Niantic Labs is adding a new augmented reality feature to Pokémon Go next month that will make the virtual creatures look even more realistic. The new feature will let real-world objects obscure parts of the pocket monsters or let them hide entirely behind environmental fixtures like trees or furniture.
The company calls the feature “reality blending,” and it’s based on the AR technique known as occlusion, which uses a mix of hardware and software features on powerful, modern smartphones to map an environment and understand the shape and depth of real-world objects. Niantic says the feature will come first to Samsung Galaxy S9 and S10 devices as well as the Google Pixel 3 and Pixel 4. More devices are expected to join the list in the future.
“Pokémon will be able to hide behind a real object or be occluded by a tree or table blocking its path, just like a pokémon would appear in the physical world,” Niantic product manager Kjell Bronder says in a blog post. “As Pokémon Go has grown throughout the years, trainers have been able to photograph, walk, and play with their pokémon buddy. This new feature brings them even closer to reality.”
Niantic first demoed its occlusion technology nearly two years ago with a video featuring a life-sized Pikachu darting around a crowded public space filled with walking pedestrians, potted plants, and benches. Each time the Pikachu would go behind a pot or stand with a person in between it and the phone camera, parts of the creature would become obscured.
[embedded content]
Niantic has built an impressive team of researchers over the last few years to help it pull off techniques like this using existing phone hardware. It’s done so through strategic acquisitions of AR and artificial intelligence companies like Escher Reality, Matrix Mill, and Sensible Object.
More recently, Niantic purchased 6D.ai, a startup dedicated to using AI to 3D map physical spaces. “The research team has grown since the 6D.ai acquisition and now includes world-renowned AR researchers from Oxford Active Vision Lab and the University College London, among others,” a Niantic spokesperson tells The Verge.
In addition to reality blending, Niantic is also giving Pokémon Go the ability to contribute to its crowdsourced 3D mapping project, as it does with its original AR game Ingress. Instead of portal scanning, as it’s called in Ingress, Niantic says players of Pokémon Go will be able to perform “PokéStop Scanning” to “contribute to building 3D maps of their “favorite PokéStops and Gyms by walking around their favorite public places and recording a stream of images with their phone.”
“Techniques such as blurring potentially recognizable objects like faces or license plates are automatically applied to information that trainers choose to send to Niantic,” Bronder explains in the blog post. “Niantic does not collect or store any personal data in connection with this information and it is not tied to specific player accounts.”
The feature will be restricted to trainers who are at the max level of 40 before it comes more broadly to lower-level players. Bronder says these images help it generate more realistic 3D maps, which in turn make the game’s blending of the real and virtual more believable. “This will allow us in the future to tie virtual objects to real world locations and provide pokémon with spatial and contextual awareness of their surroundings,” he says. “For instance, this awareness will help Snorlax find that perfect patch of grass to nap on or give Clefairy a tree to hide behind.”
Of course, COVID-19 is complicating these large, collaborative AR games at the moment. Niantic has taken some measures to ensure its games can be played safely from your home, The company laid out a plan back in March to continuously update its games during the pandemic to make easier to record indoor exercise as in-game activity and enhancing the social, exploration, and live event features to better suit players during shelter-in-place.
That’s resulted in some nice new features, including remote raiding in Pokémon Go and Harry Potter: Wizards Unite’s in-game transportation option to ferry players to the “fortresses” that typically require your physical presence at a real-world landmark or location.
