Apple’s Vision Pro Is a Breakthrough in Photorealistic Mixed Reality

News of Apple’s Vision Pro has been dominating the tech industry since its unveiling at the 2023 Worldwide Developers Conference (WWDC) on Monday. Described as the “world’s first spatial computer,” the futuristic goggle-like piece of hardware operates on a new system called visionOS and has been making headlines, not merely because it marks Apple’s foray into AR/VR but also because of its $3,500 USD price tag. (For context, the Vision Pro’s closest competitor, the Meta Quest 3, starts at $499 USD).

Hypebeast was on-site at WWDC 2023 at Apple Park in Cupertino, California, and had the opportunity to be among the “first people in the world,” per an Apple employee, to test out the prototypal Vision Pro – in fact, several Apple employees mentioned that they hadn’t yet tried it yet themselves.

Following CEO Tim Cook’s two-and-a-half-hour kickoff keynote, I was brought to an airy, cube-shaped building called the Fieldhouse, where I was given an iPhone to scan my face to determine my correct headband size, a process identical to setting up FaceID.

I was then brought to a vision specialist who asked if I wear glasses since the Vision Pro can’t be worn on top of them. For those who do, Apple has partnered with vision provider Zeiss to input your prescription into magnetic lenses that snap into the headset.

While images of the Vision Pro shared by Apple show only a singular headband wrapping around the back of a user’s head, the version I demoed had an additional perpendicular strap running across the top. Despite the measurements taken earlier, it still needed to be tightened manually for a proper fit, but was lightweight, at a little less than a pound. The Vision Pro isn’t a totally wireless device – it comes attached to a silver battery pack, which is smaller than an iPhone and offers two hours of use.

Apple

Once snug on my head, the Vision Pro could be turned on using a large button on the top right-hand side called the Digital Crown, the same button on the AirPods Max. On the left is another button that acts as a camera shutter to capture photos and videos, though that wasn’t used during the demo. The headset has users follow another FaceID-style setup, this time, training the camera to track the irises of their eyes.

The homepage of the device is a floating assortment of apps — just like an iPhone, sans static background. Users simply look at a given app to target it with their eyes, and tap their thumb and pointer fingers together to confirm their selection. A user’s fingers don’t have to be in front of the camera while performing this action, and can simply be resting in one’s lap, seemingly out of view.

The first app Apple showed off was Photos, displaying images in a 4K display that’s of higher quality than any other VR headset I’ve sampled, including the Quest models and Dell’s HTC. Scrolling is, again, similar to an iPhone but takes place in thin air with your fingers pinched together. Tapping on a photo dims the room around it, so that the image is the main focus. My first taste of the Vision Pro’s immersive capabilities came in the form of a Panorama image captured in Iceland, which wraps around the room to make you feel as if you’re standing in the photo. I then viewed some samples of spatial photos and videos that can be captured using the Vision Pro. These images rendered their subjects – a group of family members celebrating a birthday – in 3D.

Apple

After closing out the Photos app, I was prompted to bring up multiple Safari windows. These windows can be moved around the room and pinched/pulled for resizing. An Apple spokesperson described this as the ultimate multi-tasking tool. All the more visually stimulating features aside – of which there are many – it’s these sort of functions that lend themselves to the company branding the Vision Pro a “spatial computer,” for the tech-savvy worker, rather than a piece of gaming hardware (of note is that no gaming was previewed during the demo).

Apple did, however, focus its development on facilitating communication between Vision Pro users, something that would be conducive to multiplayer gameplay. I was asked to pick up a FaceTime from another Vision Pro user, an Apple employee calling in across the company’s campus. She eerily appeared as what Apple called a 3D “persona,” an AI-generated likeness of the actual person I was talking to from the shoulders up, captured in real time by the 12 cameras situated throughout the Vision Pro. FaceTime calls, along with messages, can be resized and reoriented on the screen to float off to the side while you’re working within another app, and the audio is spatial: the direction of the call corresponded to where its widget was located in the room. For example, When I moved the FaceTime app to the left side of the screen, sound was emitted from a speaker on the left side of the Vision Pro.

Among the more impressive aspects of the Vision Pro was “Environments,” a simulated outdoor setting that a user can simply hang out in or set as their permanent background. The “Environment” I sampled was Mount Hood, a photorealistic rendering of the real-life volcano in Oregon. I was able to adjust my level of immersion by conveniently turning the digital ground, to the point where I had a 1080-degree perspective of the volcano, sky and ground and its surroundings. From the lake in front of me to the dirt I was standing on, it truly felt like I was there.

Apple

During the keynote, Apple frequently made mention of mental and physical health. To this end, the Vision Pro offers its own fleet of meditations and other health-minded features, though it’s hard to imagine these as primary uses for the device. I was taken on a one-minute “mindfulness meditation,” where 3D rainbow triangles gently swirled around a black room, but didn’t get to experience any fitness features. It’s worth keeping an eye out to see if Apple incorporates other kinds of fitness content, like virtual workout classes, into the final model.

The demo then pivoted to entertainment. I watched Avatar as a standalone screen, laid against the backdrop of the room, and then again on “Cinema” mode, blacking out the background to emulate a personal movie theater. The quality was immaculate but the technology is not new: other headsets can offer a similar experience.

Apple also let those experiencing the demo into a feature they haven’t yet announced publicly: Immersive Video. It’s essentially Apple’s own take on 3D, 1080-degree streaming that transports the user into a given setting. Again, this isn’t particularly groundbreaking as it’s available in headsets like Quest 2 but the quality of Apple’s demo was like no other. Some of the experiences streamed in Immersive Video were environmental settings like swimming with sharks or climbing a mountain, along with more streamer-friendly clips like an NBA game. It’ll be interesting to see whether Apple partners with television studios and leagues to offer Immersive Video for real-time streaming, particularly for sporting events.

Apple

To round off the demo, I was presented with a fun interactive spatial experience, where I was asked to hold out my finger and a virtual butterfly landed on it. Set in a prehistoric environment, I was then asked to walk around the room as a dinosaur explored the terrain, its head and eyes rotating to follow my every move.

After my demo, I’m resolute that Vision Pro is a groundbreaking device with incomparable quality and graphics and above all else, mind-bending immersion. It’s still hard to see it as a “spatial computer,” rather than a mixed-reality headset, albeit, a truly impressive one. The question of whether it’s worth the price, even if it’s arguably better than its competitors, still looms. And with several months to refine the Vision Pro before it’s expected to launch in 2024, Apple likely has a few tricks up its sleeve and naturally, will be enlisting the resources of the larger developer community at its fingertips to build more apps and features.