Mark Zuckerberg has so many VR headset prototypes to show us

Meta’s Reality Labs division has revealed new prototypes in its roadmap toward lightweight, hyper-realistic virtual reality graphics. The breakthroughs remain far from consumer-ready, but the designs — codenamed Butterscotch, Starburst, Holocake 2, and Mirror Lake — could add up to a slender, brightly lit headset that supports finer detail than its current Quest 2 display.

Meta CEO Mark Zuckerberg and Reality Labs chief scientist Michael Abrash, along with other Reality Labs members, presented their work at a virtual roundtable last week. The event focused on designs that Meta refers to as “time machines”: bulky proofs of concept meant for testing one specific feature, like a super-bright backlight or super-high-resolution screen. “I think we’re in the middle right now of a big step forward towards realism,” Zuckerberg told reporters. “I don’t think it’s going to be that long until we can create scenes with basically perfect fidelity.” Display tech isn’t the only unsolved piece of that puzzle, but it’s an area where Meta’s intensive VR hardware research gives it a leg up.

Zuckerberg reiterated plans to ship a high-end headset codenamed Project Cambria in 2022, following its initial announcement last year. Cambria supports full VR as well as mixed reality, thanks to high-resolution cameras that can pass a video feed to an internal screen. It will also ship with eye tracking, a key feature for future Meta headsets. From there, Zuckerberg says Meta is planning two lines of VR headsets: one that will remain cheap and consumer-focused, like today’s Quest 2, and one that will incorporate the company’s newest technology, aimed at a “prosumer or professional-grade” market. That tracks with reports that the company is already planning updates to Cambria and the Quest 2, although those prototypes weren’t discussed on the call.

Meta’s VR headsets sit alongside a separate lineup of augmented reality smart glasses, which are meant to project images onto the real world instead of blocking it with a screen. Meta recently scaled back the launch of its first-generation AR glasses, and in general, VR screens have reached consumers much faster than AR holograms. But Meta’s prototypes demonstrate how far the company thinks it has left to go.

Butterscotch is an attempt at a near-retina-quality headset display — something you can find in high-end headsets from companies like Varjo, but not the current Meta lineup. The design is “nowhere near shippable” and required roughly halving the Meta Quest 2’s 110-degree field of view. But it offers about 2.5 times the resolution of the Quest 2’s (sort of) 1832 x 1920 pixels per eye, letting users read the 20/20 vision line on an eye chart. Zuckerberg says it offers about 55 pixels per field-of-view degree, slightly short of Meta’s 60-pixel-per-degree retina standard and a bit lower than Varjo’s 64 pixels per degree.

Starburst is even less shippable than Butterscotch but tests a similarly impressive upgrade. The bulky design uses a powerful lamp — requiring handles to support its weight — and produces high dynamic range (HDR) lighting with 20,000 nits of brightness. “This one is wildly impractical to consider as a product direction for the first generation, but we’re using it as a testbed for further research and studies,” says Zuckerberg. “The goal of all this work is to help us identify which technical paths are going to allow us to make meaningful enough improvements that we can start approaching visual realism.”

Holocake 2 moves in the opposite direction, exploring Meta’s options for making VR headsets thinner and lighter. It’s the successor to a 2020 design built on holographic optics, a light-bending technique that lets a nearly flat panel stand in for a thick refractive lens. The result could be as thin as sunglasses, but Meta is still working on developing a self-contained light source that would power them — almost certainly a laser, not the OLEDs commonly used today. “We’ll need to do a lot of engineering to achieve a consumer-viable laser that meets our specs: that’s safe, low-cost, and efficient, and that can fit a slim VR headset,” says Zuckerberg. “Honestly, as of today, the jury is still out on a suitable laser source.”

The presentation also discussed Half Dome, a long-running series of prototypes that can shift focal planes depending on where users are looking. These varifocal optics began as a clunky mechanical system in 2017 and later switched to an array of liquid crystal lenses, and according to internal Meta research, they can create a more convincing (and physically comfortable) illusion of depth in VR.

Meta described Half Dome’s tech as “almost ready for prime time” back in 2020, but today, Zuckerberg was more measured. “This stuff is pretty far off,” he said in response to a question about the “prime time” comment. “We’re working on it, we really want to get it into one of the upcoming headsets, I’m confident that we will at some point, but I’m not going to pre-announce anything today.”

Reality Labs will discuss more research, including how to more accurately capture real-world footage for mixed reality, at August’s SIGGRAPH tradeshow.

The designs above exist as actual hardware that Zuckerberg showed off briefly during the event. But Meta also revealed one prototype, called Mirror Lake, that’s essentially aspirational and has never been built. The design looks more like a pair of ski goggles than Meta’s bulky Quest hardware, and it would incorporate the thin optics of Holocake 2, the HDR capabilities of Starburst, and the resolution of Butterscotch. “It shows what a complete next-gen display system could look like,” said Abrash.

On top of these features, Mirror Lake would include an outward-facing display that projects an image of the user’s eyes, reducing the sense of physical separation for people outside the headset. Meta showed this slightly uncanny feature off in a prototype last year, and it may not be the only company interested in the concept: Apple has reportedly considered a similar feature for its rumored headset. The idea is tailored for a mixed reality world where Meta has staked much of its future — but today, the company is emphasizing the incremental steps along the way.