by 
With Quest Pro being discontinued and Quest Pro 2 apparently canceled, the future of Codec Avatars remains unknown.
As you may remember, in 2023, Lex Fridman, a popular podcaster and computer scientist, conducted an hour-long metaverse interview with Meta’s CEO, Mark Zuckerberg. This was their third podcast together, but this time, it was recorded entirely using Codec Avatars, Meta’s proprietary tool that allows for ultra-realistic digital representations of ourselves.
Although Codec Avatars had been around for some time, this was their first major mainstream use, and the public response was overwhelmingly positive. This positive reception has led many to believe that Codec Avatars might become the default avatar system moving forward, especially considering that Meta is also working on a lower-fidelity version of Codec Avatars. These lower-fidelity avatars are generated using neural networks, which in turn are trained on the bigger, higher-fidelity cousins from the Pittsburgh lab. They are less computationally expensive and suitable in a wider variety of circumstances while remaining impressive in terms of quality. Generating such avatars would only require a smartphone with a depth sensor, lowering the barrier to entry even further. However, Meta has since reshuffled their hardware roadmap, making the future of Codec Avatars less obvious.
Obstacles and perils
Codec Avatars need eye tracking to run. There have been whispers about Codec Avatars running without eye tracking, which would make them Quest 2 and 3 compatible, or even with no tracking at all — as a non-XR avatar supplement — but this speculation has been contradicted by Mark Rabkin, who mentioned, “necessary sensors including eye tracking” for the photorealistic avatars to work.
Because the Meta Quest family of headsets includes neither eye nor face tracking, they are automatically excluded from the equation. Recently, we learned that the Pro line has also been
discontinued. According to insiders, Meta could not find a way to get the price below $1,000 and abandoned the project. So, what hardware, if any, can we look forward to that could support Codec Avatars in the future?
There are three possible scenarios. Let’s start with the least appealing.
Option one is that Meta will give up on Codec Avatars altogether. This seems quite unlikely, given how much positive attention Codec Avatars have garnered over the years, most notably after being used in the Lex Fridman episode. They have become somewhat of a brand name compared with Apple Vision Pro Personas and other avatar systems being developed.
Nothing is written in stone, though. Meta’s CTO Andrew Bosworth remarked in an interview recently that all projects must go through a very lengthy path before they are approved for deployment. The process includes four stages and many projects end up getting cancelled very late into their development. This means that neither Orion nor Puffin are guaranteed to make it to the finish line. The same applies to Codec Avatars. Until Meta officially announces Codec support, we must acknowledge there is at least a small chance the project might be put on hold.
Option two is that Codec Avatars are not coming soon, but will eventually. At this year’s Connect, Meta wowed everyone with their Orion AR glasses, and one of the demos used in that presentation included Codec Avatars. Orion has very limited processing power compared to VR headsets, so to see Meta experiment with Codec Avatars even with such restrictive specs proves how serious they are about bringing this feature to its users.
All of this is nice, but at the same time, Orion is an early prototype that Meta doesn’t expect to launch any time before 2030. A less impressive version of the glasses codenamed Puffin is rumoured to launch around 2027, so that’s another possibility.
Option three, perhaps the most optimistic, is that Quest 4 (codenamed Pismo) will become the first Quest headset to include eye tracking. This would immediately put any doubts surrounding Codec Avatars to rest and so far, signs are promising. We’ve seen job listings that suggest eye tracking on Pismo. We’ve also seen job listings related to Codec Avatars that signal continued research and ongoing development. Finally, there’s a comment from Meta’s VP Mark Rabkin, who said about Codec Avatars, “I think probably, if we do really well, it should be possible in the next generation.”
Summary
Yasir Sheikh, the lead researcher behind Codec Avatars, once said that running Codec Avatars on standalone headsets is “ten miracles away.” He then revised it to five miracles and now to “few miracles away” so it seems we’re slowly getting there. Over the years we’ve seen so many twists and turns in its development, expectations ranging from Codecs coming to all Meta’s headsets, all the way to doubt that they will ever come to any headset at all.
Personally, given all of the information, I continue to hold my opinion from last year, that Codec Avatars will eventually make their way into standalone headsets, and moreover, that it will happen sooner rather than later.
Mat Pawluczuk is an XR/VR writer and content creator. As with all AR Insider contributors, his opinions are his own.
More from AR Insider…
