by 
XR Talks is a series that features the best presentations and educational videos from the XR universe. It includes embedded video, as well as narrative analysis and top takeaways. Speakers’ opinions are their own.
A widescale realization in the XR world in 2018 was that AR doesn’t just automatically work the way you think it will. For all those little subtle functions that simulate realistic presence (occlusion, persistence, etc.), there’s a long list of really hard computational problems to solve.
Many of those problems will be addressed by the AR Cloud. As leading AR cloud thinker, 6D.ai CEO Matt Miesnieks (see our past interview) asserted at the VRARA Global Summit, AR can connect us on deep and immersive levels. But at this point, it’s just too siloed to fulfill that promise.
“Natively, as a medium, it’s a deeply immersive and sensory experience that engages us in ways that no other medium can,” he said (video embedded at bottom). “The sad fact is today, all of our AR experiences are pretty much islands…. how do you build the infrastructure to connect apps?”
6D is working on some of these problems by enabling the creation of a shared 3D mesh of the world, upon which AR apps can do all of those things we envision. But Miesnieks admits that the AR cloud we need will include several other data sources — the “plurality” we’ve examined.
“I’m talking about the AR cloud in a singular way, but really it’s a whole universe of different services,” he said from the stage at the Vancouver based conference. “It’s not necessarily one company that’s going to own them all, but in aggregate we’re referring to it as the AR cloud.”
6D also addresses a chicken and egg dilemma: Developers won’t build AR apps without an AR cloud in place. And the AR cloud won’t be built without apps to assemble the 3D mesh data. 6D crowdsources the mesh construction via API. Or, foreshadowing Charlie Fink’s upcoming book*:
“My friend Charlie talks about painting the world with data,” said Miesnieks. “Before you can paint the world with data, you need a digital model of the world to paint on. Up to now, getting that model has always been a chicken-and-egg [challenge] for any application developer.”
The other challenge according to Miesnieks is that realistic and positionally accurate AR has been possible for years… but mostly with marker-based AR. As a practical note, the hard part is gaining that same level of functionality with moving objects and real-world conditions where it “just works.”
“If you’re willing to ask a consumer to print a piece of paper and put it on the ground with a marker on it, you could get a good AR experience,” said Miesnieks. “Today’s challenge for multiplayer, persistence and occlusion is how do you get it to be something that just works for everybody.”
We also need a broader range of functionality in order to guide AR’s development into mainstream-friendly apps. Gaming rules today, a la Pokemon Go — a function of the developer talent that has populated the sector — but non-gaming apps could make AR really scale.
“If you look at it through the lens of a gaming or entertainment product, you’re missing 90 percent of the potential,” he said, “A seamless persistent experience is fundamental for AR to get outside of a game console-sized market, and eventually reach out into a smartphone-sized market.”
But the ultimate shift will happen on more deeply structural levels. This traces back to the evolution of computing. We’ve had operating systems that have common APIs that developers call to access the hardware. Then, some of that functionality was moved to the cloud, a la AWS.
But the next step in that evolution is a sort of cloud-based operating system for the real world. This would distribute a spatial understanding of the physical world. That’s 3-D mesh data, but also deeper contextual understanding of the people, objects and motions of the physical world.
“A real-world operating system is really going to be the platform that we believe is going to be ultimately the most valuable asset.” he said. “It’s a platform transition similar to Google’s search index or Facebook’s social graphs — a hugely important topic that’s going to affect our lives.”
See the entire presentation below.
For deeper XR data and intelligence, join ARtillry PRO and subscribe to the free AR Insider Weekly newsletter.
Disclosure: The author of this post is involved with the San Francisco chapter of the VR/AR Association (mentioned above). He nor AR Insider received payment for producing this article. Disclosure and ethics policy can be seen here.
* Editor’s note: the author of this post is contributing a chapter to Fink’s upcoming book, The World Will Be Painted with Data. See our review of his first book here.