by 
There’s a hot take in the XR world that AI is the force multiplier it has been waiting for. And that notion is valid. But it’s also incomplete. It sells XR short by positioning it only as the recipient of value. But the inverse is equally true, as our friend Ori Inbar often says. AI needs XR.
One of the many places this concept applies is world models, a prevailing theme at Mawari’s Kai event where we – along with Inbar and other thought leaders – attended and spoke. World models continue to gain momentum in terms of their potential to define AI’s next era.
Stepping back, AI’s explosion in value and investment over the past few years has been enormous and expansive. But it’s also limited. AI’s manifestations have mostly been in large language models and in processing human knowledge that is in text form and confined to the web.
But the true potential lies in the physical spaces that we occupy. Among other things, this requires expanding from language models to vision-language models (VLMs) and world models. What does this have to do with XR? Key ingredients for world models are things that XR does best.
Inputs and Outputs
So what are those capabilities that XR brings to the table? We’ll split the answer into inputs and outputs. For inputs, world models need geometrical and contextual understanding of the physical world, among other things. This is a capability XR has been refining for years.
For true spatial experiences, such as Snap Specs, glasses have to understand their environment. That includes the contours of the space (geometric) and contextual understanding (semantic). This all needs to be processed before digital elements can be integrated spatially.
That precursor to spatial computing, and all the work that has gone into it, accelerates the creation of world models. We’re talking about well-funded efforts from Meta, Google, Snap, and others – meaning that capabilities are rich and world models don’t have to reinvent the wheel.
As for outputs, XR devices will be the UI – or at least one of them – for experiences that are based on world models. For example, everything from visual intelligence to consumer shopping to avatar-based assistants will be best experienced spatially. This points to AR glasses.
Another valuable output for world models will be physical AI. This wide-reaching area includes AI’s manifestations in the physical world, such as robotics. We’re talking about robots that automate everything from household chores to industrial maintenance and assembly.
Corner Cases
Of course, spatial mapping won’t be the only ingredient. Models themselves are critical and diverge into several approaches. The challenge is that, unlike text, the physical world is complex, unpredictable, and filled with “corner cases,” says XR veteran Matt Miesnieks.
These physical-world complexities range from physics to the vagaries of human behavior. And when you add up myriad physical-world anomalies, the unpredictability compounds. As world models develop, LLM tendencies to hallucinate will seem like a minor problem in retrospect.
Accordingly, Miesnieks sees world models as untenable if we just throw LLM-based principles at them. His stealth startup develops JEPA-based approach to train models on a conceptual understanding of the world – similar to how the human brain works – to predict outcomes.
Infrastructure is also critical, especially in the delivery of experiences based on world models. Use cases above – from visual intelligence to embodied assistants/avatars – require more robust and native infrastructure to deliver spatial experiences reliably and at scale.
That’s where Mawari factors in with a suite of technologies from 3D rendering to a decentralized network that crowdsources compute resources from geographically-distributed nodes. This is meant to alleviate latency issues that plagued cloud GPU plays such as Google Stadia.
Puzzle Pieces
When it comes to constructing world models, Mawari is innovating there as well. This includes collaborations with cities in Japan to process vast reaches of sensor data and other municipal signals. As co-founder & CEO Luis Oscar Ramirez says, world models need a world.
Mawari will also apply the crowdsourced principles noted above in the collection of spatial data. This includes its upcoming Pulse Node, which AI glasses wearers can activate to collect spatial data. As they move through physical spaces and use XR, they’ll help build world models.
This is analogous to Waze. Users benefit from the network while incentivized to share data back. This, along with Mawari’s crowdsourced efforts above, is built on the principles of decentralized physical infrastructure networks (DePIN), which could underpin world models.
Coming full circle, all of this is enabled by XR. We often hear about the convergence of AI and XR, such as intelligent eyewear (user-facing) and streamlined workflows (developer-facing). But the ultimate convergence point could be to accelerate the age of world models.
*Much of the above will be the topic of PHAI – an event focused on physical AI and co-located with AWE USA 2027. Learn more here.