by 
Though it’s often painted as a technology that will steal XR’s thunder, AI is the best thing that’s happened to XR in a while. AI can be an intelligence backbone for XR, while XR can be the front-end graphical interface for AI. The two technologies complete each other in lots of ways.
That goes for user-facing XR experiences, such as generating world-immersive overlays and lenses on the fly through text prompts. And it applies to creator-facing work, such as streamlining workflows through the time-saving and capability-boosting power of generative AI.
Both angles were recently advanced by Snap. In Lens Studio 5.0, it launched GenAI suite. This gives lens creators generative AI tools to do things like rapidly prototype 3D designs using text prompts. It also teased the other angle noted above: user-facing generative lenses.
The convergence of these technologies was also explored in a recent report by our research arm, ARtillery Intelligence. As such, it joins our weekly report-excerpt series, with the latest below. This week’s segment zeroes in on AI’s potential to automate XR creator workflows.
Assistive AI
In the last installment of this series, we looked at the user-facing integrations of XR and AI, such as digital-assistant functions for smart glasses. Now we switch gears to creator-facing functions, such as tapping into generative AI’s ability to construct creative assets based on text prompts.
Using the same framework, XR developers can generate 3D assets. It could even evolve with neural radiance fields (NeRFs) for 3D scenes. We’re already starting to see XR leaders move in this direction, including Snap’s generative AI functions baked into Lens Studio 5.0.
Such efforts could involve finished works, but will more likely be assistive. In other words, XR developers can lean on AI for rote parts of their workflows, or to generate placeholder elements. For example: background imagery or secondary elements that otherwise consume energy.
Eventually, AI could go a step further in creating 3D experiences and environments holistically. This could be similar to the ”builder bot” concept that Meta introduced to develop 3D worlds with text prompts. It doesn’t exist yet but is a potential target for AI’s role in XR development.
Beyond creating 3D assets, AI could be an inspirational tool. In other words, generative AI can help creators rapidly ideate and prototype ideas. Think of this like storyboarding XR experiences by testing out concepts via voice prompts. This could unlock and inspire creative pathways.
We speculate on this direction because it’s already being applied in non-XR areas. For example, artists use tools like Midjourney to test out ideas and brainstorm creative directions. And writers have begun to use ChatGPT to write things that serve as a starting point for their work.
The Scoby
Sticking with that last concept, generative AI’s use as an inspirational tool is already evident. During an energetic panel discussion about AI at AWE USA 2023, Lamina1 Chief Product Officer and XR veteran Tony Parisi provided a first-hand example of how AI can be used in this way.
During his then-recent Twitter Spaces with fellow musicians, a word emerged that Parisi wasn’t familiar with: scoby (a living culture of bacteria and yeast, used to ferment kombucha). As a joke, he asked ChatGPT to write a screenplay about a scoby terrorizing a local neighborhood.
The result was so outrageous and comical that it gave life to an offshoot of the Twitter Spaces where participants did table reads for the script, and related themes that others created. Though Parisi’s excitement over AI’s creative benefits is evident from this anecdote, it’s not a silver bullet.
“A lot of people think that AI is the solution to all of our content creation and authoring problems in XR,” said Parisi. “I would say that’s a somewhat lazy way to think about it. Clearly we’re seeing a lot of generative AI assisting creation and authoring. But it’s not the whole equation […] You can’t oversimplify it to say that you’re going to speak, and worlds will happen.”
In that light, AI should augment but not replace human creativity in XR and other fields. The good news is that it isn’t good enough to replace human work in creative fields. And based on XR’s nuance, XR developers’ work won’t be replaced by AI for the foreseeable future.
More from AR Insider…
