by 
XR Talks is a series that features the best presentations and educational videos from the XR universe. It includes embedded video, as well as narrative analysis and top takeaways. Speakers’ opinions are their own. For a deeper indexed and searchable archive, subscribe to ARtillery PRO.
There are few people in the AR sector that have a rap sheet to rival Avi Bar-Zeev. With a combination of sheer talent and serendipity, he’s landed at almost every major player doing significant things for spatial computing. This gives him more perspective than anyone out there.
As he discusses in a fireside chat with WXR Fund’s always-insightful Amy LaMeyer at a recent AWE Nite SF (video below), that resume is peppered with a few startups, including his current company, RealityPrime. But it also includes Disney, Google Earth, Microsoft, Amazon and Apple.
He first got the attention of Microsoft when blogging about the detriments of 3D immersive universes like Second Life. Secretly working on something similar, Microsoft brought him in to help. But from there, he made his way to the team that was building what became the Hololens.
“It let me in the door to meet the right people to get in to the group that was building the Hololens before anybody knew what it was it was. Nobody really knew what AR was. All we were supposed to do was to figure out what the next-generation Xbox was supposed to be. But they wanted something that would scare them. They wanted something that would be different, and so that’s what we came up with. So I got to work on the very first versions of that and the first prototypes.”
Around 2012, he then landed at Amazon where he worked on the device that would become the recently-announced Echo Frames (audio AR). The idea was a lighter-weight and consumer-viable piece of hardware that was devoid of optical components — basically the opposite of the Hololens.
Then came Apple where there are several frames missing from the film reel of his professional career. That’s a cute way of saying he can’t talk about that period, and rightly so. Apple’s plans in AR are the most anticipated — and well-guarded — work happening in the sector today.
So you get the point… he’s seen a lot and worked on more projects that will impact the future of spatial computing than anyone. Though you wouldn’t tell from his charm and humility, he’s the pinnacle of technical authority and thought leadership in AR today. So what are his takeaways?
First, there’s the classic question posed to the handful of folks that lived through previous “waves” of XR excitement: What’s different this time? Bar-Zeev believes that the ideas and ambitions are the same, but the underlying technology clearly enables a lot more, especially smartphones.
“[In] 1992, we had all the same ideas […] But now feels different for a couple reasons. One, on the VR side, it was cell phones that pushed all the technology costs way down to make things possible […] Some of those improvements are making it easier to be able to do AR.”
But there are significant challenges ahead. Bar-Zeev notes technical barriers that aren’t often discussed. Optical factors like field of view get a lot more airtime, but equally important are practical variables like heat. Fortunately, some factors correlate to the progression of Moore’s Law.
“Just think about having something on your head for 18 hours a day that’s warm. No one’s going want to do that […] The main contributor to heat is essentially waste in silicon […] So the better the silicon gets, the smaller the process in the silicon, the less heat is wasted and the better things get. So a lot of the things that we want are just strictly gated on these silicon processes. When they get better and better we can have more processing on the head with the same battery or even less battery but less heat, which makes it viable. As of now, it would be hard to say that anybody could make a great small product that could run for 18 hours a day and you would be happy with it. It would have to really be off most of the time, and you see that with watches. Watches work because they’re mostly off. If they had to run at full speed all the time, your wrist would know.”
Speaking of underlying technologies that have a discernable path of advancement, 5G will be a spatial computing enabler. That’s not a novel comment, but Bar-Zeev brings new (and approachable) technical insights to a discussion that’s usually mired in generalities and jargon.
His take is that offloading compute resources to nearby stations — a tenet of 5G and edge computing — will require less of the device itself. That lessens requirements like onboard compute, which in turn alleviates some of the above practical issues like heat and hardware bulk.
“Especially for AR, the headsets have to be light. So you could argue that whatever the state of the art of rendering is, we’re going to want the headset to be close to that, and there may not be enough power to do it […] One of the very first patents I wrote that Microsoft was about how to make the glasses a distributed process where you might have a something in the room — in this case it would be your Xbox — that was doing most of the work and the glasses would be doing that last microsecond read of where am I, what am I looking at, and doing that fix-up. But a lot of the work would happen off the box, and that’s still I think a good way to look at the problem. I think the people working on 5g know this really well. It’s not just about the bandwidth […] we need better networking that that sustains itself for hundreds of people sitting in the same space using devices at the same time.”
Battery life can likewise benefit from edge computing and 5G’s low-range attributes. It’s about systems that require less battery or that use it more efficiently. This is less of an issue with tethered experiences such as PC VR systems. But spatial computing’s future hinges on mobility.
“It’s a function of how much energy your radios need to transmit the same amount of information. Maybe they can do more information faster, but if they’re using a technology that’s optimized for a very large distance, they might be operating at high power, whereas if the technology is optimized to be in close range — and maybe it can beam form so that it’s only sending power in your direction at a given time — that can save a lot of energy for both receiver and transmitter. So there are a lot of these things that have to come together to make this work. That’s not to say you couldn’t do something cool and wireless today or even something cool that’s tethered today, but the future obviously has to be wireless.”
Lastly, Bar-Zeev is encouraged by what he sees in innovations like Tilt Five, whose successful Kickstarter concluded the day of this interview. Like we heard recently from Jeri Ellsworth, the success formula is all about focus on one specific area and nailing the user experience.
“I bought four. We have a family of four and I’m really excited […] I think the Kickstarter has been super successful. I think that we need to see that follow-on success in the market which I think we will see. I want to see that thing at Best Buy. I want to see it be the thing that you do on a Friday night with your friends […] I think the company has a window of opportunity to come in and sell something that’s practical that everybody can use. It’s super easy to get into and there’s almost no learning curve and it’s cheap […] Ultimately I think it’ll do really well and it just shows that if you pick an application — something that’s specific and really nail that use case — you can find a lot of people who are going to be excited about it. That’s the same advice I give to other companies […] Pick your use case, solve it and just be the best at that one thing.”
See the full interview below, coded to start at the right point. Below that, you’ll find the full footage of the meetup program, including a data presentation from our research arm ARtillery Intelligence.
Interview Only
Entire Program
For deeper XR data and intelligence, join ARtillery PRO and subscribe to the free AR Insider Weekly newsletter.
Disclosure: AR Insider has no financial stake in the companies mentioned in this post, nor received payment for its production. Disclosure and ethics policy can be seen here.
Header image credit: AWE, YouTube