by 
This article is the latest in AR Insider’s editorial contributor program. Find out more or contact us to participate here. Authors’ opinions are their own.
Why is Apple Buying NextVR
by Devon Copley
The coronavirus pandemic is impacting every industry, and the AR/VR space is no exception. Some AR/VR firms have seen dramatic increases in interest — especially those, like my company Imeve, who are focused on collaboration tools like AVATOUR which can lessen the impact of distance. But other startups in AR/VR are facing revenue challenges amid a grim funding environment. Two such companies apparently found ports in the storm last week: NextVR reportedly is being acquired by Apple, and 6D.ai was acquired by Niantic. Today I’ll address the NextVR acquisition; I’ll write a bit about 6D.ai as well later this week.
Apple and NextVR
The reported ~$100m price for NextVR is a disappointing exit for the firm. The company began with 3DTV broadcasting technology in 2009, pivoted skillfully to VR during the halcyon days of 2015–2016, and attracted funding to the tune of $115m from media investors like Comcast Ventures and MSG. Unfortunately the audiences, and revenues, didn’t materialize. NextVR failed to close a Series C investment in late 2018, leading to layoffs of 40% of their staff in January 2019. From that point, given the limited audience for live VR content, it was hard to see how NextVR could survive as a going concern. The coronavirus-related cancellation of real-world live events — notably the NBA games, for which NextVR surely paid a pretty penny — must have been the final straw.
The obvious question is: what does NextVR have that could be worth $100m to Apple? The answer, as ever, must be some combination of people, deals, technology, and IP.
An aside: some analysts have made a fuss about NextVR being VR technology, which supposedly doesn’t fit with Apple’s focus on AR tech. There’s no contradiction here. AR and VR are best understood not as opposites, but as alternate consumption modalities that rely on the same set of underlying technologies. NextVR’s content streams will look great in an AR headset with sufficient field-of-view. And of course, Apple’s AR/VR strategy is both, and.
NextVR’s people number less than 100 (83 are listed in LinkedIn), mostly in Southern California, mostly consisting of software engineers and production crew. Clearly there’s some valuable technical folks on the team but much of this group is likely duplicative of Apple’s existing capabilities, and a number of key engineers saw the writing on the wall and left during the last year. Perhaps this acquisition is an easy way for Apple to scale up as they’re building out their AR/VR teams, but it’s hard to see how NextVR’s staff would justify the purchase price.
NextVR’s content deals seem quite limited. AFAIK they have only announced single-year deals with any of their content partners, notably the NBA. (One has to wonder what sort of protections they had in the unlikely event of the NBA season being canceled? The expensive lawyer who wrote that contract either earned his fee or … didn’t.) In any case it’s hard to believe there’s any content deal that would still be in effect by the time Apple launches its AR/VR headsets in 2022 and 2023. This can’t have contributed much to the purchase price either.
NextVR’s technology is … more interesting. I’m not referring to the tech that was actually in production for their VR broadcasts, which was largely derived from their 3DTV pipeline and relied heavily on existing video technology. It’s their next-gen 6-degree-of-freedom (6DoF) capability that’s interesting. They showed an early version of the tech at Mobile World Congress in February 2019. A year later, does NextVR actually have nearly-production-ready technology for capturing, delivering, and rendering 6DoF video, or was this a science project with little practical application?
Hard to say. But this leads us to…
NextVR’s IP. This has to be the main reason why Apple would be considering spending $100m on this company. NextVR has 38 issued patents and a bunch of applications, nearly all of which relate to their production pipeline. I reviewed these patents, and they mostly fall into four major categories: UX, camera configuration, compression optimization, and a final category which I’ll call mesh+texture encoding. Taken together, it’s a strong portfolio, but the last category in particular is the most interesting.
Mesh+texture encoding (there’s no widely-agreed name for this technique AFAIK, but this name makes sense to me) is perhaps the most practical means to deliver a limited 6DOF capability using existing video encoders and decoders. It combines standard video textures with 3D mesh content, using “frame packing” to combine multiple textures into a single standard video frame. Similar techniques are used by Microsoft’s volumetric format and MPEG’s nascent “Metadata for Immersive Video” (MIV) standard. This approach is likely to deliver good results for live events well before true volumetric approaches do, albeit only for a “limited 6DOF” or “3DOF+” sort of experience, where the user can only move their head within a small volume rather than being able to walk around an entire captured space. It’s not the only approach for 3DoF+, but it’s possibly the most promising one.
NextVR started working on mesh+texture encoding quite a while ago. The earliest issued patent of theirs that I found on the topic has a priority date of February 2016. I was on the Nokia OZO team at this time and we were among the most advanced teams in the world in this space; our R&D team in Tampere found their way to a mesh+texture approach that same year. The NextVR guys started barking up this particular tree early, and they’ve been awarded at least 6 patents that are directly related to the technique, with priority dates in 2016 and 2017; I’m sure there are more applications pending as well.
My guess is that these 6DOF video patents are the most valuable ones in the NextVR portfolio, and are the primary drivers of the purchase price. These patents give Apple a strong IP defense against Microsoft as well as the MPEG consortium members.
Still … $100m? Seems steep. So let’s look back at a couple of the other components: technology and people.
I’m guessing NextVR must have more than a science project. They must be far enough along that their tech could actually be deployed as part of Apple’s headset launch. 2022 is coming up awfully soon, and a 3DOF+ live sports experience is something nobody has ever seen before — it’s just the kind of “wow” content that Apple needs. If you want to launch with that content, you need to produce that content. And while Apple has plenty of software engineers, they don’t exactly have a big live event production team. Maybe that tech, and those people, are significant components of the deal after all.
If it all comes together, Apple’s AR/VR headset might launch with that courtside experience we’ve been promised for so long. You’ll get the high resolution, high-FOV 3DOF+ live feed, and also be able to see your friend sitting next to you on the sofa — not to mention your beer. It sounds great to me — especially during these grim sports-free days.
Devon Copley is Co-founder & CEO of Imeve. A version of this post originally appeared on the company’s blog, contributed here with the author’s permission.