This article is the latest in AR Insider’s editorial contributor program. Find out more or contact us to participate here. Authors’ opinions are their own.  

7 Ways Spatial Computing is Evolving Advertising & Marketing

Advertising Enters the Next Dimension of Computing
by Tom Emrich

25 years ago, AT&T bought the first-ever banner ad on the web as part of its “You Will” campaign which fittingly featured commercials depicting an internet-enabled future. This 2D rectangle changed the world of advertising as brands and marketers gained a new way to reach the masses with a level of measurement never seen before.

Today brands and marketers are facing another transformation as advertising enters the next dimension through spatial computing.

Spatial computing is the shift of computing from 2 dimensions to 3 dimensions. To enable this shift, computers need the ability to understand space — the environment around it and everything in it including people. Computers are able to do this using sensors, such as cameras, and artificial intelligence.

Once computers understand space, we can use these informed computers to edit our real world by placing digital objects and information in it and view this all through a transparent digital screen like smart glasses or even our smartphone: this is Augmented Reality or AR.

Or we can put on device on our head (a head-mounted display or HMD) that completely occludes us from the real world allowing us to enter a completely simulated environment which we can walk around and interact in because the computer is sensing our physical actions: this is Virtual Reality or VR.

Like the waves of web, mobile and social before it (SoLoMo anyone), spatial computing: AR and VR will provide new opportunities for brands and marketers to reach, engage and understand consumers.

Here are just seven ways I see spatial computing evolve advertising and marketing.

1. The Evolution of Display Ads

Augmented Reality ads are not new. Marketers have been running ads in Snapchat for a number of years now. But a number of major changes on mobile have unleashed AR’s full potential as a killer new ad format which is expected to reach a $2.6 Billion ad spend by 2022.

The first change was with Snapchat itself with the introduction of AR programmatic advertising in 2018 which dramatically reduced the price of augmented reality advertising to the same price as regular Snap ads.

The second change was in Facebook’s support of 3D and AR ads in the newsfeed starting with a pilot with Michael Kors which used AR to let consumers try on sunglasses. Facebook had previously tested AR ads in Messenger with a variety of brands reporting successes: ASUS saw 10 times the engagement with their AR experience; Nike sold out of the Kyrie 4s shoe in under an hour; and Kia increased engagement with its dealers.

The third major change came from Unity which recently announced that it will now be serving AR ads on mobile. Unity has one of the world’s largest mobile ad serving businesses and at the end of last year they partnered with Fossil to pilot this new ad format in mobile games.

Finally, a breakthrough in WebAR thanks to startup called 8th Wall makes mobile AR ads possible outside of an application. 8th Wall recently partnered with Amazon Sumerian on a WebAR ad experience where, with just a click of a banner ad, an AR experience placed a 3D Spiderman into your space for you to interact with directly in your mobile browser.

With opportunities both within large app ecosystems and outside these closed gardens with WebAR, there has never been a better time for marketers to move beyond the simple 2D banner ad.

One huge opportunity for AR ads which goes beyond the 2D banner format is its ability to allow users to “try before you buy”. Experiences like the one below from Michael Kors in the Facebook newsfeed gives users the ability to put a pair of sunglasses on their face rather than having to imagine what it would look like on them from a picture of a model. This type of ad also lets a brand market multiple styles and products within a single experience.

Michael Kors is not the first to tap into this opportunity. Ikea, Sephora, Warby Parker and Houzz are just a few more brands using AR to help users try out virtual goods to convert to physical sales. AR try-on experience have been reported to increase sales, Houzz has said that AR ads make consumers 11 times more likely to buy, and they have the potential to reduce returns and improve overall customer satisfaction.

But AR try-on is not limited to smartphones. Instead of displaying signage in-store which features a model marketing merchandise, let consumers see themselves in the ware with the help of a Magic Mirror. Magic Mirrors, like the one shown below from FXGear, require no download or device to be able to show a consumer what they look like in inventory that may or may not even be in store.

In virtual reality, the try before you buy opportunity is a realized as a simulation which provides a safe environment to use new products. This is exactly what Lowe’s is doing with its Holoroom. The Lowe‘s Holoroom is being used to let customers test drive power tools and other home renovation equipment so that they feel comfortable not just buying it but also using it when they get home. The hope is that the interactivity and training will lead to increased purchase conversion. Lowe’s says that early versions of a similar VR training program for employees led to a 127% increase in confidence compared to traditional training procedures — they are hoping to bring these same results to the customer space.

2. The Evolution of Out of Home and Product Placement

It took nearly 20 years, but 2019 is the year when digital advertising will overtake spend on traditional media (billboards, newspapers, TV etc.). This should come as no surprise in a time when over 50% of the world is online and digital advertising can provide a cost-effective means to reach users with the added value of metrics offline methods can’t compete with. While our current digital advertising presents a massive opportunity it has its limitations as it is stuck behind a pane of glass and in this confined to our time online. With spatial computing, all the world’s an ad.

AR pioneer and investor, Ori Inbar, once said that “the Web Economy was defined by “clicks on links” (CPM/CPC) and the AR Cloud-based spatial economy will transition to what I like to call “clicks on bricks”. And he is right! Spatial computing digitizes every place and everything in it turning them into an ad opportunity. And the same goes for VR, where the world itself is digital. In this way, both AR and VR are evolving traditional advertising methods such as OOH (out of home) and product placement, making them just as measurable and effective (if not more) as digital advertising.

A good example of this in virtual reality can be seen in a recent campaign by The Coca-Cola Company on the Advrty platform. Advrty wants to offer a new avenue of monetization for virtual reality developers which doesn’t involve full field-of-view pop-up ads with a platform aimed at designing and incorporating nonintrusive product placement. The core of its platform is based on the ability to accurately track user views and interactions. While billboards are the most obvious opportunity in a virtual environment, Advrty expects to go well beyond this with the ability to swap out products and other 3D elements in VR with branded ones.

Where Advrty uses the simulated environment in VR to serve up ads, Ogilvy‘s recent campaign with Burger King shows what can happen when the real world around you can just as easily serve up ads on physical structures using augmented reality. Using the Burger King app on mobile, consumers were asked to find ads from competitive burger brands and set them on “fire” using AR in return for a free meal. Once the rival ad is burned, a Burger King ad is shown in its place.

While not swapping anything out, Google is attempting to make the world an ad through its visual search app–Google Lens. Lens uses your camera to find information on what it sees including places, plants, animals and objects including links to purchase where relevant. Google Lens can now identify over a billion products and every time you scan something in the world you provide information on what you are seeing — a huge opportunity for Google to understand you beyond your online behavior.

https://youtu.be/F2brcDqKTHI

Turning the physical world into a digital opportunity is something you can also see in this example from 19 Crimes wine. Using the Living Wines AR app, 19 Crimes brought its packaging to life with the prisoners featured on its bottles telling their stories right from the label on the wine bottle. Here 19 Crimes not only turned a printed label into a video, but the same engagement metrics you would expect with a video ad can now be measured from a consumer interacting with a physical bottle.

3. The Evolution of Brand Ambassadors

Social media has given rise to a new brand ambassador, the Influencer. Influencer Marketing is expected to grow to an $8 billion industry by 2020 with 62% of marketers saying they are growing their influencer marketing budgets. But while 39% of all Instagram accounts with 15,000+ followers are active influencers, they better watch their back because there is something coming that may take their place — the rise of a new brand ambassador that goes beyond just taking photos and videos of your product. An ambassador that doesn’t USE your brand, they BECOME it through Camera Effects and Avatars.

I call them the “Incarnations”.

Camera Effects such as branded face filters like the one below for (RED) by Snap Creator extraordinare, Cyrene Quiamco, is a good example. Here Cyrene goes beyond taking a pic of her wearing, or using a (RED) product — she has turned herself into (RED). She is also able to enter an AR portal which transports her to a 360-degree environment made entirely out of the (RED) brand.

Filters like these go viral in days with numbers in the millions of views. The right “Incarnation” can use a branded experience like this to tell stories which align with your brand messaging which can translate into huge success. Of course there is also the risk that your brand can also end up on the face of someone that tells a completely different story, one that goes off brand, and for this reason there may soon be a need to work in a more structured manner with “Incarnations” much in the same way as influencer marketing is formalized today.

One interesting thing to note is that the creators of camera effects are themselves becoming influencers and thus a great target for collaboration. On Instagram there is a flurry of activity where users are frantically trying to find new camera effect creators to follow, as following a creator means you will get their effects first, so that they can be the first to use a filter and boast their find by changing their face in their stories.

With Facebook, Instagram, TikTok, and WeChat all supporting face filters, the opportunity for brands and marketers to tap into this opportunity has grown far past just Snapchat. Third-party SDKs, like SVRF which adds camera effects to any app; advancements in Web AR, which does away with requiring an app; and next level emojis, such as Animoji, Memoji, Bitmojis, Facemojis and other messaging apps like SelfMoji and ToonPack available in your smartphone’s messaging and camera app option drawer, are quickly making camera effects ubiquitous.

But like any mobile interaction, smartphone face filters are quick, snackable and fleeting. It is on desktop that we are seeing the use of camera effects for longer periods of time. Snapchat and YouTube both recently launched desktop apps which allow streamers to swap their faces for something virtual. And in Japan there are more than 4,000 “Vtubers”, Virtual YouTube stars, which choose to take on virtual characters rather than their real world identity. VTubers are livestreaming, receiving gifts from fans and even making appearances at real life concerts, all with the help of screens— but as soon as smartglasses become mainstream it is easy to see how these “Incarnations” could opt to use their virtual identities 24/7 which would be just as persistent to the world as their birth identity.

But you don’t have to wait until we have AR smartglasses on our face to imagine what a world with virtual identities could look like, avatars are already being used in virtual reality. Like camera effects, avatars present new opportunities for brands whether this be to offer branded characters for use, claim those that have gone rogue, or to provide branded accessories and clothing for use. A great place to explore characters, branded and non-branded, and get to know a new community of VR-only influencers is in VR Chat — a community of 50,000 created worlds and growing.

4. The Evolution of Brand Identity

Brand identity is a crucial component to succeed in today’s world. How your brand is presented, from your logo and typeface to your tone, sits at the core of every marketing decision you make. In the 2D world, your brand identity is expressed mainly with your website. But in the 3D world, your brand will come to life quite literally with branded worlds and the personification of your brand identity in a virtual being.

Brands are already doing a lot of work to translate their brand identity into chatbots. Chatbots are expected to power 85% of all customer service interactions by 2020 and result in a cost savings of $8B annually by 2022 . But a disembodied AI is just the start to personifying your brand and they may not be doing a great job at building a relationship with the consumer. A recent study from Stanford found embodied assistants are trusted more by users which is just one reason why your chatbot requires a face as a virtual being. It becomes even more necessary in a world beyond the website with AR and VR.

Quantum Capture is lowering the cost and time to create a virtual human. Its platform, CTRL Human, uses a high quality human scan and brings it to life complete with facial and body movements by hooking it in to your chatbot backend. The example below is a partnership with REinVR, a VR real estate solution, which shows how a virtual human can be used to sell condos in VR. This same character could be used in AR, or on a 2D screen like a kiosk or your website for customer support, sales, and even training for your staff.

If there is a particular person you prefer to embody your chatbot, photogrammetry rigs can be used to capture a person to be used for this purpose. In this way, you could double an existing celebrity to have their digital twin be the face of your brand.

Volumetric capture can also create volumetric video that can be used in VR and AR experiences. Although not connected to an AI, thus limiting real-time interactivity, volumetric video could be used to convey canned messages to viewers. Microsoft Capture Studio is offering such services scanning celebrities and performers for entertainment and education purposes.

Virtual humans are getting more and more real. Check out the example below of Mica, Magic Leap’s intelligent agent. Just imagine a world where Siri, Alexa and Google Assistant have bodies. In the near future, there may be an opportunity for the user to select what character they wish these digital assistants to take form — of which a branded character, clothing or accessories could be an option for their choosing.

And it’s important to note that these beings don’t have to look like real humans, heck they don’t have to be human at all. Fable Studios is building Lucy, an AI character that currently acts as a self aware drawn character in the “Wolves in the Walls” series but eventually will become her a personified OS much in the same way Mica from Magic Leap is aiming to be.

5. The Evolution of Brand Activations

Effective marketing strategies typically attempt to evoke a feeling to align the user with the vibe of a product or brand and get them in the right frame of mind to convert to purchase. Brand activations are a compelling way to achieve this but creating a branded environment and experience can be costly and limited not just by budget but by physical space and the laws of physics. AR’s ability to remix reality and VR’s ability to world build provide a brand new opportunity to create brand activations which do not abide by these limits and completely immerse the user in an experience and environment aimed at increasing sales and brand loyalty.

A great example of this is from a new startup from Walmart’s Store №8 called Spatial&. Spatial& creates immersive retail experiences by tapping into the emotional power of VR. Their offering — called “V Commerce” — is a new form of immersive merchandising where store and story are one. V-Commerce transports the customer to a branded world and story which creates a new path to purchase and emotional connection to the brand using the medium of VR.

Spatial&’s first V-Commerce example is in collaboration with Dreamworks for “How to Train Your Dragon: Hidden World”. Sixteen Walmart locations are featuring this experience which takes the customer into the world of “How to Train Your Dragon” powered by HP VR tech and Voyager motion chairs. After experiencing the 5-minute VR story the customer is immediately taken to the merchandise shop where they can plush toys, action figures, DVDs and more.

Where VR can be used to transport a customer into a branded world, AR can be used to augment a physical location to bring magic to an experience. For me there is no better example of this than Eminem’s AR experience at last year’s Coachella. Coachella 2018 attendees used mobile AR to bring the Eminem stage to life complete with over-the-top world scale characters such as a holographic version of Eminem, sky effects and stage extensions.

Imagine a retail store for the holidays coming to life with AR without the need for much physical dressing, or a window that is dressed being brought to life and taken to the next level because of AR. As you are devising strategies for experiential marketing, AR and VR should be explored especially in cases where big ideas are finding real world limits.

6. The Evolution of Customer Support

A large part of a brands relationship with a consumer happens after the purchase when the product or solution requires support in assembly or use. While online has taken the traditional paper manual and made it more interactive with things like video and call centers and chatbots offer support around the clock, our current methods can still leave consumers confused and unsatisfied. Videos and manuals are flat and disconnected from the product and call center staff and chatbots lack the vision necessary to see the situation and are therefore forced to rely on what frustrated consumers saying by text or verbally over the phone.

AR and VR is a game changer in customer support. VR simulations and AR instructional overlays are creating more intuitive opportunities to educate the user on product use and guide installation and assembly in a much more effective and natural way. Remote assistance, telepresence and social VR experiences are all providing a way for consumers and brand staff to connect, collaborate and troubleshoot in the same space saving time, reducing the costs related to travel and dramatically improving customer satisfaction.

A good example of this today is the Streem app. Streem combines augmented reality, remote video and uses contextualized data to help brands power an exceptional customer experience. With Streem, brands can see what their customers see using the smartphone camera which leads to diagnosing and resolving issues quickly and efficiently, guiding and assisting customers with simple, intuitive tools such as drawing or using cursors and, with computer vision, automatically gather information such as measurements and recognizing parts. Streem is currently in use for product support, instant and remote insurance claims and home improvement quotes and even help to document a home inspection. Its solution is saving technicians a dramatic amount of time they would have spent in transit to reach these destinations to assist customers while leaving customers instantly gratified.

Telepresence solutions, such as the one below from NYC’s startup Spatial, takes remote assistance one step further. Spatial turns the space around you into a shared augmented workplace. Remote users can collaborate, search, brainstorm and share content as if they were in the same room. Spatial’s avatars allow remote users to “teleport” to the physical space which, through mixed reality headsets like HoloLens, allow those physically present in the room to virtually feel face to face with lifelike avatars and shared walls. Spatial’s solution is interoperable which means that users can participate using desktop, mobile and mixed reality headsets. This type of tool is going to dramatically change agency work — imagine how collaboration between agency and client changes with something like Spatial. But it is easy to see how this same solution can be used to improve customer support and elevate remote sales.

A very similar value proposition is offered in VR. MeetinVR is an online meeting and collaboration solution which allows users to meet in the same virtual reality space. Being in a simulation has its benefits. As all users in this space are avatars, virtual customer support staff could be branded. You could also choose to brand the environment. And as the outside world is removed, this setting lends itself to an increase in focus and engagement. In fact MeetinVR boasts a 25% higher attention span in its software compared to traditional video conferencing.

While Streem, Spatial and MeetinVR are good examples of how AR can be an effective tool for live interactions between support staff and customer, pre-programmed AR and VR instructional solutions are disrupting the paper manual and video tutorials. Hyundai, Mercedes-Benz, and Genesis are just a few car companies that have all used AR to evolve the paper manual with overlays that appear over automotive parts which help vehicle owners do anything from get better acclimated with their engines to changing the oil.

7. The Evolution of Consumer Data

Source: Cognitive3D, Accenture, Kellogg

The web made the consumer measurable with views, clicks, conversions, demographics and online behavior. Mobile gave us location. And with social media we were given insight into people’s online relationships. While this data helps us build and sell better products, it is just the tip of the iceberg especially since we really don’t know much about the user once they leave their desk or put their phone away.

Spatial computing will unlock the ability to know everything about a user 24/7 and measure all interactions with brands both on and offline as wearable solutions provide computers with ongoing access to what the user is seeing, where they are going, what they are doing and even how they might be feeling. While this data presents a massive opportunity, it will also require a new set of tools and skills to harness it; ethical and regulatory discussions to govern it; and a complete change in business strategy and success measurement to utilize it correctly.

While we are not completely at the point where we can monitor users to this extent, we are already seeing this shift start to happen in pilot programs such as the one below powered by leading spatial analytics company Cognitive3D. Accenture used Cognitive3D’s spatial computing analytic platform and Qualcomm’s reference headset, complete with eye-tracking, to build a virtual reality merchandising solution for Kellogg for the launch of its new Pop Tart Bites. The solution immersed the user in a simulated store environment where they were monitored through the space to watch them shop, pick up products and place them in their carts. The data led to an entirely different merchandising conclusion than what was derived by traditional methods. Rather than placing the product on higher shelves, which conventional testing indicated, the VR solution demonstrated the optimal placement was on the lower shelves. This change led to an increase in brand sales by 18% during testing.

While this type of test was done in VR today, when smartglasses hit the faces of the mainstream we will be able to measure this same scenario in real-time at an actual store. But we don’t even need to wait for smartglasses to gain new insights from AR. Mobile AR presents new opportunities such as interactions with 3D assets and 3D spaces, and potentially what users are seeing as they engage with AR experiences with their camera on.

While Cognitive3D’s example is demonstrating what happens when a computer can see what a user is doing in space. Computer vision can also be used to monitor who is in the space and how they might be feeling. A great example of this is Affectiva, a startup from Boston spun out of MIT, that monitors people in vehicle, detecting head pose and facial expressions, to interpret levels of drowsiness or distraction and expressions of joy, laughter, anger and surprise.

While technology like Affectiva is not yet widely available, developers are starting to get access to facial data from depth-sensing solutions like the iPhone X True Depth Sensing Kit and more robust biometric sensing such as eye-tracking is quickly becoming a staple feature in head-mounted displays such as the HoloLens 2 and the Vive Pro Eye. In this way, the opportunity to really know your user using AR and VR is closer than you think.

Today, sentiment analysis can be performed on text comments online and insights can be gleaned from hearts, likes and emojis on social posts. Tomorrow, computers will use biometric information from available sensors to measure how engaged a user is and potentially how they may be feeling. Imagine what brands and marketers could do with this type of information to better campaigns, product development, and overall satisfaction of the consumer.

On Privacy and Security

While the need for privacy and security in spatial computing experiences justifies its own post, it would be amiss for me not to stress how critical it is for any brand or marketer to keep these two elements at the forefront of what they do in this space. While protecting consumer data is not unique to spatial computing, the nature of this next wave does collect a much deeper and wider data set and therefore it is even more critical that consumers be made to feel safe as they interact with AR and VR. Transparency in data use, tools that give control back to the consumer and clear value for the data exchange will be critical keys to combatting privacy concerns while decisions you make on data storage, encryption and access permission will impact the security of your solution.

Final Thoughts

The spatial computing wave is upon us and like the web banner it is set to completely disrupt how brands and marketers reach, convert and understand consumers. While AR and VR technologies are still in their infancy, there are many platforms today with enough scale — mobile AR and standalone VR headsets like the Oculus Go and Quest (coming this Spring) — which provides a massive opportunity to test, experiment and realize substantial ROI. Brands and marketers that start to understand how this shift in dimension — from 2D to 3D — is changing ad opportunities, in-house team skills, while posing new concerns and challenges will be those that have a leg up in the decades to follow.

Tom Emrich is VP of Product at 8th Wall. A version of this post originally appeared on his Medium.