by 
“Trendline” is AR Insider’s series that examines trends and events in spatial computing, and their strategic implications. For an indexed library of spatial computing insights, data, reports and multimedia, subscribe to ARtillery PRO.
The augmented reality subsector known as audio AR has a new player: Foursquare. Known as Marsbot for AirPods, a new Foursquare Labs project features a lightweight virtual assistant that proactively whispers geographically-activated local recommendations in your ear.
Before diving into the details and dynamics, what is audio AR? As we’ve examined over the past few years, this AR modality eschews common graphical connotation in favor of audible augmentation. It can inform you about your surroundings or translate foreign languages on the fly.
Audio AR’s advantages include discreetness. It avoids the style crimes of AR glasses (or an upheld smartphone) in favor of subtle audio. It’s also advantaged by hearables’ growing installed base and all-day use case. And tech giants like Apple are highly motivated to push the hardware.
That hardware foundation is the first step, and that’s where we are now. Though there’s an all-day use case, hearables are idle most of the time. The next step will be to fill that time with textured and intelligent audio content, likely scaled through an iOS-orbiting SDK akin to ARkit or tvOS.
Secret Service
That brings us back to Marsbot for Airpods. The app taps Foursquare’s rich foundation of location data as content and geo-triggers for Audio AR experiences. This could include audio alerts for a new gastropub in your neighborhood or audio content left by friends in your social graph.
Here are a few audio examples provided by Foursquare (click to play):
[PLAY] Blue Bottle Coffee is the #1 rated coffee shop in Greenwich Village according to Foursquare.
[PLAY] Los Tacos No. 1 is a taco shop that made Eater’s list of “38 Essential Restaurants”
[PLAY] Sotto 13, on the south side of W 13th Street, is an 8.6 rated Italian Restaurant according to Foursquare
[PLAY] Alex is at The Scratcher!
Foursquare’s places data (care of its Pilgrim SDK) will form the foundation for these experiences that could develop over time as more content and relevance signals are applied. Foursquare will also optimize the UX over time in terms of how, and how often, alerts are delivered.
Marsbot could also evolve with deeper levels of AI that infer intent and act more like a personal assistant. By ingesting data such as your calendar and consumer preferences, it could deliver useful suggestions beyond just local businesses, turning us all into secret service agents.
“Right now virtual assistants like Alexa and Siri need to be summoned, Foursquare Founder and Labs Lead Dennis Crowley told us. “But we imagine a future where these assistants are much more proactive –– virtually tapping you on the shoulder when something (or someone!) interesting is nearby. You can imagine these assistants recommending great coffee shops or taco spots to you as you walk by them, or reminding you to pick up your prescription as you walk by the pharmacy or to grab milk as you walk by the local grocery store.”
Drilling Down
Drilling down on other particulars, Foursquare provides a few qualifiers and signs of how Marsbot for AirPods could evolve. The key term here is experiment, as Foursquare Labs wants to test these concepts in the wild and demonstrate location data’s value in developing AR magic.
Here are some of Crowley’s notes on the program, verbatim, with more depth available here.
— Right now, Marsbot for AirPods is only available for iPhone. Despite its name it will work with any type of headphones or earbuds, wireless or wired.
— It’s designed to work best when you’re walking around, though it’ll also work if you’re running or riding your bike. It’ll even work when you’re driving, which was originally a bug but it was one of those “bugs that felt more like a feature,” so we left it in. We’d love to hear your feedback on this.
— It’ll work anywhere in the world, but you’ll probably find the most pre-loaded content in big cities like NYC, San Francisco, Chicago, Seattle or Los Angeles (aka: places Foursquare has offices — ps: we’re hiring!)
— It also works better outside than inside. Marsbot for AirPods is NOT a liner walking tour (“turn left, walk 100 steps, turn right”). It’s more of an “open-ended, audio-first, augmented reality experience triggered by whatever path you chose to travel as you go about your day”. That said, it’ll work better outside (think: big open spaces with street addresses and intersections) than inside (e.g. offices w/ conference rooms). You can leave audio snippets at a specific place (coffee shop, park) or at any latitude/longitude (hiking trail, bike path, random intersection, mountaintop) on earth.
— The experience is designed to be light. You may only hear one audio snippet a day, or you may go days without hearing any audio snippets. In the spirit of “an app you don’t have to use,” we didn’t want you to have to fiddle with settings or “mute” buttons because the app was too chatty. It’s worth noting that the experience may be *really chatty* the first few days as it gets used to your surroundings.
— That said, version 1.0 of this thing does not have a “follow” mode –– meaning you can hear everything and anything created by someone else, and they can hear everything and anything created by you. This may create chaos, or it could create something interesting. If you hear anything offensive, please just use the app to flag it, and we’ll take care of it.
Moving Target
Stepping back, one thing that makes Marsbot unique is that, outside of work done with the now-defunct BoseAR platform, we haven’t seen purpose-built audio AR apps. Though SDKs could soon standardize audio AR development, as noted, Foursquare is being proactive with Marsbot.
And it’s the right company to do it. Though individuals outside of the local media and commerce spheres sometimes think of Foursquare as that check-in app from the 2010s, the company is many years into a fruitful pivot to build the location layer for the internet (see our writing here).
This provides a rich dataset for AR experiences, congruent with the principles of the AR cloud. In fact, we’ve long predicted that the AR cloud will benefit from location intelligence players like Foursquare with unique data on not only places but nuanced consumer interaction.
But because AR isn’t a prevalent consumer behavior yet, Foursquare knows that it can make the most impact where there are lower adoption barriers — per audio AR’s advantages. That’s why Marsbot is an experiment, as Foursquare knows AR will continue to be a moving target.
“The purpose of Marsbot for AirPods was never to attract millions of users,” said Crowley, “but rather to showcase how contextual-aware technologies will shape the future of AR and how Foursquare’s technology can be the foundation upon which those experiences are built.”
More from AR Insider…
