This article is the latest in AR Insider’s editorial contributor program. Find out more or contact us to participate here. Authors’ opinions are their own.  

Best practices for bringing AR applications to the field

Learn how industrial giant ABB is using Unity and augmented reality to transform field maintenance procedures into a completely paperless process.

by Nick Davis

We recently invited members of ABB’s IS Innovation and Digital Scouting team, Maciej Włodarczyk and Rafał Kielar, to walk us through how they used Unity to develop a new digital field operator system. This multiplatform application runs on mobile devices and the HoloLens, and it’s designed to improve the efficiency and safety of the field operators maintaining and servicing equipment on industrial sites.

Let’s explore the problems ABB sought to solve for its clients and some of their key learnings from the development process (you can also see a recent webinar that explored their insights, and a session at Unity Copenhagen below).

What are some challenges faced by maintenance operators?

For ABB’s customers, two key personnel are involved in this process: the field operators responsible for maintaining and servicing equipment, and control room operators who supervise the process and are located in another part of the facility.

Due to the dangerous nature of the tasks involved, field operators have traditionally undergone time-consuming, expensive training programs before working on-site.

Once they are in the field, however, it is difficult to assign and track performed service procedures, which are done on paper. This leads to further communication issues between field and control room operators, who often need to exchange information in real-time.

How did ABB solve these challenges?

ABB used the Unity Editor and Microsoft’s Mixed Reality Toolkit (MRTK) to test prototypes quickly and eventually build a production-ready software application called ABB Ability™ Augmented Field Procedures. This multiplatform application completely digitizes the field operator experience with remote-enabled augmented reality technologies.

The application provides several advantages over the traditional, paper-based workflow. This system:

— Allows any field operator to follow digitized procedures and become an expert without costly training
— Integrates the field and distributed control systems to enable real-time data capture (versus processing paper-based forms afterward)
— Ensures that the latest version of a procedure is always followed (rather than using an outdated document)
— Connects field and control room operators for real-time communication using Microsoft Remote Assist

What are some best practices for designing wearable AR applications for field use?

Based on their experiences, Włodarczyk and Kielar from ABB shared numerous best practices for those developing similar applications related to training, guidance, and maintenance use cases. In this post, we focus on several best practices for the HoloLens application, centered around the user interface (UI) and ergonomy of interactions.

Check out the webinar for a complete list of ABB’s recommendations, including location/device recognition, hologram positions, and more.

Make the UI unobtrusive

The placement of the UI should not interfere with a user’s ability to perform tasks.

As seen in the image above, the UI should not obscure the user’s view. UI elements that block the real-life objects that the user needs to interact with can pose a safety hazard.

Make it easy to control what the user sees

To minimize clutter in the user’s field of view, allow navigation menus to be accessed on request rather than being omnipresent. In the video above, notice how the user controls the visibility of the menu with a gesture.

In order to avoid blocking the field of view, some may think smaller menus and buttons make sense. On the contrary, these should be large enough to be easily targeted by gaze and selected by gestures.

Make interactions intuitive, and automate as many of them as possible

Włodarczyk and Kielar needed to make their app easier and more convenient to use than the paper-based Standard Operating Procedures (SOP) their clients were used to.

That led them to automate as much of the experience as possible in order to limit the number of interactions the user needed to perform (e.g., having a window automatically appear following a gesture, as shown in the video above). It’s also important to provide clear instructions (e.g., the “tap to dock” message shown in the video) to ensure that the next step is always clear.

Users should also be given the flexibility to select the interaction mode of their choice, such as voice commands, gaze, and gestures.

Pressure-test with users who are new to AR

Since most field operators have limited experience with AR but will be the end-users of these applications, it’s critical to share the app with a test group comprised of these individuals. They will be a source of valuable feedback that will help you to reduce complexity and streamline your app to its core components.

Nick Davis is Senior Content Marketing Manager for Automotive, Transportation & Manufacturing at Unity Technologies. A version of this article originally appeared on the Unity Blog, published here with permission.  

More from AR Insider…