Imagine spending months developing a unique AR experience only to realize it looks uncanny, fuzzy or washed out. Lighting is a fundamental design aspect that impacts user perception and rendering consistency. Understand how to approach development and avoid design mistakes.

Why Should You Care About Improving Lighting?

Proper lighting is fundamental for augmented experiences, whether developers create floating virtual overlays or photorealistic 3D models. It influences everything from immersion to usability.

The human brain knows what realistic environments look like. A simulation that does not meet its expectations will cause distress. Research shows that visual discomfort is among users’ chief complaints when using AR. They report blurry vision, fatigue or sore eyes. These ailments stem from visual inconsistencies, poorly rendered shadows, unnatural blur and too-fast motion.

Another reason to care about optimizing augmented experiences is the potential profitability. As AR marketing becomes more popular, consumers will grow accustomed to higher-end designs. Smaller developers will have to compete with the expectations larger companies set.

Leveraging AR for commercial purposes is generally advantageous. A 2022 Meta study found that marketing campaigns that incorporated AR ads achieved nearly triple the brand lift compared to those that did not. However, this success is not guaranteed. Developers must optimize augmented experiences to improve consumer awareness, perception and purchase intent.

Common Lighting Challenges Developers Face

AR relies on device sensors and cameras. Bright light creates overexposure, making detecting surfaces and tracking movements difficult. Low-light environments pose similar challenges. They introduce noise, obscuring the visual markers needed for positioning. Both can cause objects to appear misaligned or poorly rendered.

Camera quality depends on the mobile phone model, so optimization is necessary for a consistent user experience. Designing lighting for AR glasses is more challenging since only a fraction of the display’s light reaches the eye. The virtual object must compete against rapid ambient brightness changes over several orders of magnitude.

Virtual elements must match the ambient light’s direction, color and intensity. Natural light has an enormous dynamic range from less than one nit to 100,000 nits. Display technology has a hard time keeping up. For example, the average laptop can manage around 300 nits, which is 30 times less bright than a white wall illuminated by the sun.

Mistakes to Avoid During Optimization

The brute-force approach involves dramatically increasing the brightness to compensate for any changes. This is not ideal, as it drains battery life quickly and can overheat the device. Also, users may dislike forced screen brightness.

At the other end of the spectrum is ray-tracing, which creates seamless soft shadows. This approach is infamously resource-intensive. Modern computer graphics cards can manage, but mobile devices cannot. You must find another way to calculate shadows.

Relying on a single light source is one of the most common mistakes novices make. Good lighting design uses multiple layers to add dimension. Ambient light is the room’s base level of light. For maximum immersion and usability, consider accent, task and dynamic lighting.

The light probe images must be omnidirectional, with pixels representing each direction of the space and pixel values corresponding to the amount of light coming from those directions. Photographing a mirror sphere is the most common method for satisfying these requirements because it acts as a miniature, omnidirectional camera, reflecting all light at a single point.

However, layering multiple equidistant photos of the entire environment to create a spheremap is much more straightforward. You can overlap the resulting 360-degree spherical image with the environment mesh to illuminate virtual objects with realistic lighting schemes. You can create a new spheremap with new photos, so this approach is suitable for mobile applications.

How to Design for Different Lighting Scenarios

Consider lighting colors. If you place a 3D virtual object in a physical space, you generally want to match the ambient light as closely as possible. However, if you create floating graphics, you should consider color psychology. Pick shades that align with your brand image or the mood you are trying to set.

Aside from hue, you must consider intensity and direction. Design your light sources to align with existing ones. Occlusion testing in rasterization — a GPU technique that determines if an object is hidden from view by other objects — is key for calculating shadows. Shadow mapping is the most popular technique because it lets you deliver dynamic lighting in real time.

Initially, the scene renders from the light’s perspective. When it fully renders, the object’s position is converted into light space and compared to the depth data stored in the shadow map. If your result is blocky or pixelated, use percentage closer filtering. It samples the shadow map around a fixed center at multiple points to create soft edges.

Keep up with the latest AR technology so you know when to adapt your techniques. Already, manufacturers are using liquid-crystal-on-silicon microdisplays for transparent AR glasses. Since they work well in high ambient light conditions, they improve image quality and brightness while reducing power consumption.

Boost the User Experience With Better Lighting

However you refine your techniques, you should prioritize iterative testing. Find out if the virtual elements will remain visible if a sunbeam cuts through the user’s vision. Discover how long it will take for the dynamic brightness to adjust if the user walks outdoors. Even if your design looks good during the initial tests, you must trial it in all lighting conditions.

Whether someone is indoors or outdoors, you must maintain color accuracy and brightness. Render highlights, reflections and shadows correctly, and ensure they change dynamically with light sources. This effort will be well worth the finished product.

Eleanor Hecks is Editor-in-Chief of Designerly Magazine where she specializes in design, development, and UX topics. Follow Designerly on X @Designerlymag.

More from AR Insider…