by 
One ongoing pastime at AR Insider is to examine ways that AR itself is broadening. The technology shouldn’t be forced into a narrow definition of graphics that overlay the physical world. “Augmentation” should include a more expansive set of ways to fuse the digital and physical.
The latest augmentation that came to our attention isn’t a new technology but one that isn’t often associated with AR: the growing trend of LED walls in film production. And there’s no better example of this trend than Industrial Light & Magic’s Stagecraft technology.
Known mostly for its use in the production of Disney’s The Mandalorian, it’s a cylindrical space whose interior walls are covered with LED displays. This lets filmmakers dynamically alter scenes, achieve continuity between foreground and background, and several other advantages.
But how does this all work, and why is this a noteworthy fusion of the digital and physical? To answer these questions, we’re featuring a handful of videos that dive into this technology for this week’s XR talks (embedded video and summarized takeaways below).
Seeing the Light
So what is this technology and why is it important? In short, it replaces the use of green screens in cinematic production, where characters perform in front of chroma-keyed backdrops that are then manipulated in post-production. It’s a common standard for blending live-action and CG.
But this longstanding practice has drawbacks that LED walls address. For one, lighting is an issue with green screens. The lighting in a given studio invariably won’t match the background animations added later. It requires post-production work to fix….and still often looks unnatural.
Worse, green screens emit a colored glow, or “spill,” that shows on reflective surfaces and must be fixed in post-production — meaning more production cost, time, and risk of unnatural appearance. LED walls conversely emit the correct lighting and color of the intended background.
That has a lot to do with the “L” and “E” in LED, which stand for light-emitting. Some post work is still needed for color and brightness matching, but the original footage is closer to finished. This makes things a lot more natural in the final cut, with less post-production rigor.
Perceived Depth
LED walls also hold cinematographic advantages beyond lighting. For example, green screens can’t recreate accurate parallax. For those uninitiated, parallax is perceived depth — or the distance between objects on an Z-axis — when perspective moves from side to side (x-axis).
With green screens, the background is at a fixed distance so parallax is never quite right. The same fixed distance is the case with LED walls, but the content can be shifted in real-time — depending on the camera angle — to synthesize proper parallax as the scene is being shot.
Beyond technical aspects, the real-time display of background scenery — as opposed to adding it later in post-production — inspires better performances. With green screens, actors have to imagine what will be added later. LED walls show the content in its full glory while shooting.
Much of this is done through 3D game-engine software. ILM specifically uses Unreal Engine. And the technology continues to rapidly evolve. Based on its advantages and cost-savings in film production, we believe it will quickly become Hollywood’s next standard for CG-heavy fare.
…But its true essence is better captured in visuals. Check out any of the videos above or below for more color…
More from AR Insider…
