The first season of The Mandalorian felt like a return to old-school STAR WARS, but the technology used to bring the series to life was cutting-edge. More than half of The Mandalorian was filmed in "an immersive and massive 20’ high by 270-degree semicircular LED video wall and ceiling with a 75’-diameter performance space, where the practical set pieces were combined with digital extensions on the screens." ILM created high-resolution digital environments which moved in relation to the camera; in short, it's basically a much more advanced form of rear-projection, which has been used in Hollywood for decades. ILM has released a behind-the-scenes featurette which explores how these virtual sets were brought to life, and it's well worth your time. Watching The Client's (Werner Herzog) office transform (:38-:41 in the above video) from a fairly bare-bones set into a complete one in the blink of an eye is actually astounding.
Want to move a mountain or add a tree? No problem. Want to spend hours shooting a scene at sunset? Let's do it. Want to switch to a location on a completely different planet? Easy. As production on The Mandalorian continued, Jon Favreau and his team began using the technology more and more, and by the end of the season, they were starting to design entire sets based on what the technology could do well.
Digital 3D environments created by ILM played back interactively on the LED walls, edited in real-time during the shoot, which allowed for pixel-accurate tracking and perspective-correct 3D imagery rendered at high resolution via systems powered by NVIDIA GPUs. The environments were lit and rendered from the perspective of the camera to provide parallax in real-time, as if the camera were really capturing the physical environment with accurate interactive light on the actors and practical sets, giving showrunner Jon Favreau, executive producer and director Dave Filoni, visual effects supervisor Richard Bluff, and cinematographers Greig Frazier and Barry Baz Idoine, and the episodic directors the ability to make concrete creative choices for visual effects-driven work during photography and achieve real-time in-camera composites on set.
This technology largely eliminated the need for location shooting, and had the added bonus of actually giving the actors something to react to beyond the usual green-screen. With production on the second season of The Mandalorian underway, it will be fascinating to see which directions Jon Favreau and his team have taken this technology.