Technology used in Mandalorian gaining adoption
The pandemic has spurred technology adoption across many sectors, and it appears it is also accelerating the use of virtual production and augmented and virtual reality enabled data in the film industry.
While it has been promised for years, the application of AR/VR technology is getting a new focus in the film industry at an “ideal time,” says Ollie Rankin, CEO of Pansensory Interactive in Vancouver.
Several Vancouver companies are working to bring virtual production technology and processes to the sector at a time when it is needed most. The conditions are right, says Rankin, but there are still barriers to entry, among them the fact that changes to the way films and television are made happen incrementally. Studios and producers are used to working a certain way and aren’t necessarily suggesting AR and VR be used in production.
“There are people in the AR/VR space developing technology to do virtual scouting and remote shoots and integrating live-action and virtual backgrounds or vice versa. But because a lot of those people aren’t at the table, it’s often not making it into the plan,” says Rankin.
Several visual effects and virtual production companies in British Columbia are active in this area, including Animatrik Film Design, which has studios in Vancouver and Los Angeles and has worked on films such as Avengers: Endgame and Aquaman, among others.
“I know of at least three visual effects companies in Vancouver that are building virtual production solutions. Animatrik has a small in-house virtual production capability, and they have been working with a couple of other studios to build out that capacity,” says Rankin.
Animatrik’s president and chief technology officer Brett Ineson says six months ago, people in the film industry had time on their hands and were worried about revenue and how to accomplish projects which has resulted in a renewed interest in the technology.
“We’ve seen a massive uptick in interest,” says Ineson. “The biggest interest is in people learning it and adopting it rather than it being deployed on actual projects. Virtual production seemed like it might be the answer, and in some cases, it probably is. It’s also fun and interesting, so it’s a perfect storm.”
In anticipation of increased demand for virtual production, Animatrik recently hired seven new people to start training on their technology. About 90 per cent of the staff has a 3D animation degree, a program offered at Capilano University in Vancouver and the Vancouver Film School.
“We’ve been promoting virtual production as a company for 10 years, but even before I started this business, I was in software working on virtual production software, and it’s always been something everyone loved but didn’t add it to their pipeline. It’s one thing COVID gave us is that people found time to really dig into it,” says Ineson.
Unity, one of the two main game engines used for virtual production and the main game engine used for VR, acquired a small B.C. studio called Digital Monarch Media in 2018 and tooled it to be focused exclusively on virtual production. And Unreal Engine, the other game engine, has made a large investment in virtual production and is the leading real-time technology in virtual production right now.
“They have been recruiting people from the visual effects and film and television production industry and running intensive bootcamps. I just finished a five-week intensive bootcamp and filmmaking hackathon — 100 of us from around the world went through the training process, and each of us produced a short-animated film at the end of it,” says Rankin.
Virtual production was used in the making of the hit Disney+ Star Wars Mandalorian TV series and the most recent version of The Lion King. It uses some of the same technology as AR/VR but doesn’t necessarily involve any virtual reality headsets or augmented reality devices as most would recognize them. Now that the industry knows how the Mandalorian was shot, it’s become the thing that people want to use, says Rankin.
The film sector in British Columbia was worth $4.1 billion in 2019. Vancouver’s special effects industry has created a demand for skilled professionals, and many graduate from Canada’s film schools. As the industry ramps back up, there will be greater demand for this ever-evolving skillset as more special effects are used in films and television.
Virtual production allows for the filming of an actor in front of an LED screen, and it is updated in real-time based on the position of the camera. Conversely, you can have actors in front of a green screen and in real-time in the camera feed or in the pipeline from the camera feed to the video village (the area around the monitor on set); you can incorporate live real-time computer-generated imagery.
“The studios and producers have seen behind the scenes footage of how the Mandalorian was done and it’s become the thing people want. It allows you to bring the visual effects earlier into the production process,” says Rankin. “So that means the art department is brought in at scouting time rather than visual effects being entirely a post-production thing. That gives all of the filmmakers more ability to collaborate on the ideation and creation of those elements rather than it being an afterthought as it has been traditionally.”
Virtual scouting – saving time and money
VR headsets are being used in virtual scouting where film production teams send crews to check out a location but with fewer people involved and more application of the technology.
If, for example, a production wants to shoot in the Grand Canyon, they can send a small team to the location and use cameras and LiDAR (Light Detection and Ranging) scanners to essentially capture scenery in three dimensions with high dynamic range colour.
“In a situation where you’re not using a real-world location, such as an alien planet, rather than having to build it physically on a sound stage, you can build it with computer graphic technology,” says Rankin. “You either start from a real-world environment or an imagined environment, but the intermediate step is when you have a full CG digital recreation of the scene or the environment of the location that exists inside a real-time game engine, usually Unreal or Unity.”
At that point, the director and director of photography can use any number of different devices such as a VR headset to teleport into that virtual environment, or they use their phone or iPad.
As they move it around, it’s as if they are filming inside that virtual environment. “Virtual scouting, at minimum, is the ability for an individual to point an imaginary camera around an imaginary place to see what it would look like with a particular lens. Where the industry is headed more and more is that it’s a collaborative experience. The director, the production designer, the cinematographer, the gaffer, and probably a visual effects supervisor and producers — can, whether using VR headsets or their phones, remote into this virtual place and line up shots,” he says.
Cost factor
While virtual production technology often brings down the cost of production, AR/VR hasn’t reached a point where it has made processes cheaper because of the upfront costs.
“For studios using LED panels, the upfront costs for building one of those stages is in the millions of dollars, so it’s not cost-effective yet, but virtual scouting can be considerably cheaper,” says Rankin. “It’s not orders of magnitude cheaper, but it is allowing things to be better because it prevents issues later or creative compromises having to be made due to ill-informed decisions made earlier on.”
Jennifer Brown is the Editor at CourseCompare.
Leave a Reply