How Creatives Are Hacking the Movie Set for Virtual Reality

A 360-degree camera can see everything in the room, so VR directors are designing new workarounds

Signe Brewster
Sep 15, 2017 · 7 min read

Listen to this story



Child: Do not try and bend the spoon. That’s impossible. Instead, only try to realize the truth.

Neo: What truth?

Child: There is no spoon.

The next time you put on a virtual reality headset, watch a live-action film and try to be like Neo. Look around and realize that as realistic as the scene might look, a film set without cameras, lights, or sound equipment is impossible. You’re not inside The Matrix, but what’s really going on here?

The 360-degree nature of VR cameras creates a new challenge for directors, who now have to figure out how to make a film where the camera records everything within sight. A VR camera looks otherworldly — like a high-tech hydrangea. The high-end versions feature a dozen or more cameras arranged in a sphere, lenses pointed out. Once you film something and stitch all the videos together, you get a 360-degree movie suitable for viewing inside a virtual reality headset.

Directors have responded to the new technology with a slew of design choices that affect everything from set design to how they rig lights. Some directors leave the room before the camera starts rolling, while others choose to remain in the room and edit themselves out later on. No matter what, filming for VR requires crews to think hard about whether every decision they make is suited for VR.

Everyone Out

The simplest way to shoot a VR film is to have everyone but the actors leave the room. A 360-degree camera sees everything, so if you don’t want the crew caught on film, they can’t physically be in the room.

On the set of the World War II–inspired The Mission, I watched a dozen people cram into a tiny bunker to set up a shot, and then file out when it finally came time to film. From outside the bunker, we could hear the two actors shouting their lines. The director couldn’t know if he got the right shot until he played the film back later.

To make someone really feel like they’re in the room with actors, you need to convince them it’s a real room. Felix & Paul Studios’ VR film Miyubi places the viewer at the center of the story as a toy robot. If you walked onto the set of Miyubi, you might not realize it is a film set. The crew transformed an entire house — from the bedrooms to the decor — to look straight out of 1982.

“We had to build very complex sets, because they had to be perfect in 360 degrees,” says studio co-founder Félix Lajeunesse. “You wouldn’t know that this was an actual cinema set. It was quite exhaustive and much longer than the way we used to dress sets when we were making [traditional] films.”

Dressing an entire house also plays into how people see the world of the movie. As we move through the real world, we don’t experience cuts to change scenes. Everything is continuous.

To make the experience as real as possible, the directors at Felix & Paul Studios decided to use cuts that are as long as possible. Having an entire house as a set allowed them to follow characters from room to room without breaking the illusion.

Director Félix Lajeunesse addresses actors in “Miyubi” while filming the movie’s Christmas scene.

Each scene ended up lasting six or seven minutes. That in turn put further weight on the staging and acting to be excellent. Lajeunesse says many actors enjoy knowing their work will appear in the final film unedited and uncut.

Post-Production Magic

Placing a viewer inside a scene makes them less accepting of shortcuts, because their feeling of presence depends on it. But props are only one part of the problem.

One of the more interesting challenges for VR filmmakers is creating film-quality lighting that blends into the set. Some directors build lighting into the scene: Two characters may be speaking on a street at night, for example, but there’s a streetlamp overhead. That’s much more believable than a ghostly glow projecting from the sidewalk.

Luckily, it’s the year 2017, and VR directors have the post-production technology to edit how a shot looks.

In Miyubi, the directors wanted well-lit rooms without having to add a million lamps for natural-looking lighting. They ended up rigging traditional set lights to the ceiling and editing out the lights later. Traditional set lights likely would have been on poles, but those would have been harder to edit out. Keeping everything on the ceiling was cleaner.

Lajeunesse says some parts of Miyubi originally had 17 crew members present in the shot. Sometimes there is no way around keeping extra people around, and it is possible to remove them in post-production.

These types of editing decisions shouldn’t be made lightly. A documentary, for example, requires a truthful representation of what really happened. When Felix & Paul Studios filmed an interview with Barack Obama in the White House, they fashioned a robotic platform that could execute the camera motions they envisioned. They set the camera into motion, and then left the room.

Directors Still Matter

A 360-degree field of view is still a novel idea. As a result, when people step into virtual reality for the first time, they tend to test their boundaries. Can they walk around? Can they pick up that object over there?

VR filmmakers have to lead viewers through the viewing process with clear indications of where they should look and what they can do. If the filmmakers fail, they risk viewers missing part of the story.

“In TV or theater, you have a captive audience. Everyone is always looking exactly straight forward,” says Maureen Fan, CEO of Baobab. “In 360 or in VR, you can look anywhere you want. It’s really important to direct the viewer’s eyes.”

Fan began her career as a UI designer, with a formative stint at Zynga. As a result, Baobab tests everything, including what directing choices keep viewers’ attention trained on the action.

She has found that the most exciting action should take place where the viewer is supposed to look — anything else is distracting. An easy way to reinforce this is to have characters within the film look toward the action. Human viewers naturally follow their gaze.

Another technique to keep attention in the right place is to create framing within the 360-degree world. In Baobab’s Invasion, an alien spaceship flies into the scene from between the peaks of two mountains. The mountains create a natural frame that draws the eye and makes the spaceship stand out as it approaches. Baobab also chose to make the film’s main character — a bunny — white so it stands out against the darker background.

While live-action films like Miyubi benefit from more realistic lighting, the animated films Baobab creates have more flexibility. Baobab’s film Rainbow Crow begins, for the most part, in the dark. Then a floodlight begins to fill the scene and gets brighter and brighter as the film progresses. It’s an effective tactic to keep the viewer’s eyes trained on the action and adds to the story’s mood in the process.

A significant piece of feeling immersed in a VR story relies on feeling emotionally connected to characters. That also plays into keeping their attention in the right place. The acting has to be good or else viewers will start to look around.

A 2015 VR film by Oculus Story Studio called Lost was widely shown as a sample experience for the new medium. The short movie places the viewer in a forest, where they watch as a dog-like robotic hand reunites with its robot owner. It’s adorable, but the film left too much distance between the viewer and the action. The viewer isn’t part of the story.

As VR directors get better at solving the technological challenges facing their emerging industry, they can’t forget the human element. Just like a traditional film director doesn’t want viewers to be more interested in their phone, a VR film director doesn’t want to make viewers feel trapped inside their headset with nothing to watch.

“I think it’s too early to be defining things,” says Fan. “It’s all about testing and prototyping. It’s about having a hypothesis and trying and seeing what happens.”

Trinity: What’s he doing?

Morpheus: He’s beginning to believe.

Written by

Science and technology journalist

Designing for Virtual Reality
Designing for Virtual Reality
Designing for Virtual Reality

About this Collection

Designing for Virtual Reality

An exploration on the lessons, trends and hurdles for designing virtual worlds, from tech journalist Signe Brewster.

An exploration on the lessons, trends and hurdles for designing virtual worlds, from tech journalist Signe Brewster.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade