The following is an excerpt from Defying Reality: The Inside Story of the Virtual Reality Revolution by David M. Ewalt.
After the bright lights of Vegas and overstimulation of the Consumer Electronics Show, I was looking forward to a change of pace. So at the end of January 2016, I flew to Salt Lake City, Utah, and then drove east into the Wasatch Back region of the Rocky Mountains. I checked into a hotel in a small town, put on my snow boots, and started walking.
A few hours later I was watching the sun rise over a quiet mountain lake, admiring snowcapped peaks off in the distance, and listening to a gentle piano sonata playing on headphones. I felt relaxed and content. And then the tranquil scene was interrupted. A long, low whistle, like breath exhaled through clenched teeth, echoed across the water. A plume of thick gray smoke traced its way along the distant shore, and then it turned, moving closer. It took me a moment to realize what I was looking at: an old steam locomotive, speeding across the lake’s surface, straight toward where I stood. I watched, transfixed, as it approached, and then moments before reaching me, it exploded, transforming into a flock of black birds that swarmed around and past as the music came to a crescendo.
When I took off the headphones and Gear VR headset, I was still in the mountains but at a different setting — a demo room in the New Frontier Exhibition at the Sundance Film Festival in Park City, Utah. The New Frontier program started in 2006 as a showcase for cutting‑edge media and technology at the annual gathering of indie filmmakers; ten years later, it was dominated by virtual reality experiences, and I was there to check out the artistic side of the VR revolution. Tranquility and relaxation would have to come later.
The exhibition was located in a gallery space on two floors of a hundred‑year‑old hotel on Park City’s Main Street, and home to more than two dozen different stations outfitted with Rift, Vive, and Gear headsets. Many of the VR experiences on dis‑ play amounted to simulated environments, like the mountain lake of Chris Milk’s Evolution of Verse. Another demo, Jake Rowell and Ben Vance’s theBlu: Encounter, took me on a dive to an underwater shipwreck, where I watched as fish and rays and ultimately a massive blue whale swam around my head.
These experiences had action and setting but no real narrative or plot. They reminded me less of modern films than of nineteenth‑century magic lantern shows, or the very first moving pictures. The idea seemed to be to emphasize the immersive nature of VR rather than tell a particular story.
Other experiences followed more traditional narratives, like Janicza Bravo’s Hard World for Small Things, a dramatic short that put the viewer in the backseat of a convertible driving around Los Angeles, watching a group of friends who get involved in an accidental shooting. It exploited the first‑person presence of VR to create a kind of you‑are‑there thrill, making the shooting seem a little more real and visceral — but since the video was just from a static 360‑degree camera and the scenes lacked any real interactivity, it felt less like a great leap into a new medium and more like a very fancy 3‑D movie.
But as I passed from station to station trying out the different VR experiences, one of them did truly surprise me. Sylvain Chagué and Caecelia Charbonnier’s Real Virtuality: Immersive Explorations was a multiuser immersive simulation built around a real‑world stage set — a roped‑off section of the room with some odd‑looking boxes and props strewn around the space. As I watched from the line while waiting to try it out, two participants donned Rift headsets modified with multiple protruding antennas and white tracking dots, a pair of tracking gloves, weird devices strapped to their shoes, and large white backpacks filled with wireless computers. They looked more like astronauts on a space‑ walk than attendees at a film festival and, as they went through the ten‑minute demo, seemed to be engaged in some equally high‑ tech and alien activity. They traced odd paths around the floor, leaning and twisting around obstacles that only they could see, occasionally picking up one of the strange‑looking props and carefully inserting it into a hole in a box resting on a pedestal in the middle of the room.
On my turn, I was paired up with another festival attendee, a middle‑aged advertising executive named Matt, and we helped each other strap on the equipment. When the headset powered up, I found myself on the bridge of a futuristic spaceship, stars shimmering out one window and Earth spinning by on the opposite side, but when I turned to where Matt had been standing, he had been replaced — instead, there was a small being with a shaved head, wire‑frame glasses, and a skintight black space suit. I raised my hand and waved at it tentatively.
The avatar cocked its head and raised its own hand, waving back, and then I heard Matt’s voice from across the room. “Hi. Wow, is that really you?”
Chagué and Charbonnier’s Sundance demo was designed to offer a room‑scale VR experience, something like the HTC Vive, but it used a custom setup of twelve infrared cameras tracking small sensors attached to the headset and to each player’s feet and hands. Objects in the real world were also marked with the sen‑ sors, allowing them to appear in the virtual world and be manipulated and tracked too. In virtual reality, the box I’d seen on a pedestal floated in midair in the spacecraft, like some kind of futuristic flight control.
A narrator’s voice invited us to explore the space, and I took a tentative step forward, like a baby learning to walk. Looking down at my feet, I could see two legs clad in a black space suit — not my legs, but they seemed to match my movement, so I inched forward, shaky at first, then took long and clunky Frankenstein stomps around the room.
As I got used to my new body, the narrator explained that the floating box was a time machine, and that we had to find special crystals in order to activate it. Clomping around, I spotted a glowing yellow mineral on the floor near the ship’s pilot seat — an obvious illusion, but when I reached down to touch it, something was actually there. In the real world, it was just a piece of wood with tracking dots glued onto it, but the tactile feedback of actually touching something real and seeing it in the virtual was pro‑ found. In that moment, I bought into the illusion entirely: I was on that spaceship, and I was about to activate a time machine.
From behind me, I heard Matt shout that he’d also found a crystal, and then we both turned and made our unsure steps toward the box in the center of the room. Color‑coded holes in the box glowed in anticipation, indicating where we should place the control rods, and we both inserted them, holding our breath. A whooshing sound filled the room, and everything turned white.
When the bright lights faded, I was standing somewhere else — on the deck of an open‑air platform moving through the urban canyons of a futuristic cyberpunk city. Neon signs for strange products and unimaginably tall skyscrapers flashed past. Matt and I explored another scene, this one with two virtual characters dancing on the craft enjoying a date, before finding two more crystals and transporting ourselves to our final — and most convincing — virtual location.
We were in some kind of underground cavern, standing on a stone platform above a floor studded with glowing stalagmites. In front of us, the time machine had now opened to hold up a wooden torch, already ablaze. Matt’s hand reached for it and then recoiled, like he’d been burnt.
“Whoa,” he said. “It’s really there!”
Even though I was a little afraid that I might burn myself, I reached out and grabbed the torch by the handle. It wasn’t hot, but it was still a shock to feel the virtual object in my hand. With a simple bit of stagecraft — one of the Real Virtuality staff must have placed the object in the box during the whiteout time travel animation — the team had provided a sense of physical proof that we’d been transported, and gave us a totem that made the illusion feel even more real.
I lifted the torch (in reality, just another stick decorated with tracking dots, animated in the virtual world to look like a light source) and waved it around, and the light from its flames illuminated the cavern. We were standing at the entrance of some kind of underground structure, like a subterranean Petra or the entrance to a haunted temple in a fantasy role‑playing game. The effect was intensified when the narrator gave us instructions that sounded like something a Dungeon Master might say: “Find the magic stone at the end of the path…but before that you will have to face your fears.”
In the words once spoken by a nerdy character on the TV show The X-Files: I didn’t spend all those years playing Dungeons & Dragons and not learn a little something about courage. I held out the torch in front of me to light the way, and stepped toward the structure’s rough‑hewn stone walls. At its opening, the floor abruptly fell away, replaced by a rickety‑looking wooden bridge suspended high above the rocky cavern floor. Two steps into this virtual world and I’d already have to face one fear — I’d have to make a daring Indiana Jones–style scramble across the gap and risk plummeting to my death.
I held my breath and my skin went clammy as I took my first tiny steps. The bridge looked so unsafe, like it could fall apart at any second, but it held and took my weight. My heart pounded as I inched forward step by step, and my head swam with a sense of vertigo, like I might actually lose my balance and fall over the side. But soon I was close to the stone floor on the far end, and with a panicked leap, I landed on the more solid floor and sighed in relief that I’d made it alive.
Then I heard a panicked voice from behind me, in the dark. “Hey! Don’t leave me!”
In my anxiety to make it into the temple, I’d forgotten about my teammate, Matt. Turning around, I could barely see him standing in near darkness at the edge of the bridge, his virtual form slightly hunched, one arm reaching toward me, imploring for help. I stepped close to my edge of the bridge and held out the torch above the gap, giving him a little more light.
“I’m so sorry!” I told him. “It isn’t that bad. Just walk across. I’ll wait for you here.”
Matt nodded, but his feet didn’t move. He crouched slightly, and his head looked down, staring through the slats of the rickety wooden bridge. Even though it was expressed through a computer‑ animated avatar, the body language was clear: he was frozen in fear, afraid to take a step, petrified that he would fall from the passage and onto the rocky floor below.
I shifted my torch into my left hand, took a step back onto the bridge, and extended my right hand out toward him. “It’s just a few steps,” I said. “Come across, I’ll help you.”
After a few moments, he took a tentative step, placing one foot on the bridge. Finding it solid, he then dashed the rest of the way across, grabbing my hand and then pulling himself off the bridge and onto the solid stone platform.
We stood there a short while, still holding on to each other, catching our breath. Then without further discussion, we walked forward together. The passageway twisted and turned, and with only the light from my torch to show our way, there was darkness ahead until I was able to reach around each corner. At one point, the light illuminated a spider the size of my hand scurrying up a wall, and I recoiled from it, bumping hard into Matt as I swept the torch at it in an attempt to burn it. Then the corridor ahead was filled with cobwebs, and as I passed through them, I winced and tried to brush them out of my hair. We jumped over another pit trap, hearts once again racing, and then crawled under a fallen stone pillar, squeezing our bodies close to the floor in order to fit through the passage.
Finally, we reached the end of our journey: a stone platform looking out into a cavern full of glowing crystals. As the narrator congratulated us on completing our journey, a basketball‑sized orb of fire floated toward us. Matt extended his hands, and it drifted into his grasp. A whooshing sound filled the room, and everything turned white again.
“You can take your gear off,” a voice said. I’d almost forgotten. I lifted the Rift from my eyes and had a moment of shock when I saw I was still in the New Frontier demo room, not an ancient dungeon, and was standing next to a middle‑aged guy wearing funny tracking gloves and a white backpack, not a space‑suited time traveler. We’d both gotten lost in the illusion, and all the time we thought we were traversing rickety bridges, tiptoeing through spider‑infested corridors, or crawling under obstacles, we’d just been pantomiming our way around an empty room while a long line of indie film fans stood around, waiting for their turn.
“That was insane!” Matt said with a huge grin on his face. “Wow, that was insane. Incredible.”
“It was!” I replied. “I knew that I was doing a VR demo, but in the moment, it was so real. I was really worried I was going to fall into that pit.”
“The bridge got me. I actually had to stop and think about it. That was insane. Bravo! That was incredible.”
Going into Sundance, I’d assumed that the future of VR movies would proceed something along the lines of the history of traditional filmmaking. I figured the first VR films would mostly capture and replicate the real world, attempting to convey the experience of being in a place you couldn’t really visit. Later attempts would look and sound better, until the simulations were basically indistinguishable from reality. If a VR documentary film promised that “you are there,” it would actually make it happen. A VR drama would work much the same way, yanking the audience out of their theater seats and onto a set where they could explore and watch the characters. I imagined being an invisible presence in the room when the Godfather promised to make “an offer he can’t refuse,” an unnoticed visitor to the Star Wars cantina, an anonymous rider on the runaway bus in Speed.
It hadn’t occurred to me that the real power of VR cinema is to completely obliterate the line between audience and performer. At the simplest level, an audience member can be a passive participant, a friend in the car, a person other characters talk to instead of just an unacknowledged viewer. At a more interactive level, their observations can actually cause the action — an actor could wait to speak until the viewer looks at him, or a scene won’t start to unfold until the viewer walks into the right room. And then there are VR films that could play out like Real Virtuality, where the viewer is actually the star of the film; exploring a set, interacting with other characters, and overcoming obstacles. It’s the difference between inserting the viewer in the Temple of Doom alongside Indiana Jones, and putting the viewer in Indy’s actual place. The future of filmmaking isn’t just the most immersive 3‑D movies ever—it’s a blend of cinema, video games, immersive theater, and fantasy role‑playing. In The Matrix, we all are Neo.
And with that thought, I pulled my parka back on and headed back out into the snowy streets of Park City.
David M. Ewalt is a contributing editor to ForbesMagazine, and the author of Of Dice and Men: The Story of Dungeons & Dragons and the People Who Play It.
