Virtual Reality is in Desperate Need of Real Storytellers

They say VR is like sex—you never forget your first experience. For some, the first time is unsettling as it was for tech columnist Farhad Manjoo (“Tripping Down a Virtual Reality Rabbit Hole,” NYT, June 22.) He complained that VR is “too immersive” because it doesn’t allow for multitasking or distraction. It’s a response I found curious after years of reading reports on how digital distraction is eating our brains.

Virtual reality, along with its less immersive cousin, augmented reality, is by some estimates projected to become a $120 billion market by 2020. But many made skeptic by reading articles such as Manjoo’s still haven’t tried it. I was one of them until recently. The thought of taking on yet another technological platform was about as appealing as colonoscopy. Then, I stumbled into the Virtual Hub of this year’s Tribeca Film Festival.

Elevator doors opened to what looked like a disco. An enormous darkened room pulsed with music and blue light from screens. People twirled in swivel chairs, wearing what looked like ski goggles, topped by huge headset earphones. Others stood waiting, watching their faces—what little of their faces was visible under machinery—alert as straphangers surveilling sitters for signs that a seat will soon vacate.

Now, watching others doing it, hearing them talk about how cool it was, I was suddenly anxious to lose my VRginity.

I looked around to see which line was the shortest. Only a few people were queued for Seeking Pluto’s Frigid Heart and soon a young man wearing a T-shirt marked “Crew” guided me into a swivel chair. He advised me to set my bag a few feet away, so my feet wouldn’t trip over it. I worried that the bag might be stolen as goggles were fitted over my head, making me blind, and I became deaf when cushioned headphones slid over my ears. But in the next moment, I forgot about my bag, the room, even my physical coordinates on the planet. A movie started on a screen just inches from my eyes, but I wasn’t watching a movie, I was moving inside it, floating bodiless through space, serenaded by surround-sound opera, hurtling between stars, over rugged Plutonian mountains, through snow flurries so real I could almost feel them pinging my incorporeal cheeks.

Some people get what’s called “VR vertigo” because of the disconnect between brain and body. Your brain is convinced your body is doing something it’s not: flying or walking or running or falling. I didn’t get vertigo as I flew over Pluto, the planet that no longer was. What I did feel was scalp-tingling recognition that what I was experiencing was an entirely new medium with the potential to impact my life as momentously as the internet had. Here was a way to be in a place where my body wasn’t.

When the movie was over and I pulled off the goggles, I saw my bag was just where I left it, undisturbed. But it was I who was dislocated, my consciousness altered by experiencing an entirely new way to tell story.

The technical term for what I was doing was engaging in a “stereoscopic VR experience.” This is the starter kit for VR—you insert a smartphone into a headset, put on the headset, plug into earphones and you’re off to the VR movies. The headset used at the festival was top-of-the-line Samsung Gear VR which you can buy online for about $100 (not including the requisite Samsung phone) but for $20 you can turn your own smartphone into a VR viewing machine with one of countless Google Cardboard VR viewers. Google Cardboard devices—most of which aren’t made of cardboard—are why VR is finally going mainstream after being touted in 1991 when helmet, gloves and other gear needed to watch it could set you back $50K.

Lines were longest for shows you couldn’t see from a swivel chair. At this point, there are two levels of VR experiences available to non-geeks like me. Level One—let’s call it that, because common parlance is in terms of manufacturer’s equipment—involves smartphone and a headset. The best way to see Level One is to sit in a swivel chair, so you can twirl around, taking in scenes all around you. The next level, Level Two, provides an even more intensely immersive experience. In Level Two VR, you not only experience a scene in 360, you (minimally at this point) take part in it. Level Two VR requires that you be tethered by wires to a computer and for this reason it’s shown in a booth like a dressing room, curtained off from the crowds, in the company of a spotter who helps you into the gear and hovers nearby to make sure you don’t kill yourself by tripping over (actual) wires or walk into (actual) walls.

My first Level Two VR experience was for “Dragonflight.” After an hour’s wait, I ducked under a curtain and was welcomed by a spotter who fit me with goggles and headset, then connected wires that cascaded down my back. I was nervous and excited as if being strapped into my first ride on a roller coaster.

What felt like television remote controls were pressed into my hands.

“Ready?” As soon as I nodded, I was lifted into the path of a fire-breathing flying dragon, his nostrils closed in on my face. I shrieked and the spotter guided me, knees wobbling, to a green grid overlay where I was prodded to sit on the dragon’s back. (In the real world—a chair.) The dragon took off, and we were flying between mountains, swooping over battlefields and instinctively I flamed my “fireguns” to eradicate fly-by Vulcan enemies. By the time the dragon alighted on a cliff, allowing me to slide off his back (the chair), I was breathless.

The show getting the most buzz was Allumette. It was twenty minutes, twice the length of other features. Its animation was astonishingly high-caliber, from a studio headed by alums of Pixar and DreamWorks. But the power of Allumette lay only partly in the quality of its production. The real reason that the waitlists were up to nine hours long, was that unlike other productions on offer, it went beyond the spectacle of its technology. It told a story.

Allumette reconceptualizes Hans Christian Andersen’s “The Little Match Girl” as a modern allegory about love and loss. It takes place in a cloud-borne village populated by thumb-sized characters. The stop-motion-like animation is wordless; the music haunting. Watching it is like peering into a diorama in which characters come to life. A few minutes into it, I realized that I didn’t have to stand in one place. I could walk around in the scene, crossing bridges, winding down streets, peeping through windows, examining things from all angles. But at first, I couldn’t make myself move. My brain knew I was standing on a linoleum floor, but my heart believed my feet were on a cloud and if I stepped off, it would be into thin air. I had to muster actual bravery to push myself forward, and as I did, I tingled with amazement at this entirely new kind of entertainment and at the possibilities I sensed in VR for storytellers.

6 CONTEMPORARY WRITERS WE’D LIKE TO SEE AS VIRTUAL REALITY STORYTELLERS

In his column, Manjoo goes on to call out VR for being “a lonely, anti-social affair”—but, hey, isn’t that what reading a novel used to be? I mean, before we figured out how to make books ping and arouse competitive instincts by flagging favorite passages of readers who got there before we did. (I don’t mean to harp on Farhad Manjoo or to denigrate his excellent work; his reticence is shared by others in both legacy and new media cautioning against VR.)

I ordered a viewer and trawled for VR content to watch. There’s some remarkable pieces available for Cardboard, including many works showcased at Tribeca, but most are non-narrative experiences delivering the thrills and chills of amusement park rides. This is to be expected, I suppose, because the first wave of content is being created by manufacturers targeting first adopters, gamers eager to interact with characters from worlds familiar to them: flying dragons, zombies, dirt rally racers. Investors see huge potential for VR in gaming, but what about the vast potential audience of millions who aren’t gamers? Film buffs and readers and watchers of long-form TV for whom the lure of VR lies not in games or in fleeting experiences like exotic safaris and planetary adventures—but in the sustained pleasures of entertainment created by combining technology with the power of narrative.

What’s out there for us can be watched in a weekend.

When televisions first came out in 1939, shows aired just twice a week, created by manufacturers who knew no one would buy their products unless there were reasons to watch it. The shows weren’t created for television; they were created for the platform that preceded it: radio. Programs were basically announcers on mics until storytellers figured out that filming stories about the Wild West could keep viewers tuning in.

Investment is pouring into the industry; what it needs now is an infusion of creatives who can figure out how to tell stories in this pioneer medium.

Writers in every media are bemoaning the dearth of paid gigs. The Author’s Guild cites a 30 percent decrease in annual income for authors since 2000. Newspapers and magazines are going out of business. Risk-averse Hollywood is doing remakes. Could VR be the equivalent of a modern WPA project, putting writers back to paid work?

I spoke to Eugene Chung, founder of Penrose Studios, that created Allumette. “It’s not only new technology we’re creating,” he said. “We’re inventing a new language of storytelling.”

I polled him and others on the VR frontier for tips on writing for the new genre. Here’s a topline of learnings I gleaned from them, and from hours of tripping down VR rabbit holes myself:

1. Pacing. It’s slower. Audiences haven’t yet learned to “read” VR. They can’t be expected to pick up on what’s happening as quickly as they would in a conventional film. A simple establishing shot is the best opening. Or a black or white screen that slowly fades to scene, allowing the viewer to get her bearings. VR creators call the choice made for the opening “the in.” Allumette‘s “in” is to open in darkness; then one by one, yellow squares light up all around you; you realize they are windows, a device that encourages a new-to-VR audience to look all around instead of simply straight ahead, as film audiences do.

2. Much of what works in film doesn’t translate to VR. Filmmakers spent decades creating visual vocabulary audiences would understand, i.e. “cut” means that action continues, even though the picture changes. But cuts, pans and zooms are too jolting in VR, at least for now. Most VR storytelling tools haven’t yet been discovered.

3. Think in spheres instead of rectangles. A film is a series of rectangular photos. A VR film is like photos stitched together and curled into a globe, with the audience at its axis. Don’t think of shots inside frames, think of viewers at the center of spherical worlds that you, the storyteller, must set-design, attending to all that is above, below and behind the viewer, in order to make the experience believable. Even if one small object is out of place, that discrepancy is enough to break the illusion. Writers of screenplays and TV series create “Bibles” which help them keep track of myriad details of the world in which characters live—writers of VR create similar compendiums to help them maintain continuity.

4. More than one thing should happen at once. Unlike film in which a story is told in linear fashion, scene following scene, actions happen simultaneously in VR. In fact, if only one thing is happening, the scene feels fake because actions rarely unfold as isolated incidents in real life.

5. Create for a viewer who can look anywhere, not just straight ahead. Instead of a scene being confined to a squared-off rectangle as it is in a film, it’s revealed in 360-degrees, more akin to how one perceives the real world. Because VR viewers are free to watch what’s happening behind, above and below, you must figure out how to direct their attention. In this way, writing for VR is more like writing for stage. For instance, on stage sound is commonly used as a device to direct viewer attention. If a gun goes off on stage, you can bet that’s where a director wants the audience to look. Sound proves a similarly useful device in VR.

At this year’s Cannes film festival, director Steven Spielberg worried that giving audiences the choice of where to look could ruin movie-making because it took away the director’s control of the story. But this reminds me of 1920s studio executives who feared that sound would destroy the art of making film. The Jazz Singer, the first “talkie” contained only about two minutes of talking; most dialogue was conveyed using caption cards, standard technique of the day. Even so, it didn’t qualify for consideration in that year’s Academy Awards, just as this year, VR films couldn’t qualify for Cannes.

6. Acknowledge the viewer, or not? In some experiences, the viewer’s presence is never acknowledged. In a story that took me to Gaza, My Mother’s Wing, the mother at a stove didn’t appear to see me as she turned and reached past me to throw something away. This momentarily artificialized the experience for me because in the real world, my proximity would have been acknowledged. And yet—my presence wasn’t acknowledged in Allumette. Perhaps because that production wasn’t filmed, it was animated, and the size of its characters so diminutive that I was comfortable walking through scenes unacknowledged, feeling like an omniscient, invisible presence.

7. Make present limits of VR technology work in your favor. In today’s technology timeline, a VR camera rig must stay static. This limitation is exploited to great benefit by VrseWrks in Catatonic where the viewer proceeds through film scenes, remaining (virtually) motionless presumably because the viewer is a patient in a wheelchair being steered through the terrors of a horror-house asylum.

8. Make something that can’t exist elsewhere. The most compelling VR content tells stories that can’t be told in what some VR enthusiasts are now calling a “flatty.” One thing that VR can pull off better than any other visual medium is the conceit of first person. Audiences can “be” the protagonist they merely watch in a film. In EWA the viewer “becomes” a teenage girl raised by a wealthy, beautiful mother. As the film unfolds, we learn that the mother is also psychotic. Audience point of view, critical to this story, could be achieved only in 360-degrees. Similarly deft use of the medium is employed by Hard World for Small Things which tells a story from multiple points of view—literally putting you into the shoes of each of the characters.

In 1895, the first film screening was one-minute long: L’Arrivée d’un Train en Gare de La Ciotat wasn’t a story, it was 60 seconds of terror for a Parisian audience who panicked, thinking a train was coming at them. Without storytellers, the medium of film might have fizzled as its naysayers predicted, a wild ride that audiences would take only once. If VR is to move beyond shrieks and sweaty palms, storytellers must play an integral role in its development.