Review: Venice VR Expanded 2020
A virtual trip to the Venice Biennale
In 2020, with the world ravaged by a pandemic, in a late summer burned by wildfires, and with a looming US election with global consequences, it can be very tempting to escape reality. Luckily, this is the year that virtual reality seems to have come into its own as a storytelling medium.
I have personally owned an Oculus Quest since shortly after its release, having calculated that the stand-alone wireless device at a relatively-low price point was my personal ticket to virtual reality. I have greatly enjoyed the games on offer, the ability to watch movies on a giant screen, but what intrigues me the most about virtual reality its potential to transport the user to distant locations real or imagined, to relive events of history, embodied as a character in the moment, or to experience new experimental methods of social interaction and computer interfaces. So when I saw the Venice Biennale was hosting their VR exhibit online this year due to the pandemic, I thought: sign me up.
I have attended the Venice Biennale in person on several occasions — it was a highlight of the several years I spent working and traveling in Europe. Visiting Venice during the Biennale seemed like a great way to stay abreast of trends in the contemporary art world, then taking in the city, which hides endless pleasures for the urban explorer. With travel to Europe out of the question, could the Biennale’s “Expanded” VR exhibit bring that same sense of connection and wonder to me, in my Seattle apartment?
The Venice Biennale began exhibiting virtual reality content as part of the show in 2017, coinciding roughly with the release of the current generation of VR hardware. The Vive and Oculus Rift were both fully head-and-hand tracked — i.e. six-degree-of-freedom (6DOF) devices, capable of displaying high-resolution, high-framerate graphics when tethered to high-end gaming PCs. With the Biennale’s imprimatur, surely the curated selections would be of high quality, and worth reviewing for anyone interested in VR narrative content. So I decided to make an effort to watch and/or experience as many of these works as I could, during the September 2nd to 12th exhibit “Venice VR Expanded 2020” aka the official Virtual Reality competition section of the Venice International Film Festival.
What follows is my review of the entire experience — from (1) initial access to (2) hardware and software setup, to (3) reviews of individual works, and last, (4) my conclusions on the state of virtual reality narrative content in 2020. Feel free to skip ahead.
But before all that…
0. What Do We Call It?
Virtual reality content tends to fall on a spectrum between “immersive video” (strictly linear narrative short films, typically 2D, 360-degree video, with only three degrees of freedom — 3DOF) and full-fledged video games (with non-linear narratives, real-time rendered 3D environments, free locomotion, interactivity with in-game objects, and a robust point/reward/achievement system). While I do enjoy both ends of this spectrum, what I’m most interested in seeing from an event like the Biennale is a subset of this range, with at least a minimum of head-and-hand tracking, some degree of interactivity (even if it’s just to move the story along), but which prioritizes the story or the aesthetic experience over gameplay and point totals. Within that narrowed range, I think we’re seeing a new medium emerge, with great storytelling potential. There’s no agreed-upon catch-all term for this type of content, but “VR narrative experience” comes close. We the audience for these “experiences” are viewers, observers, users, players, actors, or all of the above. In the following notes, I’ll use “viewer” or “player” depending on how much interaction is involved in the experience. To simplify things I’ll say “viewed” vs “played,” since most of these are primarily passive experiences with light interactivity. In my conclusion I’ll attempt to put names to some of the techniques used in this new medium.
1. Initial Access
The Biennale website is a mess, but dedicated VR enthusiasts will eventually find the main instructions for access here: https://www.labiennale.org/en/cinema/2020/venice-vr-expanded-web-section
The exhibit was split into free and paid sections, with most live events and performances beyond the paywall. I did not opt for paid accreditation, so my review is limited to the free content.
The free content was divided into three sections — First, there were several social environments accessible in VR Chat, where you could interact with other visitors and view previews of the exhibit content. Second, the HTC Viveport app, which included both 3DOF and 6DOF content (it seems that most of the solo-viewer, standalone content was included here, but perhaps not). Third, the Facebook Oculus TV app featured 3DOF content. I tried the VR Chat environment, but spent most of my time accessing 6DOF content via the HTC Viveport app. Once I realized that Viveport was the easiest — and maybe only — way to get to the content, I quickly abandoned the other options.
The VR exhibit was apparently simultaneously hosted by several art museums and other venues around the world; but I cannot imagine anyone wanting to share a VR headset with strangers during a pandemic, let alone spend hours in a headset in a single visit — there were dozens of “experiences” available, averaging about 15 minutes each — I’ve spent about an hour per night over the last week or so watching these for review, and I still haven’t seen them all. Home viewing is clearly the way to go for a virtual exhibit.
2. Hardware and Software Requirements.
While the 3DOF content could theoretically be viewed on previous-generation VR gear, most of the content exhibited can only be viewed using an HTC Vive or Oculus Rift, with a gaming PC with a high-powered graphics card. I used a workaround for the Oculus Quest: a ShadowPC subscription — which gives you streaming access to a PC with an appropriate graphics card — and Virtual Desktop software for Quest (via Sidequest) which allows you to connect to a PC wirelessly, similar to the way Oculus Link allows Quest users access to Rift content. This is a fairly complicated setup, and not for everyone, but I already had it working, so it was only a matter of downloading and signing up for VR Chat and Viveport, and downloading the individual apps in Viveport. In short, anyone with a Vive or Rift would be able to access this content easily.
3. Reviews of individual works
VR Chat
Initially thinking that VR Chat served as the main gateway to the content, I installed and loaded it, signed as a new user, and skipped the optional avatar customization. The Biennale content was nowhere to be found in the initial VR Chat environment or menu system (no co-branding deal?), but could be found by searching for “Venice VR Expanded,” which returned two VR Chat environments to explore.
The first I tried was a gondola arrival sequence, which took me down several canals, past blocky building volumes with flat-mapped facades. I appreciated the intention. The second environment was not much better. After a loading screen, I stepped off the gondola onto a small platform in front of a low brick wall, expanding to the horizon. Behind me, the infinite sea, slightly rolling with animation. Stepping up a short staircase, a red carpet led to some basic titles and credits for the exhibit on one short wall, perpendicular to a very long hallway, seemingly modeled after the former rope-making halls at the Venice Arsenale, one of the Biennale’s most iconic venues. The hall is punctuated by a series of evenly spaced doors; spaced maybe 10–15 feet apart, each side of the hall, and connected back to a central red carpet. Where a long hall may be ideal for winding rope, and makes for an interesting venue for installation art, this is perhaps the single worst interface I’ve encountered for accessing VR content.
Each door was identical, with text describing the title and authors of the content hidden beyond. This text was sized to be illegible from the central spine. VR Chat’s default locomotion system is the “blink teleportation” type, where you point to the floor where you desire to move, click, and your vision will briefly blank while you are moved into place. In this hallway environment, with its infinite identical doors, it is incredibly difficult to determine where you are and where you are going, I gave up on trying to find individual exhibits through this interface, and contented myself with checking a few doors at random, ones I could get to without blinking down the entire hallway. (Apparently the paid “Garden” area was much more appealing, and I apologize to the designers of this space, who have detailed the challenges involved in its creation in a blog post here: https://skarredghost.com/2020/09/04/venice-vr-expanded-lessons/ )
Beyond each door was one of the curated VR experiences — or so I hoped. Several contained a single environment or handful of character models from the VR experience. At least one had a theater environment playing a 2D trailer on loop. I soon realized that this whole hallway environment was full of previews, and got out, frustrated.
HTC Viveport
This also required setting up a sign-in (and skip the paid “viveport infinity” option), but once installed “Venice VR Expanded” was prominently featured on the home page of the app, a plain windows app store, similar to Steam or the Oculus store. It was clear and intuitive, and easy enough to add individual VR experiences from the Venice collection to my personal library. I set a few to download, filling up my hard drive, and saved more for later. I quickly hit my disk space limits (the largest of the Biennale experiences ran 71 GB!) and had to strategize which ones to download, and which (other) games and apps I could uninstall to make room. Viveport also includes a VR environment similar to Steam Home, I could access my downloaded content through the Viveport VR app, or through Steam Home, but I believe I could only download new content through the Windows app. Anyway, easy enough, and multiple ways to find and access the content. I used Viveport to access all the individual works below.
I’ve listed the following in alphabetical order, and only included those I was able to successfully experience in their entirety, with no major tracking, graphical, or audio problems (For example, I was unable to get “Soundself” audio working, and “Ajax” consistently miscalibrated my head height at floor level, “Blindspot” controls were too choppy to deal with).
I’ve listed titles only; to my knowledge, all of these were released in 2020. Full credits can be found on the exhibition listing page, here: https://www.labiennale.org/en/cinema/2020/venice-vr-expanded
“Agence”
This is mostly a game, where you are an unnamed omnipotent force floating around a tiny planet populated by creatures who do various algorithmically-defined things or change color based on their proximity to flowers that you cause to grow on their tiny planet. It seems like some emergent effects are possible here, and it often feels like an old-school world-building game (like populous, early sim city games). Overall it’s a pretty chill experience, but falls a little outside the type of VR experience I’m trying to define here.
“Beat”
Very cute, but feels like a game studio’s tech demo: a handful of adorable mechanical robots (think Wall-E mixed with a rusty boiler) do a series of tasks, with minor help from the observer, you. This is a diorama type VR experience, and mostly unfolds in front of you as various platforms raise and lower like miniature stage sets. The central mechanic is a floating beating heart which is mapped laggingly to your hand, and which you must put into position every now and then to the keep the story moving along. I’ll admit — when the robots got dull, I’d wave it around.
So, neat, cute, nice visuals but doesn’t really have any narrative depth. Might be a nice VR demo for children or elders.
“The Book of Distance”
A heartfelt and moving tribute to the writer/director’s grandfather, who emigrated from Japan to Canada in the 1930s, then was interned during WW2, featuring archival materials and full-scale reenactments of scenes from his life in appealing flat-shaded art style. Many of the sets feel theatrical, as in built for the stage, and the staging is carefully executed, for example the viewer will hear something and turn to find a new area illuminated. This is a really effective technique, and the story unfolds with great pacing, ramping up or down for emotional impact.
The interactivity is less compelling, another example when the viewer is asked to complete a minor task, like pulling a lever, to advance the story. I think it would work just as well if the only thing you could do with your hands was examine the various photos and documents that the story often places in front of you. This is one of my favorites of the exhibit.
“Il Dubbio, Episode 1”
A linear narrative experience of the “Disneyland ride” variety, with scenes popping to life out of darkness, and very few parts of the scene actually animated. An odd thing to do in VR, and makes it feel unfinished. The story is a loose meditation on art and doubt, told through the figures of Leonardo Da Vinci and a contemporary artist whose name I forget. We never see these figures, but we are shown their work, illuminated by candles or spotlight. This could be a good series with more effort put into animations. The environments art style is appealing, and the scans of the artworks are pretty good. Imagine if this allowed you to explore the artist’s studio, or see and interact with a portfolio of their work! Unfortunately, this is not that.
“Gnomes and Goblins”
An appealing game demo, with some nice environments to explore and cute character design, but no narrative (in this demo, at least) and thus outside the scope of my reviews here.
“Gravity”
Parable about the truths of the universe, featuring two brothers, whose world is a zero-gravity void filled with familiar detritus from ours: chairs, mechanical equipment, a chess board, etc. One brother discovers a mysterious dot at the bottom of their world, and investigates.
The brothers are scaled down, about cat size, giving the viewer a sense of omniscience, though we are subject to the same sensations of floating or falling as the brothers do in their story, giving us a great sense of presence and, eventually, velocity, as the collection of floating objects serve as a field and ground, which is then manipulated around us.
Interactive elements are subtle: point to float in any direction, grab to pick up an object — which are neat and highly textured objects that you can examine while the story plays out.
Despite the immersion of the void-world, I’d call this a diorama style experience, where most of the action plays out on a central stage, with the viewer looking into from without. Another highlight.
“Great Hoax: Moon Landing”
A movie director enlists you to act as an astronaut and help fake a moon landing for Taiwan. You embody the astronaut, in a fluffy suit that makes the inverse-kinematics animation more convincing, and in a smart move you can almost always see your character’s movements, on a monitor in scene or by looking down at your shadow. Because many of the tasks that advance the story involve striking a pose, and because the IK is convincing, it all works wonderfully and gives a great sense of presence and — spoiler alert — culminates in a joyous dance party! Some light political commentary and a handful of good jokes — visual and in dialog — keep things punchy. Nice!
“The Hangman at Home”
An adaptation of some written work (diary or fiction? I didn’t do my research) about a hangman’s home life, this breaks down into three types of environment, intercut. The first type is all hand-drawn chalk wireframe objects, and it looks great. Your hands are similarly sketched, implying that you are the hangman in this scenario. And you can move and interact with certain objects. This is the menu system, and allows you to select from several objects in the room, in any order you choose.
The scenes that follow are of the second type, which are short silent vignettes, with hand-drawn characters on 2D surfaces staggered and framed like a puppet show. You as viewer can move around, but only to get a better view, and it’s a little disconcerting when the characters take the time to stare directly at you at key moments in the story. Finally, there’s one fully-3D rendered scene where the diorama matches the aesthetic of the chalk menu. All in all I thought this was an interesting story and a great framework for a non-linear narrative, but the story never roped me in.
“Here”
Based on the (awesome) graphic novel where each page is the same view of a single room, at different points in time, sometimes in subframes and juxtaposed to moving effect. This is that, but you are in the room.
Hand-drawn and video-captured textures are perfectly mapped onto the subtly-shaped 3D assets, making the art style really seamless and unitizing of the whole experience. The sub-frames of the comic are expressed as white-outlined 2D picture frames, 3D boxes in space, or sometimes, world-spanning diagonal sweeps — this is a masterclass in VR transitions, and should be studied.
This is one of the best pieces of VR narrative content that I’ve seen, in this exhibit or otherwise. So good I don’t mind the lack of interactivity or the short length. In fact, I’ll be back to see what I missed.
“Hush”
Could be described as “ambient” VR: no interactivity, no story, no characters, just a slowly evolving scene with spatial audio. We find ourselves on a beach, with some goings-on in the distance, then we’re engulfed by waves and are underwater in a kelp forest. And… that’s pretty much it. The best part of this is the spatial audio, which is the clue to turn and see something supposedly interesting happening. I suppose this might be used for relaxation or meditation.
This is a great example of the developers not recognizing the limits of the platform. Most of this plays out at night or in a murky underwater environment, and most VR headsets use screens that cannot represent subtle gradations of color, especially in low brightness. The result here is color banding, which is especially distracting in VR because it is a 2D glitch overlaid on your 3D view, with an effect somewhat like wearing dirty glasses.
”Man Under Bridge”
Interviews with a writer in 1960s Helsinki, illustrated with hand-drawn characters on 2D surfaces, who act out the scenes described, around you. There are three of these short vignettes, one of which is primarily 3D (inside a whiskey bottle!), one purely 2D, 360-degree animation, and a third that’s a mix of the two techniques. To me, it comes across as a test, the designers trying different techniques, but none of them really work for me, mainly because the 3D textures are pretty far removed from the art style of the 2D sketches, and it doesn’t merge well. The interviews aren’t anything particularly interesting, just snapshots or a different world.
“Minimum Mass” (2020)
A diorama-style linear narrative VR experience about a couple’s struggle to conceive, with some light horror elements. The only interactivity is in the viewer’s control of viewing angle: you can grab and rotate the small scenes, which I found somewhat distracting and added nothing to the narrative. The environments are rendered with a slightly gritty realism, highly detailed and nicely done, but the character models tipped into the uncanny valley for me, especially in a scene where you meet them at full scale.
I wouldn’t call this a failure, it’s nicely done, but it demonstrates how challenging it can be to add effective interactivity to a mainly passive experience, and like many VR shorts has a pacing issue: too slow, as if the designers expect viewers to take in every detail in each scene, while tighter pacing could be much more effective, and leave more to the imagination.
“Mirror: The Signal”
A stranded-astronaut story, mostly dialog-free, where the viewer is set centrally in scenes, and turns to follow the protagonist as she explores the detailed environment, and some strange phenomena. This is a true-scale story, the viewer’s head level set slightly higher than the astronaut, which comes into play in the concluding scene, where a mostly passive narrative experience gets a little interactive twist, a pleasant surprise. Ends on a cliffhanger, and I would gladly watch more episodes.
Nifty environment design. Unfortunately, the story is set mostly at night and some detail in the rendering is lost: some areas of the scene are just muddy black, and not apparently intentionally. Like many VR stories, it relies too heavily on “the butterfly trick” where a character actively moves around the viewer to direct their attention to specific things, it always feels like pointing, when the viewer could be encouraged to turn by other means: level of detail, and lighting seem to be the main tools, though spatial audio is also helpful.
“Once Upon A Sea”
A documentary, plus something a little mystical, explores the ecological destruction of the dead sea told through interviews with three people who live, or try to, nearby.
This is excellent: the story unfolds between two types of scenes with excellent pacing and hardware-appropriate production choices throughout. This is one that works within the technological limits. First there are nicely-detailed 3D environmental scenes, either scanned from real locations or very carefully modeled, within which you can navigate around a limited area and examine objects that advance the story. These interactions relate to and lead into 360 videos, either at eye level with the interviewees or drone shots, floating above the landscape. Exploring the landscapes in 3D first, I believe I’m more accepting of the 2D 360 video, that can normally break immersion. Good stuff.
“Queerskins: Ark”
A sweet love story told through motion-captured choreography, rendered in a beautiful particle-effect art style, often against a backdrop of a 3D-modeled beach scene with a 2D 360 video backdrop, a nice technique I’d like to see more of. There is no interaction aside from a unique locomotion that allows you to move through the space by way of moving your arms into different positions, somewhat like the dance going on around you.
Unfortunately, this central story is bookended by literally-uncomfortable 3D, 360 video capture sequences, in 3DOF — i.e. no tracking. If you can’t keep your head completely still — indeed, if you want to turn around and take in the scene — this will inevitably lead to a headache-inducing effect where your head moves but your eyes don’t get the corresponding parallax your brain expects. Until this can be resolved (tracking is a tech limitation, but camera placement also helps), I would recommend VR producers stick to 2D 360 video where they need live recording, and 6DOF fully 3D environments otherwise.
“Vajont”
A narrative VR experience where you embody a woman who tries to convince her husband to move away from their home, down river from a massive hydroelectric dam under construction, based on true events.
The embodied-character presentation is fairly rare in these types of VR experiences, perhaps because it feels so much like an exposition scene in a game. Here, you have hands, arms, body, but the inverse kinematics are poorly done and I found it distracting to have my character’s body flopping around when I moved my hands or teleported across the room. In fact all locomotion and interaction was poorly done, and I had to make multiple attempts to trigger the interactions that advance the story, such as fiddling with a slide projector, turning on a faucet, shutting a window, etc. These flaws pulled me completely out of the narrative and I found myself bored until the ending and credits, when the real story is revealed in floating text, which feels like a missed opportunity when you have the tools of VR at your disposal.
4. Conclusions: VR narrative experiences in 2020
So what is the state of the “VR narrative experience” world in 2020, based on the curated selection at the Venice Biennale? This new medium is rapidly becoming robust enough to tell powerful stories, on the level of short films or video games. The best of these experiences are designed with the limitations of the hardware in mind. The format seems to have settled into a 15–25 minute runtime as the ideal — which seems like a good limit for less experienced VR users, who might get fatigued after a short time. Longer experiences seem more palatable if there’s more to do — i.e. greater interactivity. The best of the bunch tend to either focus on one type of presentation/scene style, and do it very well, or switch between scene styles quickly to keep things fresh and limit the time a viewer has to notice flaws.
The most effective techniques for drawing a viewer’s attention, or transitioning between scenes, seem to draw more from video games and theater traditions, versus cinema. Finding a balance between interactivity and narrative drive is a persistent challenge. Complicated controls can decrease your sense of prescience; grasp/push/pull/gaze/wave interactions are much more intuitive than “push the A button to move forward” (where’s the A button?).
The primary aspect of VR narrative content that makes it a new medium, and not just a mash-up of video games, immersive theater, and cinema, is presence — whether you are embodied in a character, observing the action from afar, or simply floating in an abstract space, this is what VR can do that no other medium can do. Your sense of scale can be manipulated in ways impossible in the real world, and you actions can propel the story through real, physical motion that is impossible in most video games (I see you, Wii Bowling). At it’s best, a VR narrative experience can engender a true sense of empathy for the protagonist of a story, or transmit a sense of loss over ecological devastation, or inspire you to seek out new experiences in the real world, based on your experience in the virtual one. All these things are enhanced by presence and interactivity. VR will not solve the world’s problems, but it is a new medium that provides creators with a powerful new tool to tell their stories.
