The Assembly: in VR, Hearing is Believing

To date, we’ve taken you on a guided tour behind the scenes of The Assembly by lifting the lid on some of the challenges faced and overcome by our Production, Design, Code and Art teams. We’ve also shown you how we’ve tackled issues around worldbuilding and first-person movement for our upcoming interactive story in VR.

Today, we’re tuning you into the game’s audio aspects and why this element is essential to making a VR game come across as believable.

It’s not just graphics that need extra oomph

One of the biggest changes to sound design we've had to work with in VR is how amplified every detail becomes when you wear a headset.

Over the years, game audio has become more and more detailed. However, for flatscreen games there is a cut-off point where the collision and geometry on an object make it only worth detailing to a certain degree. After all, you can only get so up close to an object in a flatscreen game, you can’t physically lean in to inspect it.

A VR headset, however, does enable the player to push their face right up against an in-game object and allows them to explore environments in a much less restricted way. This gives us opportunities to increase the detail in the audio. Not only does this confirm to expectations of how the real world operates, thus amplifying believability and maintaining immersion, but also it's a reward for the player’s curiosity.

When standing close to an object in a traditional flatscreen game, you would normally find two or three layers of detail, graded differently by distance. For example, a machine would emit loud ambient sounds from afar, then as you approach the sound of individual gears become audible.

In VR, however, we have the scope to make much of this machine’s workings audible, by positioning sounds much more tightly to the object’s geometry. We also play around with the level realism for dramatic effect. Whereas usually a monitor wouldn't have a lot sound emanating from it, adding a slight electrical charge that’s audible if you're in really close draws the player in for longer. Alternatively, having a machine with tiny individual audio details all around it can compel the player to do a full circle, as its sparks their curiosity.

What’s truly exciting is that there's no reason why some of these effects couldn't be used as game mechanics in and of themselves, such as hearing important information behind a wall – lets come back to that in a bit.

Sounds like it’s all in your head

For now, let’s talk about talking to yourself. Or rather, voiceovers – a standard way of pushing the story forwards. In a movie or flatscreen games, you may be engaged in the story and identify with a character, but because the action takes place on a screen, there's always going to be a certain degree of disconnection between what you're seeing/playing and your own perspective on it.

VR, on the other hand, is a much more intimate experience. Thanks to a sense of presence, in VR you really feel as though you’re inhabiting the character that you’re playing as – it’s like you’re in their skin. When you add a voiceover into the mix, it can lead to a strange situation whereby the player feels as though they’re experiencing a protagonist's talking, without actually speaking themselves.

To compound the issue, The Assembly not only has two main protagonists, but also their uses of narration are different. Cal plays the game mostly in isolation, therefore the main bulk of his narration is internal. Madeline, however, always has another character in the scenes to react to, therefore much of her dialogue is said out loud. For both, we want to convince players that the voiceover they’re hearing is their own voice, internal or external, as the character they’re inhabiting.

Consequently, to make character narration feel right for players, we needed to position the audio in 3D virtual space to sound like it's actually coming from inside the player's head, almost as though it's their own train of thoughts that they’re hearing. In this way, we can convey a convincing narrative while maintaining presence. After testing this solution, we found that not only do players get used to having another person’s voice relaying their thoughts, it can in fact compel them to do what their character’s inner-monologue is telling them.

Out of sight but front of mind

That’s not the only way we rely on positional audio in VR. We also use it to expand the game’s narrative in various ways, such as by letting the player hear snatches of conversations or strange experiments through the walls, forming their own mini-stories.

Through 3D sound, we’ve been able to artificially expand the size of our maps. For example, you can hear lift systems moving through the walls, or people and heavy machinery moving around above and around you. This creates the sense of a huge, bustling workplace without needing to create gigantic levels, giving the player a sense that there are even more areas of the Assembly’s bunker than they get to visit over the course of the game. Moreover, creating these kind of assets in audio is less processor and memory intensive than doing the same in visuals, meaning we don’t suffer a performance hit with this technique.

3D audio can also maintain continuity across maps. By having a recognisable sound from loud laboratory on one floor be audible on the floor below, the player can have a sense of how the floors interconnect with each other.

We can also use 3D audio as a mechanic. For example, distant conversations and doors closing out of sight can serve as waypoints, guiding the player towards new discoveries.

Even characters who are 'absent' from the game visually can be fleshed out through in-game audio. By listening to voicemails or using the music systems in their rooms, player can peruse one of these NPCs record collections and get some sense of their character. 

Music to our ears

A musical score is one of the most effective ways to add a sense of drama, set the tone of a particular setting, or give an environment a specific atmosphere. However, given that everything in VR needs to be placed somewhere in 3D space – just like real-life – we weren’t sure to start off with whether or not a traditional backing soundtrack would break the players’ sense of immersion.

Originally, we were only going to have positional music in our environments, played through radios and stereos. However, after further some experimentation, we found that a traditional interactive soundtrack work really, really well in a VR game. Strangely enough, it feels very natural to have a personal soundtrack follow you around and we’ve used it to amplify some of the game’s more emotional moments.

To sound real, audio needs to be in 3D

It should now be pretty clear that in VR, your ears are just as important as your eyes. Being able to hear where something is in a room before you’re able to see it makes any environment that much more convincing.

Think about it - you walk into a room and hear a fan above you to the left. You then look up and see that it is, in fact, above and to the left. In this way, positional audio amplifies a player’s sense of presence and heightens their level of immersion.

Ultimately, 3D sound in VR brings players into the world of the game in a way that conventional 2D stereo sound simply cannot. 

We’re here for you, virtually

If you have any questions about The Assembly, feel free to send us a message via The Assembly's Facebook page or tweet us with the hashtag #TheAssemblyVR.

If you’d like to know more, you can also find nDreams on Facebook, reach us on Reddit, watch our videos on YouTube and check out our board on Pinterest for more screenshots.