Tag Archive for ‘audio design’ at

Tag Archive for 'audio design'

Game Audio: Immersion & Interaction

Published on Friday, March 13, 2009 in Game Audio, Games, Inspirations, Music, SFX and Theory. 0 Comments

The title of my talk at Nordic Game Jam 2009 was: “Game Audio: Immersion & Interaction”. In this article I will highlight some of the main points of my talk.

As an audio designer my focus is players audio perception in interactive game environments. How a player is immersed into the game play and how sounds encourage the player to interact. I know that the topic of game audio is huge. There is not much research on the area and there are still many questions regarding subjects such as:

Sound engines
Adaptive music
Dialogue
How to avoid repetitions
Player profiles

At Nordic Game Jam – we do not have the time to create a state-of-the-art First-Person Shooter – we have less than 48 hours. So that is the reason why I have chosen to focus on casual and edutainment games with examples from some of the projects that I have worked on.

This article is divided into 4 parts:

Immersion & Interaction
Casual games
Edutainment
Conclusion

My main objective is how you can tell a story with few and meaningful sound effects, jingles and music compositions, when you are working with game design and/or technical limitations. For instance if you are working on audio design for interactive TV applications – your technical limitation is the hardware. On the other hand – if you are working on audio design for a PC game for children – your limitation is the game design and the storage medium.

IMMERSION & INTERACTION
Why is game audio interesting? And how does player actually perceive audio?

Audio as a physical event
When we look at the graphic interface – it is a 2-dimensional representation or simulation of a 3D graphic environment, displayed on a flat TV or PC screen. But audio is not a simulation. Audio is in fact the only true 3-dimensional physical event that happens during game play. Audio has the properties of pressure and frequency, as it propagates through air and occupies an expanding volume. So the sound wave travels through air from the headphones or TV monitors to our ears. This physical event makes us react physically by making haptic spasmodic inputs on the controller.

Acoustic spaces
When we talk about audio as a physical event during game play – we also have to talk about acoustic spaces. There are 2 kinds of acoustic or resonating spaces that we perceive when we play games:

The perception of a real resonating space – which is the physical space that player is placed in – listening to the output from TV or PC monitors.
The perception of a virtual resonating space – which is the graphical space combined with the sound output from game engine.

It is the union of these two resonating spaces that makes player immerse physically into the virtual game. Meaning that the player is the centre of the gaming event as an immobile figure and s/he is in control of the acoustic space represented on screen.

Audio as a story through references
Audio is not only a physical event. It also gives us information based on our references. So it is audio as a physical event in combination with the referential elements of sound that makes us immerse into the game play.

In for instance First-Person shooters audio is very important because you have to use your audio perception skills to navigate to a certain place or to localise an enemy. As a game player we use audio as a cue to interact during the game play.

Some sounds have ‘alarm’ functions and some sounds have the function of putting us in a certain mood. So sound and music design can contain many elements of information that can be interpreted by the game player – either as something that player has to act upon or something that player has to feel or recognise.

There are many terminologies on the meaning of sound effects and music. But I will go through some of the different functions of sound and music supported by the screenshots and audio examples below.

CASUAL GAMES
Working on limited platforms such as the Set-Top-Box – audio is also important, because you have to tell a story with short and very precise sounds. As an audio designer you have to put a lot of information in a sound file that is less than 1 second in duration.

Another restriction is that the STB can only play one sound at a time. I will also in this article explain the implications of the absence of sound and what this means for players audio experience.

The controllers that players are using on this type of games is a TV remote, where they press arrow buttons and a ‘SELECT’ button to play.

Casual Games – Example 1
In the 1st example – the graphical setting gives us associations to the Wild West. The game play is like ‘Bejeweled’.

The player is “The Gambler” and has to play against different opponents. The intention is to make the opponent lose all his money – so when you make a match with gold bars or silver coins – the opponent loses money.

Gamblers Quest – Audio extract from game play

Sound design on this game is realistic recorded sounds used in a different context:

Card flop: Executed when navigating on menu, this sound supports players action on screen. It supports the players control of the game
Bottle toast: Executed when player gets a match
Cash register – Executed when there is a match with gold bars and the opponent loses money

The jingle is composed imitating piano roll music and gives us instantly references to Western movies and/or silent movies. So the intention of the audio design is to give player a perception of being in a ‘saloon’ in a historical setting from the beginning of the 20th century.

This also means that sounds can give us information on where we on:

A historical time line: It can give us a clue on whether we are in the past or present
Location: Based on our references, we know that we are in the USA and not in China

But sound and music design also relies on player’s prior experience – if player has never seen a Western movie before or does not know where USA is, s/he would probably just interpret the sounds as being ‘realistic’ and will maybe not acknowledge the humour in them.

Casual Games – Example 2

Example 2 is from a puzzle game, where the visuals are a futuristic universe. As a player you have to turn some connectors to make the electric spark enter the receptor. When player enters the receptor – the level is complete.

Ben10 – Audio extract from game play

The audio design in this puzzle game is ‘imitated reality’ sounds. We do not know how the future sounds, so we imitate the reality. We make the sounds according to how we think it will sound like in the future.

There is a positive feedback sound, which is executed when the electric spark enters a receptor. The audio has an ascending pitch. The negative feedback sound, which is executed when the electric spark dies, imitates an electric circuit short-cut.

The most important sound is when player is rotating the connectors. The function of this sound is to give player instant feedback on the interaction that is performed on screen. The focus is to tell player that s/he is in control of the game.

This game can give player associations to other science fiction entertainment media, but if the player has seen the original cartoon episode – that this game is based on – then s/he will recognize the style of sound design and appreciate the similarity of the ‘mechanic’ and ‘electric’ sound universe.

Casual Games – Example 3

In this last casual game example the graphical setting is an open sea. The game is a puzzle and as a player you have to navigate to collect corks to get points. When you have collected the message bottle – you have completed the level. The challenge is to avoid dangers such as the pirate ships. This example is from the first level of the game, so I want you to focus on the absence of sound.

Flapjack – Audio extract from game play

All sounds are in cartoon style. The positive feedback sounds are:

Musical bell: Executed when the adventure bar is full
Cork: Executed when player picks up a cork. Sound character can be interpreted as realistic, but because it is used in a different context it goes into the group of ‘cartoon’ sounds

The negative feedback sounds are:

Car horn: Executed when player attempts to go to a place outside the highlighted area. The sound has an alarm function.
Canon shot: Executed by Pirate ship. The sound has an alarm function as it is the enemy that player has to be aware of and avoid

Jingle is composed using a single accordion instrument. It gives player references to the original cartoon episode or players prior experience from similar ‘sailor’ cartoons, movies or music, where the accordion instrument is used.

But the most interesting part in this audio extract is the long pause before the last sound execution. It is the absence of sound. Silence can sometimes be very effect full in games – like for instance in horror, but in this example it feels a bit tedious.

Basically my point is that it is important to think the sound into the game at a very early stage. Ask questions like:

What impact is audio going to have on the game player?
How can we integrate audio in the interaction in the best possible ways?

So it is important to think of audio as a merged part of game design and interaction – and not just an effect.

EDUTAINMENT GAMES
The next 3 examples are from an edutainment game from the Pixeline series developed by Krea Medie. In these examples I would like to explore how you can use music and ambience in 2D graphics and how you can create a spatial depth in the aural perception.

Edutainment – Example 1

The first example is a screenshot with an extract from the background ambience music. The story is about the girl Pixeline who is beamed up to an UFO. From the UFO she will help her friends to explore different parts of the human body – such as the blood veins, the ears, brain, heart and so fort. In this example I would like to focus on how the background music is adding a spatial depth to the 2D graphics.

UFO ambience – Audio extract from background music and sound

The background ambience is a mixture of ‘bleeping’ sounds and music. The intention of the ‘bleeping’ sounds is to create a ‘resonating room’ that supports the many buttons that are present graphically on screen – but also to support the flying mini UFO

My intention with the music was to add a spatial depth to the 2-dimensional graphics. To give listener a perception of an open space – a place with ‘no roof’. So I had to aurally stretch the 2D graphic representation into something that gives player the perception of a deep space. To create that feeling of open space – I used contrasting timbres such as ‘natural’ vs. ‘synthetic’ sounds in the music score.

Bright bell sounds and cymbals with long decay times where also used to create a sense of an ‘open space’. I wrote the composition in low and high frequency notes, and I also used quartal harmony to give the listener a perception of an open tonal direction.

Edutainment – Example 2

In this example Pixeline is going to explore the blood veins in the human body – and the game play is about navigating inside the blood veins and avoid things such as cholesterols.

Music supports the level design with fast and slow tempo:

When blood veins are wide – music is in high tempo and player can move quickly
Blood Veins 1

When blood veins are narrow – music is slow and player has to be careful
Blood Veins 2

The water stream sounds in the music score give player the perception of being ’under water’ and a perception of an ’enclosed’ space. There are also synthetic sounds in music score imitating ‘submarine’ and ‘whale’ sounds. This gives player references to action entertainment movies and cartoons. These musical effects also help player to establish a perception of being on an ‘underwater’ mission.

Edutainment – Example 3

This level is about the ear and how we perceive sounds. The game play takes place in a detective agency with 3 characters: the Brain, the Eye and the Ear. These 3 characters have to co-operate by guessing the sounds in the right order. My focus in this audio example is how musical instruments are used as sound effects in the music score.

Spy Game – Audio extract from background music

Music is composed using a harmonic progression that is very common for agent/action movies – such as James Bond or Bourne Identity. The intention is to give player references to these previous entertainment action and suspense movies with elements of a story line that has to be revealed or a puzzle that has to be solved. There is not much animation on the level – so to create a sense of being in an office – I added some percussion effects in music score. There is for instance the ‘brrrrrrr-ding’ sound that imitates an old type writer.

CONCLUSION
So what is the conlusion? We have established that audio is:

A physical 3-dimensional event
The physical gate through which player immerse into the virtual environment
Telling stories through references

We should really think of audio as action. When we make inputs on the controller – we get an instant audio feedback from the game engine – that we again react and act upon through the controller.

Interactive audio design can add those extra details to the game play that can not always be said in words or in graphics. It helps us immerse to the virtual environment through the resonating acoustic spaces but also through the story telling elements that we interpret based on our references from real life or from other entertainment media.

Audio Design Project: Project Horror

Published on Wednesday, January 21, 2009 in Audio Implementation, Game Audio, Games, Music, SFX and Theory. 0 Comments

Participated as audio designer for a team of postgraduate students at IT University, Copenhagen.

The postgraduate thesis was an examination of game players reaction on ‘horror’ effects used in computer games. First part of the project was to develop a level, where player navigates through empty school buildings and investigates the rooms. The player character is locked inside the school buildings and has to find a key to escape. Second part of the project was to test the level design on a group of experienced and inexperienced game players.

I was assigned to first part of the project being responsible for sound and ambience creation on the game project.

To see and hear the result – please click on the video below:

Game Audio: Audio design in BioShock

Published on Sunday, December 28, 2008 in Game Audio, Games, Inspirations, Music, SFX and Theory. 0 Comments

The First-Person Shooter game BioShock was in 2007 rewarded having the best game audio content. Having Grimshaw’s acoustic ecology theories in mind (please see the article serie Game Audio: The Acoustic Ecology of The First-Person Shooter), I will review the music design, sound design and dialogue in BioShock – the main focus being music design.

MUSIC DESIGN

The in-game music score in BioShock is based on composition techniques inspired from the 20th Century Western classical music. The score is recorded using a live orchestra and according to composer Gary Schyman (from Audio Talk on GDC 2007), the musicians have partly been given the freedom to contribute with their own interpretations during the recording sessions. The music score is written by Gary Schyman, but it has also evolved through experimentation and collaboration with musicians during the recording sessions.

Uploading the game and entering the main menu – I am immediately set in the nostalgic mood of Rapture. The subtle piano sounds when navigating on the main menu create a small melodic piece. The combination of this melody and the mellow ambience sounds in the background enhance game player’s sense of being put in a kind of ‘dream world’.

The underlying music score is used very subtle during action sequences, where player has to battle enemies. Meaning that the action music score is low in volume and varied. The variations work well in the sense that you do not hear the score being repeated continuously when playing the same sequence over and over again.

The background and action music also blends well with the ambience sounds – meaning that there is a clear link between the character of ambience sounds and music score. This helps game player to sense subtle emotional tensions during the whole game play. So when player is confronted with new challenges – the action music is perceived less ‘in your face’.

As I said earlier the music score of Gary Schyman is inspired by composition techniques from 20th Century music composers such as Xenakis, Pierre Boulez, Ligeti and Messiaen – just to mention a few. The intention of these composers was to work with atonality to create new music scales, tonal centres, timbres and spatiality in music perception and interpretation. And the reason why these composition techniques work so well in Schyman’s music score, during the in-game battle sequences, is because they help player to establish a perception of tension and ‘chaos’ during game play.

The combination of popular jazz music from the 20th Century combined with the use of a classical live orchestra establishes another dimension in the acoustic ecology of BioShock. The two distinct music genres create a contrast in game players perception – not only in instrumentation, tonality and medium (pure digital live recording vs an old analogue record player) – but also in the in-game narrative. Assuming that player knows or has heard the popular jazz songs from the 20th Century, then s/he might listen to the lyrics during game play. If we assume that s/he is listening to the lyrical content of the songs – then s/he may also be able to acknowledge the re-contextualisation of the lyrics in the in-game narrative that s/he is in, as an active player character. In other words – the popular jazz songs suddenly has a new meaning.

SOUND DESIGN & DIALOGUE

The sound design in BioShock is based on sounds related to real life, while others are interpretations of the visual reality. One of the SFX that caught my attention as not being either was the SFX for the wooden doors with glass windows that slides up and closes appearing on the level ‘Medical Pavilion’. The audio sounds like a recording of a ‘metal archive drawer’ that slids open and closes. The sound does not correspond to the ‘wooden’ material that the doors seem to be made of – and therefore it caught my attention every time I passed the doors.

The high quality dialogue performance in BioShock has a considerable effect to the overall perception of game play. The use of audio diaries and the dialogues performed by the various Slicer characters make the game more ‘alive’, as they slowly reveal fragments of the story line during game play. This also makes the dialogue serve the purpose of ‘inviting’ the player deeper into the story line by adding a ‘past’ dimension to the narrative – and creating a link between past and present.

After a while – just before halfway through the game – I start noticing the repetitions of the dialogues from the Thuggish, Spider and Leadhead Slicers – and that annoyed me a bit. The first time I noticed the repetitious Slicer dialogues in the game, I thought it had a powerful effect to the narrative that was going to be revealed to me as a player. I thought of the Slicer dialogues as a mean to reveal the fragments of character stories in the game – the characters being ‘mentally disturbed’ and ‘unhappy’ people. But when the repetitions continued without adding more content to the story apart from giving me hint of an alarm cue to prepare myself to kill the Slicer – the repetition then had no further purpose for me. And as a player – repetitions without further purpose – creates a momentum of annoyance.

Another issue that I noticed, was the use of piano-roll jingle at the vending machines. The purpose is to lead players attention to the vending machine, where s/he can buy more bullets or health. It is a ‘happy tune’ and gives player associations to a merry-go-round. For every level the tune is the same and after a while I tried to ignore it. From an audio design perspective – it would have been very obvious to make – for instance – scale variations on this tune. Meaning that for every level player progress to – the tune is slightly varied.

CONCLUSION

The high quality audio design in BioShock is mainly reflected in the music design – the combination of a live orchestra and licensed music – and the way that it is used during game play. The use of audio diaries to reveal the story in fragments serves the purpose of ‘inviting’ the game player to be a part of an emotional story.

It is important to notice that the sound designer Emily Ridgway had an active role in the process of additional story writing. This also means that audio design – on this project – has been a high priority all the way in the development process from pre-production to production.

Game Audio: The Acoustic Ecology Of The First-Person Shooter – Part 3

Published on Thursday, June 5, 2008 in Game Audio, Games, Inspirations, SFX and Theory. 0 Comments

This post is the 3rd and last in the serie on Mark Grimshaws PhD thesis “The Acoustic Ecology Of The First-Person Shooter”. The subjects of this post are on how players experience the immersion into the acoustic ecology of the 3D world through the sonic experience.

Diegesis and Immersion
When designing games – the aim is to enhance the player and immerse her/him into the game play. Looking at the subject from an aural perspective the focus is the player, as s/he is has several functions when forming the acoustic ecology in game. It is the player that:

Immerses in the acoustic ecology
Participates in the acoustic ecology
Performs to create, sustain and transform the acoustic ecology

Previous work on digital game audio has borrowed the terminologies of diegetic and non-diegetic sounds from film theory and tried to put them in context of digital computer games.

Diegetic sounds refer to sounds that are closely connected with the narrative of visuals – like for instance – a man stand in front of a door, raises his right hand and bangs on the door. The viewer then hears the sound of a hand knocking on a thick piece of wood imitating the action taking place in the visuals
Non-diegetic sounds refer to sounds that do not have a direct connection to the narrative in the visuals such as for instance a music score.
Another theorist, Curtis (1992), prefers the terms ‘isomorphic’ and ‘iconic’ referring to the way musical instruments are used in for instance Disney cartoons.

Looking at music as a non-diegetic sound, Grimshaw suggests that music in FPS games can have a considerable effect on actions experienced by player, as FPS game music exists in between diegetic and non-diegetic sounds. An example of this gray area in between diegetic and non-diegetic sounds is for instance a player who runs through a level and shoots the enemies in such a rhythmic manner that her/his gunshot executions are synchronised with the rhythmic patterns in the music score.

Because of the complexity of a digital game play in terms of player’s interaction with the game world and because of the multiple diegetic sound outcomes – Grimshaw suggests that diegetic sound in digital games should

be defined as the sound that emanates from the game play environment, objects and characters and that is defined by that environment, those objects and characters.

By this he means that it is difficult to define the sounds as being strictly diegetic or non-diegetic as they are all at all times subject to change, as player interacts in the acoustic ecology. In contrast to the above, the non-diegetic sounds can be defined by sound events that occur prior to and following game play, like for instance UI sounds, game and level configuration menus and music score, that gives the player specific cues on points gained, or that s/he is in or outside the game, or that s/he has completed a mission.

According to Grimshaw the problem of using film sound terminology in game audio lies in the major differences between the two media. We can roughly say that in a digital game:

Sounds can have an impact on the actions of a player
Sounds are not necessarily heard by all the players at the same time

Grimshaw suggests a solution for creating a distinction between the diegetic sounds heard during game play, by introducing the following categories:

Ideodiegetic sounds – This terminology refers to diegetic sounds that can only be heard by one player, which includes all sounds that derive from within that particular player’s resonating space – sounds that come from character or other sources in the area. Ideodiegetic sounds are furthermore classified into: exodiegetic sounds which are sounds heard by player that are not triggered by player and kinediegetic sounds which are sounds that are triggered and heard by player
Telediegetic sounds – This terminology refers to a sound that is heard by and responded to by one player, where the actions of that response has a consequence for another player later on.

All diegetic sounds in the acoustic ecology have an importance and consequence for player, as player has to interact and react upon events and objects within the game world displayed on screen.

Perceptual Realism & Immersion
The use of authentic audio recordings, of for instance real weapons, when creating the weapon sound effects for a FPS game, is currently the only way for game designers to simulate or emulate a ‘real world realism’ in a 3D graphic game environment.

Sound is an illusory and real-time 3D event that player acts upon during game play. The use of caricature and convention sounds, rather than authentic sounds, in the resonating spaces establishes the paraspaces in which player has to act on and it is the player’s task to make the distinctions between virtual and real sounds.

The aim of a FPS game designer is to make the player believe that s/he is within the game environment. Sound is one of the key factors to immersion in 3D virtual environments and the creation of ‘perceptual realism’. It is the use of sound, with its physical 3-dimensionality, in the digital media that creates the possibility for player to mentally and physically immerse within the acoustic ecology of game.

The Acoustic Ecology of the FPS Game
Players interaction and immersion within the acoustic ecology is based on players prior experience from outside and inside the game.

A player who hears a music piece on a specific level might for instance interact within the acoustic environment differently depending on various facts.

If player is musical: S/he will use different modes of listening depending on how familiar s/he is with that particular piece of music style or genre

If player is new to the game: Player might use the music to navigate and explore the level to find out what kinds of affordances there are in the area to prioritize

If player is experienced: The music is appreciated as a keynote sound – meaning that its main function is to immerse player into the 3D game world, indicating the spaces and areas player is moving within.

The acoustic community is a place where acoustic information is exchanged in between the inhabitants. An example is ,for instance, a player that hears a signal sound of an action that s/he does not see visually, which turns out to have a consequence for her/his further game play.

According to Grimshaw the acoustic ecology is a system and a cycle that is dependent on player’s controller inputs – a system that changes with time as player interacts and gives response to the soundscape.

CONCLUSION

The thorough conceptual framework gives an understanding of sounds, soundscapes, acoustic spaces and their functions used in FPS games and how they effect the player’s gaming experience.

Mark Grimshaw investigates with his thesis how players interact in the acoustic ecology of the gaming environments alone and with other players. He examines player’s audio experience and immersion into the soundscapes of the 3D world as a physical event that is open for interpretation. He also argues why audio is an important part of player’s gaming experience in FPS games.

One could also argue that Mark Grimshaw’s thesis is giving some of the answers to why audio is an important part when improving player’s gaming experience, as he argues that sound is the only true physical dimension of the imitated 3D world displayed on a computer or TV screen.

There are also the implications of how the soundscape is designed to be perceived in contrast to how the soundscape is actually perceived or interpreted by player – forming the acoustic ecology. This is again based on player’s prior gaming, social or cultural experience.

The thesis “The Acoustic Ecology Of The First-Person Shooter” is a thorough analysis of the FPS game genre, which also gives a conceptual framework and tools for analysing digital game audio used in other game genres.

Game Audio: The Acoustic Ecology Of The First-Person Shooter – Part 2

Published on Tuesday, June 3, 2008 in Game Audio, Games, Inspirations, SFX and Theory. 0 Comments

This article is the 2nd part of a serie investigating Mark Grimshaw’s PhD Thesis “The Acoustic Ecology Of The First-Person Shooter”. My previous post discussed some of the main terminologies that Mark Grimshaw explores and this post will continue by dicussing the acoustic spaces that are formed when game player interacts as a part of the acoustic ecology.

ACOUSTIC SPACES

When discussing game audio and acoustic spaces – the dimension of space and time are important factors. Acoustic spaces in acoustic ecologies can be understood in two ways:

A space with physical parameters of sound propagation
A specific location in a game; like a harbour or dungeons

Looking at the spatial dimensions of an acoustic ecology, Grimshaw suggests the terms: resonating spaces and paraspaces.

A sound event in a space can be a perceived as a combination of many functions and meanings, as it depends on player’s interpretation and socio-cultural experience. Grimshaw makes the following distinctions between the perceptions of spaces:

1st perception of space: Sound is perceived as a 3-dimensional event and is therefore the primary medium that gives player a perception of the illusionary space. The illusionary space being represented by a 3D graphic environment and displayed on a 2D TV or computer screen.
2nd perception of space: The 3D sounds are triggered by the game engine and distributed with the images and animations displayed on screen. The perceived sounds that represent a real acoustic resonating space is then combined with the images displayed on screen, creating a second virtual space. This gives the player a perception that the objects on screen exist within the virtual resonating spaces. Meaning that the real-time created acoustic resonating space gives player a perception of an affordance of objects that exist within the 3D game environment.

Looking at the above there are two spaces formed around the player. These spaces change according to player’s actions, continuously creating new affordances to player. This is one of the perceptual keys to player’s immersion with the game world.

Resonating spaces
A resonating space involves the physical properties and functions that are similar to an acoustic space with parameters such as reverberation, localization and sound propagations. By using the term resonating space Grimshaw introduces the concept of acoustics: sound that propagates as mechanical waves travelling from sound source through air. Sound as a physical event that propagates and expands in the volume of the containing space and reflective surfaces.

But the fact is that the physical dimensions of an acoustic wave is an illusion, as the sound source is the output from the TV speakers or headphones and not the actions player performs in the 3D game world. So according to Grimshaw, a resonating space is a physical space dimension that is:

A physical, acoustic representation of the 3D game world
Dynamic and changes over time according to player inputs on controller

Two important parameters that can not be ignored, when discussing resonating spaces regarding spatiality and sound are: volume and time. They are important because of the following:

Volume is a physical parameter. It can be represented by the Cartesian coordinate system and can therefore also be perceived as the physical representation of the Cartesian coordinate system that is coded in the game engine
Time is the temporal dimension of a sound propagating through air as mechanical waves

When looking at the physical dimensions of a game player and the perceived acoustics during game play, we acknowledge that the physical reality is reversed.

In the real world we locate sound sources by physically moving our heads, but in a gaming environment this not possible. The only physical body parts that player moves during game play are the hand movements on the controller – and occasional whole-body reactions to unexpected sounds.

The in-game sound sources change position or volume in relation to player’s position in the 3D game environment – and not the opposite which is the case in the real world, where we physically move our body to locate a sound source.

This tells us that the player is the centre of the gaming event as an immobile figure and s/he is in control of the acoustic space represented on screen as a 3D world.

Some other physical dimensions and parameters that are important to have in mind when considering player’s control of the acoustic space are:

Low frequency sounds – Low frequency sounds are difficult to localize for the human ear. But during the audio production of sound effects, such as guns and explosions, low frequencies below 1500 Hz are usually boosted to make the effect more ‘powerful’. Because of the difficulty in locating the sound source, the player is then involved in the sound itself and not its relation to the 3D game world. This also contributes to the fact that player’s audio experience is a perception of a closed acoustic space.
Sound as a true physical event - Sound events are the only true three-dimensional components of the virtual game world with the physical parameters of time, amplitude, speed and volume. This is therefore also the only physical dimension from which player can immerse into the virtual game world.

Paraspaces
Grimshaw refers to paraspaces as the spatial dimension within the acoustic ecology that provides affordances involving player to experience location, time, cultural or social factors in game.

During a FPS game, a player will interact with the 3D world in many locations such as the docks, a harbour, dungeons, hotel lobby or a marketplace. A space that indicates a distinct location may have different functions and significations in reality. This also means that the perception and understanding of the affordances in a paraspace depends on player’s cultural and social experience.

Grimshaw gives the example of a paraspace scenario including sounds such as: car traffic, Big Ben and people talking English with a Cockney accent. Based on the cultural experience of player, s/he will then understand the location as being London in United Kingdom. If player has never been in United Kingdom, s/he will then maybe understand the paraspace as just being a big city and not being able to point out the exact country.

Ambience and the use of ambient sounds on locations in games may sometimes be musical or they may be a combination of musical content and sounds together. This means that ambient sounds used in paraspaces may be real or they may be imaginative and ‘not real’. The player will then again interpret the perceived location or ‘paraspace’ relying to her/his cultural experience – or gained experience during training in game.

There is also the function of real-time signal processing to consider when discussing paraspaces. When a reverberation effect is added to the sounds it is the intention of audio designer to give player a sense of the physical dimensions of the paraspace – such as a cathedral, cavern or hallway.

Overall a paraspace can be divided into locational and temporal paraspaces. The temporal paraspace is then divided even further by Grimshaw into: temporal period and temporal progression.

To describe the temporal paraspace – Grimshaw introduces four terms describing sounds with spatial functions:

Choraplasts – Audio that contributes to the perception of a resonating space
Topoplasts – Audio that indicates a location and therefore has a paraspatial function
Chronoplasts – Audio giving the perception of time progression
Aionoplasts – Audio that sets the game world in a ‘historical’ frame – a past, present, future or immediate timeframe

At the same time it should be remembered that the above terms are not static, as their functions and meanings can change based on player’s actions or interpretations. As the acoustic space is mediated by player perception and players categorization of sounds within the resonating space, the perception of sounds will be subject to change.

Summary
This post is the 2nd in a serie of articles dealing with Mark Grimshaw’s PhD Thesis “The Acoustic Ecology Of The First-Person Shooter”. An important part of game players immersion in the virtual 3D gaming environment is the physical dimensions of audio. Acoustic spaces can be perceived from the following perspectives:

A resonating space, where: Sound is a true physical event that travels from source through air as mechanical waves and expands in the containing volume of space and reflective surfaces
A paraspace, where: Sound as a location indicates the place a player is navigating within. A paraspace can at the same time be divided into two categories: temporal period and temporal progression. The temporal progression indicating that time is moving forward and the temporal period indicating the history of a location.

My next post will be the 3rd and last article in this serie exploring Grimshaw’s views on game player’s diegesis and immersion into the acoustic ecology of the virtual 3D gaming world.