Interactive audio design where the heck do we go from here
This presentation is the property of its rightful owner.
1 / 26

Interactive Audio Design: Where the Heck do we go from Here?? PowerPoint PPT Presentation

Interactive Audio Design: Where the Heck do we go from Here??. Brian Schmidt A software company. Premises. We are at the most exciting time: The creation and evolution of a new media and art form: Interactive Audio Design and Delivery

Download Presentation

Interactive Audio Design: Where the Heck do we go from Here??

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Interactive audio design where the heck do we go from here

Interactive Audio Design:Where the Heck do we go from Here??

Brian Schmidt

A software company



  • We are at the most exciting time:

    • The creation and evolution of a new media and art form: Interactive Audio Design and Delivery

  • The issues are still fuzzy, but getting clearer by the day

  • Plenty of BIG issues still need to be solved

Our holy grail

Our Holy Grail

The creation and delivery audio soundtracks for interactive entertainment that give the illusion that they were created post-production, but are generated or assembled at run-time.

Our holy grail paraphrased

Our Holy Grail (paraphrased)

Make games stop sounding like $%%&$@ games!!!

Our holy grail how we find it where the heck do we go

Our Holy Grail:How we find it(where the heck do we go)

  • Identify challenges

  • Look at people, roles, aesthetic issues

  • Identify tools, technologies, to be addressed

Challenges above and beyond linear entertainment

Challenges Above and Beyond Linear Entertainment

  • Music and sound effects need to FIT.

    • Not “kinda fit,” “Sorta go with,” “almost get it,” “be generally the right mood…”

  • Music and sound need to be as tightly integrated to action as linear media, without having the advantage of precognition

  • Delivery platforms vary

    • Leads to “least common denominator” implementations

Challenges above and beyond linear entertainment1

Challenges Above and Beyond Linear Entertainment

  • Difficult to harness nuance and musicianship of artists and performers and have soundtrack fit with media

    • Pre-scored music captures nuance at expense of flexibility

    • Musicians perform on non-traditional instruments, if at all

    • Too much reliance on keyboard as input medium

Challenges above and beyond linear entertainment2

Challenges Above and Beyond Linear Entertainment

  • All media decisions in an interactive product interact with and have ramifications with the rest of the product

    • Most obvious: Processor/Frame rate concerns

    • Also other resource usage issues (cd, dvd, memory etc.)

    • Audio Architect’s role is to scope the project to maximize audio in context of game

New paradigms mean expanded job descriptions

New paradigms mean expanded Job Descriptions

  • Specialization

    • Composer

    • Interactive arranger

    • Sound Designer

    • Audio Integration Engineer

    • Audio Architect

  • Describes tasks, not necessarily headcount

    • Headcount comes with maturity of industry

Audio job categories composer

Audio Job Categories:Composer

  • Writes da music

  • Probably orchestrates the music

  • Creates musical themes, building blocks

  • Works within constraints provided by Audio Architect

Audio job categories interactive arranger

Audio Job Categories: Interactive Arranger

  • Analogous to arranger/orchestrator for Film music

  • Formats composed pieces for integration into game

  • Works closely with rest of game team

Audio job categories sound effects designer

Audio Job Categories:Sound Effects Designer

  • Identifies required sound effects for title

    • Leads “spotting” sessions

  • Creates sound elements

  • Works within constraints provided by Audio Architect

  • Provides direction to Audio Implementation Engineer

Audio job categories audio implementation engineer aie

Audio Job Categories:Audio Implementation Engineer (AIE)

  • Responsible for integrating audio elements into game

    • Integrating = “programming”

  • Works with programmer to implement audio, spot queue points, etc.

  • Analogous to “Audio Engineer” in tradition post-production

  • Works closely with Interactive Arranger and Sound Designer.

  • Helps arranger/sound designer utilize technology to the fullest

Audio job categories audio architect

Audio Job Categories:Audio Architect

  • Scopes and defines suitable audio implementation strategies with “the big picture” in mind

  • Works closely with rest of game and audio team

    • Identify resource issues

    • Tracks and estimates performance implications

    • On the project from Day Zero

  • Provides blueprint for audio implementation that works within the entire game

  • Maximizes audio without negatively impacting game as a whole

Aesthetic issues

Aesthetic issues

  • As long as sounds are just “wave files”; as long as music is “fire and forget,” games will continue to sound like “games.”

Aesthetic issues1

Aesthetic Issues

  • Compositional Techniques

    • Some borrowed from film?

      • Foreshadowing

      • Musical “fake out”

    • Some New

      • How interactive does it need to be

      • Composing non-linear music that still has form

  • Use of performers for real-time rendered scores

Aesthetic issues2

Aesthetic Issues

  • Designing sound effects designed to be played/listened to multiple times

    • Avoiding predictability and monotony

  • “Sounds for every situation”

  • The above weighed against the need to provide information

    • Far more important than for linear media

Technology major areas for advancement what we should be working on

Technology: Major Areas for Advancement(What we should be working on)

  • Production Tools and Technology

  • Delivery Technology

Production tools and technologies

Production Tools and Technologies

  • Non-linear sequencers

    • Allow composers to write music designed to be played back interactively

  • Deep musical representations

    • Provides more “knobs” on musical control

      • Mood, key, intensity, etc.

  • Tools to author to specific run-time rendered environments

    • Musically relevant sound generation tools

    • Audio designer-friendly access to rendering and playback parameters

Inter production tools and technologies 2

Inter-Production Tools and Technologies (2)

  • Audio integration environment

    • Perhaps the #1 most important issue keeping audio quality down

    • Analogous to the post-production studio

  • Audio Integration and contextual auditioning must be independent from rest of game development process

    • Economics and production realities demand it

    • Development Tools that do not require significant interaction with the rest of the development process

Download powerpoint document

Create Media

Give to Programmer


Listen in Context

Download powerpoint document

That process is broken

What s broken

What’s Broken?

  • Too much reliance on programmer

  • Turn around far too long

  • “It’s easiest to implement” is often an implementation criteria

  • Weak link

  • Better question “What’s not broken?”

Download powerpoint document

  • Process must be more similar to linear media “post production”

  • Audio integration must be able to be performed by audio experts

  • Can not rely on re-compile, etc. for “edit, listen, modify” loop

  • Technological issues must be well understood by those creating the content

Delivery technologies required

Delivery Technologies Required

  • Extraordinarily high quality run-time rendered music and sound effects

    • “Studio in the box”

    • Sophisticated synthesis technologies

  • High level delivery technologies

  • Seamless integration of pre-rendered and run-time rendered audio

  • Multi-faceted audio delivery systems

    • Provides different ways to provide audio for specific situations



  • We are 150 miles into a 1000 mile journey

    • Non-trivial progress but a long way to go

  • Lots of great technologies to invent and deliver

  • Lots of great creative challenges ahead

  • This room has the best collection of people on the planet to tackle these issues.

  • Login