Misc topics: Audio and PCG

1. Misc topics: Audio and PCG • Mark Nelson • mjas@itu.dk Fall 2012 www.itu.dk

2. Other parts of game engines Audio Animation Debugging/toolchain Procedural components Etc.

3. Audio Traditionally not a big part of engine design Sound design can impact a game significantly, but was seen as mainly an aesthetic concern Bigger technical concern more recently with 3d positional audio, lots of simultaneous sounds, etc.

4. Types of audio in games Two familiar types: Music Sound effects

5. Music Typically plays in the background Significant length, plus looping Infrequent changes Per-level or per-segment music Can be tied to in-game status

6. Music Main technical concern: Keep it playing with a minimum of hassle No skips, low CPU/memory usage Format depends on hardware: On limited hardware, avoid audio files MIDI driving a wavetable synth Specialized sound hardware, e.g. SNES sound module Otherwise, commonly use Ogg Vorbis

7. Sound effects Happen in response to events Can be near-instantaneous or extended Keycard beep, gunshot, explosion NPC voice clip

8. Sound effects Main technical conerns: Integrate into the engine’s event system Trigger with low latency Simultaneous sounds? Old-school approach has a new sound effect cut off an old one Still common to have a limit, avoid things going haywire from trying to play dozens of sound effects at once

9. Environmental audio Sounds with position in the world Can be sound effects, but often ongoing Waterfalls, machinery, crackling bonfire, television In the background like music, but without the disembodied, comes-from-nowhere aspect

10. Simple environmental audio Give sound sources an (x,y,z) position E.g. In Unity, attach an AudioSource to a GameObject Attenuate volume by distance from player Optimize by having a max audible distance and doing a collision-detection-style search to only play nearby sounds

11. Sound attenuation Sound energy falls off by 1/d2 Or 1/d if measured in decibels (a logarithmic unit) Doubling or halving distance to a source equals about +/-6 dB Can use other attenuation curves Exaggerated positional effects Account for un-modeled features of the environment

12. Sound position Attenuation accounts for distance Do something else for orientation? Sounds coming from the left, in front, behind, etc.

13. Sound position Two-channel (stereo) case is easiest Basic model: Consider headphones as capturing what comes into each of an avatar’s ears Shift volume towards left or right based on angle

14. Sound position Not just level differences, but time differences: Slight delay in sound reaching the far ear On the order of hundreds of microseconds, but perceptible through phase differences Instead of engine-level delay (too precise), can slightly phase-offset the stereo channels

15. Sound position Additional complex stuff going on Sound travels through the head to the far ear: ”head-transfer function” Reverberations: sound echoes, and/or feels like it fills a room rather than point source

16. Sound position Games commonly support more than 2-channel audio now as well Better 3d reproduction, but also more complex Map 3d position to 5.1 speakers 5.1 speakers no longer match 1:1 with ears, so may need to model those effects separately Often this is a whole API/subsystem, possibly outsourced

17. 3d audio middleware OpenAL OpenGL-like interface for 3d audio Open source FMOD Commercial game-engine audio middleware 3d audio API Low-level sound/event management Used in Unity

18. Procedural content generation Algorithmic generation, on- or offline, of what would traditionally be hand-created assets Terrain, levels, vehicles, trees, rocks, …

19. Offline v. online Offline procedural content generation (PCG) External tool generates assets Assets are then used in an engine the normal way Purpose is to reduce costs or allow greater range of content Online PCG Assets or components of them are generated at runtime Closer integration with the engine Various motivations, from compression to allowing infinite worlds

20. Online PCG for compression An early use: Elite (1984) Universe, planet locations, even planet names are algorithmically generated Avoids storing them as data on limited-memory machines Algorithmic PCG, with hand-coded algorithm: Store a smallish seed and a smallish algorithm Pick a seed and expand out to content on the fly

21. Algorithmic PCG Works well if variation either: Has a precise mathematical description (fractals, etc.) Doesn’t have strong constraints, so some weirdness is ok Fractal terrain generation

22. Algorithmic PCG user control Not fully controllable: algorithm does what it does Two common ways of taking user input: Algorithm takes parameters that control its behavior ”Seed” is more than a number; instead is a starting point that’s expanded

23. Algorithmic PCG parameter control

24. Algorithmic PCG as seed-content-expansion SpeedTree: the de-facto standard for videogame trees In this case, also interactive and real-time in a nice editor

25. Infinite worlds Common use of algorithmic, online PCG Rogue dungeon generation Minecraft terrain generation Infinite Mario

26. Infinite worlds: Rogue Rogue dungeon generation: Start with a grid of rooms Mark a random starting room as ’connected’ If current room has unconnected neighbors, connect to one of them randomly, then make that room the current room Otherwise, pick a random unconnected room and repeat Stop when all rooms are connected to the dungeon Entrance in the first room chosen, exit in the last roomto be connected

27. Infinite worlds: Minecraft 3d Perlin noise with some tweaks Generates based on a 2d world grid, filled/unfilled When any player gets near enough to an unfilled part of the map, fill in that grid’s terrain

28. Infinite worlds: Infinite Mario Grammar-based generation Extracted common Mario level elements into a grammar specifying how they combine As player goes right, generates new parts of the level

29. Intelligence in PCG Instead of a hand-coded algorithm, generator can do some kind of intelligent evaluation of the content Ensure constraints like level-playability are respected Optimize a function encapsulating aesthetic judgment

30. Intelligence in PCG: SketchaWorld

31. Intelligence in PCG Other examples: Genetic algorithms optimizing aesthetic or ”fun” criteria (lots of work by Julian Togelius et al) Constraint-solvers finding optimal map according to set of constraints (answer-set-programming-based level generator for Warzone 2100)

32. Simulation-based PCG Simulate a ”real” process to produce content Attempt to recreate how the content would’ve actually been generated in real life (with simplifications) For example: Produce canyons by simulating millions of years of erosion Simulate city-building using agent models

33. Simulation-based PCG: Transport Tycoon Transportation-oriented game, where you build railroads, roads, bus lines, etc. to connect cities Cities grow or shrink according to a model of how the transportation affects their economies In-game PCG as a game mechanic Fuzzy line to ”normal” simulation gaming

34. Simulation-based PCG: Dwarf Fortress World generated by insane levels of simulation Generates terrain, simulations millions of years of sedimentation and erosion Simulates thousands of years of the world’s history, battles, civilizations, etc… Then game starts

35. PCG resources Lots more can be found: PCG wiki: http://pcg.wikidot.com/ ITU PCG course: https://blog.itu.dk/MPGG-E2012/course-plan/ PCG mailing list

36. Project You have about 1 month to do additional investigation into an aspect of game engines Many possibilities!

37. Project ideas Can build something Extend Project 2 to a full-featured renderer, write a raytracer, an AI system Extend an existing engine, e.g. via a Unity plugin Almost anything engine-related Can compare or test something Implement multiple ways of doing something and benchmark Compare implementing the same thing in, e.g., Unity vs. UDK

38. Project parameters Hand-ins are individual, but can collaborate Build different parts of a bigger system One person builds a system and another person tests things Presentation in class December 7 Hand-in December 12