Audio Engine Programming

1. Audio Engine Programming Lecture 4 Introduction to FMOD programmers API

2. 3D Sound • Sounds have position and velocity • There is a listener component • Relationship between the two • Attenuation (with distance) • Occlusion (low-pass filter) • Doppler (relative velocities) • Lots of “psycho-acoustic” options

3. Fmod audio development • There are three primary components: • The Sound Designer • The Sandbox • The API • We’ll use all three • Start with Sound Designer

4. Data Driven vs. Low Level API • Low-Level API • Everything is done using programming • The programmer has complete control • The benefits are lower memory overhead

5. Continued… • Data-Driven API • Takes assets from FMOD Designer • Sound Designer has control over the audio • Integrates them into the code using the EventSystem object

6. Key Features • The data-driven approach of the EventSystem: • Support’s event logic • Allows the Sound Designer to fine-tune or ‘tweak’ the final mix without the need for Programmer intervention • Creates a logical separate of concerns for the Programmer and Sound Designer

7. Overview of event model FEV event data FSB audio data FMOD EX FMOD EventSystem FMOD Low Level

8. Assets from the sound designer • The build process creates the following assets: • .FEV file - contains the event meta data • .FSB file(s) - contains the audio data • The ‘Programmer report ‘- a human readable list of event names, sound definitions [Optional] • A ‘header’ file - A source code header file [Optional] • Cache files - Files used to speed up subsequent builds (These files are not required by the Programmer)

9. Linking the library • Include the EventSystem header file • #include "fmod_event.hpp“ • Declare a pointer to an EventSystem • FMOD::EventSystem *eventsystem; • Create the object (allocate memory) • FMOD::EventSystem_Create(&eventsystem); • Initialize the object • eventsystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL);

10. Loading files • Set the media path (location of files) • eventsystem->setMediaPath((char *)MEDIA_PATH); • Load the .FEV file • eventsystem->load("examples.fev", 0, 0); • Load an event group • eventsystem->getGroup("examples/examples", FMOD_EVENT_DEFAULT, &eventgroup); • Load an individual event • eventgroup->getEvent(“car", FMOD_EVENT_DEFAULT, &car);

11. Loading Audio • The required .FSB file(s) are listed in .FEV file • Both the .FEV and .FSB files should be located in the directory specified with setMediaPath() • The location of audio within the .FSB files is stored in the .FEV • FMOD will load the audio data from the .FSB files as required for an event, automatically

12. Loading Options • FMOD_EVENT_DEFAULT • Specifies default loading behaviour • Event data for the whole group is NOT cached • Loading will block any other task until complete. • FMOD_EVENT_NONBLOCKING • FMOD will use a thread to load the data • Use Event::getState to find out when loading is complete.

13. Continued... • FMOD_EVENT_INFOONLY • Does not allocate instances or load data • Creates a ‘handle’ that allows the programmer to get information from the event

14. EVENT Playback • Each events have a number of methods and attributes • To start playback of an event: • example->start(); • To stop an event: • Example->stop();

15. Updating • Updating the event system should be called once per 'game' tick (or once per frame) • eventsystem->update(); • This allows the FMOD Engine to refresh data such as: • the position of 3D sounds • changes to volume, pitch, effects • as well as internal mechanisms

16. Demonstration • Simple Event • Demonstrates the necessary code to start playback of an event

17. PROVIDING Parameter DATA • Get the parameter called ‘test’ • example->getParameter(“test", &test); • test will be a EventParameter object • To get the range of the parameter • test->getRange(&rangemin, &rangemax); • To set the value of the parameter • test->setValue(value);

18. Demonstration • Parameter • Demonstrates the necessary code to provide parameter data to an event

19. PROVIDING a KEY-OFF • If an event parameter is currently sustaining on a ‘sustain point’,  triggering a keyoff will release it and allow the parameter to continue • test->keyoff()

20. Demonstration • Keyoff • Demonstrating the necessary code to leave a sustain point

21. Selecting sounds at run-time • The sound to be played can be selected at runtime • This is required for situations such as ‘colour commentary’ in sport games or other situations that require dynamic dialogue • The are two methods: • Programmer Sounds • Programmer Selected

22. Demonstration • Programmer Sound • When a sound definition containing a ‘ProgrammerSound’ is to be played, a callback is made • The code must then provide a sound object at runtime just before it's played • Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSound’ method

23. Demonstration • Programmer Selected • When a sound definition with the playmode ‘ProgrammerSelected’ is to be played, a callback is made • The code must then provide the index number at runtime just before it's played • The index number identifies which file within the sound definition is to be played • Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSelected’ method

24. Closing the Event System • Unload event data • eventsystem->unload(); • Release audio data in memory • fsb->release(); • Release memory and close eventsystem memory • eventsystem->release();

25. Setting up a Project • Take the advice of the video tutorials • Have a separate folder • Copy sounds into a “sounds” directory • Keeps the safe • Can have sub-directories • Create an “output” directory

26. Designer Interface Can delete this if you want, but need to have at least one group

27. Events • Used to define sound properties for an event • Can be comprised of • One sound • Several sounds randomly chosen • Monophonic vs Polyphonic • Can randomize pitch and attenuation • Events can be • Simple • Multi-track

28. A Simple Event

29. Granular Sounds These sounds aren’t looping Allows for a sounds to occur between time ranges Allows for polyphony

30. Event Options Can vary attenuation (dropoff with distance) Can vary pitch Creates a sound considerably smaller than a “soundtrack” Plays forever and is always random!

31. Sound Definitions Required for multi-track events

32. Multi-track Events Comprised of sound defs in layers

33. Multi-track Events • Careful when adding sounds • Choose sound def • Ambient noise is looping • Other sounds are granular, so choose “Oneshot” • Have parameters • Not based on time!

34. Multi-track Events Can cross-fade and set fade out time

35. Effects

36. Parameter Properties Can define range and velocity (to simulate time)

37. Engine Designer • fmod can specifically work with engine sounds • Need for Speed 2 • Based on the “load” of the engine • Right-click on a sound instance->properties • Auto-pitch • Window->fmod Engine Designer

38. The Build Process • Know which platform you’re targeting • Changes are applied only to that platform • Project->Clean, Project->Build

39. Interactive Music • Comprised of several short segments of music • Intro music • Darkening or discovery • Fighting/intense fighting • Release • Before you begin, you must know • Tempo of music (beats per minute – bpm) • Beats per measure (time signature)

40. Cues

41. Themes and Auditioning

42. Transitioning via Parameters

43. Other Things • Supports deployment of multiple languages • Can deploy different builds based on • Platform • Language

44. A Look at the API • Basic API • Designer API • Built on basic API • Can read .fev files • Written for C/C++ • #include <fmod.h> // C • #include <fmod.hpp> // C++ • Read the documentation fmodex.chm

45. API Parts • A System is the fmod engine • A Sound is the raw resource • 2D uses FMOD_2D • 3D uses FMOD_3D result = system->createSound(“music.wav", FMOD_2D, 0, &sound);

46. API Parts • A Channel is an instance of a Sound • Each time you play a sound, you get a new channel • Channels can start out paused • You can set the • Volume (0.0f – 1.0f) • Pan (-1.0f – 1.0f) • Frequency (in Hz as a float) • Always use FMOD_CHANNEL_FREE to pick for you

47. #include<iostream> #include<fmod.hpp> #include<fmod_errors.h> usingnamespacestd; usingnamespaceFMOD; voidmain(){ FMOD_RESULTresult; System*system; // Create the fmod system. We only need one of these result=System_Create(&system); // Specify a max number of simultaneous channels result=system->init(100,FMOD_INIT_NORMAL,0); Sound*sound; // Decompress the entire mp3 into 16-bit PCM in memory result=system->createSound("winning.mp3",FMOD_DEFAULT,0,&sound); if(result!=FMOD_OK){ cout<<"Couldn't open it! "<<FMOD_ErrorString(result)<<endl; } Channel*channel; // Used for setting volume, pan, pausing... // You have the option of passing that channel as the last parameter result=system->playSound(FMOD_CHANNEL_FREE,sound,false,&channel); channel->setFrequency (44100.0f); if(result!=FMOD_OK){ cout<<"Couldn't play it! "<<FMOD_ErrorString(result)<<endl; } while(true){ cout<<"1"; system->update(); // Now required in fmod } }

48. Virtual Voices • Supports “virtual voices” when hardware is limited • Set from System::init() • System::getCPUUsage() • Try to keep < 1000 • May need to set the priority of a channel if it’s important • channel->isVirtual()

49. 3D Sound in fmod • Sound attenuation • Logarithmic • Set the mindistance of a sound channel to start attenuation • Bee = 0.1f • Jet = 50.0f • Leave max distance alone (default is 10,000) • Set 3D settings with System::set3DSettings() • Doppler • Distance factor (in cm, m, feet) • Rolloff scale (attenuation models)

50. 3D Sound fmod uses a left-handed coordinate system result=system->init(100, (FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D), 0); With velocity, you must pass it as metres per second velx= (posx-lastposx) * 1000 / timedelta; vel = 0.1 * 1000 / 16.67 = 6 meters per second vel = 0.2 * 1000 / 33.33 = 6 meters per second