Physics and Sound

1. Physics and Sound Zhimin & Dave

2. Motivation • Physical simulation • Games • Movies • Special effects

3. What is done now? • Foley artist Lucasfilm Foley Artist

4. How sound works • Transverse waves • Doppler effect Vibration Pressure Wave Perception

5. Why sound rendering is hard • Very small time step • Ideally 44.1kHz • Small time step at lest 10-6 • Stability

6. Sound Rendering • Method for synchronized soundtracks and animations • Sound scripts • Sound rendering pipeline • Key framing audio, Approximate refraction, Different models for sound generation, etc.

7. Sound Rendering • Sound rendering definition • The process of creating a composite sound track from component sound objects • Sound scripts • Control sound instantiation and modification • User specified • From system, ex. simulation, periodic, etc.

8. Sound Rendering Pipeline Modeling Vibrational Analysis Instantiate Sound Rendering Propagation Resample

9. Vibrational Analysis • Generate prototype sound, characteristic of each object and interaction with other objects • Physically Based • FEM, Natural vibration modes, Approximation • Friction and Turbulence • Procedural Sounds (what they used)

10. Instantiate Sound • Prototype sound instantiate • Attach to some 3D object • Start energy of sound wave • Use of sound scripts

11. Propagate • Physical Simulation • Beam tracing • Frustum tracing • Refraction • Effects - reverberation, refraction

12. Resampling • Sound prototype to sound track • Sound at time steps (key-frame audio) • Amplitude - linear interpolation • Frequency - high order approximation or very dense sampling • Like texture mapping • Object space algorithm • Image space algorithm • Warning: aliasing problems similar to texture mapping, solved similar methods

13. Sound Rendering Demo

14. Sound Rendering Pipeline Modeling Vibrational Analysis Instantiate Sound Deformable body Simulation Rendering Propagation Resample

15. Deformable objects • Piggy back on top of deformable body simulator • Use audible vibrational component of simulation • Generate sound pressure and model sound propagation

16. Requirements • Temporal resolution higher then 10-5 • Dynamic deformation • FEM • mass spring • Explicit Surface Representation • Physical Realism, if not • sounds will reveal errors in motion • strange sound responses

17. Modeling surface vibration • I will put the details of this here (commming soon)

18. Deformable objects Demo

19. Modeling Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Physical Shapes Rendering Propagation Resample

20. NOTE to Ming • Some of the slides on Sound of Physical Shapes I am going to replace with an example using a Circular membrane, since I have pictures that it may be more clear.

21. The Sound of Physical Shapes Sounds of physical shapes are important for enhance realism Sounds produced by colliding objects depend on object material, shape and the location of contact. Sounds provide listeners with clues about the object.

22. The framework • Precompute characteristic frequencies • Divide object boundary into small regions • Determine the amplitudes of excitation for an impulse applied to a region • Normalize energies energies, so only scaling is needed when rendering

23. Vibrating Shapes from Impact • Calculating the Modes: • Vibration of the object obeys the wave equation: • Initial conditions: • Norm of the eigenfunctions:

24. Vibrating Shapes from Impact • Calculating the Modes: • Solution to the wave equation: • are related to eigenvalues of • That is

25. Vibrating Shapes from Impact • Calculating the Modes: • Calculating the amplitudes of the vibration modes • Time averaged energy of the vibration • (if mass distributed uniformly)

26. Mode Amplitudes from Impact Location • Initial conditions: • ,the k-dimensional Dirac delta function • The amplitudes are: • The energy is:

27. Sound Sources from Vibrating Shapes • Compute sound emitted from the vibration: • Intensity of the sound emitted: • It integrates the intensity of the vibration.

28. Sounds and Material Properties • Each mode decays with decay time: • is the internal friction parameter, • which depends on material only. • Having emulated the external damping of object, • sound-wave:

29. Sound Map • Store the pre-computed frequency spectrum • , and the excitation spectrum for a • number of locations on the surface. • Treat two sounds and as the same, if the • the difference between them , • where is a threshold.

30. Live Demo http://www.cs.ubc.ca/spider/kvdoel/app6JavaSound/demo.html

31. Sound Rendering Pipeline Modeling Vibrational Analysis Instantiate Sound Rigid body Simulation Rendering Propagation Resample

32. Sound from Rigid body simulations • Precompute shape and frequency of deformation modes • Result, quick computation of vibrational response at runtime • Numerically via eigen-decomposition of system of matrices from FEM model • Model sound from geometric description and material paarameters

33. Modal Analysis • Zhimin and I were planning on ommiting the calculation of the eigenvectors from this so she can speak about it in Nikunj’s paper, as it uses the same process.

34. Rigid body simulation • Normal rigid body simulation • One change, export collision forces to sound synthesizing process. • One thing… • Sound synthesis can reveal flaws of bad rigid body simulator

35. Deforming the model • Need to be able to generate K, C and M • Mass spring, FEM etc. • FEM used, because of past success, see [OSG02] for details

36. A Sound Generation Alg • Details of the alg here

37. How to speed up • Eval 18 use previous values e^{\omega(t+s)} = e^{\omega*t} + e^{\omega*s}

38. Other effects • Cost of additional impulses is low (analytical solution) • Convolve with Gaussian kernel • Force for soft collisions • Noise for small scale friction

39. Rigid body Demo