Audio Concepts in Plain English: 3D and I3DL2

1. Audio Conceptsin Plain English:3D and I3DL2 Scott Selfon Xbox Advanced Technology Group

2. Overview • What’s so great about “3D”? • Simulating aspects of 3D sound • Audio engine implementations • I3DL2 • Advanced 3D Challenges for Games • Q & A

3. Why bother with “3D” audio? • Screen = 45-90 degree visual ‘window’ • What’s happening in the other 270-315 degrees? • Sound informs player of distances, obstacles, environment • Competing with TV & movie ‘experience’

4. Typical 3D Sound Presentation Priorities • Location • Distance • Speed • Geometry • Sound emitter orientation • World obstacles

5. Locating a Sound in 3D Space • Speaker panning • Cheap, can be effective • Doesn’t cover surround for 2-speaker scenarios • HRTF – Head Related Transfer Function • Simulates 3D spatialization based on shapes of ‘generic’ head and ears • Filtering (shape of ears, sound waves not directly reaching ears) • Delays (sound reaches one ear slightly before the other

6. Distance in the Real World • Sounds attenuate (get quieter) with distance • Approximately half as loud (-6 dB) for every doubling of distance • Sounds “muffled” with distance • Air is a low-pass filter • Sound travels slowly vs. light • ~340 m/sec (vs. 300,000 km/sec for light) • All values vary with humidity, altitude, temperature, etc.

7. Distance Simulation: Volume • Typical distance model (DSound, OpenAL): • Minimum distance (where to start attenuating) • Maximum distance (where to stop attenuating) • Rolloff (how quickly to attenuate with distance) • Others simplify to: • Minimum distance (where to start attenuating) • Distance to silence (where sound is silent; rolloff implied from here – linear or log)

8. Distance Attenuation Example

9. Distance and Rolloff • Rolloff factor – adjust attenuation curve relative to “Real World” • .5: Less rolloff (sound louder at given distance) • 2.0: More rolloff (sound quieter at given distance) • Rolloff also influenced by minimum distance • Remember, half volume at doubled distance

10. Rolloff Factor and Attenuation

11. Minimum Distance and Attenuation

12. Rolloff Confusion • Maximum distance isn’t silence • Sound not further attenuated • Might abruptly stop voice (PC) • Distance factor unrelated to rolloff • Sets units (meters [1.0], feet [0.3048], etc.) • Used for doppler shifting

13. Rolloff Model Challenges • Unintuitive calculations to determine distance to silence • spreadsheet to help • Hard for sound designer to experiment without game-exposed controls • Simplification options • Replace max dist/rolloff with “distance to silence” • Arbitrary rolloff curves

14. Simplifying Rolloff • Distance to silence • Trivial to calculate • Arbitrary user-defined point curves • Custom curvaturecontrol

15. Simulating Other Distance Cues • Filtering with distance • EAX supports • Programmable LPFs in other engines can be used for this • Delay with distance (speed of sound) • Rarely simulated by engines • Can programmatically delay sound triggers • Perceived in games as ‘lag’?

16. Speed Perception in the Real World • Doppler Effect • Sound/listener moving closer pitched up • Sound/listener moving apart pitched down

17. Doppler Effect Simulation • Programmer updates listener/source velocities • DSound: Relative velocity used • OpenAL: Separate velocities used • Distance factor • Units do matter here (m/sec) • Doppler factor • Exaggerate (>1.0) or lessen (<1.0) effect

18. Sound Geometryin the Real World • Few sounds are truly omnidirectional • Most volume projected in one direction • Few sounds are truly “point” sources • Sounds interact with the environment • Environment interaction (“reverb”) • Filtered by obstacles

19. Sound Geometry Simulation • Directional sound emitters • Inside/outside sound cone angles • Outer volume attenuation • 3D spatial orientation for emitter • Non-point emitter solutions • Simulate with multiple points • Simulate with relative positioning • Pre-rendered multichannel • More sophisticated engines

20. Environmental Simulation • Simulation of sound propagation paths • Direct-path = signal reaches listener directly from emitter • Reflected paths = ‘echoes’ off walls, obstacles, etc. • Some delay versus direct-path

21. Basic Reverb Simulation • Direct-path: sound presented “intact” • Distance attenuation • Low pass filtering based on obstacles/distance • Reflected path: delays and filtering • Often less noticeable distance attenuation • Early reflections = discrete echoes • Late reflections = ‘reverberation’ (more dense, complex, decaying echoes)

22. Obstacle Simulation: Obstruction • Objects may ‘block’ the sound’s path to the listener • Direct-path diffracted by or transmitted through obstacle • Reflected pathgenerally notaffected

23. Obstacle Simulation: Occlusion • Sound is being played in another environment • Direct, reflectedpaths bothfiltered

24. An Introduction to I3DL2 • Designed by Interactive Audio Special Interest Group (IASIG – www.iasig.org) • “Interactive 3D Audio Rendering Guidelines, Level 2” • Provides standard for simulating: • 3D positioning (attenuation, doppler) • Environmental reverberation • Occlusion/obstruction materials

25. I3DL2 Concepts • Listener = describes environment • Timings for early/late reflections (“Room”) • Differences between high and low frequency behavior • Source = per-emitter settings • Occlusion/obstruction • Exaggerate/reduce environment’s effect on this emitter

26. I3DL2 Environmental Reverb(“Listener”) • Decay time = time for reverb to fadeto -60 dB

27. I3DL2 Reverb Settings • 30 pre-defined high level presets… • Generic environments (cave, bathroom, etc.) • ‘music’ reverbs • …or 12 low-level properties • Adjust timings: Reflections_delay, Reverb_delay, Decay_time • Set volumes: Room, Reflections, Reverb, Room_rolloff_factor • Adjust behavior of high vs. low frequencies: HF_reference, Room_HF, Decay_HF_ratio • Reverb timbre: Diffusion (“graininess”), Density (“hollowness”)

28. I3DL2 Sound Emitters(“Source”) • Volume adjustment • Direct-path level: Direct, Direct_HF • Room (reverb) rolloff factor and level: Room_rolloff_factor, Room, Room_HF • Occlusion/obstruction control • 8 presets supplied (doors, walls, etc.) • Volume attenuation: Obstruction, Obstruction_LF_ratio, Occlusion, Occlusion_LF_ratio

29. I3DL2 Reverb and Occlusion/Obstruction Demos

30. I3DL2 Limitations • Reverberation flexibility restrictions • I3DL2 doesn’t handle multiple simultaneous environments (listener only) • Can be expensive (CPU, memory) • “Morphable” reverb settings are hard • No sense of game geometry • Game must test for and set occlusion/obstruction • Early reverb reflections are static, non-directional • Reverb transitions

31. Advanced 3D Challenges • Multiple players, one set of speakers • Present sound relative to closest listener • Reproduce sound for all listeners • Doppler issues • Overlayed soundscapes • Speaker biasing? (P1=left, P2=right) • Audio for complex geometry • Rivers • Non-cubical environments

32. Advanced 3D Challenges • 3D integration with multichannel audio • 4- vs. 5- channel 3D? • Morphing 4.0/5.1 sounds to point sources? • Multiple 3D sources presenting same audio (PA system?) • Sync and Doppler challenges • More sophisticated sound propagation simulation • Complex, distant environments (pathfinding?) • AI response to sound propogation (enemies can hear you)

33. Q & A • Resources • OpenAL: http://www.openal.org/ • DirectSound: http://msdn.microsoft.com/ • I3DL2, IA-SIG: http://www.iasig.org/ • http://www.midi.org/ • Audio physics resources • http://hyperphysics.phy-astr.gsu.edu/hbase/sound/soucon.html • http://www.gmi.edu/~drussell/Demos/doppler/doppler.html • Questions, comments?