Auralization

1. Auralization Lauri Savioja (Tapio Lokki) Helsinki University of Technology, TKK

2. AGENDA, 8:45 – 9:20 • Auralization, i.e., sound rendering • Impulse response • Basic principle + Marienkirche demo • Source signals and modeling of directivity of sources • Modeling from perceptual point of view • Dynamic auralization • Evaluation of auralization quality • Spatial sound reproduction • Headphones • Loudspeakers

3. 7 meters 10 meters Impulse response of a room

4. Impulse response of a room

5. Impulse response • A linear time-invariant system (LTI) can be modeled with an impulse response • The output y(t) is the convolution of the input x(t) and the impulse response h(t) • Discrete form (convolution is sum)

6. Measured (binaural) impulse response of Tapiola concert hall

7. Two goals of room acoustics modeling • Goal 1: room acoustics prediction • Static source and receiver positions • No real-time requirement • Goal 2: auralization, sound rendering • Possibly moving source(s) and listener, even geometry • Both off-line and interactive (real-time) applications • Need of anechoic stimulus signals (Binaural rendering, Lokki, 2002)

8. Goal 2: Auralization / sound rendering • “Auralization is the process of rendering audible, by physical or mathematical modeling, the sound field of a source in a space, in such a way as to simulate the binaural listening experience at a given position in the modeled space.” (Kleiner et al. 1993, JAES) • Sound rendering: plausible 3-D sound, e.g., in games • 3-D model  spatial IR* dry signal = auralization

9. Auralization • Goal: Plausible 3-D sound, authentic auralization • The most intuitive way to study room acoustic prediction results • Not only for experts • Anechoic stimulus signal • Reproduction with binaural or multichannel techniques • Impulse response has to contain also spatial information

10. Auralization, input • Input data: • Anechoic stimulus signal(s) ! • Geometry + material data • source(s) and receiver(s) locations and orientations

11. Auralization, modeling • Source(s): omnidirectional, sometimes directional • Medium: • physically-based sound propagation in a room • perceptual models, i.e., artificial reverb • Receiver: spatial sound reproduction (binaural or multichannel)

12. Marienkirche, concert hall in Neubrandenburg (Germany)

13. source – medium – receiver (Savioja et al. 1999, Väänänen 2003)

14. Source Modeling – stimulus signal • Stimulus • Sound signal synthesis • Anechoic recordings

15. Source Modeling - Radiation • Directivity is a measure of the directional characteristic of a sound source. • Point sources • omnidirectional • frequency dependent directivity characteristics • Line and volume sources • Database of loudspeakers http://www.clfgroup.org/

16. Anechoic stimulus signals • In a concert hall typical sound source is an orchestra • Anechoic recordings needed • Directivity of instruments also needed • We have just completed such recordings • Demo • All recordings with 22 microphones • Recordings are publicly available for Academic purposes • Contact: Tapio.Lokki@tkk.fi • http://auralization.tkk.fi

17. Sound field decomposition (Svensson, AES22nd 2002) diffuse reflections handled by surface sources

18. Computation vs. human perception Computation vs. Frequency resolution Computation vs. Time resolution (Svensson & Kristiansen 2002)

19. Two approaches Perceptually-based Physically-based (Väänänen, 2003)

20. Auralization: Two approaches (1) • Perceptually-based modeling • Impulse response is not computed with a geometry • A ”statistical” response is applied • Psychoacoustical (subjective) parameters are applied in tuning the response • e.g. reverberation time, clarity, warmness, spaciousness • Applications: music production, teleconferencing, computer games...

21. Auralization: Two approaches (2) • Physically-based modeling • Sound propagation and reflections of boundaries are modeled based on physics. • Impulse response is predicted based on the geometry and its properties depend on surface materials, directivity and position of sound source(s) as well as position and orientation of the listener • Applications: prediction of acoustics, concert hall design, virtual auditory environments for games and virtual reality applications, education, ...

22. Dynamic auralization (≈sound rendering) • Method 1: A grid of impulse responses is computed and convolution is performed with interpolated responses: • Applied in the CATT software (http://www.catt.se) • Method 2: ”Parametric rendering”

23. Typical Auralization System 1. Scene definition 2. Parametric presentation of sound paths 3. Auralization with parametric DSP structure

24. Auralization parameters • For the direct sound and each image source the following set of auralization parameters is provided: • Distance from the listener • Azimuth and elevation angles with respect to the listener • Source orientation with respect to the listener • Reflection data, e.g. as a set of filter coefficients which describe the material properties in reflections

25. Treatment of one image source – a DSP view • Directivity • Air absorption • Distance attenuation • Reflection filters • Listener modeling • Linear system • Commutation • Cascading (Adapted from Strauss, 1998)

26. Auralization block diagram

27. Treatment of each image source

28. Late reverberation algorithm • A special version of feedback delay network (Väänänen et al. 1997)

29. A Case Study: a Lecture Room

30. Image sources 1st order

31. Image sources up to 2nd order

32. Image sources up to 3rd order

33. Distance attenuation

34. Distance attenuation (zoomed)

35. Gain + air absorption

36. Gain + air and material absorption

37. All monaural filtering

38. All monaural filtering (zoomed)

39. Treatment of each image source

40. Only ITD for pure impulse

41. Only ITD for pure impulse (zoom)

42. ITD + minimum phase HRTF

43. Monaural filterings + ITD

44. Monaural filterings + ITD + HRTF

45. Auralization block diagram

46. Reverb

47. Image sources + reverberation

48. Image sources + reverberation

49. Image sources + reverberation

50. Dynamic Sound Rendering • Dynamic rendering • Properties of image sources are time variant • The coefficients of filters are changing all the time • Every single parameter has to be interpolated • In delay line pick-ups the fractional delay filters have to be used to avoid clicks and artifacts • Late reverberation is static • Update rate  latency