1 / 48

Physically Based Sound

Physically Based Sound. COMP259 Nikunj Raghuvanshi. Overview. Background FEM Simulation Modal Synthesis (FoleyAutomatic) Comparison/Conclusions. Motivation. Sounds could in-principle be produced automatically, just like graphics: Sound Rendering

genera
Download Presentation

Physically Based Sound

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Physically Based Sound COMP259 Nikunj Raghuvanshi

  2. Overview • Background • FEM Simulation • Modal Synthesis (FoleyAutomatic) • Comparison/Conclusions

  3. Motivation • Sounds could in-principle be produced automatically, just like graphics: Sound Rendering • Sound Rendering has not received much research effort • Main Goal: Automatic generation of non-music, non-dialogue sound

  4. Sound Production Today • Movies: Foley Artists http://www.marblehead.net/foley/index.html • Games: Anyone noticed the huge sound directory in Unreal Tournament?

  5. PBS: Sound Production in Nature • Collisions/Other interactions lead to surface vibrations • Vibrations create pressure waves in air • Pressure waves sensed by ear Vibration Propagation Perception Surface Vibration Pressure Wave Ear

  6. Main Aims of PBS • Physics simulator gives contact/collision information • Assign material properties for sound, Wood, concrete, metal etc. • Sound simulator generates sound using this data (in real time?)

  7. Challenges • Sound must be produced at a minimum of ~44,000 Hz • Extremely High Temporal Resolution (timesteps in the range of 10-6-10-8 s) • Stiffness of underlying systems (eg. Metallic sounds. K/m~=108) • Stability may require even smaller timesteps

  8. Two Approaches • FEM deformable simulationO'Brien, J. F. et. al., “Synthesizing Sounds from Physically Based Motion.” SIGGRAPH 2001. • FoleyAutomatic (Modal Synthesis)Kees van den Doel et. Al., “FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation.” SIGGRAPH 2001.

  9. Main ideas • Deformable Simulation (arguably) much more “physically based” • Foley Automatic: Additive Synthesis Component Sinusoids Sound Signal

  10. Overview • Background • FEM Simulation • Modal Synthesis (FoleyAutomatic) • Comparison/Conclusions

  11. Simulation Requirements • Temporal Resolution • Simulate Vibration as well as Propagation • Vibration Modeling: Deformable Model for Objects • Propagation Modeling: Explicit Surface Representation • Physical/Perceptual Realism

  12. System Structure

  13. Vibration Modelling • FEM with Tetrahedral Elements • Linear Basis Functions, green’s strain • Explicit Time Integration • Typically #nodes = 500, #elements = 1500, dt = 10-6-10-7 s

  14. Sound Propagation Modelling • Fluid Dynamic FEM simulation of surrounding air? Very expensive. Instead… • Employ Huygen’s Principle: Pressure Wave may be seen as sum of pressure wavelets Receiver Receiver Pressure Wave Pressure “Wavelets”

  15. Surface Vibrations and Sound Pressure contribution of a patch, Unit Normal Velocity Density of Air Sound Propagation Speed in Air Acoustic Impedance of Air

  16. Surface Vibrations and Sound • Approximate differential elements with surface triangles • Apply band pass filters: • Low pass: windowed sinc filter • High pass: DC blocking filter • Result: Pressure known for all surface triangles

  17. Putting it all together Pressure/Signal at Receiver Filtered Average Pressure Area of Triangle Visibility Term Receiver Vibration Approximation of Beam Pattern Distance Falloff

  18. Propagation Delay Accumulation Buffer 1 Receiver Distance from Source d1 t1= d1/c 2 Source d2 t2= d2/c Receiver t=0 Sound Propagation Speed

  19. Results: Capabilities • General models • Generated sounds are accurate • Stereo Sound • Doppler’s Effect

  20. Demo

  21. Results: Accuracy

  22. Results: Speed Scene TimeStep(s) Nodes/Elems Time/Audio Time Bowl 10-6 387/1081 91.3/4.01 mins Clamped Bar 10-7 125/265 240.4/1.26 mins Vibraphone 10-7 539/1484 1309.7/5.31 mins (~1 day) Timings on a 350MHz SGI Origin MIPS R12K processor

  23. Overview • Background • FEM Simulation • Modal Synthesis (FoleyAutomatic) • Comparison/Conclusions

  24. Features • Modal resonance model of solids • Location dependent sounds • Impact, slide, roll excitation models • Real-time, low latency • Easy integration with simulation/animation • Practical • Do not model propagation of sound from source to receiver

  25. Synthesis Method Sound Samples Emission Vibration Force User Propagation Listener Speakers

  26. Vibration Surface u(x,t) of body responds to external contact force F(x,t) u(x,t) F(x,t) Strain Functional Speed of Sound Under suitable boundary conditions, the solution to the PDE is a sum of sinusoids

  27. Emission Sound pressure s(t) linear functional L of surface vibration u(x,t) u(x,t) L s(t) Note that propagation is not modeled in above

  28. The Modal Synthesis Model u(x,t) F(p,t) s(t) L “The response u(x,t) of an arbitrary solid object to an external force can be described as a weighted sum of damped sinusoids” Impulse response/modal model Since L is linear, it implies at s(t) must be a sum of damped sinusoids too

  29. Example: A 1D string a1 ak a0 1st Mode 2nd Mode …Higher modes Frequency = f0 Frequency = f1= 2*f0 Frequency = fk= k*f0 +...+ + Main Idea: Sum contributions of all the modes The point of impact decides the proportions in which the modes are to be mixed: ak. Therefore, ak is a function of p, the point of impact The frequencies and damping parameters are a property of theobject, and independent of how the object is hit

  30. The Modal Synthesis Model u(x,t) F(p,t) s(t) L Impulse response, modal model Kth mode: Gain Factor Point Damping Vibration of impact Term Frequency Parameters measured experimentally

  31. Force Modeling At runtime: Find gain parameters given the location, strength and kind of force. Synthesize sound from previous equation. • Impact • Sliding • Rolling Wavetable Stochastic

  32. Impact Forces • Duration: hardness (T) • Magnitude: energy transfer (w) • Multiple micro-collisions Example:

  33. Sliding/Scraping Micro-collisions lead to noisy audio-force

  34. Sliding/Scraping • Wavetable approach • Store force parameters • Modulate amplitude with energy transfer • Modulate rate with contact speed • Synthesis Approach • Fractal noise represents roughness • Filter through reson filter • Resonance ~ contact speed • Width ~ randomness of surface

  35. Rolling No relative surface motion • Differences with sliding: • Smoother: Use low pass • More damping • Harder to create • Less understood • Essential coupling?

  36. Rolling: Smooth Surfaces • Polyhedral objects do not lead to smooth rolling forces • Instead use smooth surfaces directly

  37. .. . • q q q Rolling: Contact Evolution • Evolve the contact in Reduced coordinates q = (u,v,s,t, ) c(u,v) d(s,t)

  38. Rolling: Contact Evolution • Piecewise parametric surfaces, loop subdivision surfaces • Explicit integration, no stabilization • Multiple contacts and conforming contacts are not handled • Used only when multiple contacts in close spatio-temporal proximity

  39. Demo

  40. Dynamic Forces Pebble-in-Wok Demo Contact force Slipping speed Rolling speed Impulses …and locations

  41. Results • 0.1% CPU time per mode • Graceful degradation of quality • The bell demo is interactive • Uses a PHANToM for interaction • Authors do not report any real timings • State that “sound quality” is perception-based and has no metric as of now

  42. Overview • Background • FEM Simulation • Modal Synthesis (FoleyAutomatic) • Comparison/Conclusions

  43. Discussion • FEM: Physically Rigorous and General • Too slow for interactive applications • Doesn’t scale well • Inappropriate to apply a 30fps technique to 44000fps? • Maybe too general for the problem domain?

  44. Discussion • Modal model exploits the vibrational nature • Higher Efficiency • But, not rigorously physically based • Finding the parameters requires experimentation and “earballing” • No rigorous correlation between physical and perceptual parameters

  45. Discussion • For Realtime: Need for a technique to cover the middle ground • Extracting modal parameters in general requires solving PDEs • Not possible to do in an automated manner • Approximate modal parameters and then use modal synthesis?

  46. Conclusion • PBS involves orders of magnitude smaller temporal and spatial scales • Research is sparse, problems are dense • Main contributions of the two papers besides vibration modeling: • FEM: Efficient modeling of sound propagation • FoleyAutomatic: Efficient, Approximate models to handle surface properties and contact forces

  47. References • O'Brien, J. F., Cook, P. R., Essl G., "Synthesizing Sounds from Physically Based Motion." The proceedings of ACM SIGGRAPH 2001, Los Angeles, California, August 11-17, pp. 529-536. • Kees van den Doel, Paul G. Kry and Dinesh K. Pai, “FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation” Computer Graphics (ACM SIGGRAPH 01 Conference Proceedings), pp. 537-544, 2001.

  48. Acknowledgements Some images were taken from the referred papers and the corresponding SIGGRAPH slides

More Related