1 / 42

Large-scale Physical Modeling Synthesis

Large-scale Physical Modeling Synthesis. NUI Maynooth, 2008. Stefan Bilbao Acoustics and Fluid Dynamics Group / Music University of Edinburgh. Abstract sound synthesis and physical modeling Physical modeling techniques Direct simulation Implementation Issues.

bendek
Download Presentation

Large-scale Physical Modeling Synthesis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Large-scale Physical Modeling Synthesis NUI Maynooth, 2008 Stefan Bilbao Acoustics and Fluid Dynamics Group / Music University of Edinburgh • Abstract sound synthesis and physical modeling • Physical modeling techniques • Direct simulation • Implementation Issues

  2. Digital Sound Synthesis: Motivations • Various goals: • Achieving complete parity with sound produced by existing instruments… • Creating new sounds and instruments… • The goal determines the particular methodology and choice of technique…and the computational complexity as well as implementation details! • Main groups of techniques: • Sampling synthesis • Abstract techniques • Physical Modeling

  3. Abstract Digital Sound Synthesis Origins…early computer hardware/software design, speech (Bell Labs, Stanford) Basic operations: delay lines, FFTs, low-order filters Some difficulties: Sounds produced are • often difficult to control… • betray their origins, i.e., they sound synthetic. • BUT: may be very efficient! Others: Subtractive Waveshaping Granular… Wavetable Synthesis (1950s) Oscillators/ Additive Synthesis (1960s) FM Synthesis (1970s) Amplitude Frequency Variable rate read Modulator Carrier Table of data FM output Sinusoid

  4. Physical Modeling Physical models: • based of physical descriptions of musical “objects” • can be computationally demanding… • potentially very realistic sound • control parameters: few in number, and perceptually meaningful • digital waveguides, modal synthesis, finite difference methods, etc.

  5. Linearity and Nonlinearity • A nonlinear system is best defined as a system which is not linear (!) • A linear system, crudely speaking: a scaling in amplitude of the excitation results in an identical scaling in amplitude of the response • Many interesting and useful corollaries… • Many physical modeling techniques are based on this simplifying assumption… Sound output (audio rate) Excitation Gestural data (control rate) Resonator Strongly nonlinear Linear (to a first approximation)

  6. Digital Waveguides (J. O. Smith, CCRMA, Stanford, 1980s--present) • A delay-line interpretation of 1D wave motion: • Useful for: strings/acoustic tubes • Waves pass by one another without interaction • Extremely efficient…almost no arithmetic! • See (Smith, 2004) for much more on waveguides… Rightward traveling wave Leftward traveling wave Add waves at listening point for output

  7. Products Using Waveguide Synthesis • Physical modeling synthesizers • Yamaha VL-1 & VL-7, 1994 • Korg Prophecy, 1995 • Sound cards • Creative Sound Blaster AWE64 • Creative Sound Blaster Live • Technology patented by Stanford University and Yamaha Yamaha VL-1 (Sound examples from: http://www-ccrma.stanford.edu/~jos/waveguide/Sound_Examples.html) (This slide courtesy of Vesa Valimaki, Helsinki University of Technology, 2008.)

  8. Waveguide Stringed Instruments (Helsinki University of Technology, Department of Acoustics and Signal Processing) • Sound examples: • Full harpsichord synthesis (Valimaki et al., 2004) • Guitar modeling (Valimaki et al., 1996) • See http://www.acoustics.hut.fi/~vpv/ for many other sound examples/related publications

  9. Modal Synthesis (Adrien et al., IRCAM, 1980s--present) • Vibration is decomposed into contributions from various modes, which oscillate independently, at separate frequencies • Basis for Modalys/MOSAIC synthesis system (IRCAM) Sound output

  10. Limitations • Digital waveguides: work well in 1D, but do not extend well to problems in higher dimensions Cannot handle nonlinearities: Linear String Nonlinear String Cannot extract efficient delay-line structures… • BUT: when waveguides may be employed, they are far more efficient than any other technique!

  11. Limitations • Modal synthesis Not computationally efficient Irregular geometries huge memory costs (storage of modes) Also cannot handle distributed nonlinearities: Nonlinear (von Karman) Plate Linear Plate • These methods are extremely useful, as first approximations…

  12. Observations These methods can be efficient, but: They are really “physical interpretations” of abstract methods: Wavetable synthesis  waveguides Additive synthesis  modal synthesis Can deal with some simple physical models this way, but not many.

  13. Physical Modelling Synthesis: Time-domain Methods • Methods are completely general— no assumptions about behaviour • Vast mainstream literature, 1920s to present. Numerical method (recursion) Output waveform Musical instrument System of equations Finite Element Methods Spectral/PseudospectralMethods Finite Difference Methods

  14. FD schemes as recursions • All time domain methods operate as recursions over values on a grid • Recursion updated at a given sample rate fs • Typical audio sample rates: • 32000 Hz • 44100 Hz • 48000 Hz • 96000 Hz

  15. FD schemes as recursions • Solution evolves over time • Output waveform is read from a point on the grid • Entire state of object is computed at every clock tick

  16. Sound Examples: Nonlinear Plate • Nonlinear plate vibration the basis for many percussion instruments: cymbals, gongs, tamtams • Audible nonlinear phenomena: subharmonic generation, buildup of high-frequency energy, pitch glides • Under struck conditions, a wide variety of possible timbres: • Under driven conditions, very much unlike plate reverberation…

  17. FD Cymbal Modeling Cymbals: an interesting synthesis problem: • Simple PDE description • Regular geometry • Highly nonlinear Time-domain methods are a very good match… A great example of a system which is highly nonlinear…linear models do not do justice to the sound! Linear model Nonlinear model Difference methods really the only viable option here…

  18. FPGA percussion instrument (R. Woods/K. Chuchasz, Sonic Arts Research Centre/ECIT, Queen’s University Belfast)

  19. FD Wind Instruments Wind instrument models: Also very easily approached using FD methods… Clarinet … Saxophone Squeaks! BUT: for simple tube profiles (cylindrical, conical), digital waveguides are far more efficient!

  20. FD Modularized Synthesis: Coupled Strings/Plates/Preparation Elements • A complex nonlinear modular interconnection of plates, strings, and lumped elements… String/soundboard connection Spring networks Prepared plate Bowed plate

  21. FD Plate Reverberation • Physical modeling…but not for synthesis! • Drive a physical model with an input waveform • In the linear case: classic plate reverberation (moving input, pickups)

  22. RenderAIR: FD Room Acoustics Simulation(D. Murphy, S. Shelley, M. Beeson, A. Moore, A. Southern, University of York, UK) • Audio bandwidth 3D models = High Memory/High Computation load. Possible Solutions? • Uses Collada (Google Earth/Sketchup) format geometry files. • “Grows” a mesh to fit the user defined geometry. • Mesh topology/FDTD-Scheme plug-ins for speed of development. • Contact and related publications: Damian Murphy, University of York, UK • dtm3@ohm.york.ac.uk • http://www-users.york.ac.uk/~dtm3/research.html

  23. Musical Example Untitled (2008)---Gordon Delap stereo, realized in Matlab(!)

  24. A general family of systems in musical acoustics • A useful (but oversimplified) model problem: • Parameters: • d: dimension (1,2, or 3) • p: stiffness (1 or 2) • c: ‘speed’ • V : d-dim. ‘volume’ • strings • acoustic tubes • room acoustics • membranes • plates • bars

  25. Computational Cost: A Rule of Thumb • Result: bounds on both memory requirements, and the operation count: # memory locations # arithmetic ops/sec Some points to note here: • As c decreases, or as V becomes larger, the “pitch” decreases and computation increases: low-pitched sounds cost more… • Complexity increases with dimension (strongly!) • Complexity decreases with stiffness(!) The bound on memory is fundamental, regardless of the method employed…

  26. Computational Costs Arithmetic operations/second, at 48 kHz: Great variation in costs… 1015 108 1016 109 106 1014 107 1012 1013 1010 1017 1011 Wind instruments Plate reverberation Small-medium acoustic spaces Large acoustic spaces Single string Bass drum Full piano Approx. limit of present realtime performance on commercially available desktop machines

  27. Difficulties: Numerical Stability • For nonlinear systems, even in isolation, stability is a real problem. • Solution can become unstable very unpredictably… • Problems for composers, and, especially: live performers! • Even trickier in fixed-point arithmetic.

  28. Parallelizability: Modal Synthesis • Each mode evolves independently of the others: • Result: independent computation for each mode (zero connectivity) • Obviously an excellent property for hardware realizations. Each mode behaves as a “two-pole” filter:

  29. Parallelizability: Explicit finite difference methods • A useful type of scheme: explicit • Each unknown value calculated directly from previously computed values at neighboring nodes • “local” connectivity… • Useful for linear problems… Update point Unknown Known

  30. Parallelizability: Implicit finite difference methods • Other schemes are implicit… • Unknowns coupled to one another (locally) • Useful for nonlinear problems… (stability!) Update group of points Unknown Known

  31. Parallelizability: Sparse matrix representations • Can always rewrite explicit updates as (sparse) matrix multiplications: Size N by N, where N is the number of FD grid locations. NNZ entries: O(N) Sparse, often structured (banded, near Toeplitz) = Next state Last state State transition matrix

  32. Parallelizability: Sparse matrix representations • Can sometimes write implicit updates as (sparse) linear system solutions: = Next state Last state • Many fast methods available: • Iterative… • Thomas-type for banded matrices • FFT-based for near-Toeplitz • Different implications regarding parallelizability!

  33. I/O: Modal Methods • Modal representations are non-local: • input/output at a given location requires reading/writing to all modes: Input Location-dependent expansion coefficients Excitation point Readout point Location-dependent expansion coefficients • Expansion coefficients calculated offline! • Must be recalculated for each separate I/O location • Multiple outputs: need structures running in parallel… Output

  34. I/O: Finite Difference Methods • Finite difference schemes are essentially local: • Input/output is very straightforward: insert/read values directly from computed grid… • O(1) ops/time step Connect excitation element/insert sample Read value

  35. I/O: Finite Difference Methods • Multichannel I/O is very simple… • No more costly than single channel! Connect excitation element/insert sample Read values • Moving I/O also rather simple • Interpolation (local) required… Read/write over trajectory

  36. Boundary conditions • Updating over interior is straightforward… • Need spcialized updates at boundary locations… • …as well as at coordinate boundaries

  37. Modularized synthesis • Idea: allow instrument designer (user) to connect together components at will: • Basic object types: • Strings • Bars • Plates • Membranes • Acoustic tubes • Various excitation mechanisms • Need to supply connection details (locations, etc.) Object 2 Object 1 Object 3

  38. Challenges: Modular Stability • Easy enough to design stable simulations for synthesis for isolated objects… Mass/spring system Ideal String • Even for rudimentary systems, problems arise upon interconnection: Stable Connection Unstable Connection • For more complex systems, instability can become very unpredictable…

  39. Energy based Modular Stability • Key property underlying all physical models is energy. • For a system of lossless interconnected objects, each has an associated stored energy H: • Each energy term is non-negative, and a function only of local state variables---can bound solution size: • Numerical methods: assure same property in recursion in discrete time, i.e., H1 H2 H4 H3 Need to ensure positivity in discrete time…

  40. Energy: Coupled Strings/Soundboard/Lumped Elements System • Can develop modular numerical methods which are exactly numerically conservative… • A guarantee of stability… • A useful debugging feature! • Returning to the plate/string/prepared elements system, Soundboard Energy Energy of Prepared Elements Energy of Strings Total Energy time

  41. Concluding remarks • Digital waveguides: • Ideal for 1D linear uniform problems: ideal strings, acoustic tubes • Extreme efficiency advantage… • Modal synthesis: • Apply mainly to linear problems • Zero connectivity • I/O difficulties (non-local excitation/readout) • Possibly heavy precomputation • Good for static (i.e., non-modular) configurations • FD schemes • Apply generally to nonlinear problems • Local connectivity • Stability difficulties • I/O greatly simplified • Minimal precomputation • Flexible modular environments possible

  42. References • General Digital Sound Synthesis: • C. Roads, The Computer Music Tutorial, MIT Press, Cambridge, Massachusetts,1996. • R. Moore, Elements of Computer Music, Prentice Hall, Englewood Cliffs, New Jersey, 1990. • C. Dodge and T. Jerse, Computer Music: Synthesis, Composition and Performance, Schirmer Books, New York, New York, 1985. • Physical Modeling (general) • V. Valimaki and J. Pakarinen and C. Erkut and M. Karjalainen, Discrete time Modeling of Musical Instruments, Reports on Progress in Physics, 69, 1—78, 2005. • Special Issue on Digital Sound Synthesis, IEEE Signal Processing Magazine, 24(2), 2007. • Digital Waveguides • J. O. Smith III, Physical Audio Signal Procesing, draft version, Stanford, CA, 2004. Available online at http://ccrma.stanford.edu/~jos/pasp04/ • V. Välimäki, J. Huopaniemi, M. Karjalainen, and Z. Jánosy, “Physical modeling of plucked string instruments with application to real-time sound synthesis,” J. Audio Eng. Soc., vol. 44, no. 5, pp. 331–353, May 1996. • V. Välimäki, H. Penttinen, J. Knif, M. Laurson, and C. Erkut, “Sound synthesis of the harpsichord using a computationally efficient physical model,” EURASIP Journal on Applied Signal Processing, vol. 2004, no. 7, pp. 934–948, June 2004. • Modal Synthesis • D. Morrison and J.-M. Adrien, MOSAIC: A Framework for Modal Synthesis, Computer Music Journal, 17(1):45—56, 1993. • Finite Difference Methods • S. Bilbao, Numerical Sound Synthesis, John Wiley and Sons, Chichester, UK, 2009 (under contract).

More Related