1 / 25

Audio design for interactive systems

Audio design for interactive systems. Camille Goudeseune Integrated Systems Laboratory, Beckman Institute. Outline. Sound libraries music instruments Digital Audio music theory Roles of sound composition. high-level multiplatform (linux+windows at least) sound libraries. Java Sound API

tashya-jimenez
Download Presentation

Audio design for interactive systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Audio design forinteractive systems Camille Goudeseune Integrated Systems Laboratory, Beckman Institute

  2. Outline • Sound libraries • music instruments • Digital Audio • music theory • Roles of sound • composition

  3. high-level multiplatform (linux+windows at least) sound libraries • Java Sound API • VSS • FMOD • SDL • SEAL • Housemarque

  4. Java Sound API • Painless for Java. Very basic. Avoid if you're ambitious. http://java.sun.com/products/java-media/sound/

  5. VSS Virtual Sound Server • www.isl.uiuc.edu/software/software.html • the 500-pound gorilla if you need stuff like 8-channel output or linux/irix/windows compatibility • steep learning curve for advanced stuff

  6. FMOD • Syzygy (the Cube) uses it for sound. • CPU-miserly • finite but featureful API • www.fmod.org

  7. SDL • large user community • open source. • Good if you want to integrate sound and graphics tightly. • www.libsdl.org

  8. SEAL Synthetic Audio Library • many C compilers supported • Wide support for hardware acceleration • www.sonicspot.com/sealsdk/sealsdk.html

  9. Housemarque • Multichannel with certain PC soundcards • www.s2.org/hmqaudio/

  10. low-level sound libraries: linux • OSS, www.linux.org.uk/OSS/ • Most distros include the free basic version • ~$30 for fancy multichannel soundcard drivers • ALSA, www.alsa-project.org • religiously open-source alternative • 0.5.10 is the stable version • 0.9.0 development version is fine too(part of the upcoming 2.5 linux kernel).

  11. low-level sound libraries: windows • MMIO • simple, universal • waveOutGetDevCaps(), waveOutWrite(), ... • DirectSound • part of DirectX • LPDIRECTSOUNDBUFFER, CreateSoundBuffer(), Lock(), Unlock(), ... • “faster” but more awkward; use a wrapper

  12. Basics of Digital Audio • File formats • 16-bit or 8-bit? • These days, 8-bit is embarrassing.Pro gear uses 32! • Stereo or mono? • Panning mono is faster + simpler than stereo. • 8 kHz? 44 kHz? • .WAV, .AIFF, .AU, .MP3 • converters: “sox” and dozens of others

  13. Basics of Digital Audio • Two tradeoffs

  14. Basics of Digital Audio • Debugging • Gaps in sound  excessive CPU use • stuttering  CPU starvation(CPU fast enough but poorly scheduled) • Different from graphics! The “frames per second” can’t degrade if the CPU is taxed.

  15. Basics of Digital Audio • Debugging • Electric-guitar distortion  clippingToo quiet: hissToo loud: clippingJust right: almost clipping • For every stage in the audio pipeline,both software and hardware.Every place you can set the volume level!

  16. Where to get sounds • Buy: fx + music libraries • Build: record it yourself • Build: synthesize it yourself • adjust an existing synth patch, a little or a lot • Steal: websearch • 8-bit might still suffice while prototyping

  17. What to do with the sounds,once you have them • common roles • alerts; acknowledgements; ambience • sound vs. image • speech vs. non-speech • synchronization with visual events • combining sounds • tips for spatializing

  18. Short and subtle is best! • “Graphical excellence gives to the viewer the greatest number of ideas in the shortest time with the least ink in the smallest space.” (Edward Tufte, The Visual Display of Quantitative Information) • This applies to alerts, acks, and ambience. • Clutter is worse with sound: no earlids!

  19. Use sound if: Simple Short Standalone Temporal Demands immediate action Use visuals if: Complex Long Referred to later Spatial Can deal with it later Sound or image for a message?

  20. Speech vs. non-speech • Main advantage: precise, simple! But... • Carries extra connotations • Tone of voice, imprecise word choice, can confuse or distract • Non-speech doesn’t interfere with conversation

  21. Synchronizing to other things • Often 100 msec is accurate enough,when passively observing a sonic anda visual event • Sync with input action might needto be tighter, 30 or even only 4 msec.

  22. Moving a probe through a dataset • Trigger sounds on button-click, or when crossing thresholds • Or, play a continuous sound. • Vector-valued data is trickier. • Nominal data: one sound per name. • Scope of probe: point, ball, shell. • Auto-size based on probe speed. • Granulation

  23. Navigating a VR world • Dense world • azimuth: click rate varies with turning rate • altitude: high/low beeps (rate varies with climb rate) • speed: vehicle-engine metaphor • Large world: quiet continuous ambiences localized to individual parts of the world. • Play only the nearest two or three.

  24. Combining sounds • It can get only so loud before clipping. • Spatial separation (“panning”). • Temporal separation (one sound at a time). • A party of drunks / a good dinner conversation. • Frequency separation. • Each layer gets its own tempo. • Heavily layered techno or orchestral music.

  25. Spatialized sound • Steady tones are worst.Bird chirps are best (they should know). • wide frequency band, complex attack • Loudspeaker array • Headphones with HRTF • Motion-tracked headphones • In the real world you move your head slightly to tell where a sound comes from.

More Related