230 likes | 321 Views
10: Audio in Interactive Digital Media. What is Sound?. All sounds are produced by the conversion of energy into vibrations. Vibration becomes a wave. When it reaches the ear, it causes the eardrum to vibrate at the same frequency. Audible Spectrum.
E N D
What is Sound? • All sounds are produced by the conversion of energy into vibrations. • Vibration becomes a wave. When it reaches the ear, it causes the eardrum to vibrate at the same frequency
Audible Spectrum • Humans can hear sounds in range of 20 Hz – 20 kHz. • Highest note on piano is approx. 4kHz.
Sound Over Time • Most sounds change frequency over time. • Represented in waveform.
Music vs. Speech Music waveform Speech waveform Waveform conveys gross character and dynamics of sound. Can provide cues in syncing media.
The Nature of Sound • Wave captures three features of sound: • Amplitude • Perceived as volume. • Frequency • Perceived as pitch. • Duration • Length of time sound lasts.
Getting Sound in and Out of the Computer • Analog to Digital Converter captures separate measures of sound amplitude. • Samples are recorded as digital numbers. • Digital values are used to recreate the analog form using a Digital to Analog Converter.
Recording Sound • Best to not do directly into computer as noise from within the computer is picked up. • Sample rate can be reduced when recording speaking.
Recording Sound • When capturing sound, adjust levels. • If amplitude is too low, sound quality is reduced; • If amplitude is too high, clipping occurs and produces distortion
Sampling Sound • Quality of the sampling depends on: • Sample resolution • Sample rate.
Sample Resolution • Number of bits to encode amplitude. • Like images, more bits used to describe information – more accurately it will be represented. • Two common sample resolutions are 8-bit and 16-bit. • 8-bit resolution captures 256 different amplitude levels. • 16-bit sound has 65,000 different levels. • CD quality sound. • Inadequate sample resolution can distort the sound.
Sample Rate • Number of samples taken in a fixed interval of time. • Stated in thousands of Hertz, or kilohertz. • CD-quality sound captures 44.1kHz to record frequencies as high as 22.05kHz. (The highest frequency the human ear can detect is 20kHz.) • Two measurements capture each cycle of the sound wave: • High value or peak • Low value or trough.
Sound File Formats • Common sampled sound file formats: • WAV • AIFF • AU • MP3 is most popular compressed format (lossy) • 10:1 file size ratio – WAV:MP3
Manipulating Sound • Common applications that allow for recording and generate effects. • Apple’s GarageBand • Adobe’s Soundbooth • Sony’s Sound Forge
Common Audio Edits • Hiss Removal • sample the hiss and remove hiss based on what you sampled. • Normalization • brings the average or peak amplitude to a target level • Time stretching • Pitch alteration
Use of Audio in Interactive Media • Ambience • Sound Effects • Auditory Feedback to aid Usability • Music • Speech
Sound on the Web • Not always needed (can be annoying) • Became popular on the Web when it became possible (late 1990s – early 2000s) • Fell out fashion on the Web • Web has become more cinematic
How to Integrate Sound • Authoring applications allow for precise syncing of audio within interactive experience.
How to Integrate Sound • HTML5 • Prior to HTML 5, there was not a standard for playing audio files on a web. • HTML 5 defines a new element which specifies a standard way to embed an audio file on a web page: the <audio> element. • For more information re: format and browser support: http://www.w3schools.com/html/html5_audio.asp
Beyond Sampled Sound • Sampled sound • All audio discussed so far: WAV, MP3, AIFF, AU, etc… • Sound is stored as a description of the sound.
Synthesized Sound • Sound is stored as a series of commands for the computer to reproduce the sounds. • Analogous to vector-based graphics. • MIDI (Musical Instrument Digital Interface). • Codes provided for: • Specific instruments • Notes • Force and duration of note • Routing commands to different instrument channels • Specialized control functions.
MIDI • Simplest system contains: • Digital musical instrument to create messages • Sound synthesizer to interpret the messages • Amplifier/speaker output system.
MIDI Videos • Composing • http://www.youtube.com/watch?v=pjPf24XQo1s • Building Your MIDI Studio • http://www.youtube.com/watch?v=JJLgwaIFePk&feature=fvw