IT 2002 – Graphics & Multimedia

1. www.hndit.com IT 2002 – Graphics & Multimedia Week 7- Audio ATI Kurunegala

2. www.hndit.com What is Audio ? • Audio is a key component of the communication. • Presence of sound greatly enhances the effect of mostly graphics presentation. • Especially in a video or with animation • Sound is the brain’s interpretation of electrical impulses being sent by inner ear through the nervous system • In multimedia projects-you can use sound in different ways.

3. www.hndit.com Audio • Sound is a continuous wave that travels through the air. • The wave is made up of pressure differences.

4. How do we hear sound? www.hndit.com

5. www.hndit.com How does the ear work? As the sound waves enter the ear, the ear canal increases the loudness of those pitches that make it easier to understand speech and protects the eardrum - a flexible, circular membrane which vibrates when touched by sound waves. The sound vibrations continue their journey into the middle ear, which contains three tiny bones called the ossicles, which are also known as the hammer, anvil and stirrup. These bones form the bridge from the eardrum into the inner ear. They increase and amplify the sound vibrations even more, before safely transmitting them on to the inner ear via the oval window. The Inner Ear (cochlea), houses a system of tubes filled with a watery fluid. As the sound waves pass through the oval window the fluid begins to move, setting tiny hair cells in motion. In turn, these hairs transform the vibrations into electrical impulses that travel along the auditory nerve to the brain itself.

6. www.hndit.com Properties of sound wave

7. www.hndit.com Properties of sound • Period- the difference between the formation of two crests(peak) is termed as period • Frequency –the number of peaks that occur in one seconds(1/period) • Human hearing frequency range: 20 Hz - 20 kHz (audio), voice is about 500 Hz to 2 kHz. • Wavelength – is the distance from the midpoint of one crest to the midpoint of the next crest

8. www.hndit.com Properties of sound • Amplitude of a sound is the measure of displacement of the air pressure wave from its mean • Bandwidth –is defined as difference between the highest and lowest frequency contained in a signal

9. www.hndit.com Analog to Digital Recording Chain ADC Microphone converts acoustic to electrical energy. It’s a transducer. Continuously varying electrical energy is an analog of the sound pressure wave. ADC (Analog to Digital Converter) converts analog to digital electrical signal. Digital signal transmits binary numbers. DAC (Digital to Analog Converter) converts digital signal in computer to analog for your headphones.

10. www.hndit.com Computer Representation of Audio • Speech is analog in nature and it is converted to digital form by an analog-to-digital converter (ADC). • A transducer converts pressure to voltage levels. • Convert analog signal into a digital stream by discrete sampling • Discretization both in time and amplitude (quantization)

11. www.hndit.com Audio Encoding Audio Waves Converted to Digital • electrical voltage input • sample voltage levels at intervals to get a vector of values: (0, 0.2, 0.5, 1.1, 1.5, 2.3, 2.5, 3.1, 3.0, 2.4,...) • A computer measures the amplitude of the waveform at regular time intervals to produce a series of numbers (samples). • The ADC process is governed by various factors such as sample rate and quantization: binary number as output

12. www.hndit.com Audio Encoding • Sampling Rate: rate at which a continuous wave is sampled (measured in Hertz) • Examples: CD standard - 44100 Hz, Telephone quality - 8000 Hz • The audio industry uses 5.0125 kHz, 11.025 kHz, 22.05 kHz, and 44.1 kHz as the standard sampling frequencies. These frequencies are supported by most sound cards. • How often do you need to sample a signal to avoid losing information?

13. www.hndit.com Audio Encoding Nyquist Theorem: Sampling rate must be at least twice as high as the highest frequency you want to represent. Capturing just the crest and trough of a sine wave will represent the wave exactly.

14. www.hndit.com Audio Encoding • Nyquist Sampling Theorem: • If a signal f(t) is sampled at regular intervals of time and at a rate higher than twice the highest significant signal frequency, then the samples contain all the information of the original signal. • Example: • CD's actual sampling frequency - 22050 Hz, • Due to Nyquist's Theorem - sampling frequency is 44100Hz.

15. www.hndit.com Nyquist’s sampling Theorem • A signal s(t) can be reconstructed from its samples, provided that it is sampled at least at a rate of twice of its maximum frequency. Maximum frequency of the Analog signal = fm Hz Sampling rate = 2fm samples /second Analog samples are called PAM (Pulse Amplitude Modulated) samples Amplitude Time Sampling time = 1/2fm seconds • Voice band width – 4000 Hz (with a guard band 900 Hz) Sample rate required to reconstruct the signal form its samples is 8000 samples / sec. Color TV - BW = 4.6 MHz

16. A high frequency signal sampled at too low a rate looks like … … a lower frequency signal. www.hndit.com Aliasing What happens if sampling rate not high enough? That’s called aliasing or foldover. An ADC has a low-pass anti-aliasing filter to prevent this. Synthesis software can cause aliasing.

17. www.hndit.com Common Sampling Rates Which rates can represent the range of frequencies audible by (fresh) ears? Most software can handle all these rates.

18. www.hndit.com Digitization Process of converting analog data into digital data is known as digitization. Ea Analog to Digital Conversion Ed Representing with a minimum loss of information Contains an infinite number of amplitudes Sampling - (PAM) Measuring the amplitude of the analog signal at regular intervals

19. www.hndit.com Audio Encoding • The best-known technique for voice digitization is Pulse-Code Modulation (PCM). • Voice 4000 Hz • What is the PCM sampling rate? • PCM provides analog samples which must be converted to digital representation. Each of these analog samples must be assigned a binary code. Each sample is approximated by being quantized as explained next.

20. www.hndit.com Pulse-Code Modulation (PCM).

21. www.hndit.com Three different sampling methods for PCM

22. www.hndit.com Pulse-Code Modulation Encoding Quantizing Done in three steps Sampling (PAM) Quantizing and Linear Encoding Binary Stream

23. www.hndit.com Audio Encoding • Quantization (sample precision):the resolution of a sample value. • Samples are typically stored as raw numbers (linear PCM format) or as logarithms (u-law or A-law) • Quantization depends on the number of bits used measuring the height of the waveform • Example: 16-bit CD quality quantization results in over 65536 values

24. 7 6 5 4 3 2 1 0 www.hndit.com 3-bit Quantization A 3-bit binary (base 2) number has 23 = 8 values. Amplitude Time — measure amp. at each tick of sample clock A rough approximation

25. 14 12 10 8 6 4 2 0 www.hndit.com 4-bit Quantization A 4-bit binary number has 24 = 16 values. Amplitude Time — measure amp. at each tick of sample clock A better approximation

26. www.hndit.com The Digital Audio Stream It’s just a series of sample numbers, to be interpreted as instantaneous amplitudes: one for every tick of the sample clock. Previous example: 11 13 15 13 10 9 6 1 4 9 15 11 13 9 This is what appears in a sound file, along with a header that indicates the sampling rate, bit depth and other things.

27. www.hndit.com Common Sampling Resolutions

28. www.hndit.com 16-bit Sample Word Length A 16-bit integer can represent 216, or 65,536, values (amplitude points). We typically use signed 16-bit integers, and center the 65,536 values around 0. 32,767 0 -32,768

29. www.hndit.com Audio File Size CD characteristics… - Sampling rate: 44,100 samples per second (44.1 kHz) - Sample word length: 16 bits (i.e., 2 bytes) per sample - Number of channels: 2 (stereo) How big is a 5-minute CD-quality sound file?

30. www.hndit.com Audio File Size How big is a 5-minute CD-quality sound file? 44,100 samples * 2 bytes per sample * 2 channels = 176,400 bytes per second 5 minutes * 60 seconds per minute = 300 seconds 300 seconds * 176,400 bytes per second = 52,920,000 bytes = 50.5 megabytes (MB)

31. 7 6 5 4 3 2 1 0 www.hndit.com DAC(Digital to Audio converter)Sample and Hold To reconstruct analog signal, hold each sample value for one clock tick; convert it to steady voltage. Amplitude Time

32. 7 6 5 4 3 2 1 0 www.hndit.com DAC: Smoothing Filter Apply an analog low-pass filter to the output of the sample-and-hold unit: averages “stair steps” into a smooth curve. Amplitude Time

33. www.hndit.com Audio Formats • Audio Formats are described by the sample rate and quantization • Voice quality: 8-bit quantization, 8000 Hz u-law mono (8kBytes/s) • 22 kHz 8-bit linear mono (22 kBytes/second) and stereo (44 kBytes/s) • CD quality 16-bit quantization, 44100 Hz linear stereo (176.4 kBytes/s = 44100 samples x 16 bits/sample x 2 (two channels)/8000)

34. www.hndit.com Audio Formats • Available formats on SUN • au - Sun File Format • wav - Microsoft RIFF/waveform Format • al - Raw A-law Data Format • u - Raw u-law Data Format • snd - NeXT File Format • Available formats on Microsoft-Windows-based systems ( RIFF formats): • Waveform audio file format for digital audio hardware • MIDI file format for standard MIDI files • Audio Video Interleaved (AVI) Indeo file format

35. www.hndit.com Audio Formats • RIFF (Resource Interchange File Format) forms the basis of a number of file formats. RIFF (similarly to TIFF - Tagged Image File Format) is a tagged file format. Tags allow applications capable of reading RIFF files to read RIFF files by another application, hence the word interchange in RIFF. • Other Formats/Players - RealPlayer 7 (Windows NT) with RealAudio, MP3 (MPEG Audio Layer 3) audio, Midi players; MP3 players (MP3.com)

36. www.hndit.com Use of Audio in Multimedia • Can use sound in multimedia projects in two ways • Content Sound provides information to audiences.(dialogs in movies or theater) • Ambient sound consists of array of background & sound effects

37. www.hndit.com Content Sound used in multimedia are : • Narration : provides information about an animation that is playing on the screen • Testimonials : these could be auditory or video sound tracks used in presentations or movies • Voice-overs : these are sound for short instructions ,for example, to navigate the multimedia application • Music –music may be used to communicate(as a song)

38. www.hndit.com Ambient Sound used in multimedia • Message reinforcement : the background sound you hear in real life, such as crowds at a ball game, can be used to reinforce the message that you wish to communicate. • Back ground music : set the mood for audience to receive and process information by starting and ending a presentation with music. • Sound effect : sound effects are used in presentation to liven up the mood and add effects to your presentations, such as sound attached to bulleted lists.

39. www.hndit.com Audio File formats

40. www.hndit.com Audio File Formats

41. www.hndit.com Audio File Formats

42. www.hndit.com Project Find all the sound files on your computer: WAV files (.wav), MIDI (.mid or .rmi), and any others (like .mp3 files) that you are able to download from the Web. Listen to several files of each format carefully, check their properties (right-clicks properties > advanced), and compare their quality with the properties like the size of file, time of the clip, compression used if any, sampling rate, etc. Make a table and record your results in the table. Mention whether these sounds are used (or can be used) as content audio or ambient audio, and explain in one sentence the message they give (if any).