Digital Audio Compression

1. Digital Audio Compression CIS 465 Spring 2013

2. Speech Compression • Compression of voice data • We have previously mentioned several methods that are used to compress voice data • mu-law and A-law companding • ADPCM and delta modulation • These are examples of methods which work in the time domain (as opposed to the frequency domain) • Often they are not even considered compression methods

3. Speech Compression • Although the previous techniques are generally applied to speech data they are not designed specifically for such data • Vocoders, instead, are • Can’t be used with other analog signals • Model speech so that the salient features can be captured in as few bits as possible • Linear Predictive Coders model the speech waveform in time • Also channel vocoders and formant vocoders • In electronic music, vocoders allow a voice to modulate a musical source (via synthesizer, e.g.)

4. General Audio Compression • If we want to compress general audio (not just speech), different techniques are needed • In particular, music compression is a more general form of audio compression • We make use of psychoacoustical modeling • Enable perceptual encoding based upon an analysis of the ear and brain perceive sound • Perceptual encoding exploits audio elements that the human ear cannot hear well

5. Psychoacoustics • If you have been listening to very loud music, you may have trouble afterwards hearing soft sounds (that normally you could hear) • Temporal masking • A loud sound at one frequency (a lead guitar) may drown out a sound at another frequency (the singer) • Frequency masking

6. Equal-Loudness Relations • If we play two pure tones, sinusoidal sound waves, with the same amplitude but different frequencies • One may sound louder than another • The ear does not hear low or high frequencies as well as mid-range ones (speech) • This can be shown with equal-loudness curves which plot perceived loudness on the axes of true loudness and frequency

7. Equal-Loudness Relations

8. Threshold of Hearing • The following image is a plot of the threshold of human hearing for pure tones – at loudness below the curve, we don’t hear a tone

9. Threshold of Hearing • A loud sound can mask other sounds at nearby frequencies as shown below

10. Frequency masking • We can determine how a pure tone at a particular frequency affects our ability to hear tones at nearby frequencies • Then, if a signal can be decomposed into frequencies, for those frequencies that are only partially masked, only the audible part will be used to set the quantization noise thresholds

11. Critical Bands • Human hearing range divides into critical bands • Human auditory system cannot resolve sounds better than within about one critical band when other sounds are present • Critical bandwidth represents the ear’s resolving power for simultaneous tones • At lower frequencies the bands are narrower than at higher frequencies • The band is the section of the inner ear which responds to a particular frequency

12. Critical Bands

13. Critical Bands • Generally, the audio frequency range for hearing (20 Hz – 20 kHz) can be partitioned into about 24 critical bands (25 are typically used for coding applications • The previous slide does not show several of the highest frequency critical bands • The critical band at the highest audible frequency is over 4000 Hz wide • The ear is not very discriminating within a critical band

14. Temporal Masking • A loud tone causes the hearing receptors in the inner ear to become saturated, and they require time to recover • This leads to the temporal masking effect • After the loud tone we cannot immediately hear another tone – post-masking • The length of the masking depends on the duration of the masking tone • A masking tone can also block sounds played just before – pre-masking (shorter time)

15. Temporal Masking • MPEG audio compression takes advantage of both temporal and frequency masking to transmit masked frequency components using fewer bits

16. MPEG Audio Compression • MPEG (Motion Picture Experts Group) is a family of standards for compression of both audio and video data • MPEG-1 (1991) CD quality audio • MPEG-2 (1994) Multi-channel surround sound • MPEG-4 (1998) Also includes MIDI, speech, etc. • MPEG-7 (2003) Not compression – searching • MPEG-21 (2004) Not compression – digital rights management

17. MPEG Audio Compression • MPEG-1 defined three downward compatible layers of audio compression • Each layer offers more complexity in the psychoacoustic model used and hence better compression • Increased complexity leads to increased delay • Compatibility achieved by shared file header information • Layer 1 – used for Digital Audio Tape • Layer 2 – proposed for digital audio broadcasting • Layer 3 – music (MPEG-1 layer 3 == mp3)

18. MPEG Audio Compression • MPEG audio compression relies on quantization, masking, critical bands • The encoder uses a bank of 32 filters to decompose the signal into sub-bands • Uniform width – not exactly aligned to crit. bands • Overlapping • A Fourier transform is used for the psycho-acoustical model • Layer 3 adds a DCT to the sub-band filtering so that layers 1 and 2 work in the temporal domain and layer 3 in the frequency domain

19. MPEG Audio Compression • PCM input filtered into 32 bands • PCM FFT transformed for PA model • Windows of samples (384, 576, 1152) coded at a time

20. MPEG Audio Compression • Since the sub-bands overlap, aliasing may occur • This is overcome by the use of a quadrature mirror filter bank • Attenuation slopes of adjacent bands are mirror images

21. MPEG Audio Algorithm • The PCM audio data is assembled into frames • Header – sync code of 12 1s • SBS format – describe how many sub-band samples (SBS) are in the frame • The SBS (384 in Layer 1, 1152 in Layers 2, 3) • Ancillary data – e.g. multi-lingual data or surround-sound data

22. MPEG Audio Algorithm • The sampling rate determines the frequency range • That range is divided up into 32 overlapping bands • The frames are sent through a corresponding 32-filter filter bank • If X is the number of samples per frame, each filter produces X/32 samples • These are still samples in the temporal domain

23. MPEG Audio Algorithm • The Fourier transform is performed on a window of samples surrounding the samples in the frame (either 1024 or 2*1024 samples) • This feeds into the psychoacoustic model (along with the subband samples) • Analyze tonal and nontonal elements in each band • Determine spreading functions (how much each band affects another)

24. MPEG Audio Algorithm • Find the masking threshold and signal-to-mask ratios for each band • The scaling factor for each band is the maximum amplitude of the samples in that band • The bit-allocation algorithm takes the SMRs and scaling factor and determines how many bits can be allocated (quantization granularity) for each band • In MP3, the bits can be moved from band to band as needed to ensure a minimum amount of compression while achieving higher quality

25. MPEG Audio Algorithm • Layer 1 has 12 samples encoded per band per frame • Layer 2 has 3 groups of 12 (36 samples) per frame • Layer 3 has non-equal frequency bands • Layer 3 also performs a Modified DCT on the filtered data, so we are in the frequency (not time) domain • Layer 3 does non-uniform quantization followed by Huffman coding • All of these modifications make for better (if more complex) performance for MP3

26. Stereo Encoding • MPEG codes stereo data in several different ways • Joint stereo • Intensity stereo • Etc. • We are not discussing these

27. MPEG File Format • MPEG files do not have a header (so you can start playing/processing anywhere in the file) • Consist of a sequence of frames • Each frame has a header followed by audio data

28. MPEG File Format

29. MPEG File Format • ID3 is a metadata container most often used in conjunction with the MP3 audio file format. • Allows information such as the title, artist, album, track number, year, genre, and other information about the file to be stored in the file itself. • Last 128 bytes of the file

30. Bit Rates • Audio (or Video) compression schemes can be characterized as either constant bit rate (CBR) or variable bit rate (VBR) • In general, higher compression can be achieved with VBR (at the cost of added complexity for code/decode) • MPEG-1 Layers 1 and 2 are CBR only • MP3 is either VBR or CBR • Average Bit Rate (ABR) is a compromise

31. MPEG-2 AAC • MPEG-2 (which is used for encoding DVDs) has an audio component as well • MPEG-2 AAC (Advanced Audio Coding) standard was aimed at transparent sound reproduction for theatres • 320 kbps for five channels (left, right, center, left-surround and right-surround) • 5.1 channel systems include a low-frequency enhancement channel (“woofer”) • AAC can also deliver high-quality stereo sound at bitrates less than 128 kbps

32. MPEG-2 AAC • AAC is the default audio format for (e.g.): YouTube, iPod (iTunes), PS3, Nintendo Dsi, etc. • Compared to MP3 • More sampling frequencies • More channels • More efficient, simpler filterbank (pure MDCT) • Arbitrary bit rates and variable frame lengths • Etc. etc.

33. MPEG-4 Audio • MPEG-4 audio integrates a number of audio components into one standard • Speech compression • Text-to-speech • MIDI • MPEG-4 AAC (similar to MPEG-2 AAC) • Alternative coders (perceptual coders and structured coders)