Fundamentals

1. Fundamentals RawesakTanawongsuwan ccrtw@mahidol.ac.th

2. Digital Data • Bits are units of data that can only have one of two values. • A byte is eight bits. • Groups of bits can be interpreted as numbers to base 2, but can also be treated as characters, colours, etc.

3. Bytes to remember • 1 byte= 8 bits • 2 bytes = 16 bits • 3 bytes = 24 bits • 4 bytes = 32 bits • . • . • .

4. Analogue and Digital Representation

5. Digitization • Converting a signal from analogue to digital form • Analogue signal can vary continuously, digital is restricted to discrete values • Two-stage process • Sampling – measure the value at discrete intervals • Quantization – restrict the value to a fixed set of quantization levels

6. Sampling and Quantization

7. Sampling and Quantization • The sampling rate is the number of samples in a fixed amount of time or space. • The quantization levels are the set of values to which a signal is quantized.

8. Analogue vs Digital

9. Benefits of digital signals • High fidelity • Noise tolerance

10. Frequency Domain • Any periodic waveform can be decomposed into a collection of frequency components • Each one is a pure sine wave • The collection of frequencies and their amplitudes represent the waveform in the frequency domain • Compute the frequency domain representation (frequency spectrum) using the Fourier Transform • Higher frequency components are associated with abrupt transitions

11. Spatial and temporal signals • Spatial and temporal signals are made up of pure sine wave components at different frequencies. • The Fourier Transform operation can be used to compute a signal’s representation in the frequency domain. • Higher-frequency components are associated with abrupt transitions.

13. Sampling Theorem • If the highest frequency component of a signal is at fh the signal can be properly reconstructed if it has been sampled at a frequency > 2fh • Nyquist rate • Undersamping leads to aliasing • Sound distortion, image 'jaggies' or Moiré patterns, jerky or retrograde motion

14. Aliasing Aliased Anti-aliased

15. Aliasing

16. Moiré Patterns

17. Too Few Quantization Levels • Reducing memory requirements by using fewer bits for each value means fewer quantization levels are available • Cannot distinguish between values that fall between levels • Images: banding and posterization • Sound: coarse hiss, loss of quiet passages, general fuzziness (quantization noise)

18. Image banding effects

19. Posterization

20. Digital Representation of Media • There are established ways of representing images, video, animation, sound and text in bits. • Media data may be represented as a textual description in a suitable language, or as binary data with a specific structural form.

21. Image • Images are displayed as arrays of pixels and represented using an internal model. Generating the pixels from the model is called rendering.

22. Bitmaps vs Vector Graphics • Images may be modelled as bitmaps or vector graphics. • A bitmap is an array of logical pixels (stored colour values) that can be mapped directly to the physical pixels on the display. • In vector graphics, the image is stored as a mathematical description of a collection of individual lines, curves and shapes making up the image, which requires computation to render it.

25. Good Things for Vector Graphics • Often smaller than bitmaps • Resolution-independent • Scalable without loss of quality • only suitable for certain sorts of synthetic image, not photographs

29. Combining Vectors & Bitmaps • Rasterize vectors • Lose all their vector properties • Trace bitmaps • Difficult and can only produce an approximation (parameterized)

30. Layers • Organizational device used in both sorts of graphics, especially useful in bitmaps • Permits separation and manipulation of different parts of a bitmapped image • Digital version of clear sheets of acetate, stacked on top of each other • Areas without coloured pixels/graphic objects are transparent so lower layers show through • Compositing – combine layers using different blending modes (digital collage)

31. Creatively Blending Layers http://www.carlvolk.com/photoshop15.htm

32. Creatively Blending Layers http://www.carlvolk.com/photoshop15.htm

33. Graphics/Image Data Types

34. RGB Components

35. Moving pictures, Videos, Animation • Moving pictures can be created as live-action or animation. • Live-action must be stored as video. • Animation may be represented in other more flexible or efficient ways. • Video frames require a lot of storage so video is invariably compressed for delivery.

37. Sound • Sound can be represented as a sequence of samples after digitization. • CD audio is sampled at 44.1 kHz, higher sampling rates are sometimes used. • Audio delivered over the Internet is compressed, often using the MP3 codec.

38. Text • A character set is a mapping from characters to character codes. • Unicode is a character set capable of representing text in all known languages. • A font is a set of character shapes, called glyphs.

40. Layout and Typography • Many aspects of layout must be controlled when text is displayed.

41. Interactivity • Interactivity is produced by executing a program in response to user input. • In multimedia, programs are often written in a scripting language, such as JavaScript or ActionScript.

42. Compression • Compression must often be applied to media data. • Compression may be lossless or lossy. • Lossless – image can be reconstructed exactly from compressed version • Lossy – some information discarded, image can only be reconstructed approximately

44. Compression • Different compression algorithms are applicable to different types of media data. Their effectiveness depends on the characteristics of the data itself.