Theoretical Foundations of Multimedia

1. Theoretical Foundationsof Multimedia Dr. Robert S. Tannenbaum Director, Academic Computing Services University of Kentucky rst@pop.uky.edu

2. Hardware that Enables Multimedia • Input and Output Devices • Virtual Reality Devices • Modems and Network Interfaces

3. Input and Output Devices Output • Monitors • Speakers and MIDI interfaces • VR helmets and immersive displays • Keyboards and OCR devices • Digital cameras, scanners, & CD-ROMs • MIDI keyboards and microphones • Video cameras and frame grabbers • Mice, track balls, joysticks, and VR gloves and wands Input

4. Monitors A simplified cathode ray tube (CRT)

5. Monitors • Pixel — a picture element; a dot of color on the screen • Three different phosphors at each pixel to create the color • RGB (Red, Green, Blue) • CYM (Cyan, Yellow, Magenta)

6. Monitors Raster Scanning

7. Monitors • Refresh rate — the frequency at which the phosphors are excited • Normally the refresh rate is given in Hertz • For flicker-free images 75 Hz or faster is desirable • The refresh rate for a projector needs to be coordinated with the monitor

8. Monitors • The digitized image to be displayed must be stored in a buffer • The stored image is said to be “bit-mapped,” because, for monochrome images, the map used just one bit per pixel • Multimedia monitors use 24 bits per pixel (8 for each color); can define >16 million colors

9. A Good Multimedia Monitor • Large enough for comfortable viewing, probably 15” or greater • Pixel size of no more than 0.28mm • Refresh rate of at least 75 Hz • Capable of displaying 24-bit color • Designed for the CPU and operating system • Ergonomically comfortable and attractive

10. Speakers and MIDI Interfaces • Storage of digitized sound files • Reproduction via digital-to-analog conversion sent to a loudspeaker • Built-in speakers often do not have sufficient fidelity • Low-powered (3- to 5-watt) external speakers or head-phones will serve a single user and provide excellent fidelity

11. Speakers and MIDI Interfaces • Storage of synthesizer command files • Create the sounds by sending the commands to a synthesizer • Musical Instrument Digital Interface (MIDI) standard (1982) • MIDI includes both a hardware and a message standard

12. Speakers and MIDI Interfaces • MIDI hardware standard defines cables, connectors, circuits, and electrical signals • MIDI message standard defines a • Device number for multiple device systems • Control segment that gives an instrument-specific command • Data segment containing the information needed by that instrument for that command

13. Alphanumeric Keyboards • For entering commands, text, and data • Each key is a switch that closes when it is depressed, sending a code to the CPU • The arrangement of the keys may vary • The most common is QWERTY • Another arrangement is Dvorak

14. Choosing a Keyboard • Does it include all of the needed characters, including command keys? • Is it ergonomically comfortable and safe, preventing repetitive stress syndrome?

15. Optical Character Recognition (OCR) • Hardware — scans the text image • Software — systematically checks the entire image for patterns of light and dark that it recognizes as alphabetic, numeric, or punctuation characters • OCR software entails pattern recognition, a sophisticated logic problem

16. Optical Character Recognition (OCR) E e E E e e E EE e eEE E e e E e e It is relatively easy for a human to recognize each of these characters as the letter “e.” For the pattern recognition logic in OCR software, this is very difficult.

17. Digital Cameras and Scanners • Real Image — a portion of what is physically present in nature • Digital Image — a representation of a real image in which individual points are encoded to represent the wavelength and intensity of light at that point • Still Image — a single snapshot of an instant; may be real or digital • Motion Image — a sequence of images that, when viewed consecutively at the appropriate rate, gives the impression of continuous motion; may be digital or analog

18. Scanners Schematic Drawing of a Scanner

19. Digital Cameras Schematic Drawing of a Digital Camera

20. Digital Cameras and Scanners • Quality of the optics and the scanning mechanism, which determines focus • Precision of the photosensitive cells, which determines the accuracy of the encoding of intensity and wavelength data • Resolution of the instrument in dots per inch, which determines graininess • Amount of storage available, which determines the total size of an image that can be digitized Image quality depends on the:

21. Dots/inch resolution of snapshot image 100 300 600 1200 Bytes required for storage 1.05 Mb 9.45 Mb 37.8 Mb 151.2 Mb Inputting Images Assuming no compression, 24 bits per pixel Memory required to store a 5” x 7” snapshot

22. Video Cameras and Frame Grabbers • Video cameras are similar to digital cameras • Except that a video camera takes image after image continuously • The output from many video cameras is analog and requires digitizing circuitry to make the image usable in a computer • Digital camcorders are now available • Frame grabber software allows the capture of a single still image from the video stream • Frame grabbed images are of rather low resolution, however, <80-90 dots/inch

23. Microphones and MIDI Keyboards • For input of sound • Microphones capture sound waves from the air as an analog signal • The analog signal must be digitized to be stored and then replayed by the computer • Digitizing at <10,000 Hz is adequate for speech; 20,000-40,000 Hz is needed for music • MIDI keyboards usually look like piano key- boards with extra switches and controls • MIDI keyboards encode and transmit musical information according to the MIDI standard