eCommerce Technology 20-751 Lecture 6: Multimedia

1. eCommerce Technology20-751Lecture 6: Multimedia

2. MULTIMEDIA • Which media can be delivered remotely? • Text, graphics, audio, video, motion picture, virtual reality • What are the limitations? • Speed, resolution, fidelity, color • How are they created? • How are they stored? • File formats: GIF, TIFF, JPEG, MPEG, ... • How are they displayed (put in web pages)?

3. Pixels (Picture Cells) Discrete array of cellsholding gray-scale data 1-bit (black or white) 8-bit (0 - 255) To represent color, use multiple pixel arrays(usually 3)

4. RGB (Additive) Color Space • R, G, B each indicated by intensity 0-255 (24 bits) • Additive color applet • Visibone color selector • RGB, RBW, HSV, RGYB, HLS, YIQ converter RGB SCREEN PHOSOPHORS CREATING COLORS BY ADDING RED, GREEN + BLUE

5. Gamut (full range of colors) Gray = human visible range 1. WIDE-GAMUT RGB 2. HDTV (SM PTE 240-M) 3. Custom RGB 4. ColorMatch RGB 5. sRGB (average monitor) CYM = cyan, magenta, yellow (subtractive)

6. YUV Color Space • Y = luminance (brightness, intensity of yellow) • U, V = chrominance (difference between a color and yellow) U = B - Y, V = R - Y • YUV is used in TV SOURCE: N. BARNETT, VA. TECH.

7. YUV Color Space Y = 0.299 R + 0.587 G + 0.114 BU = 0.492 (B - Y)V = 0.877 (R - Y)

8. Bit Depth 256 COLORS 16,777,216 COLORS 2 COLORS: B, W 16 COLORS

9. Dithering • File contains color information system can’t display • Dithering is approximating a non-displayable color • Image dithering: replace with 2 or more colors • Screen dithering (substitution): replace with 1 color IMAGE-DITHERED VERSION OF MARKED DETAIL AREA SOURCE: R. LINEBACK

10. Palettes • The palette is the range of colors available for display • A palette is browser-safe if it can be displayed on all popular browsers • A browser-safe palette • Netscapecolor palette map

11. Netscape (Non-Dithering) Palette RGB (0%, 20%, 40%, 60%, 80%, 100%) 0% = 0 = Hex 00 20% = 51 = Hex 33 40% = 102 = Hex 66 60% = 153 = Hex 99 80% = 204 = Hex CC 100% = 255 = Hex FF 6 choices for each of R, G, B = 216 colors SOURCE: WEBREFERENCE.COM

12. Palette of Indexed Colors • Color values indicate one of a set of an arbitrary set of colors, not RBG mixture. (LUT = look-up table) SOURCE: HANDBOOK OF GRAPHIC FILE FORMATS

13. Compression • There’s never enough space • File (string) is a representation of information • A representation (encoding) has a size • Compression means reducing encoding size • When can a string be compressed? • 0000100001000010000100001000010000100001 • (00001)1000 means “00001” repeated 8 times • But some 40-bit strings will get longer this way

14. Compression • An encoding is a “program” for generating the string • Strings having short programs can be compressed • Strings without short programs (Chaitin-Kolmogorov “random” strings) can’t be compressed • Compression limit: • n bits can only encode 2n different strings • Dirichlet pigeonhole principle • If p+1 pigeons fly into p holes, some hole has more than one pigeon • Try to encode > 2n strings in n bits loses information

15. RUN-LENGTH CHARACTER Run-Length Encoding (RLE) • A run is a repeated character • Represent a run by two bytes: • String “AAAAAAbbbXXXXXt” (15 characters)become “6A3b5X1t” (8 characters) • Advantages of RLE: fast, easy to code • What happens in RLE if no character is repeated?

16. 3-Byte Run-Length Encoding For repeated characters, use 3 bytes: “flag” + 2- byte RLE. For solo characters, use 1 byte. SOURCE: HANDBOOK OF GRAPHIC FILE FORMATS

17. Bitmap (BMP) Format • Header • Bitmap data (indexed color or RGB) • 1, 4, 8, 16, 24, 32 - bit color • Palette • RLE, otherwise uncompressed • Generally very large

18. Dictionary Compression • Idea: choose a dictionary with < 65,535 words • Number the words: • administration (3773), administrations (3774), fascinating (11012), xylophone (43722) • Every word can be represented by 2 bytes. (2 bytes = 16 bits = 0..65535) • Works for text • Doesn’t work for graphics

19. LZW (Lempel-Ziv-Welch) Compression • Start dictionary with all 1-character strings • C is next input character, P is empty • Is P | C in the dictionary? • If so, let P = P | C • If not, • Output the code for P • Add P | C to the dictionary (and give it a code) • Set P = C • Repeat from “Is” until last character is seen, then output the code for P LZW Explanation Page (McGill)

20. LZW (Lempel-Ziv-Welch) Compression ALGORITHM: Is P | C in the dictionary? If so, let P = P | C If not, Output the code for P P | C dict (new code) Set P = C A B B A B A B A C & 1 2 3 4 5 6 7 8 9 Dictionary = (1) A (2) B (3) C Step Pos P C Dict. Output 1 - A 1 1 A B (4) A B 1 2 2 B B (5) B B 2 3 3 B A (6) B A 2 4 4 A B 5 6 AB A (7) A B A 4 6 7 A B 7 8 AB A 8 9 ABA C (8) A B A C 7 6 10 C 3 1 2 2 4 7 3 = A B B A B A B A C! Dictionary not stored!

21. Graphic Image File (GIF) Format GIF identifier Screen dimensions Indexed color palette Position and size of image Indexed color palette LZW compressed data

22. 1004 BYTES 941 BYTES 1452 BYTES 6685 BYTES 1803 BYTES 1239 BYTES GIF Compression • Since LZW is used, compression depends on randomness of image SOURCE: KILLERSITES.COM

23. Compression Tradeoffs • Size v. Quality .bmp 1000 KB, compression factor = 1 .gif 172 KB, compression = 5.8 .jpg 47 KB, high quality, compression = 21 .jpg 13 KB, low quality, compression = 75 • Size v. processing time • time to compress • time to decompress • not necessarily equal

24. Lossless v. Lossy Compression • A method of compression is “lossless” if no two strings become identical when compressed • RLE and LZW are lossless • Lossy compression: at least two strings compress to the same result • Example: Compress a 32-bit string by taking just the first (leftmost) 16 bits (truncation) • If both A and B yield the compressed string C, we have “lost” the ability to recover the original • Lossy compression is OK for images if all possible decodings of C “look” almost the same

25. JPEG Format • JPEG = Joint Photographic Experts Group • Lossy compression • Discards visual info not seen by human eye (perceptual encoding) • slight changes in color not visible • slight changes in intensity easily visible • So, sacrifice color info in favor of intensity • Typical compression of 25:1 without degradation • User can control quality

26. COMPRESSION RATIO 13.5 SOURCE: NCSU

27. COMPRESSION RATIO 32.4 SOURCE: NCSU

28. JPEG Compression CHANGE COLOR SPACE REDUCE COLOR INFO DISCRETE COSINE TRANSFORM DISCARD HIGH FREQUENCIES COMPRESS RESULT

29. JPEG Format • Operates on blocks of 8 x 8 pixelsin the R, G and B planes • Must transform RGBinto a color spacethat separatescolor frombrightness:

30. Color Conversion and Downsampling • Must convert from RGB into a color space that separates color from brightness • Use YUV (human eye most sensitive to yellow) • Keep the Y (yellow luminance) data • Clip (shorten) the U, V data (U, V represent color differences so they are not very important)

31. Discrete Cosine Transform SOURCE: LULEÅ UNIV.

32. Discrete Cosine Transform

33. JPEQ Quantization (Truncation) • Idea: Threshold the values so that low ones (most of them) become zero:

34. Zig-Zag Run-Length Encoding • Zig-zag through the matrix so the tail of the sequence is all zeroes: 168, 45, 67, 12, 20, 7, 3, 3, 5, 2, 0, 0, 5, 3, 2, 0, 2, 2, 0, 0, 0, 0, . . . 64 numbers reduced to 18, a factor of ~ 4

35. JPEG Quality v. Compression One image is uncompressed (313 KB). One image is compressed by a factor of 13.6 (23 KB). Which is which? SOURCE: E. DELP, PURDUE

36. JPEG Quality v. Compression NW: Quality = 75, Compression = 13.6 NE: Q = 20, C = 37 SW: Q= 5, C = 78 SE: Q = 3, C= 96

37. JPEG Compression Example • Street scene from Simon Fraser University • Interactive JPEG example (Simon Fraser)

38. Sequential JPEG Image appears from top down:

39. Progressive JPEG Image “comes in” to focus:

40. Motion • Capture viewer • Requires concentration • Utilizes natural animal attention to movement • Involvement • Realism • Real-world object move • Convey message • Direct viewer’s attention

41. Delivering Video • Uncompressed video • 640 x 480 resolution • 24-bit color = 7.37 megabits/frame • 30 frames per second = 210 megabits/sec. • 1920 x 1080 = 1.5 Gbps • Compressed video • MPEG-2. Broadcast quality. 15 Mbps • AVI, QuickTime. 500 Kbps • RealNetworks. Designed for network speeds.20-200 Kbps. (At 20 Kbps, compression factor is greater than 10,000.)

42. Streaming • Begin delivering content without full download • Lower latency • Low client storage requirement • Can start streaming at any point (not necessarily beginning • Live streaming possible (real-time compression) • Disadvantages • Lower quality, also affected by network connection • More time needed to compress • Need special server

43. pnm METAFILE pnm METAFILE True Streaming 1. USER REQUESTS STREAMING MEDIA 4. BROWSER INVOKES PLUG-IN 2. BROWSER with video plug-in BROWSER REQUESTS pnm METAFILE REALPLAYER PLUG-IN PLUG-IN REQUESTS STREAM 5. STREAM CLIENT SIDE SERVER SIDE 3. 6. REALSERVER UPLOADS STREAM SERVER UPLOADS METAFILE WEB SERVER REALSERVER pnm METAFILE STREAM

44. Q A &