Digital Media

1. Digital Media Lecture 4: Bitmapped images: Compression & Convolution Georgia Gwinnett College School of Science and Technology Dr. Jim Rowan

2. Refer to Supplemental text: • Introduction to images   • Ways to store an image as numbers: Bitmapped Graphics   • Ways to store an image as numbers: Vector Graphics   • Image encoding techniques: Bitmapped Images   • BItmappedimages: JPEG compression and human vision   • BItmappedimages: Dithering and Posterization • Integer screen coordinates and bounding boxes   • Image encoding: Messing around with the color coding   • Art and Mathematics Collide! (of Bezier Curves and Convolution)   • A compete convolution

3. Bitmapped image compression Consider this image: With no compression... RGB encoding => 64 x 3 = 192 bytes 64 pixels

4. A Side Note We’ve been talking about RGB encoding for images… So… How many different colors can you make if using a 24 bit RGB color scheme? 2**24 = 16,777,216 different colors

5. Bitmapped image compression Color table representation: With 2 colors: #0 - 0 100 100 #1 - 255 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 64 pixels -6 bytes for the color table -64 X 1 = 64 / 8 = 8 bytes for the pointer table 6 + 8 = 14 bytes total

6. Bitmapped image compressionRun length encoding: 9RGB 6RGB 2RGB 6RGB 2RGB 6RGB 2RGB 6RGB 2RGB 6RGB 2RGB 6RGB 9RGB 9(0,0,255) 6(255,0,0) 2(0,0,255)6(255,0,0) 2(0,0,255) 6(255,0,0)2(0,0,255) 6(255,0,0) 2(0,0,255)6(255,0,0) 2(0,0,255) 6(255,0,0)9(0,0,255) = 52 bytes 64 pixels

7. Bitmapped image compressionRun length encoding: RLE gets its advantage when there are a number of pixels next to one another that are the same color This advantage is dependent on the CONTENT of the image. Why? Could RLE result in a larger image? How?

8. Bitmapped image compressionRun length encoding: Consider this image: RLE compression... 1RGB 1RGB 1RGB 1RGB 1RGB... 1RGB 1RGB 1RGB -> 256 bytes 64 pixels

9. Bitmapped image compressionLossless or Lossy? 24 bit RGB encoding is lossless What about table encoding? Lossless and lossy It depends on…? What about RLE encoding? Lossless

10. Bitmapped image compressionJPEG • JPG is Lossy • Best suited for natural photographs • Fine details with continuous tone changes • JPEG takes advantage of the fact that humans don’t perceive the effect of high frequencies accurately • High frequency components are associated with abrupt changes in image intensity… like a hard edge

11. Bitmapped image compressionJPEG JPEG finds these high frequency components by treating the image as a matrix Using the Discrete Cosine Transform (DCT) to convert an array of pixels into an array of coefficients The DCT is expensive computationally so the image is broken into 8x8 pixel squares and applied to each of the squares

12. Bitmapped image compressionJPEG The high frequency components do not contribute much to the perceptible quality of the image So JPEG encodes these frequencies at different quantization levels… The low frequency components are with greater detail than the high frequency changes. ==>JPEG uses more storage space for the more visible elements of an image

13. Bitmapped image compressionJPEG Why use a lossy compression technique? It is effective for the kinds of images it is intended for ==> 95% reduction in size But it suffers from artifacts like edges that blur... WHY? Can the difference be seen? HMMMmmmm…

14. Bitmapped image compressionOriginal file zoomed in

15. Bitmapped image compressionComparing tiff & jpeg

16. Bitmapped image compressionJPEG Again… Why use a lossy compression technique? For this example: The original .tiff file is 1,300,000 bytes The jpeg file is only 156,000 bytes When viewing the two full size images you cannot see the minute differences Only zooming in to the image shows the artifacts caused by jpeg compression

17. Side Note!To make matters worse… • Humans are a mess! • The human vision system is very complex • Upside down • Split- left side of eye to right side of brain • Right side of eye to left side of brain • Cones and rods not uniformly distributed • Cones and rods are upside down resulting in blind spots in each eye that we just ignore! • Partially responsible for making lossy techniques work… you don’t see what you think you see ==>

18. Optical illusions Humans are a mess! See Additional Class Information: Illusions

19. Bitmapped image compressionreturning toResizing issues…

20. Resizing issues Here the original 4x4 image is doubled in both dimensions to 8x8 by adding pixels But this example is pretty simple because the original is all one color…

21. Resizing issues If you double both dimensions you have to add pixels... But what color? ? ? ? ? Well, the answer is… it depends!

22. Resizing issues If you double both dimensions you have to add pixels... But what color? ? ? ? ? The simplest approach would be to copy the color but… You can consider the colors that surround the original pixel

23. Resizing issues If you cut each of the dimensions in half you remove… (8x8 -> 4x4)=> 64 - 16 = 48 pixels You have to remove 3/4 of the pixels! 16 pixels 64 pixels How do you decide which pixels to remove?

24. Resizing issues 1 One answer: throw them away! Here it works... but only because it is a solid color 2 3

25. Resizing issues But what if it is multi-colored? You can use the information in the surrounding pixels to influence the remaining pixel 1 2 3 How do you do this? Remember… it’s just numbers in there!

26. Convolution Underlies much of bitmapped image processing including downsizing and filters like blur and sharpen 1 2 3

27. Blur filter: Convolution under the sheets

28. Convolution: How it works Images are collections of numbers Images as piles of numbers: Art and Mathematics Collide! (of Bezier Curves and Convolution)

29. Convolution: How it works Convolution uses a convolution matrix (in this case 3 X 3) to process the original image one pixel at a time

30. Convolution: How it works Resulting in a completely new image… But how?

31. Convolution: How it works

32. Convolution: How it works

33. Convolution: How it works

34. Convolution: How it works 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 = 0 + 85 + 0 + 0 + 85 + 0 + 0 + 85 + 0 = 255

35. Convolution: How it works

36. Convolution: How it works 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 0+ 3/9 x 0 + 0/9 x 0= 0 + 85 + 0 + 0 + 85 + 0 + 0 + 0 + 0 = 170

37. Convolution: How it works

38. Convolution: How it works 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 0 + 3/9 x 0+ 0/9 x 0+ 0/9 x 255 + 3/9 x 255 + 0/9 x 255 = 0 + 85 + 0 + 0 + 0 + 0 + 0 + 85+ 0 = 170

39. Convolution: How it works

40. Convolution: How it works 0/9 x 0 + 3/9 x 0 + 0/9 x 0 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 = 0 + 0 + 0 + 0 + 85+ 0 + 0 + 85+ 0 = 170

41. Convolution: How it works

42. Convolution: How it works 0/9 x 255 + 3/9 x 255+ 0/9 x 255 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 + 0/9 x 255 + 3/9 x 255 + 0/9 x 255 = 0 + 85 + 0 + 0 + 85 + 0 + 0 + 85+ 0 = 255

43. Convolution: How it works

44. Convolution: How it works • In short… • the math is simple • there’s a lot of multiplication • there’s a lot of addition • just keeping track of where you are is really the only issue here!