CGMB 324: Multimedia System design

1. CGMB 324: Multimedia System design Chapter 04: Multimedia Element II – Image (Part III)

2. Objectives • To understand the concept and theory of image compression • To understand the two categories of image compression • To describe some of the commonly used compression techniques for image • To apply images in multimedia systems

3. Image Compression • image files (especially raster images) tend to be very large. • It can be useful or necessary to compress them for ease of storage or delivery. • However, while compression can save space or assist delivery, it can slow or delay the opening of the image, since it must be decompressed when displayed. • Some forms of compression will also compromise the quality of the image.

4. Image Compression • Compressions rely on two main strategies: • getting rid of redundant information ('redundancy reduction') • getting rid of irrelevant information ('irrelevancy reduction'). • Redundancy reduction is often used when the image is being encoded (or re-encoded). • It looks for patterns and repetitions that can be expressed more efficiently. • If, for example, there are 25 black pixels in a row, it is clearly better to record the information for one pixel and state that the next 24 are all the same, than to record each pixel separately. • This particular example is known as Run-Length Encoding (RLE).

5. Image Compression • Irrelevancy reduction aims to remove or alter information that makes little or no difference to the perception of the image. • This usually happens prior to the encoding and involves a transformation of the image. • Some of an image's colour information, for example, can be safely simplified without being perceptible to the human eye. • However, when carried to an extreme this sort of compression becomes visibly obvious and compromises the quality of the image.

6. Compression Categories • It is common to classify compressions into two groups: 'lossless' and 'lossy‘ • Lossless compressions are based on redundancy reduction and typically concentrate on more efficient ways of encoding the image data. • More sophisticated lossless compressions will also transform the colour information, but only in order to store or deliver the image and always in a way that can be reversed. • The key point to grasp about lossless compression is that no information is irretrievably lost in the process. • Once decompressed, a lossless image will always appear exactly the same as the original, uncompressed, image.

7. i) Lossless Compression • A commonly used lossless compression is the LZW (named after its developers, Lempel, Ziv and Welch). • LZW operates by default within the GIF format and is an optional compression within the TIFF format. • Like run-length encoding, LZW looks for recurrent patterns in the image's raster data. • It replaces these with codes, giving the most common patterns the shortest codes and storing all the definitions in a separate dictionary. • Although LZW is a very good compression, it has become subject to patent claims, so newer image file formats have avoided using it

8. i) Lossless Compression - LZW • Compressor algorithm w = NIL; add all possible charcodes to the dictionary for (every character c in the uncompressed data) do if ((w + c) exists in the dictionary) then w = w + c; else add (w + c) to the dictionary; add the dictionary code for w to output; w = c; endif done add the dictionary code for w to output; display output;

9. i) Lossless Compression - LZW • Decompressor algorithm read a char k; output k; w = k; while (read a char k) do if (index k exists in dictionary) then entry = dictionary entry for k; else if (k == currSizeDict) entry = w + w[0]; else signal invalid code; endif output entry; add w+entry[0] to the dictionary; w = entry; done

10. RLE (Run Length Encoding) • a very simple form of lossless data compression in which runs of data (that is, sequences in which the same data value occurs in many consecutive data elements) are stored as a single data value and count, rather than as the original run. • This is most useful on data that contains many such runs; for example, simple graphic images such as icons, line drawings, B&W images, cartoons-style graphics

11. RLE (Run Length Encoding) • Also called Packed Bits encoding • E.g. ‘abcddddddcbbbbabcdef’ • Those 20 characters are stored as 20 bytes (1 byte each) ordinarily. • However, using RLE, it can be shortened to ‘abc6dc4babcdef’  14 bytes only • In general, it is a bit more sophisticated than this. • The common approach is to use only 7 of the 8 bits to indicate the run length, this is normally interpreted as a signed byte. • The range might be -127 to 128.

12. RLE (Run Length Encoding) • If the length byte is positive it indicates a replicated run (run of the following byte). • If the number is negative then it indicates a literal run, i.e. that number of following bytes is copied as is. • To illustrate this, the following sequence of bytes would encode the example string given (abcddddddcbbbbabcdef ), it requires 17 bytes • -3 a b c 6 d -1 c 4 b -6 a b c d e f

13. RLE (Run Length Encoding) • RLE will not always result in a compression. • Consider a string where the next character is different from the current character. • Every 127 bytes will require a extra byte to indicate a new literal run length • The best case is when 128 identical characters follow each other, this is compressed into 2 bytes instead of 128 giving a compression ratio of 64

14. RLE (Run Length Encoding) • General disadvantage of RLE scheme: • When groups of adjacent pixels change rapidly, the run length will be shorter. • It could take more bits for the code to represent the run length than the uncompressed data  negative compression. • It is a generalization of zero suppression, which assumes that just one symbol appears particularly often in sequences.

15. The following 3 images illustrate the different extremes, • The first image contains runs along each row and will compress well. • The second image is the same as the first but rotated 90 degrees so there are no runs giving worse case and a larger file. This suggests a natural extension to RLE for images, that is, one compresses vertically and horizontally and uses the best, the flag indicating which one is used is stored in the image header. • The last case is the best scenario where the whole image is a constant value.

16. ii) Lossy Compression • based on irrelevancy reduction strategies, but will usually also employ some redundancy strategies, particularly in their encoding. • transform and simplify the image information  larger reductions in file size than lossless compressions. • The lossless LZW compression cut the file size to three quarters or two thirds of the original, even halve if the image has few colours, like a logo. • A lossy compression like the JPEG will reduce the file size to as little as 1% of the original, although anything less than 10% is likely to visually distort the image.

17. ii) Lossy Compression • The trade off, however, is that a lossy compression is by definition irreversible - it permanently disposes of information. • The JPEG is the best known lossy compression. • Its irrelevancy strategy is based on the characteristics of human visual perception. • Relying on the fact that people can more easily distinguish brightness (luminance) than colour (chrominance), JPEG concentrates its compression on the colour information within the image.

18. Applying Images In Multimedia Systems

19. Applying Images In MM Systems • There are a lot of things to consider when applying images into multimedia systems • Images  meant to draw peoples’ attention to a certain matter or make clear words that might be ambiguous. • There are few things more irritating than a poor image used to explain something. • Multimedia systems demand high quality (resolution) images and graphics, often regardless of what they are used for. • Much like good language used to express an idea is always better than poor language

20. A poor quality image used in a multimedia system is not good. If faced with the choice using a poor image and not using one at all, sometimes, it is better to choose the latter. A high quality image on the other hand, is always welcome and much more pleasing to the eye. Sometimes, proper adjustment of color, contrast, aspect ratio and cropping can make good an otherwise dull photo.

21. Applying Images In MM Systems – Effects & Impressions • We also need to create the proper effect with images. • Often, an image is used to deliver a direct message of fear or sorrow to the user. Apart from being a high quality image, it must also be taken well (from the right angle, distance etc). • In some extreme cases, a picture is so good, that compressing it to a lossy format causes it to lose its glitter. In these cases, it is allowed to leave the image uncompressed (e.g. as *.bmp, instead of *.jpg). • One of the techniques used by image compression, is to reduce the number of colors stored in an image. This usually doesn’t matter much because we don’t notice the difference anyway. But sometimes, we do.

22. Applying Images In MM Systems – Effects & Impressions “… and so the shark lunged at him from the ocean. Mr. Abrahoy froze in terror.” Perhaps part of a story you’re telling and this image was your choice to depict what is being said.

23. Maybe this would be a better and more accurate image for various reasons. “… and so the shark lunged at him from the ocean. Mr. Abrahoy froze in terror.”

24. Applying Images In MM Systems - Representation • We also use images to give a ‘face’ to our audio (probably narration) in multimedia applications. • There are many reasons for this. • For example, if we have a nice male or female voice for narration, it gives more of a ‘human feel’ if there is a photo (perhaps computer generated, or even a fictional person) that the user could relate to. • The human mind works by association. • If there was only a voice, we would have nothing much to associate and remember it by.

25. Applying Images In MM Systems “…what do you want!?” “… and the latest updates are…” “I’m sorry, but you failed.” • If there is a photo, every time we hear the voice, we tend to imagine it is a real person telling us something; thereby giving more attention to it.

26. Applying Images In MM Systems – File Size & Quality • Though JPEG compression provides us with high quality images at small file sizes, we must remember that collectively, images do add up. • Also, JPEG has a variety of quality settings you can play with. • For example, an image, such as the photo of a house; can be saved with perhaps just a 50% quality factor. • However, a person’s photo, often needs to be saved in excess of a 70% quality factor, especially if it is likely to be focused or examined closely by the user

27. Applying Images In MM Systems – Colors & Space • Sometimes, the very nature of the image can effect the file size. • Image compression uses a variety of techniques to save space. • For example, the more colors that exist in an image, it is likely that the compressed file will be larger than one with fewer colors. • This mean that in images where there are many shades of dark colors, the compressed image might be larger than one which is mainly white. After compression (gif), whitish images might display more noise than dark ones. 300x225 pixels 12.3KB Web-color Before compression (jpg) 300x225 pixels 17.7 KB 24-bits color

28. Applying Images In MM Systems – Colors & Space • Generally, less complex images occupy less space than simple ones (e.g. images with flat colors). • Also, where colors are limited, it is often better to save lighter colored images rather than dark ones. • For example, this sometimes effects the file size of GIF color images which only use 256 colors.

29. Applying Images In MM Systems – Color Correction • Color correction is another important factor when dealing with images. • Sometimes, the image itself is of sufficient resolution and clarity, but the colors are a bit dull. • Most photo editors (even the simple ones), allow you to bring out details that would otherwise be hidden. This makes the photo richer.

30. Applying Images In MM Systems – Color Correction • Human eyes are sensitive to light, therefore, a bright, glossy picture, delivers more impact than a dull one. • If a scanner is used to obtain the image, color corrections is almost always mandatory. • Remember to always work with the image in its uncompressed form, though.

31. Applying Images In MM Systems – Color Correction • The picture on the left is clear enough, but it would benefit from some color correction. • The one on the right has a good color balance and doesn’t require any adjustments.

32. Applying Images In MM Systems – Aspect Ratio • The photo on the left lacks luster, but after some color correction and aspect ratio adjustment (right), it can be improved to what you see on the right. • Reducing the aspect ratio (width/height) allows us to make the objects of focus look bigger/wider than they are; like the monkey in the photo. • Wherever possible, try to improve your images. • Always keep a backup of the original, though – in case things go wrong.

33. Invert Original Image Cut-Out Graphic Pen Diffuse Glow Applying Images In MM Systems – Special Effects • Special effects are also something easily achieved with images editing tools. • We can create a variety of diverse effects from the use of only just a few functions/filters.