Image Processing and Coding

1. Image Processing and Coding

2. Review of Lecture #2 In the first lecture, we covered the • Signal? • Time-domain description • Waveform • Periodic vs. non-periodic signals • Frequency-domain description • Periodic signals • Sinusoidal signals • Fourier series for periodic signals

3. Image • Rich info. from visual data • Examples of images around us natural photographic images; artistic and engineering drawings scientific images (satellite, medical, etc.) • “Motion pictures” => video movie, TV program; family video; surveillance and highway/ferry camera • A picture is worth 1000 words. • A video is worth 1000 sentences?

4. Image Processing • Enhancement and restoration • Remove artifacts and scratches from an old photo/movie • Improve contrast and correct blurred images

5. Image Processing • Composition (for magazines and movies), Display, Printing … • Transmission and storage • images from oversea via Internet, or from a remote planet • Information analysis and automated recognition • Providing “human vision” to machines Face detection in ’98 @ CMU CS, http://www.cs.cmu.edu/afs/cs/Web/People/har/faces.html

6. An Example • A Navigation system study for Accident Scene Analysis and Obstacle Avoidance Based on Mobile Multimedia sensor Platform

7. Image Processing • Medical imaging for diagnosis and exploration • Security, forensics and rights protection • Encryption, hashing, digital watermarking, digital fingerprinting … invisible Watermark Visible Watermark

8. DigitalImage • “Exactness” • Perfect reproduction without degradation • Perfect duplication of processing result • Convenient & powerful computer-aided processing • Can perform sophisticated processing through computer hardware or software • Even kindergartners can do some! • Easy storage and transmission • 1 CD can store hundreds of family photos! • Paperless transmission of high quality photos through network within seconds

9. Examples of Digital Image Processing • Compression • Manipulation and Restoration • Restoration of blurred and damaged images • Noise removal and reduction • Morphing

10. (a) original lenna image (b) corrupted lenna image (c) concealed lenna image 25% blocks in a checkerboard pattern are corrupted corrupted blocks are concealed via edge-directed interpolation Examples of Digital Image Processing • Applications • Face detection • Visible and invisible watermarking • Error concealment and resilience in video transmission

11. 2010 International Conf. on Image Processing

12. y x I(x,y) y x What is An Image? • Grayscale image • A grayscale image is a functionI(x,y) of the two spatial coordinates of the image plane. • I(x,y) is the intensity of the image at the point (x,y) on the image plane. • I(x,y) takes non-negative values • assume the image is bounded by a rectangle [0,a][0,b] • I: [0, a]  [0, b]  [0, inf ) • Color image • Can be represented by three functions, R(x,y) for red, G(x,y) for green, and B(x,y) for blue.

13. Matlab Example x=imread('lena.bmp'); imshow(x); [n,m]=size(x); x(5:40,5:40)=0; imshow(x) x(90:150, 90:150)=255; Imshow(x)

14. Different Ways to View an Image (More generally, to view a 2-D real-valued function) Intensity visualization over 2-D (x,y) plane In 3-D (x,y, z) plot with z=I(x,y);red color for high value and blue for low Equal value contour in (x,y) plane

15. 255 (white) 0 (black) Sampling and Quantization • Computer handles “discrete” data. • Sampling • Sample the value of the image at the nodes of a regular grid on the image plane. • A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j). • Quantization • Is a process of transforming a real valued sampled image to one taking only a finite number of distinct values. • Each sampled value in a 256-level grayscale image is represented by 8 bits.

16. Image Compression • Savings in storage and transmission • Multimedia data (esp. image and video) have large data volume • Difficult to send real-time uncompressed video over current network • Accommodate relatively slow storage devices • -they do not allow playing back uncompressed multimedia data in real time • 1x CD-ROM transfer rate ~ 150 kB/s • 320 x 240 x 24 fps color video bit rate ~ 5.5MB/s => 36 seconds needed to transfer 1-sec uncompressed video from CD

17. PCM coding • How to encode a digital image into bits? • Sampling and perform uniform quantization • “Pulse Coded Modulation” (PCM) • 8 bits per pixel ~ good for grayscaleimage/video • 10-12 bpp ~ needed for medical images • Reduce # of bpp for reasonable quality via quantization • Quantization reduces # of possible levels to encode • Visual quantization: companding, contrast quantization, dithering, etc. • Halftone use 1bpp but usually upsampling ~ saving less than 2:1

18. DiscussiononImprovingPCM • Quantized PCM values may not be equally likely • – Can we do better than encode each value using same # bits? • Example • P(“0” ) = 0.5, P(“1”) = 0.25, P(“2”) = 0.125, P(“3”) = 0.125 • If use same # bits for all values • Need 2 bits to represent the four possibilities if treat • If use less bits for likely values “0” ~ Variable Length Codes (VLC) • “0” => [0], “1” => [10], “2” => [110], “3” => [111] • Use 1.75 bits on average ~ saves 0.25 bpp! • Bring probability into the picture • Now use prob. distr. to reduce average # bits per quantized sample

19. Variable Length Code • If the probabilities of (a, b, c, d) were (1/2,1/4,1/8,1/8), the expected number of its used to represent a source symbol would be 1.7500 bits/per symbol:

20. Entropycoding • Idea. use less bits for commonly seen values • How many # bits needed? • Limit of compression => “Entropy” • Measures the uncertainty or amount of avg. information of a source • Definition: H = bits • e.g., entropy of previous example is 1.75 • Can’t represent a source perfectly with less than avg. H bits per sample • Can represent a source perfectly with avg. H+bits per sample ( Shannon Lossless Coding Theorem) • “Compressability” depends on the sources • Important to decode coded stream efficiently without ambiguity

21. E.g.ofEntropyCoding:HuffmanCoding • Variable length code • assigning aboutbits for the ith value • has to be integer# of bits per symbol • Step-1 • Arrange in decreasing order and consider them as tree leaves • Step-2 • Merge two nodes with smallest prob. to a new node and sum up prob. • Arbitrarily assign 1 and 0 to each pair of merging branch • Step-3 • Repeat until no more than one node left. • Read out codeword sequentially from root to leaf

22. HuffmanCoding(cont’d)

23. HuffmanCoding:Pros&Cons • Pro • Simplicity in implementation (table lookup) • For a given block size Huffman coding gives the best coding efficiency • Con • Need to obtain source statistics • Improvement • Code a group of symbols as a whole to allow fractional # bit per symbol • Arithmetic coding • Lempel-Ziv coding or LZW algorithm • “universal”, no need to estimate source statistics

24. Discussion: Coding a Binary Image • How to efficiently encode it? • – “ 0 0 0 0 1 1 0 0 0 1 0 1 0 0 0 0 0 0 1 1 1 …” • Run-length coding (RLC) • Code length of runs of “0” between successive “1” • run-length of “0” ~ # of “0” between “1” • good if often getting frequent large runs of “0” and sparse “1” • – E.g., => (4) (0) (3) (1) (6) (0) (0) … … • – Assign fixed-length codeword to run-length • Or use variable-length code like Huffman to further improve • RLC Also applicable to general binary data sequence with sparse “1” (or “0”)

25. Any Questions?

26. Things To Do Next Topics • Find your partners for the class project • Feb. 27, Thursday (email me the team information) • Each group has 4 members • Check out the class website about • lecture notes • reading assignments Image Compression-JPEG