Image Processing and Analysis (ImagePandA) 5 – Image Restoration and Reconstruction

1. Image Processing and Analysis (ImagePandA) 5 – Image Restoration and Reconstruction Christoph Lampert / Chris Wojtan Based on slides by Selim Aksoy, Bilkent University TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.:

2. Image interpolation

3. Image interpolation

4. Image interpolation

5. Nearest neighbor interpolation

6. Image interpolation

7. Image interpolation

8. Nearest neighbor interpolation First, find the x and y values of the nearest pixels int[] px = new int[4]; int[] py = new int[4]; px[0] = Math.floor(x); py[0] = Math.floor(y); px[1] = Math.ceil(x); py[1] = Math.floor(y); px[2] = Math.floor(x); py[2] = Math.ceil(y); px[3] = Math.ceil(x); py[3] = Math.ceil(y);

9. Nearest neighbor interpolation Next, find the nearest pixel intnx, ny; // x,ycoords of nearest neighbor double minDist = Double.MAX_VALUE; for(inti=0; i<4; i++) { double dist = Math.abs(x-px[i])+Math.abs(y-py[i]); if(dist<minDist) { minDist = dist; nx=px[i]; ny=py[i]; } } Finally, assign the value of the nearest pixel pixelVal = image.get(nx,ny);

10. Nearest neighbor interpolation

11. Piecewise linear interpolation

12. Piecewise Linear interpolation 0 1

13. Piecewise linear interpolation

14. Piecewise cubic interpolation

15. Piecewise Cubic (Hermite) interpolation

16. 2D interpolation Nearest neighbor Bi-linear Bi-cubic

17. Resizing images How can we generate a half-sized version of a large image? Adapted from Steve Seitz, U of Washington

18. Resizing images 1/8 1/4 Throw away every other row and column to create a 1/2 size image (also called sub-sampling). Adapted from Steve Seitz, U of Washington

19. Resizing images 1/2 1/4 (2x zoom) 1/8 (4x zoom) Does this look nice? Adapted from Steve Seitz, U of Washington

20. Sampling and aliasing Adapted from Steve Seitz, U of Washington

21. Sampling and aliasing http://www.youtube.com/watch?v=ckmtG8CZjDA

22. Sampling and aliasing • Errors appear if we do not sample properly. • Common phenomenon: • High spatial frequency components of the image appear as low spatial frequency components. • Examples: • Wagon wheels rolling the wrong way in movies. • Checkerboards misrepresented in ray tracing. • Striped shirts look funny on color television.

23. Resizing images 1/8 1/4 Throw away every other row and column…… regular sampling of a high-frequency function! Adapted from Steve Seitz, U of Washington

24. Resizing images ALIASING 1/2 1/4 (2x zoom) 1/8 (4x zoom) Does this look nice? Adapted from Steve Seitz, U of Washington

25. Resizing images • Regular sampling of a high-frequency image causes aliasing! • Solution: smooth the image (remove high frequencies) first! Gaussian 1/8 Gaussian 1/4 Gaussian 1/2 Adapted from Steve Seitz, U of Washington

26. Resizing images Gaussian 1/2 Gaussian 1/4 (2x zoom) Gaussian 1/8 (4x zoom) Adapted from Steve Seitz, U of Washington

27. Gaussian pyramids Adapted from Michael Black, Brown University

28. Gaussian pyramids Adapted from Michael Black, Brown University

29. De-noising

30. Removing Noise

31. Inverting the Degradation Function

32. Inverting the Degradation Function PSF Original motion-blurred image Clean image of The Beehive Nebula http://www.sweaglesw.com/cs448/

33. Inverting the Degradation Function

34. Inverting the Degradation Function

35. Texture Synthesis • Incrementally growing pixels • Given an incomplete neighborhood • Find similar neighborhoods from the example, copy pixels ?

36. Texture Synthesis • Incrementally growing pixels • Given an incomplete neighborhood • Find similar neighborhoods from the example, copy pixels

37. Texture Synthesis • Incrementally growing pixels • Given an incomplete neighborhood • Find similar neighborhoods from the example, copy pixels • Image Quilting • Cut up the example into patches (not just pixels) • Choose patches which line up the best • Stitch the patches together along optimal seems

38. Texture Synthesis • Incrementally growing pixels • Given an incomplete neighborhood • Find similar neighborhoods from the example, copy pixels • Image Quilting • Cut up the example into patches (not just pixels) • Choose patches which line up the best • Stitch the patches together along optimal seems • Can use this to fill holes in an image • Instead of some example, use the image itself • http://www.youtube.com/watch?v=NH0aEp1oDOI

39. Super-resolution • How do we increase image size? • Interpolation? • Need to fill in missing details EMBIGGEN INTERPOLATION

40. Super-resolution • How do we increase image size? • Interpolation? • Need to fill in missing details • Combine multiple images at sub-pixel off-sets • Synthesize details from examples • Steal details from other images • Copy details from same image (at different scales) • http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html