Besov Bayes Chomsky Plato

1. Multiscale Geometric Image Processing Besov BayesChomsky Plato Richard Baraniuk Rice University dsp.rice.edu Joint work with Hyeokho ChoiJustin RombergMike Wakin

2. Image Processing • Analyzing, modeling, processingimages • special class of 2-d functions • ex: digital photos • Problems: • representation, approximation, compression, estimation, restoration, deconvolution, detection, classification, segmentation, …

3. Low-Level Image Structures • Prevalent image structures: • smooth regions - grayscale regularity • smooth edge contours - geometric regularity • Must exploit both for maximum performance

4. Lossy Image Compression • Given image approximate using bits • Error incurred = distortionExample: squared error

5. Rate-Distortion Analysis as • Asymptotic analysis; maximize rate achievable region

6. Transform Coding • Quantize coefficients of an image expansion coefficients basis, frame Ex: Fourier sinusoids DCT (JPEG) wavelets curvelets …

7. Transform Coding • Quantize coefficients of an image expansion coefficients basis, frame quantize to total bits

8. Wet Paint Test Image

9. JPEG Compressed (DCT) PSNR = 26.32dB @ 0.0103bpp

10. MultiscaleImage Processing

11. Multiscale Image Analysis • Analyze an image a multiple scales • How? Zoom out and record information lost in wavelet coefficients … info 1 info 2 info 3

12. Wavelet-based Image Processing • Standard 2-D tensor product wavelet transform

13. Transform-domain Modeling • Transform-domain modeling and processing transform coefficient model

14. Transform-domain Modeling • Transform-domain modeling and processing transformvocabulary coefficient model

15. Transform-domain Modeling • Transform-domain modeling and processing • Vocabulary + grammar capture image structure • Challenging co-design problem transformvocabulary coefficient grammar model

16. Nonlinear Image Modeling • Natural images do not form a linear space! • Form of the “set of natural images”? + =

17. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2

18. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2

19. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2 Complicated: “Manifold” structure RN2

20. Wedge Manifold

21. Wedge Manifold 3 Haar wavelets

22. Wedge Manifold 3 Haar wavelets

23. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2 Complicated: “Manifold” structure RN2

24. Stochastic Projections • Projection = “shadow”

25. Higher-D Stochastic Projections • Model (partial) wavelet coefficient joint statistics • Shadow more faithful to manifold

26. Manifold Projections …

27. Wavelet-based Image Processing • Standard 2-D tensor product wavelet transform

28. Wavelet Coefficient Sparsity • Most

29. Wavelet Quadtree Structure • Wavelet analysis is self-similar • Image parent square divides into 4 children squares at next finer scale

30. Wavelet Quadtree Correlations • Smooth regionsmallwc’s tumble down tree • Edge/ridge regionlarge wc’s tumble down tree

31. Zero Tree Compression • Idea: Prune wavelet subtrees in smooth regions Z • tree-structured thresholding • spend bits on (large) “significant” wc’s • spend no bits on (small) wc’s… • zerotree symbolZ = {wc and all decendants = 0}

32. Zero Tree Compression • Label pruned wavelet quadtree with 2 stateszero-tree- smooth region (prune)significant- edge/texture region (keep) S smooth smooth Z Z S not smooth Z smooth S S S S

33. Zero Tree Compression • Label pruned wavelet quadtree with 2 stateszero-tree- smooth region (prune)significant- edge/texture region (keep) • Can optimize placement of Z and S with dynamic programming (SFQ) [Xiong, Ramchandran, Orchard] S smooth smooth Z Z S not smooth Z smooth S S S S

34. Wet Paint Test Image

35. JPEG Compressed (DCT) PSNR = 26.32dB @ 0.0103bpp

36. JPEG2000 Compressed (Wavelets) PSNR = 29.77dB @ 0.0103bpp

37. SFQ Compressed (Wavelets) PSNR = 29.77dB @ 0.0103bpp

38. SFQ WC Code Map green = significantblue = zero tree

39. SFQ Zoom

40. The JPEG2000 Disappointment • JPEG2000: replace DCT with wavelet transform • Does not improve on decay rate of JPEG! • WHY?neither DCT norwavelets are theright transform JPEG (DCT) JPEG2000 (wavelets)

41. Wavelet Challenges • Geometrical info not explicit • Inefficient-large number of large wc’s cluster around edge contours, no matter how smooth

42. Wavelets and Cartoons 13 26 52 • Even for a smooth C2 contour, which straightens at fine scales…

43. 2-D Wavelets: Poor Approximation 13 26 52 • Even for a smooth C2 contour, which straightens at fine scales… • Too many wavelets required! -term wavelet approximation not

44. 2-D Wavelets: Poor Approximation 13 26 52 • Even for a smooth C2 contour, which straightens at fine scales… • Too many wavelets required! -bit wavelet approximation not

45. Solution 1: Upgrade the Transform 13 26 52 • Introduce anisotropic transform • curvelets, ridgelets, contourlets, … • Optimal error decay rates for some simple images

46. Solution 2: Upgrade the Processing 13 26 52 • Replace coefficient thresholding by a new wc modelthat captures anisotropicspatial correlations

47. Our Goal: New Image Models • Wavelet coefficient models that capture both geometric and textural aspects of natural images • Optimal error decay rates for some image class

48. Cartoon Processing

49. Geometry Model for Cartoons C2 boundary • Toy model: flat regions separated by smooth contours • Goal: representation that is • sparse • simply modeled • efficiently computed • extensible to texture regions

50. 2-D Dyadic Partition • Multiscale analysis • Partition; not a basis/frame • Zoom in by factor of 2 each scale