Wedgelets: A Multiscale Geometric Representation for Images

1. Wedgelets: AMultiscaleGeometricRepresentation for Images Michael Wakin Rice University dsp.rice.edu Joint work with Richard Baraniuk, Hyeokho Choi, Justin Romberg

2. Computational Harmonic Analysis • Representation • Analysis study through structure of should extract features of interest • Approximation exploit sparsity of coefficients basis, frame

3. Nonlinear Approximation few big sorted index many small

4. Nonlinear Approximation few big sorted index • -term approximation: use largestindependently • Greedy / thresholding

5. Error Approximation Rates • Optimize asymptotic error decay rate as

6. 1-D Piecewise Smooth Signals • smooth except for singularities at a finite number of 0-D points Fourier sinusoids: suboptimal greedy approximation and extraction wavelets: optimal greedy approximation extract singularity structure

7. 2-D Piecewise Smooth Signals • smooth except for singularities along a finite number of smooth 1-D curves • Challenge: analyze/approximategeometric structure geometry texture texture

8. Wavelet-based Image Processing • Standard 2-D tensor product wavelet transform

9. Wavelet Challenges • Current wavelet methods suboptimal for 2-d piecewise smooth functions • Geometrical info not explicit • Inefficient -large number of significant WCs cluster around edge contours, no matter how smooth

10. Wavelets and Cartoons 13 26 52 • Even for a smooth C2 contour, which straightens at fine scales…

11. 2-D Wavelets: Poor Approximation 13 26 52 • Even for a smooth C2 contour, which straightens at fine scales… • Too many wavelets required! -term wavelet approximation not

12. Response: A Pursuit of Sparsity I. Anisotropic, Directional Frames • curvelets • contourlets • bandelets • etc…

13. Response: A Pursuit of Sparsity I. Anisotropic, Directional Frames • curvelets • contourlets • bandelets • etc… II. Geometric Tilings / Dyadic Partitions • wedgelets • beamlets • platelets • etc…

14. Geometry Model for Cartoons C2 boundary • Toy model: flat regions separated by smooth contours • Goal: representation that is • sparse • simply modeled • efficiently computed • extensible to texture regions (RichB -> Saturday)

15. 2-D Dyadic Partition • Multiscale analysis • Zoom in by factor of 2 each scale • Represent image on each block • Partition/tiling not a basis/frame

16. 2-D Dyadic Partition = Quadtree • Each parent node has 4 children at next finer scale • Pruning reflects partitioning • Leaf nodes decorated with atoms

17. Wedgelet Representation [Donoho]  r • Build a cartoon using wedgelets on dyadic squares

18. Wedgelet Representation  r • Build a cartoon using wedgelets on dyadic squares • Quad-tree structuredeeper in tree finer curve approximation • Decorate leaves with (r,q) parameters

19. Wedgelet Representation • Prune wedgelet quadtree to approximate local geometry (adaptive) • New problem: How to find the proper approximation?

20. Wedgelet Inference • Find representation / prune tree to balance a fidelity vs. complexity trade-off

21. Wedgelet Inference • Find representation / prune tree to balance a fidelity vs. complexity trade-off • For Comp(W) –measure the complexity of the wedgelet representation(quadtree + (r,q)) • Donoho: Comp(W) = #leaves NLA-like solution

22. Wedgelet Inference • Find representation / prune tree to balance a fidelity vs. complexity trade-off • Exponential computational cost to solve brute force • Dynamic programming to the rescue!

23. distortion complexity

24. distortion Each leaf isindependent! cost

25. Decision: Keep parent or split into 4 children?

26. Top-Down: Greedy 3 3 1 1 2 1 1 3 4 4 1 • Label each node of the tree with • Move down from tree root and prune when 15 15 6 4 6 4 3

27. Bottom-Up: Dynamic Programming 1 1 3 2 1 3 1 4 1 4 3 2 3 2 3 1 1 1 • Label each node of the tree with • Move up from the leaves and prune whenelse 15 15 6 4 6 4 3 3

28. Bottom-Up: Dynamic Programming 1 3 1 1 3 1 4 1 3 4 1 1 3 2 2 • Label each node of the tree with • Move up from the leaves and prune whenelse 15 15 6 4 5 4 3

29. Bottom-Up: Dynamic Programming 1 3 1 1 3 1 4 1 3 4 1 1 3 2 2 • Label each node of the tree with • Move up from the leaves and prune whenelse 15 9 6 4 5 4 3

30. Summary: Dynamic Programming • Simple recursive bottom-up algorithm • Efficient: O(N) for N-node tree • Finds optimal solution provided cost is additivewith independent components

31. Wedgelet Inference • Find representation / prune tree to balance a fidelity vs. complexity trade-off • Optimal approximation • Near optimal rate distortion decay • Near optimal estimation [Donoho]

32. #Leaves Complexity Penalty • Accounts for wedgelet partition size, but not wedgelet orientation #leaves #leaves “smooth” “simple” “rough” “complex”

33. Multiscale Geometry Model (MGM)

34. Multiscale Geometry Model (MGM) • Decorate each tree node with orientation (r,q) and then model dependencies thru scale

35. Multiscale Geometry Model (MGM) • Decorate each tree node with orientation (r,q) and then model dependencies thru scale • Insight: Smooth curve Geometric innovations small at fine scales • Model: Favor small innovations over large innovations (statistically)

36. Multiscale Geometry Model (MGM) • Wavelet-like geometry model: coarse-to-fine prediction • model parent-to-child transitions of orientations small innovations large

37. MGM • Wavelet-like geometry model: coarse-to-fine prediction • model parent-to-child transitions of orientations • Markov-1 statistical model • state = (r,q) orientation of wedgelet • parent-to-child state transition matrix

38. MGM • Markov-1 statistical model • Joint wedgelet Markov probability model: • Complexity = Shannon codelength == number of bits to encode

39. MGM and Edge Smoothness “smooth” “simple” “rough” “complex”

40. MGM Inference • Find representation / prune tree to balance the fidelity vs. complexity trade-off • Efficient solution via dynamic programming

41. Wedgelet Coding of Cartoon Images • Choosing wedgelets = rate-distortion optimization Shannon code length to encode

42. Predictive Wedgelet Coding C2contour (1-d) Optimal rate-distortion performance compared to for leaf-only encoding

43. Proof: Start with leaf-encoding Consider a wedgelet at scale j Block size is 2-jx2-j O(2-2j) 2-j C2 curve contained in strip with width ~ length2

44. Choose discrete wedgelet dictionary with tick marks of spacing O(2-2j) O(2-2j) 2-j

45. Discrete Wedgelet Dictionary

46. A discrete wedgelet exists that deviates from curve by only O(2-2j); L2 distortion on this block = O(2-3j) O(2-2j) 2-j Since curve is C2, there are O(2j) such blocks Total distortion D~O(2-2j)

47. Number of bits (bitrate) to encode tick marks = O(j) bits per block; thus, total rate R ~ O(j2j) O(2-2j) 2-j Combining: for leaf-only encoding

48. Can reduce bitrate by predicting wedgelets down through scale (MGM); consider scale j+1… O(2-2(j+1)) 2-(j+1) 2-(j+1)

49. Finer scale wedgelets within O(2-2(j+1)) of curve Coarse scale wedgelet within O(2-2j) of curve O(2-2(j+1)) 2-(j+1) 2-(j+1) Given the parent, there are only O(1) possible wedgelets on each finer block. Thus, rate is O(1) bits per block, or R~O(2j) bits total. This gives the optimal D(R)~R-2.

50. Summary: Wedgelets • Adaptive dyadic partition • Fit to data using fidelity/complexity optimization • Improve using MGM statistical model for parent-to-child wedgelet orientation correlations • MGM parent-to-child modeling akin to “wavelets” for wedgelets • Optimal approximation and R/D rates [Dynamic programming as a useful tool] • Only suited for cartoon images