Besov Bayes Chomsky Plato

1. Multiscale Geometric Image Analysis Besov BayesChomsky Plato Richard Baraniuk Rice University dsp.rice.edu Joint work with Hyeokho ChoiJustin RombergMike Wakin

2. Low-Level Image Structures • Smooth/texture regions • Edge singularities along smooth curves (geometry) geometry texture texture

3. Computational Harmonic Analysis • Representation Fourier sinusoids, Gabor functions, wavelets, curvelets, Laplacian pyramid coefficients basis, frame

4. Computational Harmonic Analysis • Representation • Analysis study through structure of might extract features of interest • Approximation uses just a few terms exploit sparsity of coefficients basis, frame

5. Multiscale Image Analysis • Analyze an image a multiple scales • How? Zoom out and record information lost in wavelet coefficients … info 1 info 2 info 3

6. Wavelet-based Image Processing • Standard 2-D tensor product wavelet transform

7. Transform-domain Modeling • Transform-domain modeling and processing transform coefficient model

8. Transform-domain Modeling • Transform-domain modeling and processing transformvocabulary coefficient model

9. Transform-domain Modeling • Transform-domain modeling and processing • Vocabulary + grammar capture image structure • Challenging co-design problem transformvocabulary coefficient grammar model

10. Nonlinear Image Modeling • Natural images do not form a linear space! • Form of the “set of natural images”? + =

11. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2

12. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2

13. Set of Natural Images Small: NxN sampled images comprise an extremely small subset of RN2 Complicated: Manifold structure RN2

14. Wedge Manifold

15. Wedge Manifold 3 Haar wavelets

16. Wedge Manifold 3 Haar wavelets

17. Stochastic Projections • Projection = “shadow”

18. Higher-D Stochastic Projections • Model (partial) wavelet coefficient joint statistics • Shadow more faithful to manifold

19. Manifold Projections …

20. Multiscale Grammars for Wavelet Transforms

21. Wavelet Statistical Properties

22. Marginal Distribution • many small S coefficients • smooth regions 0

23. Marginal Distribution • a fewlarge L coefficients • edgeregions 0

24. Wavelet Mixture Model state: S or L • wc: Gaussian withS or Lvariance

25. Magnitude Correlations • Persistenceof wc’s onquadtree • SS • LL

26. Wavelet Hidden Markov Tree A state: S or L • wc: Gaussian withS or Lvariance

27. Wavelet Statistical Models • Independent wc’s w/ generalized Gaussian marginal • processing: thresholding (“coring”) • application: denoising • Correlated wc magnitudes • zero tree image coder [Shapiro] • estimation/quantization (EQ) model [Ramchandran, Orchard] • JPEG2000 encoder • graphical models: • Gaussian scale mixture [Wainwright, Simoncelli] • hidden Markov process on quadtree (HMT) • processing: tree-structured algorithmsEM algorithm, Viterbi algorithm, … • applications: denoising, compression, classification, …

28. Barbara

29. Barbara’s Books

30. Denoising Barbara’s Books noisy books DWT HMT WT thresholding

31. Denoising Barb’s Books Wavelets and Subband Coding Vetterli and Kovacevic My Life as a DogParis Hilton Numerical Analysis of Wavelet Methods Albert Cohen

32. HMT Image Segmentation

33. HMT Image Segmentation

34. Zero Tree Compression • Idea: Prune wavelet subtrees in smooth regions Z • tree-structured thresholding • spend bits on (large) “significant” wc’s • spend no bits on (small) wc’s… • zerotree symbolZ = {wc and all decendants = 0}

35. New Multiscale Vocabularies

36. Geometrical info not explicit • Modulations around singularities (geometry) • Inefficient-large number of significant wc’s cluster around edge contours, no matter how smooth

37. Wavelet Modulations • Wavelets are poor edge detectors • Severe modulation effects

38. Wavelet Modulations • Wavelet wiggles…so its inner product with a singularity wiggles signal L scale S L S

39. Wavelet Modulation Effects • Wavelet transform of edge

40. Wavelet Modulation Effects • Wavelet transform of edge • Seek amplitude/envelope • To extract amplitude need coherent representation

41. 1-D Complex Wavelets [Grossman, Morlet, Lina, Abry, Flandrin, Mallat, Bernard, Kingsbury, Selesnick, Fernandes, …] • real waveleteven symmetryimaginary waveletodd symmetry • Hilbert transform pair(complex Gabor atom) • 2x redundant tight frame • Alias-free; shift-invariant • Coherent wavelet representation (magnitude/phase)

42. 2-D Complex Wavelets[Lina, Kingsbury, Selesnick, …] • 4x redundant tight frame • 6 directional subbandsaligned along 6 1-D manifold directions • Magnitude/phase • Even/odd real/imag symmetry • Almost Hilbert transform pair (complex Gabor atom) • Almost shift invariant • Compute using 1-D CWT -75 +75 +45 +15 -15 -45 real imag

43. Wavelet Image Processing

44. Coherent Wavelet Processing real part +i imaginarypart

45. Coherent Wavelet Processing |magnitude| x exp(iphase)

46. Coherent Image Processing [Lina] magnitude FFT

47. Coherent Image Processing [Lina] magnitude phase FFT

48. Coherent Image Processing [Lina] magnitude phase FFT CWT

49. Coherent Wavelet Processing feature magnitude phase 1 edge Lcoherent “speckle” Lincoherent > 1 edge smooth S undefined

50. Coherent Segmentation feature magnitude phase 1 edge Lcoherent “speckle” Lincoherent > 1 edge smooth S undefined