1 / 39

Lecture 7

Lecture 7. Patch based methods: nonlocal means, BM3D, K-SVD, data-driven (tight) frame. Outline. Nonlocal methods Nonlocal means BM3D Dictionary learning methods K-SVD Data-driven tight frame Low rank models. Nonlocal Methods. Local means and BM3D.

jjimenez
Download Presentation

Lecture 7

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 7 Patch based methods: nonlocal means, BM3D, K-SVD, data-driven (tight) frame

  2. Outline • Nonlocal methods • Nonlocal means • BM3D • Dictionary learning methods • K-SVD • Data-driven tight frame • Low rank models

  3. Nonlocal Methods Local means and BM3D

  4. Denoising: Gaussian Smoothing Revisited *h = * *h *h h

  5. Perona-Malik: Anisotropic Filtering • Edges smooth only along edges • “Smooth” regions smooth isotropically gradient

  6. Ideal: Spatially Adaptive Smoothing • Non uniform smoothingDepending on image content: • Smooth where possible • Preserve fine details How? *h3 *h2 *h1

  7. Bilateral Filtering (Smith and Brady, 1997) • Formula

  8. Slides taken from Sylvain Paris, Siggraph 2007 Gaussian Smoothing * output input * * Same Gaussian kernel everywhere Averages across edges blur

  9. Slides taken from Sylvain Paris, Siggraph 2007 Bilateral Filtering * output input * * Kernel shape depends on image content Avoids averaging across edges

  10. Nonlocal Means (NLM): Motivation • Assume a static scene • Consider multiple images at different times • The signal remains constant • varies over time with zero mean

  11. Nonlocal Means (NLM): Motivation Average multiple images over time

  12. Nonlocal Means (NLM): Motivation Average multiple images over time

  13. Nonlocal Means (NLM): Motivation Average multiple images over time

  14. Redundancy in natural images Glasneret al. (2009)

  15. Single image “time-like” denoising Unfortunately, patches are not exactly the same simple averaging just won’t work

  16. NLM • Buades, Cole and Morel (2005) Use a weighted average based on similarity

  17. From Bilateral Filters to NLM Patch similarity

  18. Performance Evaluation Gaussian Smoothing Anisotropic Filtering Bilateral Filtering NLM Windowed Weiner Hard WT Soft WT Buadeset al. (2005)

  19. Variational Formulation of NLM • Gilboa and Osher(2008) • Nonlocal (partial) derivative • Nonlocal gradient： • Inner product: • With such inner product, we can define divergence • And Laplacian

  20. Variational Formulation of NLM • Gilboa and Osher (2008) • Nonlocal TV • Nonlocal ROF • Nonlocal TV-

  21. Variational Formulation of NLM • Gilboa and Osher (2008)

  22. Application in Surface Denoising • Dong et al. (2008)

  23. Application in Surface Denoising • Dong et al. (2008)

  24. Application in 4D CT Reconstruction • Tian et al. (2011)

  25. What’s Next? • The idea of grouping sounds good reduces mixing • Denoise = “extract the common (the signal)” • NLM: common = weighted average • Can a sparser representation do better?

  26. BM3D: Dabov et al. (2007) • Block Matching 3D collaborative filtering • Group patches with similar local structure (BM) • Jointly denoise each group (3D) • Smart Fusion of multiple estimates

  27. BM3D: Dabov et al. (2007) Single BM3D Estimate Block matching Inverse 3D transform Filter / thresholding R R R 3D grouping Denoised 3D group 3D transform

  28. BM3D: Dabov et al. (2007) • For every noisy reference block: • Calculate SSD between noisy blocks • If SSD<thr add to group

  29. BM3D: Dabov et al. (2007) • 3D transform 3D transform Image Patch Domain Sparse Domain

  30. BM3D: Dabov et al. (2007) • Collaborative filtering • Use hard thresholding or Wiener filter • Each patch in the group gets a denoised estimate • Unlike NLM – where only central pixel in reference patch got an estimate Filter / Thresholding Noisy patches Denoised Patches R R

  31. BM3D: Dabov et al. (2007) Collaborative filtering R R thr thr Multiple BM3D Estimate R R R R R Collaborative filtering R

  32. BM3D: Dabov et al. (2007) R R t t ? Fusion R R R R R R

  33. BM3D: Dabov et al. (2007) • Each pixel gets multiple estimates from different groups • Naive approach Average all estimates of each pixel …. not all estimates are as good • SuggestionGive higher weight to more reliable estimates

  34. BM3D: Dabov et al. (2007) • Give each estimate a weight according to denoising quality of its group • Quality = Sparsity induced by the denoising Hard thresholding Weiner filtering

  35. BM3D: Dabov et al. (2007) • Noise may result in poor matching Degrades de-noising performance • Improvements: • Match using a smoothed version of the image • Perform BM3D in 2 phases: • Basic BM3D estimate improved 3D groups • Final BM3D • Variational formulation: BM3D frame Danielyan, Katkovnikand Egiazarian. BM3D frames and variational image deblurring. IEEE TIP. 21(4):1715-28. 2012.

  36. BM3D: Dabov et al. (2007) • Results

  37. Dictionary Learning K-SVD, Data-Driven Tight Frame

  38. Sparse Coding and Dictionary Learning

  39. Dictionary Selection • Which D to use? • A fixed set of basis: • Steerable wavelet • Contourlet • DCT Basis • …… • Data adaptive dictionary – learn from data • K-SVD (-norm) http://bicmr.pku.edu.cn/~dongbin/Teaching_files/%E5%9B%BE%E5%83%8F%E5%A4%84%E7%90%86%E4%B8%AD%E7%9A%84%E6%95%B0%E5%AD%A6%E6%96%B9%E6%B3%95-18-19/ClassMaterials/K-SVD.pptx • Data-Driven Tight Frame (DDTF) http://bicmr.pku.edu.cn/~dongbin/Teaching_files/%E5%9B%BE%E5%83%8F%E5%A4%84%E7%90%86%E4%B8%AD%E7%9A%84%E6%95%B0%E5%AD%A6%E6%96%B9%E6%B3%95-18-19/ClassMaterials/DDTF.pdf

More Related