Inverse Halftoning via Nonlocal Regularization

1. Inverse Halftoning via Nonlocal Regularization Xin Li West Virginia University This work is partially supported by NSF CCF-0914353

2. What is Inverse Halftoning? halftoning X: continuous-tone original inverse halftoning Y: halftoned (B/W) ^ X: continuous-tone estimated

3. Evolutionary Path of Inverse Halftoning Inverse Halftoning Inverse Problems Image Restoration Regularization theory Image Prior

4. What is State-of-the-Art

5. A Tantalizing Hypothesis Wavelet-based (Xiong et al.) Gradient-based (Kite et al.) LUT-based (Mese et al.) Hybrid LMS-MMSE (Chang et al.) Iterative filtering-based (Wong) Are they fundamentally equivalent? – all based on the local models (singularities in images are characterized by local intensity variations).

6. Hierarchy of Mathematical Spaces Metric space: a set with a notion of distance Hilbert-space: a complete Inner-product space General relativity Fixed-point theorems Game theory Dynamic systems Quantum mechanics Fourier/wavelet analysis Learning theory PDE(e.g., Total-Variation) Mathematical constructivism (Poincare, Brouwer, Weyl …) Mathematical formalism (Hilbert, Ackermann, Von Neumann …)

7. Filtering as Projection • Examples • Linear filtering (low-pass vs. high-pass) • Nonlinear filtering/diffusion • Bilateral filtering • Wavelet/DCT shrinkage • Nonlocal filtering (BM3D, nonlocal TV)

8. “Phase Space” of Image Signals SA-DCT TV BM3D Nonlocal-TV Nonlocal filters Local filters

9. Alternating Projections C1 X1 X∞ X0 X2 C2 Projection-Onto-Convex-Set (POCS) Theorem: If C1,…,Ck are convex sets, then alternating projection P1,…,Pk will converge to the intersection of C1,…,Ck if it is not empty C1 : observation constraint set C2 : regularization constraint set

10. Graduated Nonconvexity (GNC) temperature of deterministic annealing  threshold or Lagrangian multiplier

11. Summary of Algorithm • Key messages: • From local to nonlocalregularization thanks to the fixed-point formulation in the metric space (PNLF depends on the clustering result or similarity matrix) • From convex to nonconvexoptimization: deterministic annealing (also-called graduated nonconvexity) is the ``black magic” behind

12. Experimental Results “o” – lena “+” – peppers MATLAB codes accompanying this work are available From my homepage: http://www.csee.wvu.edu/~xinl/

13. Image Comparison Results (I) TV-based (PSNR=30.91dB) This work (PSNR=32.90dB) wavelet-based (PSNR=31.95dB) TV-based (PSNR=30.92dB) This work (PSNR=32.64dB) wavelet-based (PSNR=31.03dB)

18. Beyond Inverse Halftoning • Image denoising • W. Dong, X. Li, L. Zhang and G. Shi, "Sparsity-based image denoising via dictionary learning and structural clustering" , CVPR'2011 (oral paper), June 2011 • Image deblurring • Xin Li , "Fine-Granularity and Spatially-Adaptive Regularization for Projection-based Image Deblurring,"IEEE Trans. on Image Processing, Vol. 20, No. 4., pp. 971-983, Apr. 2011. • Weisheng Dong, Xin Li, Lei Zhang, and Guangming Shi, “Sparsity-based image deblurring with locally adaptive and nonlocally robust regularization,” accept to Proc. IEEE International Conference on Image Processing (ICIP), 2011 • Image coding • X. Li, "Collective sensing: a fixed-point approach in the metric space," SPIE Conf. on Visual Comm. and Image. Proc. (VCIP), Jul. 2010 • Super-resolution • Weisheng Dong, Guangming Shi, Lei Zhang, and Xiaolin Wu, “Super-resolution with nonlocal regularized sparse representation,” in Proc. SPIE Visual Communications and Image Processing (VCIP), July 2010 • Compressed sensing • X. Li, “The magic of nonlocaPerona-Malik diffusion”, IEEE Signal Processing Letter, vol. 18, no. 9, pp. 533-534, Sep. 2011 Source code collection for reproducible research http://www.csee.wvu.edu/~xinl/source.html