Understanding and evaluating blind deconvolution algorithms

Understanding and evaluating blind deconvolution algorithms Anat Levin1,2, Yair Weiss1,3, Fredo Durand1, Bill Freeman1,4 1MIT CSAIL, 2Weizmann Institute, 3Hebrew University, 4Adobe

Blind deconvolution ? ? kernel blurred image sharp image • Rich literature, no perfect solution • Fergus et al. 06, Levin 06, Jia 07, Joshi et al. 08, Shan et al. 08 • In this talk: • No new algorithm • What makes blind deconvolution hard? • Quantitatively evaluate recent algorithms on the same dataset

Blind deconvolution blurred image sharp image noise blur kernel Unknown, need to estimate Input (known) ? ?

Natural image priors Gaussian: -x2 Laplacian: -|x| -|x|0.5 Log prob -|x|0.25 x x Derivative histogram from a natural image Parametric models Derivative distributions in natural images are sparse:

Sparse priors in image processing • Denoising • Simoncelli et al., Roth&Black • Inpainting • Sapiro et al., Levin et al. • Super resolution • Tappen et al. • Transparency • Levin et al. • Demosaicing • Tappen et al., Hel-Or et al. • Non blind deconvolution • Levin et al.

Naïve MAPx,k estimation Find a kernel k and latent image x minimizing: Convolution constraint Sparse prior Should favor sharper x explanations

The MAPx,k paradox P( , )>P( , ) kernel kernel Latent image Latent image Claim 1: Let be an arbitrarily large image sampled from a sparse prior , and Then the delta explanation is favored

The MAPx,k failure sharp blurred ?

The MAPx,k failure 45x45 windows 25x25 windows 15x15 windows simple derivatives [-1,1],[-1;1] FoE filters (Roth&Black) Red windows = [ p(sharp x) >p(blurred x) ]

The MAPx,k failure - intuition > k=[0.5,0.5] P(step edge) P(blurred step edge) cheaper sum of derivatives: < P(impulse) P(blurred impulse) cheaper sum of derivatives:

Blur reduces derivative contrast < P(sharp real image) P(blurred real image) cheaper Real image row: Noise and texture behave as impulses - total derivative contrast reduced by blur

Why does MAPx,k fail? • Too few measurements? Fails even with infinitely large image • Wrong prior? Fails even for signals sampled from the prior • Choice of estimator

MAPk estimation MAPx,k-estimate x,k simultaneously argmaxP(,|) x k y MAPk-estimate k alone, marginalize x argmaxP(|) k y = P(,|)dx x k y

Results in this paper: Let be an arbitrarily large image sampled from a sparse prior , and Then Claim 1- MAPx,k estimator fails: The delta explanation is favored Claim 2- MAPk estimator succeeds: is maximized by the true kernel

Intuition: dimensionality asymmetry MAPx,k– Estimationunreliable. Number of measurements always lower than number of unknowns: #y<#x+#k MAPk– Estimation reliable. Many measurements for large images: #y>>#k kernel k sharp image x blurred image y Large, ~105 unknowns Small, ~102 unknowns ~105 measurements

Approximate MAPk strategies • Marginalization on x is challenging to compute • Approximation strategies: • Independence assumption in derivatives space: • Levin NIPS06 • Variational approximation: • Miskin and Mackay 00, Fergus et al.SIGGRAPH06 • Laplace approximation: • Brainard and Freeman 97, Bronstein et al. 05

Evaluation on 1D signals Exact MAPk MAPx,k Favors delta solution Favor correct solution despite wrong prior! MAPk, variational approximation (Fergus et al.) MAPk, Gaussian prior

Ground truth data acquisition 4 images x 8 kernels = 32 test images Data available online: http://www.wisdom.weizmann.ac.il/~levina/

Comparison Fergus et al. SIGGRAPH06 MAPk, variational approx. Ground truth MAPx,k MAPk, Gaussian prior Shan et al. SIGGRAPH08 adjusted MAPx,k

Evaluation Fergus, variational MAPk Shan et al. SIGGRAPH08 MAPx,k sparse prior MAPk, Gaussian prior 100 Successes percent 80 60 40 20 Cumulative histogram of deconvolution successes : bin r = #{ deconv error < r }

Problem: uniform blur assumption is unrealistic Variation of dot traces at 4 corners Note: opposite conclusion by Fergus et al., 2006

Summary • Good estimator is more important than correct prior: • - MAPk approach can do deconvolution even with Gaussian prior • - MAPx,k approach fails even with sparse prior • Spatially uniform blur assumption is invalid • Compare blind deconvolution algorithms on the same dataset, Fergus et al. 06 significantly outperforms all alternatives Ground truth data available online

Understanding and evaluating blind deconvolution algorithms