500 likes | 505 Views
Statistical Performance Analysis by Loopy Belief Propagation in Bayesian Image Modeling. Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University, Japan http://www.smapip.is.tohoku.ac.jp/~kazu/. Collaborators D. M. Titterington (University of Glasgow, UK)
E N D
Statistical Performance Analysis byLoopy Belief Propagation in Bayesian Image Modeling Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University, Japan http://www.smapip.is.tohoku.ac.jp/~kazu/ Collaborators D. M. Titterington (University of Glasgow, UK) M. Yasuda (Tohoku University, Japan) S. Kataoka (Tohoku University, Japan) IW-SMI2010 (Kyoto)
MRF, Belief Propagation and Statistical Performance • Geman and Geman (1986): IEEE Transactions on PAMI • Image Processing by Markov Random Fields (MRF) • Tanaka and Morita (1995): Physics Letters A • Cluster Variation Method for MRF in Image Processing • CVM= Generalized Belief Propagation (GBP) • Nishimori and Wong (1999): Physical Review E • Statistical Performance Estimation for MRF • (Infinite Range Ising Model and Replica Theory) Is it possible to estimate the performance of belief propagation statistically? IW-SMI2010 (Kyoto)
Outline • Bayesian Image Analysis by Gauss Markov Random Fields • Statistical Performance Analysis for Gauss Markov Random Fields • Statistical Performance Analysis in Binary Markov Random Fields by Loopy Belief Propagation • Statistical Analysis of Trajectory in EM algorithm in Bayesian Image Analysis • Concluding Remarks IW-SMI2010 (Kyoto)
Image Restoration by Bayesian Statistics Original Image IW-SMI2010 (Kyoto)
Image Restoration by Bayesian Statistics Noise Transmission Original Image Degraded Image IW-SMI2010 (Kyoto)
Image Restoration by Bayesian Statistics Noise Transmission Original Image Degraded Image Estimate IW-SMI2010 (Kyoto)
Image Restoration by Bayesian Statistics Noise Transmission Posterior Original Image Degraded Image Estimate Bayes Formula 11 March, 2010 7 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)
Image Restoration by Bayesian Statistics Noise Transmission Posterior Original Image Degraded Image Estimate Assumption 1: Original images are randomly generated by according to a prior probability. Bayes Formula 11 March, 2010 8 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)
Image Restoration by Bayesian Statistics Noise Assumption 2: Degraded images are randomly generated from the original image by according to a conditional probability of degradation process. Transmission Posterior Original Image Degraded Image Estimate Bayes Formula 11 March, 2010 9 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)
Assumption: Prior Probability consists of a product of functions defined on the neighbouring pixels. Bayesian Image Analysis Prior Probability IW-SMI2010 (Kyoto)
Bayesian Image Analysis Assumption: Prior Probability consists of a product of functions defined on the neighbouring pixels. Prior Probability 11 March, 2010 11 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)
Bayesian Image Analysis Assumption: Prior Probability consists of a product of functions defined on the neighbouring pixels. Prior Probability Patterns by MCMC. 11 March, 2010 12 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto)
Bayesian Image Analysis Assumption: Degraded image is generated from the original image by Additive White Gaussian Noise. V:Set of all the pixels IW-SMI2010 (Kyoto)
Bayesian Image Analysis Assumption: Degraded image is generated from the original image by Additive White Gaussian Noise. 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 14
Bayesian Image Analysis Assumption: Degraded image is generated from the original image by Additive White Gaussian Noise. 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 15
Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate IW-SMI2010 (Kyoto)
Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate Data Dominant Smoothing 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 17
Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate Bayesian Network Data Dominant Smoothing 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 18
Bayesian Image Analysis Degraded Image Original Image Degradation Process Prior Probability Posterior Probability Estimate Bayesian Network Data Dominant Smoothing 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 19
Image Restorations by Gaussian Markov Random Fields and Conventional Filters Original Image Degraded Image Restored Image V: Set of all the pixels Gauss Markov Random Field (3x3) Lowpass (5x5) Median IW-SMI2010 (Kyoto)
Statistical Performance by Sample Average of Numerical Experiments Original Images IW-SMI2010 (Kyoto)
Statistical Performance by Sample Average of Numerical Experiments Noise Pr{G|F=f,s} Original Images Observed Data 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 22
Statistical Performance by Sample Average of Numerical Experiments Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images Observed Data Estimated Results 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 23
Statistical Performance by Sample Average of Numerical Experiments Sample Average of Mean Square Error Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images Observed Data Estimated Results 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 24
Statistical Performance Estimation Original Image Degraded Image Additive White Gaussian Noise Restored Image Posterior Probability Additive White Gaussian Noise IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Gauss Markov Random Fields s=40 s=40 a a IW-SMI2010 (Kyoto)
= > = Statistical Performance Estimation for Binary Markov Random Fields Light intensities of the original image can be regarded as spin states of ferromagnetic system. = > = Free Energy of Ising Model with Random External Fields It can be reduced to the calculation of the average of free energy with respect to locally non-uniform external fields g1, g2,…,g|V|. IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Binary Markov Random Fields IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Markov Random Fields Spin Glass Theory in Statistical Mechanics Loopy Belief Propagation Multi-dimensional Gauss Integral Formulas 0.8 0.6 0.4 s=40 0.2 s=1 a a IW-SMI2010 (Kyoto)
Image Restoration of Loopy Belief Propagation a*=0.465 IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Markov Random Fields IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Binary Markov Random Fields 11 March, 2010 IW-SMI2010 (Kyoto) 33
Statistical Performance Estimation for Binary Markov Random Fields Original Images 11 March, 2010 IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 35
Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 36 Estimated Results
Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 37 Estimated Results
Statistical Performance Estimation for Binary Markov Random Fields Noise Pr{G|F=f,s} Posterior Probability Pr{F|G=g,a,s} Original Images 11 March, 2010 IW-SMI2010 (Kyoto) 38 Estimated Results
Statistical Performance Estimation for Binary Markov Random Fields Loopy Belief Propagation IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Binary Markov Random Fields Loopy Belief Propagation 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 40
Statistical Performance Estimation for Binary Markov Random Fields Loopy Belief Propagation Loopy Belief Propagation 11 March, 2010 IW-SMI2010 (Kyoto) IW-SMI2010 (Kyoto) 41
Statistical Performance Estimation for Markov Random Fields IW-SMI2010 (Kyoto)
Statistical Performance Estimation for Markov Random Fields IW-SMI2010 (Kyoto)
Marginal Likelihood Maximization of Marginal Likelihood by EM Algorithm Q-Function EM Algorithm IW-SMI2010 (Kyoto)
EM Algorithm for Gauss Markov Random Fields Exact Generalized Belief Propagation Loopy Belief Propagation IW-SMI2010 (Kyoto)
EM Algorithm for Gauss Markov Random Fields Numerical Experiments for Standard Image Statistical Behaviour of EM Algorithm IW-SMI2010 (Kyoto)
0.8 0.6 EM Algorithm for Binary Markov Random Fields 0.4 0.2 Numerical Experiments for Snapshot Images 1 2 3 4 5 0.8 0.6 0.4 0.2 Statistical Behaviour of EM Algorithm 1 2 3 4 5 IW-SMI2010 (Kyoto)
Summary • Formulation of probabilistic model for image processing by means of conventional statistical schemes has been summarized. • Statistical performance analysis of probabilistic image processing by using Gauss Markov Random Fields has been shown. • One of extensions of statistical performance estimation to probabilistic image processing with discretestates has been demonstrated. IW-SMI2010 (Kyoto)
References • K. Tanaka and D. M. Titterington: Statistical Trajectory of Approximate EM Algorithm for Probabilistic Image Processing, Journal of Physics A: Mathematical and Theoretical, vol.40, no.37, pp.11285-11300, 2007. • M. Yasuda and K. Tanaka: The Mathematical Structure of the Approximate Linear Response Relation, Journal of Physics A: Mathematical and Theoretical, vol.40, no.33, pp.9993-10007, 2007. • K. Tanaka and K. Tsuda: A Quantum-Statistical-Mechanical Extension of Gaussian Mixture Model, Journal of Physics: Conference Series, vol.95, article no.012023, pp.1-9, January 2008 • K. Tanaka: Mathematical Structures of Loopy Belief Propagation and Cluster Variation Method, Journal of Physics: Conference Series, vol.143, article no.012023, pp.1-18, 2009 • M. Yasuda and K. Tanaka: Approximate Learning Algorithm in Boltzmann Machines, Neural Computation, vol.21, no.11, pp.3130-3178, 2009. • S. Kataoka, M. Yasuda and K. Tanaka: Statistical Performance Analysis in Probabilistic Image Processing, Journal of the Physical Society of Japan, vol.79, no.2, article no.025001, 2010. IW-SMI2010 (Kyoto)
Correlation for Prior Probability IW-SMI2010 (Kyoto)