Physical Fluctuomatics 11th Probabilistic image processing by means of physical models

1. Physical Fluctuomatics11th Probabilistic image processing by means of physical models Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ Physical Fluctuomatics (Tohoku University)

2. Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 7. • Kazuyuki Tanaka: Statistical-mechanical approach to image processing (Topical Review), Journal of Physics A: Mathematical and General, vol.35, no.37, pp.R81-R150, 2002. • Muneki Yasuda, Shun Kataoka and Kazuyuki Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp.137-179, Advanced Communication Media CO., Ltd., December 2010 (in Japanese). Textbooks Physical Fluctuomatics (Tohoku University)

3. Contents • Probabilistic Image Processing • Loopy Belief Propagation • Statistical Learning Algorithm • Summary Physical Fluctuomatics (Tohoku University)

4. Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability. Image Restoration by Probabilistic Model Noise Transmission Original Image Degraded Image Bayes Formula Physical Fluctuomatics (Tohoku University)

5. The original images and degraded images are represented by f = (f1,f2,…,f|V|) and g = (g1,g2,…,g|V|), respectively. Image Restoration by Probabilistic Model Original Image Degraded Image Position Vector of Pixel i i i fi: Light Intensity of Pixel i in Original Image gi: Light Intensity of Pixel i in Degraded Image Physical Fluctuomatics (Tohoku University)

6. Probabilistic Modeling of Image Restoration Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels. i j Random Fields Product over All the Nearest Neighbour Pairs of Pixels Physical Fluctuomatics (Tohoku University)

7. gi gi fi fi Probabilistic Modeling of Image Restoration Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels. or Random Fields Physical Fluctuomatics (Tohoku University)

8. Bayesian Image Analysis Degraded Image Prior Probability Degradation Process Original Image Posterior Probability V：Set of All the pixels E：Set of all the nearest neighbour pairs of pixels Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability. Physical Fluctuomatics (Tohoku University)

9. We have some choices to estimate the restored image from posterior probability. In each choice, the computational time is generally exponential order of the number of pixels. Estimation of Original Image Maximum A Posteriori (MAP) estimation (1) (2) Maximum posterior marginal (MPM) estimation (3) Thresholded Posterior Mean (TPM) estimation Physical Fluctuomatics (Tohoku University)

10. Prior Probability of Image Processing Digital Images (fi=0,1,…,q-1) Sampling of Markov Chain Monte Carlo Method E: Set of all the nearest-neighbour pairs of pixels Ferromagnetic Paramagnetic Fluctuations increase near α=1.76…. V: Set of all the pixels Physical Fluctuomatics (Tohoku University)

11. Bayesian Image Analysis by Gaussian Graphical Model Prior Probability V:Set of all the pixels Patterns are generated by MCMC. E:Set of all the nearest-neighbour pairs of pixels Markov Chain Monte Carlo Method Physical Fluctuomatics (Tohoku University)

12. Degradation Process for Gaussian Graphical Model Degradation Process is assumed to be the additive white Gaussian noise. V: Set of all the pixels Gaussian Noise n Degraded Image g Original Image f Degraded image is obtained by adding a white Gaussian noise to the original image. Histogram of Gaussian Random Numbers Physical Fluctuomatics (Tohoku University)

13. Bayesian Image Analysis Degraded Image Prior Probability Degradation Process Original Image Posterior Probability V：Set of All the pixels E：Set of all the nearest neighbour pairs of pixels Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability. Physical Fluctuomatics (Tohoku University)

14. Bayesian Image Analysis Prior Probability Original Image Degradation Process Degraded Image 画素 Posterior Probability Computational Complexity Marginal Posterior Probability of Each Pixel Physical Fluctuomatics (Tohoku University)

15. Posterior Probability for Image Processing by Bayesian Analysis Additive White Gaussian Noise Physical Fluctuomatics (Tohoku University)

16. Bayesian Network and Posterior Probability of Image Processing Bayesian Network Posterior Probability Probabilistic Model expressed in terms of Graphical Representations with Cycles V: Set of all the pixels E: Set of all the nearest neighbour pairs of pixels Physical Fluctuomatics (Tohoku University)

17. Markov Random Fields Markov Random Fields ∂i: Set of all the nearest neighbour pixels of the pixel i Physical Fluctuomatics (Tohoku University)

18. Contents • Probabilistic Image Processing • Loopy Belief Propagation • Statistical Learning Algorithm • Summary Physical Fluctuomatics (Tohoku University)

19. 3 1 4 2 5 Loopy Belief Propagation Probabilistic Model expressed in terms of Graphical Representations with Cycles V: Set of all the pixels E: Set of all the nearest neighbour pairs of pixels Physical Fluctuomatics (Tohoku University)

20. 3 1 4 5 2 Loopy Belief Propagation Simultaneous fixed point equations to determine the messages Physical Fluctuomatics (Tohoku University)

21. i i j Belief Propagation Algorithm for Image Processing Step 1: Solve the simultaneous fixed point equations for messages by using iterative method Step 2: Substitute the messages to approximate expressions of marginal probabilities for each pixel and compute estimated image so as to maximize the approximate marginal probability at each pixel. Physical Fluctuomatics (Tohoku University)

22. Belief Propagation in Probabilistic Image Processing Physical Fluctuomatics (Tohoku University)

23. Contents • Probabilistic Image Processing • Loopy Belief Propagation • Statistical Learning Algorithm • Summary Physical Fluctuomatics (Tohoku University)

24. Hyperparametersa, b are determinedso as to maximize the marginal likelihoodPr{G=g|a,b}with respect to a, b. Statistical Estimation of Hyperparameters Original Image Degraded Image Marginalized with respect to F Marginal Likelihood Physical Fluctuomatics (Tohoku University)

25. Maximum Likelihood in Signal Processing Incomplete Data Original Image =Parameter Degraded Image =Data Marginal Likelihood Hyperparameter Extremum Condition Physical Fluctuomatics (Tohoku University)

26. Maximum Likelihood and EM algorithm Degraded Image =Data Incomplete Data Original Image =Parameter Marginal Likelihood Q -function Hyperparameter Expectation Maximization (EM) algorithm provide us one of Extremum Points of Marginal Likelihood. Extemum Condition Physical Fluctuomatics (Tohoku University)

27. Maximum Likelihood and EM algorithm Physical Fluctuomatics (Tohoku University)

28. Posterior Probability Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model V:Set of all the pixels Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula E:Set of all the nearest-neghbour pairs of pixels |V|x|V| matrix Multi-Dimensional Gaussian Integral Formula Physical Fluctuomatics (Tohoku University)

29. Iteration Procedure in Gaussian Graphical Model Bayesian Image Analysis by Exact Solution of Gaussian Graphical Model Physical Fluctuomatics (Tohoku University)

30. Image Restorations by Exact Solution and Loopy Belief Propagation of Gaussian Graphical Model Exact Loopy Belief Propagation Physical Fluctuomatics (Tohoku University)

31. Image Restorations by Gaussian Graphical Model Belief Propagation Original Image Degraded Image Exact MSE:325 MSE:315 MSE: 1512 Lowpass Filter Median Filter Wiener Filter MSE: 411 MSE: 545 MSE: 447 Physical Fluctuomatics (Tohoku University)

32. Image Restoration by Gaussian Graphical Model Belief Propagation Exact Original Image Degraded Image MSE236 MSE: 260 MSE: 1529 Median Filter Wiener Filter Lowpass Filter MSE: 224 MSE: 372 MSE: 244 Physical Fluctuomatics (Tohoku University)

33. Image Restoration by Discrete Gaussian Graphical Model Original Image Q=4 a(t) Degraded Image s=1 MSE:0.98893 SNR:1.03496 (dB) Belief Propagation MSE: 0.36011 s(t) SNR: 5.42228 (dB) Physical Fluctuomatics (Tohoku University)

34. Image Restoration by Discrete Gaussian Graphical Model Original Image Q=4 a(t) Degraded Image s=1 MSE: 0.98893 SNR: 1.07221 (dB) Belief Propagation MSE: 0.28796 s(t) SNR: 6.43047 (dB) Physical Fluctuomatics (Tohoku University)

35. Image Restoration by Discrete Gaussian Graphical Model Original Image Q=8 a(t) Degraded Image s=1 MSE:0.98893 SNR:1.89127 (dB) Belief Propagation MSE: 0.37697 s(t) SNR: 6.07987 (dB) Physical Fluctuomatics (Tohoku University)

36. Image Restoration by Discrete Gaussian Graphical Model Original Image Q=8 a(t) Degraded Image s=1 MSE:0.98893 SNR:0.93517 (dB) Belief Propagation s(t) MSE: 0.32060 SNR: 5.82715 (dB) Physical Fluctuomatics (Tohoku University)

37. Contents • Probabilistic Image Processing • Loopy Belief Propagation • Statistical Learning Algorithm • Summary Physical Fluctuomatics (Tohoku University)

38. Summary • Image Processing • Bayesian Network for Image Processing • Design of Belief Propagation Algorithm for Image Processing Physical Fluctuomatics (Tohoku University)

39. Practice 10-1 For random fieldsF=(F1,F2,…,F|V|)T andG=(G1,G2,…,G|V|)T and state vectors f=(f1,f2,…,f|V|)T andg=(g1,g2,…,g|V|)T, we consider the following posterior probability distribution: Here ZPosterior is a normalization constant and E is the set of all the nearest-neighbour pairs of nodes. Prove that and where ∂i denotes the set of all the nearest neighbour pixels of the pixel i. Physical Fluctuomatics (Tohoku University)

40. Practice 10-2 Make a program that generate a degraded image g=(g1,g2,…,g|V|)from a given image f=(f1,f2,…,f|V|)in which each state variable fi takes integers between 0 and q-1 by according to the following conditional probability distribution of the additive white Gaussian noise: Give some numerical experiments for s=20 and 40. Degraded Image g Gaussian Noise n Original Image f Histogram of Gaussian Random Numbers Physical Fluctuomatics (Tohoku University)

41. Practice 10-3 Make a program for estimating q-valued original images f=(f1,f2,…,f|V|)from a given degraded image g=(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution for q valued images: Give some numerical experiments. K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 . M. Yasuda, S. Kataoka and K. Tanaka: Computer Vision and Image Processing 3 (Edited by Y. Yagi and H. Saito): Chapter 6. Stochastic Image Processing, pp.137-179, Advanced Communication Media CO., Ltd., December 2010 (in Japanese). Physical Fluctuomatics (Tohoku University)

42. Practice 10-4 Make a program for estimating original images f=(f1,f2,…,f|V|)from a given degraded image g=(g1,g2,…,g|V|) by using a belief propagation algorithm for the following posterior probability distribution based on the Gaussian graphical model: Give some numerical experiments. Kazuyuki Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) , Chapter 8 and Appendix G. Physical Fluctuomatics (Tohoku University)