東北大学 大学院情報科学研究科 応用情報科学専攻 田中 和之 (Kazuyuki Tanaka) kazu@smapip.is.tohoku.ac.jp

1. 物理フラクチュオマティクス論Physical Fluctuomatics応用確率過程論Applied Stochastic Process 第5回グラフィカルモデルによる確率的情報処理5th Probabilistic information processing by means of graphical model 東北大学 大学院情報科学研究科 応用情報科学専攻 田中 和之(Kazuyuki Tanaka) kazu@smapip.is.tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ 物理フラクチュオマティクス論(東北大)

2. 今回の講義の講義ノート 田中和之著： 確率モデルによる画像処理技術入門， 森北出版，2006． 物理フラクチュオマティクス論(東北大)

3. Contents • Introduction • Probabilistic Image Processing • Gaussian Graphical Model • Statistical Performance Analysis • Concluding Remarks 物理フラクチュオマティクス論(東北大)

4. Contents • Introduction • Probabilistic Image Processing • Gaussian Graphical Model • Statistical Performance Analysis • Concluding Remarks 物理フラクチュオマティクス論(東北大)

5. Markov Random Fields for Image Processing Markov Random Fields are One of Probabilistic Methods for Image processing. • S. Geman and D. Geman (1986): IEEE Transactions on PAMI • Image Processing for Markov Random Fields (MRF) (Simulated Annealing, Line Fields) • J. Zhang (1992): IEEE Transactions on Signal Processing • Image Processing in EM algorithm for Markov Random Fields (MRF) (Mean Field Methods) 物理フラクチュオマティクス論(東北大)

6. Hyperparameter Estimation Statistical Quantities Estimation of Image Markov Random Fields for Image Processing In Markov Random Fields, we have to consider not only the states with high probabilities but also ones with low probabilities. In Markov Random Fields, we have to estimate not only the image but also hyperparameters in the probabilistic model. We have to perform the calculations of statistical quantities repeatedly. We can calculate statistical quantities by adopting the Gaussian graphical model as a prior probabilistic model and by using Gaussian integral formulas. 物理フラクチュオマティクス論(東北大)

7. Purpose of My Talk • Review of formulation of probabilistic model for image processing by means of conventional statistical schemes. • Review of probabilistic image processing by using Gaussian graphical model (Gaussian Markov Random Fields) as the most basic example. K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A: Math. Gen., vol.35, pp.R81-R150, 2002. Section 2 and Section 4 are summarized in the present talk. 物理フラクチュオマティクス論(東北大)

8. Contents • Introduction • Probabilistic Image Processing • Gaussian Graphical Model • Statistical Performance Analysis • Concluding Remarks 物理フラクチュオマティクス論(東北大)

9. Image Representation in Computer Vision Digital image is defined on the set of points arranged on a square lattice. The elements of such a digital array are called pixels. We have to treat more than 100,000 pixels even in the digital cameras and the mobile phones. 物理フラクチュオマティクス論(東北大)

10. Image Representation in Computer Vision At each point, the intensity of light is represented as an integer number or a real number in the digital image data. A monochrome digital image is then expressed as a two-dimensional light intensity function and the value is proportional to the brightness of the image at the pixel. 物理フラクチュオマティクス論(東北大)

11. Noise Reduction by Conventional Filters The function of a linear filter is to take the sum of the product of the mask coefficients and the intensities of the pixels. Smoothing Filters 202 202 192 190 192 190 219 173 202 120 202 120 218 218 100 110 100 110 It is expected that probabilistic algorithms for image processing can be constructed from such aspects in the conventional signal processing. Markov Random Fields Probabilistic Image Processing Algorithm 物理フラクチュオマティクス論(東北大)

12. Bayes Formula and Bayesian Network Prior Probability Data-Generating Process Bayes Rule Posterior Probability A Event A is given as the observed data. Event B corresponds to the original information to estimate. Thus the Bayes formula can be applied to the estimation of the original information from the given data. B Bayesian Network 物理フラクチュオマティクス論(東北大)

13. Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability. Image Restoration by Probabilistic Model Noise Transmission Original Image Degraded Image Bayes Formula 物理フラクチュオマティクス論(東北大)

14. The original images and degraded images are represented by f = {fi} and g = {gi}, respectively. Image Restoration by Probabilistic Model Original Image Degraded Image Position Vector of Pixel i i i fi: Light Intensity of Pixel i in Original Image gi: Light Intensity of Pixel i in Degraded Image 物理フラクチュオマティクス論(東北大)

15. gi gi fi fi Probabilistic Modeling of Image Restoration Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels. or Random Fields 物理フラクチュオマティクス論(東北大)

16. Probabilistic Modeling of Image Restoration Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels. i j Random Fields Product over All the Nearest Neighbour Pairs of Pixels 物理フラクチュオマティクス論(東北大)

17. p p = > = It is important how we should assume the function F(fi,fj) in the prior probability. Prior Probability for Binary Image i j We assume that every nearest-neighbour pair of pixels take the same state of each other in the prior probability. Probability of Neigbouring Pixel i j 物理フラクチュオマティクス論(東北大)

18. p p = > = i j Probability of Nearest Neigbour Pair of Pixels Prior Probability for Binary Image ？ Which state should the center pixel be taken when the states of neighbouring pixels are fixed to the white states? > Prior probability prefers to the configuration with the least number of red lines. 物理フラクチュオマティクス論(東北大)

19. = > = Prior Probability for Binary Image p p ？-？ Which state should the center pixel be taken when the states of neighbouring pixels are fixed as this figure? > = > Prior probability prefers to the configuration with the least number of red lines. 物理フラクチュオマティクス論(東北大)

20. p What happens for the case of large umber of pixels? Patterns with both ordered states and disordered states are often generated near the critical point. Covariance between the nearest neghbour pairs of pixels lnp small p large p Sampling by Marko chain Monte Carlo Critical Point (Large fluctuation) Disordered State Ordered State 物理フラクチュオマティクス論(東北大)

21. Pattern near Critical Point of Prior Probability We regard that patterns generated near the critical point are similar to the local patterns in real world images. Covariance between the nearest neghbour pairs of pixels ln p small p large p similar 物理フラクチュオマティクス論(東北大)

22. Contents • Introduction • Probabilistic Image Processing • Gaussian Graphical Model • Statistical Performance Analysis • Concluding Remarks 物理フラクチュオマティクス論(東北大)

23. Bayesian Image Analysis by Gaussian Graphical Model Prior Probability V:Set of all the pixels Patterns are generated by MCMC. E:Set of all the nearest-neighbour pairs of pixels Markov Chain Monte Carlo Method 物理フラクチュオマティクス論(東北大)

24. Bayesian Image Analysis by Gaussian Graphical Model Degradation Process is assumed to be the additive white Gaussian noise. V: Set of all the pixels Gaussian Noise n Degraded Image g Original Image f Degraded image is obtained by adding a white Gaussian noise to the original image. Histogram of Gaussian Random Numbers 物理フラクチュオマティクス論(東北大)

25. Bayesian Image Analysis Degraded Image Prior Probability Degradation Process Original Image Posterior Probability V：Set of All the pixels E：Set of all the nearest neighbour pairs of pixels Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability. 物理フラクチュオマティクス論(東北大)

26. We have some choices to estimate the restored image from posterior probability. In each choice, the computational time is generally exponential order of the number of pixels. Estimation of Original Image Maximum A Posteriori (MAP) estimation (1) (2) Maximum posterior marginal (MPM) estimation (3) Thresholded Posterior Mean (TPM) estimation 物理フラクチュオマティクス論(東北大)

27. Hyperparameters a, s are determinedso as to maximize the marginal likelihood Pr{G=g|a,s} with respect to a, s. Statistical Estimation of Hyperparameters Original Image Degraded Image Marginalized with respect to F Marginal Likelihood 物理フラクチュオマティクス論(東北大)

28. A Posteriori Probability Bayesian Image Analysis Gaussian Graphical Model 物理フラクチュオマティクス論(東北大)

29. Average of Posterior Probability Gaussian Integral formula 物理フラクチュオマティクス論(東北大)

30. Posterior Probability Bayesian Image Analysis by Gaussian Graphical Model V:Set of all the pixels Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula E:Set of all the nearest-neghbour pairs of pixels |V|x|V| matrix Multi-Dimensional Gaussian Integral Formula 物理フラクチュオマティクス論(東北大)

31. Statistical Estimation of Hyperparameters Original Image Degraded Image Marginalized with respect to F Marginal Likelihood 物理フラクチュオマティクス論(東北大)

32. Calculations of Partition Function Gaussian Integral formula (A is a real symmetric and positive definite matrix.) 物理フラクチュオマティクス論(東北大)

33. Exact expression of Marginal Likelihood in Gaussian Graphical Model Multi-dimensional Gauss integral formula We can construct an exact EM algorithm. 物理フラクチュオマティクス論(東北大)

34. Iteration Procedure in Gaussian Graphical Model Bayesian Image Analysis by Gaussian Graphical Model 物理フラクチュオマティクス論(東北大)

35. Image Restoration by Markov Random Field Model and Conventional Filters Original Image Degraded Image V:Set of all the pixels Restored Image MRF (3x3) Lowpass (5x5) Median 物理フラクチュオマティクス論(東北大)

36. Contents • Introduction • Probabilistic Image Processing • Gaussian Graphical Model • Statistical Performance Analysis • Concluding Remarks 物理フラクチュオマティクス論(東北大)

37. Performance Analysis Sample Average of Mean Square Error Additive White Gaussian Noise Posterior Probability Signal Observed Data Estimated Results 物理フラクチュオマティクス論(東北大)

38. Statistical Performance Analysis Degraded Image Original Image Additive White Gaussian Noise Restored Image Posterior Probability Additive White Gaussian Noise 物理フラクチュオマティクス論(東北大)

39. Statistical Performance Analysis 物理フラクチュオマティクス論(東北大)

40. Statistical Performance Estimation for Gaussian Markov Random Fields = 0 物理フラクチュオマティクス論(東北大)

41. Statistical Performance Estimation for Gaussian Markov Random Fields s=40 s=40 a a 物理フラクチュオマティクス論(東北大)

42. Contents • Introduction • Probabilistic Image Processing • Gaussian Graphical Model • Statistical Performance Analysis • Concluding Remarks 物理フラクチュオマティクス論(東北大)

43. Summary • Formulation of probabilistic model for image processing by means of conventional statistical schemes has been summarized. • Probabilistic image processing by using Gaussian graphical model has been shown as the most basic example. 物理フラクチュオマティクス論(東北大)

44. References K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) . K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A, 35 (2002). A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing, Proceedings of IEEE, 90 (2002). 物理フラクチュオマティクス論(東北大)

45. Problem 5-1: Derive the expression of the posterior probability Pr{F=f|G=g,a,s} by using Bayes formulas Pr{F=f|G=g,a,s}=Pr{G=g|F=f,s}Pr{F=f,a}/Pr{G=g|a,s}. Here Pr{G=g|F=f,s} and Pr{F=f,a} are assumed to be as follows: [Answer] 物理フラクチュオマティクス論(東北大)

46. Problem 5-2: Show the following equality. 物理フラクチュオマティクス論(東北大)

47. Problem 5-3: Show the following equality. 物理フラクチュオマティクス論(東北大)

48. Problem 5-4: Show the following equalities by using the multi-dimensional Gaussian integral formulas. 物理フラクチュオマティクス論(東北大)

49. Problem 5-5: Derive the extremum conditions for the following marginal likelihood Pr{G=g|a,s} with respect to the hyperparameters a and s. [Answer] 物理フラクチュオマティクス論(東北大)

50. Problem 5-6: Derive the extremum conditions for the following marginal likelihood Pr{G=g|a,s} with respect to the hyperparameters a and s. [Answer] 物理フラクチュオマティクス論(東北大)