Physical Fluctuomatics 12th Bayesian network and belief propagation in statistical inference

1. Physical Fluctuomatics12th Bayesian network and belief propagation in statistical inference Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University kazu@tohoku.ac.jp http://www.smapip.is.tohoku.ac.jp/~kazu/ Physics Fluctuomatics (Tohoku University)

2. Textbooks • Kazuyuki Tanaka: Mathematics of Statistical Inference by Bayesian Network, Corona Publishing Co., Ltd., October 2009 (in Japanese). Physics Fluctuomatics (Tohoku University)

3. A B Fundamental Probabilistic Theory for Image Processing Joint Probability and Conditional Probability Probability of Event A=a Joint Probability of Events A=a and B=b Conditional Probability of Event B=b when Event A=a has happened. Physics Fluctuomatics (Tohoku University)

4. Marginal Probability of Event B Fundamental Probabilistic Theory for Image Processing A B D C Marginalization Physics Fluctuomatics (Tohoku University)

5. Causal Independence Fundamental Probabilistic Theory for Image Processing A B C A B C Physics Fluctuomatics (Tohoku University)

6. Causal Independence Fundamental Probabilistic Theory for Image Processing A B C D A B C D Physics Fluctuomatics (Tohoku University)

7. A B C Fundamental Probabilistic Theory for Image Processing Causal Independence Physics Fluctuomatics (Tohoku University)

8. Fundamental Probabilistic Theory for Image Processing A A B B Directed Graph Undirected Graph C C Physics Fluctuomatics (Tohoku University)

9. Simple Example of Bayesian Networks Physics Fluctuomatics (Tohoku University)

10. Simple Example of Bayesian Networks Physics Fluctuomatics (Tohoku University)

11. Simple Example of Bayesian Networks Physics Fluctuomatics (Tohoku University)

12. Simple Example of Bayesian Networks Physics Fluctuomatics (Tohoku University)

13. Simple Example of Bayesian Networks Physics Fluctuomatics (Tohoku University)

14. Simple Example of Bayesian Networks Undirected Graph C C Directed Graph S R S R W W Physics Fluctuomatics (Tohoku University)

15. Belief Propagation for Bayesian Networks Belief propagation cannot give us exact computations in Bayesian networks on cycle graphs. Applications of belief propagation to Bayesian networks on cycle graphs provide us many powerful approximate computational models and practical algorithms for probabilistic information processing. Physics Fluctuomatics (Tohoku University)

16. Simple Example of Bayesian Networks Physics Fluctuomatics (Tohoku University)

17. 1 2 3 4 6 5 7 8 Joint Probability of Probabilistic Model with Graphical Representation including Cycles Directed Graph Undirected Hypergraph Physics Fluctuomatics (Tohoku University)

18. 1 2 3 4 6 5 7 8 Marginal Probability Distributions Physics Fluctuomatics (Tohoku University)

19. 1 2 1 2 3 4 3 4 6 5 6 5 7 8 7 8 Approximate Representations of Marginal Probability Distributions in terms of Messages Physics Fluctuomatics (Tohoku University)

20. 1 2 3 4 3 4 6 5 6 5 7 8 7 8 Approximate Representations of Marginal Probability Distributions in terms of Messages Physics Fluctuomatics (Tohoku University)

21. 1 2 1 2 3 4 3 4 3 4 6 5 6 5 6 5 7 8 7 8 7 8 Basic Strategies of Belief Propagations in Probabilistic Model with Graphical Representation including Cycles Approximate Expressions of Marginal Probabilities Physics Fluctuomatics (Tohoku University)

22. 1 2 1 2 3 4 3 4 6 5 6 7 8 Simultaneous Fixed Pint Equations for Belief Propagations in Hypergraph Representations Belief Propagation Algorithm Physics Fluctuomatics (Tohoku University)

23. Fixed Point Equation and Iterative Method Fixed Point Equation Iterative Method Physics Fluctuomatics (Tohoku University)

24. Belief Propagation for Bayesian Networks Belief propagation can be applied to Bayesian networks also on hypergraphs as powerful approximate algorithms. Physics Fluctuomatics (Tohoku University)

25. 1 2 3 4 6 5 7 8 Numerical Experiments Belief Propagation Exact Physics Fluctuomatics (Tohoku University)

26. 1 2 3 4 6 5 7 8 Belief Propagation Numerical Experiments Physics Fluctuomatics (Tohoku University)

27. 2 1 3 4 6 5 7 8 Linear Response Theory Physics Fluctuomatics (Tohoku University)

28. 1 2 3 4 6 5 7 8 Numerical Experiments Physics Fluctuomatics (Tohoku University)

29. Summary • Bayesian Network for Probabilistic Inference • Belief Propagation for Bayesian Networks Physics Fluctuomatics (Tohoku University)

30. Practice 11-1 Compute the exact values of the marginal probability Pr{Xi} for every nodes i(=1,2,…,8), numerically, in the Bayesian network defined by the joint probability distribution Pr{X1,X2,…,X8} as follows: • Each conditional probability table and probability table is given in Figure 3.12 and Table 3.11 in • Kazuyuki Tanaka: Mathematics of Statistical Inference by Bayesian Network, Corona Publishing Co., Ltd., October 2009. Physics Fluctuomatics (Tohoku University)

31. Practice 11-2 Make a program to compute the approximate values of the marginal probability Pr{Xi} for every nodes i(=1,2,…,8) by using the belief propagation method in the Bayesian network defined by the joint probability distribution Pr{X1,X2,…,X8} as follows: • Each conditional probability table and probability table is given in Figure 3.12 and Table 3.11 in • Kazuyuki Tanaka: Mathematics of Statistical Inference by Bayesian Network, Corona Publishing Co., Ltd., October 2009. • The algorithm has appeared explicitly in the above textbook. Physics Fluctuomatics (Tohoku University)