Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University, Japan

Statistical-Mechanical Approach to Probabilistic Inference--- Cluster Variation Method and Generalized Loopy Belief Propagation --- Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University, Japan kazu@statp.is.tohoku.ac.jp http://www.statp.is.tohoku.ac.jp/~kazu/index-e.html University of Glasgow

Contents Introduction Probabilistic Inference Cluster Variation Method Generalized Loopy Belief Propagation Linear Response Numerical Experiments Concluding Remarks University of Glasgow

Bayes Formula Probabilistic Inference Probabilistic Model Introduction Belief Marginal Probability Probabilistic models on networks with some loops =>Good Approximation Probabilistic models on tree-like networks with no loops =>Exact Results Probabilistic Inference and Belief Propagation Belief Propagation Generalization University of Glasgow

Probabilistic model with no loop Belief Propagation Introduction (Lauritzen, Pearl) Probabilistic model with some loops Probabilistic model with no loop Transfer Matrix Transfer Matrix = Belief Propagation Statistical Mechanics and Belief Propagation Recursion Formula for Beliefs and Messages Bethe/Kikuchi Method Cluster Variation Method Probabilistic model with some loops University of Glasgow

Introduction • Review of generalized loopy belief propagation and cluster variation method. • Calculation of correlations between any pair of nodes by combining the cluster variation method with the linear response theory Purpose University of Glasgow

Probabilistic Inference University of Glasgow

Probabilistic Inference Probabilistic Inference and Probabilistic Model University of Glasgow

Probabilistic Inference Probabilistic Model and Beliefs Statistics: Marginal Probability Statistical Mechanics: One-body distribution Probabilistic Inference: Belief University of Glasgow

Cluster Variation Method Probabilistic Inference and Probabilistic Model University of Glasgow

Cluster Variation Method University of Glasgow

Generalized Loopy Belief Propagation Extreme Condition of Kullback-Leibler Divergence Message Passing Algorithm University of Glasgow

Generalized Loopy Belief Propagation Expression of Marginal Probability in terms of Messages University of Glasgow

Generalized Loopy Belief Propagation University of Glasgow

Linear Response University of Glasgow

Cluster Variation Method Numerical Experiments Exact Beliefs University of Glasgow

Cluster Variation Method Numerical Experiments Exact Correlation Functions University of Glasgow

Summary • Review of Cluster Variation Method and Loopy Belief Propagation • Calculation of Correlation Function by means of Linear Response Theory and Cluster Variation Method Concluding Remarks Future Problem • Learning of Hyperparameters by means of Maximum Likelihood Estimation University of Glasgow

Kazuyuki Tanaka Graduate School of Information Sciences Tohoku University, Japan