1 / 23

On the Power of Belief Propagation: A Constraint Propagation Perspective

On the Power of Belief Propagation: A Constraint Propagation Perspective. Rina Dechter. Bozhena Bidyuk. Robert Mateescu. Emma Rollon. Distributed Belief Propagation. 4. 3. 2. 1. 1. 2. 3. 4. Distributed Belief Propagation. 5. 5. 5. 5. 5. How many people?.

ama
Download Presentation

On the Power of Belief Propagation: A Constraint Propagation Perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On the Power of Belief Propagation:A Constraint Propagation Perspective Rina Dechter BozhenaBidyuk RobertMateescu EmmaRollon

  2. Distributed Belief Propagation

  3. 4 3 2 1 1 2 3 4 Distributed Belief Propagation 5 5 5 5 5 How many people?

  4. Distributed Belief Propagation Causal support Diagnostic support

  5. Belief Propagation in Polytrees

  6. BP on Loopy Graphs • Pearl (1988): use of BP to loopy networks • McEliece, et. Al 1988: IBP’s success on coding networks • Lots of research into convergence … and accuracy (?), but: • Why IBP works well for coding networks • Can we characterize other good problem classes • Can we have any guarantees on accuracy (even if converges)

  7. Arc-consistency • Sound • Incomplete • Always converges (polynomial) A B 1 1 2 2 < 3 3 A B < = D C D C < 1 1 2 2 3 3

  8. 1 1 A A A A 2 3 A A 2 3 AB AC AB AC B A B C B A B C 4 5 4 5 ABD BCF ABD BCF 6 F D 6 DFG F D DFG Flattening the Bayesian Network Flat constraint network Belief network

  9. 1 A A A 2 3 AB AC A B C B 5 4 ABD BCF 6 F D DFG Belief Zero Propagation = Arc-Consistency Updated relation: Updated belief:

  10. 1 A A 3 A 2 AB AC B A C B 5 4 ABD BCF 6 F D DFG Flat Network - Example

  11. 1 A A 3 A 2 AB AC B A C B 5 4 ABD BCF 6 F D DFG IBP Example – Iteration 1

  12. 1 A A 3 A 2 AB AC B A C B 5 4 ABD BCF 6 F D DFG IBP Example – Iteration 2

  13. 1 A A 3 A 2 AB AC B A C B 5 4 ABD BCF 6 F D DFG IBP Example – Iteration 3

  14. 1 A A 3 A 2 AB AC B A C B 5 4 ABD BCF 6 F D DFG IBP Example – Iteration 4

  15. 1 A A 3 A 2 AB AC B A C B 5 4 ABD BCF 6 F D DFG IBP Example – Iteration 5

  16. IBP – Inference Power for Zero Beliefs • Theorem: Iterative BP performs arc-consistency on the flat network. • Soundness: • Inference of zero beliefs by IBP converges • All the inferred zero beliefs are correct • Incompleteness: • Iterative BP is as weak and as strong as arc-consistency • Continuity Hypothesis: IBP is sound for zero - IBP is accurate for extreme beliefs? Tested empirically

  17. Algorithms: IBP IJGP Experimental Results We investigated empirically if the results for zero beliefs extend to ε-small beliefs (ε > 0) • Measures: • Exact/IJGP histogram • Recall absolute error • Precision absolute error • Network types: • Coding • Linkage analysis* • Grids* • Two-layer noisy-OR* • CPCS54, CPCS360 YES Have determinism? NO * Instances from the UAI08 competition

  18. Networks withDeterminism: Coding N=200, 1000 instances, w*=15

  19. Nets w/o Determinism: bn2o w* = 24 w* = 27 w* = 26

  20. NetswithDeterminism: Linkage Exact Histogram IJGP Histogram Recall Abs. Error Precision Abs. Error pedigree1, w* = 21 Absolute Error Percentage pedigree37, w* = 30 Absolute Error Percentage i-bound = 3 i-bound = 7

  21. The Cutset Phenomena & irrelevant nodes • Observed variables break the flow of inference • IBP is exact when evidence variables form a cycle-cutset • Unobserved variables without observed descendents send zero-information to the parent variables – it is irrelevant • In a network without evidence, IBP converges in one iteration top-down X Y X

  22. Nodes with extreme support Observed variables with xtreme priors or xtreme support can nearly-cut information flow: A B1 C1 EB EC B2 C2 D EC EB ED Average Error vs. Priors

  23. Conclusion: For Networks with Determinism • IBP converges & sound for zero beliefs • IBP’s power to infer zeros is as weak or as strong as arc-consistency • However: inference of extreme beliefs can be wrong. • Cutset property (Bidyuk and Dechter, 2000): • Evidence and inferred singleton act like cutset • If zeros are cycle-cutset, all beliefs are exact • Extensions to epsilon-cutset were supported empirically.

More Related