Bayesian Networks Bucket Elimination Algorithm

1. Bayesian NetworksBucket Elimination Algorithm 主講人：虞台文 大同大學資工所 智慧型多媒體研究室

2. Content • Basic Concept • Belief Updating • Most Probable Explanation (MPE) • Maximum A Posteriori (MAP)

3. Bayesian NetworksBucket Elimination Algorithm Basic Concept 大同大學資工所 智慧型多媒體研究室

4. Satisfiability Given a statement of clauses (in disjunction normal form), the satisfiability problem is to determine whether there exists a truth assignment to make the statement true. Examples: Satisfiable A=True, B=True, C=False, D=False Satisfiable?

5. Resolution can be true if and only if can be true.   unsatisfiable

6. BucketA BucketB BucketC BucketD Direct Resolution Example: Given a set of clauses and an order d=ABCD Set initial buckets as follows:

7. BucketA BucketB BucketC BucketD Direct Resolution Because no empty clause () is resulted, the statement is satisfiable. How to get a truth assignment?

8. BucketA BucketB BucketC BucketD Direct Resolution

9. Direct Resolution

10. Queries on Bayesian Networks • Belief updating • Finding the most probable explanation (mpe) • Given evidence, finding a maximum probability assignment to the rest of variables. • Maximizing a posteriori hypothesis (map) • Given evidence, finding an assignment to a subset of hypothesis variables that maximize their probability. • Maximizing the expected utility of the problem (meu) • Given evidence and utility function, finding a subset of decision variables that maximize the expected utility.

11. Bucket Elimination • The algorithm will be used as a framework for various probabilistic inferences on Bayesian Networks.

12. Preliminary – Elimination Functions Given a function h defined over subset of variables S, where X S, Eliminate parameterX fromh Defined overU = S– {X}.

13. Preliminary – Elimination Functions Given a function h defined over subset of variables S, where X S,

14. Preliminary – Elimination Functions Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively, Defined over

15. Preliminary – Elimination Functions Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,

16. Bayesian NetworksBucket Elimination Algorithm Belief Updating 大同大學資工所 智慧型多媒體研究室

17. Goal Normalization Factor

18. A C B F D G Basic Concept of Variable Elimination Example:

19. Basic Concept of Variable Elimination Example:

20. Basic Concept of Variable Elimination G(f) D(a, b) F(b, c) B(a, c) C(a)

21. Basic Concept of Variable Elimination BucketG BucketD BucketF BucketB BucketC BucketA

22. Basic Concept of Variable Elimination BucketG BucketD BucketF BucketB BucketC BucketA

23. Basic Concept of Variable Elimination

24. Basic Concept of Variable Elimination

25. 0.7 0.1 0.7 0.1 0.7 0.1 0.7 0.1 Basic Concept of Variable Elimination

26. Basic Concept of Variable Elimination

27. Basic Concept of Variable Elimination

28. Basic Concept of Variable Elimination

29. Bucket Elimination Algorithm

30. Complexity • The BuckElim Algorithm can be applied to any ordering. • The arity of the function recorded in a bucket • the numbers of variables appearing in the processed bucked, excluding the bucket’s variable. • Time and Space complexity is exponentially grow with a function of arity r. • The arity is dependent on the ordering. • How many possible orderings for BN’s variables?

31. A C B F D G Consider the ordering AFDCBG. Determination of the Arity BucketG BucketB 1 G 4 BucketC B 1 ,3 C BucketD 0 ,2 D BucketF ,1 0 F BucketA 0 A

32. A C B 1 1 F G D 4 4 B G 3 1 C 2 0 D 1 0 F 0 0 A d Given the ordering, e.g., AFDCBG. Determination of the Arity The width of a graph is the maximum width of its nodes. w(d) = 4 w*(d) = 4 w(d): width of initial graph for ordering d. w*(d): width of induced graph for ordering d. Width of node Width of node G B C Induced Graph D Initial Graph F A

33. Definition of Tree-Width Goal: Finding an ordering with smallest induced width. Greedy heuristic and Approximation methods Are available. NP-Hard

34. Summary • The complexity of BuckElim algorithm is dominated by the time and space needed to process a bucket. • It is time and space is exponential in number of bucket variables. • Induced width bounds the arity of bucket functions.

35. A C B F D G Exercises • Use BuckElim to evaluate P(a|b=1) with the following two ordering: • d1=ACBFDG • d2=AFDCBG Give the details and make some conclusion. How to improve the algorithm?

36. Bayesian NetworksBucket Elimination Algorithm Most Probable Explanation (MPE) 大同大學資工所 智慧型多媒體研究室

37. MPE Goal: evidence

38. MPE Goal:

39. xi Notations

40. MPE Let

41. Xn MPE Some terms involve xn, some terms not. Xn is conditioned by its parents. Xnconditions its children.

42. Xn MPE xnappears in these CPT’s Not conditioned by xn Conditioned by xn Itself

43. MPE Process the next bucket recursively. Eliminate variable xnatBucketn.

44. A C B F D G Example

45. A C B F D G Example Consider ordering ACBFDG BucketG BucketD BucketF BucketB BucketC BucketA

46. Bucket Elimination Algorithm

47. Exercise Consider ordering ACBFDG

48. Bayesian NetworksBucket Elimination Algorithm Maximum A Posteriori (MAP) 大同大學資工所 智慧型多媒體研究室

49. MAP Given a belief network, a subset of hypothesized variablesA=(A1, …, Ak), and evidence E=e, the goal is to determine

50. A C B F D G Example Hypothesis (Decision) Variables g = 1