1 / 28

Bucket Elimination Tutorial #9 by Ma’ayan Fishelson

Learn about Bucket Elimination Algorithm for Bayesian network inference, with examples and explanation of probabilistic inference tasks.

marknewton
Download Presentation

Bucket Elimination Tutorial #9 by Ma’ayan Fishelson

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bucket Elimination Tutorial #9 by Ma’ayan Fishelson

  2. Bucket Elimination Algorithm • An algorithm for performing inference in a Bayesian network. • Similar algorithms can be used in other domains, such as constraint satisfaction.

  3. Bayesian Network • X = {X1,…,Xn} is a set of random variables. • A BN is a pair (G,P): • G is a directed acyclic graph over nodes that represent the random variables X. • P = {Pi|1 ≤i ≤n}. Pi, defined on , is the conditional probability table associated with node Xi. Pi = P(Xi | pa(Xi)) • The BN represents a probability distribution over X.

  4. Bayesian Network Moralized Graph A A P(A) B C B C P(B|A) P(C|A) E E P(E|B,C) D D P(D|A,B) F F P(F|E) Bayesian Network - Example P(A,B,C,D,E,F) = P(A)P(B|A)P(C|A)P(D|A,B)P(E|B,C)P(F|E)

  5. Some Probabilistic Inference Tasks… • Belief Assessment: find P(Xi = xi | e). • Most Probable Explanation (MPE): find

  6. Solution – Bucket Elimination Algorithms Given an ordering of the variables X1,…Xn: • Distribute P1,…,Pn into buckets B1,…,Bn. is the highest in order in Ai. • Process the buckets in reverse order: BnB1. (Whenprocessing bucket Bi, multiply all the probability tables in Bi and eliminate the bucket’s variable Xi by summing over all its possible values.) • Place the resulting function in the bucket of the highest variable (in the order) that is in its scope.

  7. Example – Belief Assessment • Compute: • P(A=a) = Σ{A=a,B,C,D,E,F}P(A,B,C,D,E,F) = • Σ{A=a,B,C,D,E,F}P(A)P(B|A)P(C|A)P(D|A,B)P(E|B,C)P(F|E) • Suppose an order A,C,B,E,D,F. and evidence that F=1. • The distribution into buckets is as follows: P(A) P(C|A) P(B|A) P(E|B,C) P(D|A,B) P(F|E) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A)

  8. P(A) P(C|A) P(B|A) P(E|B,C) P(D|A,B) P(F|E) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) P(A) P(C|A) P(B|A) P(E|B,C) λF(E) P(D|A,B) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) Example – Computing P(A=a) • To process B6: Assign F=1, get λF(E) = P(F=1|E) • Place λF(E) in bucket B4(E).

  9. P(A) P(C|A) P(B|A) P(E|B,C) λF(E) P(D|A,B) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) P(B|A) λD(A,B) P(A) P(C|A) P(E|B,C) λF(E) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) Computing P(A=a) (cont. 1) • To process B5: λD(A,B) = ΣDP(D|A,B) • Place λD(A,B) in bucket B3(B).

  10. P(A) P(C|A) P(B|A) λD(A,B) P(E|B,C) λF(E) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) P(B|A) λD(A,B) λE(B,C) P(A) P(C|A) B2(C) B4(E) B5(D) B6(F) B1(A) B3(B) Computing P(A=a) (cont. 2) • To process B4: λE(B,C) = ΣEP(E|B,C)λF(E) • Place λE(B,C) in bucket B3(B).

  11. P(B|A) λD(A,B) λE(B,C) P(A) P(C|A) B2(C) B4(E) B5(D) B6(F) B1(A) B3(B) P(C|A) λB(A,C) P(A) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) Computing P(A=a) (cont. 3) • To process B3: λB(A,C) = ΣBP(B|A)λD(A,B) λE(B,C) • Place λB(A,C) in bucket B2(C).

  12. P(A) P(C|A) λB(A,C) B2(C) B4(E) B5(D) B6(F) B3(B) B1(A) P(A) λC(A) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) Computing P(A=a) (cont. 4) • To process B2: λC(A) = ΣCP(C|A)λB(A,C) • Place λC(A) in bucket B1(A).

  13. P(A) λC(A) B2(C) B3(B) B4(E) B5(D) B6(F) B1(A) Computing P(A=a) (cont. 5) • To compute the belief P(A=a), we multiply P(A)λC(A). • We obtain a function of A.

  14. B E A C D D C B C E B A A D E Order B,C,D,E,A. w*(d1) =4 Order E,D,C,B,A. w*(d2) =2 Moral graph Bucket Elimination Complexity • The time and space complexity is bounded by the size of the probability tables that are created by the algorithm. • w*(d) is the induced width (max clique size) of the moral graph along ordering d.

  15. Constraint Satisfaction Problem (CSP) • Input: • A set of variables: X = {X1,…,Xn}. • A set of constraints: R = {R1,…,Rm}. • For each constraint Ri: • Scope of Ri: • Ri is a relation over the variables in Ai. • Output: • An assignment to X that is consistent with R.

  16. Constraint Graph • Nodes: {X1,…Xn}. • Edges: For simplicity, assume that constraints are defined on maximal cliques of the graph.

  17. A A1/A1 O A2/A2 1 2 A A1/A2 A A2/A2 3 4 O A1/A2 5 Example of CSP: Find Possible Genotypes A pedigree typed at the ABO and AK1 loci: • Possible haplotypes: h1=A,A1; h2 = A,A2; h3=O,A1; h4 = O,A2; • Possible genotypes (ordered): gij = hi/hj(father/mother) Ind #1: g11, g13, g31 Ind #2: g44 Ind #3: g12, g21, g14, g41, g23, g32 Ind #4: g22, g24, g42 Ind #5: g34, g43 Constraints: R13 = {<g11,g{12,14}>, <g{13,31},g{12,14,32}>}, similarly for R23, R35, and R45.

  18. Bucket Elimination for CSP • Given an order X1,…, Xn: • Distribute constraints into buckets, according to the highest variable in the constraint’s scope. • Process the buckets one by one starting from Xn: • Join the constraints (with equal value for the eliminated variable), in the eliminated bucket and remember the value of the eliminated variable. • Put the new constraint in the bucket of the highest variable in its scope.

  19. A B C D E Constraint graph Example of CSP: Find Possible Genotypes (cont. 1) Constraints: • RAC = {<g11,g{12,14}>, <g{13,31},g{12,14,32}>}; • RBC = {<g44, g14>}; • RCE = {<g{14,41,23,32} g{34,43}}; • RDE = {<g{24,42} g34>}

  20. Find Possible Genotypes (cont. 2) • Suppose an order: A, B, C, D, E. • The distribution into buckets is as follows:

  21. Find Possible Genotypes (cont. 3) • When processing BE: • compute RCD = {<g{23,32,14,41}, g{24,42}>}. • put RCD into bucket BD. • remember the value g34 for E.

  22. Find Possible Genotypes (cont. 4) • When processing BD: • compute RC = {<g{23,32,14,41}}. • put RC into bucket BC. • remember the values g24,g42 for D.

  23. Find Possible Genotypes (cont. 5) • When processing BC: • compute RAB = {<g{11,13, 31}, g44>}. • put RC into bucket BB. • remember the value g14 for C.

  24. Find Possible Genotypes (cont. 6) • When processing BB: • compute RA = {<g{11,13, 31}>}. • remember the value g44 for B.

  25. A A1/A1 O A2/A2 1 2 A A1/A2 A A2/A2 3 4 O A1/A2 5 A|A or A|O or O|A A1|A1 O|O A2|A2 1 2 A|O A1|A2 A|O or O|A A2/A2 3 4 O|O A1|A2 5 Find Possible Genotypes (cont. 7) • By backtracking we get the • possible values: • Ind #1 (A): g11, g13, g31 • Ind #2 (B): g44 • Ind #3 (C): g14 • Ind #4 (D): g24, g42 • Ind #5 (E): g34

  26. Bucket Elimination Algorithm

  27. Bucket Elimination Algorithm

  28. Bucket Elimination Algorithm • More variations of the algorithm • http://www.ics.uci.edu/~csp/R48a.pdf

More Related