1 / 122

Probabilistic Inference Lecture 3

Probabilistic Inference Lecture 3. M. Pawan Kumar pawan.kumar@ecp.fr. Slides available online http:// cvc.centrale-ponts.fr /personnel/ pawan /. Recap of lecture 1. Exponential Family. P( v ) = exp {- Σ α θ α Φ α ( v ) - A( θ )}. Sufficient Statistics. Log-Partition Function.

borna
Download Presentation

Probabilistic Inference Lecture 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Probabilistic InferenceLecture 3 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvc.centrale-ponts.fr/personnel/pawan/

  2. Recap of lecture 1

  3. Exponential Family P(v) = exp{-ΣαθαΦα(v) - A(θ)} Sufficient Statistics Log-Partition Function Parameters Random Variables V = {V1,V2,…,Vn} Random Variable Va takes a value or label va va L = {l1,l2,…,lh} Labeling V = v

  4. Overcomplete Representation P(v) = exp{-ΣαθαΦα(v) - A(θ)} Sufficient Statistics Log-Partition Function Parameters There exists a non-zero c such that Σα cαΦα(v) = Constant

  5. Pairwise MRF P(v) = exp{-ΣαθαΦα(v) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Sufficient Statistics Parameters for all Va V, li  L Ia;i(va) θa;i θab;ik for all (Va,Vb)  E, li, lk L Iab;ik(va,vb)

  6. Pairwise MRF P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Πaψa(va) Π(a,b)ψab(va,vb) Probability P(v) = Z A(θ) : log Z ψa(li) : exp(-θa;i) ψa(li,lk) : exp(-θab;ik) Parameters θ are sometimes also referred to as potentials

  7. Pairwise MRF P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Labeling as a function f : {1, 2, … , n} {1, 2, …, h} Variable Va takes a label lf(a)

  8. Pairwise MRF P(f) = exp{-Σaθa;f(a) -Σa,bθab;f(a)f(b)- A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Labeling as a function f : {1, 2, … , n} {1, 2, …, h} Variable Va takes a label lf(a) Energy Q(f) = Σaθa;f(a) + Σa,bθab;f(a)f(b)

  9. Pairwise MRF P(f) = exp{-Q(f)- A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Labeling as a function f : {1, 2, … , n} {1, 2, …, h} Variable Va takes a label lf(a) Energy Q(f) = Σaθa;f(a) + Σa,bθab;f(a)f(b)

  10. Inference maxv ( P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} ) Maximum a Posteriori (MAP) Estimation minf ( Q(f) = Σaθa;f(a) + Σa,bθab;f(a)f(b) ) Energy Minimization P(va = li) = ΣvP(v)δ(va = li) P(va = li, vb = lk) = ΣvP(v)δ(va = li)δ(vb= lk) Computing Marginals

  11. Recap of lecture 2

  12. Definitions Energy Minimization f* = arg min Q(f; ) Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) Min-marginals s.t. f(a) = i qa;i= min Q(f; ) Reparameterization ’   Q(f; ’) = Q(f; ), for all f

  13. + Mab;k ’b;k= b;k + Mba;i ’a;i = a;i ’ab;ik = ab;ik - Mab;k - Mba;i Belief Propagation Pearl, 1988 General form of Reparameterization Reparameterization of (a,b) in Belief Propagation Mab;k = mini { a;i + ab;ik } Mba;i = 0

  14. Belief Propagation on Trees Va Vb Vc Vg Vh Vd Ve Forward Pass: Leaf  Root Backward Pass: Root  Leaf All min-marginals are computed

  15. Computational Complexity • Each constant takes O(|L|) • Number of constants - O(|E||L|) O(|E||L|2) • Memory required ? O(|E||L|)

  16. Belief Propagation on Cycles b;1 a;1 b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Remember my suggestion? Fix the label of Va

  17. Belief Propagation on Cycles b;1 b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Equivalent to a tree-structured problem

  18. Belief Propagation on Cycles b;1 a;1 b;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Equivalent to a tree-structured problem

  19. Belief Propagation on Cycles b;1 a;1 b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc This approach quickly becomes infeasible Choose the minimum energy solution

  20. Vincent Algayres Algorithm b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Compute zero cost paths from all labels of Va to all labels of Vd. Requires fixing Va.

  21. Speed-Ups for Special Cases ab;ik= 0, if i = k = C, otherwise. Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

  22. Speed-Ups for Special Cases ab;ik= wab|i-k| Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

  23. Speed-Ups for Special Cases ab;ik= min{wab|i-k|, C} Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

  24. Speed-Ups for Special Cases ab;ik= min{wab(i-k)2, C} Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

  25. Lecture 3

  26. Ising Model P(v) = exp{-ΣαθαΦα(v) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {0, 1} Neighborhood over variables specified by edges E Sufficient Statistics Parameters for all Va V, li  L Ia;i(va) θa;i θab;ik for all (Va,Vb)  E, li, lk L Iab;ik(va,vb) Ia;i(va): indicator for va = li Iab;ik(va,vb): indicator for va = li, vb = lk

  27. Ising Model P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {0, 1} Neighborhood over variables specified by edges E Sufficient Statistics Parameters for all Va V, li  L Ia;i(va) θa;i θab;ik for all (Va,Vb)  E, li, lk L Iab;ik(va,vb) Ia;i(va): indicator for va = li Iab;ik(va,vb): indicator for va = li, vb = lk

  28. Interactive Binary Segmentation Foreground histogram of RGB values FG Background histogram of RGB values BG ‘1’ indicates foreground and ‘0’ indicates background

  29. Interactive Binary Segmentation More likely to be foreground than background

  30. Interactive Binary Segmentation θa;0 proportional to -log(BG(da)) θa;1 proportional to -log(FG(da)) More likely to be background than foreground

  31. Interactive Binary Segmentation More likely to belong to same label

  32. Interactive Binary Segmentation θab;ik proportional to exp(-(da-db)2) if i ≠ k θab;ik = 0 if i = k Less likely to belong to same label

  33. Outline • Minimum Cut Problem • Two-Label Submodular Energy Functions • Move-Making Algorithms

  34. Directed Graph D = (N, A) 10 n1 n2 Two important restrictions 3 2 (1) Rational arc lengths (2) Positive arc lengths n3 n4 5

  35. Cut D = (N, A) • Let N1 and N2 such that • N1 “union” N2 = N • N1 “intersection” N2 = Φ 10 n1 n2 3 2 • C is a set of arcs such that • (n1,n2)  A • n1  N1 • n2  N2 n3 n4 5 C is a cut in the digraph D

  36. Cut D = (N, A) N1 What is C? 10 n1 n2 {(n1,n2),(n1,n4)} ? 3 2 {(n1,n4),(n3,n2)} ? ✓ n3 n4 {(n1,n4)} ? 5 N2

  37. Cut D = (N, A) N1 N2 What is C? 10 n1 n2 {(n1,n2),(n1,n4),(n3,n2)} ? 3 2 ✓ {(n4,n3)} ? n3 n4 {(n1,n4),(n3,n2)} ? 5

  38. Cut D = (N, A) N2 N1 What is C? 10 n1 n2 ✓ {(n1,n2),(n1,n4),(n3,n2)} ? 3 2 {(n3,n2)} ? n3 n4 {(n1,n4),(n3,n2)} ? 5

  39. Cut D = (N, A) • Let N1 and N2 such that • N1 “union” N2 = N • N1 “intersection” N2 = Φ 10 n1 n2 3 2 • C is a set of arcs such that • (n1,n2)  A • n1  N1 • n2  N2 n3 n4 5 C is a cut in the digraph D

  40. Weight of a Cut D = (N, A) 10 n1 n2 3 Sum of length of all arcs in C 2 n3 n4 5

  41. Weight of a Cut D = (N, A) 10 n1 n2 3 w(C) = Σ(n1,n2) C l(n1,n2) 2 n3 n4 5

  42. Weight of a Cut D = (N, A) N1 What is w(C)? 10 n1 n2 3 3 2 n3 n4 5 N2

  43. Weight of a Cut D = (N, A) N1 N2 What is w(C)? 10 n1 n2 5 3 2 n3 n4 5

  44. Weight of a Cut D = (N, A) N2 N1 What is w(C)? 10 n1 n2 15 3 2 n3 n4 5

  45. st-Cut s D = (N, A) 1 2 A source “s” 10 n1 n2 A sink “t” 3 2 • C is a cut such that • s  N1 • t  N2 n3 n4 5 7 t 3 C is an st-cut

  46. Weight of an st-Cut s D = (N, A) 1 2 10 n1 n2 3 w(C) = Σ(n1,n2) C l(n1,n2) 2 n3 n4 5 7 t 3

  47. Weight of an st-Cut s D = (N, A) 1 2 What is w(C)? 10 n1 n2 3 3 2 n3 n4 5 7 t 3

  48. Weight of an st-Cut s D = (N, A) 1 2 What is w(C)? 10 n1 n2 15 3 2 n3 n4 5 7 t 3

  49. Minimum Cut Problem s D = (N, A) 1 2 Find a cut with the minimum weight !! 10 n1 n2 3 2 C* = argminC w(C) n3 n4 5 7 t 3

  50. Solvers for the Minimum-Cut Problem Augmenting Path and Push-Relabel n: #nodes m: #arcs U: maximum arc length [Slide credit: Andrew Goldberg]

More Related