1 / 54

Efficiently Solving Dynamic Markov Random Fields using Graph Cuts

Efficiently Solving Dynamic Markov Random Fields using Graph Cuts . Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University. Overview. Recycling Solutions. solve. S A. differences between A and B. P B*. Simpler problem. A and B similar. S B. Overview.

buzz
Download Presentation

Efficiently Solving Dynamic Markov Random Fields using Graph Cuts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Efficiently Solving Dynamic Markov Random Fields using Graph Cuts Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University

  2. Overview Recycling Solutions

  3. solve SA differences between A and B PB* Simpler problem A and B similar SB Overview PA cheaper operation PB computationally expensive operation

  4. Overview SA PA differences between A and B PB* PB SB

  5. Contributions • A dynamic algorithm for the st-mincut problem. • Exact • Handle arbitrary changes • Minimize dynamic energy functions. • Efficient image segmentation in videos

  6. Outline of the Talk • Markov Random Fields • Energy minimization using graph cuts • Minimizing dynamic energy functions • Experimental Results

  7. Outline of the Talk • Markov Random Fields • Energy minimization using graph cuts • Minimizing dynamic energy functions • Experimental Results

  8. Markov Random Fields • Set of random variables (x) taking values from a label set e.g. • stereo (disparity) • image segmentation (‘fg’ or ‘bg’) • Energy of a configuration • E(x) = - log Pr(x|D) + constant • Less energy = More Probability

  9. MRF for Image Segmentation Boykov and Jolly [ICCV 2001], Blake et al. [ECCV 2004] EnergyMRF = Unary likelihood Contrast Term Uniform Prior (Potts Model) Maximum-a-posteriori (MAP) solution x*= arg min E(x) x Data (D) Unary likelihood Pair-wise Terms MAP Solution

  10. st-mincut s t Labelling problem MAP solution Solving MRFs using Graph cuts • Construct graph to represent the MRF energy • binary labels (Kolmogorov and Zabih [ECCV 2002]) • multiple labels (Ishikawa [PAMI 2003]) • Compute st-mincut by finding the max-flow • Obtain the exact MAP solution MRF Graph Cut

  11. Outline of the Talk • Markov Random Fields • Energy minimization using graph cuts • Minimizing dynamic energy functions • Experimental Results

  12. Energy Minimization using Graph cuts Graph Construction for Binary Random Variables EMRF(a1,a2) Source (0) a1 a2 Sink (1)

  13. Energy Minimization using Graph cuts EMRF(a1,a2) =2a1 Source (0) 2 t-edges (unary terms) a1 a2 Sink (1)

  14. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1 Source (0) 2 a1 a2 5 Sink (1)

  15. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 Source (0) 2 9 a1 a2 5 4 Sink (1)

  16. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 +2a1ā2 Source (0) 2 9 a1 a2 2 5 4 n-edges (pair-wise term) Sink (1)

  17. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 1 a1 a2 2 5 4 Sink (1)

  18. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 1 a1 a2 2 5 4 Sink (1)

  19. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 Cost of st-cut = 11 1 a1 a2 a1 = 1 a2 = 1 2 5 4 EMRF(1,1) = 11 Sink (1)

  20. Energy Minimization using Graph cuts EMRF(a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 Source (0) 2 9 Cost of st-cut = 8 1 a1 a2 a1 = 1 a2 = 0 2 5 4 EMRF(1,0) = 8 Sink (1)

  21. Energy Minimization using Graph cuts • Most probable (MAP) configuration ≡minimum cost st-cut. • st-mincut is in general a NP-hard problem • - negative edge weights • Solvable in polynomial time • non-negative edge weights • corresponds to sub-modular (regular) energy functions

  22. Computing the st-mincut from Max-flow algorithms Source (0) • The Max-flow Problem • Edge capacity and flow balance constraints 2 9 • Notation • Residual capacity • (edge capacity – current flow) • Augmenting path 1 a1 a2 2 5 4 • Simple Augmenting Path based Algorithms • Repeatedly find augmenting paths and push flow. • Saturated edges constitute the st-mincut. • [Ford-Fulkerson Theorem] Sink (1)

  23. Computing the st-mincut Source (0) Key Observation 9 + α 2 Adding a constant to both the t-edges of a node does not change the edges constituting the st-mincut. 1 a1 a2 2 4 + α 5 Sink (1) E (a1,a2) = 2a1 + 5ā1+ 9a2 + 4ā2 + 2a1ā2 +ā1a2 E*(a1,a2 ) = E(a1,a2) + α(a2+ā2) = E(a1,a2) + α [a2+ā2 =1]

  24. Outline of the Talk • Markov Random Fields • Energy minimization using graph cuts • Minimizing dynamic energy functions • Experimental Results

  25. Minimizing dynamic energy functions Problem Given a solution of [min Ea], compute [min Eb] • Partial Solution • Boykov and Jolly, Interactive Image Segmentation [ICCV01] • - limited to unary energy terms (t-edge capacities) • Our Contribution • Arbitrary changes in the energy (graph)

  26. Minimizing dynamic energy functions Problem Given a solution of [min Ea], compute [min Eb] - Compute max-flow in Gb given the solution of Ga • Solution • Use the flows in Gato compute the st-mincut on Gb • Some flows may be more than new edge capacity! • Edge capacity constraints violated.

  27. Satisfying Constraints • Possible Solution • Push excess flow back through the augmenting paths • Extremely computationally expensive • O(m) time for finding such path • Our Solution • - Re-parameterize the problem to satisfy constraints • Constant time per update operation [O(1)] [ m = number of edges ]

  28. Dynamic Image Segmentation Image Segmentation Obtained Flows in n-edges

  29. Maximum flow MAP solution First segmentation problem Ga difference between Ga and Gb residual graph (Gr) second segmentation problem updated residual graph G` Gb Our Algorithm

  30. Graph Re-parameterization s flow/residual capacity 0/7 0/1 0/5 xi xj 0/9 0/2 0/4 t G original graph

  31. Graph Re-parameterization Edges cut s flow/residual capacity 5/2 1/0 0/7 0/1 Compute Maxflow 3/2 0/5 xi xj xi xj 0/12 0/9 st-mincut 2/0 4/0 0/2 0/4 t t Gr G residual graph original graph

  32. Update t-edgeCapacities s 5/2 1/0 3/2 xi xj 0/12 2/0 4/0 t Gr residual graph

  33. Update t-edgeCapacities s capacity changes from 7 to 4 5/2 1/0 3/2 xi xj 0/12 2/0 4/0 t Gr residual graph

  34. excess flow (e) = flow – new capacity = 5 – 4 = 1 Update t-edgeCapacities s capacity changes from 7 to 4 5/-1 1/0 3/2 xi xj edge capacity constraint violated! (flow > capacity) 0/12 2/0 4/0 t G` updated residual graph

  35. excess flow (e) = flow – new capacity = 5 – 4 = 1 add e to both t-edges connected to node i Update t-edgeCapacities s capacity changes from 7 to 4 5/-1 1/0 3/2 xi xj edge capacity constraint violated! (flow > capacity) 0/12 2/0 4/0 t G` updated residual graph

  36. Update t-edgeCapacities excess flow (e) = flow – new capacity s = 5 – 4 = 1 capacity changes from 7 to 4 5/0 1/0 add e to both t-edges connected to node i 3/2 xi xj edge capacity constraint violated! (flow > capacity) 0/12 2/1 4/0 t G` updated residual graph

  37. Update n-edgeCapacities s • Capacity changes from 5 to 2 5/2 1/0 3/2 xi xj 0/12 2/0 4/0 t residual graph Gr

  38. Update n-edgeCapacities s • Capacity changes from 5 to 2 • - edge capacity constraint violated! 5/2 1/0 3/-1 xi xj 0/12 2/0 4/0 t Updated residual graph G`

  39. Update n-edgeCapacities s • Capacity changes from 5 to 2 • - edge capacity constraint violated! • Reduce flow to satisfy constraint 5/2 1/0 3/-1 xi xj 0/12 2/0 4/0 t Updated residual graph G`

  40. Update n-edgeCapacities s • Capacity changes from 5 to 2 • - edge capacity constraint violated! • Reduce flow to satisfy constraint • causes flow imbalance! 1/0 5/2 2/0 excess xi xj 0/11 deficiency 2/0 4/0 t Updated residual graph G`

  41. Update n-edgeCapacities s • Capacity changes from 5 to 2 • - edge capacity constraint violated! • Reduce flow to satisfy constraint • causes flow imbalance! • Push excess flow to/from the terminals • Create capacity by adding α = excess to both t-edges. 1/0 5/2 2/0 excess xi xj 0/11 deficiency 2/0 4/0 t Updated residual graph G`

  42. Update n-edgeCapacities s • Capacity changes from 5 to 2 • - edge capacity constraint violated! • Reduce flow to satisfy constraint • causes flow imbalance! • Push excess flow to the terminals • Create capacity by adding α = excess to both t-edges. 5/3 2/0 2/0 xi xj 0/11 3/0 4/1 t Updated residual graph G`

  43. Update n-edgeCapacities s • Capacity changes from 5 to 2 • - edge capacity constraint violated! • Reduce flow to satisfy constraint • causes flow imbalance! • Push excess flow to the terminals • Create capacity by adding α = excess to both t-edges. 5/3 2/0 2/0 xi xj 0/11 3/0 4/1 t Updated residual graph G`

  44. Complexity analysis of MRF Update Operations *requires k edge update operations where k is degree of the node

  45. Improving the Algorithm • Finding augmenting paths is time consuming. • Dual-tree maxflow algorithm [Boykov & Kolmogorov PAMI 2004] • Reuses search trees after each augmentation. • Empirically shown to be substantially faster. • Our Idea • Reuse search trees from previous graph cut computation • Saves us search tree creation tree time [O(#edges)] • Search trees have to be modified to make them consistent with new graphs • Constrain the search of augmenting paths • New paths must contain at least one updated edge

  46. Reusing Search Trees c’ = measure of change in the energy • Running time • Dynamic algorithm (c’ + re-create search tree ) • Improved dynamic algorithm (c’) • Video Segmentation Example - Duplicate image frames (No time is needed)

  47. Outline of the Talk • Markov Random Fields • Energy minimization using graph cuts • Minimizing dynamic energy functions • Experimental Results

  48. Experimental Analysis • Compared results with the best static algorithm. • - Dual-tree algorithm [Boykov & Kolmogorov PAMI 2004] • Applications • Interactive Image Segmentation • Image Segmentation in Videos

  49. Experimental Analysis Interactive Image segmentation (update unary terms) EnergyMRF= additional segmentation cues user segmentation cues static: 175 msec dynamic : 80 msec dynamic (optimized): 15 msec static : 175 msec

  50. ExperimentalAnalysis Image segmentation in videos (unary & pairwise terms) EnergyMRF= Image resolution: 720x576 static: 220 msec dynamic (optimized): 50 msec Dynamic Graph Cuts Graph Cuts

More Related