1 / 240

GraphCut -based Optimisation for Computer Vision

GraphCut -based Optimisation for Computer Vision. Ľubor Ladický. Overview. Motivation Min-Cut / Max-Flow (Graph Cut) Algorithm Markov and Conditional Random Fields Random Field Optimisation using Graph Cuts Submodular vs. Non-Submodular Problems Pairwise vs. Higher Order Problems

maisie-mcconnell
Download Presentation

GraphCut -based Optimisation for Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GraphCut-based Optimisation for Computer Vision Ľubor Ladický

  2. Overview • Motivation • Min-Cut / Max-Flow (Graph Cut) Algorithm • Markov and Conditional Random Fields • Random Field Optimisation using Graph Cuts • Submodular vs. Non-Submodular Problems • Pairwise vs. Higher Order Problems • 2-Label vs. Multi-Label Problems • Recent Advances in Random Field Optimisation • Conclusions

  3. Overview • Motivation • Min-Cut / Max-Flow (Graph Cut) Algorithm • Markov and Conditional Random Fields • Random Field Optimisation using Graph Cuts • Submodular vs. Non-Submodular Problems • Pairwise vs. Higher Order Problems • 2-Label vs. Multi-Label Problems • Recent Advances in Random Field Optimisation • Conclusions

  4. Image Labelling Problems Assign a label to each image pixel Geometry Estimation Image Denoising Object Segmentation Depth Estimation Sky Building Tree Grass

  5. Image Labelling Problems • Labellings highly structured Unprobable labelling Impossible labelling Possible labelling

  6. Image Labelling Problems • Labellings highly structured • Labels highly correlated with very complex dependencies • Neighbouring pixels tend to take the same label • Low number of connected components • Classes present may be seen in one image • Geometric / Location consistency • Planarity in depth estimation • … many others (task dependent)

  7. Image Labelling Problems • Labelling highly structured • Labels highly correlated with very complex dependencies • Independent label estimation too hard

  8. Image Labelling Problems • Labelling highly structured • Labels highly correlated with very complex dependencies • Independent label estimation too hard • Whole labelling should be formulated as one optimisation problem

  9. Image Labelling Problems • Labelling highly structured • Labels highly correlated with very complex dependencies • Independent label estimation too hard • Whole labelling should be formulated as one optimisation problem • Number of pixels up to millions • Hard to train complex dependencies • Optimisation problem is hard to infer

  10. Image Labelling Problems • Labelling highly structured • Labels highly correlated with very complex dependencies • Independent label estimation too hard • Whole labelling should be formulated as one optimisation problem • Number of pixels up to millions • Hard to train complex dependencies • Optimisation problem is hard to infer Vision is hard !

  11. Image Labelling Problems • You can • either • Change the subject from the Computer Vision to the History of Renaissance Art Vision is hard !

  12. Image Labelling Problems • You can • either • Change the subject from the Computer Vision to the History of Renaissance Art • or • Learn everything about Random Fields and Graph Cuts Vision is hard !

  13. Foreground / Background Estimation Rother et al. SIGGRAPH04

  14. Foreground / Background Estimation Data term Smoothness term Data term Estimated using FG / BG colour models Smoothness term where Intensity dependent smoothness

  15. Foreground / Background Estimation Data term Smoothness term

  16. Foreground / Background Estimation Data term Smoothness term • How to solve this optimisation problem?

  17. Foreground / Background Estimation Data term Smoothness term • How to solve this optimisation problem? • Transform into min-cut / max-flow problem • Solve it using min-cut / max-flow algorithm

  18. Overview • Motivation • Min-Cut / Max-Flow (Graph Cut) Algorithm • Markov and Conditional Random Fields • Random Field Optimisation using Graph Cuts • Submodular vs. Non-Submodular Problems • Pairwise vs. Higher Order Problems • 2-Label vs. Multi-Label Problems • Applications • Conclusions

  19. Max-Flow Problem source 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink • Task : • Maximize the flow from the sink to the source such that • 1) The flow it conserved for each node • 2) The flow for each pipe does not exceed the capacity

  20. Max-Flow Problem source 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink

  21. Max-Flow Problem flow from the source source flow from node i to node j 9 5 capacity 4 2 2 set of edges 5 3 2 3 1 1 5 6 3 2 set of nodes conservation of flow 5 6 8 sink

  22. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink

  23. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink

  24. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink flow = 3

  25. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 5-3 4 2+3 2 5 3-3 2 3 1 1 6-3 5 +3 3 2 5 6 8-3 sink flow = 3

  26. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 2 4 5 2 5 2 3 1 1 3 5 3 3 2 5 6 5 sink flow = 3

  27. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 2 4 5 2 5 2 3 1 1 3 5 3 3 2 5 6 5 sink flow = 3

  28. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9 2 4 5 2 5 2 3 1 1 3 5 3 3 2 5 6 5 sink flow = 6

  29. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 9-3 2 4 5 2 5 2 +3 3-3 1 1 3 5 3 3-3 2 +3 5 6 5-3 sink flow = 6

  30. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 6 2 4 5 2 5 2 3 1 1 3 5 3 3 2 5 6 2 sink flow = 6

  31. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 6 2 4 5 2 5 2 3 1 1 3 5 3 3 2 5 6 2 sink flow = 6

  32. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 6 2 4 5 2 5 2 3 1 1 3 5 3 3 2 5 6 2 sink flow = 11

  33. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 6-5 2 4 5 2+5 5-5 2 3 1+5 1 3 5-5 3 3 2 5-5 6 2 sink flow = 11

  34. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2 4 5 7 2 3 1 6 3 3 3 2 6 2 sink flow = 11

  35. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2 4 5 7 2 3 1 6 3 3 3 2 6 2 sink flow = 11

  36. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2 4 5 7 2 3 1 6 3 3 3 2 6 2 sink flow = 13

  37. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2-2 4 5 7 2-2 3 +2 1 6 3 3 +2 3 2-2 6 2-2 sink flow = 13

  38. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 4 5 7 3 2 1 6 3 3 2 3 6 sink flow = 13

  39. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 4 5 7 3 2 1 6 3 3 2 3 6 sink flow = 13

  40. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 4 5 7 3 2 1 6 3 3 2 3 6 sink flow = 15

  41. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 4-2 5 7 3 2 1 3-2 6 3+2 2-2 3 +2 6-2 sink flow = 15

  42. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2 5 7 3 2 1 6 1 5 3 2 4 sink flow = 15

  43. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2 5 7 3 2 1 6 1 5 3 2 4 sink flow = 15

  44. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Find the path from source to sink While (path exists) flow += maximum capacity in the path Build the residual graph (“subtract” the flow) Find the path in the residual graph End 1 2 5 7 3 2 1 6 1 5 3 2 4 sink flow = 15

  45. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Why is the solution globally optimal ? 1 2 5 7 3 2 1 6 1 5 3 2 4 sink flow = 15

  46. Max-Flow Problem S source Ford & Fulkerson algorithm (1956) Why is the solution globally optimal ? 1. Let S be the set of reachable nodes in the residual graph 1 2 5 7 3 2 1 6 1 5 3 2 4 sink flow = 15

  47. Max-Flow Problem S source Ford & Fulkerson algorithm (1956) Why is the solution globally optimal ? 1. Let S be the set of reachable nodes in the residual graph 2. The flow from S to V - S equals to the sum of capacities from S to V – S 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink

  48. Max-Flow Problem A source Ford & Fulkerson algorithm (1956) Why is the solution globally optimal ? 1. Let S be the set of reachable nodes in the residual graph 2. The flow from S to V - S equals to the sum of capacities from S to V – S 3. The flow from any A to V - A is upper bounded by the sum of capacities from A to V – A 9 5 4 2 2 5 3 2 3 1 1 5 6 3 2 5 6 8 sink

  49. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Why is the solution globally optimal ? 1. Let S be the set of reachable nodes in the residual graph 2. The flow from S to V - S equals to the sum of capacities from S to V – S 3. The flow from any A to V - A is upper bounded by the sum of capacities from A to V – A 4. The solution is globally optimal 5/5 8/9 2/4 0/2 0/2 5/5 3/3 2/2 3/3 0/1 0/1 5/6 5/5 3/3 0/2 5/5 2/6 8/8 sink flow = 15 Individual flows obtained by summing up all paths

  50. Max-Flow Problem source Ford & Fulkerson algorithm (1956) Why is the solution globally optimal ? 1. Let S be the set of reachable nodes in the residual graph 2. The flow from S to V - S equals to the sum of capacities from S to V – S 3. The flow from any A to V - A is upper bounded by the sum of capacities from A to V – A 4. The solution is globally optimal 5/5 8/9 2/4 0/2 0/2 5/5 3/3 2/2 3/3 0/1 0/1 5/6 5/5 3/3 0/2 5/5 2/6 8/8 sink flow = 15

More Related