Discrete Optimization Lecture 1

1. Discrete OptimizationLecture 1 M. Pawan Kumar pawan.kumar@ecp.fr

2. Background • Ph.D., Computing • Oxford Brookes University, UK, 2003-2007 • Postdoc, Department of Engineering • University of Oxford, UK, 2008 • Postdoc, Computer Science Department • Stanford University, USA, 2009-2011 • Optimization and Learning for Computer Vision

3. Image Segmentation How ? Cost function Models our knowledge about natural images Optimize cost function to obtain the segmentation

4. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Each vertex corresponds to a pixel Edges define a 4-neighbourhood grid graph Assign a label to each vertex from L = {obj,bkg}

5. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Vertex Cost Cost of a labelling f : V  L Cost of label ‘bkg’ high Cost of label ‘obj’ low

6. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Vertex Cost Cost of a labelling f : V  L Cost of label ‘bkg’ low Cost of label ‘obj’ high UNARY COST

7. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Edge Cost Cost of a labelling f : V  L Cost of same label low Cost of different labels high

8. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Per Edge Cost Cost of a labelling f : V  L Cost of same label high PAIRWISE COST Cost of different labels low

9. Image Segmentation Graph G = (V,E) Object - white, Background - green/grey Problem: Find the labelling with minimum cost f*

10. Image Segmentation Graph G = (V,E) Problem: Find the labelling with minimum cost f*

11. Object Detection How ? Once again, by defining a good cost function

12. Object Detection H T 1 L1 L2 L3 L4 Graph G = (V,E) Each vertex corresponds to a part - ‘Head’, ‘Torso’, ‘Legs’ Edges define a TREE Assign a label to each vertex from L = {positions}

13. Object Detection H T 2 L1 L2 L3 L4 Graph G = (V,E) Each vertex corresponds to a part - ‘Head’, ‘Torso’, ‘Legs’ Edges define a TREE Assign a label to each vertex from L = {positions}

14. Object Detection H T 3 L1 L2 L3 L4 Graph G = (V,E) Each vertex corresponds to a part - ‘Head’, ‘Torso’, ‘Legs’ Edges define a TREE Assign a label to each vertex from L = {positions}

15. Object Detection H T 3 L1 L2 L3 L4 Graph G = (V,E) Cost of a labelling f : V  L Unary cost : How well does part match image patch? Pairwise cost : Encourages valid configurations Find best labelling f*

16. Object Detection H T 3 L1 L2 L3 L4 Graph G = (V,E) Cost of a labelling f : V  L Unary cost : How well does part match image patch? Pairwise cost : Encourages valid configurations Find best labelling f*

17. General Problem MAP Estimation f* = argminfE(f; ) E(f; ) = ∑u u(f(u))+ ∑(u,v) uv(f(u),f(v)) • Expensive to optimize exactly • Analysis of existing approximate solutions • Novel, accurate approximate solutions

18. Learning Given …. “Car”

19. Learning …. Estimate θ such that f* = argminfE(f; ) E(f; ) = ∑u u(f(u))+ ∑(u,v) uv(f(u),f(v))

20. Syllabus • Shortest Path Algorithms • NP-Complete Problems • Minimum Spanning Tree Algorithms • MAP Estimation for Trees • Minimum Cut, Maximum Flow, MAP Estimation http://ai.stanford.edu/~pawan/teaching/optimization.html

21. Schedule • Five lectures • Friday 10th February, 2012. 8:00am – 11:15am • Monday 27th February, 2012. 9:45am – 11:15am • Monday 5th March, 2012. 9:45am – 1:00pm • Friday 9th March, 2012. 8:00am – 9:30am • Monday 12th March, 2012. 9:45am – 1:00pm • Two lab sessions • Monday 27th February, 2012. 11:30am – 1:00pm • Friday 9th March, 2012. 9:45am – 11:15am • One exam • Monday 19th March, 2012. Time: TBA

22. Grading • Groups of two students for assignments • Grades based on reports • Honor code • Two lab sessions • Monday 27th February, 2012. 11:30am – 1:00pm • Friday 9th March, 2012. 9:45am – 11:15am 40% • One exam • Monday 19th March, 2012. Time: TBA 60%

23. During the Class … Better to ask twice than to lose your way once.

24. Outline • Graph Preliminaries • Undirected Graphs • Directed Graphs • Complexity Preliminaries • Shortest Path Algorithms

25. Undirected Graphs v0 G = (V, E) v1 v4 v2 v5 v3 v6 ‘n’ vertices or nodes V ‘m’ edges E: unordered pairs from V

26. Neighboring or Adjacent Vertices v0 G = (V, E) v1 v4 v2 v5 v3 v6 Connected by an edge e = (u,v) = (v,u). ‘v0’ and ‘v1’ adjacent, ‘v0’ and ‘v5’ not adjacent, …

27. Parallel Edges v0 G = (V, E) v1 v4 v2 v5 v3 v6 Represented by the same pair of vertices.

28. Loops v0 G = (V, E) v1 v4 v2 v5 v3 v6 Edges that connect a vertex to itself.

29. Simple Graphs v0 G = (V, E) v1 v4 v2 v5 v3 v6 Graphs without parallel edges and without loops.

30. Degree of a Vertex v0 G = (V, E) v1 v4 v2 v5 v3 v6 Number of edges incident on the vertex. deg(v0) = 2, deg(v1) = 2, deg(v4) = 3, …

31. Regular Graph v0 G = (V, E) v1 v4 v2 v5 v3 v6 Degrees of all vertices are equal.

32. Complete Graph v0 G = (V, E) v1 v4 v2 v5 Kn v3 v6 Graph is simple. Any two distinct vertices are adjacent.

33. Are All Complete Graphs Regular? G = (V, E) v4 v2 v5 YES Degree of a vertex in Kn? n-1

34. Walk v0 G = (V, E) v1 v4 v2 v5 v3 v6 Sequence P = (v0,e1,v1,…,ek,vk), ei = (vi-1,vi) v0, (v0,v4), v4, (v4,v2), v2, (v2,v5), v5, (v5,v4), v4

35. Path v0 G = (V, E) v1 v4 v2 v5 v3 v6 Sequence P = (v0,e1,v1,…,ek,vk), ei = (vi-1,vi) Vertices v0,v1,…,vk are distinct

36. Length of a Walk v0 G = (V, E) v1 v4 Odd Walk Even Walk v2 v5 v3 v6 Number of edges: k Length of above walk = 5

37. Length of a Walk v0 G = (V, E) 4 2 v1 v4 l: E R 2 1 1 5 v2 v5 1 -2 v3 v6 4 Sum of lengths of all edges: Σil(ei) Length of above walk = 4+2+5-2+4 = 13

38. Closed Walk v0 G = (V, E) v1 v4 v2 v5 v3 v6 Sequence P = (v0,e1,v1,…,ek,vk), ei = (vi-1,vi) v0 = vk

39. Circuit v0 G = (V, E) v1 v4 1-circuit? 2-circuit? v2 v5 v3 v6 Sequence P = (v0,e1,v1,…,ek,vk), ei = (vi-1,vi) v0 = vk Vertices v0,v1,…,vk are distinct

40. Outline • Graph Preliminaries • Undirected Graphs • Directed Graphs • Complexity Preliminaries • Shortest Path Algorithms

41. Directed Graphs (Digraphs) v0 D = (V, A) v1 v4 v2 v5 v3 v6 ‘n’ vertices or nodes V ‘m’ arcs A: ordered pairs from V

42. Neighboring or Adjacent Vertices v0 D = (V, A) v1 v4 v2 v5 v3 v6 Connected by an arc a = (u,v). ‘v0’ is the inneighbor of ‘v4’

43. Neighboring or Adjacent Vertices v0 D = (V, A) v1 v4 v2 v5 v3 v6 Connected by an arc a = (u,v). ‘v4’ is the outneighbor of ‘v0’

44. Parallel Arcs v0 D = (V, A) v1 v4 v2 v5 v3 v6 Represented by the same ordered pair of vertices.

45. Loops v0 D = (V, A) v1 v4 v2 v5 v3 v6 Arcs that connect a vertex to itself.

46. Simple Graphs v0 D = (V, A) v1 v4 v2 v5 v3 v6 Graphs without parallel arcs and without loops.

47. Underlying Undirected Graph v0 D = (V, A) v1 v4 v2 v5 v3 v6 Graphs obtained by ignoring orientation of arcs.

48. Underlying Undirected Graph v0 G = (V, E) v1 v4 v2 v5 v3 v6 Graphs obtained by ignoring orientation of arcs.

49. Indegree of a Vertex v0 D = (V, A) v1 v4 v2 v5 v3 v6 Number of arcs entering the vertex. indeg(v0) = 1, indeg(v1) = 1, indeg(v4) = 2, …

50. Outdegree of a Vertex v0 D = (V, A) v1 v4 v2 v5 v3 v6 Number of arcs leaving the vertex. outdeg(v0) = 1, outdeg(v1) = 1, outdeg(v2) = 2, …