C&O 355 Lecture 19

1. C&O 355Lecture 19 N. Harvey TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAA

2. Topics • Solving Integer Programs • Basic Combinatorial Optimization Problems • Bipartite Matching, Minimum s-t Cut,Shortest Paths, Minimum Spanning Trees • Bipartite Matching • Combinatorial Analysis of Extreme Points • Total Unimodularity

3. Mathematical Programs We’ve Seen • Linear Program (LP) • Convex Program • Semidefinite Program (SDP) • Integer Program (IP) (where f is convex) Can be efficiently solvede.g., by Ellipsoid Method (where X is symmetric matrixcorresponding to x ) Cannot be efficiently solvedassuming P  NP

4. Computational Complexity NP coNP • If you could efficiently (i.e., in polynomial time) decide if every integer program is feasible, then P = NP • And all of modern cryptography is broken • And you win $1,000,000 • … Can graph be coloredwith · k colors? Does every coloringof graph use ¸ k colors? NPÅcoNP Is integer programfeasible? Is integer programinfeasible? P Sorting, string matching, breadth-first search, … Is LP feasible?

5. Combinatorial IPs are often nice • Maximum Bipartite Matching(from Lecture 2) • Given bipartite graph G=(V, E) • Find a maximum size matching • A set Mµ E s.t. every vertex has at most one incident edge in M

6. Combinatorial IPs are often nice • Maximum Bipartite Matching(from Lecture 2) • Given bipartite graph G=(V, E) • Find a maximum size matching • A set Mµ E s.t. every vertex has at most one incident edge in M The blue edges are a matching M

7. Combinatorial IPs are often nice • Maximum Bipartite Matching(from Lecture 2) • Given bipartite graph G=(V, E) • Find a maximum size matching • A set Mµ E s.t. every vertex has at most one incident edge in M • The natural integer program • This IP can be efficiently solved, in many different ways

8. Combinatorial IPs are often nice • Max-Weight Perfect Matching • Given bipartite graph G=(V, E). Every edge e has a weight we. • Find a maximum-weight perfect matching • A set Mµ E s.t. every vertex has exactly one incident edge in M 3 1 2 4 2 5 2 1

9. Combinatorial IPs are often nice • Max-Weight Perfect Matching • Given bipartite graph G=(V, E). Every edge e has a weight we. • Find a maximum-weight perfect matching • A set Mµ E s.t. every vertex has exactly one incident edge in M 3 1 2 The blue edges are a max-weight perfect matching M 4 2 5 2 1

10. Combinatorial IPs are often nice • Max-Weight Perfect Matching • Given bipartite graph G=(V, E). Every edge e has a weight we. • Find a maximum-weight perfect matching • A set Mµ E s.t. every vertex has exactly one incident edge in M • The natural integer program • This IP can be efficiently solved, in many different ways

11. Combinatorial IPs are often nice • Minimum s-t Cut in a Graph(from Lecture 12) • Let G=(V,E) be a graph. Fix two vertices s,t2V. • An s-t cut is a set FµE such that, if you delete F, then s and t are disconnectedi.e., there is no s-t path in G\F = (V,E\F). t s

12. Combinatorial IPs are often nice • Minimum s-t Cut in a Graph(from Lecture 12) • Let G=(V,E) be a graph. Fix two vertices s,t2V. • An s-t cut is a set FµE such that, if you delete F, then s and t are disconnectedi.e., there is no s-t path in G\F = (V,E\F). t s These edges are a minimum s-t cut

13. Combinatorial IPs are often nice • Minimum s-t Cut in a Graph(from Lecture 12) • Let G=(V,E) be a graph. Fix two vertices s,t2V. • An s-t cut is a set FµE such that, if you delete F, then s and t are disconnected. • Want to find an s-t cut of minimum cardinality • Write a (very big!) integer program. Make variable xe for every e 2 E. Let P be set of all s-t paths. • This IP can be efficiently solved, in many different ways

14. Combinatorial IPs are often nice • Shortest Paths in a Digraph • Let G=(V,A) be a directed graph. Every arc a has a “length” wa¸0. • Given two vertices s and t,find a path from s to t of minimum total length. 3 4 2 7 8 t 1 2 5 1 s 2 2 2 3 1 1

15. Combinatorial IPs are often nice • Shortest Paths in a Digraph • Let G=(V,A) be a directed graph. Every arc a has a “length” wa¸0. • Given two vertices s and t,find a path from s to t of minimum total length. 3 4 2 7 8 t 1 2 5 1 s 2 2 2 3 1 1 These edges form a shortest s-t path

16. Combinatorial IPs are often nice • Shortest Paths in a Digraph • Let G=(V,A) be a directed graph. Every arc a has a “length” wa¸0. • Given two vertices s and t,find a path from s to t of minimum total length. • There is a natural IP for this problem that can be efficiently solved, in many different ways.

17. Combinatorial IPs are often nice • Minimum Spanning Tree in a Graph • Let G=(V,E) be a graph. Every edge e has a weight we. • An spanning treeis a set FµE with no cycles, such that F contains a path between every pair of vertices. 3 4 2 7 8 1 2 5 1 2 8 2 3 1 1

18. Combinatorial IPs are often nice • Minimum Spanning Tree in a Graph • Let G=(V,E) be a graph. Every edge e has a weight we. • An spanning treeis a set FµE with no cycles, such that F contains a path between every pair of vertices. 3 4 2 7 8 t 1 2 5 1 s 2 8 2 3 1 1 These edges are a minimumspanning tree There is an s-t path in the tree

19. Combinatorial IPs are often nice • Minimum Spanning Tree in a Graph • Let G=(V,E) be a graph. Every edge e has a weight we. • An spanning treeis a set FµE with no cycles, such that F contains a path between every pair of vertices. • There is a natural IP for this problem that can be efficiently solved, in many different ways.

20. How to solve combinatorial IPs? • Two common approaches • Design combinatorial algorithm that directly solves IP • Often such algorithms have a nice LP interpretation • Relax IP to an LP; prove that they give same solution; solve LP by the ellipsoid method • Need to show special structure of the LP’s extreme points • Sometimes we can analyze the extreme points combinatorially • Sometimes we can use algebraic structure of the constraints.For example, if constraint matrix is Totally Unimodularthen IP and LP are equivalent • We’ll see examples of these approaches

21. Perfect Matching Problem • Let G=(V, E) be a bipartite graph. Every edge e has a weight we. • Find a maximum-weight, perfect matching • A set MµE s.t. every vertex has exactly one incident edge in M • Write an integer program (IP) • Relax integrality constraints, obtain an LP (LP) (xe·1 is implicit) c • Theorem: Every BFS of (LP) is actually an (IP) solution!

22. Combinatorial Analysis of BFSs • Lemma:Every BFS of perfect matching (LP) is an (IP) solution. • Proof: Let x be BFS, suppose x not integral. • Pick any edge e1={v0,v1} with 0 < xe1 < 1. • The LP requires) there is another edge e2={v1,v2} with 0 < xe2 < 1. • The LP requires) there is another edge e3={v2,v3} with 0 < xe3 < 1. • Continue finding distinct edges until eventually vi=vk,i<k • We have ei+1={vi,vi+1}, ei+2={vi+1,vi+2}, …, ek={vk-1,vk}.(all edges and vertices distinct, except vi=vk)

23. Combinatorial Analysis of BFSs • Let x be BFS of matching (LP). Suppose x not integral. • WLOG, e1={v0,v1}, e2={v1,v2}, …, ek={vk-1,vk} and v0=vk. • These edges form a simple cycle, of even length. (Even length since G is bipartite.) • Define the vector: • Claim: If |²| is sufficiently small, then x+²d is feasible • So x is convex combination of x+²d and x-²d, both feasible • This contradicts x being a BFS. ¥

24. How to solve combinatorial IPs? • Two common approaches • Design combinatorial algorithm that directly solves IP • Often such algorithms have a nice LP interpretation • Relax IP to an LP; prove that they give same solution; solve LP by the ellipsoid method • Need to show special structure of the LP’s extreme points • Sometimes we can analyze the extreme points combinatorially • Sometimes we can use algebraic structure of the constraints.For example, if constraint matrix is Totally Unimodularthen IP and LP are equivalent

25. LP Approach for Bipartite Matching • Let G=(V, E) be a bipartite graph. Every edge e has a weight we. • Find a maximum weight matching • A set Mµ E s.t. every vertex has at most one incident edge in M • Write an integer program (IP) • Relax integrality constraints, obtain an LP (LP) (xe·1 is implicit) c • Theorem: Every BFS of (LP) is actually an (IP) solution!

26. Total Unimodularity • Let A be a real mxn matrix • Definition: Suppose that every square submatrix of A has determinant in {0, +1, -1}. Then A is totally unimodular (TUM). • In particular, every entry of A must be in {0, +1, -1} • Lemma:Suppose A is TUM. Let b be any integer vector. Then every basic feasible solution of P = { x : Ax·b } is integral. • Proof: Let x be a basic feasible solution. Then the constraints that are tight at x have rank n. Let A’ be a submatrix of A and b’ a subvector of b corresponding to n linearly independent constraints that are tight at x. Then x is the unique solution to A’ x = b’, i.e., x = (A’)-1 b’. Cramer’s Rule: If M is a square, non-singular matrix then(M-1)i,j = (-1)i+jdetMdel(j,i) / det M. Submatrix of M obtained by deleting row j and column i

27. Total Unimodularity • Let A be a real mxn matrix • Definition: Suppose that every square submatrix of A has determinant in {0, +1, -1}. Then A is totally unimodular (TUM). • Lemma:Suppose A is TUM. Let b be any integer vector. Then every basic feasible solution of P = { x : Ax·b } is integral. • Proof: Let x be a basic feasible solution. Then the constraints that are tight at x have rank n. Let A’ be the submatrix of A and b’ the subvector of b containing n linearly independent constraints that are tight at x. Then x is the unique solution to A’ x = b’, i.e., x = (A’)-1 b’. Cramer’s Rule: If M is a square, non-singular matrix then(M-1)i,j = (-1)i+jdetMdel(j,i) / det M. Thus all entries of (A’)-1 are in {0, +1, -1}. Since b’ is integral, x is also integral. ¥

28. Operations Preserving Total Unimodularity • Let A be a real mxn matrix • Definition: Suppose that every square submatrix of A has determinant in {0, +1, -1}. Then A is totally unimodular (TUM). • Lemma:Suppose A is TUM. Let b be any integer vector. Then every basic feasible solution of P = { x : Ax·b } is integral. • Claim: Suppose A is TUM. Then is also TUM. • Proof: Exercise on Assignment 5. ¤ • Corollary: Suppose A is TUM. Let b be any integer vector. Then every basic feasible solution of P = { x : Ax·b, x¸0 } is integral. • Proof: By the Claim, is TUM. So apply the Lemma to . ¥

29. Bipartite Matching & Total Unimodularity • Let G=(U[V, E) be a bipartite graph. • So all edges have one endpoint in U and the other in V. • Let A be the “incidence matrix” of G.A has a row for every vertex and a column for every edge. Note: Every column of A has exactly two non-zero entries. • Lemma: A is TUM. • Proof: Let Q be a kxksubmatrix of A. Argue by induction on k. If k=1 then Q is a single entry of A, so det(Q) is either 0 or 1. Suppose k>1. If some column of Q has no non-zero entries, then det(Q)=0. • 1 if vertex w is an endpoint of edge e • Aw,e = • 0 otherwise

30. 1 if vertex v is an endpoint of edge e • Av,e = • Let G=(U[V, E) be a bipartite graph. Define A by • Lemma: A is TUM. • Proof: Let Q be a kxksubmatrix of A. Assume k>1. If some column of Q has no non-zero entries, then det(Q)=0. Suppose jth column of Q has exactly one non-zero entry, say Qt,j0 Use “Column Expansion” of determinant: , where t is the unique non-zero entry in column j. By induction, detQdel(t,j) in {0,+1,-1} )det Q in {0,+1,-1}. • 0 otherwise

31. 1 if vertex v is an endpoint of edge e • Av,e = • Let G=(U[V, E) be a bipartite graph. Define A by • Lemma: A is TUM. • Proof: Let Q be a kxksubmatrix of A. Assume k>1. If some column of Q has no non-zero entries, then det(Q)=0. If jth column of Q has exactly one non-zero entry, use induction. Suppose every column of Q has exactly two non-zero entries. • For each column, one non-zero is in a U-row and the other is in a V-row. So summing all U-rows in Q gives the vector [1,1,…,1]. Also summing all V-rows in Q gives the vector [1,1,…,1]. So (sum of U-rows) – (sum of V-rows) = [0,0,…,0]. Thus Q is singular, and det Q = 0. ¥ • 0 otherwise

32. 1 if vertex v is an endpoint of edge e • Av,e = • Let G=(U[V, E) be a bipartite graph. Define A by • Lemma: A is TUM. • So every BFS of P = { x : Ax·1, x¸0 } is integral. • We can rewrite the LP max { wTx : x2P } as • For every objective function w, this LP has an optimal solution at a BFS. (Since P is bounded) • So for every vector w, the LP has an integraloptimal solution x. • Since 0·xe·1, and x is integral, we actually have xe2 {0,1}. • So every optimal LP solution is actually an (optimal) IP solution.) So we can solve the IP by solving the LP and returning a BFS. • 0 otherwise (xe·1 is implicit)

33. How to solve combinatorial IPs? • Two common approaches • Design combinatorial algorithm that directly solves IP • Often such algorithms have a nice LP interpretation • Relax IP to an LP; prove that they give same solution; solve LP by the ellipsoid method • Need to show special structure of the LP’s extreme points • Sometimes we can analyze the extreme points combinatorially • Sometimes we can use algebraic structure of the constraints.For example, if constraint matrix is Totally Unimodularthen IP and LP are equivalent