C&O 355 Lecture 9

1. C&O 355Lecture 9 N. Harvey http://www.math.uwaterloo.ca/~harvey/ TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAAA

2. Outline • Complementary Slackness • “Crash Course” in Computational Complexity • Review of Geometry & Linear Algebra • Ellipsoids

3. Duality: Geometric View • We can “generate” a new constraint aligned with c by taking a conic combination(non-negative linear combination)of constraints tight at x. • What if we use constraints not tightat x? -x1+x2· 1 x2 x x1 + 6x2· 15 Objective Function c x1

4. Duality: Geometric View • We can “generate” a new constraint aligned with c by taking a conic combination(non-negative linear combination)of constraints tight at x. • What if we use constraints not tightat x? -x1+x2· 1 x2 x Doesn’t prove x is optimal! x1 + 6x2· 15 Objective Function c x1

5. Duality: Geometric View • What if we use constraints not tightat x? • This linear combination is a feasible dual solution,but not anoptimal dual solution • Complementary Slackness: To get an optimal dual solution, must only use constraints tight at x. -x1+x2· 1 x2 x Doesn’t prove x is optimal! x1 + 6x2· 15 Objective Function c x1

6. Weak Duality Primal LP Dual LP Theorem:“Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy. Proof:cT x = (AT y)T x = yT A x ·yT b. ¥ Since y¸0 and Ax·b

7. Weak Duality Primal LP Dual LP Theorem:“Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy. Proof: When does equality hold here? Corollary:If x and y both feasible and cTx=bTy then x and y are both optimal.

8. Weak Duality Primal LP Dual LP Theorem:“Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy. Proof: When does equality hold here? Equality holds for ith term if either yi=0 or

9. Weak Duality Primal LP Dual LP Theorem:“Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy. Proof: Theorem:“Complementary Slackness”Suppose x feasible for Primal, y feasible for dual, and for every i, either yi=0 or .Then x and y are both optimal. Proof: Equality holds here. ¥

10. General ComplementarySlackness Conditions Let x be feasible for primal and y be feasible for dual. for all i,equality holds eitherfor primal or dual and for all j,equality holds eitherfor primal or dual , x and y areboth optimal

11. Example • Primal LP • Challenge: What is the dual? • What are CS conditions? • Claim: Optimal primal solution is (3,0,5/3).Can you prove it?

12. Example Dual LP Primal LP • CS conditions: • Either x1+2x2+3x3=8 or y2=0 • Either 4x2+5x3=2 or y3=0 • Either y1+2y+2+4y3=6 or x2=0 • Either 3y2+5y3=-1 or x3=0 • x=(3,0,5/3) ) y must satisfy: • y1+y2=5y3=0y2+5y3=-1 ) y = (16/3, -1/3, 0)

13. Complementary Slackness Summary • Gives “optimality conditions” that must be satisfied by optimal primal and dual solutions • (Sometimes) gives useful way to compute optimum dual from optimum primal • More about this in Assignment 3 • Extremely useful in “primal-dual algorithms”.Much more of this in • C&O 351: Network Flows • C&O 450/650: Combinatorial Optimization • C&O 754: Approximation Algorithms

14. We’ve now finished C&O 350! • Actually, they will cover 2 topics that we won’t • Revised Simplex Method: A faster implementation of the algorithm we described • Sensitivity Analysis: If we change c or b, how does optimum solution change?

15. Computational Complexity • Field that seeks to understand how efficiently computational problems can be solved • See CS 360, 365, 466, 764… • What does “efficiently” mean? • If problem input has size n, how much time to compute solution? (as a function of n) • Problem can be solved “efficiently” if it can be solved in time ·nc, for some constant c. • P = class of problems that can be solved efficiently.

16. Computational Complexity • P = class of problems that can be solved efficientlyi.e., solved in time ·nc, for some constant c, where n=input size. • Related topic: certificates • Instead of studying efficiency, study how easily can you certify the answer? • NP = class of problems for which you can efficiently certify that “answer is yes” • coNP = class of problems for which you can efficiently certify that “answer is no” • Linear Programs • Can certify that optimal value is large • Can certify that optimal value is small • (by giving primal solution x) • (by giving dual solution y)

17. Computational Complexity NP coNP Can graph be coloredwith ¸ k colors? Does every coloringof graph use · k colors? • Open Problem: Is P=NP? • Probably not • One of the 7 most important problems in mathematics • You win $1,000,000 if you solve it. NPÅcoNP Is LP value ¸ k? Manyinterestingproblems Manyinterestingproblems Is m a prime number? P Sorting, string matching, breadth-first search, …

18. Computational Complexity NP coNP Can graph be coloredwith ¸ k colors? Does every coloringof graph use · k colors? • Open Problem: Is P=NPÅcoNP? • Maybe… for most problems in NPÅcoNP, they are also in P. • “Is m a prime number?”. Proven in P in 2002. “Primes in P” • “Is LP value ¸ k?”. Proven in P in 1979. “Ellipsoid method” NPÅcoNP Is LP value ¸ k? Manyinterestingproblems Manyinterestingproblems Is m a prime number? P Sorting, string matching, breadth-first search, … Leonid Khachiyan

19. Review • Basics of Euclidean geometry • Euclidean norm, unit ball, affine maps, volume • Positive Semi-Definite Matrices • Square roots • Ellipsoids • Rank-1 Updates • Covering Hemispheres by Ellipsoids

20. 2D Example Ellipsoid T(B) Unit ball B