Convex Relaxations

1. Convex Relaxations May 2, 2004 Ben Recht

2. Outline • Lagrangian Duality • Linear Programming and Combinatorics • Non-convex quadratic programming • Positivstellensatz and Polynomial Programming

3. Lagrangian Duality • General Problem

4. Lagrangian Duality • General Problem • Lagrangian:

5. Lagrangian Duality • From Calculus: search for • Lagrangian:

6. Lagrangian Duality • Equivalent Optimization • sup is infinite unless constraints satisfied • Lagrangian:

7. Lagrangian Duality • Equivalent optimization: • Consider • This is the dual problem

8. Visualization f(x) (g(x), h(x)) optimum Search over half spaces containing the epigraph of the function (m,l,1)

9. Visualization f(x*) (m*,l*,1)

10. Visualization f(x*) (m*,l*,1)

11. Visualization duality gap

12. Linear Programming Duality • Lagrangian

13. Linear Programming Duality • Minimize with respect to x

14. Linear Programming Duality • Minimize with respect to x

15. Linear Programming Duality • Minimize with respect to x Either 0 or -1

16. Linear Programming Duality • Minimize with respect to x Either 0 or -1 Independent of x

17. Linear Programming Duality • Form the Dual

18. Linear Programming Duality • Primal • Dual

19. Integer Programming • Primal

20. Integer Programming • Primal • Dual

21. Integer Programming • Primal • Dual the same dual – dual dual is just the LP without integer constraints

22. Integer Programming and Combinatorics • Primal Dual Methods (shortest path, network flows) • Total Unimodularity • Guarantee integer solutions • Total Dual Integrality and Min-max Relations • Prove problem in NPÅcoNP • Branch and Bound, Branch and Cut

23. Quadratic Programming • Problem is convex only when the Ai are positive semidefinite.

24. Nonconvex Quadratically Constrained Quadratic Programs • Consider the general problem: no assumption on definiteness.

25. NCQ2P • Consider the general problem: no assumption on definiteness.

26. NCQ2P • Simplified presentation

27. NCQ2P • Form the Lagrangian

28. NCQ2P • Form the Lagrangian

29. NCQ2P • Form the Lagrangian Taking inf over x gives 0 or -1

30. NCQ2P • Dual • This is a semidefinite program

31. NCQ2P • Dual • Dual Dual

32. NCQ2P SDP Relaxation

33. Dual Dual

34. Dual Dual

35. Dual Dual

36. Dual Dual

37. Dual Dual • Recovered same relaxation • This technique doesn’t generalize (duality does!)

38. Bounding the gap • For Aº0

39. Bounding the gap • For Aº0

40. Bounding the gap • For Aº0

41. Bounding the gap • For Aº0

42. Bounding the gap • For Aº0 • Take x=sign(y), y~N (0,Z*). Then

43. The MAX-CUT Relaxation • Invented by Goemans and Williamson • Guarantees accuracy of 88% for the MAX-CUT problem. An algorithm with accuracy of 95% would prove P=NP. • Specific instance of the “A0” matrix in the relaxation we discussed. • Generalizes to MAX-2-SAT, MAX-SAT, graph coloring, MAX-DICUT, etc.

44. MAX-CUT • Let G=(V,E) be a graph and let w:E!R be an arbitrary function. A cut in the graph is a partition of the vertices into two disjoint sets V1 and V2 such that V1[ V2 = V. Let F(V1) denote the set of edges which have exactly one node in V1. • The weight of the cut is defined w(F) = f2 F w(f) • Problem: find the partition which maximizes w.

45. Graph: G=(V,E) • Maximum-Cut

46. Graph: G=(V,E) • Maximum-Cut

47. Graph: G=(V,E) • Maximum-Cut

48. Graph: G=(V,E) • Maximum-Cut Easy for bipartite graphs. In general, NP-Hard

49. Petersen Graph Classic Counterexample Maximum-Cut = 12

50. Petersen Graph Classic Counterexample Maximum-Cut = 12