1 / 71

Convex Programming

Convex Programming. Brookes Vision Reading Group. Huh?. What is convex ??? What is programming ??? What is convex programming ???. Huh?. What is convex ??? What is programming ??? What is convex programming ???. Convex Function. f(t x + (1-t) y) <= t f(x) + (1-t) f(y). Convex Function.

vartouhi
Download Presentation

Convex Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Convex Programming Brookes Vision Reading Group

  2. Huh? • What is convex ??? • What is programming ??? • What is convex programming ???

  3. Huh? • What is convex ??? • What is programming ??? • What is convex programming ???

  4. Convex Function f(t x + (1-t) y) <= t f(x) + (1-t) f(y)

  5. Convex Function Is a linear function convex ???

  6. Convex Set Region above a convex function is a convex set.

  7. Convex Set Is the set of all positive semidefinite matrices convex??

  8. Huh? • What is convex ??? • What is programming ??? • What is convex programming ???

  9. Objective function Example Constraints Programming • Objective function to be minimized/maximized. • Constraints to be satisfied.

  10. Example Optimal solution Vertices Objective function Feasible region

  11. Huh? • What is convex ??? • What is programming ??? • What is convex programming ???

  12. Convex Programming • Convex optimization function • Convex feasible region • Why is it so important ??? • Global optimum can be found in polynomial time. • Many practical problems are convex • Non-convex problems can be relaxed to convex ones.

  13. Convex Programming • Convex optimization function • Convex feasible region • Examples ??? • Linear Programming • Refer to Vladimir/Pushmeet’s reading group • Second Order Cone Programming • What ??? • Semidefinite Programming • All this sounds Greek and Latin !!!!

  14. Outline • Convex Optimization • Second Order Cone Programming (SOCP) • Semidefinite Programming (SDP) • Non-convex optimization • SDP relaxations • SOCP relaxations • Optimization Algorithms • Interior Point Method for SOCP • Interior Point Method for SDP 2 out of 3 is not bad !!!

  15. Outline • Convex Optimization • Second Order Cone Programming (SOCP) • Semidefinite Programming (SDP) • Non-convex optimization • SDP relaxations • SOCP relaxations • Optimization Algorithms • Interior Point Method for SOCP • Interior Point Method for SDP

  16. Second Order Cone • || u || < t • u - vector of dimension ‘d-1’ • t - scalar • Cone lies in ‘d’ dimensions • Second Order Cone defines a convex set • Example: Second Order Cone in 3D x2 + y2 <= z2

  17. Hmmm ICE CREAM !! x2 + y2 <= z2

  18. Linear Objective Function Affine mapping of SOC Second Order Cone Programming Minimize fTx Subject to || Ai x+ bi || <= ciT x + di i = 1, … , L Constraints are SOC of ni dimensions Feasible regions are intersections of conic regions

  19. Example

  20. Why SOCP ?? • A more general convex problem than LP • LP  SOCP • Fast algorithms for finding global optimum • LP - O(n3) • SOCP - O(L1/2) iterations of O(n2∑ni) • Many standard problems are SOCP-able

  21. SOCP-able Problems • Convex quadratically constrained quadratic programming • Sum of norms • Maximum of norms • Problems with hyperbolic constraints

  22. SOCP-able Problems • Convex quadratically constrained quadratic programming • Sum of norms • Maximum of norms • Problems with hyperbolic constraints

  23. QCQP Minimize xT P0 x + 2 q0T x + r0 Subject to xT Pi x + 2 qiT x + ri Pi >= 0 || P01/2 x + P0-1/2 x ||2 + r0 -q0TP0-1p0

  24. QCQP Minimize xT P0 x + 2 q0T x + r0 Subject to xT Pi x + 2 qiT x + ri Minimize t Subject to || P01/2 x + P0-1/2 x || < = t || P01/2 x + P0-1/2 x || < = (r0 -q0TP0-1p0)1/2

  25. SOCP-able Problems • Convex quadratically constrained quadratic programming • Sum of norms • Maximum of norms • Problems with hyperbolic constraints

  26. Sum of Norms Minimize  || Fi x + gi || Minimize  ti Subject to || Fi x + gi || <= ti Special Case: L-1 norm minimization

  27. SOCP-able Problems • Convex quadratically constrained quadratic programming • Sum of norms • Maximum of norms • Problems with hyperbolic constraints

  28. Maximum of Norms Minimize max || Fi x + gi || Minimize t Subject to || Fi x + gi || <= t Special Case: L-inf norm minimization

  29. You weren’t expecting a question, were you ??

  30. SOCP-able Problems • Convex quadratically constrained quadratic programming • Sum of norms • Maximum of norms • Problems with hyperbolic constraints

  31. Hyperbolic Constraints x >= 0 , y >= 0 w2 <= xy || [2w; x-y] || <= x+y

  32. Let’s see if everyone was awake !

  33. Outline • Convex Optimization • Second Order Cone Programming (SOCP) • Semidefinite Programming (SDP) • Non-convex optimization • SDP relaxations • SOCP relaxations • Optimization Algorithms • Interior Point Method for SOCP • Interior Point Method for SDP

  34. Linear Objective Function Linear Constraints Semidefinite Programming Minimize C  X Subject to Ai X = bi X >= 0 Linear Programming on Semidefinite Matrices

  35. Why SDP ?? • A more general convex problem than SOCP • LP  SOCP  SDP • Generality comes at a cost though • SOCP - O(L1/2) iterations of O(n2∑ni) • SDP - O((∑ni)1/2) iterations of O(n2∑ni2) • Many standard problems are SDP-able

  36. SDP-able Problems • Minimizing the maximum eigenvalue • Class separation with ellipsoids

  37. SDP-able Problems • Minimizing the maximum eigenvalue • Class separation with ellipsoids

  38. Minimizing the Maximum Eigenvalue Matrix M(z) To find vector z* such that max is minimized. Let max(M(z)) <= n max(M(z)-nI) <= 0 min(nI - M(z)) >= 0 nI - M(z) >= 0

  39. Minimizing the Maximum Eigenvalue Matrix M(z) To find vector z* such that max is minimized. Max -n nI - M(z) >= 0

  40. SDP-able Problems • Minimizing the maximum eigenvalue • Class separation with ellipsoids

  41. Outline • Convex Optimization • Second Order Cone Programming (SOCP) • Semidefinite Programming (SDP) • Non-convex optimization • SDP relaxations • SOCP relaxations • Optimization Algorithms • Interior Point Method for SOCP • Interior Point Method for SDP

  42. Non-Convex Problems Minimize xTQ0x + 2q0Tx + r0 Subject to xTQix + 2qiTx + ri < = 0 Qi >= 0 => Convex Non-Convex Quadratic Programming Problem !!! Redefine x in homogenous coordinates. y = (1; x)

  43. Let’s solve this now !!! Non-Convex Problems Minimize xTQ0x + 2q0Tx + r0 Subject to xTQix + 2qiTx + ri < = 0 Minimize yTM0y Subject to yTMiy < = 0 Mi = [ ri qiT; qi Qi]

  44. Non-Convex Problems • Problem is NP-hard. • Let’s relax the problem to make it convex. • Pray !!!

  45. Outline • Convex Optimization • Second Order Cone Programming (SOCP) • Semidefinite Programming (SDP) • Non-convex optimization • SDP relaxations • SOCP relaxations • Optimization Algorithms • Interior Point Method for SOCP • Interior Point Method for SDP

  46. Bad Constraint !!!! No donut for you !!! SDP Relaxation Minimize yTM0y Subject to yTMiy < = 0 Minimize M0 Y Subject to Mi  Y < = 0 Y = yyT

  47. SDP Problem SDP Relaxation Minimize yTM0y Subject to yTMiy < = 0 Minimize M0 Y Subject to Mi  Y < = 0 Y >= 0 Nothing left to do …. but Pray Note that we have squared the number of variables.

  48. Example - Max Cut • Graph: G=(V,E) • Maximum-Cut

More Related