Bi-Parametric Convex Quadratic Optimization

1. Bi-Parametric Convex Quadratic Optimization Tamás Terlaky Lehigh University Joint work with Alireza Ghaffari-Hadigheh and Oleksandr Romanko RUTCOR 2009: Dedicated to the 80th Birthday of Professor András Prékopa

3. Outline • Introduction • Quadratic optimization, optimal partition • Uni-Parametric quadratic optimization • Bi-Parametric quadratic optimization • Numerical illustration • Fundamental properties • Algorithm • Conclusions and future work

4. Introduction: Parametric Optimization General framework of parametric optimization • Multidimensional parameter is introduced into objective function and/or constraints • The goal is to find • – optimal solution • – optimal value function • Generalization of sensitivity analysis • Applications: multi-objective optimization

5. Multi-objective optimization: OBJECTIVE SPACE f2 identify Pareto frontier (all non-dominated solutions) f*2 f1 f*1 Introduction: Multi-Objective Optimization as Parametric Problem • Multi-objective optimization with weighting method: • Parametric formulation:

6. Bi-parametric QO generalizes three models: uni-parametric QO • Bi-Parametric Convex Quadratic Optimization (PQO) problem: Introduction: Quadratic Optimization and Its Parametric Counterpart • Convex Quadratic Optimization (QO) problem:

7. Sensity Analysis: Just be careful!

8. Optimal Partition for QO Primal Dual • Convex Quadratic Optimization problems: • Optimality conditions: • Maximally complementary solution: • LO: and - strictly complementary solution • QO: , but may not hold • maximally complementary solution maximizes the number • of non-zero coordinates in and IPMs !!!

9. Optimal Partition for QO • The optimalpartition of the index set {1, 2,…, n} is The optimalpartition is unique!!! • An optimal solution is maximally complementary iff: • Example: for maximally complementary solution with:

10. Uni-Parametric Quadratic Optimization • Primal and dual perturbed problems: • For some we are given the maximally complementary optimal solution of and with the optimal partition . • The left and right extreme points of the invariancy interval: - invariancy interval - transition points

11. Gengyang Gengyang Gengyang (1) Solve two auxiliary problems (2) Uni-Parametric QO: Optimal Partition in the Neighboring Invariancy Interval • How to proceed from the current invariancy interval to the next one? z z z

12. Uni-param QO: NumericalIllustration Solver output type lu B N T () ----------------------------------------------------------------------------------------------------------------- transition point -8.00000 -8.00000 3 5 1 4 2 -0.00 invariancy interval -8.00000 -5.00000 2 3 5 1 4 8.502 + 68.00 + 0.00 transition point -5.00000 -5.00000 2 1 3 4 5 -127.50 invariancy interval -5.00000 +0.00000 1 2 3 4 5 4.002 + 35.50 - 50.00 transition point +0.00000 +0.00000 1 2 3 4 5 -50.00 invariancy interval +0.00000 +1.73913 1 2 3 4 5 -6.912 + 35.50 - 50.00 transition point +1.73913 +1.73913 2 3 4 5 1 -9.15 invariancy interval +1.73913 +3.33333 2 3 4 5 1 -3.602 + 24.00 - 40.00 transition point +3.33333 +3.33333 3 4 5 1 2 0.00 invariancy interval +3.33333 Inf 3 4 5 1 2 0.002 - 0.00 + 0.00

13. Bi-Parametric Quadratic Optimization • Primal and dual perturbed problems: • Invariancy regions instead of invariancy intervals • Illustrative example:

14. Bi-Parametric Quadratic Optimization • Illustrative example Invariancy regions

15. Bi-Parametric Quadratic Optimization • Illustrative example: Optimal value function

16. Optimal partition is constant on invariancy regions. • Invariancy regions that are transition lines or singletons are called trivial regions. Otherwise, they are called non-trivial invariancy regions. • Invariancy region is a convex set and its closure is a polyhedron that might be unbounded. • The optimal value function is a bivariate quadratic function on invariancy region : Bi-Parametric Quadratic Optimization • The optimal value function is continuous and piecewise bivariate quadratic • The boundary of a non-trivial invariancy region consists of a finite number of line segments.

17. where Bi-Parametric QO: Algorithm • Idea: reduce bi-parametric QO problem to a series of uni-paramteric QO problems with

18. Start from , determine the optimal partition • Choose , and • Solve where • Solve where • Now, two points and on the boundary of the invariancy region are known • Consider cases and Bi-Parametric QO: Algorithm

19. Bi-Parametric QO: Algorithm • Case

20. : and Bi-Parametric QO: Algorithm • Case • Case

21. : and • : and Bi-Parametric QO: Algorithm • Case • Case

22. : and • : and • : back to the first or the second case Bi-Parametric QO: Algorithm • Case • Case

23. Bi-Parametric QO: Algorithm • Invariancy region exploration

24. vertex edge Bi-Parametric QO: Algorithm • Enumerating all invariancy regions To-be-processed queue Completed queue cell

25. Conclusions and Future Work • Developed an IPM-based technique for solving bi-parametric problems that • extends the results of the uni-parametric case • allows solving both bi-parametric linear and bi-parametric quadratic optimization problems • systematically explores the optimal value surface • Polynomial-time algorithm in the output size • Applications in finance, IMRT, data mining • Improving the implementation • Extending methodology to • Parametric Second Order Conic Optimization • Multi-Parametric Quadratic Optimization

26. References • A. B. Berkelaar, C. Roos, and T. Terlaky. The optimal set and optimal partition approach to linear and quadratic programming. In Advances in Sensitivity Analysis and Parametric Programming, T. Gal and H. J. Greenberg, eds., Kluwer, Boston, USA, 1997. • A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Sensitivity Analysis in Convex Quadratic Optimization: Simultaneous Perturbation of the Objective and Right-Hand-Side Vectors. Algorithmic Operations Research, Vol. 2(2), 2007. • A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. Bi-Parametric Convex Quadratic Optimization. To appear in Optimization Methods and Software, 2009. • A. Ghaffari-Hadigheh, O. Romanko, and T. Terlaky. On Bi-Parametric Programming in Quadratic Optimization. Proceedings of EurOPT-2008, 2008.