Gradient Projection for bound-constrained QPs

1. Gradient Projection for bound-constrained QPs Sophea Chan May 29, 2012

2. Gradient Projection for bound-constrained QPs Overview • Bound constrained QPs • Gradient Projection Method • Conjugate Gradient Method • Projected Search • Implementation Sophea Chan

3. Gradient Projection for bound-constrained QPs Bound constrained QPs • G is symmetric nxn matrix • l and u are vectors of lower and upper bound • q : Rn → R, q is strictly convex and n is large Sophea Chan

4. Gradient Projection for bound-constrained QPs Subproblemof QPs • Wk is subset of active constraints => The solution of subproblem is a feasible direction. Sophea Chan

5. Gradient Projection for bound-constrained QPs Gradient Projection Method - ∇q is the gradient of q with respect to the inner product 〈.,.〉 Sophea Chan

6. Gradient Projection for bound-constrained QPs Gradient Projection Method Example: Given X0  and a tolerance  in (0,1), we define an approximation of QP problem as any vector X   such that (5) it shows that q(X*) = 0 whenever X*is a solution of the problem. This definition also holds whenever X is sufficiently close to X*and in the face of  that contains X*. The face of that contains X is Sophea Chan

7. Gradient Projection for bound-constrained QPs Gradient Projection Method Sophea Chan

8. Gradient Projection Method Sophea Chan

9. Gradient Projection for bound-constrained QPs Gradient Projection Method An approximation minimizer is required (6) Where P is the Projection into the feasible , P(X) is the closest point to X in . is a measure of optimality fairly common. P projected into  , then we can compute P(x) = mid (l, u, X) (7) where mid(l, u, X) is the vector whose it component is the median of the set {li, ui, Xi}. Sophea Chan

10. Gradient Projection for bound-constrained QPs Conjugate Gradient Method • If Xk lies in the same face as the QP and dksolve the equation above, then Xk + dk is the solution of QP. • dk is search direction Note that this is an unconstrained QP problem in free variables. Sophea Chan

11. Gradient Projection for bound-constrained QPs Conjugate Gradient Method Sophea Chan

12. Gradient Projection for bound-constrained QPs Conjugate Gradient Method Sophea Chan

13. Gradient Projection for bound-constrained QPs Algorithm GPCG • Generate gradient projection iterates y0, y1, … with y0 = Xk. Set yjk -> Xk, where jk is the first index. • Generate conjugate gradient iterates W0, W1, … with W0 = 0. Set dk = ZkWjk, where jk is the first index. • Use a projected search to define Xk+1. If B(Xk+1) = A(Xx+1), continue the conjugate gradient method Sophea Chan

14. Gradient Projection for bound-constrained QPs Implement Example and solution will give on blackboard. Sophea Chan

15. Gradient Projection for bound-constrained QPs References • J.J. More and G. Toraldo. On the solution of large quadratic programming problems with bound constraints. SIAM Journal on Optimization, 1(1), 93113, 1991. • J.J. More and G. Toraldo. Algorithms for bound constrained quadratic programming problems. Numerische Mathematik, 55(4), 377400, 1989. • J. Nocedal, S.J. Wright. Numerical Optimization, Springer, 2006. Ch. 16.7, pp. 485490. Sophea Chan