Download
1 / 9
100 likes | 365 Views
Nonlinear Programming. In this handout Gradient Search for Multivariable Unconstrained Optimization KKT Conditions for Optimality of Constrained Optimization Algorithms for solving Convex Programs. Multivariable Unconstraint Optimization. max f(x 1 ,…,x n ) No functional constraints.
E N D
Nonlinear Programming In this handout Gradient Search for Multivariable Unconstrained Optimization KKT Conditions for Optimality of Constrained Optimization Algorithms for solving Convex Programs
Multivariable Unconstraint Optimization maxf(x1,…,xn) No functional constraints. Consider the case when f is concave. The necessary and sufficient condition for optimality is that all the partial derivatives are 0. But in most cases, the system of equations obtained that way can’t be solved analytically. Then a numerical search procedure must be used.
The Gradient Search Procedure The gradient at a specific point x=x’ The rate at which f increases is maximized in the direction of the gradient. Keep moving in the direction of the gradient until f stops increasing.
Constrained Optimization maxf(x1,…,xn) subject to gi(x1,…,xn) ≤ bi x1,…,xn ≥ 0 The necessary conditions for optimality are called the Karush-Kuhn-Tucker conditions (or KKT conditions), because they were derived independently by Karush (1939) and by Kuhn and Tucker (1951).
KKT conditions The conditions are also sufficient for optimality if f is concave and gi’s are convex.
KKT conditions • The KKT conditions are also sufficient for optimality if f is concave and gi’s are convex. • ui’s can be interpreted as dual variables; then KKT conditions are similar to the complementary slackness conditions of linear programming. • For relatively simple problems, KKT conditions can be used to derive an optimal solution. For example, KKT conditions are used to develop a modified simplex method for quadratic programming. • For more complicated problems, it might be impossible to derive a solution directly from KKT conditions. But they can be used to check whether a proposed solution is optimal (close to optimal).
Algorithms for solving Convex Programming problems Most of the algorithms fall into one of the following three categories. • Gradient algorithms, where the gradient search procedure is modified to keep the search path penetrating any constraint boundary. • Sequential unconstrained algorithms convert the original constrained problem to a sequence of unconstrained problems whose optimal solutions converge to the optimal solution of the original problem (for example, the barrier function method used in interior-point methods for linear programming).
Algorithms for solving Convex Programming problems (cont.) 3) Sequential-approximation algorithms. These algorithms replace the nonlinear objective function by a succession of linear or quadratic approximations. Particularly suitable for linearly constrained optimization problems. One example is Frank-Wolfe algorithm for the case of linearly constrained convex programming.
