OR II GSLM 52800

1. OR IIGSLM 52800 1

2. Outline • inequality constraint 2

3. NLP with Inequality Constraints • min f(x) • s.t. • hi(x) = 0 for i = 1, …, p • gj(x)  0 for j = 1, …, m • in matrix form • min f(x) • s.t. • h(x) = 0 • g(x) = 0 3

4. Binding Constraints • Inequality constraint gj(x)  0 is binding at point x0 if gj(x0) = 0 • =-constraint: always binding 4

5. Regular Points • K: the set of binding (inequality) constraints • x* subject to h(x) = 0 & g(x)  0 is a regular point • if Thi(x*), i = 1, …, p, & Tgj(x*), j K, are linearly independent 5

6. FONC for Inequality Constraints(Karush-Kuhn-Tucker Necessary Conditions; KKT Conditions) • for a regular point x* to be a local min, there exist *p and *m such that In Matrix Form: f(x*) +*Th(x*) + * Tg(x*) = 0(stationary condition; dual feasibility) h(x*) = 0, g(x*) 0(primal feasibility) * Tg(x*) = 0(Complementary slackness conditions) *  0(Non-negativity of the dual variables) 6

7. The Convex Cone Formed by Vectors • v1 = (1, 0), v2 = (0, 1) • {(x, y)|v1+v2 , ,   0} The convex cone formed by (1, 0) and (0, 1): {(x, y)|v1+v2 , ,   0} (0, 1) v1 The convex cone formed by v1 and v2: {(x, y)|v1+v2 , ,   0} (1, 0) v2 7

8. Geometric Interpretation of the KKT Condition constraint 1: g1(x)  0 constraint 2: g2(x)  0 g2(x0) g1(x) = 0 f(x0) g1(x0) x0 f(x0) g2(x) = 0 x0 cannot be a minimum 8

9. Geometric Interpretation of the KKT Condition constraint 1: g1(x)  0 constraint 2: g2(x)  0 g2(x0) g1(x) = 0 g1(x0) f(x0) x0 f(x0) g2(x) = 0 x0 cannot be a minimum 9

10. Geometric Interpretation of the KKT Condition constraint 1: g1(x)  0 region decreasing in f constraint 2: g2(x)  0 g2(x0) g1(x) = 0 g1(x0) f(x0) x0 f(x0) g2(x) = 0 x0 cannot be a minimum 10

11. Geometric Interpretation of the KKT Condition region decreasing in f constraint 1: g1(x)  0 constraint 2: g2(x)  0 g2(x0) g1(x) = 0 g1(x0) f(x0) x0 f(x0) g2(x) = 0 x0 cannot be a minimum 11

12. Geometric Interpretation of the KKT Condition region decreasing in f constraint 1: g1(x)  0 constraint 2: g2(x)  0 g2(x0) g1(x) = 0 g1(x0) f(x0) x0 f(x0) g2(x) = 0 x0 can be a minimum 12

13. A Necessary Condition for x0 to be a Minimum • when f in the convex cone of g1 and g2 • f(x0) = 1g1(x0) + 2g2(x0), i.e., • f(x0) + 1g1(x0) + 2g2(x0) = 0 g2(x0) g1(x) = 0 g1(x0) f(x0) x0 f(x0) g2(x) = 0 13

14. Effect of Convexity • convex objective function, convex feasible region set  the KKT condition  necessary & sufficient • convex gj& linear hi 14

15. Example 10.11 of JB • KKT conditions 15

16. Example 10.11 of JB • Tf(x) = (4(x1+1), 6(x24)) • Tg1(x) = (2x1, 2x2) • Tg2(x) = (-1, -1) • possibilities 16

17. Example 10.11 of JB • case 1 = 0, 2 = 0 (both constraints not binding) • x1 = -1, x2 = 4, violating 17

18. Example 10.11 of JB • case 1 > 0, 2 > 0 (both constraints binding) • solving g2(x0) g1(x0) f(x0) g1(x0) g2(x0) f(x0) 18

19. Example 10.11 of JB • case 1 = 0, 2 > 0 (the second constraint binding) • solving, 2 < 0 19

20. Example 10.11 of JB • case 1 > 0, 2 = 0 (the first constraint binding) • solving, 20

21. KKT Condition with Non-negativity Constraints • min f(x), • s.t. gi(x)  0, i = 1, …, m, • x  0 21

22. KKT Condition with Non-negativity Constraints • define L(x, ) = f(x) + Tg(x) 22

23. KKT Condition with Non-negativity Constraints • define L(x, ) = f(x) + 1Tg(x) - 2Tx 23

24. Example 10.12 of JB • L(x, ) = • KKT condition • (a) 2x1 8 +   0, 8x2 16 +   0 • (b) x1+ x2 5  0 • (c) x1(2x1 8 + ) = 0, x2(8x2 16 + ) = 0 • (d) (x1+ x2 5) = 0 • (e) x1 0, x2 0,  0   24

25. Example 10.12 of JB • eight cases, depending on whether x1, x2, and = 0 or > 0 • on checking, the case that • x1+x2 5 is binding, x1 > 0, x2 > 0  x= (3.2, 1.8) and  = 1.6 • satisfying the KKT condition • regular point • convex objective with linear constraint  KKT being sufficient 25