1 / 23

OR II GSLM 52800

OR II GSLM 52800. Outline. some terminology differences between LP and NLP basic questions in NLP gradient and Hessian quadratic form contour, graph, and tangent plane. feasible region. C. the neighborhood of a point for a given . . D. B. A.

rona
Download Presentation

OR II GSLM 52800

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OR IIGSLM 52800 1

  2. Outline • some terminology • differences between LP and NLP • basic questions in NLP • gradient and Hessian • quadratic form • contour, graph, and tangent plane 2

  3. feasible region C the neighborhood of a point for a given   D B A Feasible Points, Solution Set, and Neighborhood • feasible point: a point that satisfies all the constraints • solution set (feasible set, feasible region): the collection of all feasible points • neighborhood of x0 = {x| |xx0| < } for some pre-specified  only the neighborhood of D is completely feasible for this  3

  4. f(x ) x3 12 x1 t s x2 x Weak and Strong; Local and Global • local minima: x1, any point in [s, t], x3 • strict (strong) local minima: x1, x3 • weak local minima: any point in [s, t] • strict global minimum: x1 • weak local maxima: any point in [s, t] 4

  5. Differences Between Linear and Non-Linear Programming • linear programming • there exists an optimal extreme point (a corner point) • direction of improvement keeps on being so unless hitting a constraint • a local optimum point is also globally optimal optimal point direction of improvement 5

  6. f(x ) x3 12 x1 t s x2 x Differences Between Linear and Non-Linear Programming • none of these necessarily holds for a non-linear program min x2 + y2, s.t. -2  x, y  2 6

  7. Basic Questions in Non-Linear Programming • main question: given an initial location x0, how to get to a local minimum, or, better, a global minimum • (a) the direction of improvement? • (b) the necessary conditions of an optimal point? • (c) the sufficient conditions of an optimal point? • (d) any conditions to simplify the processes in (a), (b), and (c)? • (e) any algorithmic procedures to solve a NLP problem? 7

  8. Basic Questions in Non-Linear Programming • calculus required for (a) to (e) • direction of improvement of f = gradient of f • shaped by constraints • convexity for (d), and also (b) and (c) • identification of convexity: definiteness of matrices, especially for Hessians 8

  9. Gradient and Hessian • gradient of f: f(x) = • in short • Hessian = f and gj usually assumed to be twice differentiable functions  Hessian is a symmetric matrix 9

  10. Gradient and Hessian • ej: (0, …, 0, 1, 0, …, 0)T, where “1” at the jth position • for small , f(x+ej) f(x) +  • in general, x = (x1, …, xn)T from x, f(x+x) f(x) + 10

  11. Example 1.6.1 • (a). f(x) = x2; f(3.5+)  ? for small  • (b). f(x, y) = x2 + y2, f((1, 1) + (x, y))  ? for small x, y • gradient f: direction of steepest accent of the objective fucntion 11

  12. Example 1.6.2 • find the Hessian of • (a). f(x, y) = x2 + 7y2 • (b). f(x, y) = x2 + 5xy+ 7y2 • (c). f(x, y) = x3 + 7y2 12

  13. Quadratic Form • general form: xTQx/2 + cTx + a, where x is an n-dimensional vector; Q an nn square matrix; c and a are matrices of appropriate dimensions • how to derive the gradient and Hessian? • gradient f(x) = Qx+c • Hessian H = Q 13

  14. Quadratic Form • relate the two forms xTQx/2 + cTx + a and f(x, y) = 1x2+2xy+3y2+4x+5y+6 • Example 1.6.3 14

  15. Example 1.6.4 • Find the first two derivatives of the following f(x) • f(x) = x2 for x[-2, 2] • f(x) = -x2 for x[-2, 2] 15

  16. Contour and Graph (i.e., Surface) of Function f • Example 1.7.1: f(x1, x2) = 16

  17. Contour and Graph (i.e., Surface) of Function f • an n-dimensional function • a contour of f: a diagram f(x) = c in the n-dimensional space for a given value c • the graph (surface function) of f: the diagram z = f(x) in the (n+1)st dimensional space as x and z vary 17

  18. Contour and Graph (i.e., Surface) of Function f • how do the contours of the one-dimensional function f(x) = x2 look like? 18

  19. An Important Property Between the Gradient and the Tangent Plane at a Contour • the gradient of f at point x0 is orthogonal to the tangent of the contour f(x) = c at x0 • many optimization results are related to the above property 19

  20. Gradient of f at x0 Being Orthogonal to the Tangent of the Contour f(x) = c at x0 • Example 1.7.3: f(x1, x2) = x1+2x2 • gradient at (4, 2)? • tangent of contour at (4, 2)? 20

  21. Gradient of f at x0 Being Orthogonal to the Tangent of the Contour f(x) = c at x0 • Example 1.7.2: f(x1, x2) = • point (x10, x20) on a contour f(x1, x2) = c 21

  22. Tangent at a Contour and the Corresponding Tangent Plane at a Surface • the above two are related • for contour of f(x, y) = x2+y2, the tangent at (x0, y0) • (x-x0, y- y0)T(2x0, 2y0) = 0 two orthogonal vectors u and v: uTv = 0 22

  23. Tangent at a Contour and the Corresponding Tangent Plane at a Surface • the tangent place at (x0, y0) for the surface of f(x, y) = x2+y2 • the surface: z = x2+y2 • defining a contour at a higher dimension: F(x, y, z) = x2+y2z • tangent plane at (x0, y0, ) of the surface: what happens when z = 23

More Related