OR II GSLM 52800 - PowerPoint PPT Presentation

OR II GSLM 52800

1 / 23
OR II GSLM 52800

OR II GSLM 52800

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

1. Outline • some terminology • differences between LP and NLP • basic questions in NLP • gradient and Hessian • quadratic form • contour, graph, and tangent plane 2

2. feasible region C the neighborhood of a point for a given   D B A Feasible Points, Solution Set, and Neighborhood • feasible point: a point that satisfies all the constraints • solution set (feasible set, feasible region): the collection of all feasible points • neighborhood of x0 = {x| |xx0| < } for some pre-specified  only the neighborhood of D is completely feasible for this  3

3. f(x ) x3 12 x1 t s x2 x Weak and Strong; Local and Global • local minima: x1, any point in [s, t], x3 • strict (strong) local minima: x1, x3 • weak local minima: any point in [s, t] • strict global minimum: x1 • weak local maxima: any point in [s, t] 4

4. Differences Between Linear and Non-Linear Programming • linear programming • there exists an optimal extreme point (a corner point) • direction of improvement keeps on being so unless hitting a constraint • a local optimum point is also globally optimal optimal point direction of improvement 5

5. f(x ) x3 12 x1 t s x2 x Differences Between Linear and Non-Linear Programming • none of these necessarily holds for a non-linear program min x2 + y2, s.t. -2  x, y  2 6

6. Basic Questions in Non-Linear Programming • main question: given an initial location x0, how to get to a local minimum, or, better, a global minimum • (a) the direction of improvement? • (b) the necessary conditions of an optimal point? • (c) the sufficient conditions of an optimal point? • (d) any conditions to simplify the processes in (a), (b), and (c)? • (e) any algorithmic procedures to solve a NLP problem? 7

7. Basic Questions in Non-Linear Programming • calculus required for (a) to (e) • direction of improvement of f = gradient of f • shaped by constraints • convexity for (d), and also (b) and (c) • identification of convexity: definiteness of matrices, especially for Hessians 8

8. Gradient and Hessian • gradient of f: f(x) = • in short • Hessian = f and gj usually assumed to be twice differentiable functions  Hessian is a symmetric matrix 9

9. Gradient and Hessian • ej: (0, …, 0, 1, 0, …, 0)T, where “1” at the jth position • for small , f(x+ej) f(x) +  • in general, x = (x1, …, xn)T from x, f(x+x) f(x) + 10

10. Example 1.6.1 • (a). f(x) = x2; f(3.5+)  ? for small  • (b). f(x, y) = x2 + y2, f((1, 1) + (x, y))  ? for small x, y • gradient f: direction of steepest accent of the objective fucntion 11

11. Example 1.6.2 • find the Hessian of • (a). f(x, y) = x2 + 7y2 • (b). f(x, y) = x2 + 5xy+ 7y2 • (c). f(x, y) = x3 + 7y2 12

12. Quadratic Form • general form: xTQx/2 + cTx + a, where x is an n-dimensional vector; Q an nn square matrix; c and a are matrices of appropriate dimensions • how to derive the gradient and Hessian? • gradient f(x) = Qx+c • Hessian H = Q 13

13. Quadratic Form • relate the two forms xTQx/2 + cTx + a and f(x, y) = 1x2+2xy+3y2+4x+5y+6 • Example 1.6.3 14

14. Example 1.6.4 • Find the first two derivatives of the following f(x) • f(x) = x2 for x[-2, 2] • f(x) = -x2 for x[-2, 2] 15

15. Contour and Graph (i.e., Surface) of Function f • Example 1.7.1: f(x1, x2) = 16

16. Contour and Graph (i.e., Surface) of Function f • an n-dimensional function • a contour of f: a diagram f(x) = c in the n-dimensional space for a given value c • the graph (surface function) of f: the diagram z = f(x) in the (n+1)st dimensional space as x and z vary 17

17. Contour and Graph (i.e., Surface) of Function f • how do the contours of the one-dimensional function f(x) = x2 look like? 18

18. An Important Property Between the Gradient and the Tangent Plane at a Contour • the gradient of f at point x0 is orthogonal to the tangent of the contour f(x) = c at x0 • many optimization results are related to the above property 19

19. Gradient of f at x0 Being Orthogonal to the Tangent of the Contour f(x) = c at x0 • Example 1.7.3: f(x1, x2) = x1+2x2 • gradient at (4, 2)? • tangent of contour at (4, 2)? 20

20. Gradient of f at x0 Being Orthogonal to the Tangent of the Contour f(x) = c at x0 • Example 1.7.2: f(x1, x2) = • point (x10, x20) on a contour f(x1, x2) = c 21

21. Tangent at a Contour and the Corresponding Tangent Plane at a Surface • the above two are related • for contour of f(x, y) = x2+y2, the tangent at (x0, y0) • (x-x0, y- y0)T(2x0, 2y0) = 0 two orthogonal vectors u and v: uTv = 0 22

22. Tangent at a Contour and the Corresponding Tangent Plane at a Surface • the tangent place at (x0, y0) for the surface of f(x, y) = x2+y2 • the surface: z = x2+y2 • defining a contour at a higher dimension: F(x, y, z) = x2+y2z • tangent plane at (x0, y0, ) of the surface: what happens when z = 23