OR II GSLM 52800

125 Views

Download Presentation
## OR II GSLM 52800

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Outline**• some terminology • differences between LP and NLP • basic questions in NLP • gradient and Hessian • quadratic form • contour, graph, and tangent plane 2**feasible region**C the neighborhood of a point for a given D B A Feasible Points, Solution Set, and Neighborhood • feasible point: a point that satisfies all the constraints • solution set (feasible set, feasible region): the collection of all feasible points • neighborhood of x0 = {x| |xx0| < } for some pre-specified only the neighborhood of D is completely feasible for this 3**f(x )**x3 12 x1 t s x2 x Weak and Strong; Local and Global • local minima: x1, any point in [s, t], x3 • strict (strong) local minima: x1, x3 • weak local minima: any point in [s, t] • strict global minimum: x1 • weak local maxima: any point in [s, t] 4**Differences Between Linear and Non-Linear Programming**• linear programming • there exists an optimal extreme point (a corner point) • direction of improvement keeps on being so unless hitting a constraint • a local optimum point is also globally optimal optimal point direction of improvement 5**f(x )**x3 12 x1 t s x2 x Differences Between Linear and Non-Linear Programming • none of these necessarily holds for a non-linear program min x2 + y2, s.t. -2 x, y 2 6**Basic Questions in Non-Linear Programming**• main question: given an initial location x0, how to get to a local minimum, or, better, a global minimum • (a) the direction of improvement? • (b) the necessary conditions of an optimal point? • (c) the sufficient conditions of an optimal point? • (d) any conditions to simplify the processes in (a), (b), and (c)? • (e) any algorithmic procedures to solve a NLP problem? 7**Basic Questions in Non-Linear Programming**• calculus required for (a) to (e) • direction of improvement of f = gradient of f • shaped by constraints • convexity for (d), and also (b) and (c) • identification of convexity: definiteness of matrices, especially for Hessians 8**Gradient and Hessian**• gradient of f: f(x) = • in short • Hessian = f and gj usually assumed to be twice differentiable functions Hessian is a symmetric matrix 9**Gradient and Hessian**• ej: (0, …, 0, 1, 0, …, 0)T, where “1” at the jth position • for small , f(x+ej) f(x) + • in general, x = (x1, …, xn)T from x, f(x+x) f(x) + 10**Example 1.6.1**• (a). f(x) = x2; f(3.5+) ? for small • (b). f(x, y) = x2 + y2, f((1, 1) + (x, y)) ? for small x, y • gradient f: direction of steepest accent of the objective fucntion 11**Example 1.6.2**• find the Hessian of • (a). f(x, y) = x2 + 7y2 • (b). f(x, y) = x2 + 5xy+ 7y2 • (c). f(x, y) = x3 + 7y2 12**Quadratic Form**• general form: xTQx/2 + cTx + a, where x is an n-dimensional vector; Q an nn square matrix; c and a are matrices of appropriate dimensions • how to derive the gradient and Hessian? • gradient f(x) = Qx+c • Hessian H = Q 13**Quadratic Form**• relate the two forms xTQx/2 + cTx + a and f(x, y) = 1x2+2xy+3y2+4x+5y+6 • Example 1.6.3 14**Example 1.6.4**• Find the first two derivatives of the following f(x) • f(x) = x2 for x[-2, 2] • f(x) = -x2 for x[-2, 2] 15**Contour and Graph (i.e., Surface) of Function f**• Example 1.7.1: f(x1, x2) = 16**Contour and Graph (i.e., Surface) of Function f**• an n-dimensional function • a contour of f: a diagram f(x) = c in the n-dimensional space for a given value c • the graph (surface function) of f: the diagram z = f(x) in the (n+1)st dimensional space as x and z vary 17**Contour and Graph (i.e., Surface) of Function f**• how do the contours of the one-dimensional function f(x) = x2 look like? 18**An Important Property Between the Gradient and the Tangent**Plane at a Contour • the gradient of f at point x0 is orthogonal to the tangent of the contour f(x) = c at x0 • many optimization results are related to the above property 19**Gradient of f at x0 Being Orthogonal to the Tangent of the**Contour f(x) = c at x0 • Example 1.7.3: f(x1, x2) = x1+2x2 • gradient at (4, 2)? • tangent of contour at (4, 2)? 20**Gradient of f at x0 Being Orthogonal to the Tangent of the**Contour f(x) = c at x0 • Example 1.7.2: f(x1, x2) = • point (x10, x20) on a contour f(x1, x2) = c 21**Tangent at a Contour and the Corresponding Tangent Plane at**a Surface • the above two are related • for contour of f(x, y) = x2+y2, the tangent at (x0, y0) • (x-x0, y- y0)T(2x0, 2y0) = 0 two orthogonal vectors u and v: uTv = 0 22**Tangent at a Contour and the Corresponding Tangent Plane at**a Surface • the tangent place at (x0, y0) for the surface of f(x, y) = x2+y2 • the surface: z = x2+y2 • defining a contour at a higher dimension: F(x, y, z) = x2+y2z • tangent plane at (x0, y0, ) of the surface: what happens when z = 23