1 / 24

Performance Surfaces

Performance Surfaces. d. *. *. F. (. x. ). =. F. (. x. ). +. F. (. x. ). (. x. –. x. ). *. d. x. x. =. x. 2. 1. d. 2. ¼. *. +. -. -. -. F. (. x. ). (. x. –. x. ). +. 2. 2. d. x. *. x. =. x. n. 1. d. n. ¼. *. +. -. -. -. -. -. F.

Download Presentation

Performance Surfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Performance Surfaces

  2. d * * F ( x ) = F ( x ) + F ( x ) ( x – x ) * d x x = x 2 1 d 2 ¼ * + - - - F ( x ) ( x – x ) + 2 2 d x * x = x n 1 d n ¼ * + - - - - - F ( x ) ( x – x ) + n ! n d x * x = x Taylor Series Expansion

  3. Example Taylor series of F(x) about x*=0: Taylor series approximations:

  4. Plot of Approximations

  5. ¶ * * * x x x x F ( ) = F ( ) + F ( ) ( x – x ) + F ( ) ( x – x ) 1 1 2 2 ¶ x ¶ x * * x x x x = = 1 2 2 ¶ 1 ¶ 2 ¼ * * x x + + F ( ) ( x – x ) + - - - F ( ) ( x – x ) n n 1 1 * * ¶ x 2 2 x x x x = = ¶ x n 1 2 1 ¶ ¼ * * x + - - - F ( ) ( x – x ) ( x – x ) + 1 1 2 2 * 2 ¶ x ¶ x x x = 1 2 Vector Case

  6. 2 2 2 ¶ ¶ ¶ x x x F ( ) F ( ) ¼ F ( ) ¶ ¶ x ¶ x ¶ x ¶ x 2 x F ( ) ¶ x 1 2 1 n ¶ x 1 1 2 2 2 ¶ ¶ ¶ ¶ x x x x F ( ) F ( ) F ( ) ¼ F ( ) 2 x x Ñ F ( ) = ¶ x 2 Ñ F ( ) = ¶ x ¶ x ¶ x ¶ x ¶ x 2 2 1 2 n 2 ¼ ¼ ¼ ¼ ¶ 2 2 2 x F ( ) ¶ ¶ ¶ ¶ x x x x F ( ) F ( ) ¼ F ( ) n 2 ¶ x ¶ x ¶ x ¶ x ¶ x n 1 n 2 n Matrix Form T * * x x x x x F ( ) = F ( ) + Ñ F ( ) ( – ) * x x = 1 T * * ¼ 2 x x x x x + - - - ( – ) Ñ F ( ) ( – ) + * 2 x x = Gradient Hessian

  7. T p x Ñ F ( ) - - - - - - - - - - - - - - - - - - - - - - - p Directional Derivatives First derivative (slope) of F(x) along xi axis: (ith element of gradient) Second derivative (curvature) of F(x) along xi axis: (i,i element of Hessian) First derivative (slope) of F(x) along vector p: T 2 p x p Ñ F ( ) Second derivative (curvature) of F(x) along vector p: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 2 p

  8. 1 1 – 1 T 1 0 p x Ñ F ( ) - - - - - - - - - - - - - - - - - - - - - - - = - - - - - - - - - - - - - - - - - - - - - - - - = - - - - - - - = 0 p 2 1 – 1 Example

  9. Plots Directional Derivatives 1.4 1.3 x2 1.0 0.5 0.0 x2 x1 x1

  10. Minima Strong Minimum The point x* is a strong minimum of F(x) if a scalar d>0 exists, such that F(x*) < F(x*+Dx) for all Dx such that d>||Dx||>0. Global Minimum The point x* is a unique global minimum of F(x) if F(x*) < F(x*+Dx) for all Dx°0. Weak Minimum The point x* is a weak minimum of F(x) if it is not a strong minimum, and a scalar d>0 exists, such that F(x*) Š F(x* + Dx) for all Dx such that d>||Dx||>0.

  11. Scalar Example Strong Maximum Strong Minimum Global Minimum

  12. Vector Example

  13. 1 T T * * ¼ x x x x x x x x x 2 F ( ) = F ( + D ) = F ( ) + Ñ F ( ) D + - - - D Ñ F ( ) D + * * 2 x x x x = = First-Order Optimality Condition For small Dx: If x* is a minimum, this implies: If then But this would imply that x* is not a minimum. Therefore Since this must be true for every Dx,

  14. Second-Order Condition If the first-order condition is satisfied (zero gradient), then A strong minimum will exist at x* if for any Dx°0. Therefore the Hessian matrix must be positive definite. A matrix A is positive definite if: for any z°0. This is a sufficient condition for optimality. A necessary condition is that the Hessian matrix be positive semidefinite. A matrix A is positive semidefinite if: for any z.

  15. 2 2 x F ( ) = x + 2 x x + 2 x + x 1 1 2 1 2 Example (Not a function of x in this case.) To test the definiteness, check the eigenvalues of the Hessian. If the eigenvalues are all greater than zero, the Hessian is positive definite. Both eigenvalues are positive, therefore strong minimum.

  16. Quadratic Functions (Symmetric A) Gradient and Hessian: Useful properties of gradients: Gradient of Quadratic Function: Hessian of Quadratic Function:

  17. l 0 ¼ 0 1 0 l ¼ 0 T 2 A ' B A B L = [ ] = = ¼ ¼ ¼ 0 0 ¼ l n Eigensystem of the Hessian Consider a quadratic function which has a stationary point at the origin, and whose value there is zero. Perform a similarity transform on the Hessian matrix, using the eigenvalues as the new basis vectors. Since the Hessian matrix is symmetric, its eigenvectors are orthogonal.

  18. T T p x p p A p 2 Ñ F ( ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - = - - - - - - - - - - - - - - - 2 2 p p n 2 å l c i i T T T T T p A p c B B L B B c c L c ( ) i = 1 - - - - - - - - - - - - - - - = - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - = - - - - - - - - - - - - - - = - - - - - - - - - - - - - - - - - - - - 2 T T T n p c B B c c c 2 å c i i = 1 T p A p l £ - - - - - - - - - - - - - - - £ l m i n m a x 2 p Second Directional Derivative Represent p with respect to the eigenvectors (new basis):

  19. 0 0 ¼ 0 T T c B p B z = = = m a x 1 0 ¼ 0 n 2 å l c T i i z A z m a x m a x i = 1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - = - - - - - - - - - - - - - - - - - - - - = l m a x 2 n z 2 å m a x c i i = 1 Eigenvector (Largest Eigenvalue) The eigenvalues represent curvature (second derivatives) along the eigenvectors (the principal axes).

  20. Circular Hollow (Any two independent vectors in the plane would work.)

  21. Elliptical Hollow

  22. 1 3 1 1 – 0.5 – 1.5 2 2 T x x x F ( ) = – - - - x – - - - x x – - - - x = - - - 1 1 2 2 4 2 4 2 – 1.5 – 0.5 Elongated Saddle

  23. 1 1 1 1 – 1 2 2 T x x x F ( ) = - - - x – x x + - - - x = - - - 1 1 2 2 2 2 2 – 1 1 Stationary Valley

  24. Quadratic Function Summary • If the eigenvalues of the Hessian matrix are all positive, the function will have a single strong minimum. • If the eigenvalues are all negative, the function will have a single strong maximum. • If some eigenvalues are positive and other eigenvalues are negative, the function will have a single saddle point. • If the eigenvalues are all nonnegative, but some eigenvalues are zero, then the function will either have a weak minimum or will have no stationary point. • If the eigenvalues are all nonpositive, but some eigenvalues are zero, then the function will either have a weak maximum or will have no stationary point. Stationary Point:

More Related