1 / 58

NLP

NLP. KKT Practice and Second Order Conditions from Nash and Sofer. Unconstrained. First Order Necessary Condition Second Order Necessary Second Order Sufficient. Easiest Problem. Linear equality constraints. KKT Conditions. Note for equality – multipliers are unconstrained

Download Presentation

NLP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. NLP KKT Practice and Second Order Conditions from Nash and Sofer

  2. Unconstrained • First Order Necessary Condition • Second Order Necessary • Second Order Sufficient

  3. Easiest Problem • Linear equality constraints

  4. KKT Conditions Note for equality – multipliers are unconstrained Complementarity not an issue

  5. Null Space Representation • Let x* be a feasible point, Ax*=b. • Any other feasible point can be written as x=x*+p where Ap=0 • The feasible region {x : x*+p pN(A)} where N(A) is null space of A

  6. Null and Range Spaces See Section 3.2 of Nash and Sofer for example

  7. Orthogonality

  8. Null Space Review

  9. Constrained to Unconstrained You can convert any linear equality constrained optimization problem to an equivalent unconstrained problem • Method 1 substitution • Method 2 using Null space representation and a feasible point.

  10. Example • Solve by substitution becomes

  11. Null Space Method • x*= [4 0 0]’ • x=x*+Zv becomes

  12. General Method • There exists a Null Space Matrix • The feasible region is: • Equivalent “Reduced” Problem

  13. Optimality Conditions • Assume feasible point and convert to null space formulation

  14. Where is KKT? • KKT implies null space • Null Space implies KKT Gradient is not in Null(A), thus it must be in Range(A’)

  15. Lemma 14.1 Necessary Conditions • If x* is a local min of f over {x|Ax=b}, and Z is a null matrix • Or equivalently use KKT Conditions

  16. Lemma 14.2 Sufficient Conditions • If x* satisfies (where Z is a basis matrix for Null(A)) then x* is a strict local minimizer

  17. Lemma 14.2 Sufficient Conditions (KKT form) • If (x*,*) satisfies (where Z is a basis matrix for Null(A)) then x* is a strict local minimizer

  18. Lagrangian Multiplier • * is called the Lagrangian Multiplier • It represents the sensitivity of solution to small perturbations of constraints

  19. Optimality conditions • Consider min (x2+4y2)/2 s.t. x-y=10

  20. Optimality conditions • Find KKT point Check SOSC

  21. In Class Practice • Find a KKT point • Verify SONC and SOSC

  22. Linear Equality Constraints - I

  23. Linear Equality Constraints - II

  24. Linear Equality Constraints - III so SOSC satisfied, and x* is a strict local minimum Objective is convex, so KKT conditions are sufficient.

  25. Next Easiest Problem • Linear equality constraints Constraints form a polyhedron

  26. x* Close to Equality Case Equality FONC: a2x = b a2x = b -a2 Polyhedron Ax>=b a3x = b a4x = b -a1 a1x = b contour set of function unconstrained minimum Which i are 0? What is the sign of I?

  27. x* Close to Equality Case Equality FONC: a2x = b a2x = b -a2 Polyhedron Ax>=b a3x = b a4x = b -a1 a1x = b Which i are 0? What is the sign of I?

  28. x* Inequality Case Inequality FONC: a2x = b a2x = b -a2 Polyhedron Ax>=b a3x = b a4x = b -a1 a1x = b Nonnegative Multipliers imply gradient points to the less than Side of the constraint.

  29. Lagrangian Multipliers

  30. Lemma 14.3 Necessary Conditions • If x* is a local min of f over {x|Ax≤b}, and Z is a null-space matrix for active constraints then for some vector *

  31. Lemma 14.5 Sufficient Conditions (KKT form) • If (x*,*) satisfies

  32. Lemma 14.5 Sufficient Conditions (KKT form) where Z+ is a basis matrix for Null(A +) and A + corresponds to nondegenerate active constraints) i.e.

  33. Sufficient Example • Find solution and verify SOSC

  34. Linear Inequality Constraints - I

  35. Linear Inequality Constraints - II

  36. Linear Inequality Constraints - III

  37. Linear Inequality Constraints - IV

  38. Example • Problem

  39. You Try • Solve the problem using above theorems:

  40. Why Necessary and Sufficient? • Sufficient conditions are good for? • Way to confirm that a candidate point • is a minimum (local) • But…not every min satisifies any given SC • Necessary tells you: • If necessary conditions don’t hold then you know you don’t have a minimum. • Under appropriate assumptions, every point that is a min satisfies the necessary cond. • Good stopping criteria • Algorithms look for points that satisfy Necessary conditions

  41. General Constraints

  42. Lagrangian Function • Optimality conditions expressed using Lagrangian function and Jacobian matrix were each row is a gradient of a constraint

  43. Theorem 14.2 Sufficient Conditions Equality (KKT form) • If (x*,*) satisfies

  44. Theorem 14.4 Sufficient Conditions Inequality (KKT) • If (x*,*) satisfies

  45. Lemma 14.4 Sufficient Conditions (KKT form) where Z+ is a basis matrix for Null(A +) and A + corresponds to Jacobian of nondegenerate active constraints) i.e.

  46. Sufficient Example • Find solution and verify SOSC

  47. Nonlinear Inequality Constraints - I

  48. Nonlinear Inequality Constraints - II

  49. Nonlinear Inequality Constraints - III

  50. Sufficient Example • Find solution and verify SOSC

More Related