- 184 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' NLP' - eagan-flowers

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### NLP

KKT Practice and Second Order Conditions from Nash and Sofer

Unconstrained

- First Order Necessary Condition
- Second Order Necessary
- Second Order Sufficient

Easiest Problem

- Linear equality constraints

Null Space Representation

- Let x* be a feasible point, Ax*=b.
- Any other feasible point can be written as x=x*+p where Ap=0
- The feasible region
{x : x*+p pN(A)}

where N(A) is null space of A

Null and Range Spaces

See Section 3.2 of Nash and Sofer for example

Constrained to Unconstrained

You can convert any linear equality constrained optimization problem to an equivalent unconstrained problem

- Method 1 substitution
- Method 2 using Null space representation and a feasible point.

Example

- Solve by substitution
becomes

Null Space Method

- x*= [4 0 0]’
- x=x*+Zv
becomes

General Method

- There exists a Null Space Matrix
- The feasible region is:
- Equivalent “Reduced” Problem

Optimality Conditions

- Assume feasible point and convert to null space formulation

Where is KKT?

- KKT implies null space
- Null Space implies KKT
Gradient is not in Null(A), thus it must be in Range(A’)

Lemma 14.1 Necessary Conditions

- If x* is a local min of f over {x|Ax=b}, and Z is a null matrix
- Or equivalently use KKT Conditions

Lemma 14.2 Sufficient Conditions

- If x* satisfies (where Z is a basis matrix for Null(A))
then x* is a strict local minimizer

Lemma 14.2 Sufficient Conditions (KKT form)

- If (x*,*) satisfies (where Z is a basis matrix for Null(A))
then x* is a strict local minimizer

Lagrangian Multiplier

- * is called the Lagrangian Multiplier
- It represents the sensitivity of solution to small perturbations of constraints

Optimality conditions

- Consider min (x2+4y2)/2 s.t. x-y=10

Optimality conditions

- Find KKT point Check SOSC

In Class Practice

- Find a KKT point
- Verify SONC and
SOSC

Linear Equality Constraints - III

so SOSC satisfied, and x* is a strict local minimum

Objective is convex, so KKT conditions are sufficient.

Next Easiest Problem

- Linear equality constraints
Constraints form a polyhedron

Close to Equality Case

Equality FONC:

a2x = b

a2x = b

-a2

Polyhedron Ax>=b

a3x = b

a4x = b

-a1

a1x = b

contour set of function

unconstrained minimum

Which i are 0? What is the sign of I?

Close to Equality Case

Equality FONC:

a2x = b

a2x = b

-a2

Polyhedron Ax>=b

a3x = b

a4x = b

-a1

a1x = b

Which i are 0? What is the sign of I?

Inequality Case

Inequality FONC:

a2x = b

a2x = b

-a2

Polyhedron Ax>=b

a3x = b

a4x = b

-a1

a1x = b

Nonnegative Multipliers imply gradient points to the less than

Side of the constraint.

Lemma 14.3 Necessary Conditions

- If x* is a local min of f over {x|Ax≤b}, and Z is a null-space matrix for active constraints then for some vector *

Lemma 14.5 Sufficient Conditions (KKT form)

- If (x*,*) satisfies

Lemma 14.5 Sufficient Conditions (KKT form)

where Z+ is a basis matrix for Null(A +) and A + corresponds to nondegenerate active constraints)

i.e.

Sufficient Example

- Find solution and verify SOSC

Example

- Problem

You Try

- Solve the problem using above theorems:

Why Necessary and Sufficient?

- Sufficient conditions are good for?
- Way to confirm that a candidate point
- is a minimum (local)
- But…not every min satisifies any given SC

- Necessary tells you:
- If necessary conditions don’t hold then you know you don’t have a minimum.
- Under appropriate assumptions, every point that is a min satisfies the necessary cond.
- Good stopping criteria
- Algorithms look for points that satisfy Necessary conditions

Lagrangian Function

- Optimality conditions expressed using
Lagrangian function

and Jacobian matrix

were each row is a gradient of a constraint

Theorem 14.2 Sufficient Conditions Equality (KKT form)

- If (x*,*) satisfies

Theorem 14.4 Sufficient Conditions Inequality (KKT)

- If (x*,*) satisfies

Lemma 14.4 Sufficient Conditions (KKT form)

where Z+ is a basis matrix for Null(A +) and A + corresponds to Jacobian of nondegenerate active constraints)

i.e.

Sufficient Example

- Find solution and verify SOSC

Sufficient Example

- Find solution and verify SOSC

Theorem 14.1 Necessary Conditions- Equality

- If x* is a local min of f over {x|g(x)=0}, Z is a null-space matrix of the Jacobian g(x*)’, and x* is a regular point then

Theorem 14.3 Necessary Conditions

- If x* is a local min of f over {x|g(x)>=0}, Z is a null-space matrix of the Jacobian g(x*)’, and x* is a regular point then

Regular point

- If x* is a regular point with respect to the constraints g(x*) if the gradient of the active constraints are linearly independent.
- For equality constraints, all constraints are active so
should have linearly independent rows.

Necessary Example

- Show optimal solution x*=[1,0]’
is regular and find KKT point

Constraint Qualifications

- Regularity is an example of a constraint qualification CQ.
- The KKT conditions are based on linearizations of the constraints.
- CQ guarantees that this linearization is not getting us into trouble. Problem is
KKT point might not exist.

- There are many other CQ,e.g., for inequalities Slater is there exists g(x)<0.
- Note CQ not needed for linear constraints.

Download Presentation

Connecting to Server..