Lecture 8 nonlinear programming models
Download
1 / 24

Lecture 8 – Nonlinear Programming Models - PowerPoint PPT Presentation


  • 151 Views
  • Uploaded on

Lecture 8 – Nonlinear Programming Models. Topics General formulations Local vs. global solutions Solution characteristics Convexity and convex programming Examples. Nonlinear Optimization. In LP ... the objective function & constraints are linear and the problems are “ easy ” to solve.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Lecture 8 – Nonlinear Programming Models' - benard


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Lecture 8 nonlinear programming models
Lecture 8 – Nonlinear Programming Models

Topics

  • General formulations

  • Local vs. global solutions

  • Solution characteristics

  • Convexity and convex programming

  • Examples


Nonlinear optimization
Nonlinear Optimization

In LP ... the objective function & constraints are linear and the problems are “easy”to solve.

Many real-world engineering and business problems have nonlinear elements and are hard to solve.


General nlp
General NLP

Minimize f(x)

s.t. gi(x) (, , =) bi, i = 1,…,m

x = (x1,…,xn) is the n-dimensional vector of decision variables

f(x) is the objective function

gi(x) are the constraint functions

bi are fixed known constants


c

x

c

x

c

x

Example 2

Max f(x) = e

e

e

1

1

2

2

n

n

s.t. Ax = b, x³0

n

å

fj (xj)

Example 3

Min

Problems with

“decreasing efficiencies”

j =1

s.t. Ax = b, x³0

fj(xj)

where each fj(xj)is of the form

xj

Examples of NLPs

4

Example 1

Max f(x) =

3x1 + 2x2

2

s.t. x1 + x2£ 1, x1³ 0, x2 unrestricted

Examples 2 and 3 can be reformulated as LPs


Nlp graphical solution method
NLP Graphical Solution Method

Max f(x1, x2) = x1x2

s.t. 4x1 + x2£ 8

x1 ³ 0, x2 ³ 0

x2

8

f(x) = 2

f(x) = 1

x1

2

Optimal solution will lie on the line g(x) = 4x1 + x2 – 8 = 0.


Solution Characteristics

Gradient of f(x) = f(x1, x2) (f/x1, f/x2)T

This gives f/x1 = x2, f/x2 = x1

and g/x1 = 4, g/x2 = 1

At optimality we have f(x1, x2) = g(x1, x2)

or x1* = 1 and x2* = 4

  • Solution is not a vertex of feasible region. For this particular problem the solution is on the boundary of the feasible region. This is not always the case.

  • In a more general case, f(x1, x2) = g(x1, x2) with  0. (In this case,  = 1.)


Nonconvex Function

global

max

stationary

point

f(x)

local

max

local

min

local

min

x

Let Sn be the set of feasible solutions to an NLP.

Definition: A global minimum is any x0S such than

f(x0)  f(x)

for all feasible x not equal to x0.


Function with Unique Global Minimum at x = (1, –3)

If g1 = x1³ 0 and g2 = x2³ 0, what is the optimum ?

At (1, 0), f(x1, x2) = 1g1(x1, x2) + 2g1(x1, x2)

or (0, 6) = 1(1, 0) + 2(0, 1), 1³ 0, 2³ 0

so 1 = 0 and 2 = 6


Function with Multiple Maxima and Minima

Min { f(x)= sin(x) : 0 x 5p}



Convexity

Convexity condition for Global Minimum

univariate f :

d2 f(x)

≥ 0 for all x

dx2

Convexity

Convex function: If you draw a straight line between any two points on f(x) the line will be above or on the line.

Concave function: If f(x) is convex than –f(x) is concave.

Linear functions are both convex and concave.


Definition of convexity
Definition of Convexity Global Minimum

Let x1 and x2 be two points in Sn. A function f(x) is convex if and only if

f(lx1 + (1–l)x2) ≤ lf(x1) + (1–l)f(x2)

for all 0 < l < 1. It is strictly convex if the inequality sign ≤ is replaced with the sign <.

1-dimensional example


Nonconvex -- Nonconave Function Global Minimum

f(x)

x


Theoretical result for convex functions

2 Global Minimum

2

2

d

f

d

f

d

f

Hessian of f at x : s2f(x) =

. . .

dx

dx

dx

dx

2

dx

1

2

1

n

1

.

.

2

d

f

.

.

.

dx

dx

.

2

1

.

.

.

2

2

d

f

d

f

. . .

dx

dx

2

dx

n

1

n

Theoretical Result for Convex Functions

A positively weighted sum of convex functions is convex:

if fk(x) k =1,…,m are convex and 1,…,m³ 0,

then f(x) = å ak fk(x) is convex.

m

k =1


Determining convexity

f Global Minimum(x)

x1 x2

Determining Convexity

One-Dimensional Functions:

A function f(x) ÎC1 is convex if and only if it is underestimated by linear extrapolation; i.e.,

f(x2) ≥ f(x1) + (df(x1)/dx)(x2 – x1) for all x1 and x2.

A function f(x) C2 is convex if and only if its second derivative is nonnegative.

d2f(x)/dx2≥ 0 for all x

If the inequality is strict (>), the function is strictly convex.


Multiple dimensional functions

Example Global Minimum: f(x) = 3(x1)2 + 4(x2)3 – 5x1x2 + 4x1

Multiple Dimensional Functions

f(x) is convex if only if f(x2) ≥ f(x1) + Tf(x1)(x2 – x1) for all x1 and x2.

Definition: The Hessian matrix H(x) associated with f(x) is the nn symmetric matrix of second partial derivatives of f(x) with respect to the components of x.

When f(x) is quadratic, H(x) has only constant terms; when f(x) is linear, H(x) does not exist.


Properties of the hessian
Properties of the Hessian Global Minimum

How can we use Hessian to determine whether or not f(x) is convex?

  • H(x) is positive definite if and only if xTHx> 0 for all x0.

  • H(x) is positive semi-definite if and only if xTHx≥ 0 for all x and there exists and x 0 such that xTHx = 0.

  • H(x) is indefinite if and only if xTHx> 0 for some x, and xTHx< 0 for some other x.


Multiple dimensional functions and convexity
Multiple Dimensional Functions and Convexity Global Minimum

  • f(x) is strictly convex (convex) if its associated Hessian matrix H(x) is positive definite (semi- definite) for all x.

  • f(x) is neither convex nor concave if its associated Hessian matrix H(x) is indefinite

The terms negativedefiniteandnegative-semi definiteare also appropriate for the Hessian and provide symmetric results for concave functions. Recall that a function f(x) is concave if –f(x) is convex.


Testing for definiteness

Definition Global Minimum: The ith leading principal submatrix of H is the matrix formed taking the intersection of its first i rows and i columns. Let Hi be the value of the corresponding determinant:

Testing for Definiteness

, where hij= 2f(x)/xixj

LetHessian, H =


Rules for definiteness
Rules for Definiteness Global Minimum

  • H is positive definite if and only if the determinants of all the leading principal submatrices are positive; i.e., Hi> 0 for i = 1,…,n.

  • His negative definite if and only if H1 < 0 and the remaining leading principal determinants alternate in sign:

  • H2 > 0, H3 < 0, H4 > 0, . . .

Positive-semidefinite and negative semi-definiteness require that all principal submatrices satisfy the above conditions for the particular case.


Quadratic functions
Quadratic Functions Global Minimum

Example 1: f(x) = 3x1x2 + x12 + 3x22

so H1 = 2 and H2 = 12 – 9 = 3

Conclusion f(x) is convex because H(x) is positive definite.


Quadratic functions1
Quadratic Functions Global Minimum

Example 2: f(x) = 24x1x2 + 9x12 + 16x22

  • so H1 = 18 and H2 = 576 – 576 = 0

  • Thus H is positive semi-definite (determinants of all submatrices are nonnegative) so f(x) is convex.

  • Note, xTHx = 2(3x1 + 4x2)2≥ 0. For x1 = -4, x2 = 3, we get xTHx = 0.


Nonquadratic functions
Nonquadratic Functions Global Minimum

Example 3: f(x) = (x2 – x12)2 + (1 – x1)2

Thus the Hessian depends on the point under consideration:

At x = (1, 1), which is positive definite.

At x = (0, 1), which is indefinite.

Thus f(x) is not convex although it is strictly convex near (1, 1).


What you should know about nonlinear programming
What You Should Know About Nonlinear Programming Global Minimum

  • How to identify the decision variables.

  • How to write constraints.

  • How to identify a convex function.

  • The difference between a local and global solution.


ad