Logic based methods for global optimization
1 / 45

Logic-Based Methods for Global Optimization - PowerPoint PPT Presentation

  • Updated On :

Logic-Based Methods for Global Optimization. J. N. Hooker Carnegie Mellon University, USA November 2003. Basic Idea Assume the problem becomes convex when certain variables are fixed.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Logic-Based Methods for Global Optimization' - siyamak

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Logic based methods for global optimization l.jpg

Logic-Based Methods for Global Optimization

J. N. Hooker

Carnegie Mellon University, USA

November 2003

Slide2 l.jpg

  • Basic Idea

  • Assume the problem becomes convex when certain variables are fixed.

    • If these variables are discrete, we can reformulate the problem as disjunctions of convex constraints.

    • If some of them are continuous, discretize them to obtain an approximate global solution.

  • Motivation is to take advantage of advanced solution methods:

    • Branch-and-bound method chooses the appropriate disjunct in each constraint.

    • Nonlinear programming method solves convex subproblems that result when disjuncts are chosen.

Slide3 l.jpg

  • Outline

  • General form of problem

  • Structural design example

  • Disjunctive formulation

  • Branch and bound with convex relaxations

    • Big-M formulation

    • Convex hull formulation

  • Logic-based outer approximation

  • Logic-based Benders decomposition

  • Branch and bound with convex quasi-relaxations

  • Realistic structural design problem

    • Solution by MILP

    • Solution by quasi-relaxation

  • Other applications

Slide4 l.jpg

If is continuous, discretize it, to get approximate global solution.

General Form of Problem

Vector of functions

Logical conditions on y

Assume that when is fixed to , we get a convex problem

convex functions of x

Slide5 l.jpg

Objective is defined in the constraints global solution.

We assume one per constraint.

Many problems have this form. If not, constraints can in principle be put into this form by change of variable.

Slide6 l.jpg

For example, consider global solution.

Use the change of variable

and the constraints have the desired form:

One yjper constraint

Logical condition

Slide7 l.jpg

load = 10 global solution.

displacement =

displacement =

load = 20

cost of steel

penalty for displacement

Structural Design Example

Choose bar thickness that minimizes cost.

This example is intended only to illustrate the algorithms. A more realistic model for structural design is presented at the end of the talk.

= compression of bar j

= thickness (cross-sectional area) of bar j

cost =

Slide8 l.jpg

Can be written in desired form: global solution.

Global optimization problem:

Hooke’s law


or many closely spaced values for continuous problem


Slide9 l.jpg

How to Solve It? global solution.

Can in principle use branch and bound by branching on .

But continuous relaxations at nodes of the search tree are in general nonconvex.

So we write the problem in disjunctive form.

Slide10 l.jpg

Disjunctive Formulation global solution.

  • Now each disjunct is convex. We will solve by:

  • Branch and bound with convex relaxations (use disjunctive programming or MINLP).

  • Logic-based outer approximation with linear relaxations.

  • Relaxations can be large when there are many disjunctions. In this case consider:

  • Logic-based Benders decomposition with discrete relaxation.

  • Branch and bound with convex quasi-relaxations (requires that constraint functions satisfy certain properties).

Slide11 l.jpg

Recall the example… global solution.

Disjunctive formulation is:

Disjuncts are convex (linear)

Slide12 l.jpg

Branch and Bound with Convex Relaxations global solution.

  • Two convex relaxations of a disjunction

    • Big-M : Write a big-M formulation with 0-1 variables and take its continuous relaxation (i.e., drop the integrality requirement on the 0-1 variables).

    • Convex hull : Write a convex hull formulation with 0-1 variables and take its continuous relaxation.

  • Two solution options

    • Disjunctive programming : branch on disjunctions.

    • Mixed integer nonlinear programming (MINLP): branch on 0-1 variables.

  • Optimal value of relaxation provides a lower bound that is used in a branch-and-bound scheme.

Slide13 l.jpg

Big-M formulation of disjunction global solution.

The disjunction:

Big-M formulation:

Where Mv is a vector of valid upper bounds on the component functions of g(x,v).

It is assumed that x is bounded above and below.

To obtain relaxation, replace


Slide14 l.jpg

Relaxation… global solution.

Projection is

Example of big-M relaxation


Slide15 l.jpg

Convex hull formulation of disjunction global solution.Stubbs & Mehrotra; Grossmann & Lee

The disjunction:

Assume each g(x,v) is bounded as well as convex. Also assume xL x xU

Write every point in the relaxation as a convex combination of points satisfying the disjuncts

Use change of variable


Slide16 l.jpg

Restore convexity by multiplying by global solution.


This is a convex hull relaxation (i.e., projects onto convex hull in x-space).

But disaggregation of x adds many new variables.

To get 0-1 formulation, replace


Slide17 l.jpg

Convex hull relaxation… global solution.

Example of convex hull relaxation


Slide18 l.jpg

Solve structural design example with big-M formulation global solution.

Disjunctive formulation:

Big-M formulation:

Solve by disjunctive programming or MINLP. Get optimal solution at the root node.


Slide19 l.jpg

Solve structural design example with global solution.convex hull formulation

Disjunctive formulation:

Convex hull formulation:

Solve by disjunctive programming or MINLP.

Slide20 l.jpg

Logic-Based Outer Approximation global solution.Türkay and Grossmann

  • Allows one to use linear relaxations. But one must solve a mixed integer linear programming (MILP) master problem repeatedly.

  • Solve a master problem containing 1st-order approximations of the disjuncts to obtain a value for y.

    • Solve with MILP, which uses linear relaxations.

  • Solve the subproblem that results when y is fixed to this value, to get value for x.

    • Compute 1st-order approximations about previously obtained values of x, y.

  • Continue until value of master problem  best value obtained in a subproblem so far.

  • Begin with warm start by precomputing 1st-order approximations about several values of (x,y).

Slide21 l.jpg

Disjunctive formulation again: global solution.

The master problem in iteration K + 1is

where (xk,yk)are solutions from previous iterations.

The nonlinear subproblem in iteration K is

Slide22 l.jpg

Solve structural design example with global solution.logic-based outer approximation

Disjunctive formulation again:

Master problem:

Disjuncts already linear

Slide23 l.jpg

Solve master problem as MILP global solution.(Big-M formulation):`

For warm start, solve subproblem for 2 y’s:

y1 = (1,1), which yields x1 = (20,20)y2= (2,2), which yields x2 = (5,10).

This results in the master problem:

Slide24 l.jpg

Solve master problem and get global solution.

which implies

Subproblem solution is

Next master problem is



Solution is

and the algorithm terminates with y = (1,2).

Slide25 l.jpg

Logic-Based Benders Decomposition global solution.Hooker and Ottosson

  • Can be useful when variables have a large number of discrete values, resulting in a large number of disjuncts.

  • Convergence can be slow.

  • Solve a master problem for y.

    • The master problem incompletely describes the projection of the original problem onto the y-space.

  • Solve the subproblem that results when y is fixed to this value.

    • Obtain Benders cut from inference dual of the subproblem.

    • Add the cut to the master problem to rule out some solutions that are no better than the previous one.

  • Continue until the master and subproblem converge in value.

  • Best to have a warm start with “don’t-be-stupid” constraints involving y.

Slide26 l.jpg

Disjunctive formulation again: global solution.

The nonlinear subproblem in iteration K is

Lagrange multiplier

Optimal value =

The master problem in iteration K + 1is

Logical Benders cuts

Slide27 l.jpg

Solve structural design example with global solution.logic-based Benders decomposition

Initial master problem:

Don’t-be-stupid constraint

One solution is

Solve subproblem:

Corresponds to y1 = 1

Lagrange multipliers

Corresponds to y2 = 1

Subproblem solution is

Slide28 l.jpg

Since the master problem is global solution.

Solution is

Continue in this fashion. Master problem in iteration 4 is:


Solution is

The algorithm terminates with y = (1,2).

Slide29 l.jpg

Branch and Bound with Convex Quasi-Relaxations global solution.

  • Does not require disjunctive formulation and is therefore useful when there are many discrete values.

  • But the constraint functions must have a certain form.

  • Solve the problem by branch and bound.

  • Obtain bounds from quasi-relaxations at each node.

  • Given problem P:

a problem Q:

is a quasi-relaxation of P if for any feasible solution x of P, there is a feasible solution xof Q with f (x )  f (x).

Thus one can obtain a valid lower bound by solving a quasi-relaxation.

Slide30 l.jpg

Consider the problem, global solution.

Theorem. Suppose that eachis either

(a) convex [for (i,j) J1] or(b) concave in yjand homogeneous in x:

[for (i,j)  J2].

Suppose also that

Then the following is a convex quasi-relaxation :

Slide31 l.jpg

Why? global solution.

Take any feasible solution of

To obtain a feasible solution of

do the following:




Slide32 l.jpg

satisfied, by global solution.construction

convex, because gj(x,y) is convex

satisfied, by above argument

satisfied, by construction

convex, because gj(x,y) is convex in x

So we have a feasible solution of the quasi-relaxation with value that is less than or equal to (in fact equal to) that of the original problem.

Slide33 l.jpg

Solve continuous version of structural design example with quasi-relaxations

Original formulation:


Put in proper form:


Concave in yj & homogeneous in 1st argument (sj, xj)

Slide34 l.jpg

The quasi-relaxation is: quasi-relaxations

Can now re-aggregate sj:

Slide35 l.jpg

Beginning of branch-and-bound tree quasi-relaxations

Total 63 nodes out of 3131 possible solutions.

Get y = (1.1, 2.0)with z0 = 1394.5

Root node

x0 = 1177.8 = (0.667,0.667) y = (1,1)



x0 = 1322 = (0,0.667) y = (1,1)

x0 = 1283 = (0.816,0.667) y = (1.45,1)



Global optimum isy = (1.126, 1.972)with z0 = 1394.1

x0 =1352  = (0,0.816) y = (1,1.45)

x0 =1900  = (0,0) y = (1,1)feasible solution

Slide36 l.jpg

Realistic Structural Design Problem quasi-relaxations


Degree of freedom i


Cross-sectional area



Bar j

Hooke’s law



Elongation bounds

Displacement bounds

Slide37 l.jpg

Solution as MI(N)LP quasi-relaxationsGhattas and Grossmann

The disjunctive formulation is

Discrete sizes for bar j

Since everything is linear, the big-M and convex hull formulations are linear.

Can solve as a mixed integer linear programming (MILP) problem.

Slide39 l.jpg

Solution with convex quasi-relaxations quasi-relaxationsBollapragada, Ghattas and Hooker

Check that the problem has the right form:


Concave (linear) in yj and homogeneous in vj, fj

Slide40 l.jpg

The quasi-relaxation is quasi-relaxations

Slide41 l.jpg

Some problem instances quasi-relaxations

10-bar cantilever truss

25-bar electrical transmission tower

Slide42 l.jpg

72-bar building quasi-relaxations

Use symmetries to help solve problem

Slide44 l.jpg

Applications of Logic-Based Methods for Nonlinear Global Optimization

  • Logic-based outer approximation applied to chemical processing network design

    • Quesada and Grossmann 1992; Türkay and Grossmann 1996

  • Disjunctive programming applied to chemical processing network design

    • Vecchietti and Grossmann 1999

  • Convex quasi-relaxations applied to truss structure design

    • Bollapragada, Ghattas and Hooker 2001

  • Disjunctive programming applied to tray placement in distillation columns

    • Barttfeld, Aguire and Grossmann 2003

  • Logic-based Benders decomposition applied to planning and scheduling (linear case)

    • Hooker 2000, 2003.; Jain and Grossmann 2001

Slide45 l.jpg

Surveys Optimization

  • I. E. Grossmann, Review of nonlinear mixed-integer and disjunctive programming techniques for process systems engineering, Carnegie Mellon University, June 2001

  • J. N. Hooker, Logic-Based Methods for Optimization, John Wiley & Sons, 2000.