Introduction to integer programming modeling and methods
This presentation is the property of its rightful owner.
Sponsored Links
1 / 70

Introduction to Integer Programming Modeling and Methods PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Introduction to Integer Programming Modeling and Methods. Michael Trick Carnegie Mellon University CPAI-OR School, Le Croisic 2002. Some History. Integer Programming goes back a long way:

Download Presentation

Introduction to Integer Programming Modeling and Methods

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Introduction to integer programming modeling and methods

Introduction to Integer Programming Modeling and Methods

Michael Trick

Carnegie Mellon University

CPAI-OR School, Le Croisic 2002


Some history

Some History

  • Integer Programming goes back a long way:

    • Schrijver takes it back to ancient times (linear diophantine equations), Euler (1748), Monge (1784) and much more.

    • “Proper” study began in the 1950s

      • Dantzig (1951): linear programming

      • Gomery (1958): cutting planes

      • Land and Doig (1960): branch and bound

    • Survey books practically every five years since

    • Tremendous practical success in last 10-15 years


Scope

Scope

  • This talk will not be comprehensive!

  • Attempt to get across main concepts of integer programming

    • Relaxations

    • Primal Heuristics

    • Branch and Bound

    • Cutting Planes


Integer program ip

Integer Program (IP)

Linear objective

Minimize cx

Subject to

Ax=b

l<=x<=u

some or all of xj integral

X: variables

Linear constraints

Makes things hard!


Rules of the game

Rules of the Game

  • Must put in that form!

  • Seems limiting, but 50 years of experience gives “tricks of the trade”

  • Many formulations for same problem


Example formulations

Example Formulations

Warehouse location: n stores, m possible warehouses; cost k[j] to open warehouse m; cost c[i,j] to handle store i out of warehouse j.

Minimize the total opening costs plus handling costs

Subject to

Each store assigned to one open warehouse


Warehouse formulation

Warehouse Formulation

Variables

x[i,j] = 1 if store i served by warehouse j; 0 otherwise

y[j] = 1 if warehouse j open; 0 otherwise

Objective

Minimize sum_j k[j]y[j]+sum_i,j c[i,j]x[i,j]


Warehouse formulation1

Warehouse Formulation

  • Constraints:

    sum_j x[i,j] = 1 for all i

    sum_i x[i,j] <= ny[j] for all j


Binary integer programs

Binary Integer Programs

  • Restrict variables to be 0-1

  • Many specialized methods

  • OR people are real good at formulating difficult problems within these restrictions


Key concepts

Key concepts

  • Relaxation R(IP)

    • “easily” solved problem such that

      • Optimal solution value is no more than that of IP

      • If solution is feasible for IP then is optimal for IP

      • If R(IP) infeasible then so is IP

    • Most common is “linear relaxation”: drop integrality requirements and solve linear program

      • Others possible: lagrangian relaxation, lagrangian decomposition, bounds relaxation, etc.


Illustration

Illustration


Linear relaxation

Linear Relaxation


Why this fetish with linear relaxations

Why this fetish with Linear Relaxations?

  • IP people are very focused on linear relaxations. Why?

    • Sometimes linear=integer

    • Linear relaxations as global constraints

    • Duals and reduced costs


Linear integer formulations

Linear=integer formulations

  • Happens naturally for some problems

    • Network flows

    • Totally unimodular constraint matrices

  • Takes more work, but defined for

    • Matchings

    • Minimum spanning trees

  • Closely associated with polynomial solvability


Duals and reduced costs

Duals and Reduced Costs

Associated with the solution of a linear program are the dual values

--- one per constraint

--- measures the marginal value of changing the right-hand-side of the constraint

--- Useful in many algorithmic ways


Dual example

Dual example

Sum_i x[i,j]-y[j] <= 0

Suppose facility j* has cost 10 and

y*[j*] = 0.

The dual value of this constraint is 4. What cost must facility j* have to be appealing?

Answer: no more than 10-4=6.


Dual example 2

Dual Example 2

  • Products 1, 2, 3 use chemicals A, B

    Maximize 3x1+2x2+2x3

    Subject to

    x1+x2+2x3 <= 10; (.667)

    5x1+2x2+x3 <= 20; (.667)

    Solution: x2=6.67 x1=1.67

    What objective must a product that uses 4 of A and 3 of B have to be appealing: at least 4.67


Final advantage of linear relaxations global

Final advantage of linear relaxations: Global

  • Linear relaxations are

    • Relatively easy to solve: huge advances in 15 years

    • Incorporate “global” information

    • Often provide good bounds and guidelines for integer program

      • Variables with very bad reduced cost likely not in optimal integer solution

      • Rounding doesn’t always work, but often gets good feasible solutions


Feasible solutions

Feasible solutions

  • Solutions that satisfy all the constraints but might not be optimal

    • Generally found by heuristics

    • Can be problem specific

  • Must have value greater than or equal to optimal value (for minimizing)


Feasible solution

Feasible Solution

Feasible solution


Fundamental branch and bound algorithm

Fundamental Branch and Bound Algorithm

Solve relaxation to get x*

If infeasible, then IP infeasible

Else If x* feasible to IP, then x* optimal to IP

Else create new problems IP1 and IP2 by branching; solve recursively, stop if prove subproblem cannot be optimal to IP (bounding)


Branching

Branching

  • Create two or more subproblems IP1, IP2,… IPn such that

    • Every feasible solution to IP appears in at least one (often exactly one) of IP1, IP2, … IPn

    • x* is infeasible to each of R(IP1), R(IP2), … R(IPn)

  • For linear relaxation, can choose a fraction xj* and have one problem with xj <=[xj] and the other with xj >= [xj]+1 ([x]: round down of x)


Illustration1

Illustration

IP1

x*

IP2


Bounding

Bounding

  • Along way, we may find solution x’ that is feasible to IP. If any subproblem has relaxation value c* >= cx’ then we can prune that subproblem: it cannot contain the optimal solution. There is no sense continuing on that subproblem.


Stopping

Stopping

  • Technique can stop early with solution within a provable percentage of optimal (compare to be relaxation value)

  • Can also modify to generate all solutions (do not prune on ties)


How to make work better

How to make work better?

  • Better formulations

  • Better relaxations (cuts)

  • Better feasible solutions (heuristics)


Formulations

Formulations

  • Different formulations of integer programs may act very differently: their relaxations might have radically different bounds

  • “Good Formulation” of integer program: provides a better relaxation value (all else being equal).


Back to warehouse example

Back to Warehouse Example

  • Alternative formulation of “Only use if open constraint”

    x[i,j] <= y[j] for all i,j

    (versus)

    sum_i x[i,j] <= ny[j]

  • Which is better?


Comparing

Comparing

  • Positives to original

    • Fewer constraints: linear relaxation should solve faster

  • Positives to disaggregate formulation

    • Much better bounds (consider having x[i,j]=1 for a particular i,j. What would y[j] be in the two formulations?)

  • (Almost) no comparison! Formulation with more constraints works much better.


Ideal

Ideal

Formulation gives convex hull of feasible integer points


Embarrassing formulations

Embarrassing Formulations

  • Some things are very hard to formulate with integer programming:

    • Traveling Salesman problem: great success story (IP approaches can optimize 15,000 city problems!), but best IP approaches begin with an exponentially sized formulation (no “good” compact formulation known).

    • Complicated operational requirements can be hard to formulate.


Further approaches

Further approaches

  • Branch and Price

    • Formulations with exponential number of variables with complexity in generating “good” variables (see Nemhauser): heavy use of dual values

  • Branch and Cut

    • Improving formulations by adding additional constraints to “cut off” linear relaxation solutions (more later)


Algorithmic details

Algorithmic Details

Preprocessing

Primal Heuristics

Branching

Cut Generation


Preprocessing

Preprocessing

  • Process individual rows to

    • detect infeasibilities

    • detect redundancies

    • tighten right-hand-sides

    • tighten variable bounds

  • Probing: examine consequences of fixing 0-1 variable

    • If infeasible, fix to opposite bound

    • If other variables are fixed, inequalities


Preprocessing1

Preprocessing

  • Much like simple Constraint Programming


Improving coefficients

Improving Coefficients

3x1-2x2 1

1. Convert to  with pos. coefficients with y1 = 1-x1

3y1 + 2x2  2

2. Note that constraint always satisfied when y1 = 1, so change coefficient 3 to 2

2y1 + 2x2 2

3. Convert back to original

x1 -x2 1


Improving coefficients1

Cuts off (1/2, 1/4) and others

Improving (?) Coefficients

1

1


Manual or automatic

Modeling issue

Automatic identification not foolproof

Generally easy to see

Can provide problem-knowledge to further reduce coefficients

Automatic issue

Many opportunities will only occur within Branch and Bound tree as variables are fixed

“More foolproof” as models change

Manual or Automatic?


Identifying redundancy and infeasibility

Identifying Redundancy and Infeasibility

Use upper and lower bounds on variables:

  • Redundancy

    3x1 - 4x2+ 2x3 6 (max lhs is 5)

  • Infeasibility

    3x1 - 4x2+ 2x3 6 (max lhs is 5)

    While very simple, can be used to fix variables, particularly within B&B tree


Pp fixing variables

PP: Fixing Variables

Simple idea: if setting a variable to a value leads to infeasibility, then must set to another value

3x1-4x2+2x3-3x4 3

Setting x4 to 1 leads to previous infeasible constraint, so x4 must be 0


Pp implication inequalities

PP: Implication Inequalities

Many constraints embed restrictions that at most one of x and y (or their complements) are 1.

This can lead to implication inequalities.


Pp implication inequalities1

PP: Implication Inequalities

Facility location

x1+x2+…+xm mx0

x0= 0  x1 = 0

x0= 0  x2 = 0, etc.

(1-x0) + x1  1 (or x1 x0)

x2 x0 , etc.

Automatic disaggregation (stronger!)


Pp clique inequalities

PP: Clique Inequalities

These inequalities found by “probing” (fix variable and deduce implications).

These simple inequalities can be strengthened by combining mutually exclusive variables into a single constraint.

Resulting clique inequalities are very “strong” when many variables combined.


Example sports scheduling

Example: Sports Scheduling

Problem: Given n teams, find an “optimal” (minimum distance, equal distance, etc.) double round robin (every team plays at every other team) schedule.

A: @B @C D B C @D

B: A D @C @A @D C

C: @D A B D @A @B

D: C @B @A @C B A


Sports scheduling

Sports Scheduling

Many formulations (not wholly satisfactory)

One method: One variable for every “home stand” (series of home games) and “away trip” (series of away games).


Variables team a

Variables (team A)

Some variables:

y1

H

x1

@B

@C

x2

@C

@D

y1

H

H

x3

@E

@F

1

2

3

4


Constraints

Constraints

  • Can only do one thing in a time slot

    y1+x1+x2  1

    x1+x2+y2  1

  • No “Away after Away”

    x1+x2+x3  1

  • No “Home after Home”

    y1+y2  1

    Additional constraints link teams


Improving formulation

Improving Formulation

Create Implication Graph

y1

H

x1

@B

@C

x2

@C

@D

y2

H

H

x3

@E

@F

1

2

3

4


Find cliques

Find cliques

  • Cliques in graph: can only have one

y1

H

x1

@B

@C

x2

@C

@D

y2

H

H

x3

@E

@F

1

2

3

4


Constraints1

Constraints

  • Can only do one thing in a time slot

    y1+x1+x2  1

    x1+x2+y2  1

  • No “Away after Away”

    x1+x2+x3 + y2 1

  • No “Home after Home”

    y1+y2 + x1 + x2  1

    Additional constraints link teams


Clique inequalities

Clique Inequalities

  • Resulting formulation is much tighter (turns formulation from hopeless to possible)

  • Idea generalized to variables and their complements

  • Can be found automatically, but may be a huge number (and clique generally hard)

  • On divide of “automatic” and “manual” modeling issue


Primal heuristics

Primal Heuristics

  • Feasible solutions at B&B nodes can greatly decrease the solution time

  • Most common: problem-specific heuristics embedded with B&B

  • Some general purpose heuristics: LP Diving, Pivot and Complement


Lp diving

LP-Diving

1. Solve LP

2. Stop if infeasible or integral

3. Fix all integral variables (or all 1’s)

4. Select fractional variable and fix to integer

5. Go to 1


Branching1

Branching

  • Two decisions: which B&B node to branch on and how to divide into two problems

  • Node selection

    • Depth First: try for integral solution

    • Best Bound: explore “appealing” nodes

    • Adaptive: Depth First first, then best bound


Branching how to branch

Branching: How to branch

  • Priorities on variables and sets is extremely important

  • Normal branching is on a variable equals 0 or equals 1, but much more complicated branching possible:

    x1+ x2+ x3 + x4 1

    Could lead to two problems:

    a) x1+ x2= 0

    b) x3 + x4=0


Branching as modeling

1

k-2

k-1

k

k+1

k+2

m

Branching as Modeling

Specially Ordered Sets of Type 2 (no more than 2 positive, must be adjacent): used to model piecewise linear functions:

x1,x2,x3,… xm

can lead to two problems:

Either all of the blue or all of the red are 0


Adding constraints

Adding constraints

  • Constraints can strengthen formulation: we have seen clique inequalities already

  • Can be added on an “as needed” basis during calculations (generally to move away from current fractional solution).


Separation problem

Separation Problem

Given a fractional solution to the LP relaxation, find an inequality that is not satisfied by the fractional solution. Algorithm should be

- fast

- yield strong inequality

- heuristics acceptable


Types of constraints

Types of Constraints

1. Feasibility: A large number of constraints is used in formulation

- connectivity constraints (i.e. TSP)

- nonlinearities

2. Problem specific facets

- blossom inequalities for matching

- comb inequalities for TSP

3. General IP constraints


Gomory constraints

Gomory Constraints

  • Any fractional solution from the simplex method can be separated:

    xb + 2.5 x1 - 3.2 x2= 2.1

    where current sol. is xb =2.1, x1 = x2= 0

    xb + 2 x1 - 4 x2 - 2= .1 - .5 x1 -.8 x2

    LHS is integer so RHS must be also

    .1 - .5 x1 -.8 x2  0

    is valid and violated by current solution


Real version of gomory cuts

y+ sum_j ajxj =d

Let d=[d]+f; let aj=[aj] + fj

t=y+sum_(j:fj<=f) [aj]xj + sum_(j:fj>f) ([aj]+1)xj

So

d-t = sum_(j:fj<=f) fjxj+sum_(j:fj>f) (fj-1)xj

Either

t<=[d] => sum_(j:fj<=f) fjxj >= f

t>=[d]+1 => sum_(j: fj>f) (1-fj)xj >= 1-f

Divide through to get RHS of 1 in each case. Result gives

sum(j:fj<=f) (fj/f)xj + sum(j: fj>f) (1-fj)/(1-f)xj >= 1

“Real Version” of Gomory Cuts


Working through example

Working through example

Previous example becomes

.5x1+.2x2 >= .9, which is a good, strong constraint

(can extend all this to mixed integer programs easily)


Gomory constraints modeling

Gomory Constraints (modeling)

  • Are we done? Very good to have available but likely not the only tool in the arsenal.


0 1 knapsack covers

0-1 Knapsack Covers

For problems that contain constraints like:

j N ajxj b

C  N is a cover if

j C aj> b

then

j C xj |C| - 1

is valid


Separation of cover inequalities

Separation of Cover Inequalities

Given fractional LP solution x*, is there a cover C for which x* violates the cover inequality?

Solvable by a binary knapsack problem (one constraint IP): can be solved heuristically, or exactly by dynamic programming


Lifting of cover inequalities

Lifting of Cover Inequalities

Constraints can be strengthened:

20x1+16x2+15x3+10x4+30x5 40

Cover on {1,2,3} leads to

x1+x2+x3+0x4+0x5 2

Is the “0” coefficient on x4 the best possible? Can solve knapsack to get

x1+x2+x3+x4+0x5 2


Lifing of cover inequalities

Lifing of Cover Inequalities

We could continue to x5 to get

x1+x2+x3+x4+x5 2

If we had done x5 first, we would have got

x1+x2+x3+0x4+2x5 2

Different lifting sequences lead to different inequalities


Modeling implications

Modeling implications

  • Cover inequalities generally part of the software, rather than the modeling (but if software has capability, do not include in model).

  • Decision is whether to use the inequalities or not essentially numerical question

  • Lifting is extremely important, as is relationship of covers in subproblems to full problem


Conclusions

Conclusions

  • There is lots to integer programming!

  • Interesting interplay between formulations and algorithms

  • Much intelligence embedded in software, making formulations a bit more “fool-proof”


Papers to read

Papers to Read

Progress in Linear Programming Based Branch and Bound Algorithms: An exposition, Ellis Johnson, George Nemhauser, and Martin Savelsbergh

MIP: Theory and Practice – Closing the Gap, Robert Bixby, Mary Fenelon, Zonghao Gue, Ed Rothberg, and Roland Wunderling


  • Login