Explorations in Artificial Intelligence. Prof. Carla P. Gomes gomes@cs.cornell.edu Module 7 Part 3 Integer Programming . Divisibility
Related searches for Explorations in Artificial Intelligence
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Prof. Carla P. Gomes
gomes@cs.cornell.edu
Module 7
Part 3
Integer Programming
Decision variables in an LP model are allowed to have any values, including noninteger values, that satisfy the functional and nonnegativity constraints. i.e., activities can be run at fractional levels.
What to do when divisibility assumption violated:
realm of integer programming!!!
Question: How many airplanes of each type should be purchased to maximize their total net annual profit?
Let S = Number of small airplanes to purchaseL = Number of large airplanes to purchaseMaximize Profit = S + 5L ($millions)subject to Capital Available: 5S + 50L ≤ 100 ($millions) Max Small Planes: S ≤ 2and
S ≥ 0, L ≥ 0.
Let S = Number of small airplanes to purchaseL = Number of large airplanes to purchaseMaximize Profit = S + 5L ($millions)subject to Capital Available: 5S + 50L ≤ 100 ($millions) Max Small Planes: S ≤ 2and
S ≥ 0, L ≥ 0S, L are integers.
StockCompany is considering 6 investments. The cash required from each investment as well as the NPV of the investment is given next. The cash available for the investments is $14,000. Stockco wants to maximize its NPV. What is the optimal strategy?
An investment can be selected or not. One cannot select a fraction of an investment.
Investment budget = $14,000
What are the decision variables?
Objective and Constraints?
Max 16x1+ 22x2+ 12x3+ 8x4+ 11x5+ 19x6
5x1+ 7x2+ 4x3+ 3x4+ 4x5+ 6x6 14
xje {0,1} for each j = 1 to 6
A hiker trying to fill her knapsack to maximum total value.
Each item she considers taking with her has a certain value and a
certain weight. An overall weight limitation gives the single
constraint.
Practical applications:
Project selection and capital budgeting allocation problems
Storing a warehouse to maximum value given the indivisibility of goods and space limitations
Subproblem of other problems e.g., generation of columns for a given model in the course of optimization – cutting stock problem (beyond the scope of this course)
A 2dimensional representation
Stock 2
Stock 1
The integer programming constraint:
If stock 2 is selected then so is stock 1x1 x2
A 2dimensional representation
Stock 3
Stock 1
The integer programming constraint:
If stock 1 is selected then stock 3 is not selectedx1 + x3 1
A 2dimensional representation
stock 5
stock 4
The integer programming constraint:
Either stock 4 is selected or stock 5 is selected, but not both.x4 + x5 = 1
Question: Should the California Manufacturing Company expand with factories and/or warehouses in Los Angeles and/or San Francisco?
Let x1 = 1 if build a factory in L.A.; 0 otherwisex2 = 1 if build a factory in S.F.; 0 otherwisex3 = 1 if build a warehouse in Los Angeles; 0 otherwisex4 = 1 if build a warehouse in San Francisco; 0 otherwiseMaximize NPV = 8x1 + 5x2 + 6x3 + 4x4 ($millions)subject to Capital Spent: 6x1 + 3x2 + 5x3 + 2x4 ≤ 10 ($millions) Max 1 Warehouse: x3 + x4 ≤ 1 Warehouse only if Factory: x3 ≤ x1x4 ≤ x2andx1, x2, x3, x4 are binary variables.
Resource Availability
Mutually exclusive decisions
Contingent decisions
Max 8x1 + 5x2 + 6x3 + 4x4;
subject to
6x1 + 3x2 + 5x3 + 2x4 <= 10;
x3 + x4 <= 1;
x3 <= x1;
x4 <= x2;
BINARY
x1;
x2;
x3;
x4;
If a product is produced, must incur a fixed setup cost.
If a warehouse is operated, must incur a fixed cost.
x – quantity of product to be manufactured
x = 0 cost =0;
x > 0 cost = C1x + C2
y = 1 x is produced; y = 0 x is not produced
Objective function becomes C1x + C2y
Additional Constraint x ≤ My
Suppose that two changes are made to the original Wyndor problem:
Optimal solution
Let D = Number of doors to produce,W = Number of windows to produce,y1 = 1 if perform setup to produce doors; 0 otherwise,y2 = 1 if perform setup to produce windows; 0 otherwise .Maximize P = 300D + 500W – 700y1 – 1,300y2subject to Original Constraints: Plant 1: D ≤ 4 Plant 2: 2W ≤ 12 Plant 3: 3D + 2W ≤ 18 Produce only if Setup: Doors: D ≤ My1 Windows: W ≤ My2andD ≥ 0, W ≥ 0, y1 and y2 are binary.
Suppose that now the only change from the original Wyndor problem is:
Let D = Number of doors to produce,W = Number of windows to produce,y1 = 1 if produce doors; 0 otherwise,y2 = 1 if produce windows; 0 otherwise.Maximize P = 300D + 500Wsubject to Original Constraints: Plant 1: D ≤ 4 Plant 2: 2W ≤ 12 Plant 3: 3D + 2W ≤ 18 Auxiliary variables must =1 if produce any: Doors: D ≤ My1 Windows: W ≤ My2 Mutually Exclusive: y1 + y2 ≤ 1 andD ≥ 0, W ≥ 0, y1 and y2 are binary.
Suppose that now the only change from the original Wyndor problem is:
Let D = Number of doors to produce,W = Number of windows to produce,y = 1 if plant 4 is used; 0 if plant 3 is usedMaximize P = 300D + 500Wsubject to Plant 1: D ≤ 4 Plant 2: 2W ≤ 12 Plant 3: 3D + 2W ≤ 18 + My Plant 4: 2D + 4W ≤ 28 + M(1 – y) andD ≥ 0, W ≥ 0, y is binary.
Locate fire stations so that each district has a fire station in it, or next to it.
1
2
3
5
6
7
Minimize the number of fire stations needed.
4
8
9
11
12
13
10
14
15
16
Set
Covers
1
2
3
1
1, 2, 4, 5
2
1, 2, 3, 5, 6
5
6
7
3
2, 3, 6, 7
4
8
9
11
12
13
10
16
13, 15, 16
14
15
16
1
2
3
7
5
6
7
4
8
9
11
11
12
13
15
10
14
15
16
Representation as Integer programxj = 1 if node j is selectedxj = 0 otherwise
Minimize x1 + x2 + … + x16
s.t. x1 + x2 + x4 + x5 1
x1 + x2 + x3 + x5 + x6 1
x13 + x15 + x16 1
xj {0, 1} for each j.
Question: How should the 3 crews be assigned 3 sequences of flights so that every one of the 11 flights is covered?
Let xj = 1 if flight sequence (paring) j is assigned to a crew; 0 otherwise. (j = 1, 2, … , 12).Minimize Cost = 2x1 + 3x2 + 4x3 + 6x4 + 7x5 + 5x6 + 7x7 + 8x8 + 9x9 + 9x10 + 8x11 + 9x12 (in $thousands)subject to Flight 1 covered: x1 + x4+ x7 + x10 ≥ 1 Flight 2 covered: x2 + x5 + x8 + x11 ≥ 1 : : Flight 11 covered: x6 + x9 + x10 + x11 + x12 ≥ 1 Three Crews: x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10 + x11 + x12 ≤ 3 andxj are binary (j = 1, 2, … , 12).
pairings
Investment budget = $14,000
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14
xj binary for j = 1 to 6
x2 = 1
x2 = 0
x2 = 1
x2 = 0
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
An Enumeration TreeOriginal problem
x1 = 0
x1 = 1
The essential idea: search the enumeration tree, but at each node
Node 1 is the original LP Relaxation
44 3/7
1
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14
0 xj 1 for j = 1 to 6
Solution at node 1:
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
The IP cannot have value higher than 44 3/7.
x1 = 0
2
Branch and BoundNode 2 is the original LP Relaxation plus the constraint x1 = 0.
44 3/7
1
44
maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6
subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14
0 xj 1 for j = 1 to 6, x1 = 0
Solution at node 2: x1 = 0 x2 = 1 x3 = 1/4 x4 = x5 = 0 x6 = 1 z = 44
x1 = 1
x1 = 0
3
2
Branch and BoundNode 3 is the original LP Relaxation plus the constraint x1 = 1.
44 3/7
1
44 3/7
44
The solution at node 1 was
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
Note: it was the best solution with no constraint on x1. So, it is also the solution for node 3. (If you add a constraint, and the old optimal solution is feasible, then it is still optimal.)
x2 = 0
4
Branch and BoundNode 4 is the original LP Relaxation plus the constraints x1 = 0, x2 = 0.
44 3/7
1
1
x1 = 0
x1 = 1
2
3
44
44 3/7
4
42
Solution at node 4: 0 0 1 0 1 1 z = 42
Our first incumbent solution!
No solution in the subtree can have a value better than 42.
No further searching from node 4 because there cannot be a better integer solution.
x2 = 1
x2 = 0
x2 = 1
x2 = 0
4
5
6
7
Branch and BoundThe incumbent solution has value 42
The incumbent is the best solution on hand.
44 3/7
1
1
x1 = 0
x1 = 1
2
3
44
44 3/7
4
42
44
44
44 1/3
We next solved the LP’s associated with nodes 5, 6, and 7
No new integer solutions were found.
We would eliminate (fatham) a subtree if we were guaranteed that no solution in the subtree were better than the incumbent.
x2 = 1
x2 = 0
x2 = 1
x2 = 0
4
5
6
7
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
8
9
10
11
12
13
Branch and BoundThe incumbent solution has value 42
44 3/7
1
x1 = 0
x1 = 1
2
3
44
44 3/7
4
42
44
44
44 1/3
13
43.75
43.5
43.25
43.8
44.3

We next solved the LP’s associated with nodes 8 13
x2 = 1
x2 = 0
x2 = 1
x2 = 0
5
6
7
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
8
9
10
11
12
Branch and BoundThe incumbent solution has value 42
44 3/7
1
x1 = 0
x1 = 1
2
3
44
44 3/7
4
42
44
44
44 1/3
x3 = 1
43.75
43.5
43.25
43.8
44.3

13
43.75
42.66
We next solved the LP’s associated with the next nodes.
We can fathom the node with z = 42.66. Why?
x2 = 1
x2 = 0
x2 = 1
x2 = 0
5
6
7
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
8
9
10
11
12
Branch and BoundThe incumbent solution has value 42
44 3/7
1
x1 = 0
x1 = 1
2
3
44
44 3/7
4
42
44
44
44 1/3
x3 = 1
43.75
43.5
43.25
43.8
44.3
.
13
43.75
42.66
x2 = 1
x2 = 0
x2 = 1
x2 = 0
5
6
7
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
8
9
10
11
12
Branch and BoundThe incumbent solution has value 42
44
1
x1 = 0
x1 = 1
2
3
44
44
4
42
44
44
44
x3 = 1
43
43
43
43
44

13
43
42
43
42
43
43
We found a new incumbent solution!
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43
x2 = 1
x2 = 0
x2 = 1
x2 = 0
5
6
7
Branch and BoundThe new incumbent solution has value 43
44
1
x1 = 0
x1 = 1
2
3
44
44
4
42
44
44
44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
8
9
10
11
43
43
43
43
44
8
9
10
11
12

13
43
42
43
42
43
43
We found a new incumbent solution!
x1 = 1, x2 = x3 = 0, x4 = 1, x5 = 0, x6 = 1 z = 43
x2 = 1
x2 = 0
x2 = 1
x2 = 0
5
6
7
Branch and BoundThe new incumbent solution has value 43
44
1
x1 = 0
x1 = 1
2
3
44
44
4
42
44
44
44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
8
9
10
11
43
43
43
43
44
8
9
10
11
12

13
If we had found this incumbent earlier, we could have saved some searching.
x2 = 1
x2 = 0
x2 = 1
x2 = 0
5
6
7
44
14
15

44
16
17

38

18
19
Finishing UpThe new incumbent solution has value 43
44
1
x1 = 0
x1 = 1
2
3
44
44
4
42
44
44
44
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
8
9
10
11
43
43
43
43
44
8
9
10
11
12

13
Notation:
NOTE: zIz*
1
x1 = 0
x1 = 1
2
3
44
44 3/7
x2 = 1
x2 = 0
x2 = 1
x2 = 0
4
4
5
6
7
42
44
44
44 1/3
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
13
43.75
8
43.5
9
43.25
10
43.8
11
44.3
12

13
Illustrating the definitionsThe incumbent solution has value 42
z* = 43 = optimal integer solution value. (We found it later in the search)
Incumbent is 0 0 1 0 1 1 zI = 42. It is the optimal solution for the subdivision 4.
The zLP values for each subdivision are next to the nodes.
1
x1 = 0
x1 = 1
2
3
44
44 3/7
x2 = 1
x2 = 0
x2 = 1
x2 = 0
4
4
5
6
7
42
44
44
44 1/3
x3 = 0
x3 = 1
x3 = 0
x3 = 1
x3 = 0
x3 = 1
13
43.75
8
43.5
9
43.25
10
43.8
11
44.3
12

13
Illustrating the definitionsThe incumbent solution has value 42
The children of node (subdivision) 1 are nodes 2 and 3.The children of node 3 are nodes 6 and 7.
LIST = { 8, 9, 10, 11, 12 } = unfathomed nodes with no active children
44 3/7

13
Branch and Bound AlgorithmINITIALIZE LIST = {original problem}Incumbent: = zI = 
SELECT:
If LIST = , then the Incumbent is optimal if it exists, and the problem is infeasible if no incumbent exists;
else, let S be a node (subdivision) from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
e.g., S = {1}
e.g., S = {13}
13
CASE 1. zLP =  (the LP is infeasible)
Remove S from LIST (fathom it)
Return to SELECT
8
8
42
43
14
Branch and Bound AlgorithmINITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the problem is infeasible if no incumbent exists;
else, let S be a node from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
CASE 2.  < zLP zI.
That is, the LP is dominated by the incumbent.
Then remove S from LIST (fathom it)
Return to SELECT
e.g., the incumbent has value 43, and node 14 is selected. zLP = 43.
14
42
Branch and Bound AlgorithmINITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the problem is infeasible if no incumbent exists;
else, let S be a subdivision from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
CASE 3. zI < zLP and xLP is integral.
That is, the LP solution is integral and dominates the incumbent.
Then Incumbent := xLP;zI := zLP
Remove S from LIST (fathomed by integrality)
Return to SELECT
e.g., node 4 was selected, and the solution to the LP was integervalued.
4
x2 = 1
7
x1 = 1
3
x2 = 0
6
Branch and Bound AlgorithmINITIALIZE
SELECT:
If LIST = , then the Incumbent is optimal (if it exists), and the problem is infeasible if no incumbent exists;
else, let S be a subdivision from LIST.
Let xLP be the optimal solution to S
Let zLP = its objective value
CASE 4. zI < zLP and xLP is not integral.
There is not enough information to fathom S
Remove S from LIST
Add the children of S to LIST
Return to SELECT
e.g., select node 3.
44 3/7
List := List – 3 + {6,7}
= 0
= 1
X
X
8
8
X
3
One does not have to have the B&B tree be symmetric, and one does not select subtrees by considering variables in order.
Choosing how to branch so as to reduce running time is largely “art” and based on experience.
= 1
= 0
X
X
1
1
= 0
= 1
X
X
2
2
= 0
X
3
= 0
X
= 1
X
3
= 0
3
= 1
= 1
X
X
= 0
= 1
3
X
X
3
3
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
New exciting area
A*  wellknown AI algorithm that is a generalization of branchandbound with LP relaxations – notion of admissible heuristic that overestimates (underestimates) the objective function for a maximization (minimization) problem (analogously to what a relaxation does)
Used in e.g. MapQuest and Darpa Challenge
It was fun to teach INFO 372! branchandbound with LP relaxations – notion of admissible heuristic that overestimates (underestimates) the objective function for a maximization (minimization) problem (analogously to what a relaxation does)
Hope you had fun too!
THE END
!!!