The Simplex Algorithm

1 / 78

# The Simplex Algorithm - PowerPoint PPT Presentation

The Simplex Algorithm. LI Xiao-lei. preface. The two-variable LP program can be solved graphically. Most real-life LPs have many variables. In many industrial applications, the simplex algorithm is used to solve LPs with thousands of constraints and variables.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## The Simplex Algorithm

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### The Simplex Algorithm

LI Xiao-lei

preface
• The two-variable LP program can be solved graphically.
• Most real-life LPs have many variables.
• In many industrial applications, the simplex algorithm is used to solve LPs with thousands of constraints and variables.
How to convert an LP to standard form
• Before the simplex algorithm can be used to solve an LP, the LP must be converted into an equivalent problem in which all constraints are equations and all variables are nonnegative. An LP in this form is said to be in standard form.
EXAMPLE 1
• Leather limited manufactures two types of belts: the deluxe model and the regular model. Each type requires 1 sq yd of leather. A regular belt requires 1 hour of skilled labor, and a deluxe belt requires 2 hours. Each week, 40 sq yd of leather and 60 hours of skilled labor are available. Each regular belt contributes \$3 to profit and each deluxe belt, \$4.
EXAMPLE 1
• Define,

X1=number of deluxe belts produced weekly

X2=number of regular belts produced weekly

• The LP is,

Max z=4x1+3x2 (LP1)

s.t. x1+ x2≤40 (Leather constraint)

2x1+ x2≤60 (Labor constraint)

x1,x2≥0

Convert inequality constraints to equality constraints
• Define for each ≤ constraint a slack variable si (si=slack variable for ith constraint).

We define,

s1=40- x1-x2 or x1+ x2+ s1=40

s2=60-2x1-x2 or 2x1+ x2 + s2=60

Note: a point satisfies the ith constraint if and only if si≥0.

LP1 in standard form

Max z=4x1+3x2 (LP1’)

s.t. x1+ x2 +S1 =40

2x1+ x2 +S2 =60

x1,x2,S1,S2≥0

Note: if constraint i of an LP is a ≤ constraint, we convert it to an equality constraint by addinga slack variable si to the ith constraint and adding the sign restriction si ≥0.

Similarly, to convert the ith ≥ constraint to an equality constraint, we define an excess variable (some times called a surplus variable) ei.

Note: if the ith constraint of an LP is a ≥ constraint, it can be converted to an equality constraint by subtracting an excess variable ei from the ith constraint and adding the sign restriction ei≥0.

example

Original LP,

max z=20x1 + 15x2

s.t. x1≤100

x2≤100

50x1+ 35x2≤6000

20x1+ 15x2≥2000

x1,x2≥0

example

Standard form LP,

max z=20x1 + 15x2

s.t. x1 + s1=100

x2 +s2 =100

50x1+ 35x2 +s3 =6000

20x1+ 15x2 -e4=2000

xi≥0(i=1,2); s3≥0(i=1,2,3); e4≥0

Preview of the simplex algorithm
• Suppose an LP with m constraints in standard form,

max z= c1x1+ c2x2+…+ cnxn (1)

s.t. a11x1+a12x2+…+a1nxn=b1

a21x1+a22x2+…+a2nxn=b2

… …

am1x1+am2x2+…+amnxn=bm

xi≥0(i=1,2,…,n)

We define,

The constraints may be written as the system of equations Ax=b

Basic and nonbasic variables
• Consider a system Ax=b of m linear equations in n variables (assume n≥m)
• DEFINATION

A basic solution to Ax=b is obtained by setting n-m variables equal to 0 and solving for the values of the remaining m variables. This assumes that setting the n-m variables equal to 0 yields unique values for the remaining m variables or, equivalently, the columns for the remaining m variables are linearly independent.

Basic and nonbasic variables
• To find a basic solution to Ax=b, we choose a set of n-m variables (the nonbasic variables, or NBV) and set each of these variables equal to 0. Then solve for the values of the remaining n-(n-m)=m variables (the basic variables, or BV) that satisfy Ax=b.
Basic and nonbasic variables
• The different choices of nonbasic variables will lead to different basic solutions.

To illustrate,

x1+x2 =3

-x2+x3=-1

We begin by choosing a set of 3-2=1 nonbasic variables. If NBV={x3} then BV={x1,x2}. We find that x1=2, x2=1, by setting x3=0. If NBV={x2} then we get x1=3, x2=0, x3=-1.

Basic and nonbasic variables
• Some sets of m variables do not yield a basic solution.

For example,

x1+2x2+x3=1

2x1+4x2+x3=3

If we choose NBV={x3}, then

x1+2x2=1

2x1+4x2=3

The system has no solution, there is no basic solution to BV={x1,x2}

Feasible solutions
• DEFINATION

Any basic solution to (1) in which all variables are nonnegative is a basic feasible solution(or bfs).

THEOREM 1

The feasible region for any linear programming problem is a convex set. Also, if an LP has an optimal solution, there must be an extreme point of the feasible region that is optimal.

THEOREM 2

For any LP, there is a unique extreme point of the LP’s feasible region corresponding to each basic feasible solution. Also, there is at least one bfs corresponding to each extreme point of the feasible region.

Feasible solutions

Illustration of Leather Limited example,

Max z=4x1+3x2 (LP1’)

s.t. x1+ x2 +S1 =40

2x1+ x2 +S2 =60

x1,x2,S1,S2≥0

• DEFINATION

For any LP with m constraints, two basic feasible solutions are said to be adjacent if their sets of basic variables have m-1 basic variables in common.

How the simplex algorithm solves LPs in a max problem
• Step1

find a bfs to the LP. We call this bfs the initial basic feasible solution. In general, the most recent bfs will be called the current bfs, so at the beginning of the problem the initial bfs is the current bfs.

• Step2

determine if the current bfs is an optimal solution to the LP. If it is not, find an adjacent bfs that has a larger z-value.

• Step3

How the simplex algorithm solves LPs in a max problem
• If an LP in standard form has m constraints and n variables, there may be a basic solution for each choice of nonbasic variables. From n variables, a set of n-m nonbasic variables can be chosen in

Different ways.

How the simplex algorithm solves LPs in a max problem
• Thus, an LP can have at most basic solutions.
• If we were proceed from the current bfs to a better bfs, we would surely find the optimal bfs after a finite number of calculations.
The Simplex Algorithm
• Step1 convert the LP to standard form.
• Step2 obtain a bfs (if possible) from the standard form.
• Step3 determine whether the current bfs is optimal.
• Step4 if the current bfs is not optimal, determine which nonbasic variable should become a basic variable and which basic variable should become a nonbasic variable to find a new bfs with a better objective function value.
• Step5 use ero(elementary row operation)’s to find the new bfs with the better objective function value. Go back to step3.
example
• The dakota furniture company manufactures desk, tables, and chairs. The manufacture of each type of furniture requires lumber and two types of skilled labor: finishing and carpentry.

48 board feet of lumber, 20 finishing hours, and 8 carpentry hours are available. A desk sells for \$60, a table for \$30, and a chair for \$20. demand for desks and chairs is unlimited, but at most five table can be sold. Dakota wants to maximize total revenue.

example
• Defining the decision variable as

x1=number of desks produced

x2=number of tables produced

x3=number of chairs produced

• Dakota is the following LP:

max z=60x1+30x2+ 20x3

s.t. 8x1+ 6x2+ x3≤48 (lumber constraint)

4x1+ 2x2+1.5x3≤20 (finishing constraint)

2x1+1.5x2+0.5x3≤8 (carpentry constraint)

x2 ≤5 (limitation on table demand)

x1,x2,x3≥0

example
• Convert the LP to standard form

max z=60x1+30x2+ 20x3

s.t. 8x1+ 6x2+ x3 +s1 =48

4x1+ 2x2+1.5x3 +s2 =20

2x1+1.5x2+0.5x3 +s3 =8

x2 +s4=5

x1,x2,x3,s1 ,s2,s3 ,s4≥0

Note: the row 0 for the objective function is

z -60x1-30x2-20x3=0

example
• Putting rows 1-4 together with row 0 and the sign restrictions yields the equations and basic variables.

Canonical form 0

example
• The feasible solution for the initial canonical form has

BV={s1,s2,s3,s4} ={0,48,20,8,5}

and NBV={x1,x2,x3}={0,0,0}

The feasible solution, z=0,s1=48,s2=20,s3=8,s4=5,x1=x2=x3=0

Note: a slack variable can be used as a basic variable for an equation if the right-hand side of the constraint is nonnegative.

example
• Is the current basic feasible solution optimal?

To determine whether there is any way that z can be increased by increasing some nonbasic variables at their current values of zero.

Note: from z=60x1+30x2+ 20x3 we can observe that x1 has the most negative coefficient in row 0. we call x1 the entering variable.

example
• Determine the entering variable

We choose the entering variable (in a max problem) to be the nonbasic variable with the most negative coefficient in row 0(ties may be broken in an arbitrary fashion).

Increasing x1 may cause a basic variable to become negative.

example
• How to increasing x1( holding x2=x3=0) ?

From row 1 s1≥0 for x1≤48/8=6

From row 2 s2≥0 for x1≤20/4=5

From row 3 s3≥0 for x1≤8/2=4

From row 4 s4≥0 for all values of x1

Note: to keep all the basic variables nonnegative, the largest that we can make x1 is min{48/8,20/4,8/2}=4. Also for any row in which the entering variable had a positive coefficient, the row’s basic variable became negative when the entering variable exceeded

right-hand side of row

coefficient of entering variable in row (10)

example
• How to increasing x1( holding x2=x3=0) ?

The ratio test

When entering a variable into the basis, compute the ratio in (10) for every constraint in which the entering variable has a positive coefficient. The constraint with the smallest ratio is called the winner of the ratio test. The smallest ratio is the largest value of the entering variable that will keep all the current basic variables nonnegative.

example
• Find a new basic feasible solution: pivot in the entering variable

In which row does the entering variable become basic?

Always make the entering variable a basic variable in a row that wins the ratio test.

In example, to make x1 a basic variable in row 3, we use elementary row operations to make x1 have a coefficient of 1 in row 3 and a coefficient of 0 in all other rows. This procedure is called pivoting on row 3; and row 3 is the pivot row. The final result is that x1 replaces s3 as the basic variable for row 3. The term in the pivot row that involves the entering basic variable is called the pivot term.

example
• To make x1 a basic variable in row 3 by performing the following ero’s.

ero 1 create a coefficient of 1 for x1 in row 3 by multiplying row 3 by ½. The resulting row is

x1+0.75x2+0.25x3+0.5s3=4 (row 3’)

ero 2 to create a zero coefficient for x1 in row 0, replace row 0 with 60(row 3’) +row 0.

z+15x2-5x3+30s3=240 (row 0’)

ero 3 to create a zero coefficient for x1 in row 1, replace row 1 with -8(row 3’)+row 1.

-x3+s1-4s3=16 (row 1’)

ero 4 to create a zero coefficient for x1 in row 2, replace row 2 with -4(row 3’)+row 2.

-x2+0.5x3+s2-2s3=4 (row 2’)

example
• x1 does not appear in row 4. we don’t need to perform an ero to eliminate x1 from row 4.

Canonical form 1

example
• The feasible solution for the canonical form 1 has

BV={z,s1,s2,x1,s4} and NBV={s3,x2,x3}

The feasible solution, z=240,s1=16,s2=4,x1=4,s4=5,x2=x3=s3=0

In obtaining canonical form 1 from the initial canonical form, we have gone from one bfs to a better (larger z-value) bfs.

Note: the initial bfs and the improved bfs are adjacent. The procedure used to go from one bfs to a better adjacent bfs is called an iteration (or sometimes, a pivot) of the simplex algorithm.

example
• Try to find a bfs that has a still larger z-value.

Rearranging row 0’ to solve for z,

z=240-15x2+5x3-30s3

• Increasing x2 by 1 will decrease z by 15.
• Increasing s3 by 1 will decrease z by 30.
• Increasing x3 by 1 will increase z by 5.

Thus we choose to enter x3 into the basis.

Note: the rule for determining the entering variable is to choose the variable with the most negative coefficient in the current row 0. since x3 is the only variable with a negative coefficient in row 0’, it should be entered into the basis.

example
• To determine how large x3 can be.
• From row 1’: s1=16+x3 all values of x3
• From row 2’: s2=4-0.5x3 x3≤4/0.5=8
• From row 3’: x1=4-0.25x3 x3≤4/0.25=16
• From row 4’: s4=5 all values of x3

The largest we can make x3 is min{4/0.5,4/0.25}=8

example
• To determine how large x3 can be.

Also can be discovered by using (10) and the ratio test:

Row 1’: no ratio (x3 has negative coefficient in row)

Row 2’:4/0.5=8

Row 3’:4/0.25=16

Row 4’:no ratio (x3 has a nonpositive coefficient in row 4)

Thus, the smallest ratio occurs in row 2’, and row 2’ wins the ratio test.

example
• Use ero’s to make x3 a basic variable

ero 1 create a coefficient of 1 for x3 in row 2’ by replacing row 2’ with 2(row 2’)

-2x2+x3+2s2-4s3=8 (row 2’’)

ero 2 to create a zero coefficient for x3 in row 0’, replace row 0 with 5(row 2’’) +row 0’.

z+5x2+10x2+10s3=280 (row 0’’)

ero 3 to create a zero coefficient for x3 in row 1’, replace row 1’ with row 2’’+row 1’.

-2x2+s1+2s2-8s3=24 (row 1’’)

ero 4 to create a zero coefficient for x3 in row 3’, replace row 3’ with -1/4(row 2’’)+row 3’.

x1+1.25x2-0.5s2+1.5s3=2 (row 3’’)

example
• x3 already has a zero coefficient in row 4’. we don’t need to perform an ero to eliminate x3 from row 4’.

Canonical form 2

example
• From canonical form 2, we find

BV={z,s1,x3,x1,s4} and NBV={s2,s3,x2}

It yields the following bfs:

z=280,s1=24,x3=8,x1=2,s4=5,s2=s3=x2=0

Since the bfs’s for canonical forms 1 and 2 have 4-1=3 basic variables in common (s1,s4,x1), they are adjacent basic feasible solutions.

Now the second iteration (or pivot) has been completed.

example
• If we rearrange row 0’’ and solve for z, we obtain

z=280-5x2-10s2-10s3

We see that increasing any nonbasic variable will cause z to decrease. this might lead us to believe that our current bfs is an optimal solution.

• Is a canonical form optimal(max problem)?

A canonical form is optimal( for a max problem) if each nonbasic variable has a nonnegative coefficient in the canonocal form’s row 0.

Summary of the simplex algorithm for a max problem
• Step 1 convert the LP to standard form.
• Step 2 find a basic feasible solution.
• Step 3 if all nonbasic variables have nonnegative coefficients in row 0, the current bfs is optimal. If any variables in row 0 have negative coefficients, choose the variable with the most negative coefficient in row 0 to enter the basis. We call this variable the entering variable.
Summary of the simplex algorithm for a max problem
• Step4 use ero’s to make the entering variable the basic variable in any row that wins the ratio test. After the ero’s have been used to create a new canonical form, return to step 3, using the current canonical form.
Representing simplex tableaus
• Rather than writing each variable in every constraint, we often used a shorthand display called a simplex tableau.
• For example,

z+3x1+x2=6

x1+s1=4

2x1+x2+s2=3

Representing simplex tableaus
• This format makes it very easy to spot basic variable:

Just look for columns having a single entry of 1 and all other entries equal to 0 (s1 and s2).

In our use of simplex tableaus, we will encircle the pivot term and denote the winner of the ratio test by *.

Using the simplex algorithm to solve minimization problems
• There are two different ways that the simplex algorithm can be used to solve minimization problems.
• We illustrate these methods by solving the following LP:

min z=2x1-3x2

s.t. x1+ x2≤4

x1- x2≤6 (LP2)

x1,x2≥0

Method 1
• We can find the optimal solution to LP 2 by solving LP 2’:

max -z=-2x1+3x2

s.t. x1+ x2≤4

x1- x2≤6 (LP2’)

x1,x2≥0

Initial tableau:

The ratio test indicates that x2 should enter the basis in row 1.

Method 1

Optimal tableau

Thus, the optimal solution to LP 2’ is

-z=12,x2=4,s2=10,x1=s1=0.

then the optimal solution to LP 2 is

z=-12,x2=4,s2=10,x1=s1=0.

Method 1
• In summary, multiply the objective function for the min problem by -1 and solve the problem as a maximization problem with objective function –z.

Note:

(optimal z-value for min problem)

= -(optimal objective function value z for max problem)

Method 2
• A simplex modification of the ximplex algorithm can be used to solve min problem directly.

Modify step3 of the simplex as follows:

If all nonbasic variables in row 0 have nonpositive coefficients, the current bfs is optimal. If any nonbasic variable in row 0 has a positive coefficient, choose the variable with the most positive coefficient in row 0 to enter the basis.

This modification works because increasing a nonbasic variable with a positive coefficient in row 0 will decrease z.

Method 2
• To solve LP 2

Initial tableau

Since x2 has the most positive coefficient in row 0, we enter x2 into the basis. the ratio test says that x2 should enter the basis in row 1.

Method 2

Optimal tableau:

• Thus, the optimal solution to LP 2 is
• z=-12,x2=4,s2=10,x1=s1=0.
Alternative optimal solutions
• If an LP has more than one optimal solution, we say that it has multiple or alternative optimal solutions.
• Reconsider the Dakota furniture example, with the modification that the tables sell for \$35 instead of \$30.
Alternative optimal solutions

Initial tableau for Dakota furniture(\$35/table)

The ratio test indicates that x1 should be entered in row 3

Alternative optimal solutions

first tableau for Dakota furniture(\$35/table)

The ratio test indicates that x3 should be entered in row 2

Alternative optimal solutions

Second (optimal) tableau for Dakota furniture(\$35/table)

The optimal solution is s1=24,x3=8 x1=2 s4=5 x2=s2=s3=0

Alternative optimal solutions
• In the optimal tableau, there is a nonbasic variable, x2, has a zero coefficient in row 0.
• What happens if we enter x2 into the basis?
Alternative optimal solutions

Another optimal tableau for Dakota furniture(\$35/table)

Alternative optimal solutions

Note: because x2 has a zero coefficient in the optimal tableau’s row 0, the pivot that enters x2 into the basis does not change row 0.

• All variables in new row 0 still have nonnegative coefficients, thus, the new tableau is also optimal.
• The pivot has not changed the value of z, an alternative optimal solution is z=280, s1=27.2, x3=11.2,x2=1.6,s4=3.4,and x1=s3=s2=0.
Alternative optimal solutions
• Any point on the line segment joining two optimal extreme points will also be optimal.
Alternative optimal solutions

For 0≤c≤1, all points are optimal.

Unbounded LPs
• There exist points in the feasible region for which z assumes arbitrarily large (in max problems) or arbitrarily small (in min problems) values.
Example 3
• Breadco Bakeries bakes two kinds of bread: french and sourdough. Each loaf of french bread can be sold for 36¢, and each loaf of sourdough bread for 30¢. A loaf of french bread requires 1 yeast packet and 6 oz of flour; sourdough requires 1 yeast packet and 5 oz of flour.
Example 3
• At present, Breadco has 5 yeast packets and 10 oz of flour. Additional yeast packets can be purchased at 3¢ each, and additional flour at 4¢/oz. formulate and solve an LP that can be used to maximize Breadco’s profits.
Example 3

Solution

Define

x1=number of loaves of french bread baked

x2=number of loaves of sourdough bread baked

x3=number of yeast packets purchased

x4=number of ounces of flour purchased

Objectiveis to maximize z=revenues – costs, where

revenues = 36x1+30x2 and costs=3x3+4x4

Thus, objective function is

max z=36x1+30x2-3x3-4x4

Example 3
• Constraint 1

Number of yeast packages used to bake bread cannot exceed available yeast plus purchased yeast.

Available yeast + purchased yeast=5+x3

Then

x1+x2≤5+x3 or x1+x2-x3≤5

Example 3
• Constraint 2

Ounces of flour used to bake breads cannot exceed available flour plus purchased flour.

Available flour + purchased flour=10+x4

Then

6x1+5x2≤10+x4 or 6x1+5x2-x4≤10

Example 3
• The LP:

max z=36x1+30x2-3x3-4x4

s.t. x1+ x2-x3 ≤5 (yeast constraint)

6x1+5x2 -x4≤10 (flour constraint)

x1,x2,x3,x4≥0

adding slack variables s1 and s2 to the two constraints.

Example 3

The ratio test indicates that x1 should enter the basis in row 2.

Example 3

The ratio test indicates that x4 should enter the basis in row 1.

Example 3

The ratio test fails to indicate the row in which x3 should enter the basis.

Example 3
• What is happening?

Holding the other nonbasic variables at zero,

x4=20+6x3

x1=5+x3

as x3 is increased, both x4 and x1 increase, and no matter how large we make x3, the inequalities x4≥0 and x1≥0 will still be true.

Note: an unbounded LP for a max problem occurs when a variable with a negative coefficient in row 0 has a nonpositive coefficient in each constraint.

Unbounded LPs
• If an LP is unbounded, one will eventually come to a tableau where one wants to enter a variable into the basis, but the ratio test will fail.
• An unbounded LP is usually caused by an incorrect formulation.
example

In the Breadco example, we allowed Breakco to pay 3+6(4)=27¢ for the ingredients in a loaf of french bread and then sell the loaf for 36¢. Thus, each loaf of french bread earns a profit of 9¢. Since unlimited purchases of yeast and flour are allowed, it is clear that our model allows Breadco to manufacture as much french bread as it desires, thereby, earning arbitrarily large profits.

example
• Our formulation of the Breadco example ignored several aspects of reality.

First, we assumed that demand for Breadco’s products is unlimited.

Second, we ignored the fact that certain resources to make bread are in limited supply.

Finally, we made the unrealistic assumption that unlimited quantities of yeast and flour could be purchased.