analysis of algorithms cs 477 677
Download
Skip this Video
Download Presentation
Analysis of Algorithms CS 477/677

Loading in 2 Seconds...

play fullscreen
1 / 37

Analysis of Algorithms CS 477/677 - PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on

Analysis of Algorithms CS 477/677. Instructor: Monica Nicolescu Lecture 12. Dynamic Programming. An algorithm design technique for optimization problems (similar to divide and conquer) Divide and conquer Partition the problem into independent subproblems

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Analysis of Algorithms CS 477/677' - nero-hampton


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
analysis of algorithms cs 477 677

Analysis of AlgorithmsCS 477/677

Instructor: Monica Nicolescu

Lecture 12

dynamic programming
Dynamic Programming
  • An algorithm design technique for optimization problems (similar to divide and conquer)
  • Divide and conquer
    • Partition the problem into independentsubproblems
    • Solve the subproblems recursively
    • Combine the solutions to solve the original problem

CS 477/677 - Lecture 12

dynamic programming1
Dynamic Programming
  • Used for optimization problems
    • Find a solution with the optimal value (minimum or maximum)
    • A set of choices must be made to get an optimal solution
    • There may be many solutions that return the optimal value: we want to find one of them
  • Applicable when subproblems are not independent
    • Subproblems share subsubproblems

CS 477/677 - Lecture 12

dynamic programming algorithm
Dynamic Programming Algorithm
  • Characterize the structure of an optimal solution
  • Recursively define the value of an optimal solution
  • Compute the value of an optimal solution in a bottom-up fashion
  • Construct an optimal solution from computed information

CS 477/677 - Lecture 12

elements of dynamic programming
Elements of Dynamic Programming
  • Optimal Substructure
    • An optimal solution to a problem contains within it an optimal solution to subproblems
    • Optimal solution to the entire problem is built in a bottom-up manner from optimal solutions to subproblems
  • Overlapping Subproblems
    • If a recursive algorithm revisits the same subproblems again and again  the problem has overlapping subproblems

CS 477/677 - Lecture 12

assembly line scheduling
Assembly Line Scheduling
  • Automobile factory with two assembly lines
    • Each line has n stations: S1,1, . . . , S1,n and S2,1, . . . , S2,n
    • Corresponding stations S1, j and S2, j perform the same function but can take different amounts of time a1, j and a2, j
    • Times to enter are e1 and e2and times to exit are x1 and x2

CS 477/677 - Lecture 12

assembly line
Assembly Line
  • After going through a station, the car can either:
    • stay on same line at no cost, or
    • transfer to other line: cost after Si,j is ti,j , i = 1, 2, j = 1, . . . , n-1

CS 477/677 - Lecture 12

assembly line scheduling1
Assembly Line Scheduling
  • Problem:

What stations should be chosen from line 1 and what from line 2 in order to minimize the total time through the factory for one car?

CS 477/677 - Lecture 12

1 structure of the optimal solution

S1,j-1

S1,j

a1,j-1

a1,j

t2,j-1

a2,j-1

S2,j-1

1. Structure of the Optimal Solution
  • Let’s consider all possible ways to get from the starting point through station S1,j
    • We have two choices of how to get to S1, j:
      • Through S1, j - 1, then directly to S1, j
      • Through S2, j - 1, then transfer over to S1, j

CS 477/677 - Lecture 12

2 a recursive solution
2. A Recursive Solution
  • f* = the fastest time to get through the entire factory
  • fi[j] = the fastest time to get from the starting point through station Si,j

f* = min (f1[n] + x1, f2[n] + x2)

CS 477/677 - Lecture 12

2 a recursive solution1
2. A Recursive Solution
  • fi[j] = the fastest time to get from the starting point through station Si,j
  • j = 1 (getting through station 1)

f1[1] = e1 + a1,1

f2[1] = e2 + a2,1

CS 477/677 - Lecture 12

2 a recursive solution2

S1,j-1

S1,j

a1,j-1

a1,j

t2,j-1

a2,j-1

S2,j-1

2. A Recursive Solution
  • Compute fi[j] for j = 2, 3, …,n, and i = 1, 2
  • Fastest way through S1, j is either:
    • the way through S1, j - 1 then directly through S1, j, or

f1[j - 1] + a1,j

    • the way through S2, j - 1, transfer from line 2 to line 1, then through S1, j

f2[j -1] + t2,j-1 + a1,j

f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)

CS 477/677 - Lecture 12

3 computing the optimal solution

increasing j

3. Computing the Optimal Solution
  • For j ≥ 2, each value fi[j] depends only on the values of f1[j – 1] and f2[j - 1]
  • Compute the values of fi[j]
    • in increasing order of j
  • Bottom-up approach
    • First find optimal solutions to subproblems
    • Find an optimal solution to the problem from the subproblems

1

2

3

4

5

f1[j]

f2[j]

CS 477/677 - Lecture 12

4 construct the optimal solution

increasing j

4. Construct the Optimal Solution
  • We need the information about which line has been used at each station:
    • li[j] – the line number (1, 2) whose station (j - 1) has been used to get in fastest time through Si,j, j = 2, 3, …, n
    • l* – the line number (1, 2) whose station n has been used to get in fastest time through the exit point

2

3

4

5

l1[j]

l2[j]

CS 477/677 - Lecture 12

fastest way a t e x n

Compute initial values of f1 and f2

FASTEST-WAY(a, t, e, x, n)
  • f1[1] ← e1 + a1,1
  • f2[1] ← e2 + a2,1
  • for j ← 2to n
  • do if f1[j - 1] + a1,j ≤ f2[j - 1] + t2, j-1 + a1, j
  • then f1[j] ← f1[j - 1] + a1, j
  • l1[j] ← 1
  • else f1[j] ← f2[j - 1] + t2, j-1 + a1, j
  • l1[j] ← 2
  • if f2[j - 1] + a2, j ≤ f1[j - 1] + t1, j-1 + a2, j
  • then f2[j] ← f2[j - 1] + a2, j
  • l2[j] ← 2
  • else f2[j] ← f1[j - 1] + t1, j-1 + a2, j
  • l2[j] ← 1

Compute the values of

f1[j] and l1[j]

Compute the values of

f2[j] and l2[j]

CS 477/677 - Lecture 12

fastest way a t e x n cont
FASTEST-WAY(a, t, e, x, n) (cont.)
  • if f1[n] + x1 ≤ f2[n] + x2
  • then f* = f1[n] + x1
  • l* = 1
  • else f* = f2[n] + x2
  • l* = 2

Compute the values of

the fastest time through the

entire factory

CS 477/677 - Lecture 12

4 construct an optimal solution
4. Construct an Optimal Solution

Alg.: PRINT-STATIONS(l, n)

i ← l*

print “line ” i “, station ” n

for j ← ndownto 2

do i ←li[j]

print “line ” i “, station ” j - 1

line 1, station 5

line 1, station 4

line 1, station 3

line 2, station 2

line 1, station 1

1

2

3

4

5

f1[j] l1[j]

9

18[1]

20[2]

24[1]

32[1]

l* = 1

f2[j] l2[j]

12

16[1]

22[2]

25[1]

30[2]

CS 477/677 - Lecture 12

dynamic programming algorithm1
Dynamic Programming Algorithm
  • Characterize the structure of an optimal solution
    • Fastest time through a station depends on the fastest time on previous stations
  • Recursively define the value of an optimal solution
    • f1[j] = min(f1[j - 1] + a1,j ,f2[j -1] + t2,j-1 + a1,j)
  • Compute the value of an optimal solution in a bottom-up fashion
    • Fill in the fastest time table in increasing order of j (station #)
  • Construct an optimal solution from computed information
    • Use an additional table to help reconstruct the optimal solution

CS 477/677 - Lecture 12

matrix chain multiplication
Matrix-Chain Multiplication

Problem: given a sequence A1, A2, …, An of matrices, compute the product:

A1  A2  An

  • Matrix compatibility:

C = A  B

colA = rowB

rowC = rowA

colC = colB

A1  A2  Ai  Ai+1  An

coli = rowi+1

CS 477/677 - Lecture 12

matrix chain multiplication1
Matrix-Chain Multiplication
  • In what order should we multiply the matrices?

A1  A2  An

  • Matrix multiplication is associative:
  • E.g.:A1  A2  A3 = ((A1  A2)  A3)

= (A1  (A2  A3))

  • Which one of these orderings should we choose?
    • The order in which we multiply the matrices has a significant impact on the overall cost of executing the entire chain of multiplications

CS 477/677 - Lecture 12

matrix multiply a b

rows[A]  cols[A]  cols[B]

multiplications

k

k

MATRIX-MULTIPLY(A, B)

ifcolumns[A]  rows[B]

then error“incompatible dimensions”

else fori  1 to rows[A]

do forj  1 to columns[B]

doC[i, j] = 0

fork  1 to columns[A]

doC[i, j]  C[i, j] + A[i, k] B[k, j]

j

cols[B]

j

cols[B]

i

i

=

*

B

A

C

rows[A]

rows[A]

CS 477/677 - Lecture 12

example
Example

A1  A2  A3

  • A1: 10 x 100
  • A2: 100 x 5
  • A3: 5 x 50
  • ((A1  A2)  A3): A1  A2 takes 10 x 100 x 5 = 5,000

(its size is 10 x 5)

((A1  A2)  A3) takes 10 x 5 x 50 = 2,500

Total: 7,500 scalar multiplications

2. (A1  (A2  A3)): A2  A3 takes 100 x 5 x 50 = 25,000

(its size is 100 x 50)

(A1  (A2  A3)) takes 10 x 100 x 50 = 50,000

Total: 75,000 scalar multiplications

one order of magnitude difference!!

CS 477/677 - Lecture 12

matrix chain multiplication2
Matrix-Chain Multiplication
  • Given a chain of matrices A1, A2, …, An, where for i = 1, 2, …, n matrix Ai has dimensions pi-1x pi, fully parenthesize the product A1  A2 An in a way that minimizes the number of scalar multiplications.

A1  A2 Ai  Ai+1 An

p0 x p1 p1 x p2 pi-1 x pi pi x pi+1 pn-1 xpn

CS 477/677 - Lecture 12

1 the structure of an optimal parenthesization
1. The Structure of an Optimal Parenthesization
  • Notation:

Ai…j = Ai Ai+1 Aj, i  j

  • For i < j:

Ai…j = Ai Ai+1 Aj

= Ai Ai+1 Ak Ak+1  Aj

= Ai…k Ak+1…j

  • Suppose that an optimal parenthesization of Ai…j splits the product between Ak and Ak+1, where i  k < j

CS 477/677 - Lecture 12

optimal substructure
Optimal Substructure

Ai…j= Ai…k Ak+1…j

  • The parenthesization of the “prefix” Ai…kmust be an optimal parentesization
  • If there were a less costly way to parenthesize Ai…k, we could substitute that one in the parenthesization of Ai…j and produce a parenthesization with a lower cost than the optimum  contradiction!
  • An optimal solution to an instance of the matrix-chain multiplication contains within it optimal solutions to subproblems

CS 477/677 - Lecture 12

2 a recursive solution3
2. A Recursive Solution
  • Subproblem:

determine the minimum cost of parenthesizing Ai…j = Ai Ai+1 Ajfor 1  i  j  n

  • Let m[i, j] = the minimum number of multiplications needed to compute Ai…j
    • Full problem (A1..n): m[1, n]
    • i = j: Ai…i = Ai m[i, i] =
  • 0, for i = 1, 2, …, n

CS 477/677 - Lecture 12

2 a recursive solution4

pi-1pkpj

m[i, k]

m[k+1,j]

2. A Recursive Solution
  • Consider the subproblem of parenthesizing Ai…j = Ai Ai+1 Ajfor 1  i  j  n

= Ai…k Ak+1…j for i  k < j

  • Assume that the optimal parenthesization splits the product Ai Ai+1 Aj at k (i  k < j)

m[i, j] =

m[i, k] + m[k+1, j] + pi-1pkpj

min # of multiplications

to compute Ai…k

min # of multiplications

to compute Ak+1…j

# of multiplications

to computeAi…kAk…j

CS 477/677 - Lecture 12

2 a recursive solution5
2. A Recursive Solution

m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj

  • We do not know the value of k
    • There are j – i possible values for k: k = i, i+1, …, j-1
  • Minimizing the cost of parenthesizing the product Ai Ai+1Aj becomes:

0if i = j

m[i, j]= min {m[i, k] + m[k+1, j] + pi-1pkpj}ifi < j

ik<j

CS 477/677 - Lecture 12

3 computing the optimal costs
3. Computing the Optimal Costs

0if i = j

m[i, j]= min {m[i, k] + m[k+1, j] + pi-1pkpj}if i < j

ik<j

  • How many subproblems do we have?
    • Parenthesize Ai…j

for 1  i  j  n

    • One subproblem for each

choice of i and j

1

2

3

n

 (n2)

n

j

3

2

1

CS 477/677 - Lecture 12

i

3 computing the optimal costs1
3. Computing the Optimal Costs

0if i = j

m[i, j]= min {m[i, k] + m[k+1, j] + pi-1pkpj}ifi < j

ik<j

  • How do we fill in table m[1..n, 1..n]?
    • Determine which entries of the table are used in computing m[i, j]

Ai…j= Ai…k Ak+1…j

    • Fill in m such that it corresponds to solving problems of increasing length

CS 477/677 - Lecture 12

3 computing the optimal costs2

second

first

m[1, n] gives the optimal

solution to the problem

3. Computing the Optimal Costs

0if i = j

m[i, j]= min {m[i, k] + m[k+1, j] + pi-1pkpj}ifi < j

ik<j

  • Length = 1: i = j, i = 1, 2, …, n
  • Length = 2: j = i + 1, i = 1, 2, …, n-1

1

2

3

n

n

j

3

Compute elements on each diagonal, starting with the longest diagonal.

In a similar matrix s we keep the optimal values of k.

2

1

i

CS 477/677 - Lecture 12

example min m i k m k 1 j p i 1 p k p j
Example: min {m[i, k] + m[k+1, j] + pi-1pkpj}

m[2, 2] + m[3, 5] + p1p2p5

m[2, 3] + m[4, 5] + p1p3p5

m[2, 4] + m[5, 5] + p1p4p5

k = 2

m[2, 5] = min

k = 3

k = 4

1

2

3

4

5

6

6

5

  • Values m[i, j] depend only on values that have been previously computed

4

j

3

2

1

i

CS 477/677 - Lecture 12

example min m i k m k 1 j p i 1 p k p j1

2

0

25000

2

1

0

7500

5000

0

Example min {m[i, k] + m[k+1, j] + pi-1pkpj}

1

2

3

Compute A1  A2 A3

  • A1: 10 x 100 (p0x p1)
  • A2: 100 x 5 (p1x p2)
  • A3: 5 x 50 (p2x p3)

m[i, i] = 0 for i = 1, 2, 3

m[1, 2] = m[1, 1] + m[2, 2] + p0p1p2 (A1A2)

= 0 + 0 + 10 *100* 5 = 5,000

m[2, 3] = m[2, 2] + m[3, 3] + p1p2p3 (A2A3)

= 0 + 0 + 100 * 5 * 50 = 25,000

m[1, 3] = min m[1, 1] + m[2, 3] + p0p1p3 = 75,000 (A1(A2A3))

m[1, 2] + m[3, 3] + p0p2p3 = 7,500 ((A1A2)A3)

3

2

1

CS 477/677 - Lecture 12

4 construct the optimal solution1
4. Construct the Optimal Solution
  • Store the optimal choice made at each subproblem
  • s[i, j] = a value of k such that an optimal parenthesization of Ai..j splits the product between Ak and Ak+1

1

2

3

n

n

j

3

2

1

i

CS 477/677 - Lecture 12

4 construct the optimal solution2
4. Construct the Optimal Solution
  • s[1, n] is associated with the entire product A1..n
    • The final matrix multiplication will be split at k = s[1, n]

A1..n = A1..k Ak+1..n

A1..n = A1..s[1, n] As[1, n]+1..n

    • For each subproduct recursively find the corresponding value of k that results in an optimal parenthesization

1

2

3

n

n

j

3

2

1

i

CS 477/677 - Lecture 12

4 construct the optimal solution3
4. Construct the Optimal Solution
  • s[i, j] = value of k such that the optimal parenthesization of Ai Ai+1 Aj splits the product between Ak and Ak+1

1

2

3

4

5

6

6

  • s[1, n] = 3  A1..6 = A1..3 A4..6
  • s[1, 3] = 1  A1..3 = A1..1 A2..3
  • s[4, 6] = 5  A4..6 = A4..5 A6..6

5

4

3

j

2

1

i

CS 477/677 - Lecture 12

readings
Readings
  • Chapter 15

CS 477/677 - Lecture 12

37

ad