1 / 30

CPSC 411 Design and Analysis of Algorithms

CPSC 411 Design and Analysis of Algorithms. Set 5: Dynamic Programming Prof. Jennifer Welch Fall 2008. Drawback of Divide & Conquer. Sometimes can be inefficient Fibonacci numbers: F 0 = 0 F 1 = 1 F n = F n-1 + F n-2 for n > 1 Sequence is 0, 1, 1, 2, 3, 5, 8, 13, ….

carloswhitten
Download Presentation

CPSC 411 Design and Analysis of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CPSC 411Design and Analysis of Algorithms Set 5: Dynamic Programming Prof. Jennifer Welch Fall 2008 CPSC 411, Fall 2008: Set 5

  2. Drawback of Divide & Conquer • Sometimes can be inefficient • Fibonacci numbers: • F0 = 0 • F1 = 1 • Fn = Fn-1 + Fn-2 for n > 1 • Sequence is 0, 1, 1, 2, 3, 5, 8, 13, … CPSC 411, Fall 2008: Set 5

  3. Computing Fibonacci Numbers • Obvious recursive algorithm: • Fib(n): • if n = 0 or 1 then return n • else return (F(n-1) + Fib(n-2)) CPSC 411, Fall 2008: Set 5

  4. Recursion Tree for Fib(5) Fib(5) Fib(4) Fib(3) Fib(3) Fib(2) Fib(2) Fib(1) Fib(2) Fib(1) Fib(1) Fib(0) Fib(1) Fib(0) Fib(1) Fib(0) CPSC 411, Fall 2008: Set 5

  5. How Many Recursive Calls? • If all leaves had the same depth, then there would be about 2n recursive calls. • But this is over-counting. • However with more careful counting it can be shown that it is ((1.6)n) • Exponential! CPSC 411, Fall 2008: Set 5

  6. Save Work • Wasteful approach - repeat work unnecessarily • Fib(2) is computed three times • Instead, compute Fib(2) once, store result in a table, and access it when needed CPSC 411, Fall 2008: Set 5

  7. More Efficient Recursive Alg • F[0] := 0; F[1] := 1; F[n] := Fib(n); • Fib(n): • if n = 0 or 1 then return F[n] • if F[n-1] = NIL then F[n-1] := Fib(n-1) • if F[n-2] = NIL then F[n-2] := Fib(n-2) • return (F[n-1] + F[n-2]) • computes each F[i] only once called memoization CPSC 411, Fall 2008: Set 5

  8. fills in F[4] with 3, returns 3+2 = 5 fills in F[2] with 1, returns 1+1 = 2 fills in F[3] with 2, returns 2+1 = 3 returns 0+1 = 1 Example of Memoized Fib F 0 1 2 3 4 5 Fib(5) 0 1 1 NIL Fib(4) 2 NIL 3 NIL Fib(3) 5 NIL Fib(2) CPSC 411, Fall 2008: Set 5

  9. Get Rid of the Recursion • Recursion adds overhead • extra time for function calls • extra space to store information on the runtime stack about each currently active function call • Avoid the recursion overhead by filling in the table entries bottom up, instead of top down. CPSC 411, Fall 2008: Set 5

  10. Subproblem Dependencies • Figure out which subproblems rely on which other subproblems • Example: F0 F1 F2 F3 … Fn-2 Fn-1 Fn CPSC 411, Fall 2008: Set 5

  11. Order for Computing Subproblems • Then figure out an order for computing the subproblems that respects the dependencies: • when you are solving a subproblem, you have already solved all the subproblems on which it depends • Example: Just solve them in the order F0, F1, F2, F3,… called Dynamic Programming CPSC 411, Fall 2008: Set 5

  12. DP Solution for Fibonacci • Fib(n): • F[0] := 0; F[1] := 1; • for i := 2 to n do • F[i] := F[i-1] + F[i-2] • return F[n] • Can perform application-specific optimizations • e.g., save space by only keeping last two numbers computed time reduced from exponential to linear! CPSC 411, Fall 2008: Set 5

  13. Matrix Chain Order Problem • Multiplying non-square matrices: • A is n x m, B is m x p • AB is n x p whose (i,j) entry is ∑aik bkj • Computing AB takes nmp scalar multiplications and n(m-1)p scalar additions (using basic algorithm). • Suppose we have a sequence of matrices to multiply. What is the best order? must be equal CPSC 411, Fall 2008: Set 5

  14. Why Order Matters • Suppose we have 4 matrices: • A, 30 x 1 • B, 1 x 40 • C, 40 x 10 • D, 10 x 25 • ((AB)(CD)) : requires 41,200 mults. • (A((BC)D)) : requires 1400 mults. CPSC 411, Fall 2008: Set 5

  15. Matrix Chain Order Problem • Given matrices A1, A2, …, An, where Ai is di-1 x di: [1] What is minimum number of scalar mults required to compute A1·A2 ·… · An? [2] What order of matrix multiplications achieves this minimum? Focus on [1]; see textbook for how to do [2] CPSC 411, Fall 2008: Set 5

  16. A Possible Solution • Try all possibilities and choose the best one. • Drawback is there are too many of them (exponential in the number of matrices to be multiplied) • Need to be more clever - try dynamic programming! CPSC 411, Fall 2008: Set 5

  17. Step 1: Develop a Recursive Solution • Define M(i,j) to be the minimum number of mults. needed to compute Ai·Ai+1 ·… · Aj • Goal: Find M(1,n). • Basis: M(i,i) = 0. • Recursion: How to define M(i,j) recursively? CPSC 411, Fall 2008: Set 5

  18. Defining M(i,j) Recursively • Consider all possible ways to split Ai through Aj into two pieces. • Compare the costs of all these splits: • best case cost for computing the product of the two pieces • plus the cost of multiplying the two products • Take the best one • M(i,j) = mink(M(i,k) + M(k+1,j) + di-1dkdj) CPSC 411, Fall 2008: Set 5

  19. Defining M(i,j) Recursively (Ai ·…·Ak)·(Ak+1 ·… · Aj) P2 P1 • minimum cost to compute P1 is M(i,k) • minimum cost to compute P2 is M(k+1,j) • cost to compute P1· P2 is di-1dkdj CPSC 411, Fall 2008: Set 5

  20. Step 2: Find Dependencies Among Subproblems M: GOAL! computing the pink square requires the purple ones: to the left and below. CPSC 411, Fall 2008: Set 5

  21. Defining the Dependencies • Computing M(i,j) uses • everything in same row to the left: M(i,i), M(i,i+1), …, M(i,j-1) • and everything in same column below: M(i,j), M(i+1,j),…,M(j,j) CPSC 411, Fall 2008: Set 5

  22. Step 3: Identify Order for Solving Subproblems • Recall the dependencies between subproblems just found • Solve the subproblems (i.e., fill in the table entries) this way: • go along the diagonal • start just above the main diagonal • end in the upper right corner (goal) CPSC 411, Fall 2008: Set 5

  23. Order for Solving Subproblems M: CPSC 411, Fall 2008: Set 5

  24. Pseudocode for i := 1 to n do M[i,i] := 0 for d := 1 to n-1 do // diagonals for i := 1 to n-d to // rows w/ an entry on d-th diagonal j := i + d // column corresponding to row i on d-th diagonal M[i,j] := infinity for k := 1 to j-1 to M[i,j] := min(M[i,j], M[i,k]+M[k+1,j]+di-1dkdj) endfor endfor endfor pay attention here to remember actual sequence of mults. running time O(n3) CPSC 411, Fall 2008: Set 5

  25. Example M: 1: A is 30x1 2: B is 1x40 3: C is 40x10 4: D is 10x25 CPSC 411, Fall 2008: Set 5

  26. Keeping Track of the Order • It's fine to know the cost of the cheapest order, but what is that cheapest order? • Keep another array S and update it when computing the minimum cost in the inner loop • After M and S have been filled in, then call a recursive algorithm on S to print out the actual order CPSC 411, Fall 2008: Set 5

  27. Modified Pseudocode for i := 1 to n do M[i,i] := 0 for d := 1 to n-1 do // diagonals for i := 1 to n-d to // rows w/ an entry on d-th diagonal j := i + d // column corresponding to row i on d-th diagonal M[i,j] := infinity for k := 1 to j-1 to M[i,j] := min(M[i,j], M[i,k]+M[k+1,j]+di-1dkdj) if previous line changed value of M[i,j] then S[i,j] := k endfor endfor endfor keep track of cheapest split point found so far: between Ak and Ak+1) CPSC 411, Fall 2008: Set 5

  28. Example M: 1: A is 30x1 2: B is 1x40 3: C is 40x10 4: D is 10x25 S: 1 1 1 2 3 3 CPSC 411, Fall 2008: Set 5

  29. Using S to Print Best Ordering Call Print(S,1,n) to get the entire ordering. Print(S,i,j): if i =j then output "A" + i //+ is string concat else k := S[i,j] output "(" + Print(S,i,k) + Print(S,k+1,j) + ")" CPSC 411, Fall 2008: Set 5

  30. Example S: <<draw recursion tree on board>> CPSC 411, Fall 2008: Set 5

More Related