Dynamic Programming (DP)

1. Dynamic Programming(DP) Simpler, Better, Faster Carl Hultquist

2. Rough definition • Dynamic programming is: • Breaking a problem up into smaller sub-problems • By finding the optimal solution to these smaller sub-problems, being able to find the solution to the bigger problem • Dynamic programming is not: • A type of programming language (like declarative and imperative)

3. Learn by example: Fibonacci numbers • We all know the Fibonacci numbers 1 1 2 3 5 8 13… and that we can write a neat definition for these as: f(0) = 1 f(1) = 1 f(n) = f(n – 2) + f(n – 1) for n>1

4. Fibonacci in code int f(int n) { if (n < 2) return 1; else return f(n – 2) + f(n – 1); }

5. Fibonacci and big-O • So we have an algorithm for finding f(n), but is it any good? What’s the big-O for finding f(n)? Answer: O(f(n)) So… is this good? Is it bad?

6. Recursive Fibonacci: bad • As an indication, f(1000)=1,318,412,525 • Surely we can do better…

7. The trick: overlapping subproblems • For some n, f(n)=f(n-2)+f(n-1) Now f(n-1)=f(n-3)+f(n-2) Note the common f(n-2) – so to calculate the value of f(n), we actually calculate f(n-2) twice. Doing this using our recursive program is wasteful – we should only need to work it out once!!!

8. A better Fibonacci #define MAX_N 10000 int f[MAX_N]; int fib(int n) { f[0] = f[1] = 1; for (int i = 2; i <= n; i++) f[i] = f[i – 2] + f[i – 1]; return f[n]; }

9. Coin counting  • Yes, today’s problem was a DP. So we want to make change of value M, we have N coin denominations, and their values are Vi for i=1…N. Now we can write the solution like this: coins(M) = min{coins(M – V1), coins(M – V2), …, coins(M – VN)} + 1 Now, doesn’t that look just a little bit like the Fibonacci definition? ;-)

10. A better solution… int N, M; int V[N]; int coins[M + 1]; coins[0] = 0; for (int i = 1; i <= M; i++) { int best = M; for (int j = 0; j < N; j++) if (V[j] <= i && coins[i – V[j]] + 1 < best) best = coins[i – V[j]] + 1; coins[i] = best; }

11. Checking for valid states int N, M; int V[N]; int coins[M + 1]; set(coins[0], coins[M], -1); coins[0] = 0; for (int i = 1; i <= M; i++) { int best = M; for (int j = 0; j < N; j++) if (V[j] <= i && coins[i – V[j]] != -1 && coins[i – V[j]] + 1 < best) best = coins[i – V[j]] + 1; coins[i] = best; }

12. An alternative coin solution: memoization int N, M; int V[N]; int cache[M + 1]; set(cache[0], cache[M], -1); int coins(int amount) { if (cache[amount] == -1) { int best = M; for (int i = 0; i < N; i++) if (V[i] <= amount && coins(amount) + 1 < best) best = coins(amount) + 1; cache[amount] = best; } return cache[amount]; }

13. Two approaches to DP • Top-down: recursion + memoization. Easier to code, but may have greater memory requirements. Also has extra stack + function call overhead. • Bottom-up: iterative, solves subproblems from the smallest one up. Sometimes harder to code and/or work out exactly what’s going on, but more efficient.

14. Backtracking • Some DP problems just call for the value of the best answer (like today’s problem). • Others make your life harder: they ask for the “path” to the solution. • Today’s problem could have done this by asking you to output the actual coins used

15. Backtracking (2) • This usually isn’t too hard: in the same way that you have an array to store your best solution, you also keep an array indicating how you got there. • For the coins problem, this array can simply store the last coin used to reach each total • You can then “backtrack” by subtracting the coin value from the total, and look at the next coin that was needed. Repeat until you hit 0 

16. Coins with backtrack int N, M; int V[N]; int coins[M + 1]; int coinUsed[M + 1]; coins[0] = 0; for (int i = 1; i <= M; i++) { int best = M; int coin = -1; for (int j = 0; j < N; j++) if (V[j] <= i && coins[i – V[j]] + 1 < best) { best = coins[i – V[j]] + 1; coin = j; } coins[i] = best; coinUsed[i] = coin; }

17. Higher dimensions • So far, we’ve just seen 1D DP problems • At the IOI, 2D (or higher!) problems crop up often • Consider this one: given a NxN grid of numbers and M co-ordinate pairs (a,b);(c,d), find the sum of the values in the grid for each of the rectangles with top-left co-ordinate (a,b) and bottom-right co-ordinate (c,d).

18. The naïve approach • For each of the M rectangles, loop over the rectangle and sum up the values. This has O(N2M). • With DP, we can instead get O(N2+M) which will usually be better (unless M is huge)

19. Getting clever… • Suppose we can compute and store the sum of all rectangles with top-left co-ordinate (1,1). Let’s stash these in a 2D array called sum[][]. • Then, to work out the sum of a rectangle from (a,b) to (c,d), we can work it out using our sum array like this: sum(a,b,c,d) = sum[c][d] – sum[c][b-1] – sum[a-1][d] + sum[a-1][b-1]

20. Cool, now how do we create our sum array? • Suppose the grid values are in an array called grid[][]. To work out sum[i][j], we just need to see that it can be calculated as: sum[i][i] = sum[i][j-1] + sum[i-1][j] – sum[i-1][j-1] + grid[i][j]

21. … which we can do with a quick 2D loop sum[0][0] = 0; sum[0][j] = 0; sum[i][0] = 0; for (int i = 1; i <= N; i++) for (int j = 1; j <= N; j++) sum[i][j] = sum[i-1][j] + sum[i][j-1] - sum[i-1][j-1] + grid[i][j];

22. Common applications • Longest common subsequence problem • Floyd’s all-pairs shortest path • Knapsack problem • Duckworth-Lewis method! … amongst many others. See: http://en.wikipedia.org/wiki/Dynamic_Programming

23. Questions?