Download Presentation
## Dynamic Programming

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Dynamic Programming**Lecture 9 Asst. Prof. Dr. İlker Kocabaş**Dynamic Programming History**• Bellman. Pioneered the systematic study of dynamic programming in the 1950s. • Etymology. • Dynamic programming = planning over time. • Secretary of Defense was hostile to mathematical research. • Bellman sought an impressive name to avoid confrontation. • "it's impossible to use dynamic in a pejorative sense" • "something not even a Congressman could object to" Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography.**Introduction**• Dynamic Programming(DP) applies to optimiza- tion problems in which a set of choices must be made in order to arrive at an optimal solution. • As choices are made, subproblems of the same form arise. • DP is effective when a given problem may arise from more than one partial set of choices. • The key technique is to store the solution to each subproblem in case it should appear**Introduction (cont.)**• Divide and Conquer algorithms partition the problem into independent subproblems. • Dynamic Programming is applicable when the subproblems are not independent.(In this case DP algorithm does more work than necessary) • Dynamic Programming algorithm solves every subproblem just once and then saves its answer in a table.**Dynamic Programming Applications**• Areas. • Bioinformatics. • Control theory. • Information theory. • Operations research. • Computer science: theory, graphics, AI, systems, …. • Some famous dynamic programming algorithms. • Viterbi for hidden Markov models. • Unix diff for comparing two files. • Smith-Waterman for sequence alignment. • Bellman-Ford for shortest path routing in networks. • Cocke-Kasami-Younger for parsing context free grammars.**Assembly-line Scheduling**• If we are given a list of which stations to use in line 1 and which to use in line 2, then T(n)=Θ(n). • There are 2n possible ways to choose stations. Thus, the solution is obtained by enumarating all posible ways and computing how long each takes would take Ω(2n).**The steps of a dynamic programming**• Characterize the structure of an optimal solution • Recursively define the value of an optimal solution • Compute the value of an optimal solution in a bottom-up fashion • Construct an optimal solution from computed information**Step 1:The structure of the fastest way through the factory**• Consider the fastest possible way(fpw) to get from the starting point through station S1,j. • If j=1, there isonly one way thatthe chassis could have gone. • For j=2, 3, ..., n there are two choices: • The chassis could have come from S1, j-1and then directly to station S1, j. • The chassis could have come from S2, j-1and then directly to station S1, j-1.**Step 1:The structure of the fastest way through the factory**(cont.) • First, suppose that the fastest way(fw) through station S1, j is throughstation S1, j-1. • The key observation is that the chassis must have taken a fw from the starting point through station S1, j-1.**Step 1:The structure of the fastest way through the factory**(cont.) • Similarly, suppose that the fw through station S2, j is throughstation S2, j-1. • We observe that the chassis must have taken a fw from tha starting point through station S2, j-1.**Step 1:The structure of the fastest way through the factory**(cont.) • We can say that for assembly-line scheduling, an optimal solution to a problem contains within it an optimal solution to sub-problems (finding the fw through either S1, j-1or S2, j-1). • We refer to this property as optimal substructure.**The fw through stations S1, j**• The fw through station S1, j-1and then directly through station S1, j. or • The fw through station S2, j-1and then through station S1, j.**The fw through station S2, j**• The fw through station S2, j-1and then directly through station S2, j. or • The fw through station S1, j-1and then through station S2, j.**Step 1:The structure of the fastest way through the factory**(cont.) • To solve the problem of finding the fw through station j of either line, we solve the subproblems of finding the fw through station j-1 on both lines. • Thus, we can build an optimal solution to an instance of the assemly line scheduling problem by building on optimal solutions to subproblems**Step 2: A recursive solution**• We define the value of an optimal solution recursively in tems of the optimal solutions to subproblems. • Let fi[j] denote the fastest possible time to get a chassis from the starting point throuhg station Si, j. • Let fi* denote the fastest time to get a chassis all the way through the factory.**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 9 3 4 Line 1 7 4 8 2 3 2 1 3 3 4 Chassis Enters 5 2 1 2 1 4 2 5 6 4 8 5 7 Line 2 Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] l1[j] f*=0 l* =0 f2[j] l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 9 3 4 Line 1 9 4 8 3 2 1 3 3 4 Chassis Enters 5 2 1 2 1 2 5 6 4 12 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 l1[j] f*=0 l* =0 12 f2[j] l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 4 Line 1 9 4 18 3 8 3 1 3 3 4 Chassis Enters 5 1 2 1 2 6 4 16 12 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 1 l1[j] f*=0 l* =0 12 16 f2[j] 1 l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 4 Line 1 9 4 18 20 8 3 1 3 4 Chassis Enters 5 2 1 2 4 16 22 12 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 1 2 l1[j] f*=0 l* =0 12 16 22 f2[j] 1 2 l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 4 18 20 8 3 3 4 Chassis Enters 2 1 2 16 22 12 25 5 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 1 2 1 l1[j] f*=0 l* =0 12 16 22 25 f2[j] 1 2 1 l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 4 18 20 32 3 4 Chassis Enters 1 2 16 22 12 25 30 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 1 2 1 1 l1[j] f*=0 l* =0 12 16 22 25 30 f2[j] 1 2 1 2 l2[j]**Step 2: A recursive solution: An example( Correction !!! )**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 4 18 20 32 3 4 Chassis Enters 1 2 16 22 12 25 30 7 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 1 2 1 1 l1[j] f*=0 l* =0 12 16 22 25 30 f2[j] 1 2 1 2 l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 35 18 20 20 32 3 Chassis Enters 2 16 22 22 12 25 30 37 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 35 1 2 1 1 1 l1[j] f*=0 l* =0 12 16 22 25 30 37 f2[j] 1 2 1 2 2 l2[j]**Step 2: A recursive solution: An example**Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 24 Line 1 9 35 18 20 20 32 3 Chassis Enters 2 16 22 22 12 25 30 37 Line 2 Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j 1 2 3 4 5 6 j 2 3 4 5 6 f1[j] 9 18 20 24 32 35+3 1 2 1 1 2 l1[j] f*=38 l* =1 12 16 22 25 30 37+2 f2[j] 1 2 1 2 2 l2[j]**Step 2: A recursive solution (cont.)**• fi[j] values give the values of optimal solutions to subproblems. • Let li[j] to be the line number 1 or 2, whose station j-1 is used in a fw throgh station Si, j. • We also define l* to be the line whose station nis used in a fw through the entire factory.**Step 2: A recursive solution (cont.)**The fw through the entire factory:**Step 3: Computing the fastest times**We can write a simple recursive algorithm based on the equations for f1[j] and f2[j] to compute the fw through the factory. However its running time is exponential. Let ri( j ) be the number of references made tofi [ j ] in a recursive algorithm:**Step 3: Computing the fastest times(cont.)**• We observe that for each value of fi[j] depends only on the values of f1[j-1] and f2[j-1]. • By computing the fi[j] values in order of increasing station numbers we can compute the fw through the factory in time.**The fastest way procedure (cont.)**line 1, station 6 line 2, station 5 line 2, station 4 line 1, station 3 line 2, station 2 line 1, station 1**Matrix-Chain multiplication**• We are given a sequence • And we wish to compute**Matrix-Chain multiplication (cont.)**• Matrix multiplication is assosiative, and so all parenthesizations yield the same product. • For example, if the chain of matrices is then the product can be fully paranthesized in five distinct way:**Matrix-Chain multiplication**MATRIX-MULTIPLY (A,B) if columns[A] ≠ rows[B] then error “incompatible dimensions” else for i←1 to rows[A] do for j←1 to columns[B] do C[i,j]←0 for k←1 to columns[A] do C[ i,j ]← C[i,j]+A[i,k]*B[k,j] return C**Matrix-Chain multiplication (cont.)**Cost of the matrix multiplication: An example:**Matrix-Chain multiplication (cont.)**• The problem: Given a chain of nmatrices, where matrix Ai has dimension pi-1xpi, fully paranthesize the product in a way that minimizes the number of scalar multiplications.**Matrix-Chain multiplication (cont.)**• Counting the number of alternative paranthesization : bn**Matrix-Chain multiplication (cont.)**• Based on the above defination we have**Matrix-Chain multiplication (cont.)**• Substituting the initial value for b we obtain**Matrix-Chain multiplication (cont.)**• Let B(x) be the generating function**Matrix-Chain multiplication (cont.)**• Substituting in B(x) we can write that B(x)**Matrix-Chain multiplication (cont.)**• Summing over the both sides**Matrix-Chain multiplication (cont.)**• One way to express B(x) in closed form is Expanding into a Taylor series about x=0 we get Comparing the coefficients of x we finally show that**Matrix-Chain multiplication (cont.)**Step 1: The structure of an optimal paranthesization(op) • Find the optimal substructure and then use it to construct an optimal solution to the problem from optimal solutions to subproblems. • Let Ai...jwhere i ≤ j, denote the matrix product Ai Ai+1 ... Aj • Anyparenthesization of Ai Ai+1 ... Aj must split the product between Ak andAk+1 fori ≤ k < j.**Matrix-Chain multiplication (cont.)**The optimal substructure of the problem: • Suppose that an op of Ai Ai+1 ... Aj splits the productbetween Akand Ak+1 then the paranthesization of the subchain Ai Ai+1 ... Ak within this parantesization of Ai Ai+1 ... Aj must be an op of Ai Ai+1 ... Ak**Matrix-Chain multiplication (cont.)**Step 2: A recursive solution: • Let m[i,j] be the minimum number of scalar multiplications needed to compute the matrix Ai...jwhere 1≤ i ≤ j ≤ n. • Thus, the cost of a cheapest way to compute A1...nwould be m[1,n]. • Assume that the op splits the product Ai...jbetween Akand Ak+1.wherei ≤ k <j. • Then m[i,j] =Theminimum cost for computing Ai...k and Ak+1...j+ the cost of multiplying these two matrices.**Matrix-Chain multiplication (cont.)**Recursive defination for the minimum cost of paranthesization: