1 / 76

Dynamic Programming: Edit Distance

Dynamic Programming: Edit Distance. DNA Sequence Comparison: First Success Story. Finding sequence similarities with genes of known function is a common approach to infer a newly sequenced gene’s function

rod
Download Presentation

Dynamic Programming: Edit Distance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dynamic Programming:Edit Distance

  2. DNA Sequence Comparison: First Success Story • Finding sequence similarities with genes of known function is a common approach to infer a newly sequenced gene’s function • In 1984 Russell Doolittle and colleagues found similarities between cancer-causing gene and normal growth factor (PDGF) gene

  3. Bring in the Bioinformaticians • Gene similarities between two genes with known and unknown function alert biologists to some possibilities • Computing a similarity score between two genes tells how likely it is that they have similar functions • Dynamic programming is a technique for revealing similarities between genes • The Change Problem is a good problem to introduce the idea of dynamic programming

  4. The Change Problem Goal: Convert some amount of money M into given denominations, using the fewest possible number of coins Input: An amount of money M, and an array of d denominations c= (c1, c2, …, cd), in a decreasing order of value (c1 > c2 > … > cd) Output: A list of d integers i1, i2, …, id such that c1i1 + c2i2 + … + cdid = M and i1 + i2 + … + id is minimal

  5. Value 1 2 3 4 5 6 7 8 9 10 Min # of coins 1 1 1 Change Problem: Example Given the denominations 1, 3, and 5, what is the minimum number of coins needed to make change for a given value? Only one coin is needed to make change for the values 1, 3, and 5

  6. Change Problem: Example (cont’d) Given the denominations 1, 3, and 5, what is the minimum number of coins needed to make change for a given value? Value 1 2 3 4 5 6 7 8 9 10 Min # of coins 1 2 1 2 1 2 2 2 However, two coins are needed to make change for the values 2, 4, 6, 8, and 10.

  7. Change Problem: Example (cont’d) Given the denominations 1, 3, and 5, what is the minimum number of coins needed to make change for a given value? Value 1 2 3 4 5 6 7 8 9 10 Min # of coins 1 2 1 2 1 2 3 2 3 2 Lastly, three coins are needed to make change for the values 7 and 9

  8. minNumCoins(M-1) + 1 minNumCoins(M-3) + 1 minNumCoins(M-5) + 1 min of minNumCoins(M) = Change Problem: Recurrence This example is expressed by the following recurrence relation:

  9. minNumCoins(M-c1) + 1 minNumCoins(M-c2) + 1 … minNumCoins(M-cd) + 1 min of minNumCoins(M) = Change Problem: Recurrence (cont’d) Given the denominations c: c1, c2, …, cd, the recurrence relation is:

  10. Change Problem: A Recursive Algorithm • RecursiveChange( M, c, d ) • ifM = 0 • return 0 • bestNumCoins infinity • for i 1 to d • ifM ≥ ci • numCoins RecursiveChange( M – ci , c, d ) • ifnumCoins + 1 < bestNumCoins • bestNumCoins numCoins + 1 • returnbestNumCoins

  11. RecursiveChange Is Not Efficient • It recalculates the optimal coin combination for a given amount of money repeatedly • i.e., M = 77, c = (1,3,7): • Optimal coin combo for 70 cents is computed 9 times!

  12. The RecursiveChange Tree 77 76 74 70 75 73 69 73 71 67 69 67 63 74 72 68 68 66 62 70 68 64 68 66 62 62 60 56 72 70 66 72 70 66 66 64 60 66 64 60 . . . . . . 70 70 70 70 70

  13. We Can Do Better • We’re re-computing values in our algorithm more than once • Save results of each computation for 0 to M • This way, we can do a reference call to find an already computed value, instead of re-computing each time • Running time M*d, where M is the value of money and d is the number of denominations

  14. The Change Problem: Dynamic Programming • DPChange(M,c,d) • bestNumCoins0 0 • form 1 to M • bestNumCoinsm infinity • for i  1 to d • ifm ≥ ci • if bestNumCoinsm – ci+ 1< bestNumCoinsm • bestNumCoinsmbestNumCoinsm – ci+ 1 • returnbestNumCoinsM

  15. DPChange: Example 0 0 1 2 3 4 5 6 0 0 1 2 1 2 3 2 0 1 0 1 2 3 4 5 6 7 0 1 0 1 2 1 2 3 2 1 0 1 2 0 1 2 0 1 2 3 4 5 6 7 8 0 1 2 3 0 1 2 1 2 3 2 1 2 0 1 2 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 0 1 2 1 2 3 2 1 2 3 0 1 2 1 2 c = (1,3,7)M = 9 0 1 2 3 4 5 0 1 2 1 2 3

  16. Manhattan Tourist Problem (MTP) Imagine seeking a path (from source to sink) to travel (only eastward and southward) with the most number of attractions (*) in the Manhattan grid Source * * * * * * * * * * * * Sink

  17. Manhattan Tourist Problem (MTP) Imagine seeking a path (from source to sink) to travel (only eastward and southward) with the most number of attractions (*) in the Manhattan grid Source * * * * * * * * * * * * Sink

  18. Manhattan Tourist Problem: Formulation Goal: Find the longest path in a weighted grid. Input: A weighted grid G with two distinct vertices, one labeled “source” and the other labeled “sink” Output: A longest path in Gfrom “source” to “sink”

  19. MTP: An Example 0 1 2 3 4 j coordinate source 3 2 4 0 3 5 9 0 0 1 0 4 3 2 2 3 2 4 13 1 1 6 5 4 2 0 7 3 4 15 19 2 i coordinate 4 5 2 4 1 0 2 3 3 3 20 3 8 5 6 5 2 sink 1 3 2 23 4

  20. MTP: Greedy Algorithm Is Not Optimal 1 2 5 source 3 10 5 5 2 5 1 3 5 3 1 4 2 3 promising start, but leads to bad choices! 5 0 2 0 22 0 0 0 sink 18

  21. MTP: Simple Recursive Program MT( n, m ) ifn=0 or m=0 returnMT( n, m ) else x  MT(n-1,m)+ length of the edge from (n- 1,m) to (n,m) y  MT(n,m-1)+ length of the edge from (n,m-1) to (n,m) returnmax { x, y }

  22. MTP: Dynamic Programming j 0 1 source 1 0 1 S0,1= 1 i 5 1 5 S1,0= 5 • Calculate optimal path score for each vertex in the graph • Each vertex’s score is the maximum of the prior vertices score plus the weight of the respective edge in between

  23. MTP: Dynamic Programming (cont’d) j 0 1 2 source 1 2 0 1 3 S0,2 = 3 i 5 3 -5 1 5 4 S1,1= 4 3 2 8 S2,0 = 8

  24. MTP: Dynamic Programming (cont’d) j 0 1 2 3 source 1 2 5 0 1 3 8 S3,0 = 8 i 5 3 10 -5 1 1 5 4 13 S1,2 = 13 5 3 -5 2 8 9 S2,1 = 9 0 3 8 S3,0 = 8

  25. MTP: Dynamic Programming (cont’d) j 0 1 2 3 source 1 2 5 0 1 3 8 i 5 3 10 -5 -5 1 -5 1 5 4 13 8 S1,3 = 8 5 3 -3 3 -5 2 8 9 12 S2,2 = 12 0 0 0 3 8 9 S3,1 = 9 greedy alg. fails!

  26. MTP: Dynamic Programming (cont’d) j 0 1 2 3 source 1 2 5 0 1 3 8 i 5 3 10 -5 -5 1 -5 1 5 4 13 8 5 3 -3 2 3 3 -5 2 8 9 12 15 S2,3 = 15 0 0 -5 0 0 3 8 9 9 S3,2 = 9

  27. MTP: Dynamic Programming (cont’d) j 0 1 2 3 source 1 2 5 0 1 3 8 Done! i 5 3 10 -5 -5 1 -5 1 5 4 13 8 (showing all back-traces) 5 3 -3 2 3 3 -5 2 8 9 12 15 0 0 -5 1 0 0 0 3 8 9 9 16 S3,3 = 16

  28. si-1, j + weight of the edge between (i-1, j) and (i, j) si, j-1 + weight of the edge between (i, j-1) and (i, j) max si, j = MTP: Recurrence Computing the score for a point (i,j) by the recurrence relation: The running time is n x m for a n by mgrid (n = # of rows, m = # of columns)

  29. A2 A3 A1 B sA1 + weight of the edge (A1, B) sA2 + weight of the edge (A2, B) sA3 + weight of the edge (A3, B) max of sB = Manhattan Is Not A Perfect Grid What about diagonals? • The score at point B is given by:

  30. max of sy + weight of vertex (y, x) where y є Predecessors(x) sx = Manhattan Is Not A Perfect Grid (cont’d) Computing the score for point x is given by the recurrence relation: • Predecessors (x) – set of vertices that have edges leading to x • The running time for a graph G(V, E) (V is the set of all vertices and Eis the set of all edges) is O(E) since each edge is evaluated once

  31. Traveling in the Grid • The only hitch is that one must decide on the order in which visit the vertices • By the time the vertex x is analyzed, the values sy for all its predecessors y should be computed – otherwise we are in trouble. • We need to traverse the vertices in some order • Try to find such order for a directed cycle ???

  32. DAG: Directed Acyclic Graph • Since Manhattan is not a perfect regular grid, we represent it as a DAG • DAG for Dressing in the morning problem

  33. Topological Ordering • A numbering of vertices of the graph is called topological ordering of the DAG if every edge of the DAG connects a vertex with a smaller label to a vertex with a larger label. • In other words, if vertices are positioned on a line in an increasing order of labels then all edges go from left to right.

  34. Topological ordering • 2 different topological orderings of the DAG:

  35. Longest Path in DAG Problem • Goal: Find a longest path between two vertices in a weighted DAG • Input: A weighted DAG G with source and sink vertices • Output: A longest path in G from source to sink

  36. max of sv= Longest Path in DAG: Dynamic Programming • Suppose vertex v has indegree 3 and predecessors {u1, u2, u3} • Longest path to v from source is: In General: sv = maxu (su + weight of edge from uto v) su1 + weight of edge fromu1to v su2 + weight of edge fromu2to v su3+ weight of edge fromu3to v

  37. Traversing the Manhattan Grid a) b) • 3 different strategies: • a) Column by column • b) Row by row • c) Along diagonals c)

  38. T A G T C C A T T G A A T Alignment: 2 row representation Given 2 DNA sequences v and w: v : m = 7 w : n = 6 Alignment : 2 * k matrix ( k > m, n ) letters of v A T -- G T T A T -- letters of w A T C G T -- A -- C 4 matches 2 insertions 2 deletions

  39. Aligning DNA Sequences n = 8 V = ATCTGATG matches mismatches insertions deletions 4 m = 7 1 W = TGCATAC 2 match 2 mismatch V W deletion indels insertion

  40. Longest Common Subsequence (LCS) – Alignment without Mismatches • Given two sequences • v = v1v2…vm and w = w1w2…wn • The LCS of vand w is a sequence of positions in • v: 1 < i1 < i2 < … < it< m • and a sequence of positions in • w: 1 < j1 < j2 < … < jt< n • such that it -th letter of v equals to jt-letter of wand t is maximal.

  41. 0 1 2 2 3 3 4 5 6 7 8 0 0 1 2 3 4 5 5 6 6 7 LCS: Example i coords: elements of v A T -- C -- T G A T C elements of w -- T G C A T -- A -- C j coords: (0,0) (1,0) (2,1) (2,2) (3,3) (3,4) (4,5) (5,5) (6,6) (7,6) (8,7) positions in v: 2 < 3 < 4 < 6 < 8 Matches shown inred positions in w: 1 < 3 < 5 < 6 < 7 Every common subsequence is a path in 2-D grid

  42. LCS Problem as Manhattan Tourist Problem A T C T G A T C j 0 1 2 3 4 5 6 7 8 i 0 T 1 G 2 C 3 A 4 T 5 A 6 C 7

  43. Edit Graph for LCS Problem A T C T G A T C j 0 1 2 3 4 5 6 7 8 i 0 T 1 G 2 C 3 A 4 T 5 A 6 C 7

  44. Edit Graph for LCS Problem A T C T G A T C j 0 1 2 3 4 5 6 7 8 Every path is a common subsequence. Every diagonal edge adds an extra element to common subsequence LCS Problem: Find a path with maximum number of diagonal edges i 0 T 1 G 2 C 3 A 4 T 5 A 6 C 7

  45. si-1, j si, j-1 si-1, j-1 + 1 if vi = wj max si, j = Computing LCS Let vi = prefix of v of length i: v1 … vi and wj = prefix of w of length j: w1 … wj The length of LCS(vi,wj) is computed by:

  46. i-1,j i-1,j -1 0 1 si-1,j + 0 0 i,j -1 si,j = MAX si,j -1 + 0 i,j si-1,j -1 + 1, if vi = wj Computing LCS (cont’d)

  47. Every Path in the Grid Corresponds to an Alignment W A T C G 0 1 2 2 3 4 V = A T - G T | | | W= A T C G – 0 1 2 3 4 4 V A T G T

  48. A T A T A T A T T A T A T A T A Aligning Sequences without Insertions and Deletions: Hamming Distance Given two DNA sequences vand w : v : w: • The Hamming distance: dH(v, w) = 8 is large but the sequences are very similar

  49. A T A T A T A T T A T A T A T A Aligning Sequences with Insertions and Deletions By shifting one sequence over one position: v : -- w: -- • The edit distance: dH(v, w) = 2. • Hamming distance neglects insertions and deletions in DNA

  50. Edit Distance • Levenshtein (1966) introduced Edit Distance between two strings as the minimum number of elementary operations (insertions, deletions, and substitutions) to transform one string into the other d(v,w) = MIN number of elementary operations to transform v w

More Related