Overlap Matching

1 / 58

# Overlap Matching - PowerPoint PPT Presentation

Overlap Matching. By Itamar Nabriski. A. Amir, R. Cole, G. Landau, R. Hariharan, M. Lewenstein, E. Porat, Overlap Matching, Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms (2001) 279-288,. Lecture Structure. Discrete Convolutions Overlap Matching Problem Definition

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## Overlap Matching

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Overlap Matching

By Itamar Nabriski

A. Amir, R. Cole, G. Landau, R. Hariharan, M. Lewenstein, E. Porat, Overlap Matching, Proceedings of the twelfth annual ACM-SIAM symposium on Discrete algorithms (2001) 279-288,.

Lecture Structure
• Discrete Convolutions
• Overlap Matching Problem Definition
• Overlap Matching Algorithm
• Reduction from Swap Matching
Definition 1

Let T be a function whose domain is {0,…,n-1}

Let P be a function whose domain is {0,…,m-1}

(we’ll view them as arrays of numbers of length n and m respectively)

The Convolution of Tand P at indexj is defined as follows:

Exempli Gratia

T=Ronaldinho

(n=10)

P=Deco

(m=4)

(assume, for now, each letter represents a number)

(TxP)[2]=

(TxP)[2]=

Thus Naïve Computional Time is O(m)

nD+ae+lc+do

Computing All The Convolutions

Since

number of possible convolutions is O(n)

Naïve Approach:

For each j pay O(m) time, for a total ofO(nm)

Devious Approach:

Using the “Fast Fourier Transform” (FFT) For each j pay O(log m) time, for a total ofO(n log m)

(machine word size must be O(log m) though)

Using Convolutions

Preprocessing

Before we perform convolutions on T andPwe preprocess each letter using a constant number of constant time functions (total O(n)), retaining the running time of O(nlogm)

Using several Convolutions

For each indexj we can preform a constant amount of convolutions, retaining the running time of O(nlogm)

Postprocessing

For each index j we can use a constant time function fto format the output of the constant number of convolutions, retaining a running time of O(nlogm)

Using Convolutions - Example

Testing For Exact Matching at index j

∑ = {a,b}

T = ababbaaa

P = abba

Thus, for example, T and P exactly match at index j = 2 :

2

Using Convolutions - Example

Testing For Exact Matching at index j

Preprocessing functions (x is a letter):

When we write prep(S),S being a string, we mean we apply prep to all characters of S

Using Convolutions - Example

Testing For Exact Matching at index j

Convolutions we will use:

Postprocessing function f:

For every index j,

Iff there is an exact matching of P and T at index j.

Using Convolutions - Example

Testing For Exact Matching at index j

P = abba

prepa(P)=0110

prepb(T)=1001

T = ababbaaa

prepa(T)=10100111

prepb(T)=01011000

prepa(T) X prepa(P)[2]=

prepb(T) X prepb(P)[2]=

j =2

= 0*1+1*0+1*0+0*1=0

= 1*0+0*1+0*1+1*0=0

F(0,0) = 1 = exact match at 2

Stractural String
• Linear structure made of segments
• Each segment can be marked or unmarked
• A Stractural String is a concatenation of segments

Exampli Gratia

Thus, the actual characters are not important

Defintion of Overlap Matching

Input

• Stractural String T (text) of length n
• Stractural String P (pattern) of length m
• m≤n
• Both T and P have some marked segments

Output

• All locations k in T where if P is aligned at k, all marked segments have overlaps of even length
Example

T = Franz_Beckenbauer

P = The_Kaiser

j=3

3

Overlaps (j=3 is not a valid overlap match – has an odd overlap):

General Preprocessing

Each segment can start at either an even or odd index and end at either an even or odd index

We will produce from Tfour new segmentsToo,Tee,ToeandTeo

Toowill have 1’s in the place of all characters belonging to segments that start and end at an odd indexand 0’s otherwise, for example:

3

7

Too= 0001111100

And analgously for the other segment types …

General Preprocessing

Since the pattern P tends to move around we will need to treat its segment indexes a bit differently

We will produce from Peight new segmentsPOoo,POee,POoe,POeo, PEoo,PEee,PEoeand PEeo

The big‘O’in POeemeansthe all segments in P that start and end at an even location relative to T’s index, when P is aligned to an odd index of T

(don’t worry there is an example in the next slide …)

And analgously for the other segment types …

General Preprocessing

T=

Too=

Tee=

Teo=

Toe=

General Preprocessing

P =

Since P is always aligned to T at some index j we treat’s P’s indexes relative to T, thus:

P =

Assume j is now odd, then for that location we will use the four PO’s:

POoo=

POee=

POeo=

POoe=

General Preprocessing

Thus for every locationj we have 16(a constant) possible number of Text-Pattern pairings:

{Too,Tee,Toe,Tee} × {PXoo,PXee,PXoe,PXee}

X=parity(j)

If we can determine, using convolutions, for each pairing if it only contains even overlaps we can solve the Overlap Matching problem in O(n log m) time

Case 1

Case 1 occurs when for Tab, Pcd either a=c or b=d

This covers 12 of the 16cases.

We now show a solution for when a=c. This covers 8 cases, we use the solution on the reverse* strings of T and P (thus ‘a’ becomes ‘c’ and ‘b’ becomes ‘d’) to solve the 4 remaining cases.

* Computing the reverse strings does not alter the run time (do it during general preprocessing)

Case 1 (a=c)

For every two marked segments St in Tab starting at index x and Sp in Pcdstarting at index y:

|x-y| is always even

(since even-even = even and odd-odd = even)

We now create a convolution that will return 0 for index j iff there is no odd overlap at j

Case 1 (a=c)

For every segment in Tab we replace the 1’s by an alrenating series of 1’s and -1’s beginning with 1.

In case where we have only even (and/or no) overlaps:

= 1 - 1 = 0

In case where we have at least one odd overlap:

5

6

3

1

= 1 – 1 + 1 = 1 > 0

Case 2

Case 2 occurs when Toe, Peo (Teo, Poe is symmetric)

If a segment in Toeis contained in a segment inPeoor vice versathen the overlap is even, otherwise overlap is odd.

2

4

8

11

Case 2

Containment Elimination Property

Convolution at index j gives zero if all overlaps are containments, otherwise it gives a positive result .

To achieve this we will actually use 3 convolutions, a combination of their output will give us the desired answer.

Case 2

Fleshing Out The Solution

For each segment St in Toe that starts at index st, replace the segment’s 1’s by st,1…1,-st

For each segment Sp in Peo that starts at index sp, replace the segment’s 1’s by sp,1…1,-sp

3

3

Case 2

Containment:

sp + (len(Sp)-2) + -sp = len(Sp)-2

st + (len(St)-2) + -st = len(St)-2

No Containment (overlap of length k):

k-2

k-2

st + (k-2) + -sp

sp + (k-2) + -st

Case 2

Problem 1

The indexes of the pattern Peo change for each indexj , raising the preprocessing time to O(m) for each convolution!

Problem 2

We need to find a way to remove “The size of the overlap -2” from the resulting convolution.

Containment

len(Sp)-2

0

0

len(St)-2

Remove “overlap - 2”

No Containment

sp - st 0<

sp + (k-2) + -st

st - sp 0<

st + (k-2) + -sp

Case 2

Solving Problem 2

Perform another convolution, The “Overlap Length Convolution”subtract its value from the main convolution.

Every segment both Toe and Peois replaced by0,1,1,….1,0giving us “size of overlap -2” for each overlap.

3

3

Overlap of length 4 :

= 0+1+1+0 = 2 = “overlap -2”

Case 2

Solving Problem 1

The trouble is with the pattern Peo segments whose indexes change in each index j. Instead treat the pattern segments relative to Peo. (“Zero Containment Convolution”)

3

4

T

P

4

3

T

P

1

2

0

Case 2

Solving Problem 1

We created a new problem, overlap convolutions can be negative and thus the overall convolution at index j can turn out to be zero when there is an odd overlap.

7

T

P

2

= 2+1-7 = -4

Case 2

Solving Problem 1

We want to get the benefits of both worlds. Towards that end we’ll add to the result a third convolution “The Shifting Convolution”. This simply corrects the problem caused by using the pattern indexes.

Every segment in T is replaced by 1,0…0,1 and every segment in P is replaced by 0,1…,1,0 and the result is multiplied by index j.

j2

3

7

4

T

= 2

P

0

1

2

2 * j = 2 * 2 = 4

This replenishes our “losses”

Case 2

Solving Problem 1

Thus, the convolution gives 0 for each containment overlap and 1 for each non-containment overlap.

1

2

T

= 1

P

1

2

T

= 0

P

Thus multiplying by j we return “one j” to each non-containment overlap

Case 2

Final Algorithm

Thus we implement the “Containment Elimination Property” by:

Zero Containment Convolution

+

Shifting Convolution

-

Overlap Length Convolution

=

Containment Elimination Property

Case 3

Case 3 occurs when Too, Pee (Tee, Poo is symmetric)

If a segment in Toois contained in a segment inPeeor vice versathen the overlap is odd, otherwise overlap is even.

1

2

3

4

7

8

10

13

Case 3 - Using Case 2

Containment:

sp + (len(Sp)-2) + -sp = len(Sp)-2

st + (len(St)-2) + -st = len(St)-2

No Containment (overlap of length k):

k-2

k-2

st + (k-2) + -sp

sp + (k-2) + -st

Case 3

We’ll use the same convolution as in Case 2 and two additional ones:

Conv1: Every segment in Too of length len replace by 0,1,2,…,len-1.

Replace Pee segments by 1,0,…,0.

1

1

T

P

Conv2: (Opposite of 1) Every segment in Pee of length len replace by 0,1,2,…,len-1.

Replace Too segmentsby 1,0,…,0.

Case 3

The first convolution gives us the length of all areas like the one marked in green:

= 3

It gives us for every two overlapping segments which St is “ahead” of Sp

If, for some overlap, the first convolution is positive the second will be zero, and vice versa.

= 0

Case 3

The reverse case (second convolution):

= 3

And the first one is now zero:

= 0

Case 3

This is true also for containments:

= 3

The convolution from Case 2 gives the same value for non containments and zero for containments.

Case 3

Thus:

Conv1 + Conv2 – ConvCase2 = positive

= containments = odd overlap

Conv1 + Conv2 – ConvCase2 = 0

= no containments = onlyeven overlaps

Overlap Matching

Algorithm Final Outcome

Each Case (1,2,3) takes O(n log m) :

1. A constant number of preprocessing functions O(n)

2. A constant number of convolutions O(n log m)

3. A constant time computable function O(1)

for a total runtime of O(n log m)

Swap Matching

Formal Definition

Let S =s1,…,s2 be a string over alphabet ∑

A swap permutation for S is a permutation

π : {1,…,n} → {1,…,n} such that:

• If π(i) = j then π(j) = I
• For all i, π(i) member of { i-1 , i , i+1 }
• If π(i) ≠ i then sπ(i) ≠ si
Swap Matching

Lemma (will not be proven):

A solution to swap matching over alphabet {a,b} of time O(f(n,m)) implies a solution of time O(log|∑|f(n,m)) over alphabet ∑.

And there exists an algorithm to do so.

A. Amir, Y. Aumann, G. Landau, M. Lewensten, N. Lewenstein, Pattern matching with swaps, J. Algorithms 37 (2) (2000) 247-266.

Swap Matching

Maximal Alternating Segment (MAS)

Swap Matching
• Lemma:
• The pattern P does NOT match in a particular alignment iff there exists a MASA in T and MASB in P such that:
• The characters of A and Bmisalign in the overlap
• The overlap is of odd length
Swap Matching

Lemma Intuition

Even overlap mismatch

Odd overlap mismatch

Swap Matching

Assume P is aligned to T at index j and we can’t swap match and the two MAS A,B do not exist:

• All MAS’ overlaps match exactly –contradiction– we don’t even need to swap.
• There exists at least one pair A,B that do not match exactly in an even overlap:

w.l.o.g overlapA=(ab)* overlapB=(ba)*

we can swap within the overlap boundaries and get the desired result - contradiction

Thus, there must be one MAS A,B that have a misaligned odd overlap

Swap Matching

Assume there exist MAS A,B that misalign in an odd overlap and P and T swap match at index j:

w.l.o.g overlapA=(ab)*a overlapB=(ba)*b

The we must swap with letters outside of the overlap but by definition of MAS this will not help and we can’t swap match. Contradiction.

Swap Matching

Algorithm

• Construct from T:
• Teven-a where all MAS with a’s on even indexes are marked segments.
• Todd-a where all MAS with a’s on odd indexes are marked segments.

0

2

6

7

8

9

T =

Teven-a =

Todd-a =

Swap Matching

Algorithm

We provide a similar construction for P :

Peven-a ,Podd-ausing P’s index !

When matching,if the indexj of T is odd we will use one for the other(Peven-abecomes Podd-aand vice versa)

0

4

Peven-a =

Aligned at T’s index 3 it becomes Podd-a:

3

7

Swap Matching

Algorithm

If index j is even, T swap matches P iff Teven-a overlap matches Podd-a at j and Todd-aoverlap matches Peven-a at j.

If index j is odd, T swap matches P iff Teven-a overlap matches Peven-a at j and Todd-aoverlap matches Podd-a at j.

Swap Matching

Algorithm – Why does it work?

An even-a MAS and an odd-a MAS will never exactly match:

0

even-aMAS

odd-aMAS

By the lemma if their overlap is odd then swap matching is not possible and this is exactly what we examine using the Overlap Matching method

Swap Matching

Algorithm

• Runtime O(n log m):
• We pay O(n) to segmentize to MAS.
• We pay O(n log m) to run overlap matching.

Thus, for an alphabet ∑ we can swap match at O(n log m log|∑|)

Improvement over previous deterministic upper bound of O(nm1/3log mlog|∑|)

A. Amir, Y. Aumann, G. Landau, M. Lewensten, N. Lewenstein, Pattern matching with swaps, J. Algorithms 37 (2) (2000) 247-266.