Combinatorial pattern matching
This presentation is the property of its rightful owner.
Sponsored Links
1 / 37

Combinatorial Pattern Matching PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on
  • Presentation posted in: General

Combinatorial Pattern Matching. CS 466 Saurabh Sinha. Genomic Repeats. Example of repeats: AT GGTC TA GGTC CTAGT GGTC Motivation to find them: Genomic rearrangements are often associated with repeats Trace evolutionary secrets Many tumors are characterized by an explosion of repeats.

Download Presentation

Combinatorial Pattern Matching

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Combinatorial pattern matching

Combinatorial Pattern Matching

CS 466

Saurabh Sinha


Genomic repeats

Genomic Repeats

  • Example of repeats:

    • ATGGTCTAGGTCCTAGTGGTC

  • Motivation to find them:

    • Genomic rearrangements are often associated with repeats

    • Trace evolutionary secrets

    • Many tumors are characterized by an explosion of repeats


Genomic repeats1

Genomic Repeats

  • The problem is often more difficult:

    • ATGGTCTAGGACCTAGTGTTC

  • Motivation to find them:

    • Genomic rearrangements are often associated with repeats

    • Trace evolutionary secrets

    • Many tumors are characterized by an explosion of repeats


L mer repeats

l -mer Repeats

  • Long repeats are difficult to find

  • Short repeats are easy to find (e.g., hashing)

  • Simple approach to finding long repeats:

    • Find exact repeats of short l-mers (l is usually 10 to 13)

    • Use l -mer repeats to potentially extend into longer, maximal repeats


L mer repeats cont d

l -mer Repeats (cont’d)

  • There are typically many locations where an l -mer is repeated:

    GCTTACAGATTCAGTCTTACAGATGGT

  • The 4-mer TTAC starts at locations 3 and 17


Extending l mer repeats

Extending l -mer Repeats

GCTTACAGATTCAGTCTTACAGATGGT

  • Extend these 4-mer matches:

    GCTTACAGATTCAGTCTTACAGATGGT

  • Maximal repeat: CTTACAGAT


Maximal repeats

Maximal Repeats

  • To find maximal repeats in this way, we need ALL start locations of all l -mers in the genome

  • Hashing lets us find repeats quickly in this manner


Hashing maximal repeats

Hashing: Maximal Repeats

  • To find repeats in a genome:

    • For all l -mers in the genome, note the start position and the sequence

    • Generate a hash table index for each unique l -mer sequence

    • In each index of the hash table, store all genome start locations of the l -mer which generated that index

    • Extend l -mer repeats to maximal repeats


Pattern matching

Pattern Matching

  • What if, instead of finding repeats in a genome, we want to find all sequences in a database that contain a given pattern?

  • This leads us to a different problem, the Pattern MatchingProblem


Pattern matching problem

Pattern Matching Problem

  • Goal: Find all occurrences of a pattern in a text

  • Input: Pattern p = p1…pn and text t = t1…tm

  • Output: All positions 1<i< (m – n + 1) such that the n-letter substring of t starting at i matches p

  • Motivation: Searching database for a known pattern


Exact pattern matching running time

Exact Pattern Matching: Running Time

  • Naïve runtime: O(nm)

  • On average, it’s more like O(m)

    • Why?

  • Can solve problem in O(m) time ?

    • Yes, we’ll see how (later)


Generalization of problem multiple pattern matching problem

Generalization of problem:Multiple Pattern Matching Problem

  • Goal: Given a set of patterns and a text, find all occurrences of any of patterns in text

  • Input: k patterns p1,…,pk, and text t = t1…tm

  • Output: Positions 1 <i <m where substring of t starting at i matches pj for 1 <j<k

  • Motivation: Searching database for known multiple patterns

  • Solution: k “pattern matching problems”: O(kmn)

  • Solution: Using “Keyword trees” => O(kn+m) where n is maximum length of pi


Keyword trees example

Keyword Trees: Example

  • Keyword tree:

    • Apple


Keyword trees example cont d

Keyword Trees: Example (cont’d)

  • Keyword tree:

    • Apple

    • Apropos


Keyword trees example cont d1

Keyword Trees: Example (cont’d)

  • Keyword tree:

    • Apple

    • Apropos

    • Banana


Keyword trees example cont d2

Keyword Trees: Example (cont’d)

  • Keyword tree:

    • Apple

    • Apropos

    • Banana

    • Bandana


Keyword trees example cont d3

Keyword Trees: Example (cont’d)

  • Keyword tree:

    • Apple

    • Apropos

    • Banana

    • Bandana

    • Orange


Keyword trees properties

Keyword Trees: Properties

  • Stores a set of keywords in a rooted labeled tree

  • Each edge labeled with a letter from an alphabet

  • Any two edges coming out of the same vertex have distinct labels

  • Every keyword stored can be spelled on a path from root to some leaf


Multiple pattern matching keyword tree approach

Multiple Pattern Matching: Keyword Tree Approach

  • Build keyword tree in O(kn) time; kn is total length of all patterns

  • Start “threading” at each position in text; at most n steps tell us if there is a match here to any pi

  • O(kn + nm)

  • Aho-Corasick algorithm: O(kn + m)


Aho corasick algorithm

Aho-Corasick algorithm


Fail edges in keyword tree

“Fail” edges in keyword tree

Dashed edge out of internal node if matching edge not found


Fail edges in keyword tree1

“Fail” edges in keyword tree

  • If currently at node q representing word L(q), find the longest proper suffix of L(q) that is a prefix of some pattern, and go to the node representing that prefix

  • Example: node q = 5 L(q) = she; longest proper suffix that is a prefix of some pattern: “he”. Dashed edge to node q’=2


Automaton

Automaton

  • Transition among the different nodes by following edges depending on next character seen (“c”)

  • If outgoing edge with label “c”, follow it

  • If no such edge, and are at root, stay

  • If no such edge, and at non-root, follow dashes edge (“fail” transition); DO NOT CONSUME THE CHARACTER (“c”)

Example: search text “ushers” with the automaton


Aho corasick algorithm1

Aho-Corasickalgorithm

  • O(kn) to build the automaton

  • O(m) to search a text of length m

  • Key insight:

    • For every character “consumed”, we move at most one level deeper (away from root) in the tree. Therefore total number of such “away from root” moves is <= m

    • Each fail transition moves us at least one level closer to root. Therefore total number of such “towards root” moves is <= m (you cant climb up more than you climbed down)


Suffix tree

Suffix tree

  • Build a tree from the text

  • Used if the text is expected to be the same during several pattern queries

  • Tree building is O(m) where m is the size of the text. This is preprocessing.

  • Given any pattern of length n, we can answer if it occurs in text in O(n) time

  • Suffix tree = “modified” keyword tree of all suffixes of text


Suffix tree collapsed keyword trees

Suffix Tree=Collapsed Keyword Trees

  • Similar to keyword trees, except edges that form paths are collapsed

    • Each edge is labeled with a substring of a text

    • All internal edges have at least two outgoing edges

    • Leaves labeled by the index of the pattern.


Suffix tree of a text

Suffix Tree of a Text

  • Suffix trees of a text is constructed for all its suffixes

ATCATG TCATG CATG ATG TG G

Keyword Tree

Suffix Tree

quadratic

Time is linear in the total size of all suffixes, i.e., it is quadratic in the length of the text


Suffix trees advantages

Suffix Trees: Advantages

  • Suffix trees build faster than keyword trees

ATCATG TCATG CATG ATG TG G

Keyword Tree

Suffix Tree

quadratic

linear (Weiner suffix tree algorithm)

Time to find k patterns of length n: O(m + kn)


Suffix trees example

Suffix Trees: Example


Multiple pattern matching summary

Multiple Pattern Matching: Summary

  • Keyword and suffix trees are used to find patterns in a text

  • Keyword trees:

    • Build keyword tree of patterns, and thread text through it

  • Suffix trees:

    • Build suffix tree of text, and thread patterns through it


Approximate vs exact pattern matching

Approximate vs. Exact Pattern Matching

  • So far all we’ve seen exact pattern matching algorithms

  • Usually, because of mutations, it makes much more biological sense to find approximate pattern matches

  • Biologists often usefast heuristic approaches (rather than local alignment) to find approximate matches


Heuristic similarity searches

Heuristic Similarity Searches

  • Genomes are huge: Smith-Waterman quadratic alignment algorithms are too slow

  • Alignment of two sequences usually has short identical or highly similar fragments

  • Many heuristic methods (i.e., FASTA) are based on the same idea of filtration

    • Find short exact matches, and use them as seeds for potential match extension


Query matching problem

Query Matching Problem

  • Goal: Find all substrings of the query that approximately match the text

  • Input: Query q = q1…qw,

    text t = t1…tm,

    n (length of matching substrings),

    k (maximum number of mismatches)

  • Output: All pairs of positions (i, j) such that the

    n-letter substring of q starting at iapproximately matches the

    n-letter substring of t starting at j,

    with at most k mismatches


Query matching main idea

Query Matching: Main Idea

  • Approximately matching strings share some perfectly matching substrings.

  • Instead of searching for approximately matching strings (difficult) search for perfectly matching substrings (easy).


Filtration in query matching

Filtration in Query Matching

  • We want all n-matches between a query and a text with up to k mismatches

  • “Filter” out positions we know do not match between text and query

  • Potential match detection: find all matches of l -tuples in query and text for some small l

  • Potential match verification: Verify each potential match by extending it to the left and right, until (k + 1) mismatches are found


Filtration match detection

Filtration: Match Detection

  • If x1…xn and y1…yn match with at most k mismatches, they must share an l -tuple that is perfectly matched, with l = n/(k + 1)

  • Break string of length n into k+1 parts, each each of length n/(k + 1)

    • k mismatches can affect at most k of these k+1 parts

    • At least one of these k+1 parts is perfectly matched


Filtration match verification

Filtration: Match Verification

  • For each l -match we find, try to extend the match further to see if it is substantial

Extend perfect match of lengthl

until we find an approximate match of length n with k mismatches

query

text


  • Login