1 / 56

Pattern Recognition

C. E. N. T. E. R. F. O. R. I. N. T. E. G. R. A. T. I. V. E. B. I. O. I. N. F. O. R. M. A. T. I. C. S. V. U. Introduction to bioinformatics 2007 Lecture 4. Pattern Recognition. Patterns Some are easy some are not. Knitting patterns Cooking recipes

jurado
Download Presentation

Pattern Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. C E N T E R F O R I N T E G R A T I V E B I O I N F O R M A T I C S V U Introduction to bioinformatics 2007 Lecture 4 Pattern Recognition

  2. PatternsSome are easy some are not • Knitting patterns • Cooking recipes • Pictures (dot plots) • Colour patterns • Maps In 2D and 3D humans are hard to be beat by a computational pattern recognition technique, but humans are not so consistent

  3. Example of algorithm reuse: Data clustering • Many biological data analysis problems can be formulated as clustering problems • microarray gene expression data analysis • identification of regulatory binding sites (similarly, splice junction sites, translation start sites, ......) • (yeast) two-hybrid data analysis (experimental technique for inference of protein complexes) • phylogenetic tree clustering (for inference of horizontally transferred genes) • protein domain identification • identification of structural motifs • prediction reliability assessment of protein structures • NMR peak assignments • ......

  4. Data Clustering Problems • Clustering: partition a data set into clusters so thatdata points of the same cluster are “similar” and points of different clusters are “dissimilar” • Cluster identification-- identifying clusters with significantly different features than the background

  5. Application Examples • Regulatory binding site identification: CRP (CAP) binding site • Two hybrid data analysis • Gene expression data analysis These problems are all solvable by a clustering algorithm

  6. Multivariate statistics – Cluster analysis C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Raw table Any set of numbers per column • Multi-dimensional problems • Objects can be viewed as a cloud of points in a multidimensional space • Need ways to group the data

  7. Multivariate statistics – Cluster analysis C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Raw table Any set of numbers per column Similarity criterion Similarity matrix Scores 5×5 Cluster criterion Dendrogram

  8. Comparing sequences - Similarity Score - • Many properties can be used: • Nucleotide or amino acid composition • Isoelectric point • Molecular weight • Morphological characters • But: molecular evolution through sequence alignment

  9. Multivariate statistics – Cluster analysis Now for sequences 1 2 3 4 5 Multiple sequence alignment Similarity criterion Similarity matrix Scores 5×5 Cluster criterion Phylogenetic tree

  10. Human -KITVVGVGAVGMACAISILMKDLADELALVDVIEDKLKGEMMDLQHGSLFLRTPKIVSGKDYNVTANSKLVIITAGARQ Chicken -KISVVGVGAVGMACAISILMKDLADELTLVDVVEDKLKGEMMDLQHGSLFLKTPKITSGKDYSVTAHSKLVIVTAGARQ Dogfish –KITVVGVGAVGMACAISILMKDLADEVALVDVMEDKLKGEMMDLQHGSLFLHTAKIVSGKDYSVSAGSKLVVITAGARQ Lamprey SKVTIVGVGQVGMAAAISVLLRDLADELALVDVVEDRLKGEMMDLLHGSLFLKTAKIVADKDYSVTAGSRLVVVTAGARQ Barley TKISVIGAGNVGMAIAQTILTQNLADEIALVDALPDKLRGEALDLQHAAAFLPRVRI-SGTDAAVTKNSDLVIVTAGARQ Maizey casei -KVILVGDGAVGSSYAYAMVLQGIAQEIGIVDIFKDKTKGDAIDLSNALPFTSPKKIYSA-EYSDAKDADLVVITAGAPQ Bacillus TKVSVIGAGNVGMAIAQTILTRDLADEIALVDAVPDKLRGEMLDLQHAAAFLPRTRLVSGTDMSVTRGSDLVIVTAGARQ Lacto__ste -RVVVIGAGFVGASYVFALMNQGIADEIVLIDANESKAIGDAMDFNHGKVFAPKPVDIWHGDYDDCRDADLVVICAGANQ Lacto_plant QKVVLVGDGAVGSSYAFAMAQQGIAEEFVIVDVVKDRTKGDALDLEDAQAFTAPKKIYSG-EYSDCKDADLVVITAGAPQ Therma_mari MKIGIVGLGRVGSSTAFALLMKGFAREMVLIDVDKKRAEGDALDLIHGTPFTRRANIYAG-DYADLKGSDVVIVAAGVPQ Bifido -KLAVIGAGAVGSTLAFAAAQRGIAREIVLEDIAKERVEAEVLDMQHGSSFYPTVSIDGSDDPEICRDADMVVITAGPRQ Thermus_aqua MKVGIVGSGFVGSATAYALVLQGVAREVVLVDLDRKLAQAHAEDILHATPFAHPVWVRSGW-YEDLEGARVVIVAAGVAQ Mycoplasma -KIALIGAGNVGNSFLYAAMNQGLASEYGIIDINPDFADGNAFDFEDASASLPFPISVSRYEYKDLKDADFIVITAGRPQ Lactate dehydrogenase multiple alignment Distance Matrix 1 2 3 4 5 6 7 8 9 10 11 12 13 1 Human 0.000 0.112 0.128 0.202 0.378 0.346 0.530 0.551 0.512 0.524 0.528 0.635 0.637 2 Chicken 0.112 0.000 0.155 0.214 0.382 0.348 0.538 0.569 0.516 0.524 0.524 0.631 0.651 3 Dogfish 0.128 0.155 0.000 0.196 0.389 0.337 0.522 0.567 0.516 0.512 0.524 0.600 0.655 4 Lamprey 0.202 0.214 0.196 0.000 0.426 0.356 0.553 0.589 0.544 0.503 0.544 0.616 0.669 5 Barley 0.378 0.382 0.389 0.426 0.000 0.171 0.536 0.565 0.526 0.547 0.516 0.629 0.575 6 Maizey 0.346 0.348 0.337 0.356 0.171 0.000 0.557 0.563 0.538 0.555 0.518 0.643 0.587 7 Lacto_casei 0.530 0.538 0.522 0.553 0.536 0.557 0.000 0.518 0.208 0.445 0.561 0.526 0.501 8 Bacillus_stea 0.551 0.569 0.567 0.589 0.565 0.563 0.518 0.000 0.477 0.536 0.536 0.598 0.495 9 Lacto_plant 0.512 0.516 0.516 0.544 0.526 0.538 0.208 0.477 0.000 0.433 0.489 0.563 0.485 10 Therma_mari 0.524 0.524 0.512 0.503 0.547 0.555 0.445 0.536 0.433 0.000 0.532 0.405 0.598 11 Bifido 0.528 0.524 0.524 0.544 0.516 0.518 0.561 0.536 0.489 0.532 0.000 0.604 0.614 12 Thermus_aqua 0.635 0.631 0.600 0.616 0.629 0.643 0.526 0.598 0.563 0.405 0.604 0.000 0.641 13 Mycoplasma 0.637 0.651 0.655 0.669 0.575 0.587 0.501 0.495 0.485 0.598 0.614 0.641 0.000 How can you see that this is a distance matrix?

  11. Multivariate statistics – Cluster analysis C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Data table Similarity criterion Similarity matrix Scores 5×5 Cluster criterion Dendrogram/tree

  12. Multivariate statistics – Cluster analysisWhy do it? • Finding a true typology • Model fitting • Prediction based on groups • Hypothesis testing • Data exploration • Data reduction • Hypothesis generation But you can never prove a classification/typology!

  13. Cluster analysis – data normalisation/weighting C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Raw table Normalisation criterion C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Normalised table Column normalisation x/max Column range normalise (x-min)/(max-min)

  14. Cluster analysis – (dis)similarity matrix C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Raw table Similarity criterion Similarity matrix Scores 5×5 Di,j= (k | xik – xjk|r)1/r Minkowski metrics r = 2 Euclidean distance r = 1 City block distance

  15. (dis)similarity matrix Di,j= (k | xik – xjk|r)1/r Minkowski metrics r = 2 Euclidean distance r = 1 City block distance EXAMPLE: length height width Cow1 11 7 3 Cow 2 7 4 5 Euclidean dist. = sqrt(42 + 32 + 22) = sqrt(29) = 5.39 City Block dist. = |4+3+2| = 9

  16. Cluster analysis – Clustering criteria Similarity matrix Scores 5×5 Cluster criterion Dendrogram (tree) Single linkage - Nearest neighbour Complete linkage – Furthest neighbour Group averaging – UPGMA Neighbour joining – global measure, used to make a Phylogenetic Tree

  17. Cluster analysis – Clustering criteria • Start with N clusters of 1 object each • Apply clustering distance criterion iteratively until you have 1 cluster of N objects • Most interesting clustering somewhere in between distance Dendrogram (tree) 1 cluster N clusters

  18. Single linkage clustering (nearest neighbour) Char 2 Char 1

  19. Single linkage clustering (nearest neighbour) Char 2 Char 1

  20. Single linkage clustering (nearest neighbour) Char 2 Char 1

  21. Single linkage clustering (nearest neighbour) Char 2 Char 1

  22. Single linkage clustering (nearest neighbour) Char 2 Char 1

  23. Single linkage clustering (nearest neighbour) Char 2 Char 1 Distance from point to cluster is defined as the smallest distance between that point and any point in the cluster

  24. Single linkage clustering (nearest neighbour) Char 2 Char 1 Distance from point to cluster is defined as the smallest distance between that point and any point in the cluster

  25. Single linkage clustering (nearest neighbour) Char 2 Char 1 Distance from point to cluster is defined as the smallest distance between that point and any point in the cluster

  26. Single linkage clustering (nearest neighbour) Char 2 Char 1 Distance from point to cluster is defined as the smallest distance between that point and any point in the cluster

  27. Single linkage clustering (nearest neighbour) Let Ci andCj be two disjoint clusters: di,j = Min(dp,q), where p  Ci and q  Cj Single linkage dendrograms typically show chaining behaviour (i.e., all the time a single object is added to existing cluster)

  28. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  29. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  30. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  31. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  32. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  33. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  34. Complete linkage clustering (furthest neighbour) Char 2 Char 1

  35. Complete linkage clustering (furthest neighbour) Char 2 Char 1 Distance from point to cluster is defined as the largest distance between that point and any point in the cluster

  36. Complete linkage clustering (furthest neighbour) Char 2 Char 1 Distance from point to cluster is defined as the largest distance between that point and any point in the cluster

  37. Complete linkage clustering (furthest neighbour) Let Ci andCj be two disjoint clusters: di,j = Max(dp,q), where p  Ci and q  Cj More ‘structured’ clusters than with single linkage clustering

  38. Clustering algorithm • Initialise (dis)similarity matrix • Take two points with smallest distance as first cluster (later, points can be clusters) • Merge corresponding rows/columns in (dis)similarity matrix • Repeat steps 2. and 3. using appropriate cluster measure when you need to calculate new point-to-cluster or cluster-to-cluster distances until last two clusters are merged

  39. Average linkage clustering (Unweighted Pair Group Mean Averaging -UPGMA) Char 2 Char 1 Distance from cluster to cluster is defined as the average distance over all within-cluster distances

  40. UPGMA Let Ci andCj be two disjoint clusters: 1 di,j = ————————pqdp,q, where p  Ci and q  Cj |Ci| × |Cj| Ci Cj In words: calculate the average over all pairwise inter-cluster distances

  41. Multivariate statistics – Cluster analysis C1 C2 C3 C4 C5 C6 .. 1 2 3 4 5 Data table Similarity criterion Similarity matrix Scores 5×5 Cluster criterion Phylogenetic tree

  42. Multivariate statistics – Cluster analysis C1 C2 C3 C4 C5 C6 1 2 3 4 5 Similarity criterion Scores 6×6 Cluster criterion Scores 5×5 Cluster criterion Make two-way ordered table using dendrograms

  43. Multivariate statistics – Two-way cluster analysis C4 C3 C6 C1 C2 C5 1 4 2 5 3 Make two-way (rows, columns) ordered table using dendrograms; This shows ‘blocks’ of numbers that are similar

  44. Multivariate statistics – Two-way cluster analysis

  45. Multivariate statistics – Principal Component Analysis (PCA) C1 C2 C3 C4 C5 C6 Similarity Criterion: Correlations 1 2 3 4 5 Correlations 6×6 • Calculate eigenvectors with greatest eigenvalues: • Linear combinations • Orthogonal 1 Project data points onto new axes (eigenvectors) 2

  46. Multivariate statistics – Principal Component Analysis (PCA)

  47. Multidimensional Scaling • Multidimensional scaling (MDS) can be considered to be an alternative to factor analysis (PCA) • It starts using a set of distances (distance matrix) • MDS attempts to arrange "objects" in a space with a particular number of dimensions so as to reproduce the observed distances. As a result, we can "explain" the distances in terms ofunderlying dimensions

  48. Multidimensional Scaling Measures of goodness-of-fit: Stress Phi = [dij – f(ij)]2 • Phi is stress value, dijis reproduced distance, ij is observed distance, f(ij) is a monotone transformation of the observed distances (good function preserves rank order of distances after scaling)

  49. Multidimensional Scaling Different cell types are multi-dimensionally scaled. The colour codes indicate clear clustering.

More Related