Razvan C. Bunescu Raymond J. Mooney

1. Integrating Co-occurrence Statistics with IE for Robust Retrieval of Protein Interactions from Medline Razvan C. Bunescu Raymond J. Mooney Arun K. Ramani Edward M. Marcotte Institute for Cellular and Molecular Biology Center for Computational Biology and Bioinformatics University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin {razvan, mooney}@cs.utexas.edu {arun, marcotte}@icmb.utexas.edu

2. Introduction • Two orthogonal approaches to mining binary relations from a collection of documents: • Information Extraction: • Relation Extraction from individual sentences; • Aggregation of the results over the entire collection. • Co-occurrence Statistics: • Compute (co-)occurrence counts over the entire corpus; • Use statistical tests to detect whether co-occurrence is due to chance. • Aim: Combine the two approaches into an integrated extraction model.

3. Outline • Introduction. • Two approaches to relation extraction: • Information Extraction. • Co-occurrence Statistics. • Integrated Model. • Evaluation Corpus. • Experimental Results. • Future Work & Conclusion.

4. Information Extraction • Most IE systems detect relations only between entities mentioned in the same sentence. • The existence & type of the relationship is based on lexico-semantic cues inferred from the sentence context. • Given a pair of entities, corpus-level results are assembled by combining the confidence scores that the IE system associates with each occurrence. In synchronized human osteosarcoma cells, cyclin D1 is induced in early G1 and becomes associatedwithp9Ckshs1, a Cdk-binding subunit.

5. Relation Extraction using a Subsequence Kernel [Bunescu et al., 2005]. • Subsequences of words and POS tags are used as Implicit Features. • Assumes the entities have already been annotated. • Exponential penalty factor  is used to downweigh longer word gaps. • Generalization of the extraction system from [Blaschke et al., 2001]. • The system is trained to ouput a normalized confidence value for each extraction. interaction of(3)PROT(3)withPROT

6. Sentences Confidences Aggregation S1 Information Extraction S2 . . . . . . Sn Aggregating Corpus-Level Results

7. Aggregation Operators • Maximum • Noisy-OR • Average • AND

8. Outline • Introduction. • Two approaches to relation extraction: • Information Extraction. • Co-occurrence Statistics. • Integrated Model. • Evaluation Corpus. • Experimental Results. • Future Work & Conclusion.

9. Co-occurrence Statistics • Compute (co-)occurrence counts for the two entities in the entire corpus. • Based on these counts, detect if the co-occurrence of the two entities is due to chance, or to an underlying relationship. • Can use various statistical measures: • Pointwise Mutual Information (PMI) • Chi-square Test (2) • Log-Likelihood Ratio (LLR)

10. Pointwise Mutual Information • N : the total number of protein pairs co-occurring in the same sentence in the entire corpus. • P(p1,p2)  n12/N : the probability that p1 and p2 co-occur in the same sentence. • P(p1, p)  n1/N : the probability that p1 co-occurs with any protein in the same sentence. • P(p2, p)  n2/N : the probability that p2 co-occurs with any protein in the same sentence. • The higher the sPMI(p1, p2) value, the less likely it is that p1 and p2 co-occurred by chance => they may be interacting.

11. Outline • Introduction. • Two approaches to relation extraction: • Information Extraction. • Co-occurrence Statistics. • Integrated Model. • Evaluation Corpus. • Experimental Results. • Future Work & Conclusion.

12. Integrated Model • [Local] The sentence-level Relation Extraction (SSK) uses information that is local to one occurrence of a pair of entities (p1,p2). • [Global] The corpus-level Co-occurrence Statistics (PMI) are based on counting all occurrences of a pair of entities (p1,p2). • [Local & Global] Achieve a more reliable extraction performance by combining the two orthogonal approaches into an integrated model.

13. Integrated Model • Rewrite sPMI as: • Instead of counting 1 for each co-occurrence, use the confidence ouput by the IE system => a weighted PMI: • Can use any aggregation operator:

14. Outline • Introduction. • Two approaches to relation extraction: • Information Extraction. • Co-occurrence Statistics. • Integrated Model. • Evaluation Corpus. • Experimental Results. • Future Work & Conclusion.

15. Evaluation Corpus • An evaluation corpus needs to provide two types of information: • The complete list of interactions mentioned in the corpus. • Annotations of protein mentions, together with their gene identifiers. • The corpus was compiled based on the HPRD [www.hprd.org] and NCBI [www.ncbi.nih.gov] databases: • Every interaction is linked to a set of Medline articles that report the corresponding experiment. • An interaction is specified as a tuple containing: • The LocusLink (EntrezGene) identifiers of the proteins involved. • The PubMed identifiers of the corresponding Medline articles.

16. Evaluation Corpus (cont’ed) Interactions (XML) (HPRD) Participant Genes (XML) (NCBI) Medline Abstracts (XML) (NCBI)

17. Gene Name Annotation and Normalization • NCBI provides a comprehensive dictionary of human genes, where each gene specifies is specified by its unique identifier, and qualified with: • an official name, • a description, • a list of synonyms, • a list of protein names. • All these names (including the description) are considered as referring to the same entity. • Use a dictionary-based annotation, similar to [Cohen, 2005].

18. Gene Name Annotation and Normalization • Each name is reduced to a normal form, by: • Replacing dashes with spaces • Introducing spaces between letters and digits • Replacing Greek letters with their Latin counterparts • Substituting Roman numerals with Arabic numerals • Decapitalizing the first word (if capitalized). • The names are further tokenized, and checked against a dictionary of 100K English nouns. • Names associated with more than one gene identifier (i.e. ambiguous names) are ignored. • The final gene name dictionary is implemented as a trie-like structure.

19. Outline • Introduction. • Two approaches to relation extraction: • Information Extraction. • Co-occurrence Statistics. • Integrated Model. • Evaluation Corpus. • Experimental Results. • Future Work & Conclusion.

20. Experimental Results • Compare four methods on the task of interaction extraction: • Information Extraction: • [SSK.Max] Relation extraction with the subsequence kernel (SSK), followed by an aggregation of corpus-level results using Max. • Co-occurrence Statistics: • [PMI] Pointwise Mutual Information. • [HG] The HyperGeometric distribution method from [Ramani et al., 2005]. • Integrated Model: • [PMI.SSK.Max]The combined model of PMI & SSK. • Draw Precision vs. Recall graphs, by ranking the extractions and choosing only the top N interactions, while N is varying.

21. Experimental Results

22. Future Work • Derive an evaluation corpus from a potentially more accurate database (Reactome). • Investigate combining IE with other statistical tests (LLR, 2). • Design an IE method that is trained to do corpus-level extraction (as opposed to sentence-level extraction).

23. Conclusion • Introduced an integrated model that combines two orthogonal approaches to corpus-level relation extraction: • Information Extraction (SSK). • Co-occurrence Statistics (PMI). • Derived an evaluation corpus from the HPRD and NCBI databases. • Experimental results show a more consistent performance across the PR curve.

24. Aggregating Corpus-Level Results • Two entities p1 and p2 are mentioned in a corpus C of n sentences, C {S1, …, Sn}. • The IE system outputs a confidence value for each of the n occurrences: • The corpus-level confidence value is computed using an aggregation operator:

25. Experimental Results • Compare [PMI] and [HG] on the task of extracting interactions from the entire Medline. • Use the shared protein function benchmark from [Ramani et al. 2005]. • Calculate the extent to which interactions partners share functional annotations, as specified in the KEGG and GO databases. • Use a Log-Likelihood Ratio (LLR) scoring scheme to rank the interactions: • Also plot the scores associated with the HPRD, BIND and Reactome databases.

26. Experimental Results