Biomedical articles per year - PowerPoint PPT Presentation

slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Biomedical articles per year PowerPoint Presentation
Download Presentation
Biomedical articles per year

play fullscreen
1 / 43
Biomedical articles per year
Download Presentation
Download Presentation

Biomedical articles per year

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Biomedical articles per year

  2. Questions of biomedical experts Yes/No question “Are there any DNMT3 proteins present in plants?” Exact Answer “Yes” Ideal Answer • “Yes. The plant DOMAINS REARRANGED METHYLTRANSFERASE2 (DRM2) is a homolog of the mammalian de novo methyltransferase DNMT3. DRM2 contains a novel arrangement of the motifs required for DNA methyltransferase catalytic activity.”

  3. Questions of biomedical experts Factoid question “What is the methyl donor of DNA (cytosine-5)-methyltransferases?” Exact Answer “S-adenosyl-L-methionine” Ideal Answer • “S-adenosyl-L-methionine (AdoMet, SAM) is the methyl donor of DNA (cytosine-5)-methyltransferases. DNA (cytosine-5)-methyltransferasescatalyze the transfer of a methyl group from S-adenosyl-L-methionine to the C-5 position of cytosine residues in DNA.”

  4. Questions of biomedical experts (III) List question “Which species may be used for the biotechnological production of itaconic acid?” Exact Answer “Aspergillusterreus”, “Aspergillusniger”, “Ustilagomaydis” Ideal Answer • “In 1955, the production of itaconic acid was firstly described for Ustilagomaydis. Some Aspergillus species, like A. itaconicus and A. terreus, show the ability to synthesize this organic acid and A. terreus can secrete significant amounts to the media. Itaconic acid is mainly supplied by biotechnological processes with the fungus Aspergillusterreus. Cloning of the cadA gene into the citric acid producing fungus A. niger showed that it is possible to produce itaconic acid also in a different host organism.”

  5. Questions of biomedical experts (III) Summary question “How do histone methyltransferases cause histone modification?” Exact Answer - Ideal Answer • “Histone methyltransferases (HMTs) are responsible for the site-specific addition of covalent modifications on the histone tails, which serve as markers for the recruitment of chromatin organization complexes. There are two major types of HMTs: histone-lysine N-Methyltransferases and histone-arginine N-methyltransferases. The former methylate specific lysine (K) residues such as 4, 9, 27, 36, and 79 on histone H3 and residue 20 on histone H4. The latter methylate arginine (R) residues such as 2, 8, 17, and 26 on histone H3 and residue 3 on histone H4. Depending on what residue is modified and the degree of methylation (mono-, di- and tri-methylation), lysine methylation of histones is linked to either transcriptionally active or silent chromatin.”

  6. Finding relevant snippets

  7. Not only texts: ontologies, linked data, …

  8. Information from structured data List question “Which forms of cancer is the Tpl2 gene associated with?” Related RDF triple • Subject: (lung cancer) • Predicate: • Object:" Related concepts (cancer) (TPL2 synonym)

  9. BioASQ Vision • Make sure this knowledge is used to the benefit of patients • Need to make it accessible to biomedical experts • Search is not effective enough • Push research in automated answering of questions • A challenge for such systems can achieve a multiplying effect

  10. What is BioASQ? A challenge funded by the European Union (FP7). • Task a: Hierarchical text classification • Organizers distribute newunclassifiedPubMed articles. • Participants assign MeSH termsto the articles. • Evaluation based on annotations of PubMedcurators. • Task b: IR, QA, summarization, … • Organizers distribute English biomedical questions. • Participants provide: relevant articles, snippets, concepts, triples, “exact” answers, “ideal” answers. • Evaluation: both automatic (GMAP, MRR, ROUGE etc.) and manual (by biomedical experts).

  11. The challenge Taska Task b

  12. Behind the scenes

  13. BioASQ Platform

  14. Datasets Task b data contain gold articles, snippets, concepts, triples, “exact” and “ideal” answers prepared by biomedical experts from around Europe.

  15. Data sources Theyinclude both text and structured info. • PubMed abstracts, PubMed Central articles,MeSH. • Gene Ontology, UniProt, Jochem, Disease Ontology.

  16. Annotation: questions and queries

  17. Annotation: snippets

  18. Annotation: answers

  19. Assessment: relevance of material

  20. Assessment: information in answers

  21. BioASQ social network

  22. Oracle

  23. Oracle

  24. Two cycles 2013 Schedule The official challenge is over, but… • Task acontinues to run each week . • An oracle for task b will be available soon. • Oracles will remain available. • Third cycle is being designed … 2014 Schedule March 2013 June 2013 August 2013 September 2013 February 2014 March 2014 May 2014 September 2014

  25. Challenge participants so far

  26. Challenge participants in each cycle

  27. Evaluation measures Task a: Hierarchical text classification Flat measures for multi-label classification: Accuracy, MiF, MaF, EBF Hierarchical measures: LCA-F (new), HF • Task b: IR, QA, summarization, … • Phase A: • standard IR measures, mean precision, mean recall, mean F-measure, MAP (used for winners selection), G-MAP • Phase B: • ‘Exact answers’ (based on type): accuracy (yes/no), strict/lenient accuracy, MRR (factoid), mean F-measure (list) • ‘Ideal answers’: manual scores from the experts {Readability, Repetition, Information Precision and Recall}, plus ROUGE

  28. First year technology/results overview • Task 1a • Mainly SVMs and learning-to-rank. • Mostly flat classification, ignoring class taxonomy. • Mediocre results by hierarchical methods. • One of the systems outperformed NLM’s system. • Task 1b • Phase A (retrieve relevant documents, concepts, snippets, triples): low performance (compared to baselines). • Phase B (formulate ‘exact’ and ‘ideal’ answers): poor performance for ‘exact’ answers (except for yes/no questions); high performance for ‘ideal’ answers (paragraph-sized summaries), but starting with gold documents, snippets etc. • Large scope for improvements, esp. in Task 1b.

  29. “Exact” answer results (batch 2/3)

  30. “Ideal” answer results (batch 2/3)

  31. Results – task a – flat measures

  32. Results – task a – hierarchical

  33. First challenge prizes

  34. Sustainability Making the challenge viable, at very low cost, after the end of the project • BioASQ Oracle • Software release and installation instructions • Benchmark datasets • BioASQ social network • Involvement of the biomedical community in the process • Attracting sponsors for prizes

  35. Project Consortium • National Centre for Scientific Research “Demokritos” -NSCR “D” (EL) • Transinsight GmbH – TI (D) • Universite Joseph Fourier- UJF (F) • University Leipzig - ULEI (D) • Universite Pierre et Marie Curie Paris 6 – UPMC (F) • Athens University of Economics and Business – Research Centre – AUEB-RC (EL)

  36. Project Consortium

  37. Get in touch! • BioASQ workshop @CLEF (Sheffield, Sept 14) • Visit www.bioasq.orgFollow@BioASQ

  38. Useful Links • BioASQ Annotation & assessment tools: • • • • BioASQ social network: • • • BioASQ platform: • • BioASQ Oracles: • • A. Kosmopoulos, I. Partalas, E. Gaussier, G. Paliouras, I. Androutsopoulos, Evaluation Measures for Hierarchical Classification: a unified view and novel approaches. Data Mining and Knowledge Discovery (To appear)