2 nd International Conference on Biomedical Ontology (ICBO’11)

1. 2nd International Conference on Biomedical Ontology (ICBO’11) Ontology-Based Analysis of Event-Related Potentials Gwen Frishkoff12, Robert Frank2, Paea LePendu3, & Snežana Nikolič1 1 Psychology & Neuroscience, Georgia State University 2 NeuroInformatics Center (NCBO), University of Oregon 3 National Center for Biomedical Ontology (NCBO), Stanford University http://nemo.nic.uoregon.edu

2. Overview • Why ontology-based analysis? • Linking Data to Knowledge in Human Neuroscience • Ontology-based analysis of ERP data • Data  Information • Pipeline for automated(and therefore objective)separation of ERP patterns andextraction of summary metrics for each pattern • Information  Knowledge • Ontology to represent metrics in semantically structured way so as to automaticallyclassify & label ERP patterns within and across experiments

3. Overview • Why ontology-based analysis? • Linking Data to Knowledge in Human Neuroscience • Ontology-based analysis of ERP data • Data  Information • Pipeline for automated(and therefore objective)separation of brainwave (ERP) patterns and automatedextraction of summary metrics, which are output to RDF • Information  Knowledge • Ontology to represent data (in RDF) and automatically(and therefore objectively)classify & label ERP patterns within and across experiments

4. “The plural of ‘anecdote’ is not ‘data’.” — Roger Brinner (Economist) Assertion #1: In a scientific domain, the priority should be to capture and track assertions about data. Corollary: To capture complex (and presently ill-defined) patterns in data, we need bottom-up (data-driven) analysis.

5. The plural of ‘data’ is not ‘knowledge’. Assertion #2: To draw meaningful inferences from data, they must be linked to a well-structured knowledge base (ontology).

6. ONTOLOGY Knowledge engineering INFORMATION Ontology mining? Data mining (i.e., analysis) DATA

7. The plural of ‘data’ is not ‘knowledge’. Assertion #3: Ontology  Semantic Structure. It cannot be automatically extracted from data (or patterns in data). Cf. Searle’s Chinese Room argument… Corollary: To build a valid ontology, we need top-down (knowledge-driven) methods (ala BFO/OBO).

8. Introduction to ERP Domain (I):The Data = Measurements of Scalp EEG EEGs (“brainwaves” or flunctuations in brain electrical potentials) are recorded by placing two or more electrodes on the scalp surface. ~5,000 ms 256-channel Geodesic Sensor Net

9. Introduction to ERP Domain (II):From EEG to Event-Related Potentials (ERP) EEG AVERAGE OVER (LOTS OF) EEG SEGMENTS ERP • ERPs (event-related potentials) are the result of averaging across multiple segments of EEG, time-locking to an event of interest.

10. Introduction to ERP Domain (III): Entities of Interest = ERP Patterns (in Data!) 120 ms ERP patterns characterized by three types of attributes: (1) TIME  latency of peak positive or peak negative potential (left) (2) SPACE  scalp topography of this potential (right); and (3) FUNCTION  experimental context in which these patterns are characteristically observed (e.g., presentation of visual stimulus)

11. What’s great about ERPs … • Tried and true method for noninvasive brain functional mapping • Direct measure neuronal activity • Whole-brain measurement (at scalp) • Millisecond temporal resolution • Portable and inexpensive • Important clinical applications (e.g., potential biomarkers for AD, presurgical planning) • Recent innovations give new windows into rich, multi-dimensional patterns • Rich spatial info (high-density EEG) • Combined temporal & spectral info (JTF) • Multimodal (EEG/ fMRI/MEG) measures 1 sec

12. If ERPs are so great…. Why are there so few meaningful applications in biomedicine? And why so few (arguably no) cross-lab meta-analyses?

13. Problem #1: Patterns superposed in space & time MEASURED DATA (THIS IS WHAT WE ACTUALLY MEASURE/OBSERVE!) LATENT (INFERRED) PATTERNS (THIS IS WHAT WE WANT TO TALK ABOUT) Superposition

14. Problem #2a: Patterns (actually, pattern labels) are like toothbrushes… Everyone has one, and nobody likes to use anyone else’s. Prosody-specific negativity Meaningfulness recognition potential N400 Effect N300 Phonological mapping negativity Medial frontal negativity fN400 old-new effect

15. Problem #2b: Conversely, different scientists use the same label for incommensurable patterns. 450 ms Consider a Hypothetical Database Query: Show me all the N400 patterns in the database. Peak latency 410 ms Will the “real” N400 please step forward? 330 ms “CANONICAL N400” 410 ms

16. Putative “N400”-labeled patterns Parietal N400 ≠ fN400 ≠ Assertion #3: We cannot ground ERP meta-analysis in prior literature (e.g., text mining). We need a reliable workflow for data analysis & classification. Parietal P600

17. Summary: Motivation for NEMO • Lots of different — and equally valid! — methods for pattern analysis • Inconsistent and subjective use of metrics and labels for pattern summary and classification • No existing methods or tools to support ERP data sharing and integration Assertion #4:The best way to address these issues is to combine data-driven methods for pattern analysis with knowledge-driven methods for ontology development and application (to interpret analysis results)

18. Neural ElectroMagnetic Ontologies • A set of formal (OWL) ontologiesfor representation of ERP domain concepts • A suite of tools for data-driven extraction and ontology-based annotation of ERP patterns • A database that includes publicly available, annotated data from our NEMO ERP consortium to demonstrate application of ontology for quantitative meta-analysis of results from studies of language and cognition

19. Overview • Why ontology-based analysis? • Linking Data to Knowledge in Human Neuroscience • Ontology-based analysis of ERP data • Data  Information • Pipeline for automated(and therefore objective)separation of ERP patterns andextraction of summary metrics for each pattern • Information  Knowledge • Ontology to represent data (in RDF) and automatically(and therefore objectively)classify & label ERP patterns within and across experiments

20. Extraction of meaningful patterns (i.e., data analylsis) FROM DATA TO INFORMATION….

21. ERP Pattern Analysis: Current Practice “Bumpology” P3 component N400 component Bumpology^2?

22. NEMO Ontology-based Analysis: Overview • ERP Pattern Extraction • ERP Metric Extraction • RDF Generation (Data Annotation) • (Metadata Entry) • ERP Pattern Classification

24. 1. NEMO Pattern Extraction • NEMO ERP Pattern Extraction Toolkit • http://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release • NEMO_ERP_Pattern_Decomposition/ • NEMO_ERP_Pattern_Segmentation/

25. Pattern Extraction I: Decomposition • Advantages: • Data-driven • Automated/ Objective • Sensitive (able to separate superposed patterns) 100ms P100 170ms N100 200ms • Disdvantages: • Requires expertise (~vanilla PCA) • Not used by majority of ERP researchers fP2 P1r/ N3 280ms 400ms P1r/ MFN • NEMO ERP Pattern Extraction Toolkit • http://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release • NEMO_ERP_Pattern_Decomposition/

26. Pattern Extraction II: Segmentation • NEMO ERP Pattern Extraction Toolkit • http://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release • NEMO_ERP_Pattern_Segmentation/

28. 2. Metric Extraction • NEMO ERP Metric Extraction Toolkit • http://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release • NEMO_ERP_Metric_Extraction/

29. Typical semi-structured representation of ERP data ERP pattern (extracted from “raw” ERP data using PCA/ICA etc.) Peak latency measurement (in ms)

30. Overview • Why ontology-based analysis? • Linking Data to Knowledge in Human Neuroscience • Ontology-based analysis of ERP data • Data  Information • Pipeline for automated(and therefore objective)separation of brainwave (ERP) patterns and automatedextraction of summary metrics, which are output to RDF • Information  Knowledge • Ontology to represent metrics in semantically structured way so as to automaticallyclassify & label ERP patterns within and across experiments

31. FROM INFORMATION TO KNOWLEDGE…. ONTOLOGY

32. NEMO Ontology-based Analysis: Overview • ERP Pattern Extraction • ERP Metric Extraction • RDF Generation (Data Annotation) • (Metadata Entry) • ERP Pattern Classification

33. Recall: Entities of interest (at Stage 1) = Patterns in Data SPACE TIME 1 sec FUNCTION  Modulation of pattern features (time, space, amplitude) in different experiment conditions

34. NEMO Ontology (in a nutshell) L3: Brain Physiological data (OBI/IAO) L1: Brain Physiological processes (BFO/OPB)

36. 3. RDF Generation # OWL Ontology Declaration / Import: GAF-LP1_NN_ERP_data <http://purl.bioontology.org/NEMO/data/GAF-LP1_NN_ERP_data> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/2002/07/owl#Ontology>. <http://purl.bioontology.org/NEMO/data/GAF-LP1_NN_ERP_data> <http://www.w3.org/2002/07/owl#imports> <http://purl.bioontology.org/NEMO/ontology/NEMO.owl>. # Instance Declaration 000: GAF-LP1_NN_ERP_data <http://purl.bioontology.org/NEMO/data/GAF-LP1_NN_ERP_data> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.bioontology.org/NEMO/ontology/NEMO.owl#NEMO_0000495>. • NEMO ERP Metric Extraction Toolkit • http://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release • NEMO_ERP_Metric_Extraction/

37. Data annotation using RDF “Triples” In RDF form: <002> <type> <NEMO_0745000> Subject – Predicate – Object In natural language = The data represented in cell Z (row A, column 1) is an instance of (“is a”) a peak latency temporal measurement (i.e., the time at which the pattern is of maximal amplitude) Note that the predicate links an instance to a class within NEMO ontology.

38. GOAL: Represent extracted information with rich, formal semantics that allow us to reason over data (both within and across datasets) RDF Graph (“triples”)

39. ERP PATTERN CLASSIFICATION

40. 5. Pattern Classification (I) (1) Temporal Criterion (2) Spatial Criterion (3) Functional Criterion

41. 5. Pattern Classification (II) RDF Data is opened in Protégé ontology editing software RDF Data loads NEMO ontology

42. 5. Pattern Classification (III) HermiT Reasoner is used to generate inferences

43. 5. Pattern Classification (IV) Instance-level information (i.e., ERP pattern instances) are successfully classified!

44. Take-home messages • For some biomedical applications it may important to capture L3 (DATA) as well as L1 (REALITY) explicitly, i.e., within the ontology • In linking the data to the ontology (e.g., for classification/labeling of patterns), it may be important consider data-driven methods for pattern analysis and metric extraction • An advantage of this approach is that we can generate relatively stable (non-controversial) representation of data (RDF artifacts), which we will archive and maintain — separate from, but linked to, the ontology — even as the ontology is uncertain & changing. • Further, robust representation of data across studies provides basis for valid quantitative meta-analysis, which provide high-quality evidence to inform pattern rules in the ontology

45. Ongoing Work & Open Issues • Evolving pattern rules to represent more complex functional criteria (i.e., expt metadata) • Temporal reasoning (can we squeeze this into DL/OWL?) • Representing uncertainty in pattern rules & classification of pattern instances (beyond Evidence Codes?) • Clinical applications: Pilot cross-lab work with aphasics (stroke & TBI patients with language disorders)

46. Acknowledgments NEMO Ontology Task Force Robert M. Frank (NIC) Dejing Dou (CIS) Paea LePendu (CIS) Haishan Liu (CIS) Allen Malony (NIC, CIS) Jason Sydes (CIS) *Snezana Nikolic (PSY, GSU) *emeritus NEMO EEG/MEG Data Consortium Tim Curran (U. Colorado) Dennis Molfese (U. Louisville) John Connolly (McMaster U.) Kerry Kilborn (Glasgow U.) Charles Perfetti (U. Pittsburgh) YOU (BIO-ONTOLOGY COMMUNITY) Special thanks to: Maryann Martone & associates (NIF) Jessica Turner (cogPO) Angela Laird (BrainMap) Sivaram Arabandi (OGMS) Funding from the National Institutes of Health (NIBIB), R01-MH084812 (Dou, Frishkoff, Malony) www.nemo.nic.uoregon.edu

47. Recent References • Frishkoff, G., Frank, R., LePendu, P., & Nikolic, S. (2011, in press). Ontology-based Analysis of Event-Related Potentials. Proceedings of the International Conference on Biomedical Ontology (ICBO'11). • Frishkoff, G., Frank, R., Sydes, J., Mueller, K., & Malony, A. (2011, subm). Minimal Information for Neural Electromagnetic Ontologies (MI-NEMO): A standards-compliant workflow for analysis and integration of human EEG. Standards in Genomic Sciences (SIGS). • Liu, H., Frishkoff, G., Frank, R. M. F., & Dou, D. (2011, subm). Integration of Human Brain Data: Metric and Pattern Matching across Heterogeneous ERP Datasets. Journal of Neurocomputing. • Frank, D. & Frishkoff, G. A. (2011, in prep.). The NEMO ERP Analysis Toolkit: Combining data-driven and knowledge-driven methods for ERP pattern analysis. Neuroinformatics. • Frishkoff, G.A., Dou, D., Frank, R., LePendu, P., and Liu, H. (2009). Development of Neural Electromagnetic Ontologies (NEMO): Representation and integration of event-related brain potentials. Proceedings of the International Conference on Biomedical Ontologies (ICBO09). July 24-26, 2009. Buffalo, NY.