General Ontology Evaluation Framework (GOEF)

1. General Ontology Evaluation Framework (GOEF) By Amruta Akut Tetherless World Constellation April 20th, 2012

2. Agenda • About GOEF • Current Status • Current Challenges • Approach In Solving Problems: MIMV, BioPortal • Next Few Steps • Future Challenges • Conclusion • References

3. About GOEF • Portal which deals with determining the fitness of an ontology for a particular use case on a quantifiable basis • Input: Use case and ontology • Output: Quantitative measures (metrics-based) indicating fitness of an ontology for the use case • Ontology evaluation follows a 2-step process • Formalize use cases • Apply metrics • VSTO CEDAR use case is considered currently

4. Current Status • Completed basic HTML implementation of GOEF screens • Understood VSTO CEDAR use case • Working on formalizing the use case into F, S and C (F, C extracted) • Working on the development of a template which can be followed by users while entering information about their use case

5. Current Challenges • “Standard” is not explicit in the document. Then how do we determine the “standard” ? • This is the case specific to this use case. There can be other problems faced as we try to formalize other use cases

6. Approach in Solving Problems • Parsing the documents by the system to extract the F, S and C would not be feasible right now (would involve NLP) • So let us develop a use case template which the users can follow for entering information about their use case • Developing such a template will involve some minimum information which our application will need for ontology evaluation • How can the MIMV be useful?

7. Approach in Solving Problems • Secondly, Bio Portal (http://bioportal.bioontology.org/) has a recommendation system which recommends ontologies based on the free text which we enter. • Can we use this idea in our project? • Is the code for this thing made available by Bio Portal ? • If yes, then we can reuse this in a way such that • Free Text = Use Case Document (Will this be helpful?)

8. Next Few Steps • Complete development of a generic use case template or the minimum information model for ontology evaluation • Determine how this can be encoded in RDF • Determine how SADI service (DataFAQ’s) can be used in ontology evaluation • Determine how S2S can be used in faceted browsing of use cases and ontologies; think of UI redesign

9. Future Challenges • What details should the use case template capture? • How do we convert the minimum information captured in the use case template into RDF dynamically? • Against which ontology should this RDF be reasoned with? Do we need to design an ontology for this? • Determine the set of metrics to be applied to every level (F,S,C). • What is the minimum criteria required to know that the requirements for a particular level has been met?

10. Conclusion • GOEF provides a quantitative measure of the fitness of an ontology for a particular use case • Currently in the phase of formalizing use cases • Developing use case templates by following MIMV seems to be feasible to extract information about the use case right now

11. References • http://www.slideshare.net/joanneluciano/luciano-pr-08849ontologyevaluationmethodsmetrics • bioportal.bioontology.org • VSTO Cedar • dl.dropbox.com/u/2168216/MiMVocab-HCLS.pdf

12. Thank You

13. Questions