1 / 29

Natural Language Interfaces to Ontologies LarKc PhD symphosium , Beijing, 14 November 2010

Natural Language Interfaces to Ontologies LarKc PhD symphosium , Beijing, 14 November 2010. Danica Damljanovi ć University of Sheffield danica@dcs.shef.ac.uk. What are Natural Language Interfaces to Ontologies ?. Ontology engineer. Domain expert. Customisation.

nerice
Download Presentation

Natural Language Interfaces to Ontologies LarKc PhD symphosium , Beijing, 14 November 2010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Natural Language Interfaces to OntologiesLarKc PhD symphosium, Beijing, 14 November 2010 DanicaDamljanović University of Sheffield danica@dcs.shef.ac.uk

  2. What are Natural Language Interfaces to Ontologies?

  3. Ontology engineer Domain expert Customisation Ontology editing (e.g. using Protege) WordNet … NLI for querying NLI for Ontology authoring Domain lexicon Domain knowledge

  4. The Objective • Increase usability of Natural Language Interfaces to ontologies • For end users: increase precision and recall • For application developers: decrease the time for customisation

  5. Previous Work: QuestIO 1.15 1.19 compare

  6. But... • Ontologies are not perfect: • ontology lexicalisations often missing or too many • ranking based on ontology structure might be misleading • Encouraging users to use keywords might be misleading • User evaluation: • defined tasks: user satisfaction reaching 90% • undefined tasks: user satisfaction low (~44%)

  7. FREyA - Feedback, Refinement, Extended VocabularyAggregator • Feedback: showing the user system interpretation of the query • Refinement: • resolving ambiguity: generating dialog whenever one term refers to more than one concept in the ontology (precision) • Extended Vocabulary: • expressiveness: generating dialog whenever an “unknown” term appears in the question (recall) • portability: no need for customisation from application developers • The dialog: • generated by combining the syntactic parsing and ontology-based lookup • learns from the user’s selections

  8. answer FREyA Workflow • SPARQL • answer Answer Type • Potential Ontology Concept (POC) • Ontology Concept (OC) • triples • NL query • OCs • POCs Indentify the Answer Type learn

  9. Find Potential Ontology Concepts CNL 2010, Marettimo, Sicily

  10. Finding Ontology Concepts

  11. Mapping POC to OCs: Ambiguities POC POC population geo:State geo:State new york geo:City geo:cityPopulation

  12. New York is a city

  13. New York is a state

  14. Ambiguous Lexicon IF THEN

  15. The User Controls the Output POC min geo:loElevation point POC geo:isLowestPointOf geo:LoPoint POC max state geo:stateArea area geo:State

  16. What is the lowest point of the state with the largest area? TRIPLES: ?firstJoker – geo:isLowestPointOf – geo:State geo:State – (max) geo:stateArea - ?lastJoker SPARQL: prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> prefix xsd: <http://www.w3.org/2001/XMLSchema#> select ?firstJoker ?p0 ?c1 ?p2 ?lastJoker where { { { ?c1 ?p0 ?firstJoker} UNION { ?firstJoker ?p0 ?c1} . filter (?p0=<http://www.mooney.net/geo#isLowestPointOf>) . } ?c1 rdf:type <http://www.mooney.net/geo#State> . ?c1 ?p2 ?lastJoker . filter (?p2=<http://www.mooney.net/geo#stateArea>) . } ORDER BY DESC(xsd:double(?lastJoker))

  17. What is the lowest point of the state with the largest area? the answer for both is Death Valley TRIPLES: ?firstJoker – (min) geo:loElevation – geo:LoPoint geo:LoPoint - ?joker3 – geo:State geo:State – (max) geo:stateArea - ?lastJoker SPARQL: prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> prefix xsd: <http://www.w3.org/2001/XMLSchema#> select ?firstJoker ?p0 ?c1 ?joker3 ?c2 ?p3 ?lastJoker where { ?c1 ?p0 ?firstJoker . filter (?p0=<http://www.moony.net/geo#loElevation>) . ?c1 rdf:type <http://www.mooney.net/geo#LoPoint> . {{ ?c2 ?joker3 ?c1 } UNION { ?c1 ?joker3 ?c2 }} ?c2 rdf:type <http://www.mooney.net/geo#State> . ?c2 ?p3 ?lastJoker . filter (?p3=<http://www.mooney.net/geo#stateArea>) . } ORDER BY ASC(xsd:double(?firstJoker)) DESC(xsd:double(?lastJoker))

  18. New Lexicon IF THEN

  19. Learning

  20. FREyA: a Natural Language Interface to Ontologies • http://gate.ac.uk/freya ESWC 2010

  21. Evaluation: correctness • Mooney GeoQuery dataset, 250 questions • 34 no dialog, 14 failed to be answered • Precision=recall=94.4%

  22. Evaluation: Learning • 10-fold cross-validation , 202 Mooney GeoQuery questions that could be correctly mapped into SPARQL and required dialog, from 0.25 to 0.48 • Errors: ambiguity and sparseness

  23. Evaluation: Ranking • Mean Reciprocal Rank: 0.76 (default ranking based on string similarity and synonym detection)

  24. Learning the Correct Ranking • Randomly selected 103 dialogs from 202 questions (343 dialogs) • MRR increased for 6% from 0.72 to 0.78

  25. Evaluation: Answer Type

  26. Evaluation: Customisation • Small empirical evaluation with 1 subject who is not familiar with ontologiesand NLP • No training, short introduction into the domain • 17 questions asked in total; 3 were cancelled by the user during one of the dialogs • 78.57% correctly answered • 21.43% failed or incorrectly answered

  27. Conclusion • Combining syntactic parsing with ontology-based lookup through user interaction can increase the precision and recall of NLIs to ontologies, • while reducing the time for customisation by shifting it from application developers to end users.

  28. Next steps • Improvement of the learning model to avoid errors due to ambiguities • point> geo:HiPoint or geo:LoPoint • Using lexicon to improve other systems

  29. More information... • D. Damljanovic, M. Agatonovic, H. Cunningham: Natural Language Interfaces to Ontologies: Combining Syntactic Analysis and Ontology-based Lookup through the User Interaction. In Proceedings of the 7th Extended Semantic Web Conference (ESWC 2010), Springer Verlag, Heraklion, Greece, May 31-June 3, 2010. PDF • D. Damljanovic, M. Agatonovic, H. Cunningham: Identification of the Question Focus: Combining Syntactic Analysis and Ontology-based Lookup through the User Interaction. In Proceedings of the 7th Language Resources and Evaluation Conference (LREC 2010), ELRA 2010, La Valletta, Malta, May 17-23, 2010. PDFD. Damljanovic. Towards portable controlled natural languages for querying ontologies. In Rosner, M., Fuchs, N., eds.: Proceedings of the 2nd Workshop on Controlled Natural Language. Lecture Notes in Computer Science. Springer Berlin/Heidelberg, Marettimo Island, Sicily (September 2010)

More Related