1 / 21

Natural Language Interfaces to Ontologies: usability and performance (Transfer report)

Natural Language Interfaces to Ontologies: usability and performance (Transfer report). Student: Danica Damljanovi ć Supervisor: Prof. Dr Hamish Cunningham. Introduction. Semantic Web Ontologies and knowledge bases Semantic search Semantic Web interfaces. Semantic Search Interfaces.

darryl
Download Presentation

Natural Language Interfaces to Ontologies: usability and performance (Transfer report)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Natural Language Interfaces to Ontologies: usabilityand performance(Transfer report) Student: Danica Damljanović Supervisor: Prof. Dr Hamish Cunningham Danica Damljanović

  2. Introduction • Semantic Web • Ontologies and knowledge bases • Semantic search • Semantic Web interfaces

  3. Semantic Search Interfaces • Protégé (Noy et al., 2001) • KIM (Kiryakov et al., 2004) • TAP (Guha et al., 2003) • GetData • Reflection • TAPSearch • SemSearch (Lei et al., 2006) • NLIs to KBs Danica Damljanović

  4. User preferences • (Kaufmann and Bernstein, 2007) • Natural Language Interfaces preferred to keywords, menu-guided, and graphical interfaces • (Linckels, 2007): • keywords preferred to NL interfaces Danica Damljanović

  5. NLIs to KBs: challenges • Robustness • Portability • What to show? • Understanding information need • Habitability • Usability Danica Damljanović

  6. Natural Language Interfaces • Usability • Effectiveness • Efficiency • User satisfaction • Who uses NLIs? • Application developers: customisation • End users: search Danica Damljanović

  7. Customisation and Retrieval Performance Danica Damljanović

  8. Ontology engineer Domain expert Customisation of NLIs to KBs Ontology editing (e.g. using Protege) WordNet … NLI for querying NLI for Ontology authoring Domain lexicon Domain knowledge

  9. End users point of view • Can the User and the System speakthe same language? Danica Damljanović

  10. Design recommendations • System Vocabulary>>User Vocabulary: • Feedback • Guided interfaces • Personalised vocabulary • User Vocabulary>> System Vocabulary: • Clarification dialogs • Query refinement • Controlling the relevance • Ranking suggestions • Defining similarity • User profiles Danica Damljanović

  11. First experiments: Question-based Interface to Ontologies Danica Damljanović

  12. NL --> SeRQL query Danica Damljanović

  13. An Example 1.15 1.19 compare

  14. Evaluation: Initial results We evaluated: • coverage and correctness • scalability and portability Danica Damljanović

  15. Comparison with Aqualog • We removed 6 questions that we knew were not supported by Aqualog • 1 conjunction query “What are the run parameters of POS Tagger and Sentence splitter?” • 1 query with brackets “Does GATE have a coreference resolution component (PR)?” • 1 query starting with “How many. . . ” • 3 queries not in a form of a full-blown question, for example “I cannot get Wordnet plugin to work“. Danica Damljanović

  16. Evaluation on scalability Danica Damljanović

  17. Expected contributions and research plan Addressing challenges without compromising retrieval performance: How existing NLIs to KBs can increase the performance without a significant additional cost for customisation? How methods for assisting the user in formulating the query can influence the retrieval performance? Richer navigation and semantic search interface which extend the concept of faceted search Improving QuestIO 9/15/2014 Danica Damljanović 24

  18. Expected contributions and research plan • Evaluation with golden standard dataset which would be one of the contributions of this PhD: • Knowledge base • Set of queries • Set of relevant SPARQL queries • Exploring methods for usability and their impact on retrieval performance

  19. Related publications Journals: • Danica Damljanovic, Kalina Bontcheva: Towards Enhanced Usability of Natural Language Interfaces. Annals of IS: Special Issue on “Semantic Web & Web 2.0”. Springer. 2009. (to appear) Workshops: • Damljanovic, D.,Bontcheva, K.. Enhanced Semantic Access to Software Artefacts. In Workshop on Semantic Web Enabled Software Engineering (SWESE) held in conjunction with ISWC'08, Karlsruhe, Germany, October 2008. Conferences: • Tablan, V., Damljanovic, D., Bontcheva, K.. A natural language query interface to structured information. In Proceedings of the 5h European Semantic Web Conference (ESWC 2008), Tenerife, Spain, June, 2008. • Damljanovic, D., Tablan, V., Bontcheva, K.: A text-based query interface to owl ontologies. In: 6th Language Resources and Evaluation Conference (LREC), Mar- rakech, Morocco, ELRA (May 2008). Posters: • Natural Language Queries for Enhanced Knowledge Access presented at Summer School on Multimedia Semantics Analysis, Annotation, Retrieval and Applications (SSMS07) , Glasgow, UK, July 15-21, 2007.

  20. Thanks • …you for your attention! • Hamish Cunningham and Kalina Bontcheva for continuous support • Valentin Tablan for the help with QuestIO • Vanesa Lopez from Open University for allowing me to play with AquaLog • Professor Abraham Bernstein and Esther Kaufmann from the University of Zurich for sharing with me the Mooney dataset in owl format. • Milan Agatonovic for continuous support

  21. References • (Noy et al., 2001)N. Noy, M. Sintek, S. Decker, M. Crubezy, R. Fergerson, and M. Musen. Creating Semantic Web Contents with Protege-2000. IEEE Intelligent Systems, 16(2):60-71, 2001. • (Kiryakov et al., 2004) A. Kiryakov, B. Popov, D. Ognyano, D. Manov, A. Kirilov, and M. Goranov. Semantic annotation, indexing and retrieval. Journal of Web Semantics, ISWC 2003 Special Issue, 1(2):671-680, 2004. • (Guha et al., 2003) R. Guha, R. McCool, and E. Miller. Semantic search. In WWW '03: Proceedings of the 12th international conference on World Wide Web, pages 700-709, New York, NY, USA, 2003. ACM. • (Lei et al., 2006) Y. Lei, V. Uren, and E. Motta. Semsearch: a search engine for the semantic web. In Managing Knowledge in a World of Networks, pages 238{245. Springer Berlin /Heidelberg, 2006. • (Kaufmann & Bernstein, 2007) E. Kaufmann and A. Bernstein. How useful are natural language interfaces to the semantic web for casual end-users? In Proceedings of the Forth European Semantic Web Conference (ESWC 2007), Innsbruck, Austria, June 2007. • (Serge Linckels, 2007) C. M. Serge Linckels. Semantic interpretation of natural language user input to improve search in multimedia knowledge base. it - Information Technologies, 49(1):40-48, 2007.

More Related