Provably Reliable QA Interfaces. http://cs.washington.edu/research/nli. Outline. Motivation Reliable NLIs Semantic tractability theory Implementation and experiments Joint work with Ana-Maria Popescu, Alex Yates, Henry Kautz, and Dan Weld. I. The Dominant UI Paradigm.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Provably Reliable QA Interfaces
Joint work with Ana-Maria Popescu, Alex Yates,
Henry Kautz, and Dan Weld.
Just Click it!
Speech + NL Understanding + Agents.
Substantial Research Challenges!
Reliable Natural Language Interfaces
Imagine “intelligent” interfaces that..
AI cannot be an excuse for incompetence
We have to think about tractable classes!
There are common situations where Semantic Interpretation is tractable.
Sentence Target expression
Easy to understand questions:
What are the Chinese restaurants in Seattle?
What Microsoft jobs require 2 years of experience?
What rivers run through Texas?
Semantically tractable questions
are quite common: 77.5% - 97%
Q: What is the second highest mountain in the US?
A: The word ‘second’ is unknown; please rephrase your query.
Q: What are the states bordering the states bordering the states bordering Montana?
Given a lexicon and a parse, IF
THENQ is Semantically Tractable.
Words (phrases) DB elements.
Precise = NLIDB implementation.
Will Precise capture “user intent?”
Match is computed via maxflow.
PRECISE graph representation
Movies = what
Actor = Woody Allen
Director = Woody Allen
Director = Paul Mazursky
“What are the Paul Mazursky films with Woody Allen?”
SQL Query + Answer Set
Mooney, PRECISE, EnglishQuery
Sets of NL questions labeled with SQL queries
Geography (846) Jobs (577) Restaurants (224)
Prevalence of semantically tractable questions
Recall = Qanswered/Q
Recall > 75 %
Precision = Qcorrect/Qanswered
PRECISE made no mistakes on semantically tractable questions