Interaction Challenges for Agents with Common Sense

1. Interaction Challenges for Agents with Common Sense • Henry Lieberman • MIT Media Lab • Cambridge, Mass. USA • Http://www.media.mit.edu/~lieber

2. Agents with Common Sense • Some AI interfaces are now beginning to make use of large knowledge bases of Common Sense • Explicitly collected • Cyc, Open Mind, ThoughtTreasure • Distilled from other sources • Semantic Web, Wikipedia, Web resources • Web mining, other resource mining

3. Interaction Challenges • Finding opportunities for applying Common Sense in interfaces • Setting users' expectations • Making interfaces fail-soft • Taking advantage of user interaction

4. Interaction Challenges • Making better mistakes • Get lots of knowledge, but not too much • Common sense inference vs. mathematical inference • Debugging • Evaluating Common Sense interfaces

5. Interaction challenges for AI/Common Sense • Many interaction challenges for Common Sense interfaces are the same as for AI in general • But some are unique or critical for Common Sense… • Can't be sure what will be known • Reasonable, rather than right • Know a little about everything, not much about anything • Don't miss the obvious • Try not to make stupid mistakes

6. Open Mind Common Sense • Asks the Web community to contribute English sentences expressing Common Sense knowledge • "The Wikipedia version of Cyc" (#2 after Cyc) • Launched in 2001 by Push Singh • Now contains ~800 Kilofacts • Freely available / Open Source • Some versions in other languages/cultures • Brazilian Portuguese, Korean, Japanese, Chinese

7. Open Mind Common Sense • English sentences parsed by POS Tagger • Pattern-directed mining of 22 relations • (isA, PartOf, UsedFor…) • Strong focus on easy integration with applications • Semantic Net (ConceptNet: Liu, Eslick) • Natural Language toolkit (MontyLingua: Liu) • Tools for: Context, Analogy, Affect and more

8. What do we mean by “Common Sense”? • Simple statements about everyday life • Things fall down, not up • A wedding has a bride and a groom • You go to a restaurant to eat • And…the ability to use that knowledge when appropriate

9. Open Mind Common Sense- Push Singh & colleagues

10. ConceptNet - Liu, Singh, Eslick

11. Common Sense projects • Predictive typing • Speech recognition -- disambiguation & error correction • Storytelling with photo libraries • Searching social networks • Macro recording using Common Sense generalization • World construction for video games • Phrasebook for tourist information • Debugging problems in e-commerce interactions

12. Common Sense projects • Video editing based on story structures • Goal-oriented interfaces for consumer electronics • Mining Common Sense from the Web • Multi-lingual and multi-cultural Common Sense; translation • Games for acquiring, verifying and using Commonsense knowledge • Commonsense "Captchas" • Understanding imprecise qualities such as affect • ShapeNet and Expectation-driven Image Recognition • Understanding sensor data using Commonsense

13. Opportunities for using Common Sense • Find UI situations that are underconstrained • Ordinary system would either take no action or do something arbitrary • Then, give user some reasonable choices • Provide intelligent defaults • Make the most likely thing easiest to do

14. Opportunities for using Common Sense • Recognize users' likely goals • Help users map from goals to actions • Sanity checking • In the case of trouble, help users debug

15. Opportunities for using Common Sense • Find situations where every little bit helps • A little bit of knowledge is better than none • A little bit of knowledge about a lot of things can be more useful than a lot of knowledge about a few things

16. Setting users' expectations • Avoid direct question-answer interfaces • Right or wrong. Only one shot. • Better to cast system in role of advisor • Making suggestions, help • Adapting interface to most likely uses • Remove unnecessary steps in the interface • The user only expects intelligent behavior only once in a while

17. Take advantage of user interaction • Repurpose input that the user gives you for other reasons • Every time the user tells the interface something, they're telling you what their interests are -- learn from it

18. Make Common Sense interfaces "fail-soft" • There should no dire consequences of being wrong or not knowing what to do • Don't interfere if the user wants to use the application without interaction with the agent • If the relevant knowledge is missing, incomplete or wrong, the user is no worse off than without the agent

19. Make better mistakes • Common Sense approaches have the advantage that when they make mistakes, they tend to make plausible mistakes • Statistical approaches can make arbitrary mistakes • Better mistakes improve user trust in interfaces

20. Evaluation of Common Sense interfaces • Evaluation is tough because • Depends on what's in the knowledge base • Depends on limited-depth and other kinds of approximate inference • Standardized tasks don't test breadth of coverage • Try to relativize testing to coverage • Start with easier cases, then move to "typical" or "hard" cases

21. Intelligent User Interfaces 2008www.iuiconf.org • Location: Canary Islands, Spain • Dates: 13-16 January 2008 • Deadline: late September 2007