Command and Natural Language

1. Command and Natural Language • Command and Natural Languages • Hieroglyphs (a 5000 year old technology) versus icons (the technology of the day) • Special languages • Mathematics • Music • Chemistry • Computers (interactive, user/programming communication)

2. Command and Natural Language • Command and Natural Languages • Goals of a language • Precision • Compactness • Ease of writing and reading • Completeness • Speed in learning • Simplicity in reducing errors • Ease of retention over time

3. Command and Natural Language • Command and Natural Languages • Tools that accelerated language development • Printing Press versus Computer • Print Press provided widespread dissemination of written work • Computer provides widespread dissemination of words via networks • Computers provide tools to manipulate language • Language is a tool to manipulate computers

4. Command and Natural Language • Command and Natural Languages • Computer Languages • Modest influences on spoken natural languages • Huge impact on the development of formal written languages • Early computers were designed to perform mathematical computations • Future is to operate on the real world • Directing robots • Controlling manufacturing • Guiding spacecraft

5. Command and Natural Language • Languages that tell computers what to do

6. Command and Natural Language • Command and Natural Languages • Scripting Languages • HyperCard • Revolution • Database Query Languages • Structured Query Languages (SQL) • The goal of the user is to create a result, not a program (3 to 20 lines) • Boolean operations (AND, OR, NOT) • Command Languages • Ephemeral • Produce an immediate result on an object • Example command line input to a browser: http://www.google.com/ • New command languages have declined as a result of direct manipulation • Typical format: Verb followed by a noun with qualifiers • PRINT MYFILE 3 COPIES • UPD CAD TN=555-555-5555 BILLTN=302-555-1234

7. Command and Natural Language • A UI Design Approach to Command Language Interfaces • Study the user’s task domain to determine the functionality required of the application • Derive a list of task actions and objects • Represent the items as the low-level interface syntax • Update Customer Record • Copy File_A File_B • Determine the frequency of specific tasks and facilitate the execution of those tasks • Determine whether destructive actions are reversible (e.g., delete) • Determine error messages • Transition diagram of how specific commands take the user to specific states

8. Command and Natural Language • Transition Diagrams iNUM oNewNum iCAD iCAD iCAR oNewCar iUPD oNewCAD iCADb iCADa iCPY oNewCPR Logon iCPRa iCPRb oNoCAD iDEL iCAD oNoCPR iCPR

9. Command and Natural Language • UPD CAD=555-555-5555 NUM=444-444-4444 NewNum=333-333-3333 iNUM oNewNum iCAD iCAD iCAR oNewCar iUPD oNewCAD iCADb iCADa iCPY oNewCPR Logon iCPRa iCPRb oNoCAD iDEL iCAD oNoCPR iCPR

10. Command and Natural Language • CPY CPR=555-555-5555 CPR=444-444-4444 iNUM oNewNum iCAD iCAD iCAR oNewCar iUPD oNewCAD iCADb iCADa iCPY oNewCPR Logon iCPRa iCPRb oNoCAD iDEL iCAD oNoCPR iCPR

11. Command and Natural Language • DEL CAD=555-555-5555 iNUM oNewNum iCAD iCAD iCAR oNewCar iUPD oNewCAD iCADb iCADa iCPY oNewCPR Logon iCPRa iCPRb oNoCAD iDEL iCAD oNoCPR iCPR

12. Command and Natural Language • Command Organization • Simplest – each command carries out a single task • Commands plus arguments • COPY FILE_A,FILE_B • DELETE FILE_A • PRINT FILE_A,FILE_B,FILE_C • Delimiters of blanks or commas • Keywords • COPY FROM=FILE_A TO=FILE_B • Options • PRINT/3 PRINTER_LOC_A FILE_A • Command to check seat availability on a flight on August 21 from Washington’s National Airport to LaGuardia Airport departing about 3:00PM • A0821DCALGA0300PM • Parsed: A 0821 DCA LGA 0300PM • Speed

13. Command and Natural Language • Command Organization • Tree Structure • 5x3x4 = 60 tasks based on 5 commands and 1 rule

14. Command and Natural Language • Command Organization • Tree Structure • 5x3x4 = 60 tasks based on 5 commands and 1 rule

15. Command and Natural Language • Command Organization • Tree Structure • 5x3x4 = 60 tasks based on 5 commands and 1 rule

16. Command and Natural Language • Command Organization • Tree Structure • 5x3x4 = 60 tasks based on 5 commands and 1 rule

17. Command and Natural Language • Hierarchical structure and congruence (Carroll 1982) Subjective ratings: 1=best, 5-worst Scores: Errors: Omissions:

18. Command and Natural Language • Naming and Abbreviations • Inconsistent abbreviation strategies (e.g., UNIX) • mkdir (make a directory) • cd (change directory) • ls (list directory) • pwd (print working directory) • Specific versus Generality (Black and Moran 1982) • Commands for inserting and deleting text • “Infrequent discriminating” command sets resulted in faster learning and superior recall

19. Command and Natural Language • Abbreviation strategies for commands • Simple truncation • Use the first, second, third letters of each command • Each command must be distinguishable by the leading string of characters • Can be the same length or variable lengths • Paste (Pa), Print (Pr) • Vowel drop (Pst, Prnt) • First and final letter (Pe, Pt) • First letter of each word in a phrase • Standard abbreviation from other contexts (Print – PRT) • Phonics (Execute – XQT)

20. Command and Natural Language • Abbreviation guidelines • Truncation is easy for users to comprehend • Fixed length is preferable to variable length • Do not incorporate endings (e.g., ING, ED, S) • Avoid the computer displaying abbreviations in messages • Command menus and keyboard shortcuts (single key press) H)elp O)ptions P)rint M)ain screen Q)uit

21. Command and Natural Language • Natural Language Interaction (NLI) • Users communicating in a familiar language (e.g., English) to a computer, and receiving response from the computer in the same language • Computers display information over 1000 times faster than people can enter commands • Better scenario is to have the computer enumerate options (display large amounts of information), and for the human to select items • This is partly the success of search engines • The success of NLI depends on the users knowledge about their task • E.g., Stock-market brokers who know their objects and actions by typing those terms as natural language • Compact command languages are more rapid and reliable • Potential application - Checkbook maintenance • Pay to University of Delaware on 4/18/06 $1525.00 • Show me all checks made out to the Deer Park • Which checks did I write this week • Direct manipulation interfaces have had more success in the market

22. Command and Natural Language • NLI Future Vision – 2001: A Space Odyssey • Astronauts set off in a spaceship controlled by HAL 9000, a revolutionary computer system that is every bit mankind's equal, and perhaps his superior. • HAL endangers the crew's lives for the sake of the mission, and eventually, kills almost all of the astronauts in the film. The astronauts have to overcome the computer. • HAL behaves in the most human fashion of all of the characters. He reaches human intelligence levels, and seems to have developed human traits of paranoia, jealousy and other emotions. By contrast, the human characters act like machines, cooly performing their tasks in a mechanical fashion, whether they be mundane tasks of operating their craft or even under extreme duress. • HAL, develops an acute emotional crisis because he can not accept evidence of his own fallibility.

23. Command and Natural Language • NLI Future Vision – 2001: A Space Odyssey • Check out this video to see the natural communication between the astronauts and the computer. • 2001 Space Odyssey Video • Ease of communication • Personal assessment of Dave’s art work • Determining what is a “personal” question • HALs superior statement of human error to computer error • Preparing the crew psychological report • HAL providing a hard copy of a report • Astronauts talking in private about whether to trust HAL • HAL reading lips

24. Command and Natural Language - AI • Global warming lead humans to near to extinction • Humans create robots known as mechas which can emulate thoughts and emotions. • David is designed to resemble a human child and to virtually feel love for its human owners. • Humans then become extinct • Humanoid’s use David’s memories, to reconstruct the Swinton home • The humanoids also create a clone Monica (his mother). She can only live for one day. • David spends the happiest day of his life playing with Monica and Teddy. • Monica tells David that she loves him as she drifts slowly away from the world.

25. Command and Natural Language • Command languages useful when • Frequent use of the system is anticipated • Users know the task and interface concepts • Screen space is at a premium • Numerous functions can be combined in a compact expression • Natural Language is currently applicable to situations where the user cannot actually touch the computer or device • Voice input • Synthesized voice output

26. Command and Natural Language • Natural Language Interaction (NLI) • Turing Test • “Computer program whose conversation is indistinguishable from a human’s” • ELIZA - emulates a Rogerian psychotherapist. • ELIZA has almost no intelligence whatsoever, only tricks like string substitution and canned responses based on keywords. The illusion of intelligence works best, however, if you limit your conversation to talking about yourself and your life. • http://www.manifestation.com/neurotoys/eliza.php3 • AskJeeves • http://www.ask.com/ • Invites natural language questions • Does keyword extraction • Returns web-page result sets

27. Command and Natural Language • Natural Language Interaction (NLI) vs. Direct Manipulation • Users seek predictable responses • Users get discouraged having to engage in clarification dialogs • Visually oriented interaction exploits • The computer’s capacity for rapid display of information • The human’s ability for rapid visual recognition

28. Command and Natural Language • More questions and answers • The role of facial expressions? • Context? • Personality? • The role of psychology in applying what is known about behavior to robots? • Education? • Living family trees? http://www.youtube.com/watch?v=pkpWCu1k0ZI http://www.youtube.com/watch?v=IhVu2hxm07E&feature=related

29. http://www.pbs.org/wgbh/nova/tech/social-robots.html

30. Command and Natural Language • Natural Language Queries (NLQ) • Natural Language front end for databases • The user types an English-like sentence • The result is a structured database search

31. Command and Natural Language • Natural Language Question Answering (NLQA) • “Who was the first Prime Minister of the European Union?” • Don’t attempt an exact answer • Users make incorrect assumptions • There is a president of the European Union • There is a European Commission • Better to return a list of results pages

32. Command and Natural Language • Text-database searching • Used by Lycos, Google, AltaVista • Stemming (attempt to reduce a word to its stem or root form) • Relevance ranking by word-frequency analysis • Filtering of common words (e.g., “the”, “of”, “in”) • Latent semantic indexing • Uses a term-document matrix which describes the occurrences of terms in documents • It is a sparse matrix whose rows correspond to documents and whose columns correspond to terms, typically stemmed words that appear in the documents. • A typical example of the weighting of the elements of the matrix:

33. Command and Natural Language • Natural Language Text Generation (NLTG) • The computer generates readable reports • “White-blood-cell count is 12,000” • “This value exceeds the normal range of 3,000 to 8,000 by 50%” • “Further examination for systemic infection is recommended” • Natural Language, Adventure Games and Instructional Systems • Example user commands • TAKE ALL OF THE KEYS • PICK UP THE SWORD • Instructional software • Guide students through the learning process • Algebra and Geometry • Feedback is provided in conversational language

34. Command and Natural Language • Applications in mobile devices – Voice Commands • Call • Recognizes verb • Accepts name or number as input • Requests clarification by providing a list of names • Confirms dialing • Useful for dialing while driving • Send Text Message • Recognizes verb • Accepts name or number as input • Opens the text composer

35. Command and Natural Language • Applications in mobile devices – Voice Read Out • Good quality now for voice read out

36. Command and Natural Language • Facebook Facial Recognition • Security • Or, hold up a photo • Mood Recognition • The API will estimate your mood automatically, with 5 values: Happy, Sad, Surprised, Angry, and Neutral.