People & Speech Interfaces CS 260 Wednesday, October 4, 2006
People & Speech • Born to be wild and understand speech • Speech as an instrument of cooperative action • What literacy has to do with speech
Wired for Speech (Nass & Brave) • “Humans are automatic experts at extracting the social aspects of speech.” • Our brains associate voice with social relationships: so what happens when we communicate with technology through speech?
Our brains can’t tell the difference • Gender stereotyping: the man (woman?) behind the curtain • pitch, pitch range • Personality: opposites (don’t) attract • volume, pitch, pitch range, speed rate
Hello? Hey. Hey! Hey, what’s up? Not much. How are you? Okay! So…what’s going on?
“Our talk exchanges do not normally consist of a succession of disconnected remarks, and would not be rational if they did. They are characteristically…cooperative efforts.” ~ H. Paul Grice
Conversational Maxims • Maxims of Quantity • Make your contribution as informative as necessary. • Do not make your contribution more informative than is necessary. • Maxims of Quality • Do not say what you believe to be false. • Do not say that for which you lack adequate evidence. • Maxims of Relevance • Be relevant. • Maxims of Manner • Avoid obscurity. • Avoid ambiguity. • Be brief. • Be orderly.
How we use language • The maxims describe how speech should be produced • Cooperative users • But speech is often subordinate to some kind of action • A sentence serves a syntactic function • A dialogue act serves a pragmatic function
Sentence Types • Constituent interrogatives • Yes/no interrogatives • Imperatives • Assertions • Each of these sentence types serve a different syntactic function
Dialogue Acts • Task Management Acts • Constitutive acts • Expressives (complimenting someone) • Declaratives (sentencing someone to prison) • Informative acts • Assertives (stating a fact) • Interrogatives (asking for information) • Obligative acts • Directives (requests) • System directives (calling the help system) • Commissives (offering something) • Dialogue Management Acts • Flow-regulating acts (beginning an exchange) • Grounds-keeping acts (clarifying a point)
Task /Dialogue Management Example System: How may I help you? DM: exchange initiator TM: offer Caller: I was trying to place a call and must have dialed the wrong number DM: acknowledgment TM: acceptance, assertion can I get credit for that? DM: turn assignment TM: request System: Do you need me to give you credit? DM: acknowledgment, turn assignment TM: offer Caller: Yes. DM: confirmation, turn release TM: acceptance
“The situation in which words are uttered can never be passed over as irrelevant to the linguistic expression.” ~Bronislaw Malinowski
Illiterate people also… • Live in poverty • Have not had formal schooling • Are marginalized • Feel inferior to educated people • May not have regular jobs • Live in literate societies
#1 All Kpelle men are rice farmers. Mr. Smith is not a rice farmer. Is he a Kpelle man? Illiterate Subject: I don’t know the man in person. I have not laid eyes on the man myself…If you know a person, if a question comes up about him you are able to answer. But if you do not know the person, if a question comes up about him, it’s hard for you to answer it. #2 In the far north all bears are white; Novaya Zemyla is in the far north. What color are the bears there? Illiterate Subject: You should ask the people who have been there and seen them…We always speak of only what we see; we don’t talk about what we haven’t seen. Cognition & Literacy Experiential, empirical, situational knowledge
Translating an interface into illiterate • Replace all text with icons • Done!
It’s not that simple… • Associating an identifier (icon, number) with a concept is a literate task • Reluctance to learning how to do arbitrary tasks • Oranges • Apples • Bananas • Please enter your selection: 2 apples
Translating into illiterate, take two • Turn it into a speech-based telephone application. • Done!
Issues with speech only interfaces • Can’t support browsing • Difficult to represent spatial and temporal data • Requires user to remember a lot of information • Forced through a deep decision tree • Memorize command for abstract tasks
Our own experience • Environmental factors • Reluctance • Attrition • Memorization of commands • Clarifications • Dialect