Loredana Cerrato. Looking for feedback expressions in speech corpora. KTH-TMH Stockholm firstname.lastname@example.org. General idea.
Looking for feedback expressions
in speech corpora
Study of the distribution, typology, use and function of verbal and non-verbal feedback expressions in human communication and in human machine communication.
Try to model human behaviour and try to reproduce it in speech technology application
Allwood J. et al. 1992, On the semantics and pragmatics of linguistic feedback
“Control strategies of the channel of communication and of the informative flux, used by participant to a conversation to support the interaction”.
”Feedback mechanism is one of the most important cohesion devices in human conversation”
”Feedback consists of a set of verbal and bodily means used, in a minimal intrusive way, both to elicit and give information about the basic communicative functions: contact, perception, comprehension, attitudinal reactions”
-Combination of verbal and non verbal expressions
Napoli, Bari, Pisa
(Collected in 1998-1999, it containes 16 dialogues transcribed at different levels: ortographic, phonetic, phonemic, prosodic and DA)
Cerrato L. 1999
Only audio recordings
Giver: gives instruction
Follower: receives the instruction to accomplish the task
park? What park?
I don´t have any parks
till the lake, then at
the park turn left.
the auditive channel of communication is maximised
no visual channel, therefore feedback expressions are produced in a great number.
3) Acoustic phonetic analysis:
With the aim of detecting acoustic cues related to the function carried out by feedback expressions:
90 are feedback expressions
A contradiction: if feedback is expressed in a ”non-intrusive way” can it be counted as a turn of its own?
Material: 4 real dialogues between
1 travel agent and 4 customers
recorded in a travel agency in Gothenburg
Göteborg Spoken Language Corpus.
Cerrato, L. 2002a
4) Acoustic phonetic analysis:
Continuation of contact
Acknowledgement of information
Expression of a point of view
·short duration and flat pitch contour
·longer duration and rising pitch contour
Is it possible to categorize visual feedback in the same way?
Need for a multimodal database
Text: transcription (MSO)
Function: reports the function of the feedback expression and the function of the utterance that precede the feedback expression
Gesture: reports the “non-verbal feedback expressions”, analysed in terms of movements.
Gaze: [should] report the direction of the speakers’ gaze
”I am active” ”I am retrieving information”
”I have understood”:explicit, implicit feedback that indicates the reception and comprehension of the message
User: I want to go to London
System: did you say you wanted to go to London?
User: I want to go to London
System: At what time do you wish to arrive in London?
Ecc.Feedback in human machine interactions
94% of users (of 33) gave feedback at least once
18% of utterances (of 1845) contained feedback expressions
Only in 6% of these cases, feedback appeared in a turn of its own (it occurred mostly at the beginning of longer sentences)
Results adapted from Bell L. & Gustafson J. 2000;
Adapt is an experimental dialogue system in the domain of Stockholm’ s real-estate.
The Adapt database was collected in 1999 by means of the Wizard of Oz technique.
It includes a total of 50 dialogues produced by 33 users.
4 dialogues from GCSL and 4 dialogues from ADAPT
Distributional, functional and acoustic analysis
Cerrato L. 2002b
Quantitative measurement of the entity of head movement related to feedback expressions with the Qualisys MacReflex motion tracking system.
A pilot study: