VICTEC and Empathic Interfaces

1. VICTEC and Empathic Interfaces Ruth Aylett Ana Paiva

2. What does VICTEC stand for? • V: Virtual • ICT: Information Communication • Technology • E: Empathic • C: Characters

3. Inspired by theatre • Use of performance to dramatise the problem • Theatre in Education • Linked workshops • But is not a cheap option • In every group some will be bullying others • Interaction is limited in conventional performance

4. Forum theatre (Boal) • An interactive form of theatre • Sections of audience ‘own’ a character • The actor meets them ‘in role’ and discusses actions • Can only act ‘in character’ • Episodic performance • Actor can halt an episode to ask for more advice • Hard to manage in school environment

5. Implementing a Virtual Forum Theatre • 3D interactive graphics and virtual actors - synthetic characters • Individual interaction based on creating empathy with the characters • Computer game technology - but NOT a game! • Provide a safe environment for children to explore different perspectives on bullying behaviour.

6. What are Intelligent SyntheticCharacters? • A synthetic character is a digital or physical entity with personality that can interact with people in real time. • Computer games characters are not very intelligent • Do not remember anything about interaction • Hard to believe in them as personalities • Thus hard to create empathy with them

7. We want them to be believable Is believability the same as realism? We believe in Mickey Mouse… Characters that are NEARLY realistic may be ‘creepy’ We have high expectations And then they clash with them How realistic should they be?

8. Empathic Characters Already much experience in animation

9. Mediating Empathy • (1) via the situation- the observer concludes the emotional state of the target from the situation the target is dealing with. (2) via emotional expressions - the observer interprets the behaviour of the target: for example, when a target cries this means that he/she is probably sad.

10. Empathic Synthetic Agents Characters show empathy towards users and other characters Users experience Empathic emotions "empathic synthetic agent" as "an agent that is able to perceive and have an internal representation of the other agent's emotions and/or experiencing an appropriate emotion as a consequence "empathy triggering synthetic agent" as "an agent that is able to, by its behaviour and features, allow the users to build an empathic relation with it"

11. Mediation by situation • Depends on narrative engagement • Collection of narratives from children • Using Kar2ouche story-boarding tool • Observation of dramatic performance • Scripting of dramas by drama students • Performances by Uni Herts students • Narratives should be: believable, dramatic, somewhat unpredictable

12. Mediation by emotional expression • Expressive behaviour linked to internal state modelling character’s affect • Facial Expression (expressive facial expressions that include emotions and communication signals) • Voice • Body expression, posture and gestures

13. Designing Characters in VICTEC: Mediating empathy through facial expression

14. Designing Characters in VICTEC: Mediating empathy though body expression

15. Designing Characters in VICTEC: Mediating empathy through exaggeration Voice: • Current Text-to-Speech technology is inadequate • Real Voice must be used to emphasize emotional content Through body expression and animation Through posture and facial expression

16. Engaging the user • The character must be believable • No rerun - what happens counts • Different scenarios and characters • Emerging story as in improvisational drama, not pre-scripted • Acting as an ‘invisible friend’. • Tamagotchis seemed to evoke empathy • Give advice about dealing with the situation and see what happens

17. A Trailer • Not the finished software, but how we think it is going to look

18. Capturing affective responses • Creating an affective loop • FearNot! Attempts to evoke empathy in the user • How could one produce an empathic response to the user? • Requires ability to detect affective state of the user • Non-invasively

19. Interaction using Sentoy • Intelligent Physical Interface • Aims at making the bridge between the real and virtual worlds, extending and enhancing the child-system interactions

20. Hardware of SenToy • Magnet (on off) • Pressure sensors • (4 positions each) • Accelerometer • (x, y, z angles)

21. Embodiment • Restrictions: • Sensors size / number • Sensors capabilities • Battery vs Autonomy • Has to handled by children (not too heavy but also robust)

22. Validation of Embodiment • A Teddy, a puppet and a Barbie were tested • - They had to express emotions and actions with every dolls: • - The subjects preferred a neutral expression • (Barbie and Teddy) • - The doll had to be malleable (Teddy) • The teddy was easier to handle, more flexible and allowed the free expression of emotions / actions

23. Gestures Emotions Reference According to Lazarus fear is associated with avoidance. Put the toy's hands in front of it's eyes or moving the toy backwards vigorously FEAR According to Lazarus action tendencies for disgust include “move away”, nausea and even vomiting. DISGUST Moving the toy slightly backwards (squeezing it slightly) Joy is portrayed with open arms, movements such as clapping or rhythmic movement according to Darwin. Swinging the toy (putting it dancing) and/or playing with its arms HAPPINESS Bend down its neck or bend down all the toy's trunk SADNESS According to Scherer sadness is expressed through slow movement inwards and head down. According to Lazarus anger is associated with “tendency to attack” To place its arms crosswise or shake the toy vigorously ANGER According to Laban surprise is associated with attention and with a sudden event and inclination of trunk backwards. “A surprise person often raises his opened hands high above his head, or by bending his arms only to the level of his face”. Open its arms backwards inclining its trunk slightly backwards too SURPRISE Built-in patterns for emotion recognition

24. Gestures Actions Swing the legs forward and backward alternately or moving the toy forward with small jumps WALK A bouncing jump in the vertical JUMP To bend down and move the arm like it was picking something or moving the toy like it was diving PICK Built-in patterns

25. SenToy’s Integration Module SenToy Hardware Signal Processing Procedure Software • The Data from the sensors is passed through radio into a receiver connected with the computer. • Then some signal processing occurs that filters the information, detects patterns and send it to a specific application integration module that provides the semantic.

26. Sentoy in Victec • Roles of SenToy in the Interaction: • Out-of-scenario interaction • As an embodiment of the user (toy as best friend). Number of toys is big restriction for evaluation • In-scenario • As a form of transparently enriching the experience. • As inspirational input to the character (eg. cheering) • As a non-intrusive recording device

27. Virtually friends • Event for 400 children in June 2004 at University of Hertfordshire: • Virtually friends: Social role-play with robots and agents • Questionnaire and interaction with the software • Activities with the robots used in research

28. Challenges - 1 • There IS no easy answer • No strategy always works • Tension between realism and a straightforward message • Do we have to present a happy ending? • Producing believable characters and varying scenarios • Pushing the technology here • Especially language for unscripted interaction

29. Challenges - 2 • Cultural differences • There is no word exactly equivalent to ‘bullying’ in German or Portuguese • School uniform and other differences • Where bullying takes place • No school dinners in Germany • Children unaccompanied to school • Now extending work to China...