1 / 50

Affective Computing for Game Design

Affective Computing for Game Design. Eva Hudlicka Psychometrix Associates Blacksburg, VA hudlicka@ieee.org psychometrixassociates.com. GAMEON-NA 2008 August 13, 2008 McGill University, Montreal, Canada. Outline. Emotions & Games: Affective Gaming Affective Computing

tallulah-raymond
Download Presentation

Affective Computing for Game Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Affective Computing for Game Design Eva Hudlicka Psychometrix Associates Blacksburg, VA hudlicka@ieee.org psychometrixassociates.com GAMEON-NA 2008 August 13, 2008 McGill University, Montreal, Canada

  2. Outline • Emotions & Games: Affective Gaming • Affective Computing • Background on Emotion Research • Computational Affective Modeling • Conclusions

  3. Fully-immersive PTSD treatment environment, Rizzo - ICT Drake’s Fortune SMILE math & science game:Villani & Wilbur, Purdue Trends in Gaming • More engaging, immersive • Multi-modal interaction (haptic controls) • Physical & emotional realism

  4. Emotiv SystemsEEG Emsense headset: EEG, heart rate,respiration, headmotion, temperature Trends in Gaming • Adaptive gaming • Games sense user state & adapt gameplay

  5. FreeDive: Breakaway Games Virtual Iraq-PTSD: Rizzo, ICT Neuromatrix: Morphonix Trends in Gaming • Serious games (fastest growing segment) • Education & Training • Exergaming • Rehabilitation & therapy • Pain reduction in healthcare • Coaching (lifestyle & training games) • Psychotherapy • … social change, government, healthcare, science, corporate training, military

  6. Trends in Gaming • Demographics • Girls • Women • Adults • Elderly

  7. We Are About Here… In Terms of the Full Potential of Gaming..

  8. Existing Games Limited In • Interaction bandwidth • Narrative complexity • Player modeling • Adaptation to players • Affective realism of game characters • Affective complexity of game characters

  9. So How Do We Get Here? We Are About Here…

  10. Emotion a Key Factor… in both “Play” & “Work” • Mediates motivation • Influences memory & learning • Key role in decision-making & problem-solving • Central factor in engagement • Key factor in serious games

  11. valence - + - arousal Affective Complexity of Games: Current ….. Image from Russell, 1997

  12. valence - + - arousal ….. & Potential Images adapted from Lim & Aylett, 2005

  13. Future Games Need To… • Recognize & adapt to players’ emotions • Understand players’ affective profiles • Increase affective complexity & autonomy of game characters • Increase visual affective realism of game characters & player avatars • Increase affective complexity of the entire game experience

  14. Affecting Gaming & Affective Computing “Affective gaming is concerned with the application of affective computing techniques to the domain of digital games. Affective gaming involves both the ‘evocation’ of emotions, as well as the detection of player emotion.” (Sykes, 2004) …which will increasingly require emotion modeling

  15. Outline • Emotions & Games: Affective Gaming • Affective Computing • Background on Emotion Research • Computational Affective Modeling • Conclusions

  16. MAX (Becker, Prendinger et al.) Breazeal De Rosis Hudlicka Overview of Affective Computing

  17. Affective Computing • Broad area of interdisciplinary research and practice relating computers and affect • “Anything that combines computing and emotions” • Term coined by Rosalind Picard (MIT Media Lab) • 1997 book “Affective Computing” (MIT Press) • “How can emotions be generated in computers, berecognized by computers, and be expressed by computers?” (Picard, Affective Computing, ‘97)

  18. Emotion Sensing & Recognition “Emotion” Expression OR? GRETA, Fiorella de Rosis, U. Bari Affective Modeling

  19. Affective Computing Includes… • Emotion sensing & recognition • via a variety of sensors from multiple modalities • Generation of ‘affective’ behaviors in machines • Facial expressions in agents and robots • Affective synthetic speech • Affect-induced behavioral variation in robots and agents • Computational models of emotion & affective phenomena • Emotion generation (via appraisal) • Emotion effects on cognition & behavior • Affective user models • Cognitive-affective architectures ..for agents & robots • Generic requirements for modeling emotion • Characterizing emotion in computational terms

  20. Methods & Techniques Relevant for Affect-Focused Game Design • Sensing & recognition of players’ emotions • Adaptive gaming • Game control • Expression of emotions by game characters • More realistic & believable behavior • Complex social interactions • Models of emotion in game characters • To support complex, autonomous behavior • To support adaptive behavior • Models of players’ emotions • Affective user models to support game adaptation • Affective game evaluation • Develop games with desired affective profiles

  21. Outline • Emotions & Games: Affective Gaming • Affective Computing • Background on Emotion Research • Computational Affective Modeling • Conclusions

  22. So What ARE Emotions? • Evaluative judgments of the: • World • Others • Self • … in light of agent’s goals & beliefs • …motivating & coordinating adaptive behavior

  23. Roles of Emotions WHAT? * Social coordination * Rapid communication of behavioral intent; WHAT? * Motivation* Homeostasis * Adaptive behavior Intrapsychic Interpersonal • HOW? • - Global interrupt system • Goal management • Prepare for coordinated actions • - Emotion generation (appraisal) • Emotion effects (processing biases) HOW? Express emotions via: -Facial expression -Speech (content & properties) -Gesture, Posture -Specific actions

  24. How Do We Recognize an Emotion if We See One? • Manifested across multiple, interacting modalities: • Somatic / Physiological(neuroendocrine - e.g., heart rate, GSR) • Cognitive / Interpretive(“Nothing is good or bad but thinking makes it so…”) • Behavioral / Motivational (action oriented, expressive, ‘visible’) • Experiential / Subjective(“that special feeling…”, consciousness) • Much terminological confusion can be attributed to a lack of consideration of these multiple modalities of emotions • e.g., Is emotion a feeling or a thought? - It’s both

  25. Cognitive Subjective Simple Fear “Signature”: Large, Approaching Object Feeling of fear Increased heart-rate; Flee? Freeze? Attacked? Crushed?

  26. States Traits “Big 5” “Giant 3” … Emotions Moods Affective States Negative Positive Complex Basic Joy Anger … Fear A Taxonomy of Affective Factors Attitudes, Preferences… Affective Factors … Pride Guilt Shame

  27. Alternative Views of Emotions • Categorical theories (discrete theories) (Tomkins, Izard, Ekman, Panskepp) • A small number of hardwired basic (fundamental) emotions exist with distinct neuromotor patterns • Dimensional theories(Russell, & Thayer) • 2D: Valence & arousal • 3D: Valence & arousal + dominance • All other emotions defined in terms of these • Componential theories(Scherer, Roseman, Smith and Ellsworth, Frijda) • Emotions result from the (parallel) evaluation of a number of appraisal dimensions (novelty, valence, goal congruence…) • Vast numbers of emotions possible in this space • Individual appraisals represent components of familiar emotions

  28. Categorical View: Basic Emotions (Cohn, 2006)

  29. Dimensional View: Arousal & Valence (Breazeal, 2003 adapted from Russell, 1997)

  30. Effects of Emotions (on cognition & behavior) Generation of Emotions (via cognitive appraisal) Cognitive-Affective Architecture Goals Stimuli Cognitive Appraisal Situations Emotions Expectations Core Processes of Emotions

  31. Appraisal Process Emotions • Domain-Independent Appraisal Dimensions • Novelty • Valence • Goal / Need relevance • Goal congruence • Agency • Coping potential • Social and self norms and values Goals (desires, values, standards) Beliefs, Expectations Existing emotions, moods, traits Emotion Generation via Appraisal Appraisal Dimensions Stimuli Perceived Recalled Imagined

  32. Facial expression Gestures Posture Behavior Blah blah blah Emotion Effects on Behavior Emotion

  33. Emotion Effects on Cognition • Emotion & cognition function as closely-coupled information processing systems • Complex feedback interactions • Emotions influence fundamental processes mediating high-level cognition: • Attention & working memory speed & capacity • Long-term memory recall & encoding • Anxiety • Attentional narrowing / threat bias / self-focus bias • Anger • Risk tolerance / impulsive action bias / attribution of hostility

  34. Outline • Emotions & Games: Affective Gaming • Affective Computing • Background on Emotion Research • Computational Affective Modeling • Conclusions

  35. Why Do We Need Emotion Models in Game Design? • Game characters’ emotions • Generate realistic & affectively-complex character behavior & believable affective expressions • … responsive to dynamically-evolving game scenario & player behavior • … in real-time • Emotion generation & emotion effects on cognition & behavior • Players’ emotions • Help recognize player emotions in real-time • Develop gameplay adaptations • Generate more realistic player avatar behavior

  36. Affective Modeling Outline • Core processes of emotion • …. Computational requirements • …. Representational alternatives • Levels of resolution (tutorial) • Affective architectures (tutorial)

  37. Effects of Emotions (on cognition & behavior) Cognitive-Affective Architecture Goals Stimuli Cognitive Appraisal Situations Emotions Expectations Core Processes of Emotions Generation of Emotions (via cognitive appraisal)

  38. Models of Emotion Generation via Cognitive Appraisal

  39. Emotions Computational Tasks for Appraisal Models Stimuli • Stimuli-to-emotion mappings • Intensity calculation • Nature of mapping process: • * Stages & functions • * Degree of variability • Integrating multiple emotions • Emotion dynamics over time • Types of stimuli: • Internal / External • Real / Imagined • Past / Present / Future • Domain specific / Abstract appraisal dimensions • Complexity of stimulus structure • Mental constructs required (e.g., goals, expectations) • Emotion attributes: • Complexity of emotion construct • * type • * intensity • * cause … • * direction • * …

  40. STIMULI FEAR high Novelty Valence low Goal relevance high Agency other Outcome probability high Goal congruence low Urgency v. high Coping potential low Norms

  41. Emotions N V ? GR World GC A C N … Stimuli --> Appraisal Dimensions --> Emotion(s) This is the difficult part!

  42. N Stimuli --> Appraisal Dimensions V GR World GC • Mental structures required: • Situations, Expectations, Beliefs, Values, Goals, Plans, Causal structures, Agent history • Large amounts of domain-specific knowledge required • Complex reasoning required • Typically implemented using symbolic representations • Rules, Semantic nets, Belief nets, Goal-Procedure hierarchies A C N …

  43. N V GR GC A C N … Appraisal Dimensions --> Emotion(s) ? • Can be easy • Vector maps directly onto n-dim space of app. dimensions • Emotion identified via some measure of proximity (Euclidian dist.) • Learning may be involved (connectionist models) • BUT… • What if the dimension values are ambiguous? • How to combine multiple emotions? • How to calculate intensity? • Individual variability • Across & within individuals • Across emotions • Across contexts

  44. Questions Regarding Representational & Reasoning Requirements • What must represented explicitly? • Time (present, past, future) • Hope needs expectations, regret needs past • Mental constructs • situations, expectations, goals • Memories • what type – declarative, episodic, procedural • Explicit representation of the self • need for complex emotions, social interaction, coping • What types of reasoning are necessary? • What-if • …to generate expectations which influence emotions • Causal explanation • ..important for attribution

  45. Representation & Reasoning Alternatives • Vector spaces (Scherer) • Connectionist (Araujo, Velasquez) • Symbolic • Rules (Marinier, Jones, Henninger, Hudlicka…) • Belief nets (Hudlicka, de Rosis,…) • Complex symbolic structures (Elliot, Reilly, Gratch & Marsella) • Appraisal frames, causal plan structures • Spreading activation over networks of processes (Breazeal) • Decision-theoretic • Decision trees • Decision theoretic formulations (Lisetti & Gmytrasiewicz, Busemeyer) • Blackboards and ‘specialists’ • Finite state machines (Kopecek) • Markov models (El Nasr) • Theorem proving (Zippora) • Dynamical systems

  46. Facial expression Gestures Posture Behavior Blah blah blah Modeling Emotion Effects Afternoon Tutorial Emotion

  47. Outline • Emotions & Games: Affective Gaming • Affective Computing • Background on Emotion Research • Computational Affective Modeling • Conclusions

  48. Affect-Focused Game Design • To achieve the next generation of engaging & effective games we need to focus on emotion • Affective considerations should influence all elements of game design • Visuals: Environment & Character appearance • Music • Narrative structure & plot • Character design • Gameplay design

  49. Affect-Focused Game Design (cont.) • Emotion research in psychology & neuroscience provides data & theories • Affective computing provides methods & techniques for… • Emotion modeling in game characters • Affective user modeling • Emotion recognition in players • Emotion expression in game characters

  50. Thank you Eva Hudlicka Psychometrix Associates Blacksburg, VA hudlicka@ieee.org psychometrixassociates.com

More Related