The Entertainment Industry Meets Simulation: The Challenge of the Holodeck - PowerPoint PPT Presentation

paul2
the entertainment industry meets simulation the challenge of the holodeck n.
Skip this Video
Loading SlideShow in 5 Seconds..
The Entertainment Industry Meets Simulation: The Challenge of the Holodeck PowerPoint Presentation
Download Presentation
The Entertainment Industry Meets Simulation: The Challenge of the Holodeck

play fullscreen
1 / 23
Download Presentation
The Entertainment Industry Meets Simulation: The Challenge of the Holodeck
405 Views
Download Presentation

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. The Entertainment Industry Meets Simulation:The Challenge of the Holodeck Bill Swartout Institute for Creative TechnologiesUniversity of Southern California

  2. The ICT is intended to serve as a nexus between the simulation and entertainment communities

  3. Institute For Creative Technologies Tech Show Biz

  4. How can we best simulate reality? • Two approaches • Military simulation • Entertainment industry

  5. Simulating Reality:Two Approaches Military Simulation • Literal: try to include as much reality in simulation as possible • Homogeneous Entertainment Industry • Goal-directed: create effect of real experience by selectively focusing on elements (real and unreal) • Hybrid Approach: combine disparate elements

  6. Movie clip

  7. To the Holodeck and beyond…

  8. Mission Rehearsal Exercise Project: New Operations

  9. Mission Rehearsal Exercise Project: ICT VR Theater

  10. What we don’t want… Movie clip

  11. New Research Areas • Speech Recognition • Natural Language Understanding • Dialogue Management • Natural Language Generation • Speech Synthesis • Gesture Generation • Perception • Personality Variation • Sound Spatialization • Interactive Story Structure

  12. Movie clip

  13. Integration Raises New Challenges: Natural LanguageProject Leaders: Ed Hovy (ISI), Kevin Knight (ISI), David Traum (ISI), Shri Narayanan (IMSC) • Speech recognition: Noisy environment • Multimodal: Face To Face (speech+gesture), Radio • Multiple Interactors (Sgt, medic, squads,…) • Messages tailored for multiple addressees/overhearers • Emotion

  14. “Squad Leaders listen up!” Challenge: Speech SynthesisProject Leaders: Lewis Johnson (ISI), Shri Narayanan (IMSC) • Most speech synthesis sounds unnatural • Even the best have limited expression (AT&T NextGen) • No command voice • Use hybrid approach: • Multiple synthesizers withdifferent “voices” • Command voice Text toSpeechSynthesis • Use pre-recorded library for highly emotional speech

  15. Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI)

  16. Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI) Movie clip

  17. Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI) • Problem addressed: • Virtual humans need emotions to be believable & realistic • Approach: • Characters have beliefs, goals, utilities • Construct plans • Events evaluated against plans • If goals thwarted (or enabled) characters react

  18. Emotional Reaction Movie clip

  19. Modeling Emotions: Issues • Integrating gesture, natural language & voice to project realistic behavior • Managing unintended interpretations • Hollywood vs. Psychology • Movie emotions quite different from natural emotions • Recognizing human emotions

  20. Challenge:Story is Critical • More than an event list • Successful stories: • Juxtapose contrasting elements • Surprise (plot twists) • Dilemmas • Things go wrong • Build to climax (and ebb)

  21. Structuring Stories for Intelligent Agents: Issues • Traditional story telling tricks require linear structure • Immersed participant needs freewill • Need “bumpers” • Possibility: director agent controls environment & characters to limit choice

  22. Reasons for Optimism • Hybrid approach • Use all the tools in the toolbox • Entertainment industry as source of knowledge and techniques • Story context constrains possibilities • Synergy from graphics, sound & emotional engagement • The mind is a willing collaborator