The entertainment industry meets simulation the challenge of the holodeck
Advertisement
This presentation is the property of its rightful owner.
1 / 23

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck PowerPoint PPT Presentation

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck. Bill Swartout Institute for Creative Technologies University of Southern California. The ICT is intended to serve as a nexus between the simulation and entertainment communities. Institute For Creative Technologies.

Related searches for The Entertainment Industry Meets Simulation: The Challenge of the Holodeck

Download Presentation

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The entertainment industry meets simulation the challenge of the holodeck

The Entertainment Industry Meets Simulation:The Challenge of the Holodeck

Bill Swartout

Institute for Creative TechnologiesUniversity of Southern California


Brief

The ICT is intended to serve as a nexus between the simulation and entertainment communities


Brief

Institute For Creative Technologies

Tech

Show

Biz


How can we best simulate reality

How can we best simulate reality?

  • Two approaches

    • Military simulation

    • Entertainment industry


Simulating reality two approaches

Simulating Reality:Two Approaches

Military Simulation

  • Literal: try to include as much reality in simulation as possible

  • Homogeneous

    Entertainment Industry

  • Goal-directed: create effect of real experience by selectively focusing on elements (real and unreal)

  • Hybrid Approach: combine disparate elements


Brief

Movie clip


To the holodeck and beyond

To the Holodeck and beyond…


Mission rehearsal exercise project new operations

Mission Rehearsal Exercise Project: New Operations


Mission rehearsal exercise project ict vr theater

Mission Rehearsal Exercise Project: ICT VR Theater


What we don t want

What we don’t want…

Movie clip


New research areas

New Research Areas

  • Speech Recognition

  • Natural Language Understanding

  • Dialogue Management

  • Natural Language Generation

  • Speech Synthesis

  • Gesture Generation

  • Perception

  • Personality Variation

  • Sound Spatialization

  • Interactive Story Structure


Brief

Movie clip


Brief

Integration Raises New Challenges: Natural LanguageProject Leaders: Ed Hovy (ISI), Kevin Knight (ISI), David Traum (ISI), Shri Narayanan (IMSC)

  • Speech recognition: Noisy environment

  • Multimodal: Face To Face (speech+gesture), Radio

  • Multiple Interactors (Sgt, medic, squads,…)

    • Messages tailored for multiple addressees/overhearers

  • Emotion


Challenge speech synthesis project leaders lewis johnson isi shri narayanan imsc

“Squad Leaders listen up!”

Challenge: Speech SynthesisProject Leaders: Lewis Johnson (ISI), Shri Narayanan (IMSC)

  • Most speech synthesis sounds unnatural

  • Even the best have limited expression (AT&T NextGen)

    • No command voice

  • Use hybrid approach:

    • Multiple synthesizers withdifferent “voices”

    • Command voice

Text toSpeechSynthesis

  • Use pre-recorded library for highly emotional speech


Challenge modeling emotions project leaders jon gratch ict stacy marsella isi

Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI)


Challenge modeling emotions project leaders jon gratch ict stacy marsella isi1

Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI)

Movie clip


Challenge modeling emotions project leaders jon gratch ict stacy marsella isi2

Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI)

  • Problem addressed:

    • Virtual humans need emotions to be believable & realistic

  • Approach:

    • Characters have beliefs, goals, utilities

    • Construct plans

    • Events evaluated against plans

    • If goals thwarted (or enabled) characters react


Emotional reaction

Emotional Reaction

Movie clip


Modeling emotions issues

Modeling Emotions: Issues

  • Integrating gesture, natural language & voice to project realistic behavior

    • Managing unintended interpretations

  • Hollywood vs. Psychology

    • Movie emotions quite different from natural emotions

  • Recognizing human emotions


  • Challenge story is critical

    Challenge:Story is Critical

    • More than an event list

    • Successful stories:

      • Juxtapose contrasting elements

      • Surprise (plot twists)

      • Dilemmas

      • Things go wrong

      • Build to climax (and ebb)


    Structuring stories for intelligent agents issues

    Structuring Stories for Intelligent Agents: Issues

    • Traditional story telling tricks require linear structure

    • Immersed participant needs freewill

    • Need “bumpers”

      • Possibility: director agent controls environment & characters to limit choice


    Reasons for optimism

    Reasons for Optimism

    • Hybrid approach

      • Use all the tools in the toolbox

    • Entertainment industry as source of knowledge and techniques

    • Story context constrains possibilities

    • Synergy from graphics, sound & emotional engagement

      • The mind is a willing collaborator


  • Login