Cs491j csmpg
Download
1 / 25

CS491j CSMPG - PowerPoint PPT Presentation


  • 224 Views
  • Uploaded on
  • Presentation posted in: Sports / Games

CS491j CSMPG. Lecture Eight: AI Deux. Roadmap. Last Time: Logic, Reasoning, Planning Today: Learning, Modeling, Emotions . Learning. Learning has numerous definitions What is its role in gaming?. Offline Learning. Learning is often slow In games much of what is to be learned is fixed.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentationdownload

CS491j CSMPG

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cs491j csmpg l.jpg

CS491j CSMPG

Lecture Eight: AI Deux

CS 491J

Computer Science of Multi-Player Games


Roadmap l.jpg

Roadmap

  • Last Time: Logic, Reasoning, Planning

  • Today: Learning, Modeling, Emotions

CS 491J

Computer Science of Multi-Player Games


Learning l.jpg

Learning

  • Learning has numerous definitions

  • What is its role in gaming?

CS 491J

Computer Science of Multi-Player Games


Offline learning l.jpg

Offline Learning

  • Learning is often slow

  • In games much of what is to be learned is fixed.

  • Learn offline, use learned policy or model online

    • Uses? Advantages? Disadvantages?

CS 491J

Computer Science of Multi-Player Games


Online learning l.jpg

Online Learning

  • Learning online provides adaptability

  • Customized to a particular player

  • Responsive to changing environments

  • Limited learning resources

  • Mixed-Mode learning?

CS 491J

Computer Science of Multi-Player Games


Supervised learning l.jpg

Supervised Learning

CS 491J

Computer Science of Multi-Player Games


Learning classification l.jpg

Learning: Classification

  • Learn a discrete decision problem:

    • Is this player attacking?

    • Am I winning or losing?

    • Is the player having fun?

  • Techniques

    • Decision Trees

    • Expert Systems/Case-Based Reasoning

    • Statistical Models

CS 491J

Computer Science of Multi-Player Games


Learning function aprox l.jpg

Learning: Function Aprox.

  • Learn a continuous output given inputs

    • Can be used for classification

    • How fast should I drive?

    • What is the probability I’ll win this battle?

    • How good is this player?

  • Techniques

    • Neural Networks

    • Statistical Models

CS 491J

Computer Science of Multi-Player Games


Causality bayesian networks l.jpg

Causality: Bayesian Networks

  • Often times we know the basic structure of how things happen

  • But we observe limited information

  • We can still reason about what happened

CS 491J

Computer Science of Multi-Player Games


Example network l.jpg

Example Network

Friendly Attack

On Lava

Enemy Attack

Unit Dies

Unit Dies

CS 491J

Computer Science of Multi-Player Games


Learning bayes nets l.jpg

Learning Bayes Nets

  • Structure learning is hard (and often unnecessary)

  • Probability distribution learning is hard (but tractable in many circumstances)

CS 491J

Computer Science of Multi-Player Games


Reinforcement learning l.jpg

Reinforcement Learning

  • Learns a policy:

    • Given a state, how should we act?

  • Unsupervised Learning (Approx.)

    • Dependent on a “Reward” function

    • Dependent on a appropriate state space

    • Dependent on a good set of actions

  • Produced the best backgammon program (TD-Gammon)

CS 491J

Computer Science of Multi-Player Games


The basic idea l.jpg

The basic idea

  • Take many different actions (initially at random, then guided)

  • Obtain a reward

  • Propogate reward backwards across actions (The “credit assignment problem”)

CS 491J

Computer Science of Multi-Player Games


Learning a policy l.jpg

Learning a Policy

  • Estimate the utility from the reward for the current state plus the expected reward for future states.

  • Given experience (i to j), update the utility:

    • U’(i) = U(i) + (delta)*(R(i)+U(j)-U(i))

CS 491J

Computer Science of Multi-Player Games


Valuing states and actions l.jpg

Valuing States and Actions

  • Given a state i and an action a, learn the utility or value of the action Q(i,a)

  • Same basic idea as TD-Learning, but update each pair, rather than each state:

  • Q’(i,a) = Q(i,a) + (delta)*(max (Q(j,a’)-Q(i,a)))

CS 491J

Computer Science of Multi-Player Games


Internal state l.jpg

Internal State

  • So far we have considered agents that plan and learn, but what about the agent’s feelings

  • Creating believable characters.

CS 491J

Computer Science of Multi-Player Games


Finite state automata l.jpg

Finite State Automata

  • A simple model for a characters inner state

  • Each node represents a state

  • Each edge represents an event in the world

  • Behavior is keyed to state in the machine

CS 491J

Computer Science of Multi-Player Games


Example l.jpg

Example

Time

Hit

Run

Stand

Time

Sighted

Sighted

Pain

Attack

Sighted

Zero Life

Die

CS 491J

Computer Science of Multi-Player Games


Slide19 l.jpg

FSMs

  • Often used

  • Advantages?

  • Disadvantages?

CS 491J

Computer Science of Multi-Player Games


Emotional modelling l.jpg

Emotional modelling

  • Continous approaches

  • Each “emotion” receives a number

  • Events update the value of the “emotion”

    Various update rules are possible

  • The set of numeric values is the player’s state

CS 491J

Computer Science of Multi-Player Games


Example21 l.jpg

Example

CS 491J

Computer Science of Multi-Player Games


Learning possibilities l.jpg

Learning Possibilities

  • Learn “realistic” states

  • Human labeled realistic/unrealistic behaviors

  • Learn to produce realistic behaviors

CS 491J

Computer Science of Multi-Player Games


Further directions l.jpg

Further Directions

  • Multi-Layer emotional models

  • Natural Language Processing & Generation

  • Cooperative emotions (empathy & more…)

CS 491J

Computer Science of Multi-Player Games


Coming soon l.jpg

Coming Soon

  • 3/23: Proposal Day

    • Pitch a game to the class

  • 3/28,3/30: First 10 Student Presentations

    • Topic to me 3 weeks before

    • Slides to me 1 week before

  • 4/11: Project Proposals

    • Binding contracts for what you will produce

CS 491J

Computer Science of Multi-Player Games


Ai in games discussion l.jpg

AI in Games: Discussion

CS 491J

Computer Science of Multi-Player Games


ad
  • Login