cs491j csmpg
Download
Skip this Video
Download Presentation
CS491j CSMPG

Loading in 2 Seconds...

play fullscreen
1 / 25

CS491j CSMPG - PowerPoint PPT Presentation


  • 227 Views
  • Uploaded on

CS491j CSMPG. Lecture Eight: AI Deux. Roadmap. Last Time: Logic, Reasoning, Planning Today: Learning, Modeling, Emotions . Learning. Learning has numerous definitions What is its role in gaming?. Offline Learning. Learning is often slow In games much of what is to be learned is fixed.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CS491j CSMPG' - LeeJohn


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
cs491j csmpg

CS491j CSMPG

Lecture Eight: AI Deux

CS 491J

Computer Science of Multi-Player Games

roadmap
Roadmap
  • Last Time: Logic, Reasoning, Planning
  • Today: Learning, Modeling, Emotions

CS 491J

Computer Science of Multi-Player Games

learning
Learning
  • Learning has numerous definitions
  • What is its role in gaming?

CS 491J

Computer Science of Multi-Player Games

offline learning
Offline Learning
  • Learning is often slow
  • In games much of what is to be learned is fixed.
  • Learn offline, use learned policy or model online
    • Uses? Advantages? Disadvantages?

CS 491J

Computer Science of Multi-Player Games

online learning
Online Learning
  • Learning online provides adaptability
  • Customized to a particular player
  • Responsive to changing environments
  • Limited learning resources
  • Mixed-Mode learning?

CS 491J

Computer Science of Multi-Player Games

supervised learning
Supervised Learning

CS 491J

Computer Science of Multi-Player Games

learning classification
Learning: Classification
  • Learn a discrete decision problem:
    • Is this player attacking?
    • Am I winning or losing?
    • Is the player having fun?
  • Techniques
    • Decision Trees
    • Expert Systems/Case-Based Reasoning
    • Statistical Models

CS 491J

Computer Science of Multi-Player Games

learning function aprox
Learning: Function Aprox.
  • Learn a continuous output given inputs
    • Can be used for classification
    • How fast should I drive?
    • What is the probability I’ll win this battle?
    • How good is this player?
  • Techniques
    • Neural Networks
    • Statistical Models

CS 491J

Computer Science of Multi-Player Games

causality bayesian networks
Causality: Bayesian Networks
  • Often times we know the basic structure of how things happen
  • But we observe limited information
  • We can still reason about what happened

CS 491J

Computer Science of Multi-Player Games

example network
Example Network

Friendly Attack

On Lava

Enemy Attack

Unit Dies

Unit Dies

CS 491J

Computer Science of Multi-Player Games

learning bayes nets
Learning Bayes Nets
  • Structure learning is hard (and often unnecessary)
  • Probability distribution learning is hard (but tractable in many circumstances)

CS 491J

Computer Science of Multi-Player Games

reinforcement learning
Reinforcement Learning
  • Learns a policy:
    • Given a state, how should we act?
  • Unsupervised Learning (Approx.)
    • Dependent on a “Reward” function
    • Dependent on a appropriate state space
    • Dependent on a good set of actions
  • Produced the best backgammon program (TD-Gammon)

CS 491J

Computer Science of Multi-Player Games

the basic idea
The basic idea
  • Take many different actions (initially at random, then guided)
  • Obtain a reward
  • Propogate reward backwards across actions (The “credit assignment problem”)

CS 491J

Computer Science of Multi-Player Games

learning a policy
Learning a Policy
  • Estimate the utility from the reward for the current state plus the expected reward for future states.
  • Given experience (i to j), update the utility:
    • U’(i) = U(i) + (delta)*(R(i)+U(j)-U(i))

CS 491J

Computer Science of Multi-Player Games

valuing states and actions
Valuing States and Actions
  • Given a state i and an action a, learn the utility or value of the action Q(i,a)
  • Same basic idea as TD-Learning, but update each pair, rather than each state:
  • Q’(i,a) = Q(i,a) + (delta)*(max (Q(j,a’)-Q(i,a)))

CS 491J

Computer Science of Multi-Player Games

internal state
Internal State
  • So far we have considered agents that plan and learn, but what about the agent’s feelings
  • Creating believable characters.

CS 491J

Computer Science of Multi-Player Games

finite state automata
Finite State Automata
  • A simple model for a characters inner state
  • Each node represents a state
  • Each edge represents an event in the world
  • Behavior is keyed to state in the machine

CS 491J

Computer Science of Multi-Player Games

example
Example

Time

Hit

Run

Stand

Time

Sighted

Sighted

Pain

Attack

Sighted

Zero Life

Die

CS 491J

Computer Science of Multi-Player Games

slide19
FSMs
  • Often used
  • Advantages?
  • Disadvantages?

CS 491J

Computer Science of Multi-Player Games

emotional modelling
Emotional modelling
  • Continous approaches
  • Each “emotion” receives a number
  • Events update the value of the “emotion”

Various update rules are possible

  • The set of numeric values is the player’s state

CS 491J

Computer Science of Multi-Player Games

example21
Example

CS 491J

Computer Science of Multi-Player Games

learning possibilities
Learning Possibilities
  • Learn “realistic” states
  • Human labeled realistic/unrealistic behaviors
  • Learn to produce realistic behaviors

CS 491J

Computer Science of Multi-Player Games

further directions
Further Directions
  • Multi-Layer emotional models
  • Natural Language Processing & Generation
  • Cooperative emotions (empathy & more…)

CS 491J

Computer Science of Multi-Player Games

coming soon
Coming Soon
  • 3/23: Proposal Day
    • Pitch a game to the class
  • 3/28,3/30: First 10 Student Presentations
    • Topic to me 3 weeks before
    • Slides to me 1 week before
  • 4/11: Project Proposals
    • Binding contracts for what you will produce

CS 491J

Computer Science of Multi-Player Games

ai in games discussion
AI in Games: Discussion

CS 491J

Computer Science of Multi-Player Games

ad