Integrating Learning in Interactive Gaming Simulators - PowerPoint PPT Presentation

Slide1 l.jpg
Download
1 / 20

  • 182 Views
  • Updated On :
  • Presentation posted in: Sports / Games

Integrating Learning in Interactive Gaming Simulators. David W. Aha 1 & Matthew Molineaux 2 1 Intelligent Decision Aids Group Navy Center for Applied Research in AI Naval Research Laboratory; Washington, DC 2 ITT Industries; AES Division; Alexandria, VA surname @aic.nrl.navy.mil.

Related searches for Integrating Learning in Interactive Gaming Simulators

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Integrating Learning in Interactive Gaming Simulators

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

Integrating Learning in Interactive Gaming Simulators

David W. Aha1 & Matthew Molineaux2

1Intelligent Decision Aids Group

Navy Center for Applied Research in AI

Naval Research Laboratory; Washington, DC

2ITT Industries; AES Division; Alexandria, VA

surname@aic.nrl.navy.mil

AAAI’04 Workshop on Challenges in Game AI

25 July 2004


Outline l.jpg

Outline

  • Motivation: Learning in cognitive systems

  • Objectives:

    • Support empirical studies w/ simulators

    • Encourage studies that address industry & military M&S concerns

  • Design: TIELT functionality & components

  • Example: Knowledge base content

  • Status: Implementation, collaborations

  • Summary

Thanks to our sponsor:


Rough anatomy of a cognitive agent l.jpg

Affect

Rough Anatomy of a Cognitive Agent

Reflective Processes

LTM

CognitiveAgent

Concepts

STM

Deliberative Processes

Learning

Other reasoning

Sentences

Communication

(language,

gesture,

image)

Prediction,

planning

Perception

Action

Reactive Processes

Sensors

Effectors

External Environment

Attention

(Brachman, 2003)


Status of cognitive learning l.jpg

Complication

The ML research community has been focusing on:

¬Rapid: Knowledge poor algorithms

¬Enduring: Learning over a short time period

¬Embedded: Stand-alone evaluations

Status of Cognitive Learning

Few deployed cognitive systems integrate techniques that exhibit rapid & enduring learning behavior on complex tasks

  • It’s costly to integrate & evaluate embedded learning techniques

Problem


Wanted a new interface thanks to w cohen others l.jpg

Supervised Learning

Supervised Learning

ML System

ML System

Reasoning

System

Reasoning

System

Interface

(standard format)

Interface

(standard format)

Database

Database

(e.g., UCI Repository)

(e.g., UCI Repository)

Cognitive Learning

Cognitive Learning

Cognitive Learning

Reasoning Modules

Reasoning Systemk

Reasoning Modules

ML Module

ML Module

ML Module

Sensors

Sensors

Sensors

ML Module

ML Module

ML Module

World

(Simulated/Real)

Worldi

(Simulated/Real)

World

(Simulated/Real)

Interface

(standard API)

Interface

(standard API)

Interface

(standard API)

ML Module

ML Modulej

ML Module

Effectors

Effectors

Effectors

(e.g., TIELT)

(e.g., TIELT)

(e.g., TIELT)

Testbed for Integrating and Evaluating Learning Techniques

Wanted: A New Interface(thanks to W. Cohen, others)

Supervised Learning

ML Systemj

Reasoning

Systemk

Interface

(standard format)

Databasei

(e.g., UCI Repository of ML Databases)


Objectives domain l.jpg

Objectives & Domain

Objective

Facilitate the evaluation of learning techniques in CogSys

  • Develop & transition TIELT to help reduce integration costs (time, $)

  • Support DARPA Challenge Problems on Cognitive Learning

  • Demonstrate research utility prior to approaching industry/military

Domain: Why interactive gaming simulators?

  • Available implementations (cheap to acquire & run)

  • Challenging problems for CogSys/ML research

  • Significant interest (military, industry, academia, funding, public)


Tielt specification l.jpg

TIELTSpecification

  • Simplifies integration & evaluation!

    • Learning-embedded reasoning systems & gaming simulators

    • Inputs: 5 descriptions (simulator I/O, game model, learning & performance tasks, reasoning system I/O, & evaluation strategy)

    • Free

  • Learning foci: Many

    • Task (e.g., learn how to execute, or advise on, a task)

    • Player (e.g., learn/predict a human player’s strategies)

    • Game (e.g., learn/refine its objects, their relations, & behaviors)

  • Learning methods: Many types

    • Supervised/unsupervised, immediate/delayed feedback, analytic, active/passive, online/offline, direct/indirect, automated/interactive

    • Learning results should be available for inspection

  • Gaming simulators: Those with challenging learning tasks

  • Reuse: Provide access to libraries of simulators & reasoning systems

    • Abstracts interface definitions from game & task models


Distinguishing tielt l.jpg

Distinguishing TIELT

  • Provides an interface for message-passing interfaces

  • Supports composable system-level interfaces


Slide9 l.jpg

Game

Player(s)

Reasoning

System

Library

Reasoning

System

Reasoning

System

Reasoning

System

Learning

Module

Learning

Module

Learning

Module

Stratagus

. . .

. . .

. . .

Learning

Module

Learning

Module

Learning

Module

Full Spectrum Command

TIELT: Integration Architecture

TIELT’s User Interface

Game

Engine

Library

Advice

Interface

Evaluation

Interface

Prediction

Interface

Coordination

Interface

TIELT

User

TIELT’s

Internal

Communication

Modules

Selected

Game

Engine

Selected

Reasoning

System

.

.

.

Learned Knowledge

(inspectable)

TIELT’s

KB

Editors

Selected/Developed Knowledge Bases

Game

Interface

Description

Reasoning

Interface

Description

Game

Model

Description

Task Descriptions

Evaluation

Methodology Description

TIELT

User

Knowledge

Base

Libraries

GMD

TDs

EMD

GID

RID

GMD

TDs

EMD

GID

RID

GMD

TDs

EMD

GID

RID


Tielt s knowledge bases l.jpg

TIELT’s Knowledge Bases

Game

Interface

Description

Defines communication processes with the game engine

Reasoning

Interface

Description

Defines communication processes with the learning system

Game

Model

Description

  • Defines interpretation of the game

    • e.g., initial state, classes, operators, behaviors (rules)

    • Behaviors could be used to provide constraints on learning

Task Descriptions

  • Defines the selected learning and performance tasks

    • Selected from the game model description

Evaluation

Methodology Description

Defines the empirical evaluation to conduct


Slide11 l.jpg

Game

Engine

Library

Game

Player(s)

Reasoning

System

Library

Reasoning

System

Reasoning

System

Reasoning

System

Processed

State

Learning

Module

Learning

Module

Learning

Module

Stratagus

Raw State

Action

Decision

. . .

. . .

. . .

Learning

Module

Learning

Module

Learning

Module

Full Spectrum Command

Example: Controlling a Game Character

TIELT’s User Interface

Advice

Interface

Evaluation

Interface

Prediction

Interface

Coordination

Interface

TIELT

User

TIELT’s

Internal

Communication

Modules

Selected

Game

Engine

Selected

Reasoning

System

Learned Knowledge

(inspectable)

TIELT’s

KB

Editors

Selected/Developed Knowledge Bases

Game

Interface

Description

Reasoning

Interface

Description

Game

Model

Description

Task Descriptions

Evaluation

Methodology Description

TIELT

User

Knowledge

Base

Libraries

GMD

TDs

EMD

GID

RID

GMD

TDs

EMD

GID

RID

GMD

TDs

EMD

GID

RID


Slide12 l.jpg

Game

Engine

Library

Game

Player(s)

Reasoning

System

Library

Reasoning

System

Reasoning

System

Reasoning

System

Processed

State

Learning

Module

Learning

Module

Learning

Module

Stratagus

Raw State

Edit

. . .

. . .

. . .

Learning

Module

Learning

Module

Learning

Module

Full Spectrum Command

Edit

Example: Updating a Game Model

TIELT’s User Interface

Advice

Interface

Evaluation

Interface

Prediction

Interface

Coordination

Interface

TIELT

User

TIELT’s

Internal

Communication

Modules

Selected

Game

Engine

Selected

Reasoning

System

Learned Knowledge

(inspectable)

TIELT’s

KB

Editors

Selected/Developed Knowledge Bases

Game

Interface

Description

Reasoning

Interface

Description

Game

Model

Description

Task Descriptions

Evaluation

Methodology Description

TIELT

User

Knowledge

Base

Libraries

GMD

TDs

EMD

GID

RID

GMD

TDs

EMD

GID

RID

GMD

TDs

EMD

GID

RID


Slide13 l.jpg

TIELT’s Internal Communication Modules

Database

Evaluation

Interface

Advice

Interface

Database

Engine

State

Evaluator

Controller

Stored

State

Current

State

Model

Updater

Learning

Translator

(Mapper)

Translated Model (Subset)

Selected

Reasoning

System

Learning Task

Selected

Game

Engine

Percepts

Action / Control

Translator

(Mapper)

Learning Outputs

Actions

Perf. Task

Game

Model

Description

Game

Interface

Description

Reasoning

Interface

Description

Task

Descriptions

Evaluation

Methodology

Description

User

Game Model

Editor

Game Interface

Editor

Reasoning

Interface Editor

Task

Editor

Evaluation

Editor


Slide14 l.jpg

Sensing the Game State

(City placement example)

1

In Game Engine, thegame begins; a colony pod is created and placed.

TIELT

Current

State

2

The Game Engine sends a “See” sensor message identifying the pod’s location.

4

Actions

Action

Translator

3

Updates

Game

Engine

1

The Model Updater receives the sensor message and finds the corresponding message template in the Game Interface Description.

Sensors

3

5

2

Model

Updater

Controller

3

4

4

Game

Model

Description

Game

Interface

Description

This message template provides updates (instructions) to the Current State, telling it that there is a pod at the location See describes.

User

Game Model

Editor

Game Interface

Editor

5

The Model Updater notifies the Controller that the See action event has occurred.


Slide15 l.jpg

Getting Decisions from the Learning System

(City placement example)

1

The Controller notifies the Learning Translator that it has received a See message.

TIELT

Controller

2

The Learning Translator finds a city location task, which is triggered by the See message. It queries the controller for the learning mode, then creates a TestInput message to send to the reasoning system with information on the pod’s location and the map from the Current State.

Selected

Reasoning

System

1

Learning

Outputs

4

Action

Translator

Translated Model

(Subset)

Learning

Module #1

Learning

Translator

Current

State

3

. . .

2

2

Learning

Module #n

Reasoning

Interface

Description

Task

Descriptions

3

The Learning Translator transmits the TestInput message to the Reasoning System.

User

4

The Reasoning System transmits output to the Action Translator.

Learning Interface

Editor

Agent

Editor


Slide16 l.jpg

Acting in the Game World

(City placement example)

1

The Action Translator receives a TestOutput message from the Reasoning System.

4.b, c

The Advice Interface receives Move and displays advice to a human player on what to do next, or makes a Prediction.

2

The Action Translator finds the TestOutput message template, determines it is associated with the city location task, and builds a MovePod operator (defined by the Current State) with the parameters of TestOutput.

TIELT

Advice

Interface

Prediction

Interface

4.b

4.c

Current

State

1

2

Actions

4.a

Action

Translator

Game

Engine

3

3

The Action Translator determines that the MoveAction from the Game Interface Description is triggered by the MovePodOperator and binds Moveusing information from MovePod.

3

2

Game

Interface

Description

Reasoning

Interface

Description

User

Game Interface

Editor

Reasoning

Interface Editor

4.a

The Game Engine receives Move and updates the game to move the pod toward its destination, or


Status july 2004 l.jpg

Example

Status (July 2004)

Initial Work

  • TIELT v1 + documentation due 9/04

    • Message protocols

      • Current: Console I/O, TCP/IP

      • Future: Library calls, HLA interface, RMI (possibly)

    • Message content: Configurable

      • Instantiated templates tell it how to communicate with other modules

    • Initialization messages: Start, Stop, Load Scenario, Set Speed

    • Game Model representations (w/ Lehigh University)

      • Simple programs

      • TMK process models

      • PDDL (language used in planning competitions)

  • Stratagus/Wargus module (Lehigh University)

  • Initial publicity (BRIMS’04, here)

  • Workshops being planned: ICCBR’05 (plus competition), ICML’05, ...?


Tielt collaboration projects 2004 05 l.jpg

TIELT Collaboration Projects (2004-05)


Slide19 l.jpg

ToEE

Platform Library

Stratagus

U. Minn-D.

Lehigh U.

Urban

Terror

FSC/R

UT Arl.

USC/ICT

RoboCup

TIELT Collaborations (2004-05)

TIELT’s User Interface

TIELT User

Advice

Interface

Evaluation

Interface

Prediction

Interface

Coordination

Interface

U.Mich.

U.Minn-D.

USC/ICT

Reasoning System Library

Game Library

TIELT’s

Internal

Communication

Modules

Soar: U.Mich

ICARUS: ISLE

DCA: UT Arlington

EE2

Learning Modules

Troika

Mad Doc

Neuroevolution:

UT Austin

FreeCiv

Others: Many

ISLE

NWU

TIELT’s

KB

Editors

Selected/Developed Knowledge Bases

Many

LU, USC

U. Mich.

Mich/ISLE

Many

Game

Interface

Description

Reasoning

Interface

Description

Game

Model

Description

Task Descriptions

Evaluation

Methodology Description

TIELT

User

Knowledge Base Libraries


Summary questions concerns l.jpg

Summary: Questions? Concerns?

  • TIELT: Mediates between a (gaming) simulator and a learning-embedded reasoning system

    • Goals:

      • Simplify running learning expts with cognitive systems

      • Support DARPA challenge problems in learning

    • Designed to work with many types of simulators & reasoning systems

  • Status:

    • v1 scheduled for completion in 9/04

      • Please see Matt Molineaux’s demo

    • 11 additional organizations about to start 1-year collaborations

      • Enhances probability that TIELT will achieve its goals


  • Login