adaptive intelligent agent in real time strategy games n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Adaptive Intelligent agent in real-time strategy games PowerPoint Presentation
Download Presentation
Adaptive Intelligent agent in real-time strategy games

Loading in 2 Seconds...

play fullscreen
1 / 38

Adaptive Intelligent agent in real-time strategy games - PowerPoint PPT Presentation


  • 186 Views
  • Uploaded on

Adaptive Intelligent agent in real-time strategy games. A Hybrid Online Case-Based Planning &Reinforcement Learning Approach. Project Members. Omar Enayet. Abdelrahman Al- Ogail. Ahmed Atta. Amr Saqr. Dr. Mostafa Aref. Dr. Ibrahim Fathy. Agenda. Project Research Area & Domain

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Adaptive Intelligent agent in real-time strategy games' - lupita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
adaptive intelligent agent in real time strategy games

Adaptive Intelligent agent in real-time strategy games

A Hybrid Online Case-Based Planning &Reinforcement Learning Approach

slide2

Project Members

Omar Enayet

Abdelrahman Al-Ogail

Ahmed Atta

AmrSaqr

Dr. MostafaAref

Dr. Ibrahim Fathy

slide3

Agenda

  • Project Research Area & Domain
  • Motivations.
  • Problem Definition
  • Objectives
  • Related Work
  • Our Methodology.
    • Offline Stage.
    • Online Stage.
  • Testing and Results.
  • Conclusion and Future Work.
  • Demo.
  • References.
slide4

Project Research Area

AI Planning

Knowledge Sharing

AI Learning

Plan then re-plan according to new givens.

Let everyone know instantly what you knew through experience.

Make the machine learn.

slide5

Project Domain

RTS Games

Real-Time Strategy Games.

Severe Time Constraints – Real-Time AI – Many Objects – Imperfect Information – Micro-Actions

slide6

Motivations

War Simulation

Experimental Relevance

Robotics

For the corporation of robots.

They constitute well-defined environments to conduct experiments.

For interest for military which uses battle simulations in training programs.

slide7

Problem Definition

Experience Loss

Static Scripts

Computer AI relies on static scripting techniques.

The Absence of sharing experience costs a lot

slide8

Problem Challenges

Predictability

Non-Adaptability

Computer Opponent actions easily predicted.

Computer Opponent doesn’t adapt to changes in human actions.

slide9

Objectives

Mobile Experience

Adaptive A.I.

Making the Computer Opponent adapt to changes like human do.

Import/Export your experience !

slide10

Related Work

  • SantiOntanon introduced Darmok 2 which is an Online Case-Based Planningsystem designed to play Wargus in 2010.
  • Eric Kok introduced : Adaptive Reinforcement Learning Agents in RTS Games, which merged BDI Agents technology with Reinforcement Learning, 2009
  • M.Johansen devised a CBR/RL system for learning micromanagementin real-time strategy games, 2009
slide12

Case

Behavior

Goal

WinWargus

Preconditions

UnitExist(Peon)

Strategy

Rush-Attack

Alive conditions

Situation

PlayerBuildings > 0

Beginning

Snippet

Shallow Features

BuildBase1

TrainForce(TinyLandForce)

TrainForce(TinyAirForce)

Attack(NearWoodPeon)

NumberOfPlayerPeons = 10

NumberOfPlayerCanonTower = 3

PlayerHasFortress = 1

.

.

Performance

Deep Features

0.6

PathExists = 1

DistanceToEmeny = 40

.

.

Eligibility

Prior Confidence

3

0.8

Case Representation

slide15

Offline Stage:

Learning from human demonstration before shipping game

case abstraction cont d
Case Abstraction (Cont’d)
  • Simplify case complexity.
  • Increase the flexibility.

Abstractor

Point(10, 137)

Unguarded Region

case acquisition cont d
Case Acquisition (Cont’d)
  • Generates Cases from Human’s game play.

Acquisition

Abstract Trace

Casebase

slide20

Online Stage:

Learning what’s the best to do while playing

slide22

Case Retrieval – Cont’d

Case

Case Base

Retriever

Goal

case adaptation cont d
Case Adaptation(Cont’d)
  • Adapts Behaviors according to current game state.
    • Removal of unnecessary actions.
    • Adaptation for unsatisfied preconditions.

Adaptation

Adapted Behavior

Behavior

case concretization cont d
Case Concretization (Cont’d)
  • Adapt the abstract actions to suit current situation.

Concreter

Unguarded Region

Point(10, 137)

case revision cont d
Case Revision (Cont’d)

Uses reinforcement learning, TD-learning SARSA(λ)

Case

Reviser

Evaluation

Used Case

conclusion
Conclusion
  • A Hybrid Architecture of case based reasoning and reinforcement learning was introduced to play strategy games.
  • The architecture merged online case based planning with Sarsa(λ) with eligibility traces. The system showed promising simulation of human behavior; however it still needs a lot extra effort and testing to become industrially capable.
  • Also, the concept of an abstract case base was introduced which opens the door for generic AI engines for games which is never implemented till the date of writing of this document.
future work
Future Work

1) Cooperative AI Agents.

2) Opponent Modelling.

3) Strategy visualization tool.

4) Generic situation assessment.

5) Learn weights of Game State through neural network.

6) Online I-Strategizers.

7) Generic Abstraction/ Concretization.

web resources
Web Resources
  • To get introduced for the whole project journey, evolution, technical summaries, presentations, discussions, meeting minutes and others visit project blog:

http://rtsairesearch.wordpress.com/

  • For full materials of papers, technical summaries, documentations, articles, external links and running version of WARGUS (our test best) use the repository link:

svn checkout http://rtsairesearch.googlecode.com/svn/trunk/ rtsairesearch-read-only

  • For downloading the latest source code for the I-Strategizer Project, please use the following:

svn checkout http://istrategizer.googlecode.com/svn/trunk/ istrategizer-read-only

references
References
  • [1] Martin Johansen Gunnerud. A CBR/RL system for learning micromanagement in real-time strategy games. In Norwegian University of Science and Technology, 2009
  • [2] SantiOntañón, NehaSugandh, KinshukMishra, Ashwin Ram. On-Line Case-Based Planning. In Computational Intelligence, 26(1):84-119, 2010.
  • [3] Brain Schwab. AI Game Engine Programming. Charles River Media, 2009.
  • [4] SantiOntañón and Kane Bonnette and PrafullaMahindrakar and Marco A. G´omez-Mart´ın and Katie Long and JainarayanRadhakrishnan and Rushabh Shah and Ashwin Ram. Learning from Human Demonstrations for Real-Time Case-Based Planning. In AAAI 2008
  • [5] Ralph Bergmann and Wolfgang Wilke,On the role of abstraction in case-based reasoning
references cont
References – Cont.
  • [6] KinshukMishra, Santiago SantiOntañón, and Ashwin Ram. Situation Assessment for Plan Retrieval in Real-Time Strategy Games. In 9th European Conference on Case-Based Reasoning (ECCBR 2009), Trier, Germany.
  • [7] NehaSugandh and Santiago SantiOntañón and Ashwin Ram . On-Line Case-Based Plan Adaptation for Real-Time Strategy Games. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008)
  • [8] Richard S. Sutton and Andrew G. Barto. Reinforcement Learning, An Introduction. MIT press, 2005.
  • [9] Wikipedia, the free encyclopedia. http://www.wikipedia.com
  • [10] Michael Buro, Call for Research in RTS AI, AAAI 2004