Multi level learning in hybrid deliberative reactive mobile robot architectural software systems
1 / 21

Mobile Robot Architectural Software Systems - PowerPoint PPT Presentation

  • Uploaded on

Multi-Level Learning in Hybrid Deliberative/Reactive Mobile Robot Architectural Software Systems. DARPA MARS Kickoff Meeting - July 1999. Georgia Tech College of Computing Prof. Ron Arkin Prof. Ashwin Ram Prof. Sven Koenig Georgia Tech Research Institute Dr. Tom Collins

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Mobile Robot Architectural Software Systems' - LionelDale

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Multi level learning in hybrid deliberative reactive mobile robot architectural software systems l.jpg

Multi-Level Learning in Hybrid Deliberative/Reactive Mobile Robot Architectural Software Systems

DARPA MARS Kickoff Meeting - July 1999

Personnel l.jpg

Georgia Tech Robot Architectural Software Systems

College of Computing

Prof. Ron Arkin

Prof. Ashwin Ram

Prof. Sven Koenig

Georgia Tech Research Institute

Dr. Tom Collins

Mobile Intelligence Inc.

Dr. Doug MacKenzie


Impact l.jpg
Impact Robot Architectural Software Systems

  • Provide the DoD community with a platform-independent robot mission specification system, with advanced learning capabilities

  • Maximize utility of robotic assets in battlefield operations

  • Demonstrate warfighter-oriented tools in three contexts: simulation, laboratory robots, and government-furnished platforms

New ideas l.jpg
New Ideas Robot Architectural Software Systems

  • Add machine learning capability to a proven robot-independent architecture with a user-accepted human interface

  • Simultaneously explore five different learning approaches at appropriate levels within the same architecture

  • Quantify the performance of both the robot and the human interface in military-relevant scenarios

Adaptation and learning methods l.jpg

Case-based Reasoning for: Robot Architectural Software Systems

deliberative guidance (“wizardry”)

reactive situational- dependent behavioral configuration

Reinforcement learning for:

run-time behavioral adjustment

behavioral assemblage selection

Probabilistic behavioral transitions

gentler context switching

experience-based planning guidance

Adaptation and Learning Methods

Available Robots and MissionLab Console

Aura a hybrid deliberative reactive software architecture l.jpg

Reactive level Robot Architectural Software Systems

motor schemas

behavioral fusion via gains

Deliberative level

Plan encoded as FSA

Route planner available

AuRA - A Hybrid Deliberative/Reactive Software Architecture

1 learning momentum l.jpg

Reactive learning via dynamic gain alteration (parametric adjustment)

Continuous adaptation based on recent experience

Situational analyses required

In a nutshell: If it works, keep doing it a bit harder; if it doesn’t, try something different

1. Learning Momentum

2 cbr for behavioral selection l.jpg

Another form of reactive learning adjustment)

Previous systems include: ACBARR and SINS

Discontinuous behavioral switching

2. CBR for Behavioral Selection

3 q learning for behavioral assemblage selection l.jpg

Reinforcement learning at coarse granularity (behavioral assem-blage selection)

State space tractable

Operates at level above learning momentum (selection as opposed to adjustment)

3. Q-learning for Behavioral Assemblage Selection

4 cbr wizardry l.jpg

Experience-driven assistance in mission specification assem-blage selection)

At deliberative level above existing plan representation (FSA)

Provides mission planning support in context

4. CBR “Wizardry”

5 probabilistic planning and execution l.jpg

“Softer, kinder” method for matching situations and their perceptual triggers

Expectations generated based on situational probabilities regarding behavioral performance (e.g., obstacle densities and traversability), using them at planning stages for behavioral selection

Markov Decision Process, Dempster-Shafer, and Bayesian methods to be investigated

5. Probabilistic Planning and Execution

Integration with missionlab l.jpg
Integration with their perceptual triggers MissionLab

  • Usability-tested Mission-specification software developed under DARPA funding (RTPC/UGV Demo II/TMR programs)

  • Incorporates proven and novel machine learning capabilities

  • Extends and embeds deliberative Autonomous Robot Architecture (AuRA) capabilities

Architecture Subsystem Specification Mission Overlay

Development process with mlab l.jpg
Development Process with Mlab their perceptual triggers

Behavioral Specification




Missionlab l.jpg
MissionLab their perceptual triggers

  • Example:

    Scout Mission

Missionlab15 l.jpg
MissionLab their perceptual triggers


Missionlab16 l.jpg
MissionLab their perceptual triggers

Example: Trashbot (AAAI Robot Competition)

Missionlab17 l.jpg
MissionLab their perceptual triggers

Reconnaissance Mission

  • Developed by University of Texas at Arlington using MissionLab as part of UGV Demo II

  • Coordinated sensor pointing across formations

Evaluation simulation studies l.jpg
Evaluation: Simulation Studies their perceptual triggers

  • Within MissionLab simulator framework

  • Design and selection of relevant performance criteria for MARS missions (e.g., survivability, mission completion time, mission reliability, cost)

  • Potential extension of DoD simulators, (e.g., JCATS)

Evaluation experimental testbed l.jpg

Drawn from our existing fleet of mobile robots their perceptual triggers

Annual Demonstrations

Evaluation: Experimental Testbed

Evaluation formal usability studies l.jpg
Evaluation: Formal Usability Studies their perceptual triggers

  • Test in usability lab

  • Subject pool of candidate end-users

  • Used for both MissionLab and team teleautonomy

  • Requires develop-ment of usability criteria and metrics

Schedule l.jpg
Schedule their perceptual triggers