sb corla s chema b ased co nstructivist r obot l earning a rchitecture l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SB-CoRLA: S chema- B ased Co nstructivist R obot L earning A rchitecture PowerPoint Presentation
Download Presentation
SB-CoRLA: S chema- B ased Co nstructivist R obot L earning A rchitecture

Loading in 2 Seconds...

play fullscreen
1 / 127

SB-CoRLA: S chema- B ased Co nstructivist R obot L earning A rchitecture - PowerPoint PPT Presentation


  • 300 Views
  • Uploaded on

SB-CoRLA: S chema- B ased Co nstructivist R obot L earning A rchitecture Yifan Tang DiLab Agenda Intro Related Work Approach Simulation Conclusion Agenda Information types, schemas, and evaluation criteria Team task solution Team task solution Team configuration

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SB-CoRLA: S chema- B ased Co nstructivist R obot L earning A rchitecture' - paul2


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
agenda

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Agenda

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

  • Introduction: background, research objectives, and motivation
  • Related work
  • Approach
  • Simulations and results
  • Conclusion and future work

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

SB-CoRLA overview

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

agenda3

Agenda

Agenda

Intro

Intro

Related Work

Related Work

Approach

Approach

Simulation

Simulation

Conclusion

Conclusion

Agenda

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

  • Introduction: background, research objectives, and motivation
  • Related work
  • Approach
  • Simulations and results
  • Conclusion and future work

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

SB-CoRLA overview

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

goal of my research

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Goal of my research
  • To develop an architecture that enables continuous robot learning
  • To extend the existing ASyMTRe architecture to enable constructivist robot learning
    • A method to learn new knowledge and skills based upon past experience
  • To explore the robot team solution search problem in a different way
    • Task allocation problems is an NP-hard search problem
why is robot learning beneficial

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Why is robot learning beneficial?
  • Tasks that have been solved before; or new task, same set of skills
    • When designer doesn’t want to reinvent the wheel
  • Development of new behaviors
    • When robot adapts to the environment through interaction
  • Unknown environment
    • When human expertise is not sufficient
  • Biological inspiration
    • Learning is widely observed in the biological world
    • Learning reduces genetic material
my dissertation builds upon the past research asymtre for software reconfiguration

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

My dissertation builds upon the past research (ASyMTRe) for software reconfiguration
  • ASyMTRe: Automated Synthesis of Multi-team member Task solution through software Reconfiguration [Parker, F. Tang, 2006]
  • Inspired by the theory of information invariants [Donald 1994,1997] and schema theory [Lyons and Arbib,1989, 2003]
  • Automatically connects schemas through matching information types to generate a task solution
  • Enables the robots to share sensory, computational, and effector capabilities
example of reconfiguring interconnections of schemas team task go to goal

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Example of reconfiguring interconnections of schemas; team task: go to goal

R1

R2

  • Case 1: R1: laser localization; R2: no environmental sensors
  • Solution:R1 uses laser to calc. its own global position and to find relative position of R2. R1 communicates global position of R2 to R2. R1 goes to goal on its own. R2 goes to goal based on assistance from R1.
  • Case 2: R1 :GPS; R2: camera
  • Solution: R1 uses GPS to localize, communicates its own position to R2; R2 uses camera to determine its location relative to R1, then to calc. its own global position. R1 goes to goal on its own. R2 goes to goal based on assistance from R1, plus its own relative positioning calc.

laser

cs1

ps3

ms1

ms1

ps1

cs2

R1

R2

GPS

Camera

cs1

ps4

ms1

ms1

ps1

cs2

L. E. Parker and F. Tang, Building Multi-Robot Coalitions through Automated Task Solution Synthesis, Proceedings of the IEEE, special issue on Multi-Robot Systems, vol. 94, no. 7, 2006: 1289-1305.

formal problem definition for asymtre

CS

ES

PS

MS

CS

Roboti

PS

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Formal problem definition for ASyMTRe
  • Set of nrobots, R = {R1, R2, …, Rn}
  • Set of Information Types, F = {f1, f2, …}
  • Environmental Sensors, ES = {es1, es2, …}
    • Input: physical sensor signal
    • Output:
  • Perceptual Schemas, PS = {ps1, ps2, …}
    • Input:
    • Output:
  • Communication Schemas, CS = {cs1, cs2}
    • Input:
    • Output:
  • Motor Schemas, MS = {ms1, ms2, …}
    • Input:
    • Output:
example of reconfiguring interconnections of schemas team task go to goal9

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Example of reconfiguring interconnections of schemas; team task: go to goal
  • Case 1: Fully Capable Robots
  • Robot 1 -> Laser Scanner + Map
  • Robot 2 -> Laser Scanner + Map
  • Case 2: Communicate Own Current Position
  • Robot 1 (Helper) -> Laser Scanner + Map
  • Robot 2 (Needy) -> Camera
  • Case 3: Communicate Other’s Current Position
  • Robot 1 (Helper) -> Laser Scanner + Map and Camera
  • Robot 2 (Needy) -> nil

L. E. Parker, Chandra, and Tang, “Enabling Autonomous Sensor-Sharing for Tightly-Coupled Cooperative Tasks”,

3rd NRL International Workshop on Multi-Robot Systems, March 2005.

Chandra, “Software Reconfigurability for Heterogeneous Robot Cooperation”, UTK M.S. thesis, Spring 2004.

case 1 fully capable robots

Red

(Laser)

Blue

(Laser)

Own Pos

Own Pos

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Case 1: fully capable robots

Fully Capable Robots:

Robot 1 -> Laser Scanner + Map

Robot 2 -> Laser Scanner + Map

Blue

Red

asymtre derived schema configurations for case 1

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ASyMTRe-derived schema configurations for case 1

Blue

Red

PS5

PS5

CS1

CS1

Map

Map

ES1

ES1

PS1

PS1

MS

MS

ES2

ES2

PS2

PS2

PS4

PS4

PS3

PS3

CS2

CS2

case 2 helper robot communicates own global position

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Case 2 : helper robot communicates own global position

Communicate Own Current Global Position

Robot 1 (Helper) -> Laser Scanner + Map

Robot 2 (Needy) -> Camera

Needy

Helper

Needy

(Camera)

Helper

(Laser)

Rel Pos

+

Own Pos

Own Pos

asymtre derived schema configurations for case 2

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ASyMTRe-derived schema configurations for case 2

Helper: Laser (ES1)

Needy: Camera (ES2)

PS5

PS5

CS1

CS1

Map

Map

ES1

ES1

PS1

PS1

MS

MS

ES2

ES2

PS2

PS2

PS4

PS4

PS3

PS3

CS2

CS2

case 3 helper robot communicates other robot s global position

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Case 3 : helper robot communicates other robot’s global position

Communicate Other’s Current Global Position

Robot 1 (Helper) -> Laser Scanner + Map and Camera

Robot 2 (Needy) -> nil

Helper

(Camera)(Laser)

Needy

Needy

Helper

Rel Pos

+

Own Pos

Own Pos

asymtre derived schema configurations for case 3

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ASyMTRe-derived schema configurations for case 3

Helper

Needy

PS5

PS5

CS1

CS1

Map

Map

ES1

ES1

PS1

PS1

MS

MS

ES2

ES2

PS2

PS2

PS4

PS4

PS3

PS3

CS2

CS2

research objectives and inspiration

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Research objectives and inspiration
  • Research objectives: To extend ASyMTRe to enable constructivist learning
    • To learn collections of schemas (“chunks”, or “SCS”) constructively, in order to store knowledge from previous search process, and to improve the efficiency for future search
  • Inspiration: Piaget’s child development theory
    • Assimilation: Reorganize existing knowledge and skills to reflect novelties in the environment
    • Accommodation: Modify existing knowledge and skills to adjust to novelties in the environment

PS5

PS5

CS1

CS1

Map

Map

ES1

ES1

PS1

PS1

MS

MS

ES2

ES2

PS2

PS2

PS4

PS4

PS3

PS3

CS2

CS2

key contributions

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Key contributions
  • Facilitates constructivist learning
    • Includes both assimilation and accommodation in the new SB-CoRLA architecture
    • Learns schema chunks
  • Enables more efficient searches
    • Re-uses schema chunks
    • Finds online team solutions more quickly, rather than searching exhaustively over all possible solutions first

First-level chunk

Second-level chunk

agenda18

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Agenda

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

  • Introduction: background, research objectives, and motivation
  • Related work
  • Approach
  • Simulations and results
  • Conclusion and future work

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

SB-CoRLA overview

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

related work

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Related Work
  • Schema Theory
    • Lyons and Arbib (1989 and 2003)
    • Arkin (1998)
  • Information Invariants
    • Donald et al. (1994 and 1997)
  • Constructivist Learning
    • Drescher (1991)
    • Chaput (2004)
  • ASyMTRe
    • F. Tang and Parker (2005 and 2006)
agenda20

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Agenda

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

  • Introduction: background, research objectives, and motivation
  • Related work
  • Approach
  • Simulations and results
  • Conclusion and future work

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

SB-CoRLA overview

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

recall the basic components in asymtre

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Recall the basic components in ASyMTRe
  • Schema
    • Presents basic robot capabilities
    • Categorizes into perceptual schema (PS), motor schema (MS), and communication schema (CS)
  • Information type (i.e. semantic content)
    • Each schema requires and produces information types
    • Inputs and outputs of schemas can be connected if their information types match
  • ASyMTRe automatically connects the schemas to generate a task solution

GPS

Camera

cs1

ps4

ms1

ms1

ps1

cs2

the special terms in sb corla

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

The special terms in SB-CoRLA
  • Sensori-Computational System (SCS)
    • SCS = “Chunk”
    • First-level chunk
    • Second-level chunk
    • Higher-level chunk
  • SCS repository
  • CA: Centralized ASyMTRe
  • RA: Randomized ASyMTRe
  • EL: Evolutionary Learning
  • ECA: Extended Centralized ASyMTRe

First-level chunk example

the sb corla architecture

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

The SB-CoRLA architecture

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

assimilation in sb corla

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Assimilation in SB-CoRLA

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

learning chunks off line

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Learning chunks (off-line)

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

saving chunks off line

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Saving chunks (off-line)

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

using chunks online

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Using chunks (online)

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

find chunks from an el solution

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution29

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution30

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution31

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution32

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution

First-level Chunk 1

find chunks from an el solution33

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution34

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution35

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution36

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution37

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution
find chunks from an el solution38

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Find chunks from an EL solution

First-level Chunk 2

combine chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Combine chunks

First-level Chunk 1

First-level Chunk 2

combine chunks40

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Combine chunks

Second-level Chunk

First-level Chunk 1

First-level Chunk 2

learning chunks off line41

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Learning chunks (off-line)

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

evolutionary learning

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Evolutionary Learning

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

what are the reasons for choosing evolutionary learning for schema chunking

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

What are the reasons for choosing evolutionary learning for schema chunking?
  • Current ASyMTRe search algorithm does not include learning
    • Does not produce schema chunks for constructivist learning
    • Has difficulty discovering certain large team solutions because it uses heuristics to search for small team size solutions first
    • Regenerates each solution from the beginning
  • Evolutionary learning enables constructivist learning
    • Solution evolves with increasing fitness value
    • Learns highly-fit schema chunks
    • Reuses schema chunks in new task assignment
approach for ensuring solution quality compare three search algorithms

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Approach for ensuring solution quality: compare three search algorithms
  • Centralized ASyMTRe search algorithm (CA) – previously developed
    • A two step, anytime algorithm
    • Greedy search that prefers small team size and lower cost solution for individual robot
  • Randomized ASyMTRe search algorithm (RA) – new
    • Similar to CA
    • Randomized search
  • Evolutionary Learning search algorithm (EL) – new
    • Uses genetic algorithm to evolve populations of team solutions
centralized asymtre search algorithm ca

Robot team

R1

R2

R3

Rn

L1

L2

L3

Lk

Potential solutions

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Centralized ASyMTRe search algorithm (CA)

1: for each robot team of n robots with up to m available schemas for each robot and up to p

inputs for each schema

2: generate a list of kpotential solutions (O(nmp))

3: sort the potential solutions in ascending order of costs (O(k log(k)))

cost = wc * (c/cmax) + wp * (1-p)

4: sort the robots in ascending order of available schemas (O(n log(n)))

5: end for

6: for each robot team sequence (O(n!))

7: for each robot in the sequence (O(n))

8: attempt to assign a potential solution to this robot (O(q))

9: if the robot cannot do the task by itself

10: attempt to find another robot that can provide help (O(nq))

11: if all robots can do the task and the cost of the solution is lower than existing solutions

12: record this solution

13: end for

1

2

3

4

randomized asymtre search algorithm ra

Robot team

R1

R2

R3

Rn

L1

L2

L3

Lk

Potential solutions

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Randomized ASyMTRe search algorithm (RA)

1: for each robot team of n robots with up to m available schemas for each robot and up to p

inputs for each schema

2: generate a list of kpotential solutions (O(nmp))

4: end for

5: for each robot team sequence (O(n!))

6: for each robot in the sequence (O(n))

7: attempt to assign a random potential solution to this robot (O(q))

8: if the robot cannot do the task by itself

9: attempt to find another random robot that can provide help (O(nq))

10: if all robots can do the task and the cost of the solution is lower than existing solutions

11: record this solution

12: end for

2

4

3

1

evolutionary learning search algorithm el

S4

S2

S3

S1

S7

S1

S7

S4

S1

S1

Solution1

Solution1’

S5

S6

S2

S3

S5

S6

S2

S3

S2

S5

S6

S5

S6

Solution2

Solution2’

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Evolutionary Learning search algorithm (EL)

1: for each robot team of n robots with up to m available schemas for each robot

2: initialize the first population of size p by randomly connecting the schemas via matching information types [O((nm2+n2)p)]

3: evaluate each individual solution of the p population

4: calculate fitness F = wc·(1-c/cmax) + wx·(1-x/xmax)+ wq·(q/qmax) + wu·(u/n) [O(n2m2)]

5: for gmax generations, perform

6: fitness proportionate selection or tournament selection [O(p)]

7: pair wise single point crossover at crossover rate =γ [O(n2m2p)]

8: single point mutation at mutation rate = δ [O(nmp)]

9: prune solutions and calculate their fitness values [O(n2m2p)]

10: record the solution with the best fitness value

11: stop if no fitness improvement for a pre-defined number of generations

12: end for

el the graph

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the graph
  • A graph, in adjacency list format, is used to represent individual team task solution;
  • Each generation in the EL process contains a number of individual team task solutions, i.e., graphs;
  • Each graph has as many nodes as the number of schemas in the robot team that is current assigned a task;
  • The edges among the graph nodes represents schemas connected with each other via matching information types, and therefore indicate information flows.

ES

Goal

el the process

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the process
  • Initialization
  • Evaluation
  • Selection
  • Crossover
  • Mutation
  • Pruning

ES

Goal

el the process50

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the process
  • Initialization
  • Evaluation
  • Selection
  • Crossover
  • Mutation
  • Pruning

ES

Goal

el initialization

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: initialization
  • Record all valid schema connections
  • For each valid schema connection, randomly connect the schemas according to:
    • Inter-robot connect rate for connections among robots
    • Intra-robot connect rate for connections within a robot

R1

R2

laser

cs1

ps3

ms1

ms1

cs2

ps1

el the process52

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the process
  • Initialization
  • Evaluation
  • Selection
  • Crossover
  • Mutation
  • Pruning

ES

Goal

el evaluation

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: evaluation
  • Delete cycles in the graph
  • Calculate
    • The costs of active schemas
    • The number of active connections between schemas
    • The number of provided information types
    • The number of assigned robots
  • Calculate the fitness value
el fitness function

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: fitness function

F = wc·(1-c/cmax) + wx·(1-x/xmax) + wq·(q/qmax) + wu·(u/n)

Costs

Complexity

Robots

Information types

el the process55

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the process
  • Initialization
  • Evaluation
  • Selection
  • Crossover
  • Mutation
  • Pruning

ES

Goal

el crossover

S2

S3

S4

S1

S1

S7

S1

S1

S2

S7

S4

S3

Solution1

Solution1’

S5

S6

S5

S6

S2

S2

S3

Solution2’

Solution2

S5

S5

S6

S6

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: crossover

Crossover point

el the process57

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the process
  • Initialization
  • Evaluation
  • Selection
  • Crossover
  • Mutation
  • Pruning

ES

Goal

el mutation

S2

S2

S3

S3

S7

S1

S1

S1

S7

S1

S4

S4

S5

S5

S6

S6

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: mutation

Solution1

Solution1’

Mutation point

S2

S3

S2

S3

Solution2

Solution2’

S5

S6

S5

S6

el the process59

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: the process
  • Initialization
  • Evaluation
  • Selection
  • Crossover
  • Mutation
  • Pruning

ES

Goal

el pruning

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning61

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning

R1

R2

  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

laser

cs1

cs2

ps3

ms1

ms1

cs1

ps1

cs2

ES

Goal

el pruning62

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

x

Goal

el pruning63

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning64

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning65

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning66

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning67

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning68

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning69

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning70

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning71

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning72

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning73

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

el pruning74

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL: pruning
  • Pruning is used to select the best solution from each generation
  • Delete cycles in the graph
  • Delete incomplete information flows
  • Delete information flows that do not lead to goal

ES

Goal

parameter settings for el ga related

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Parameter settings for EL: GA related
parameter settings for el fitness related

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Parameter settings for EL: fitness related

F = wc·(1-c/cmax) + wx·(1-x/xmax) + wq·(q/qmax) + wu·(u/n)

other parameter settings for el

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Other parameter settings for EL
harvesting chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Harvesting chunks

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

harvesting first level chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Harvesting first-level chunks
  • Extract first-level chunks that output one single information type from the best solution generated by EL
  • Delete duplicate first-level chunks (unique first-level chunks remain)
  • Sort unique first-level chunks
    • In ascending order of costs, the number of active schemas, and the number of robots
example of a first level chunk

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Example of a first-level chunk
eca online solution search using chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ECA: online solution search using chunks

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Chunks

Assimilation/Chunking

eca builds second level chunks using first level chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ECA builds second-level chunks using first-level chunks
  • Combine first-level chunks to generate second-level chunks for each robot type in the team and for each required information types in the task
  • Delete duplicate second-level chunks (unique second-level chunks remain)
  • Sort unique second-level chunks
    • In ascending order of costs, the number of active schemas, and the number of robots
example of first level chunks and their combination second level chunk

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Example of first-level chunks and their combination second-level chunk
eca searches solutions online using second level chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ECA searches solutions online using second-level chunks
  • Sort robot team according to robot types
  • Repeat for all permutations of robot types
    • For each robot, assign the best second-level chunk
    • Generate team costs based on active schemas
  • The end solution is a collection of updated second-level chunks with active schema list.

Robot team

RT1

RT2

RT1

RTm

C1

C2

Ck1

C1

C2

Ck2

Chunks

limitations of the chunking process

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Limitations of the chunking process
  • Parameter settings in the EL process requires expertise from the user
  • Does not include higher-level chunks beyond first-level and second-level chunks
  • Cannot handle unknown information types and unknown robot types
limitations of the chunking process86

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Limitations of the chunking process
  • Parameter settings in the EL process requires expertise from the user
  • Does not include higher-level chunks beyond first-level and second-level chunks
  • Cannot handle unknown information types and unknown robot types
hybrid process

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Hybrid process

Team task solution

Team

configuration

Information types

and schemas

Robot Team

Offline

Evolutionary

Learning (EL)‏

Parsing

Task

Task definition

EL Solution

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

CA

RA

Online solution search

ECA

Chunks

Assimilation/Chunking

agenda88

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Agenda

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

  • Introduction: background, research objectives, and motivation
  • Related work
  • Approach
  • Simulations and results
  • Conclusion and future work

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

SB-CoRLA overview

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

simulated applications

A

B

C

D

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Simulated applications
  • A: Multi-Robot

Transportation

  • B: Box Pushing
  • C: Robot Formation
  • D: Limited Resource
testing objectives

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
testing objectives91

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
time breakdown for ca ra and chunking

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Time breakdown for CA, RA, and chunking
el needs more pre processing time than ca and ra for applications a b and c

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL needs more pre-processing time than CA and RA, for applications A, B, and C

Similar results for

application A and C

el needs less pre processing time than ca and ra for application d

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL needs less pre-processing time than CA and RA, for application D
eca and ra need less time than ca to find the first solution for applications a b and c

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ECA and RA need less time than CA to find the first solution, for applications A, B, and C

Similar results for

application A and B

eca needs less time than ca and ra to find the first solution for application d

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ECA needs less time than CA and RA to find the first solution, for application D
simulation findings

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Simulation findings

EL needs less pre-processing time than CA and RA for complicated applications

ECA is always faster than CA and RA in online search

ECA can find solution where CA and RA cannot

testing objectives98

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
time breakdown for ca ra and chunking99

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Time breakdown for CA, RA, and chunking
time to evolve one generation in el

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Time to evolve one generation in EL
harvesting needs very little time 0 01 second

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Harvesting needs very little time (<0.01 second)

Similar results for

application B, C, D

simulation findings102

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Simulation findings

The graph confirms that EL needs quadratic time for generating one generation

EL needs very little time for harvesting

testing objectives103

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
el improves the solution fitness

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL improves the solution fitness

Similar results for

application B, C, D

el increases the number of assigned robots

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL increases the number of assigned robots

Similar results for

application B, C, D

el complexity development

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL complexity development

Similar results for

application B, C, D

el cost development

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

EL cost development

Similar results for

application B, C, D

simulation findings108

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Simulation findings

EL is able to improve the solution fitness over time

testing objectives109

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
eca and ca generates solutions with less costs than ra for applications a and b

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

ECA and CA generates solutions with less costs than RA, for applications A and B

Similar results for

application B

slide111

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Chunks extracted from incomplete EL solutions can be used in ECA to create complete robot team solutions

Similar results for

application B, C, D

simulation findings112

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Simulation findings

 Chunks extracted from EL solution can be used to generate comparable team task solutions to CA and better than RA

testing objectives113

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
chunking vs asymtre advantages and disadvantages

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Chunking vs. ASyMTRe, advantages and disadvantages
  • Chunking generates second-level chunks for each robot type, while ASyMTRe generates potential solutions for each individual robot;
  • Chunking harvests and reuses first-level chunks, while ASyMTRe generates new potential solutions from scratch;
  • Chunking uses EL to evolve highly fit chunks, while ASyMTRe either uses greedy search (CA) or randomized search (RA);
  • Chunking can find solutions in case when ASyMTRe (CA) cannot. However, often CA can generate good solution fast.
testing objectives115

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Testing objectives
max no improvement 200 instead of default value 20

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

MAX_NO_IMPROVEMENT = 200 (instead of default value 20)
more generations do not necessarily enhance the solution quality

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

More generations do not necessarily enhance the solution quality

Similar results for

application A, C, D

mutation rate makes a difference

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Mutation rate makes a difference
simulation findings119

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Simulation findings

 EL is sensible towards different parameter settings

 The default parameter settings work well

overall simulation findings

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Overall simulation findings
  • EL needs less pre-processing time than CA and RA for complicated applications;
  • EL is able to increase the solution fitness over time;
  • Chunks extracted from EL solution can be used to generate team task solutions comparable to CA and better than RA;
  • ECA is always faster than CA and RA in online search;
  • ECA can find online solutions where CA and RA cannot.

The chunking process finds high-quality online solutions more

quickly by reusing learned chunks, hence provides a valid

foundation for continuous learning in a schema-based robot

system.

agenda121

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Agenda

Information

types, schemas,

and evaluation

criteria

Team task solution

Team task

solution

Team

configuration

Information types

and schemas

Robot Team

Feedback

Offline

Evolutionary

Learning (EL)‏

  • Introduction: background, research objectives, and motivation
  • Related work
  • Approach
  • Simulations and results
  • Conclusion and future work

Parsing

Task

Online

Goal-Directed

Feedback-Based

Learning

Task definition

EL Solution

SB-CoRLA overview

Harvesting

Information types

and schemas

General

SCS Repository

Chunks

Schemas

Chunks

Evaluation

New

schemas

and chunks

Available

schemas

and chunks

Information types

and schemas

Specific

SCS Repository

Online

Solution Searching (ECA)‏

Schemas

and chunks

Chunks

Accommodation

Assimilation/Chunking

recall the goal of my research

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Recall the goal of my research
  • To develop an architecture that enables continuous robot learning
  • To extend the existing ASyMTRe architecture to enable constructivist robot learning
    • A method to learn new knowledge and skills based upon past experience
  • To explore the robot team solution search problem in a different way
    • Task allocation problems is an NP-hard search problem
conclusion

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Conclusion
  • SB-CoRLA includes both assimilation (implemented) and accommodation (to be implemented)
    • Based on ASyMTRe
    • Enables continuous learning
  • Chunking includes three new algorithms that enables constructivist robot learning
  • EL explores the search space in a different way than CA using genetic algorithm
future work

Task

Robot team

Indexing system

SCS repository

……

Relevant

schema chunks

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Future work
  • Developing SCS repository with an indexing system
    • Generating higher-level chunks
    • Developing re-usable chunks
  • Including human knowledge
  • Gaining more insight for parameter settings
  • Implementing accommodation
motivational example for constructivist learning

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Motivational example for constructivist learning
  • Consider three different, yet related robot behaviors:
    • Follow: keep leader in view, while remaining in close proximity
    • Track: keep object in view, no need to move constantly to keep close
    • Shadow: keep object in view, while avoiding being detected by object
  • All three behaviors need similar skills, while having their own specific need for different skills.
accommodation in sb corla

Agenda

Intro

Related Work

Approach

Simulation

Conclusion

Accommodation in SB-CoRLA