learning interaction protocols through imitation a data mining approach n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Learning Interaction Protocols through imitation A data mining approach PowerPoint Presentation
Download Presentation
Learning Interaction Protocols through imitation A data mining approach

Loading in 2 Seconds...

play fullscreen
1 / 67

Learning Interaction Protocols through imitation A data mining approach - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

Artificial Intelligence, Adv (E) (2013). Do not distribute beyond this class. Learning Interaction Protocols through imitation A data mining approach. Yasser Mohammad Nishida Lab. Situated Modules. Used in many systems until now mainly with Robovie Situated modules are executed in serial.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Learning Interaction Protocols through imitation A data mining approach' - kalila


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
learning interaction protocols through imitation a data mining approach

Artificial Intelligence, Adv (E) (2013)

Do not distribute beyond this class

Learning Interaction Protocols through imitationA data mining approach

Yasser Mohammad

Nishida Lab.

situated modules
Situated Modules
  • Used in many systems until now mainly with Robovie
  • Situated modules are executed in serial

[Ishiguro et al. 1999]

route guidance listener 2006
Route Guidance Listener (2006)

Structure

Adjustment

Redesign

Tune/Adapt (Supervised)

Parameter

Adjustment

Analyze Human Human Interactions

Implement Model

Evaluate Model

Model

Controller

2 WOZ experiments using motion captured data

[Kanda et al. 2007]

engineering vs learning approaches
Engineering vs. Learning Approaches

Standard Engineering Approach

Structure

Adjustment

Redesign

Tune/Adapt (Supervised)

Parameter

Adjustment

Analyze Human Human Interactions

Implement Model

Evaluate Model

Model

Controller

Learning/Imitation Approach

Adapt (Unsupervised)

Parameter &

Structure Adjustment

Collect Human Human Interactions

Develop

Interact

Training Data

Controller

example scenarios
Example Scenarios

Gaze Control During Listening

Implicit

Guided Navigation

Explicit

bird s eye view

Model of Actions

Communication Protocol

Model of Commands

Bird’s Eye View

Shared ground

Learned models

and protocol

Adapted models

and protocol

models

and protocol

models

and protocol

Watch

Mimic

Interact

Adapt

Learner Robot

Co-action

Learned action

Adapted actions

action

action

Primordial Knowledge

Model

Our Long Term Model

Watch External Behavior

Learn Actions’ Model

Learn Commands’ Model

Learn Communication Protocol

Commands

Feedback

  • Main Insights
  • Learning By Watching is Ubiquitous in humans
  • Learning Actions and Commands are related
  • Change in Behavior is what matters

Actions

Operator

Actor

Learner

Interaction

basic architecture
Basic Architecture

Activation Level

Execution Time

Behavioral Influence

design procedure
Design Procedure

Structure

Adjustment

Redesign

Tune/Adapt (Supervised)

Parameter

Adjustment

Analyze Human Human Interactions

Implement Model

Evaluate Model

Model

Controller

Redesign

Evaluate

Structure

Adjustment

Analyze Task & Required Basic Actions

Decide Required Behavior

(H-H Interactions)

Learn Parameters

(FPGA)

Intentions

Processes

example gaze control during listening
Example: Gaze Control during Listening

Sensors

Perception Processes

Behavior Processes

Intentions

floating point genetic algorithm
Floating Point Genetic Algorithm

Select 2 individuals and generate 4:

Calculate probability of passing:

Crossover

1. Calculate probabilities over 1~m:

2. Calculate P(mutation@ k) as:

3. Select mutation site  according to P(mutation @ k)

4. Mutate parameter  using:

Mutation

Eliting

Tournament

Cross Over

Mutation

fpga preliminary evaluation
FPGA – Preliminary Evaluation
  • Fitness function:
  • 100 generations
  • 100 individuals
  • Two comparison algorithms

Proposed>A1

p=0.0133

Proposed>A2

p=0.0032

[Mohammad & Nishida 2010d]

applications gaze control
Applications – Gaze Control
  • Fixed Structure Gaze Controller (18 parameters)
  • Dynamic Structure Gaze Controller (7 parameters)

[Mohammad & Nishida 2010d]

applications gaze control1
Applications – Gaze Control
  • Fixed vs. Dynamic Structure GC
    • Six novel sessions
    • Four control GCs
      • Follow
      • Stare
      • Random

[Mohammad & Nishida 2010d]

learning by watching imitation mimicry
Learning by watching/imitation/mimicry

2. Learn

1. Watch

Command stream

Action stream

Interaction Protocol

Discovery Phase

Constrained Motif Discovery

Learner

Commands

Discrete Commands

Discrete Actions

320000310

23340003204402

Feedback

Actions

Association Phase

Baysian Network Induction

Operator

Actor

learned

Interaction Protocol

Behavior Generation Model

offline

3. Act

Commands

Piecewise Linear Controller Gen.

Controller Generation

Feedback

Actions

Robot/Agent Controller

Feedback Controller

online

Operator

Learned Actor

building blocks
Building Blocks
  • Behavior Discovery
    • Motif Discovery
      • Change Point Detection
  • Behavior Association
    • Bayesian Network Induction
      • Causality Analysis
  • Behavior Generation
    • Piecewise Linear Controller Generation
  • Behavior Adaptation
    • Bayesian Network Combination
gaze control data collection experiment
Gaze Control:Data Collection Experiment
  • 44 participants
    • ages 19-37 (27% females)
    • Untrained to interact with robots
  • Two objects (chair/stepper)
    • Easily assembled (7 steps both)
    • Not so easy (2 ordering steps both)
  • Two roles:
    • Instructor: explains about a single object three times:
      • Good listener
      • Bad listener
      • Robot
    • Listener: listens to two explanations about two objects:
      • Good listener
      • Bad listener
gaze control evaluation experiment
Gaze Control:Evaluation Experiment
  • Internet poll
  • 35 subjects
  • Watch 2 videos:
    • ISL learned controller
    • carefully designed controller
  • Age (ranged from 24 to 43 with an average of 31.16 years).
  • Gender (8 females and 30 males).
  • Experience in dealing with robots (ranged from I never saw one before to I program robots routinely).
  • Expectation of robot attention in a range from 1 to 7 (4 +-1.376).
  • Expectation of robot's behavior naturalness in a range from 1 to 7 (3.2 +-1.255).
  • Expectation of robot's behavior human-likeness in a range from 1 to 7 (3.526 +-1.52).
building blocks1
Building Blocks
  • Behavior Discovery
    • Motif Discovery
      • Change Point Detection
  • Behavior Association
    • Bayesian Network Induction
      • Causality Analysis
  • Behavior Generation
    • Piecewise Linear Controller Generation
  • Behavior Adaptation
    • Bayesian Network Combination
1 behavior discovery
(1) Behavior Discovery

Proposed

Advantages

  • Utilizes relation between actions and commands
  • removes irrelevant dimensions
  • No need for separate clustering step
  • No predefined model

Command Stream

Action Stream

Robust Singular

Spectrum Transform

Discover

Change Points

Discover

Change Points

Granger-Causality

Maximization

Natural Delay Discovery

X

Constrained

Motif Discovery

Discover Motifs

Discover Motifs

Remove Irrelevant

Dimensions

Remove Irrelevant

Dimensions

motif discovery
Motif Discovery
  • Given a timeseries (an ordered list of real numbers), find approximately recurring subsequences

Chiu 2013

motif discovery1
Motif Discovery
  • Given a time series X(t) find recurring patterns of length L using distance function D
catalano s algorithm

Motif

Catalano’s Algorithm

candidate

noise

comparison

Compare

Keep top k rather than best only

[Catalano 2006]

constrained motif discovery
Constrained Motif Discovery
  • Given a time series X(t) find recurring patterns of length between L1 and L2 using distance function D

subject to the constraint P(t), where P(t) is an estimation of the probability that a motif occurrence exists near time step t.

A motif is likely near here

dgcmd
DGCMD

Signal

Tcon

Constraint

[Mohammad & Nishida 2009]

dgcmd1
DGCMD
  • Advantages:
    • Controlled Exhaustiveness (# candidates).
    • Controlled Sensitivity (Tc).
    • No random subwindow as needed by some MD algorithms.
    • No upper bound on motif size as needed by most MD algorithms.
  • Disadvantages:
    • Can become quadratic if # candidates is large.
    • Sensitive to outlier segments (long subwindows of outliers).
dgcmd evaluation
DGCMD – Evaluation
  • 50440 time series
  • Variable length (102~106)
  • Variable noise level (0~20%PP)
  • Variable motif types
  • Variable # of occurrences
  • Motif Discovery Algorithms:
    • Projections (most accurate)
    • Catalano et al. (fastest)
  • Constrained Motif Discovery Alg.:
    • MCFull
    • MCInc
    • DGCMD

[Mohammad & Nishida 2009]

how good is the constraint
How good is the constraint?

Probability of discovering a motif

- not using the constraint

- using the constraint

Relative entropy between constraint

and motif locations

Entropy of the constraint

Number of motif occurrences

Window length

Average motif length

Time series length

[Mohammad & Nishida 2010a]

how to get the constraint
How to get the constraint?
  • Main insight
    • The generating dynamics change near the beginning and end of motifs.
    • We need to find points in the time series where generating dynamics change
building blocks2
Building Blocks
  • Behavior Discovery
    • Motif Discovery
      • Change Point Detection
  • Behavior Association
    • Bayesian Network Induction
      • Causality Analysis
  • Behavior Generation
    • Piecewise Linear Controller Generation
  • Behavior Adaptation
    • Bayesian Network Combination
change point discovery
Change Point Discovery
  • Given a time series X(t) find for every time step the probability that X(t) is changing form (underlying dynamics are changing!!)
available techniques
Available Techniques
  • CUMSUM
    • Detects only mean change
  • Inflection Point Detection
    • Assumes any variation is a change!!
  • Autoregressive Modeling
    • Assumes a specific generating model
  • Mixtures of Gaussians
    • Assumes a specific generating model
  • Discrete Cosine Transform
    • Finds only global changes
  • Wavelet Analysis
    • Tons of parameters
  • Singular Spectrum Transform (SST) [Ide et al. 2005]
    • Most General, no ad-hoc adjustment
main idea
Main idea
  • At every point
    • Use few values before it to represent the past: H
    • Use few values after it to represent the future: G
    • Compare the past with the future. The more dissimilar, the highest the score

H is a hyper plan

Future

Past

G

H

G is a set of Eigen vectors

singular spectrum transform
Singular Spectrum Transform

Parameters

w

n

Future

Past

g,

m

G

H

l

Future

Change

Angle

[Ide et al. 2005]

numeric example
Numeric Example
  • X(t)={-4,-3,-2,-1,0,1,2,3,4,-1,1,-1,1,-1,1,-1,1}
  • Parameters:
    • w=g=4,n=m=2,l=1
  • At t=6

Future

SVD

Change

Angle

numeric example continued
Numeric Example (Continued
  • X(t)={-4,-3,-2,-1,0,1,2,3,4,-1,1,-1,1,-1,1,-1,1}
  • Parameters:
    • w=g=4,n=m=2,l=1
  • At t=6

Future

Change

Angle

numeric example1
Numeric Example
  • X(t)={-4,-3,-2,-1,0,1,2,3,4,-1,1,-1,1,-1,1,-1,1}
  • Parameters:
    • w=g=4,n=m=2,l=1
  • At t=10

Future

SVD

Change

Angle

numeric example continued1
Numeric Example (Continued
  • X(t)={-4,-3,-2,-1,0,1,2,3,4,-1,1,-1,1,-1,1,-1,1}
  • Parameters:
    • w=g=4,n=m=2,l=1
  • At t=6

Change

Angle

singular spectrum transform1
Singular Spectrum Transform
  • Advantages:
    • No predefined generation model.
    • Comparably few parameters (5).
    • PCA using SVD works for ANY matrix so no ad-hoc preprocessing is needed.
    • Linear in the length of the time series.
  • Disadvantages
    • Still there are 5 parameters hard to select.
    • Specificity degrades very fast with increased noise level.
    • Inadequate for time series with no background signal.
robust singular spectrum transform
Robust Singular Spectrum Transform

Parameters

w,n

Future

Past

G

H

Future

Change

Angles

[Mohammad & Nishida 2009b]

rsst vs sst real world data
RSST vs. SST – Real world data
  • Explanation Scenario
  • 22 participants
  • 3 conditions:
    • Natural listening
    • Unnatural listening
    • Robot
  • Physiological Sensors:
    • Respiration
    • Skin Conductance
    • Pulse
building blocks3
Building Blocks
  • Behavior Discovery
    • Motif Discovery
      • Change Point Detection
  • Behavior Association
    • Bayesian Network Induction
      • Causality Analysis
  • Behavior Generation
    • Piecewise Linear Controller Generation
  • Behavior Adaptation
    • Bayesian Network Combination
behavior association
Behavior Association

After Discovering Basic Motifs in both actions and commands and detecting their occurrence in all time series as in this graph

Command 1

Action 1

use the natural delay between commands and actions calculated during the discovery phase.

For every command-action pair calculate the joint-activation of them by the number of occurrences of the action within the natural delay interval of the command.

Use the joint-activation values to induce a Baysian Network describing the relation between actions and commands

Mohammad & Nishida 2009

causality based delay estimation
Causality Based Delay Estimation

To find delay between and

Regress actions using actions & gestures

Regress actions using actions only

Compare residues

Calculate g-causality statistic

Find the delay that maximizes g-causality

[Mohammad & Nishida 2009c]

example associating actions and gestures
Example: Associating Actions and Gestures
  • Guided Navigation Scenario

Correct prediction 95.2%.

[Mohammad & Nishida 2009c]

building blocks4
Building Blocks
  • Behavior Discovery
    • Motif Discovery
      • Change Point Detection
  • Behavior Association
    • Bayesian Network Induction
      • Causality Analysis
  • Behavior Generation
    • Piecewise Linear Controller Generation
  • Behavior Adaptation
    • Bayesian Network Combination
behavior controller generation
Behavior Controller Generation

Convert the Baysian Network learned into L0EICA controller

Command  Process & Action  Process & Link  Effect Channel

Motor Babbling

PLGC

[Mohammad & Nishida 2010c]

motor babbling
Motor Babbling

generate a straight line in one dimension while minimizing disturbance

to all others

Mohammad & Nishida 2010c

slide53
PLGC

[Mohammad & Nishida 2010c]

building blocks5
Building Blocks
  • Behavior Discovery
    • Motif Discovery
      • Change Point Detection
  • Behavior Association
    • Bayesian Network Induction
      • Causality Analysis
  • Behavior Generation
    • Piecewise Linear Controller Generation
  • Behavior Adaptation
    • Bayesian Network Combination
accumulation phase
Accumulation Phase

e

a

d

b

f

c

e

a1

d4

b2

f5

1

4

c 3

6

2

5

3

6

abn combination
ABN Combination
  • Main assumption
    • Action nodes are more compatible than gesture nodes
  • Algorithm
    • Associate action nodes with similar stored pattern
      • Set of action node association links
    • Associate gesture nodes with similar stored pattern
      • Set of action node association links
    • Calculate Link Competence Index for association links
      • Set of LCIs for gestures and actions
    • Resolve association link conflicts using LCIs
      • Final ABN
associating action gesture nodes
Associating action/gesture nodes
  • Compile AN1 and AN2 lists {every action node}
  • Calculate
  • Calculate for all nodes and order them
  • Create a link iff

for any

  • Set

Gesture association links are calculated the same way

[Mohammad & Nishida 2010c]

lci calculation
LCI Calculation

[Mohammad & Nishida 2010c]

guided navigation
Guided Navigation
  • Roles: Actor & Operator
  • Protocol: explicit
  • Nonverbal Behaviors:
    • Operator’s Gesture
    • Actor’s motion
  • Sensors
    • Accelerometers (BPACK)
    • Motion Capture (PhaseSpace)
  • Procedure
    • Offline experiment
    • Online experiment

Guided Navigation

  • Task Oriented
  • Explicit Protocol
  • 1 way Interaction

[Mohammad and Nishida 2009c]

guided navigation online experiment
Guided Navigation – Online experiment
  • 18 subjects (6 days)
  • Task: operator in GN scenario
  • Procedure
    • WOZ session (training & familiarization)
    • 3 Sessions on these conditions:
      • WOZ
      • Per-participant learner
      • Accumulating learner

[Mohammad and Nishida 2010c]

experimental setup
Experimental Setup

[Mohammad and Nishida 2009c]

action and gesture streams
Action and Gesture Streams

Action Stream

5D

Gesture Stream

6D

guided navigation online examples1
Guided Navigation – Online Examples
  • Number of failures:
    • Per-Participant  1/18
    • Accumulating  4/17
conclusions
Conclusions
  • Unsupervised learning of interaction protocols is possible using three main data mining technologies:
    • Motif discovery <The Heart>
    • Change point discovery <The speeding engine>
    • Causality analysis <Natural delays>
  • Several algorithms for solving these three problems were introduced and are available (among others) in source code as a MATLAB toolbox called CPMD
  • We have shown that it is possible to learn both implicit interaction protocols (gaze control) and explicit interaction protocols (guided navigation) without explicit modeling
  • By manipulating learned BNs, it is possible to improve the interactive behavior of agents over time based on interactions with multiple people
references
References

[Ishiguro et al. 1999] Ishiguro, H.; Kanda, T.; Kimoto, K.; Ishida, T., "A robot architecture based on situated modules," Intelligent Robots and Systems, 1999. IROS '99. Proceedings. 1999 IEEE/RSJ International Conference on , vol.3, no., pp.1617,1624 vol.3, 1999

[Catalano 2006] Joe Catalano, Tom Armstrong, and Tim Oates. Discovering patterns in real-valued time series. In Knowledge Discovery in Databases: PKDD 2006, pages 462–469, 2006.

[Chiu 2003] B. Chiu, E. Keogh, and S. Lonardi, “Probabilistic discovery of time series motifs,” in KDD ’03: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. New York, NY, USA: ACM, 2003, pp. 493–498.

[Ide 2005] T. Ide and K. Inoue, “Knowledge discovery from heterogeneous dynamic systems using change-point correlations,” in Proc. SIAM Intl. Conf. Data Mining, 2005.

[Kanda et al. 2007] Kanda, T., Kamasima, M., Imai, M. et al. “A Humanoid Robot That Pretends to Listen to Route Guidance from a Human”, Auton. Robots, Vol. 22, Number 1, pages 87-100, 2007.

[Mohammad and Nishida 2009a] Yasser Mohammad, Toyoaki Nishida, Shogo Okada, “Unsupervised Simultaneous Learning of Gestures, Actions and their Associations for Human-Robot Interaction," . IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009, pp.2537-2544, 11-15 Oct. 2009

[Mohammad and Nishida 2009b] Yasser Mohammad and Toyoaki Nishida, Robust Singular Spectrum Transform, The Twenty Second International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA/AIE 2009), June 2009, Taiwan, pp 123-132.

[Mohammad and Nishida 2009c] Yasser Mohammad and Toyoaki Nishida, Measuring Naturalness During Close Encounters Using Physiological Signal Processing, The Twenty Second International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems (IEA/AIE 2009), June 2009, Taiwan, pp. 281-290

[Mohammad and Nishida 2009d]] Yasser Mohammad and Toyoaki Nishida, Using Physiological Signals to Detect Natural Interactive Behavior, Applied Intelligence, 13(1) 79-92

[Mohammad 2009 PhDThesis] Yasser Mohammad, Autonomous Development of Natural Interactive Behavior for Robots and Embodied Agents, PhD Thesis, Kyoto University, September 2009

[Mohammad and Nishida 2010a] Yasser Mohammad, Toyoaki Nishida, Learning Interaction Protocols using Augmented Baysian Networks Applied to Guided Navigation, Taipei, Taiwan, IROS 2010.

[Mohammad and Nishida 2010c] Yasser Mohammad, Toyoaki Nishida, Learning Interaction Protocols using Augmented Baysian Networks Applied to Guided Navigation, Taipei, Taiwan, IROS 2010.

[Mohammad and Nishida 2010d] Yasser Mohammad and Toyoaki Nishida, Controlling Gaze with an Embodied Interactive Control Architecture, Applied Intelligence, Vol. 32, No. 2, 2010, pp 148-163