learning and remapping in the sensory motor system l.
Skip this Video
Loading SlideShow in 5 Seconds..
Learning and Remapping in the Sensory Motor System PowerPoint Presentation
Download Presentation
Learning and Remapping in the Sensory Motor System

Loading in 2 Seconds...

play fullscreen
1 / 43

Learning and Remapping in the Sensory Motor System - PowerPoint PPT Presentation

  • Uploaded on

Learning and Remapping in the Sensory Motor System. F.A. Mussa-Ivaldi Northwestern University. In natural arm movements, complex motions of the joints lead to simple hand kinematics. From Morasso 1981. Is the shape of trajectories the byproduct of dynamic optimization?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Learning and Remapping in the Sensory Motor System' - infinity

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
learning and remapping in the sensory motor system

Learning and Remapping in the Sensory Motor System

F.A. Mussa-Ivaldi

Northwestern University

in natural arm movements complex motions of the joints lead to simple hand kinematics
In natural arm movements, complex motions of the joints lead to simple hand kinematics

From Morasso 1981


Is the shape of trajectories the byproduct of dynamic optimization?

(Uno, Kawato and Suzuky, 1989)

dynamic adaptation
Dynamic Adaptation

Unperturbed movements

Shadmehr & Mussa-Ivaldi, 1994



Initial response

Post-adaptation kinematics


The shape of trajectories is not a by-product of dynamic optimization.

It reflects and is consistent with the Euclidean properties of ordinary space

what is space
What is space ?
  • Intuitive Concept

Geometry, from a physical standpoint is the totality of the laws according to which rigid bodies mutually at rest can be placed with respect to each other… “Space” in this interpretation is in principle an infinite rigid body – or skeleton - to which the position of all other bodies is related.”

A. Einstein, 1949

  • Less Intuitive Concepts

Signals spaces, configuration spaces, state spaces, feature spaces, etc...

ordinary space




Ordinary Space

Two equivalent statements

  • Euclid
  • Pythagoras
euclidean symmetry

Euclidean (L2) Norm

Orthogonal transformation

Euclidean Symmetry

Among all possible norm only the L2 (Euclidean) norm is invariant under rigid transformations (e.g. rotations, translations).

Size does not depend upon position and orientation.

signal spaces
Signal spaces
  • Neither the visual nor the motor signals spaces are Euclidean.
  • Yet, we perceive Euclidean symmetry and we coordinate motor commands to control independently movement amplitude and direction

It is a common misconception that the visual system provides the primal information about this symmetry.

  • In fact it is likely for the opposite to be true: we learn the properties of space through active perception
Is the bias toward Euclidean symmetry a guiding principle for the emergence of new motor programs?
the motor system can reassign degrees of freedom to novel tasks
The motor system can reassign degrees of freedom to novel tasks

Glove Talk. Sidney Fels and Geoffrey Hinton, 1990



The properties of space are captured by the motor system when forming novel motor programs whose purpose is to control the motion of an endpoint

Remapping Space


(Hand Configuration)

2 Screen Coordinates

F.A. Mussa-Ivaldi, S. Acosta, R. Scheidt, K.M. Mosier

the task
The task
  • The subject must place the cursor corresponding to the hand inside a circular target
  • Experiment begins with the cursor inside a target.
  • New target is presented – Cursor is suppressed.
  • The subject must try to make a single rapid movement of the fingers so as to place the (invisible) cursor inside the new target and stop.
  • As the hand is at rest after this initial movement, the cursor reappears
  • The subject corrects by moving the cursor under visual guidance inside the target
  • Only the initial open loop movement is analyzed
some examples
Some examples

Part 2

Part 1





some observations
Some observations
  • Weird task. Cannot be represented before the experiment
  • Body and task endpoint are physically uncoupled
  • Only feedback pathway is vision
  • Dimensional imbalance
  • Metric imbalance

Aspect-Ratio: AB/SE



Single Subject





  • Cursor movements DO NOT become straighter !
extended training
Extended training
  • 4 consecutive days
  • 2 groups of subjects
  • No-Vision subjects (P2) just engage in the task for 4 days
  • Vision subjects (P3) train with continuous visual feedback trials. But they are tested in the same no-vision condition as the other group.
  • The two groups receive the same amount of training
extended training results
Extended Training Results
  • Both group have the same learning trend.
  • Continuous visual feedback of endpoint movements facilitates the formation of straighter open-loop trajectories

Error reduction and straightening of motion are independent processes

  • Straightening of motion is facilitated when vision and movement execution are concurrent

The subjects learn to solve an ill-posed inverse problem (from cursor location to hand gesture) and in doing so, they learn the geometrical structure of the map, i.e. a specific partition of degrees of freedom.

task space and null space motion
Task-space and Null-space Motion





practical applications body machine interfaces bmi
Practical Applications:Body-Machine Interfaces (BMI)


  • Identify Residual Control Signals (Movements, EEG, EMG, Multiunit…)
  • Form a first “vocabulary” of commands (faster, slower, right, left, MIDI, etc…)
  • Machine learns to respond appropriately
  • Patient practice and leans new commands



Sensing garments (Smartex -Pisa)

  • 52 smeared sensors on lycra
  • No metallic wires
  • External connection realized by metallic press stud
the interactive lms procedure 1
The interactive LMS procedure - 1

1)Set an arbitrary starting map:

Endpoint vector dim(x) =2

“Body” vector dim(h)=20

the interactive lms procedure 2
The interactive LMS procedure - 2

2) Collect movements toward a set of K targets:


Movements (endpoint coordinates)

Movements (body coordinates)

the interactive lms procedure 3
The interactive LMS procedure - 3

3) Calculate the error (in endpoint coordinates)

For each movement

For a whole test epoch

the interactive lms procedure 4
The interactive LMS procedure - 4

4) Calculate the gradient of the error (w.r.t. A)

5) Update the A-matrix in the direction of the gradient

4) Repeat from step 2 until convergence.


Sensory-Motor Cognition IS NOT an oxymoron !