Feedback, Adaptation, Learning or Evolution: How Does the Brain Coordinate and Time Movements?

1. Feedback, Adaptation, Learning or Evolution: How Does the Brain Coordinate and Time Movements? Amir Karniel Department of Biomedical Engineering Ben Gurion University of the Negev The studies presented were done in collaboration with: Gideon Inbar, Ronny Meir, and Eldad Klaiman - Technion Sandro Mussa-Ivaldi - Northwestern University The first workshop of THE CENTER FOR MOTOR RESEARCH December 18-21, 2003 Jerusalem in Motion

2. Outline • The Hierarchy of Wide Sense AdaptationEquilibrium trajectories and internal models Reaching movements muscle models and adaptation • Adaptation to Force Perturbations Time representation Sequence learning and switching • Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal models • Summary and Future Research Jerusalem in Motion

3. yd x y F(x) F-1(yd) Two Important Concepts in the Theory of Motor Control Equilibrium Inverse Model Feldman Bizzi et al. +Minimum Jerk, Flash and Hogan +Force fields, primitives, Mussa-Ivaldi Albus (cerebellum) Inbar and Yafe (signal adaptation) +feedback error, Kawato +distal teacher, Jordan Adaptation Change of ImpedanceChange of the inverse Jerusalem in Motion

4. MJT Reaching movements • Feed-Forward Control • Invariant Features: Roughly straight line, bell shaped speed profile (Flash & Hogan 1985) • Key Questions • What is the origin of the invariance ? • How do we handle external perturbations ? Jerusalem in Motion

5. A Hill-type mechanical muscle model The viscose element B is not a constant ! Jerusalem in Motion

6. Linear Vs. Nonlinear Muscle Model Linear model The nonlinear Hill-type model The physiologically plausible nonlinear model can produce the typical speed profile with a simple control signals Karniel and Inbar (1997) Biol. Cybern. 77:173-183 Jerusalem in Motion

7. Other typical features of rapid movements are also facilitated by the nonlinear muscle properties In this set of simulations the one-fifth power law model was used. Karniel and Inbar (1999) J. Motor Behav. 31:203-206 Jerusalem in Motion

8. Force Field After Learning No Force After-Effects Force Field Initial Exposure Adaptation to force perturbations • Force field exposure  recovery of unperturbed pattern • Removal of field  “after-effects” • (Shadmehr & Mussa-Ivaldi 1994) Modified with permission from Patton and Mussa-Ivaldi Jerusalem in Motion

9. Hierarchical system with feedback adaptation and learning Internal models for control Desired Target Adaptation Learning Dynamics determine the control signal (e.g., EPH, CPG, …) Feedback Musculoskeletal system Actual Performance Jerusalem in Motion

10. The hierarchy of wide sense adaptationKarniel and Inbar (2001), Karniel (In preparation) Change Scale Structural Change Evolution Learning Functional Change Parameters Change Adaptation Time Scale Feedback No Change mSec Minutes Years Myears Jerusalem in Motion

11. Outline • The Hierarchy of Wide Sense AdaptationEquilibrium trajectories and internal models Reaching movements muscle models and adaptation • Adaptation to Force PerturbationsTime representation Sequence learning and switching • Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal models • Summary and Future Research Jerusalem in Motion

12. Example: Plant & Environment Controller Internal Representation of the field Force Field What is the structure of the modifier ? Could it be a function of position, velocity, time, … ? What are the limitations of adaptation? Key Questions: Jerusalem in Motion

13. Time Representation  These systems are indistinguishable therefore • The existence of time variable isn’t sufficient to define time representation. • It is sufficient to consider the following form: Jerusalem in Motion

14. Time Representation - Definition The system is said to be capable of time representation if there exists a deterministic function h(x) such that for any u(t). The system is said to be capable of time representation of up to T seconds with ε accuracyif there exists a deterministic function h(x) such that for t<T and for any u(t). Jerusalem in Motion

15. The experiment Number of movements ~100 ~500 ~100 Null Learning Generalization No external field External Force field time/state/sequence dependent Jerusalem in Motion

16. Time Varying Force Field The force field is not correlated with the movement initiation, therefore there is no way to use state information. Only time representation would allow adaptation and after-effects for this field. Jerusalem in Motion

17. Result: No adaptation to this TV force field The maximum distance from a straight line during “learning” A control experiment with the viscous curl field Karniel and Mussa-Ivaldi (2003) Biol. Cybern. Jerusalem in Motion

18. B- B+ 1 1 Vy 0 Vy 0 -1 -1 -1 0 1 -1 0 1 Vx Vx Viscous Curl Force Field Jerusalem in Motion

19. Result: There is Significant Adaptation with This Sequence of Force Fields The maximum distance from a straight line during “learning” A control experiment with the viscous curl field Jerusalem in Motion

20. Direction Error Calculation 1. Find the Euclidean distance from a straight line at the point of maximum velocity(The feed-forward part of the movement) 2. If the deviation is to the right multiply by –1 “B+” 3. If the curl field in the sequence is B- multiply by –1 Therefore: Positive DE: Yielding to the field Negative DE: Over resisting the field DE is Positive Jerusalem in Motion

21. Catch trials – After Effects A few trials without force field were introduced unexpectedly. The left bar is the mean of the error (DE) during these trials in the first part of the learning. The right bar is in the last part. Significant expectation to the correct field after learning i.e., learning of an internal model of the force field Jerusalem in Motion

22. Mid – Summary • No adaptation in the case of the time dependent force field • Adaptation in the case of the simplest sequence of curl viscous fields with four targets. What is learned in the second case? Jerusalem in Motion

23. During the learning it is possible to assign a unique force field to each movement instead of learning the sequence of force fields. The generalization phase would violate this representation. Odd and Even Movement B+B- Force Field: Jerusalem in Motion

24. 1. Analysis of errors in the last part where diagonal movements are introduced Refuting the Sequence Learning Assumption The same sequence is applied in this part; sequence learning predicts similar errors Force Field: B+B- Jerusalem in Motion

25. Distance Error Analysis of movements in part 1 and part 5 Left bar: Catch trials in part 1. Middle bar: Movements in part 5 that are inconsistent with the learning phase. Right bar: Movements in part 5 that are consistent with the learning phase. All movements are consistent with the sequence of force field. The sequence learning assumption predicts similar errors in the right two bars that is smaller than the first, left bar However, ANOVA of the data shows similar error in the first two bars and significantly smaller error in the right bar! Jerusalem in Motion

26. Refuting the Sequence Learning Assumption We found that when the perturbation can be modeled both as a function of sequence and as a function of the state, the brain generates a state dependent model. Can we design an experiment where only sequence representation would allow adaptation? Would the brain adapt to this perturbation? We tried to train subject with the same sequence but with three targets. In this case one needs to follow the temporal sequence in order to adapt Jerusalem in Motion

27. Result: No Adaptation to the Sequence of Force Fields! The maximum distance from a straight line during “learning” A control experiment with the viscous curl field Karniel and Mussa-Ivaldi (2003) Biol. Cybern. 89:10-21 Jerusalem in Motion

28. Catch trials – No After Effects A few trials without force field were introduced unexpectedly. The left bar is the mean of the error (DE) during these trials in the first part of the learning. The right bar is in the last part. No significant expectation to the correct field after learning i.e., no learning of an internal model to the sequence! Jerusalem in Motion

29. Mid – Summary (2) • No adaptation in the case of time dependent force field • Adaptation when the temporal sequence coincide with single state mapping • No adaptation in the case of sequence of force fields Karniel and Mussa-Ivaldi (2003) Biol. Cybern. 89:10-21 Maybe it is too difficult to construct two internal models simultaneously Multiple Models Conjecture (“soft” version): If each force field is experienced separately and enough time is given for consolidation of each model, then the multiple model would be constructed Jerusalem in Motion

30. Day 1 Day 2 Day 3 Day 4 Karniel and Mussa-Ivaldi EBR 2002 Early Training Late Training Late Training Catch-Trials Jerusalem in Motion

31. Result: Clear learning of each perturbation, but No evidence for ability to utilize multiple models and context switching Error [DE, mm] during early and late training Day 1 Day 2 Day 3 Day 4 Error [DE, mm] during catch trials (Subject E) Jerusalem in Motion

32. Does the brain employs clocks counters or switches ? In contrast to artificial devices that are based on clock counters and switches the brain seems to prefer state dependent maps Jerusalem in Motion

33. Outline • The Hierarchy of Wide Sense AdaptationEquilibrium trajectories and internal models Reaching movements muscle models and adaptation • Adaptation to Force Perturbations Time representation Sequence learning and switching • Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal models • Summary and Future Research Jerusalem in Motion

34. Bimanual Coordination (1) • Preference for in-phase symmetry • Stable vs. Unstable • Homologous muscles Figure from Kelso and Schöner (1988) Jerusalem in Motion

35. Bimanual Coordination (2) • It was recently shown that the preference for symmetry in bimanual coordination is perceptual Figure from Mechsner et al. (2001) Jerusalem in Motion

36. Bimanual Coordination (3) • Untrained individuals are unable to produce non-harmonic polyrhythms • However, with altered feedback (gear) they are able to generate symmetrical movement of the flags and non-symmetrical movements of the hands. • Again: The preference for symmetry is perceptual • Figure from Mechsner et al. (2001) Jerusalem in Motion

37. Bimanual Coordination (4) • The preference for symmetry was explained in terms of stable solution of dynamic system without employing internal models. • Following the vast literature about reaching movements we propose an alternative Hypothesis: The brain contains internal representation of the transformation between the perceptual level and the execution level in order to maintain the symmetry invariance in face of altered feedback or other external perturbations. • Predictions: 1. Learning curves, 2. After effects Jerusalem in Motion

38. Bimanual Index Tapping Experiment • The right hand received slower feedback such that when the display shows rotation at equal speeds the subject eventually produces a non-harmonic polyrhythm, with a left/right tapping frequency ratio of 2/3 Jerusalem in Motion

39. Learning Curve Regression (Standardized Data) From: Karniel A, Klaiman E, and Yosef V, Society for Neuroscience 2003 Jerusalem in Motion

40. After-Effect IndicationsThe last 60 seconds of each half in the experiment Jerusalem in Motion

41. Bimanual Adaptation Hypothesis • Symmetry Invariance • Adaptable transformation from the perception level to the execution level • After effects • The structure, learning rates and generalization capabilities are subjects for future research Jerusalem in Motion

42. Future Research • Relative role of each level, muscles, spinal cord, central nervous system • The structure of internal models (learning capabilities and generalization capabilities) • Virtual Haptic Reality • The Robo-Sapiens age Mathematical Analysis, Simulation, Experiments Jerusalem in Motion

43. Turing-like test for motor intelligence:The Robo-Sapiens age Building a robot that would be indistinguishable from human being Jerusalem in Motion