Understanding Control Variables in Grasping and Visuomotor Control

1. Cursus Doelgericht Handelen(BPSN33) R.H. Cuijpers, J.B.J. Smeets and E. Brenner(2004). On the relation between object shape and grasping kinematics. J Neurophysiol, 91: 2598-2606. R.H. Cuijpers, E. Brenner and J.B.J. Smeets (2006). Grasping reveals visual misjudgements of shape. Exp Brain Res 175:32-44

2. Topics • 1st hour: Control Variables in Grasping • Opposing views on visuomotor control • Research question • 2nd hour: Grasping elliptical cylinders • Real cylinders • Which positions? • How to get there? • Virtual cylinders • Constant haptic feedback • Veridical haptic feedback • If time permits: Modeling grip planning • Conclusions

3. Control variables in grasping Many levels of description: • Activity motor neurons • Muscle activity (EMG) • Posture (Joint angles) • Kinetics (Forces, torques) • Kinematics (Position, speed etc.) • Task level high Degrees of Freedom (DoF) low

4. Control variables in grasping How does the brain ‘plan’/compute the desired motor neuron output? • If movements are planned in task space: • little computational power needed for planning stage • But … • Need to solve DoF-problem (Motor primitives) • Cannot control everything (Stereotypic movements) • Need low-level on-line control (e.g. stiffness control)

5. Control variables in grasping • What is/are the correct level(s) of description for movement planning and visuomotor control? Method of research in visuomotor control: • Manipulate visual information / haptic feedback / proprioceptive feedback • Measure effect on motor output • Variables that have an effect are ‘controlled’ • Variables that have no effect are redundant Haptic = by touch Proprioceptor = sensory receptor in muscles, tendons or joints

6. Opposing views on visuomotor control Fingertip positions and object size • Milner & Goodale: perception vs. action • Franz et al: common source model • Smeets & Brenner: position vs. size Fingertip positions and object orientation • Glover & Dixon: planning vs. on-line control • Smeets & Brenner: position vs. orientation

7. perception vs. action Goodale (1993); Milner, Goodale (1993) • RV: lesions in occipito-parietal cortex (dorsal). • DF: damage in ventrolateral occipital areas due to CO poisoning.

8. perception vs. action • Dorsal pathway for guiding movements (should be veridical) • Ventral pathway for perception (perception of shape, colour etc.)

9. perception vs. action Agliotti, De Souza, Goodale (1995): • Grip aperture NOT influenced by size-illusion. • Due to separate processing of information for perception and action.

10. Common source model • Franz et al (2000): equal effects of illusion

11. Position vs. size Brenner, Smeets (1996): • Size-illusion does not affect grip aperture, but does affect the initial lifting force. • Explanation: not size information is used but position information. They are inconsistent.

12. Planning vs. on-line control Glover & Dixon (2001) • Relative effect of illusion decreases with time  Illusion mainly affects planning

13. Position vs. orientation Smeets et al. (2002) • Assumption: illusion affects orientation, not position • Also explains data of Glover and Dixon

14. Research Question: How is shape information used for grasping? • The visually perceived shape is deformed • Shape (ventral) determines where it is best to grasp an object (dorsal) • Grip locations not veridical • Shape information could be used during planning (ventral) or on-line control (dorsal) • Grip errors arise early or late in the movement

15. Grasping elliptical cylinders:real cylinders

16. Experimental design • seven 10cm tall cylinders • elliptical circumference with fixed 5cm axis • variable axis: 2, 3, 4, 5, 6, 7 and 8 cm

17. Experimental Design

18. Experimental design • Optotrak recorded traces of fingertips • 2 distances x 7 shapes x 6 orientations = 84 trials • 3 repetitions • 10 subjects

19. Experimental Design

20. Example

21. Which positions? • Geometry: grasping is stable at principle axes

22. Which positions? • Principle axes preferred. But systematic errors…

23. Which positions? • Systematic "errors" depending on orientation.

24. Which positions? • Scaling grip orientation  0.7 except for aspect ratios close to 1,  0.5 Scaling grip orientation = slope + 1

25. Comfortable grip Suppose:grip orientation = mixture between cylinder orientation + comfortable grip Prediction: Slope a =w-1 Offset b = -(w-1)f0

26. Thus … • Subjects grasp principle axes, but make systematic errors • Cannot be explained by comfort of posture • Additional effect of deformation of perceived shape

27. How to get there?

28. How to get there?

29. How to get there? Gradual increase: grip errors were planned that way High correlation despite errors! Sudden drop at end: Grip aperture automatically corrected Correlation much higher for max. grip aperture than final grip aperture

30. Thus … • Systematic errors already present in the planning of the movement • Maximum Grip Aperture reflects planned size rather than true size

31. Grasping virtual cylinders

32. Experimental design

33. Experimental Design

34. Experimental design

35. Experimental design • Constant haptic feedback: • Real cylinder is always circular • Virtual cylinders: 15 aspect ratios, 3 orientations • Veridical haptic feedback: • Virtual and real cylinders are the same, 7 aspect ratios and 2 orientations

36. Constant haptic feedback • Only half of the subjects scale their grip orientation • If they do, the scaling of grip orientation is similar to real objects (0.42)

37. Constant haptic feedback • Subjects hardly scale their max. grip aperture • Scaling of max. grip aperture is much smaller than for real objects (0.14 instead of 0.57)

38. Thus • Inconsistent haptic feedback reduces scaling gains Possible cause: • All subjects scale their grip aperture based on the felt size • Scaling of grip orientation based on seen orientation for only half of the subjects, and the felt orientation for the other half

39. Veridical haptic feedback • Similar pattern of grip orientations for all subjects • Scaling of grip orientation (0.58) close to those for real objects (0.60)

40. Veridical haptic feedback • All subjects adjust their maximum grip aperture • Scaling of max. grip aperture (0.39) much higher and closer to real objects (0.57)

41. Thus With consistent haptic feedback • Scalings of grip orientation and grip aperture close to those for real cylinders • Less variability between subjects

42. Comparison of experiments Real Cylinders Consistent Feedback Inconsistent Feedback

43. Thus • Natural grasping of virtual cylinders requires veridical haptic feedback • Grip orientation and grip aperture can be scaled independently

44. Modeling grip planning

45. Modeling grip planning • Physical constraints • Grip force through centre of mass • Grip force perpendicular to surface • Optimal grip along major or minor axis • Biomechanical constraints • For a given cylinder location there is a most comfortable grip • Evident when grasping circular cylinder

46. Modeling grip planning • Assumptions: • The planned grip orientation is a weighted average of the optimal and the comfortable grip orientation • The weights follow from the expected cost functions for comfort and mechanical stability

47. Modeling grip planning If Then (required)

48. Modeling grip planning • Perceptual errors change the perceived cylinder orientation • The comfortable posture may also be uncertain

49. Modeling grip planning If distributions are Gaussian with zero mean, we get: For the circular cylinder w=0, so that:

50. Modeling grip planning • Each grip axis may be grasped in different modes: • Model predicts probability of each mode