slide1 l.
Skip this Video
Loading SlideShow in 5 Seconds..
From Mere Embodiment to the Cyborg Mind PowerPoint Presentation
Download Presentation
From Mere Embodiment to the Cyborg Mind

Loading in 2 Seconds...

play fullscreen
1 / 92

From Mere Embodiment to the Cyborg Mind - PowerPoint PPT Presentation

  • Uploaded on

From Mere Embodiment to the Cyborg Mind Andy Clark Cognitive Science Program Indiana University As of Fall 2004 Dept of Philosophy Edinburgh University Scotland disembodied symbolic abstract rule-following clunky chunky high-level slow brittle…..etc embodiment

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'From Mere Embodiment to the Cyborg Mind' - jaden

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

From Mere Embodiment to the Cyborg Mind

Andy Clark

Cognitive Science Program

Indiana University

As of Fall 2004

Dept of Philosophy

Edinburgh University
















perception-action systems






Just how different ARE these two paradigms really?Just what is it about embodiment that (when taken seriously) really matters for the very shape of a science of the mind?

What is the RELATION between those two apparently very different sets of target activities (between abstraction - rich reflective reason and more basic capacities for embodied action)? If embodiment matters, does it still matter ‘all the way up’?


Two Questions √

2.Embodiment: What Really



4.Incorporation versus Use

5.Mind and Reason

6.Can the be a Science of Hybrids?


Two Questions √

2.Embodiment: What Really


3.Sensory Substitution

4. Incorporation versus Use

5.Mind and Reason

6. Can there be a Science of Hybrids?


Mere Embodiment

Modest Embodiment

Profound Embodiment


For Shakey, the body and the environment were first and foremost problems to be solved.

The environment was the problem arena.

The sensors detected the lay-out in that arena.

The reasoning system planned a solution.

The body was just another problem, that then needed to be micro-managed so as to put the solution into practice.


Passive Dynamic Walkers (PDW’s)

  • (Andy Ruina Lab, Cornell: original work by Tad McGeer)
  • No actuation except gravity, and no control system, except for a mechanical knee.
  • Inner and outer legs are paired to constrain it from falling over sideways.
  • Surprisingly, PDW’s are capable (when set on a gentle incline) of very stable, human-looking walking.

Systematically pushing, damping and tweaking a system in which Passive Dynamic effects play a very major role

  • = a very simple example of
  • ecological control

Ecological Control

  • An ecological control system is one in which goals are not achieved by micro- managing every detail of the desired action or response, but by making the most of robust, reliable sources of relevant order in the bodily or worldly environment of the controller.
  • (See also Rob Wilson on ‘exploitative control’).

systems that are specifically designed so as to constantly search for opportunitiesto make the most of body and world, checking for what is available, and then (at various time-scales and with varying degrees of difficulty) integrating it deeply, creating new functional wholes.

i.e. a dynamically adaptive form of ecological control.


Any control system that is highly engineered so as to be able to learn to make maximal problem-simplifying use of an open-ended variety of internal, bodily, or external sources of order.

= dynamically adaptive ecological control


“Ecological control” names an overall effect not a single mechanism.

That effect (the achievement of delicate adaptive balances between environmental, neural and bodily dynamics) comes in many degrees and flavors, all the way from more-or-less hard-wired ecological balances to learnt-on-the-fly ecological balances.


The third hand is controlled by EMG signals detected by electrodes placed on four strategic muscle sites on Stelarc’s legs and abdomen. The third hand is controlled by Stelarc’s brain via muscle commands to these sites that are then relayed to the prosthesis.

Since these sites are not normally used for hand control, the third hand can be moved independently of the other two


Stelarc simply feels as if he wills the third hand to move, just as he wills his biological hands to move.

In each case, the control is fluent and intuitive. It seems to require no special effort or conscious focus.


Normally, you don’t feel as if you are (for example) USING your hand to do the washing up.

Instead, you just feel as if YOU are washing up.

Your hand functions as what some philosophers call TRANSPARENT EQUIPMENT : equipment through which you can act on the world without first willing an act on anything else.

(see Heidegger (1927) on the ‘ready-to-hand’)


The same is true in the biological case.

The human infant must learn, by trial and error and practice, which neural commands bring about which bodily effects,and must then practice until she is skilled enough to issue those commands without conscious effort (so the body becomes transparent equipment)


A monkey, with implanted electrodes monitoring brain activity, learns to control a joystick to move a cursor to get rewards.

The monitoring computer learns what neural commands correspond to what joystick motions.

Next, the joystick is disconnected.

The monkey discovers, though, that it can still use its own neural commands ( as transmitted by the implanted electrodes and decoded by the monitoring computer ) to directly control the cursor.

Finally, the commands are diverted to control a distant robot arm, whose motions are reflected in the on-screen cursor movements, thus closing the loop.


“the dynamics of the robot arm (reflected by the cursor movements) become incorporated into multiple cortical representations…we propose that the gradual increase in behavioral performance…emerged as a consequence of a plastic re-organization whose main outcome was the assimilation of the dynamics of an artificial actuator into the physiological properties of frontoparietal neurons”

Carmena et al

Public Library of Science: Biology

Vol 1: 2:2003 p.205


Whereas modest embodiment treats the body as a fixed (though highly significant) resource, profound embodiment is characterized by constant learning and re-calibration.

Biological forms of embodiment, unlike a lot of current work in robotics, all tend towards the ‘profound’ end of this spectrum, though we primates seem especially plastic and well-engineered for multiple embodiment and fluent tool -use.


Even the minds that, in the movie The Matrix, populate the Matrix ‘dream-world’ count as profoundly embodied, since those minds display the same adaptive ecological control abilities as our own. For example, a Matrixer could learn to fluently incorporate a Stelarc-style third-hand, or to use a thought-controlled robot arm.

It is just that the physical dynamics of the new components would be held in place by the Machines’ computer simulation rather than worldly physics.


These kinds of minds are promiscuously body-and-world exploiting. They are forever testing and exploring the possibilities for incorporating new resources and structures deep into their problem-solving regimes.

They are indeed the minds of Natural-Born Cyborgs (shameless plug): systems continuously re-negotiating their own limits, components, and (as we’ll next see) data-stores and interfaces.


Two Questions √

2.Embodiment: What Really



4. Incorporation versus Use

5.Mind and Reason

6.A Science of Hybrids?




Sensory Substitution (TVSS)

Work by Paul Bach y

Rita and



STOP feeling the tickling on the back and START to report rough, quasi-visual experiences of looming objects etc.

After a while, a ball thrown at the head causes instinctive and appropriate ducking.

The causal chain is ‘deviant’: it runs via the systematic input to the back.

But the nature of the information carried, and the way it supports the control of action, is distinctive of the visual modality.


“TVSS systems [have] been sufficient to perform complex perception and ‘eye’-hand co-ordination tasks. These have included face recognition, accurate judgment of speed and direction of a rolling ball with over 95% accuracy in batting the ball as it rolls over a table edge, and complex inspection-assembly tasks”.

Bach-y-Rita 2001


The head-mounted camera was under the subject’s motor control. This meant that the brain could, in effect experiment via the motor system, giving commands that systematically varied the input, so as to begin to form hypotheses about what information the tactile signals might be carrying.

For example, you hear someone approaching from the left, turn the camera that way, and see what tactile pattern corresponds to this event…


The motor system operating the camera could be changed, eg to a hand-held camera, with no loss of acuity.

The touch pad, too, could be moved to new bodily sites.

Also, there was no confusion: an itch scratched under the grid caused no ‘visual’ effects.

(see Bach y Rita and Kercel “Sensory Substitution and the Human-Machine Interface” Trends in Cognitive Sciences 7:12:2003)


Leprosy patients who have lost feeling in their hands.

Fitted with a sensor-laden glove that transmits signals to a forehead mounted tactile disc-array, they report feeling sensations of touch at the fingertips .

This is because the motor-control over the sensors runs via commands to the hand, so the sensation is projected to that site.

Bach y Rita and Kercel op cit.

(notice that this has clear implications for tele-presence based touch, etc).


Tactile Flight Suit (US Navy)

Jacket delivers small puffs of air controlled by complex sensors that determine if a plane or helicopter is tilting to the right or left or forward or backward.

The pilot feels a puff-induced vibrating sensation on the side of the body corresponding to the direction of tilt, and can control the vehicle’s response by moving their body so as to cancel the puff/vibration.


The suit is so good at transmitting and delivering information in an intuitive way that it allows even inexperienced helicopter pilots to perform difficult tasks such as holding the helicopter in a stationary hover, while military fighter pilots can use it to fly blindfold.

The suit thus rapidly links the pilot to the aircraft in the same kind of closed loop interaction that linked Stelarc and the third hand, or the monkey and the robot arm, or the blind person and the TVSS system

While wearing the suit, the helicopter itself behaves very much like an extended body/sensory sheath for the pilot.


What matters, in each case, is the provision of closed-loop signaling so that motor commands affect sensory input.

What varies is the amount of training (and hence the extent of deeper neural changes) required to fully exploit the new agent-world circuits thus created.


The specific details of the (old or new) circuitry by which the world is engaged fall ‘transparent’ in use.

The conscious agent is aware of the oncoming ball, not of seeing the ball, or (by the same token) of using a tactile substitution channel to detect the ball.

The pilot becomes aware of the plane’s tilt and slant, not of the puffs of air…


Integrated but constantly negotiable platforms of sensing, moving and (as we’ll see later) reasoning.

Platforms able to fluidly incorporate new bodily and sensory kit so as to engage (in the service of goal-directed activity) a larger and potentially hostile world.

= Profoundly Embodied Agents


Two Questions √

2.Embodiment: What Really


3.Adaptive Ecological Control√

4. Incorporation versus Use

5.Mind and Reason

6.Can there be a Science of Hybrids?


You are making quite a song and a dance out of this, what with talk of “incorporation of new bodily and sensory kit” and so on.

But we all know we can use tools and stuff, and learn to use them better (more ‘transparently’ if you must).

Why talk of extended bodies and reconfigured users, rather than talk of the same old embodied user just being in command of a new tool?


Any control system that is highly engineered so as to be able to learn to make maximal problem-simplifying use of anopen-ended varietyof internal, bodily, or external sources of order.

= dynamically adaptive ecological control


“Pre-motor, parietal and putaminal neurons that respond both to somatosensory information from a given body region (ie the somatosensory Receptive Field; sRF) and to visual information from the space (visual Receptive Field;vRF) adjacent to it”

Maravita and Iriki “Tools for the body (schema)”

Trends in Cognitive Sciences vol 8:2:2004, p. 79

For example, some respond to somatosensory stimuli (light touches) at the hand AND to visually presented stimuli near the hand, so as to yield an action-relevant coding of visual space.


After 5 minutes of rake-use, the responses of some bi-modal neurons whose original vRFs picked out stimuli near the hand now expanded to include the entire length of the tool, “as if the rake was part of the arm and forearm” (op cit).

And other bi-modal neurons, that previously responded to visual stimuli within the space reachable by the arm, now had vRFs that covered the space accessible by the arm-rake combination.


“Such vRF expansions may constitute the neural substrate of use-dependent assimilation of the tool into the body-schema, suggested by classical neurology”

And note that:

“any expansion of the vRF only followed active, intentional use of the tool not its mere grasping by the hand”

(op cit 80,81).

(see also work on haptic and dynamic touch- eg Turvey and Carello (1995))


In a patient whose neglect selectively affected space close to (one side of) the body, use of a stick extends the neglect to the whole area reachable by the tool.

See Berti and Frassinetti “When Far Becomes Near: Re-mapping of space by tool use” Journal of Cognitive Neuroscience 12 (2000) 415-420


“The brain makes a distinction between ‘far space’ (the space beyond reaching distance) and ‘near space’ (the space within reaching distance)” and that “…simply holding a stick causes a remapping of far space to near space. In effect the brain, at least for some purposes, treats the stick as though it were a part of the body”

Berti and Frassinetti (op cit)


The changes (for most tool uses) take time to occur, and lag slightly behind fluency of use.

No doubt they contribute to fluency of use. But they do so in a rather distinctive fashion.

Contrast: Representing the shape and dimensions of a hand-held tool and then INFERRING that you can now reach such and such an object, and altering the shape of these vRFs ( +perhaps other neural tweaks too) so that objects within range are now automatically classified as reachable.


What I dubbed adaptive ecological control is actually achieved by the use of neural resources that are multiply and delicately tuneable so as to factor in the properties of a soft-assembled system of components.

This is not the same as just representing those properties and features and then engaging in micro-managed deployment of resources.

Rather, the effect is to create new body-schemas pretty much ‘on-the-fly’


The genuine sense in which we are OF our world and not just IN it.

See Heidegger, Haugeland, Varela.


Two Questions √

2.Embodiment: What Really



4. Incorporation versus Use√

5.Mind and Reason

6.Can there be a Science of Hybrids?


Could anything like the notion of ‘incorporation’ (rather than mere use) get a grip in the domain of mind and cognition?

Could human minds be extended and augmented by technological tweaks, or is it still just the same old mind with a shiny new tool (I think this is often evolutionary psychology’s line)?


External information-processing resources are also apt for temporary or long-term recruitment and incorporation by processes quite analogous to those of basic adaptive ecological control.

To the extent that such processes operate, we are not just bodily and sensorily but also cognitively permeable agents.

















……just fill in your favorites….


Our cognitive permeability is related to the ability of the brain to learn complex routines that make implicit commitments to the robust availability of certain bodies of information while carrying out on-line problem solving.

This manifests as the delicate temporal tuning of multiple automatic calls to interacting sub-routines (including calls to bodily action and motion) that simply factor in that availability.

(for two nice, and quite different examples, see Kirsh and Maglio’s (1991) account of expert skill at Tetris and Ballard et als (2000) account of performance in a ‘block-copying’ task)


The experiential reflection of this kind of systemic commitment is that once specific bodies of information are thus ‘factored in’ we feel as if we “already know them”. This feeling is better glossed as “already being in command of the information”.

This may be the informational equivalent of equipment falling ‘transparent in use’


Our ability to perform fast, frequent, intelligent (Yarbus (1967)) saccades allows us to make maximal use of any stable local environment as a store of information (a kind of ‘external memory’)

See O’Regan (1992), Noe and O’Regan (2000)

In this way, some say, the stable local environment becomes poised for real incorporation into information-driven problem solving routines.

See eg work by Ballard and colleagues (eg 2000), Churchland et al (1994).


Experimental evidence from so-called Change Blindness (as well as magic tricks and undetected film continuity errors) shows dramatic gaps in our (on-the-spot, conscious) knowledge.

  • (Big lit. and complex issues, including substantial non-conscious uptake)
  • And yet we feel as if we are constantly aware of a great deal of detail concerning our current visual surroundings.

The way that a large body of information concerning the rich and varied detail in the current scene is indeed poised, in normal circumstances, for active retrieval and use in the service of our current goals.

  • Most of the information is left ‘out in the world’, but its easy availability is what yields our (correct) sense of knowing a rich and detailed scene.
  • (hence my paper rejecting the claims of a ‘grand illusion’ in Journal of Consciousness Studies (2002)).

The feeling of seeing all that detail in the scenes is really a reflection of something implicit in the overall problem-solving organization in which vision participates.

That organization ‘assumes’ the (ecologically normal) ability to retrieve more detailed info when needed, so we feel (correctly, in an important sense) that we are already in command of the detail.



Your feeling that you know what month this is.

Its not due to your constantly rehearsing the answer in your conscious mind (always saying, “April, April, April”).

Rather, it is due to your implicit knowing that this is the kind of thing you know and can usually retrieve (in this case from bio-memory) at will.


If confronted by a repeated series of this -month’s- name involving problems, the brain would simply build a set of interlocking routines that took for granted the availability, at low metabolic cost, of that information.


Just as the experienced brain need not (though it sometimes can) represent the shape of the tool and infer the available reach, so too it need not (though it sometimes can) first represent the availability (externally or internally) of the information and then infer that it can find what it needs by accessing a given resource.

When no such processes of resource representation and inference intervene, the tool, whether physical or informational, should be counted as (temporarily) incorporated into the problem-solving whole.

(establishing this was the main point of Clark and Chalmers (1998))


Gray and Fu (Cognitive Science, In Press)

Task: Programming a VCR (on-screen simulation)

Set-up so that successful programming could be achieved either by retrieval of knowledge from the world or from biological memory.

From-world retrieval involved saccading to an on-screen box, which in one condition (free access) was plainly visible and in another (gray box) required uncovering by mouse move and click

Experimenters thus systematically varied the effort required to deploy each source of information, using time-taken as the measure of effort



Milliseconds matter!

The subjects robustly settled on whatever strategy yielded the least-cost (measured by time) information retrieval.

Whichever mix yielded the least-effort solution (with effort measured by time) was recruited and calls to that date store, whether internal or external, became ‘built-in’ into the dominant strategy.

(This was so even when the least-effort-by-time solution was non-optimal for accuracy)


“the time spent retrieving something from memory is weighed the same as time spent in perceptual-motor activity” and that it is therefore a mistake to “presume the privileged status of any location or type of operation”

Gray and Fu (Cognitive Science, in press)


“The cognitive control of interactive behavior minimizes effort by using a least-effort combination of all the mechanisms available to it. All mechanisms or sub-systems are on the table. There is no reason to think that one mechanism or subsystem has a privileged status in relation to another”

“The embodiment level is the right level of description for functional cognition”

Gray and Fu (Cognitive Science, in press)


Body parts, sensory channels, non-biological body parts, sensory subsitution systems, internal information storage and processing, and external information storage and processing are all fair game for adaptive ecological control.

They are just resources apt for dynamic recruitment into the whirl of problem-solving activity and hence for incorporation, not just use.


“The integrative processes by which the brain adapts to control interaction are relatively agnostic concerning the source of the structure participating in the process...”

Christensen (In Press)


Two Questions √

2.Embodiment: What Really



4.Incorporation versus Use√

5.Mind and Reason√

6. Can there be a Science of Hybrids?


Adams and Aizawa (2001)

  • Transcranialism
  • The transcranialist (yuch) holds that
  • “cognitive processes extend in the physical world beyond the bounds of the brain and the body”(43)
  • (typical offenders include: Clark and Chalmers, Clark, Dennett, Donald and Hutchins)

Science tries to carve nature at its joints” (51)

But the causal arrangements whereby external stuff contributes to reason and action are very different to those whereby internal stuff does.

For example, , biological memory systems “display a number of what appear to be law-like regularities, including primacy effects, recency effects, chunking effects and others” (61)

By contrast, our external props and tools (notebooks, PC’s, rolodexes) form a varied bunch, with nothing much in common either with each other or with the biological substrate itself


‘transcranial [extended] processes are not likely to give rise to interesting scientific regularities. There are no laws covering humans and their tool-use over and above the laws of intercranial [inner] human cognition and the laws of the physical tools’ (61)


The work by Nicolelis, by Bach-y-Rita, and by Gray and Fu are all examples (in very different domains) of the systematic exploration of hybrid (biological and non-biological) systems that function (on various time-scales) as integrated problem-solving wholes.


There are (at least) two perfectly legitimate objects of study hereabouts: the multiple and exquisite forms of bio-side plasticity that support this ability and the new functional wholes themselves


Neural plasticity has long been treated as if it were just a ploy designed to cope flexibly with early insult or injury, and one whose prime efficacy is limited to an early developmental window.

As if it were “ a reactive rather than an active property of the immature system” (240)

Joan Stiles ‘Neural Plasticity and Cognitive Development’ Developmental Neuropsychology 18: 2: 2000 p. 237-272


“The construct of plasticity is best defined by complex dynamic processes in which the biological system progressively adapts to contingencies of input and the demands of the learning environment…. They are not…ancillary, optional or reactive. Rather, plasticity is a fundamental and essential property…not unique to development”

Mature form and function emerge as stabilized plasticity (see Christensen, In Press).

Stiles, op cit p 252


First, the neural plasticity and adaptivity does not itself constitute the knowledgeable agent so much as a resource that underpins an ongoing and never-ending search for participative components and information stores.

It is just the biological bootstrap that allows our full physical and cognitive agency to come into being.


Second, the new functional wholes that are discovered by the operation of the system are often themselves the integrated systems that participate in subsequent rounds of resource-seeking and annexation.

(A simple example was Stelarc learning to use that pencil with the third-hand)


‘ratchet effects’ whereby hybrid systems (level 1) come to incorporate still further layers of structure, becoming hybrid systems (level 2) that go on to seek out still further layers of structure, etc….

(for a nice exploration of this in the cognitive realm, see Kim Sterelny’s new book Thought in a Hostile World)


A kind of ‘basic science’ of the biological plasticity and multiple specific adaptations that originally poised us for the long journey into hybrid space.


A science of the new functional wholes that (repeatedly) result, and of the ratchet effects in which they participate.


What are the basic mechanisms of adaptive ecological control?

What is the basic tool-kit for understanding adaptive ecological control, and does it require the use of more than the standard information-processing notions of representation and computation (e.g. dynamical field theory)?


What extra tweaks enable we humans to be such world-class experts at such control?

What is the basic tool-kit for understandinghybrid functional wholes, and the complex cultural-evolutionary scenarios in which they participate?


What exactly happens when biological endowments for plasticity and for adaptive ecological control interact with material symbolic artifacts such as spoken words and written inscriptions?

Given all that plasticity and permeability, how should we best identify and analyze minds, persons, agents, environments, and perceptual and cognitive systems?

What about the cognitive/non-cognitive divide itself: does it still make sense when everything is just a resource on a level playing field?


Agents for whom body, sensing, world and technology are all just resources for soft assembled processes of dynamically adaptive ecological recruitment, yielding a permeable and repeatedly reconfigurable agent/world boundary.


For the profoundly embodied agent, the world is not something locked away behind the fixed veil of a certain skin-bag, a reasoning engine and a primary sensory sheath.

Instead, it is a resource apt for active recruitment and use, in ways that bring new forms of embodied intelligence into being.


Short bibliography

Bach-y-Rita, P and Kercel, S (2003) “Sensory Substitution and the Human-Machine Interface” Trends in Cognitive Sciences 7:12:541-546

Carmena, J et al (2003) “learning to Control a Brain-Machine Interface for Reaching and Grasping by Primates” Public Library of Science: Biology: 1:2: 193-208

Available free in the archive at:

Clark, A (2003) Natural-Born Cyborgs: Minds, Technologies and the Future of Human Intelligence (Oxford University Press:NY)

González, J and Bach-y-Rita, P (ms) “Perceptual Adaptive Recalibration: Tactile Sensory Substitution in Blind Subjects”

Gray, W and Fu, W (In Press) “Soft constraints in interactive behavior: the case of ignoring perfect-knowledge in-the-world for imperfect knowledge in-the-head” Cognitive Science

Mussa-Ivaldi, F and Miller, L (2003) “Brain-machine interfaces: computational demands and clinical needs meet basic neuroscience” Trends in Cognitive Sciences 26:6:329-334