How does the brain decide what to look at next
Download
1 / 59

How does the brain decide what to look at next? - PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on

How does the brain decide what to look at next?. John Findlay University of Durham. (acknowledgements to Val Brown). Saccades are quintessentially voluntary movements. The gaze selects informative detail but eye scans also appear random and arbitrary.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' How does the brain decide what to look at next?' - yosef


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
How does the brain decide what to look at next

How does the brain decide what to look at next?

John Findlay

University of Durham

(acknowledgements to Val Brown)


Saccades are quintessentially voluntary movements.The gaze selects informative detail but eye scans also appear random and arbitrary

from Yarbus (1967)





Multiple interconnected maps offer the possibility of selection by biased competition
Multiple interconnected maps offer the possibility of selection by biased competition

“ Some kind of short-term description of the information currently needed must be used to control competitive bias in the visual system, such that inputs matching that description are favoured in the visual cortex.”

(Desimone and Duncan, Annual Review of Neuroscience, 1995)


Biased competition creates salience maps
Biased competition creates salience maps selection by

  • A salience map is a two-dimensional map in which a single scalar quantity (salience) is represented at each point.

  • Biased competition results in similarity to the search target being represented as salience.

  • Instantiated in various models

    Itti L and Koch C (2000). A saliency-based search mechanism for overt and covert shifts of attention. Vision Research, 40, 1489-1506.

    Hamker FH (2004). A dynamic model of how feature cues guide spatial attention. Vision Research, 44, 501-521.


Saccade target selection in the superior colliculus
Saccade target selection in the superior colliculus selection by

The SC is the main final site for selection of saccade destinations


Activity in the superior colliculus related to saccadesWurtz R H (1996). Vision for the control of movement. The Friedenwald Lecture Investigative Ophthalmology and Visual Science, 37, 2131-2145.


How does the brain decide what to look at next1
How does the brain decide what to look at next? saccades

  • During visual search, biased competition creates a salience map and processes, probably in the SC, select the point of highest salience to convert to an orienting saccade.

  • Supported by detailed studies of saccades during visual search (Findlay, Vision Research, 1997; Motter and Belky, Vision Research, 1998a,b)


Task: saccades

Search for a

red cross

(Look at it)

Findlay J M (1997). Saccade target selection during visual search. Vision Research, 37, 617-631


Properties of first saccades findlay 1997
Properties of first saccades saccades(Findlay, 1997)

Short latency (~ 250 ms) – very similar for saccades to target and to distractor


Properties of first saccades findlay 19971
Properties of first saccades saccades(Findlay, 1997)

Short latency (~ 250 ms) – very similar for saccades to target and to distractor

Frequently (75%) on target when target is in inner ring, occasionally (26%) when target in outer ring.

.


Properties of first saccades findlay 19972
Properties of first saccades saccades(Findlay, 1997)

Short latency (~ 250 ms) – very similar for saccades to target and to distractor

Frequently (75%) on target when target is in inner ring, occasionally (26%) when target in outer ring.

Incorrect saccades go preferentially to distractor sharing a feature with the target.


Monkey visual search saccades(Motter & Belky, Vision Research, 38, 1007-1022; 1885-1815, 1998)

  • Monkeys trained to search for a conjunction target (colour and orientation)


Saccade selection in visual search
Saccade selection in visual search saccades

  • The conclusion in both the Findlay and the Motter & Belky studies was that the biased competition/ salience map approach provided the most satisfactory account of saccadic selection.

  • In particular, no evidence for a rapid covert attentional scan (favoured by many psychologists).

  • This conclusion was reached earlier in physiological studies of single cell responses in the visual system of primates carrying out search tasks

  • FEF Schall & Hanes (1993)

  • IT Chelazzi, Miller, Duncan & Desimone (1993)


How does the brain decide what to look at next during visual search
How does the brain decide what to look at next during visual search ?

  • During visual search, biased competition creates a salience map and processes, possibly in the SC, select the point of highest salience to convert to an orienting saccade.


How does the brain decide where to look next
How does the brain decide where to look next? during visual search ?

  • Selection from a salience map is basic

  • Supplementary processes

    • Inhibition of return

    • Saccade pipelining

    • Strategies

  • Implicit learning

  • Contingent learning

  • Neuro-economics (Glimcher)

  • Task specific requirements for information acquisition

  • (Land, Hayhoe, Ballard)


How does the brain decide where to look next1
How does the brain decide where to look next? during visual search ?

  • Selection from a salience map is basic

  • Supplementary processes

    • Inhibition of return

    • Saccade pipelining

    • Strategies

  • Implicit learning

  • Contingent learning

  • Neuro-economics (Glimcher)

  • Task specific requirements for information acquisition

  • (Land, Hayhoe, Ballard)

NO OTHER ATTENTIONAL SELECTION


  • Selection from a salience map is basic during visual search ?

    How does the system avoid ‘salience loops’?

    (B is the most salient location when A is fixated, then A becomes the most salient when B is fixated)

    Salience map alone would give A > B > A > B . . . . .

    Proposed answer - Inhibition of Return (IOR)

    An attended location is subject to some form of inhibition

    when attention is shifted elsewhere

    Klein R M and MacInnes W J (1999). Inhibition of return is a foraging facilitator in visual search. Psychological Science, 10, 346-352


The rings task
The rings task during visual search ?

Scan order partly specified

Centre - red – free scan through blacks - blue

Count target letters and make Yes/No response


The Rings Task during visual search ?

3 6 9 12

Scan through the rings, starting with the red and

ending with the blue

(Scans from 6 individuals)


Rings task typical eye scan
Rings task - typical eye scan during visual search ?


Deviations from sequential scan during visual search ?

BACKTRACK 1

OMISSION

BACKTRACK 1

BACKTRACK 1

BACKTRACK 1

BACKTRACK 2


Trials with deviations from sequential scan
Trials with deviations from sequential scan during visual search ?

* Error rate on trials with standard scan 4 %


Backtracking in visual search during visual search ?

BACKTRACK 1

Found by other workers

(Motter & Belky, 1998)

(Peterson et al. 2001)

BACKTRACK 1

BACKTRACK 1

  • IOR time course may relate to visual processing

  • slower with increased processing demands, so not always immediate

  • Backtracking sequences may be pre-planned

  • (pipelined saccades)

  • In A1, B, A2 fixation sequences, the B fixation was normal duration (228 ms) but A1 and A2 were both shorter than normal (~ 170 ms).

  • The saccade following a backtracking sequence tended to follow the direction of the last saccade in the sequence.


Backtracking in visual search during visual search ?

BACKTRACK 1

Found by other workers

(Motter & Belky, 1998)

(Peterson et al. 2001)

X

BACKTRACK 1

BACKTRACK 1

  • IOR time course may relate to visual processing

  • Backtracking sequences may be pre-planned

  • (pipelined saccades)

  • In A1, B, A2 fixation sequences, the B fixation is normal duration but A1 and A2 are both shorter than normal

  • The saccade following a backtracking sequence tends to follow the direction of the last saccade in the sequence


Visually guided and memory guided saccades
Visually-guided and memory-guided saccades during visual search ?

How does the brain decide which?

Hikosaka et al (2000) argue for basal ganglia pathway (blue route)

Inhibitory effects on SC, others excitatory

Separate sets of cells in caudate and in SNr are active during visually guided and memory guided movements.


Directional strategies
Directional strategies during visual search ?

(Convex Hull)

COUNT

THE

DOTS


Directional strategies1
Directional strategies during visual search ?

COUNT

THE

DOTS

Directional strategies are one form of memory (Gilchrist & Harvey)


No other attentional selection
NO OTHER ATTENTIONAL SELECTION during visual search ?

‘Visual attention selects the saccade target’

Statement supported by the finding that visual information at the destination point of a forthcoming saccade receives preferential pre-processing

(Deubel and Schneider, 1996; Kowler et al. 1995)

Biased competition is a form of attentional selection but does not operate in a localised region of the visual field.

Visual attention is commonly thought of as selection of a localised region.


Visual attention selects the saccade target
‘Visual attention selects the saccade target’ during visual search ?

Why I don’t like this statement

1. It’s getting close to a homunculus view

2. Localised visual attention should be able to eliminate distractor interference.


Visual attention should be able to eliminate distractor interference.

next

saccade ?


Visual attention should be able to eliminate distractor interference.

Attentional spotlight selects next target


Visual attention should be able to eliminate distractor interference.

Attentional spotlight selects next target


Visual attention should be able to eliminate distractor interference.

Attentional spotlight selects next target

No effective spotlight


Visual attention should be able to eliminate distractor interference.

Attentional spotlight selects next target

Global effect

No effective small spotlight


The Global Effect distractor interference

Saccades to neighbouring target pairs tend to land towards

a centre-of-gravity position.

Findlay, 1981, 1982; Deubel, 1982; Ottes, Van Gisbergen and Eggermont, 1984

Reliably found with onset stimuli: does it occur in free scanning?


How accurate are scanning saccades
How accurate are scanning saccades ? distractor interference

SACCADE

TO CENTRE

OF GRAVITY ?


Is there a global effect in free scanning
Is there a global effect in free scanning ? distractor interference

  • Are saccades less accurate when there is a distractor present in the critical sector (as defined by Walker et al. 1997)?

Percentage of inaccurate saccades


Probability of inaccurate saccade distractor interference

for distractors in different

locations relative to

the target


accuracy distractor interference

coding

4

2

1

3

Is accuracy higher follower longer fixations ?

Accuracy is highest following short

duration fixations (although distractors

still decrease it).

This is the opposite to a speed-accuracy

trade off.


Attentional selection and saccades
Attentional selection and saccades distractor interference

Saccades during a free scan of a set of identical elements show the global effect. Thus no evidence here for a spotlight-like attentional selection.

In most practical situations, elements are not identical; hence biased competition will act to reduce the global effect


How does the brain decide where to look next? distractor interference

  • Selection from a salience map is basic

  • Supplementary processes influencing salience

    • Inhibition of return

    • Saccade pipelining

    • Strategies

    • Implicit learning

    • Etc. etc. etc.


The end
THE END distractor interference

Thank you for your attention


Kowler distractor interferenceet al (1995)


accuracy distractor interference

coding

4

2

1

3

Do distractors that have been already scanned reduce accuracy ?

Accuracy is reduced both by scanned

distractors and by new ones.


How replicable are scanning patterns ? distractor interference

(repeat run with one subject - different trial order)

REPLICA TRIALS

NON REPLICA TRIALS

NEAR REPLICA TRIALS

REPLICAS 26% overall, 68% ring count 3

REPLICAS and NEAR REPLICAS 45% overall



Saccade direction histograms distractor interference

Heuristic scanners

Strategic scanners


Saccade direction change histograms distractor interference


Saccade Landing Points distractor interference

SACCADES WITH NO DISTRACTOR IN SECTOR

Subject PB

Accuracy is largely independent of saccade size


Saccade undershoot distractor interference

10%

5%

1%


Saccade variability : on-axis distractor interference

10%

10%

5%

5%

2%

1%


Saccade variability : off-axis distractor interference

5%

2%

1%




ad