Binocular disparity and stereopsis
Download
Sponsored Links
This presentation is the property of its rightful owner.
1 / 164

Binocular disparity and Stereopsis PowerPoint PPT Presentation


  • 175 Views
  • Uploaded on
  • Presentation posted in: General

Binocular disparity and Stereopsis. Bruce Cumming. Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health. Put red lens over left eye, blue lens over right eye Stereo anaglyph by Prof. Michael Greenhalgh, Australian National University (with permission).

Download Presentation

Binocular disparity and Stereopsis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Binocular disparity and stereopsis

Binocular disparity and Stereopsis

Bruce Cumming

Laboratory of Sensorimotor Research,

National Eye Institute,

National Institutes of Health


Binocular disparity and stereopsis

Put red lens over left eye, blue lens over right eye

Stereo anaglyph by Prof. Michael Greenhalgh, Australian National University (with permission).


Stereopsis

stereopsis

L

R


Correspondence problem

correspondence problem

left eye’s image

right eye’s image


Random dot patterns

random-dot patterns

  • a completely unnatural stimulus

  • image changes every few ms

  • no recognisable objects e.g. faces

  • each dot has dozens of identical potential matches

  • and yet a clear perception of depth!


Neurons and depth perception

Neurons and depth perception

  • A simple model to generate disparity signals.

  • How neurons reflect this.

  • Some psychophysical limits this explains.

  • Further processing.


Binocular disparity and stereopsis

head image from Royal Holloway University of London Vision Research Group (with permission)


Binocular disparity and stereopsis

Right retina

Receptive Field

Left retina

Fovea


Disparity selective neuron

*

*

Disparity-selective neuron

Right RF

R

Left RF

L


Basic building block

basic building-block

  • inner product of image with receptive field

Pos(v)


Binocular disparity and stereopsis

….

….S= response

+ 1

+ 1

-0.1


Binocular disparity and stereopsis

=l

=r

Left RF

+

S

+

Right RF


Binocular disparity and stereopsis

Output (spike rate)

l1

r1

r2

l2

l2

r1

Input (membrane V)


Circuitry for complex cell

BS 2

BS 3

BS 4

Circuitry for complex cell

left right

binocular simple cells

RF1

BS 1

complex cell

RF2

Cx

If RF2 = -RF 1 in both eyes, then half squaring then summing is equivalent to simply squaring.


Energy model

square the result

sum over many such subunits

add together

energy model

convolution of left eye’s image with jth left receptive field

convolution of right eye’s image with jth right receptive field


Binocular disparity and stereopsis

L

R

L

R

Right Stimulus Position

Complex cell

Left Stimulus Position

Model

Ohzawa et al, 1990


Disparity selective neuron1

Disparity-selective neuron

Right RF

R

Left RF

L


Binocular disparity and stereopsis

L

R

L

R

L

R

L

R

Right Stimulus Position

Complex cell

Left Stimulus Position

Model

Ohzawa et al 1990


Disparity selective neuron2

*

*

Disparity-selective neuron

Right RF

R

Left RF

L


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Left RF

Right RF

-d

d

0

Correlation

-50

0

50

Disparity (pixels)


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Patern 1

Patern 2

Patern 3

Patern 4

Correlation

Patern 5

Mean

-50

0

50

Disparity (pixels)


Binocular disparity and stereopsis

d

-d

0

Disparity


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Right RF

Left RF

Correlation

0

Disparity


Deangelis ohzawa and freeman 1991

DeAngelis, Ohzawa and Freeman, (1991)

Cat simple cell RF maps


For single subunits simple

For single subunits (simple)

  • Odd symmetric disparity tuning implies phase disparity

  • Even symmetry around non-zero disparity implies position disparity

True for complex cells if:

  • All subunits have same phase disparity

  • All subunits have same position disparity.


Binocular disparity and stereopsis

Monkey complex cells

duf043

duf065

60

40

Firing rate (spikes/s)

20

0

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

0.8

-1.4

-0.9

-0.4

0.1

0.6

1.1

1.4

Disparity (degrees)


So far

So far:

  • Energy model measures cross-correlation after filtering.

  • V1 contains a bank of filters measuring these correlations after displacements of both phase and position.


Disparity selective neuron3

*

*

Disparity-selective neuron

Right RF

R

Left RF

L


Binocular disparity and stereopsis

the neuronal response

cf: 0.06 cpd

80

60

response

[spikes/sec]

40

20

0

0

0.5

1

1.5

2

time [sec]


Binocular disparity and stereopsis

the neuronal response

cf: 0.06 cpd

80

60

40

20

0

0

0.5

1

1.5

2

response

[spikes/sec]

cf: 0.5 cpd

80

60

40

20

0

0

0.5

1

1.5

2

time [sec]


Binocular disparity and stereopsis

f1

relative modulation

corrugation-frequency

[cpd]

relative modulation

cf: 0.06 cpd

80

60

40

f0

20

0

0

0.5

1

1.5

2

response

[spikes/sec]

cf: 0.5 cpd

80

60

40

20

0

0

0.5

1

1.5

2

time [sec]


Binocular disparity and stereopsis

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

SDsf [cpd]

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

RM (f1/f0)

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

sf [cpd]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

1/(2πSDrf) [degree-1]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

response

[spikes/sec]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

vertical position [°]

output exponent:

1 2 4

2

1.5

1

corrugation cutoff [cpd]

0.5

0

n=19

r=0.45

0

0.5

1

1.5

2

1/(2*π*SD of RF height)

[degree-1]


Binocular disparity and stereopsis

Predicted from mean V1 response

(mean ecentricity 3.7º)


Temporal impulse response lgn

Temporal impulse response (LGN)

10ms

Reppas, Usrey and Ried (2000)


Binocular disparity and stereopsis

drifting luminance grating

80

disparity modulation

1.5

40

40

response [spikes/sec]

tf cutoff for

drifting luminance grating [Hz]

1

20

relative modulation [f1/f0]

0

n=27

0.5

0

0

20

40

0

tf cutoff for

disparity modulation [Hz]

1

10

100

temporal frequency [Hz]

temporal frequency [Hz]

Temporal frequency tuning for contrast and disparity


Summary

Summary

  • We don’t solve the correspondence problem dot-by-dot.

  • Is this enough?


Monocular response

Monocular response

=

×


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Correlation

-50

0

50

RF Disparity (pixels)


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Correlation

-50

0

50

RF Disparity (pixels)


Disparity is two dimensional

Disparity is two-dimensional


Binocular disparity and stereopsis

P’

P

direction of gaze

nodal point

fovea


Binocular disparity and stereopsis

Epipolar line


Binocular disparity and stereopsis

Y

Z

X


Binocular disparity and stereopsis

Y

Z

X


Binocular disparity and stereopsis

Y

Z

X


Binocular disparity and stereopsis

probability density function for disparities encountered during natural viewing

-15

-10

-5

vertical disparity (degrees)

0

5

10

15

-10

0

10

20

30

horizontal disparity (degrees)


Binocular disparity and stereopsis

probability density function for disparities encountered during natural viewing

-1

-0.5

vertical disparity (degrees)

0

0.5

1

-1

-0.5

0

0.5

1

horizontal disparity (degrees)


Binocular disparity and stereopsis

-6

-4

-2

0

vertical disparity

2

4

6

-15

-10

-5

0

5

10

15

20

25

30

horizontal disparity


Binocular disparity and stereopsis

Preferred 2-D Disparity

0.6

0.6

0.4

0.2

0.0

Vertical Disparity (degrees)

-0.2

-0.4

-0.6

-0.8

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.6

Horizontal Disparity (degrees)


Binocular disparity and stereopsis

Stevenson and Schor (1997)


Schreiber et at 2001

Schreiber et at (2001)


Searching for matches

Searching for Matches

  • Not a 2-D problem.

  • Vertical extent of RF may be enough to deal with most epipolar lines.


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

-50

0

50

correlation

RF Disparity (pixels)


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

-50

0

50

correlation

RF Disparity (pixels)


Size disparity correlation

Size-disparity correlation

Smallman and MaCleod (1994)

1/(threshold contrast)

Spatial Period of Center Frequency

Disparity range (min)

Binocular phase

Center spatial frequency


Binocular disparity and stereopsis

Size-disparity correlation (2)

Prince and Eagle (1999)

R

L


Binocular disparity and stereopsis

Stimulus Disparity

1

0.5

0

-0.5

-1

-50

0

50

correlation

RF Disparity (pixels)


Binocular disparity and stereopsis

Stimulus Disparity

Stimulus Disparity

1

.5

0

-.5

-1

-50

0

50

correlation

0

50

RF Disparity (pixels)


Binocular disparity and stereopsis

2 cpd

0.5 cpd

Threshold

2 cpd + 0.5 cpd

2 cpd, half cycle

Farell, Li and McKee (2004)


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

correlation

-50

0

50

Disparity (pixels)


Tsai and victor 2003

Tsai and Victor (2003)


Anti correlated stimuli

anti-correlated stimuli

left eye’s image

right eye’s image

black  white


Binocular disparity and stereopsis

1

.5

0

-.5

-1

-50

0

50

correlation

0

50

RF Disparity (pixels)


Energy model simulation

4

3.5

3

2.5

2

1.5

1

0.5

0

-50

-40

-30

-20

-10

0

10

20

30

40

50

energy model simulation

correlated stimuli

simulated firing rate

anti-correlated stimuli

disparity


Binocular disparity and stereopsis

200

180

160

140

120

100

80

60

40

20

0

120

Cell rb313

Cell rb332

100

80

60

Firing Rate (spikes/s )

40

20

0

-0.4

-0.2

0.0

0.2

0.4

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

Disparity (degrees)

Correlated disparity

Anticorrelated disparity


Binocular disparity and stereopsis

Correlated disparity

Anticorrelated disparity

Energy model prediction

120

100

80

60

Firing Rate (spikes/s )

40

20

0

-0.4

-0.2

0.0

0.2

0.4


Binocular disparity and stereopsis

Correlated disparity

Anticorrelated disparity

120

Cell rb313

100

80

60

weaker response for anti-correlatedstimuli

Firing Rate (spikes/s )

40

20

0

-0.4

-0.2

0.0

0.2

0.4


What the energy model gets wrong

what the energy model gets wrong

 quantitative response to anticorrelation

  • real cells respond more weakly to anticorrelated stimuli


Monocular stimuli

left right

monocular stimuli

35

30

25

20

firing rate (spikes / s)

15

10

5

0


This cell is monocular

left right

“this cell is monocular”

35

30

25

20

firing rate (spikes / s)

15

10

5

0


Disparity tuning curve

35

left right

left right

disparity tuning curve

30

25

20

firing rate (spikes / s)

15

10

left right

5

0

-1.5

-1

-0.5

0

0.5

1

disparity (degrees)


Left eye has purely inhibitory effect

left eye has purely inhibitory effect

35

30

25

-

20

firing rate (spikes / s)

15

10

5

0

-1.5

-1

-0.5

0

0.5

1

disparity (degrees)


Binocular disparity and stereopsis

1

0.8

0.6

Disparity Discrimination Index (DDI)

0.4

0.2

0

0

0.2

0.4

0.6

0.8

1

Ocular Dominance Index


Binocular disparity and stereopsis

but -!

  • this isn’t possible in the energy model.


Binocular disparity and stereopsis

=ON region of RF

=OFF region of RF

the energy model says that each eye sends both excitatory and inhibitory input

receptive fields

BS


Binocular disparity and stereopsis

=ON region of RF

=OFF region of RF

the energy model says that each eye sends both excitatory and inhibitory input

receptive fields

BS


Disparity tuning curve1

35

Inhibition from left eye

left right

disparity tuning curve

30

25

left right

20

firing rate (spikes / s)

15

10

uncorrelated

left right

5

0

-1.5

-1

-0.5

0

0.5

1

disparity (degrees)


Binocular disparity and stereopsis

Response rates to random dots

140

Ideal monocular neuron

120

100

80

Ideal binocular neuron

monocular

60

40

20

0

0

20

40

60

80

100

120

binocular uncorrelated


What the energy model gets wrong1

what the energy model gets wrong

 quantitative response to anticorrelation

  • real cells respond more weakly to anticorrelated stimuli

     cells where one eye always inhibits firing

  • not possible within the energy model


Energy model1

D = 2 L* R

energy model:

  • disparity tuning curve is the cross-correlation of the left and right eye’s receptive fields.

C = [vL+vR]2 = vL2 + vR2 + 2 vLvR


Binocular disparity and stereopsis

disparity tuning curve

left eye’s receptive field

right eye’s receptive field

0.35

0.35


How to test

Cx

S

S

S

how to test

  • measure receptive fields?


How to test1

Cx

S

S

S

how to test

  • measure receptive fields?

  • not possible for complex cells.

  • make the comparison in Fourier space.

  • this works for simple and complex cells.


Energy model2

energy model:

  • disparity tuning curve is the cross-correlation of the left and right eye’s receptive fields:

    D = 2 L* R

  • the Fourier power spectrum of the disparity tuning curve is the product of the Fourier amplitude spectra of the left and right eye’s receptive fields:

    FT2(D) = 2 FT(L)FT(R)


Binocular disparity and stereopsis

0

0

2

4

6

8

-4

x 10

2

1

0

-1

-2

0

0

2

4

6

8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

position

spatial frequency tuning curve

receptive field (RF)

firing rate

spatial frequency

RF cross-section

Fourier spectrum

Fourier transform

amplitude

spatial frequency


If the energy model is right

0

2

4

6

8

if the energy model is right:

  • then by obtaining the cell’s spatial frequency tuning….

  • we obtain the Fourier amplitude spectrum of the RF profile.

firing rate

spatial frequency


Binocular disparity and stereopsis

2

2.5

1.5

2

1

1.5

4

1

0.5

3

0.5

0

0

2

4

6

8

2

0

0

2

4

6

8

1

0

0

2

4

6

8

monocular spatial frequency tuning curves

left eye

right eye

x

spatial frequency

spatial frequency

Fourier spectrum of disparity tuning curve

=

spatial frequency


Spatial frequency tuning

60

50

40

firing rate (spikes/s)

30

20

10

0

0

0.1

1

10

0.1

1

10

spatial frequency (cycles per degree)

spatial frequency (cycles per degree)

spatial frequency tuning

left eye

right eye


Product of fitted monocular spatial frequency tuning curves

60

0.05

50

0.04

0

40

30

0.03

20

0.02

10

0

product of fitted monocular spatial frequency tuning curves

0.01

0.1

0.2

0.5

1

2.5

5

10

15

spatial frequency (cycles per degree)


Disparity tuning curve2

100

90

80

70

60

50

40

30

20

10

0

-1.5

-1

-0.5

0

0.5

1

1.5

disparity tuning curve

firing rate (spikes/s)

disparity (degrees)


Binocular disparity and stereopsis

60

0.05

100

50

90

0.04

80

40

70

60

30

0.03

50

20

40

0.02

30

10

20

10

0

0.1

1

10

0

-1.5

-1

-0.5

0

0.5

1

1.5

firing rate (spikes/s)

firing rate (spikes/s)

disparity (degrees)

spatial frequency (cycles/degree)

0.05

0.04

0.03

normalized units

0.02

0.01

0

0.02

0.05

0.1

0.2

0.5

1

2.5

5

10

15

spatial frequency (cycles per degree)


Binocular disparity and stereopsis

Peak frequencies differ

too much power at DC

product of fitted spatial frequency tuning curves

0.05

0.04

Fourier transform of fitted disparity tuning curve (minus baseline)

0.03

normalized units

0.02

0.01

0

0.01

0.02

0.05

0.1

0.2

0.5

1

2.5

5

10

15


Binocular disparity and stereopsis

40

duf065

Right

35

60

Left

30

25

40

20

Firing rate (spikes/s)

15

20

10

5.0

0.0

0

-1.4

-0.9

-0.4

0.1

0.6

1.1

1.4

0.05

0.10

1.0

10

30

Disparity (degrees)

Spatial Frequency (cpd)


Binocular disparity and stereopsis

duf043

50

70

60

40

Right

Left

40

Firing rate (spikes/s)

20

20

0

0

0.05

0.10

1.0

10

30

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

0.8

Disparity (degrees)

Spatial Frequency (cpd)


What the energy model gets wrong2

what the energy model gets wrong

 quantitative response to anticorrelation

  • real cells respond more weakly than predicted to anticorrelated stimuli

     suppressive effect from one eye

  • not possible within the energy model

     mismatch between disparity frequency and response to gratings

  • real disparity tuning curves have more power at low frequencies than predicted


How can we fix the problem

how can we fix the problem?

  • one simple modification to the energy model.

  • keeps all the successes of the energy model.

  • but fixes all these problems at a stroke!


Binocular disparity and stereopsis

energy model

disparity-selective complex cell

receptive fields

images

binocular simple cell

BS

Cx


Read s modified version

= half-wave rectification

Read’s modified version

monocular simple cells

disparity-selective complex cell

receptive fields

images

binocular simple cell

MS

BS

Cx

MS


Read s modified version1

Read’s modified version

monocular simple cells

disparity-selective complex cell

receptive fields

images

binocular simple cell

MS

BS

Cx

MS


Suppression from one eye

input purely inhibitory

cell never fires

suppression from one eye

monocular simple cells

disparity-selective complex cell

receptive fields

images

binocular simple cell

MS

BS

Cx

MS


Problems our model solves

problems our model solves

  • suppressive effect from one eye

    • inhibitory synapse after monocular simple cell


Binocular disparity and stereopsis

0.05

60

0.05

100

50

90

0.04

0.04

80

40

70

0.03

60

30

0.03

50

20

0.02

40

0.02

30

10

20

0.01

10

0

0.1

1

10

0

0

-1.5

-1

-0.5

0

0.5

1

1.5

0.02

0.05

0.1

0.2

0.5

1

2.5

5

10

15

firing rate (spikes/s)

firing rate (spikes/s)

disparity (degrees)

spatial frequency (cycles/degree)

normalized units

spatial frequency (cycles per degree)


Binocular disparity and stereopsis

0.05

60

0.05

100

50

90

0.04

0.04

80

40

70

0.03

60

30

0.03

50

20

0.02

40

0.02

30

10

20

0.01

10

0

0.1

1

10

0

0

-1.5

-1

-0.5

0

0.5

1

1.5

0.02

0.05

0.1

0.2

0.5

1

2.5

5

10

15

firing rate (spikes/s)

firing rate (spikes/s)

disparity (degrees)

spatial frequency (cycles/degree)

normalized units

spatial frequency (cycles per degree)


Binocular disparity and stereopsis

100

when vL and vR are negatively correlated, this tends to be negative

90

80

70

60

50

40

pulling the response down below the uncorrelated level

30

20

10

0

-1.5

-1

-0.5

0

0.5

1

1.5

firing rate (spikes/s)


Binocular disparity and stereopsis

100

90

when vL and vR are negatively correlated, this is zero

80

70

60

50

40

30

pushing the response up closer to the uncorrelated level

20

10

0

-1.5

-1

-0.5

0

0.5

1

1.5

firing rate (spikes/s)


Binocular disparity and stereopsis

energy model our modified version

disparity tuning curve

0

0

-50

0

50

-50

0

50

disparity

disparity

no power at DC

increased power at DC

Fourier power spectrum

0

0

0

0.02

0.04

0.06

0

0.02

0.04

0.06

spatial frequency

spatial frequency


Threshold at zero

threshold at zero

monocular simple cells

receptive fields

binocular simple cell

complex cell

MS

BS

Cx

MS


Increased threshold

increased threshold

monocular simple cells

receptive fields

binocular simple cell

complex cell

MS

BS

Cx

MS


Binocular disparity and stereopsis

energy model our modified version

zero threshold

high threshold

disparity tuning curve

0

0

0

-50

0

50

-50

0

50

-50

0

50

disparity

disparity

disparity

no power at DC

increased power at DC

maximum power at DC

Fourier power spectrum

0

0

0

0

0.02

0.04

0.06

0

0.02

0.04

0.06

0

0.02

0.04

0.06

spatial frequency

spatial frequency

spatial frequency


Problems our model solves1

problems our model solves

  • suppressive effect from one eye

    • inhibitory synapse after monocular simple cell

  • mismatch between disparity frequency and response to gratings

    • threshold boosts power at low frequencies


Anticorrelation

anticorrelation

 image in one eye replaced with negative

 one of the convolutions changes sign

 disparity-modulated term inverts; amplitude unchanged:

 a consequence of the linearity of the model


Modified model

modified model

anticorrelation: convolution changes sign

clearly disparity-modulated term no longer simply inverts


Binocular disparity and stereopsis

MS

MS

-40

-20

0

20

40

Disparity

Energy model

Modified model

2

1.5

Response

1

0.5

0

-40

-40

-20

0

20

40

-20

0

20

40

Disparity

Disparity


Problems our model solves2

problems our model solves

  • suppressive effect from one eye

    • inhibitory synapse after monocular simple cell

  • mismatch between disparity frequency and response to gratings

    • threshold boosts power at low frequencies

  • quantitative response to anticorrelation

    • with high enough thresholds, arbitrarily low amplitude ratios can be obtained


Heterogeneity

heterogeneity

  • real neurons vary greatly in behavior.

  • some well-described by energy model.

  • complex cells have many binocular subunits:

  • perhaps some are like the energy model

    • linear binocular combination

  • others are like our modified version

    • threshold prior to binocular combination


Heterogeneity1

MS

BS

MS

BS

heterogeneity

some binocular subunits as in our model…

Cx

…others as in the original energy model

complex cells receive input from many binocular subunits.


Summary1

summary

  • the energy model gives a good qualitative account of disparity-tuned neurons.

  • it has been widely used in computational models.

  • there are a number of discrepancies when it is compared with quantitative data.


Summary2

summary

  • A simple, plausible modification removes these discrepancies.

  • Consequences for models of later processing largely unexplored.


Extrastriate cortex

Extrastriate cortex

  • V2, V4, MT, and MST all show responses to anticorrelated RDS, like V1.

  • IT does not response to anticorrelated RDS.

  • Does the solution have to be represented explicitly?


Conclusion

conclusion

  • Good understanding of the mechanisms of disparity selectivity in primary visual cortex, without invoking complex network interactions.

  • provides a firm basis for understanding the computations enabling stereo vision.


Binocular disparity and stereopsis

Put red lens over left eye, blue lens over right eye

Stereo anaglyph by Prof. Michael Greenhalgh, Australian National University (with permission).


Plus a prediction

plus… a prediction

  • Consider case where convolutions are equal and opposite: vL=-vR

  • Original energy model: they cancel out

  • Our version: no cancellation


Disparate drifting grating

disparate drifting grating

right eye

left eye


Typical simple cell response

typical simple cell response

  • one burst of firing per cycle of the stimulus.

firing rate

time (one stimulus cycle)


Phase difference 0 o

phase difference 0o

right eye

MS

BS

MS

left eye


Phase difference 0 o1

phase difference 0o

…half a cycle later

right eye

MS

BS

MS

MS

left eye


Phase difference 180 o

phase difference 180o

right eye

MS

BS

+

MS

left eye


Phase difference 180 o1

phase difference 180o

…half a cycle later

right eye

+

MS

BS

MS

left eye


Binocular disparity and stereopsis

o

0

o

60

o

120

interocular phase difference

o

180

o

240

o

300

time (2 stimulus periods)

energy model modified version


Binocular disparity and stereopsis

o

0

o

60

o

120

interocular phase difference

o

180

o

240

o

300

time (2 stimulus periods)

energy model modified version


Binocular disparity and stereopsis

-180o

-90o

0o

90o

180o

interocular phase difference

spikes / s

time (1 stimulus period)


Summary3

summary

  • we postulate that some binocular simple cells receive input via monocular simple cells.

  • straightforward, physiologically plausible mechanism.

  • extends our repertoire so that we can account for all known observations.

  • even predicted something before it was observed!


Binocular disparity and stereopsis

Stereo anaglyph by Michael Greenhalgh, Australian National University.

Put red lens over left eye, blue lens over right eye


Long term goal of our work

long-term goal of our work

to understand:

  • the algorithm the brain uses for stereoscopic depth perception.

  • how this algorithm is implemented physiologically.

  • where this occurs within the brain.


Binocular disparity and stereopsis

The stereo correspondence problem


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Patern 1

Patern 2

Patern 3

Patern 4

Correlation

Patern 5

Mean

-50

0

50

Disparity (pixels)


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Left RF

Right RF

-90

90

0

Correlation

90

0

-90

Disparity


Binocular disparity and stereopsis

Z

Y

“straight ahead”

yL

X

“straight ahead”

yR

xL

L

xR

R


Binocular disparity and stereopsis

1

0.5

0

-0.5

-1

Patern 1

Patern 2

Patern 3

Patern 4

Correlation

Patern 5

Mean

-50

0

50

Disparity (pixels)


Basic building block1

basic building-block

  • inner product of image with receptive field

“ON” region

Pos(v)


Basic building block2

“OFF” region

basic building-block

  • inner product of image with receptive field

Pos(v)


Binocular disparity and stereopsis

Right RF

Left RF

d

0


Energy model simulation1

-50

-40

-30

-20

-10

0

10

20

30

40

50

energy model simulation

correlated stimuli

simulated firing rate

uncorrelated stimuli

disparity


Shape of disparity tuning

shape of disparity tuning

  • a key prediction of the energy model.

  • demonstrating this result would be strong evidence for the energy model.


Binocular disparity and stereopsis

5.0

4.0

3.0

Preferred Grating Frequency (cpd)

2.0

1.0

0.0

0.0

0.5

1.0

1.5

2.0

2.5

Disparity Frequency (cpd)


Binocular disparity and stereopsis

5.0

4.0

3.0

Preferred Grating Frequency (cpd)

2.0

1.0

0.0

0.0

0.5

1.0

1.5

2.0

2.5

Disparity Frequency (cpd)


Binocular disparity and stereopsis

Peak

low

0.05

0.04

0.03

normalized units

0.02

0.01

0

0.01

0.02

0.05

0.1

0.2

0.5

1

2.5

5

10

15


Binocular disparity and stereopsis

Response at 0.05cpd

Response at peak

Peak frequency

5.0

1.0

4.0

0.8

3.0

0.6

Monocular grating

2.0

0.4

1.0

0.2

0.0

0.0

0.0

0.5

1.0

1.5

2.0

2.5

0.0

0.2

0.4

0.6

0.8

1.0

FT of disparity tuning


Binocular disparity and stereopsis

70

45

40

60

35

50

30

40

25

20

30

3

15

2.5

20

2

10

1.5

1

10

5

0.5

0

0

0

-0.5

-1.5

-1

-0.5

0

0.5

1

0.1

1

10

0

0.5

1

1.5

2

2.5

3

3.5

4

firing rate (spikes/s)

firing rate (spikes/s)

disparity (degrees)

spatial frequency (cycles/degree)

normalized units

spatial frequency (cycles per degree)


Binocular disparity and stereopsis

1

.5

0

-.5

-1

-50

0

50

correlation

0

50

RF Disparity (pixels)


  • Login