visual decoding can you see what i see n.
Skip this Video
Loading SlideShow in 5 Seconds..
Visual decoding… Can you see what I see? PowerPoint Presentation
Download Presentation
Visual decoding… Can you see what I see?

Loading in 2 Seconds...

play fullscreen
1 / 18

Visual decoding… Can you see what I see? - PowerPoint PPT Presentation

  • Uploaded on

Visual decoding… Can you see what I see?. Sophia Yang 4-22-2010. Visual pathway . retina o ptic chiasm t halamus, lateral geniculate nucleus (LGN) o ccipital cortex, primary visual cortex (V1) v entral and dorsal streams, downstream processing

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Visual decoding… Can you see what I see?

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
visual pathway
Visual pathway
  • retina
  • optic chiasm
  • thalamus, lateral geniculate nucleus (LGN)
  • occipital cortex, primary visual cortex (V1)
  • ventral and dorsal streams, downstream processing

**retinotopic mapping in lower visual areas

retinotopy in v1
Retinotopy in V1

Visual fields mapped retinotopically to V1

  • visual fields
retinotopic mapping in cat v1
Retinotopic mapping in cat V1

Visual stimulus

Corresponding V1 activity

decoding with fmri
Decoding with fMRI
  • functional magnetic resonance imaging (fMRI) methods
  • analyze neural activity (BOLD signals) in response to stimuli
  • traditional univariate analyses
    • Single voxels… ROIs involvement
  • multivariate analyses, machine learning approaches
    • classification, reconstruction, decoding!
reconstructing images
Reconstructing images
  • Miyawaki et al. (2008)
overview of visual decoding projects
Overview of visual decoding projects
  • classify, identify, reconstruct images
  • establish a systematic mapping between visual stimuli and brain activity
  • activity to stimulus
    • evaluate mapping
  • stimulus to activity
    • inversion

Goal: reconstruct visual contrast image (10x10 square grey-scale patches)

Input -

fMRI signals from all V1 and V2 voxels (algorithm later selects relevant voxels and weights) while viewing a contrast image

Output -

reconstructed image


  • Multivoxel patterns of fMRI signals in low visual areas
  • Multiscale visual representation
presentation of image stimuli
Presentation of image stimuli

Random image session: 6s random contrast image, followed by 6s rest

Figure image session: 12s geometric or alphabet shape contrast image, followed by 12s rest


The reconstruction algorithm

r = given fMRI signals

I(x|r) = the reconstruction image

= local image basis

x = spatial position in the image

Cm(r) = predicted contrast of local image basis

= combination coefficient of local image basis

training local decoders
Training local decoders
  • Local decoders defined to predict the mean contrast of each local image basis
  • Individually trained with fMRI data and the corresponding class labels representing the mean contrast values
  • Each local decoder consists of a multi-class classifier (classify fMRI data samples into the classes defined by the mean contrast values)
  • Predetermined relevant voxels selection and individual voxel weights (“sparse logistic regression,” Yamashita et al., 2008)
  • Lineardiscriminant function for contrast class k:
  • rd = fMRI signal of voxeld
  • wkd= weight parameter for voxeld and contrast class k
  • wk0 = bias
  • Probability that r belongs to contrast class k:
  • Softmax function
  • K = number of contrast classes
  • Predicted contrast class for mth local image basis Cm(r) chosen as contrast class with highest probability
reconstruction demonstration
reconstruction demonstration


applications of brain decoding
Applications of brain decoding?
  • Neural prosthetics, BMI, HCI
    • Control prosthetic device using neural activity in motor cortex
  • Reconstruct contents of visual imagery, dreams
  • Can you see what I see?
  • Reconstruction of the subjective contents of human perception
  • Ethical issues?