1 / 29

Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Fast Readout of Object Identity from Macaque Inferior Tempora Cortex. Chou P. Hung, Gabriel Kreiman, Tomaso Poggio, James J.DiCarlo McGovern Institute for Brain Research, Brain and Cognitive Sciences, MIT. Object Recognition is difficult: trade-off between selectivity and invariance.

louvain
Download Presentation

Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fast Readout of Object Identity from Macaque Inferior Tempora Cortex Chou P. Hung, Gabriel Kreiman, Tomaso Poggio, James J.DiCarlo McGovern Institute for Brain Research, Brain and Cognitive Sciences, MIT

  2. Object Recognition is difficult:trade-off between selectivity and invariance • Selectivity • Many different images can correspond to the same type of object • Invariance • Similar activation patterns can correspond to different objects

  3. The end station of the ventral stream in visual cortex is IT

  4. Can we readout what the monkey is seeing?

  5. Single electrode recordings • Anterior inferior temporal cortex: highest visual area in the ventral “what” pathway • Spiking activity in AIT shows selectivity for complex shapes

  6. Can we “read-out” the subject’s object percept from IT? • number of sites for reliable, real-time performance • temporal properties (onset + integration scale) of object information • neural code for different tasks • invariance to object position, size, pose, illumination, clutter • recognition: ‘classification’ vs. ‘identification’? • spatial scale of object information (single unit, multi-unit, LFP) • stability of these neuronal codes? • improvement with experience? • …

  7. 77 objects, 8 classes

  8. Recording at each recording site during passive viewing • 77 visual objects • 10 presentation repetitions per object • presentation order randomized and counter-balanced

  9. One-versus-all classification • g classes (g=8): G1, …, Gg(toys, monkey faces, vehicles, etc.) • For each class i, build a binary classifier fi (toys vs. rest, monkey faces vs. rest, etc.) • sjlabeled examples (j=1,…,n), • For each example j, compute the output of each classifier (e.g. pi=sj．fi) • Take prediction that maximizes pi • One-versus-all is not worse than other methods (Rifkin et al, 2003)

  10. Comparison of different statistical classifiers

  11. Decoding the population response Categorization 8 groups

  12. Pattern of mistakes made by the classifier

  13. Very rapid read-out of object information

  14. Categorization and Identification

  15. IT representation is invariant to changes in position and size

  16. IT representation is invariant to changes in position and size

  17. IT representation is invariant to changes in position and size

  18. Neural code in IT: time resolution

  19. Neural code in IT: latency and integration time

  20. Reading out another type of object info: scale and location

  21. How are different kinds of information coded?

  22. Reading out another type of object info: stimulus onset

  23. Specific wiring significantly improves classifier performance

  24. Extrapolation to novel pictures within the same categories

  25. Strong overlap between the best neurons for categorization and identification

  26. The SNR for categorization and identification are positively correlated

  27. Invariance to scale and position

More Related