Praveen K. Pilly Stephen Grossberg

1. TEMPORAL DYNAMICS OF DECISION-MAKING DURING MOTION PERCEPTION IN THE VISUAL CORTEX(2008)Vision Research, 48, 1345-1373 Praveen K. Pilly Stephen Grossberg

2. Cognitive decision-making Decision-Making? Perceptual decision-making

3. Motivation

4. How does the brain make perceptual decisions? How do we decide the direction of a moving object embedded in clutter? How does the brain perform a direction discrimination task in a context-appropriate manner? Main Questions

5. Motion Direction Discrimination Experiments VALUABLE PARADIGM Train monkeys to discriminate the direction of a random dot motion stimulus report the judgment via a choice saccade Record behavior and area LIP neuronal responses Shadlen & Newsome, 2001 Roitman & Shadlen, 2002

6. Random Dot Motion Stimulus Signal dots move from frame n to frame n+3, frame n+3to frame n+6, and so on Interleaving of 3 uncorrelated random dot sequences Coherence level: the fraction of dots moving non-randomly 60 Hz frame rate

7. Two-Alternative Forced Choice Task Right or Left? MORE AMBIGUITY 3.2%

8. Two-Alternative Forced Choice Task Right or Left? LESS AMBIGUITY 51.2%

9. Two Experimental Contexts FIXED DURATION (FD) REACTION TIME (RT) Unlimited viewing duration before saccade in the judged direction Fixed viewing duration before saccade in the judged direction Shadlen and Newsome, 2001 Roitman and Shadlen, 2002 Roitman and Shadlen, 2002

10. Accuracy of decisions in both FD and RT tasks as a function of coherence Reaction time of decisions in the RT task as a function of coherence for correct and error trials Area LIP neuronal responses during correct and error trials in both FD and RT tasks for various coherences Correlation between the temporal dynamics of LIP responses and saccadic behavior (accuracy, reaction time of decisions) Differences between sensory MT/MST and decision LIP responses Data from the Experiments

11. ‘BAYESIAN INFERENCE’IN THE BRAIN Beck et al., 2008; Gold & Shadlen, 2001, 2007; Jazayeri & Movshon, 2006; Ma et al., 2006; Pouget et al., 2003; Rao, 2004 Existing Proposals / Models Abstract; Non-neural; Propose explicit Bayesian decoders in brain areas Rev. Thomas Bayes NEURAL MODELS Ditterich, 2006a, 2006b; Mazurek et al., 2003; Wang, 2002 Do not clarify important computations that need to occur between the motion stimulus and saccadic response Have a number of issues that need to addressed Treatise on Man (Rene Descartes)

12. MOtion DEcision (MODE) Model Contextual gating of response Choice of saccadic response Winning direction chosen and fed back to MT Pool signals over multiple orientations, opposite contrast- polarities, both eyes, multiple depths, and a larger spatial range FT signals are strengthened, ambiguous signals weakened Evidence accumulation amplifies feature tracking (FT) signals Local directional signals Non-directional signals Random dot motion input MOTION BCS: Grossberg et al., 2001 Berzhanskaya et al., 2007

13. Feature tracking signals Ambiguous signals Percept Motion Processing from Retina to Area MST Geometric aperture problem BARBERPOLE ILLUSION How do sparse feature tracking signals capture so many ambiguous signals to determine the global motion direction?

14. Local Directional Signals Grossberg et al., 2001 Null direction inhibition model Barlow & Levick, 1965 Fried et al., 2002, 2005

15. Short-Range Motion Signals Local directional processes can be fooled by low coherence multiple dots interleaving of uncorrelated dot sequences

16. Do Random Dot Motion Stimuli pose an Aperture Problem? INFORMATIONAL APERTURE PROBLEM

17. MT-MST Circuit: Motion Capture Directionally-asymmetric feedback inhibition from area MST to area MT across space Inter-directional competition across space in area MST

18. MT and MST Responses during Stimulus Viewing pref MT null Britten et al., 1993 MODEL SIMULATIONS MST MT

19. Informational Aperture Problem 51.2% coherence Rightward motion Directional short-range filters (V1)

20. Effectiveness of the motion capture process is limited by coherence level and also viewing duration Informational Aperture Problem Resolution 51.2% coherence Rightward motion Area MST

21. LIP Recurrent Competitive Field (RCF) Noise-saturation problem Recurrent on-centeroff-surround shunting network Self-normalizes total activity like computing real-time probabilities Grossberg, 1973+ RCFs have also been used to model reach decisions in dorsal premotor cortex Cisek, 2006

22. Stochastic LIP RCF

23. RT Task SimulationsSample Correct Trials RT Task

24. LIP Responses during RT Task Correct Trials Roitman & Shadlen, 2002 Simulations More coherence in preferred direction causes: Fastercell activation More coherence in opposite direction causes: Fastercell inhibition Coherence stops playing a role in the final stages of LIP firing for preferred choices

25. FD Task SimulationsSample Correct Trials FD Task The “gain of the LIP response is greater in the RT version of the task” when compared to the FD task Roitman & Shadlen, 2002

26. LIP Responses during FD Task Correct Trials Roitman & Shadlen, 2002 Simulations More coherence in preferred direction causes: Fastercell activation Highermaximal cell activation More coherence in opposite direction causes: Fastercell inhibition Lowerminimal cell activation

27. Accuracy of Decisions 50 50 Mazurek et al., 2003 Roitman & Shadlen, 2002 Simulations More coherence in the motion causesmore accurate decisions RT task accuracy is slightly better than FD task accuracy at lower coherences (< 25%)

28. Effect of Viewing Duration on Accuracy in FD Task Gold & Shadlen, 2003 Simulations

29. LIP Responses in the RT Task during Correct and Error Trials Roitman & Shadlen, 2002 Simulations LIP encodes the perceptual decision regardless of the direction and strength of the dots, unlike sensory MT/MST neurons

30. LIP Response Dynamics correlate with Reaction Time 6.4% Roitman & Shadlen, 2002 Simulations

31. Speed of Decisions (RT Task)Correct (-) and Error (- -) Trials Roitman & Shadlen, 2002 Simulations More coherence in the motion causesfaster reaction time RTs on error trials are greater than those on correct trials

32. Slower Error Trial RTs? Brownian motion process At low coherences, the LIP cell dynamics are controlled more by cellular noise processes As time passes, the likelihood of a wrong LIP cell being chosen increases Slower RT indirectly explains slower rate of change in LIP responses on error trials

33. “ logarithm of the likelihood ratio (logLR) provides a natural currency for trading off sensory information, prior probability and expected value to form a perceptual decision ” Gold & Shadlen, 2001 Is Motion Direction Discrimination an example ofBayesian Decision-Making? S1: direction d S2: opposite direction D I: spatio-temporal input logLR is proposed to be equivalent to opponent motion read-out How does this explain decision-making properties in response to a variety of perceptual stimuli and task conditions?

34. Bayesian inference in the brain? Gold & Shadlen, 2001, 2007 Knill & Pouget, 2004 Pouget et al., 2003; etc. Bayesian Inference is a Popular Hypothesis Probabilistic nature of decision-making in response to uncertainty Neuronal variability This approach does provides an intuitive framework Does it disclose brain mechanisms underlying perception and decision-making?

35. “… We question the popular wisdom that the brain operates as an information-processing device that performs probabilistic inference …” Shadlen et al., 2008 “… a categorical decision is readout by a Bayesian decoder ... Our work suggests that explicit representation of probability densities by neurons might not be necessary …”  Furman & Wang, 2008 “… This generality is part of its [Bayes’ rule] broad appeal, but is also its weakness in not proving enough constraints to discover models of any particular science …” Grossberg & Pilly, 2008 Brain without Bayes

36. Comparison to other Neural Models Wang, 2002; Wong & Wang, 2006; Mazurek et al., 2003;Ditterich, 2006a, 2006b Our model goes beyond alternative models: Uses the real-time perceptual stimuli used in the experiments Does not make many of the specialized assumptions of previous models Clarifies the different roles of sensory MT/MST and decision LIP cells Simulates the effect of viewing duration on the psychometric function Incorporates the difference in LIP responsiveness to the two task conditions Considers the visual contribution to LIP response due to choice target Simulates the entire time course of LIP responses during both tasks on both correct and error trials Highlights the important role of BG in contextually gating the saccadic response …

37. MODE Model Predictions Gradual resolution of the informational aperture problem in area MT Pack & Born, 2001 Explanation for the lack of coherence-independent initial transient pause in LIP activity in the FD task, unlike the RT task Lower LIP activity, before motion onset, in multiple-choice tasks Volitional top-down mechanism to make ‘forced choices’ Churchland et al., 2007 Stimulus manipulations such as: higher dot density more interleaved sequences briefer signal dots should: decrease accuracy increase reaction times have influences on MT, MST, and LIP responses similar to those that occur due to lowering motion coherence