Continuity between perception and imagination in the predictive coding framework

1. Continuity between perception and imagination in the predictive coding framework Nicolas Alzetta CoNGA: Cognition and Neuroscience Group of Antwerp Dec 14th 2016

2. Origins of predictive coding in cognitive science Hermann Helmholtz: How can we escape from the world of the sensations of our nervous systems into the world of real things ? We are guided by the answers nature delivers when we query it, using unconscious perceptual inference based on our prior learning. Connectionism & machine learning: models of unsupervised learning that relied heavily on recurrence and prediction Bayesianism: the brain as inference machine that infers causes from sensory input Data compression: eg. JPEGs: the computational cost of coding color of every pixel is too high.

3. The Bayesian brain Nervous system maintains probabilistic models which are updated by sensory information according to (an approximation of) Bayes’ theorem

4. Main ideas of PC • The brain continuously makes generative models (hypotheses) at multiple levels of the cortical hierarchy. These statistical models predict the activity from lower areas and are sent down the hierarchy through recurrent connections (top down). • Each model is compared to the activation at the level below. If there is a mismatch, the so-called prediction error is sent up through feedforward connections (bottom-up). From: Stefanics, et al. 2015

5. Main ideas of PC (2) • At each level of the hierarchy, the generative model is continuously updated on the basis of the prediction error. The aim of the process is to minimize prediction error, in which case the model has optimal accuracy. This is also described as ‘minimizing surprise’. • An other way of minimizing the prediction error is not by updating the model at the higher level, but by action in order to change the input signal. • The conscious percept is the prediction (hypothesis) with the highest overall posterior probability and hence the lowest the prediction error. • Depending on context, input might be less precise (low signal to noise ratio). In that case, the impact of the prediction error on the generative model is smaller. Attention on this framework is the optimization precision.

6. Main ideas of PC (3) • Depending on context (low signal to noise ratio, high uncertainty), input might be less precise. In that case, the impact of the prediction error on the updating of the generative model is smaller. • Attention, on this picture, serves to optimize the precision of the prediction error signal. It ‘turns up’ or ‘lowers the volume’ of signal.

7. Explanatory strengths • Offers a solution to (among other things): • Binding problem • Binocular rivalry • Contextual effects

8. Weaknesses Relatively little neurological evidence But: Friston (2009) argues that the sources of forward connections are the superficial pyramidal cell population and the sources of backward connections are the deep pyramidal cells Too general and therefore unfalsifiable. Not all attentional effects can be explained using precision optimization.

9. Mental imagery • According to PC, mental imagery consists simply of generative models without error messages coming from the lowest levels of the hierarchy to update them. There is no fundamental divide between perceiving and imagining. Empirical support: • Reddy et al. study using ‘brain reading’: reconstruct properties of a stimulus from fMRI data, by extracting multivoxel response patterns from which you infer (decode) properties of the stimulus that caused them. • First subjects viewed objects. A decoding algorithm was developed that could successfully classify properties of the objects (food, face, tool, …). • The same was done when subjects imagined the objects. A different decoding algorithm was successfully developed.

10. Mental imagery • Then, both algorithms were swapped, without significantly impacting the classification results. This indicates a deep similarity between neuronal activation patterns in both perception and imagery. • This could only be done in ventral temporal cortex. Not in V1 & V2 where decoding was only possible in perception cases. The claim I wish to defend (…), is thus that animals able to perceive a complex external world of interacting causes using the characteristic resources of prediction-driven learning will be animals capable of the endogenous generation of sensory like states. (Clark 2016) A good imaginer is a good perceiver ? Is that why artists see more ?

11. Questions? Sources: Friston, K., and Kiebel, S. (2009). Predictive coding under the free-energy principle. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 1211–1221. Stefanics, G., Kremláček, J., and Czigler, I. (2014). Visual mismatch negativity: a predictive coding view. Front. Hum. Neurosci. 8:666, 1–19. doi: 10.3389/fnhum.2014.00666