Caching and replay of place sequences in a Temporal Restricted Boltzmann Machine model of the hippocampus Sue Becker, Dept. of Psychology Neuroscience & Behaviour, McMaster Univ., [email protected] and Geoff Hinton, Dept. of Computer Science, Univ. of Toronto. 1. Summary .
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Caching and replay of place sequences in a Temporal Restricted Boltzmann Machine model of the hippocampus
Sue Becker, Dept. of Psychology Neuroscience & Behaviour, McMaster Univ., [email protected] and Geoff Hinton, Dept. of Computer Science, Univ. of Toronto
3. Learning in RBMs and TRBMs
5. Training patterns
Place tuning in linear track environment
The hippocampus is thought to be critical for the rapid formation and cued recall of complex event memories. For example, a set of hippocampal place cells that fires in a particular sequence during exploration may fire in the same sequence during sleep [1,2,3]. We propose a novel model of the hippocampal memory system that reconciles a wide range of neurobiological data. Here we address the question of how the hippocampus encodes sequences rapidly, and what is the function of sequence replay. Our model draws upon two recent developments of the Restricted Boltzmann Machine (RBM): 1) a hierarchy of sequentially trained RBM's , and 2) the extension to sequential inputs employed in the Temporal RBM . The top two layers of the model, representing the dentate gyrus (DG) and CA fields, are connected via undirected links to form an autoassociator, allowing the model to generate memories of coherent events, rather than generating top-level unit states independently. The CA region also has directed connections from previous CA states, representing the CA3 recurrent connections. Thus the probability distribution learned over the visible and hidden units is conditioned on earlier states of the autoassociator. The model is trained by contrastive Hebbian learning, with data-driven and generative phases providing the statistics for the positive and negative Hebbian updates respectively. This is broadly consistent with Hasselmo's proposal that the hippocampus oscillates between encoding and recall modes within each theta cycle . When trained on a variety of spatial environments (fig 5), the hippocampal units develop place fields (fig 6), while the CA time-delayed recurrent collaterals encode the sequential structure of the data (fig 7). It has been widely assumed that sequence replay is required for memory consolidation, in which rapidly stored hippocampal memories are gradually transferred to cortex. Alternatively, the hippocampus may always be required to recall complex associative memories, while replay allows the brain to maintain consistent forward and generative models in the hippocampal-cortical system .
3.1 Restricted Boltzmann Machines
Figure 4: Spatial locations are coded by Boundary Vector Cells (BVCs) . Each BVC’s tuning curve is a product of 2 Gaussians, tuned to distance from and direction to environmental boundaries
Probabilities of states P(V,H) are
proportional to their Boltzmann
In an RBM, conditional probabilities of H & V can be computed efficiently:
Figure 6: Place tuning of several CA units
Figure 5: The model was trained on 3 environments, a linear track, a U-shaped maze and a rectangular box.
Left: U-maze, with training locations shown by blue dots.
Right: the reconstruction of the boundaries from the BVC representation of the location marked X in the U-maze.
3.2 Contrastive divergence learning 
Maximizing likelihood requires settling to equilibrium.
Instead, minimize diff. between 2 K.L. divergence terms:
2. Data not explained by most HC models
6. Simulation results
Figure 7: Model’s reconstruction of a sequence of spatial locations along the linear track.
3.3 Temporal Restricted Boltzmann Machines
This is one example of a class of models called TRBM discussed in .
The model learns spatial environments gradually, but can rapidly cache state sequences in its temporal generative model (CA recurrent collaterals)
Making the temporal connections bi-directional requires propagation of states backwards in time, accounting for reverse sequence replay.
The addition of sparseness constraints sharpens place-tuning and leads to a grid-like representation
Figure 1. Cascaded architecture of the HC.
Figure 2. Left:Theta-modulated gamma ripples (fig 2A from Chrobak et al 2000) .
Right: Sharp waves (fig 1 A, C from Chrobak & Buszaki, 1994).  Reproduced with permission.
 Wilson, M.A. and McNaughton, B.L. (1994), Science 265(5172):676-679
 Louie, K; Wilson, MA (2001), Neuron 145-156
 Lee, A.K. and Wilson, M.A. (2002) Neuron 36(6):1183-1194
 Hinton, G. E., Osindero, S. and Teh, Y. (2006), Neural Computation 18(7):1527-1554.
 Sutskever, I. and Hinton, G. E. (2006) Technical Report UTML TR 2006-003.
 Hasselmo, M.E. et al (2002), Neural Computation, 14(4): 793-817.
 Kali, S. and Dayan, P. (2004) Nature Neuroscience 7(3):286-294
 Chrobak,J.J., Lorincz,A. & Buszaki,G. (2000) Hippocampus 10(4):457-65
 Chrobak & Buszaki, (1994) J Neurosci 14:6160-6170
 Hinton, G.E. (2002) Neural Computation, 14(8):1711–1800
 Hartley, T., Burgess, N., Lever, C., Cacucci, F. and O'Keefe, J. (2000), Hippocampus 10(4):369-379
4. A TRBM model of the hippocampus
from Fig 2
Lee & Wilson
Figure 3. Top: Smoothed raster plots of spikes vs sequentially traversed spatial locations (5-second time scale).
Bottom:Same sequence of 6 place cells fires during slow wave sleep (150msec time scale).
Figure 5: Examples of boundary reconstructions from BVC input patterns (left) and from model’s reconstruction of input (right) averaged over 5 binary stochastic samples
Figure 4. Simplified HC architecture used here.
This work was supported by funding from the Natural Sciences and Engineering Research Council of Canada (SB, GEH) and CIAR (GEH)