1 / 53

Network correlations: A maximum entropy approach

Network correlations: A maximum entropy approach. John Beggs Aonan Tang Indiana University Department of Physics. David Hubel. The brain is a complex system. Is there any way we could “simplify” it?. If so, could such an approach be applied to other complex systems?. Outline.

keelia
Download Presentation

Network correlations: A maximum entropy approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Network correlations: A maximum entropy approach John Beggs Aonan Tang Indiana University Department of Physics David Hubel

  2. The brain is a complex system. Is there any way we could “simplify” it? If so, could such an approach be applied to other complex systems?

  3. Outline • Correlated states • Maximum entropy • Spatial correlations • Temporal correlations

  4. Outline • Correlated states • Maximum entropy • Spatial correlations • Temporal correlations

  5. Data collected in collaboration with Alan Litke, UC Santa Cruz

  6. 512 electrode array

  7. Correlated activity Yet pairwise correlations are often weak!

  8. Consider only single time bins Consider only ensembles of ~10 neurons Schneidman, et al, 2006

  9. Puzzle: If four neurons each had a 0.01 probability of firing, what would be the probability of observing all 4 neurons firing at the same time? P = (0.01)4 = 0.00000001

  10. But the data suggest that neurons are not independent Schneidman, et al, 2006

  11. Multi-neuron firing patterns can be observed far more often than an independent model would predict: Schneidman, et al, 2006

  12. A special model which considers only firing rate and pairwise correlations can do a good job of predicting correlated states: Let’s see how this model works Schneidman, et al, 2006

  13. Outline • Correlated states • Maximum entropy • Spatial correlations • Temporal correlations

  14. Main features of model: • Incorporate only firing rate and pairwise correlations • Assume parsimony (Occam’s Razor)

  15. Main features of model: • Incorporate only firing rate and pairwise correlations • Assume parsimony (Occam’s Razor)

  16. spikes Represent activity in each neuron as a spin: -1 -1 -1 -1 1 1 -1 1 -1 1 -1 -1 -1 -1 -1

  17. Average activity in a neuron is given by the time average of this spin: < -1 -1 -1 -1 1 1 -1 1 -1 1 -1 -1 -1 -1 -1 >

  18. < -1 -1 -1 -1 1 1 -1 1 -1 1 -1 -1 -1 -1 -1 > < 1 -1 -1 -1 -1 1 1 -1 -1 -1 1 1 -1 -1 -1 > We can also look at the time average of the product of spin pairs:

  19. Extract from the data: Firing rate: Pairwise correlations:

  20. The observed quantities <si> and <sisj> will influence the spins: (Ising model)

  21. Where <si> is the local magnetic field hi And <sisj> is the couplingJij hi = Jij = (Ising model)

  22. h2 = +0.5 J12 = -0.3 h1 = -0.9 J23 = -0.3 J13 = +0.9 h3 = -0.9 For example:

  23. How well each spin agrees with its connected neighbors How well each spin agrees with its local magnetic field A particular choice of h and J values will assign different energies to each spin configuration:

  24. Main features of model: • Incorporate only firing rate and pairwise correlations • Assume parsimony (Occam’s Razor)

  25. Puzzle: What does it mean to have maximum parsimony? Or to have the least complex model? Can we quantify this somehow?

  26. If you had to draw a curve through these points, what would you draw?

  27. If you had to draw a curve through these points, what would you draw? Probably not!

  28. If you had to draw a curve through these points, what would you draw?

  29. If you had to draw a curve through these points, what would you draw? Why is this more likely?

  30. Probability 1 2 3 4 5 Number If you were going to pull a number (1, 2, 3, 4, 5) out of a hat, what probability distribution would you assume?

  31. 5 Probability 4 3 2 1 Number If you were going to pull a number (1, 2, 3, 4, 5) out of a hat, AND I told you the mean value of the distribution was 4, what probability distribution would you assume?

  32. 5 Probability Probability 4 1 2 3 4 5 3 2 1 Number Number In the first case, you chose the maximum entropy distribution with no constraints In the second case, you chose the maximum entropy distribution subject to the constraint of a given mean

  33. Maximizing entropy is maximizing uncertainty. Maximizing entropy is also maximizing simplicity.

  34. Probability Energy Regarding the energies of the system: Assume that there is a mean energy Assume further that there is a maximum and a minimum energy How will the system populate these energy states? Maximum Entropy

  35. Probability Energy This is just a Boltzmann distribution:

  36. Now we can see how the different spin configurations will be distributed: The lowest energies are the most probable; the highest energies are the least probable.

  37. So, our task is to select a set of h and J values that give a distribution of spin configurations most like that observed in the data h and J are really Lagrange multipliers, and this is a problem from the calculus of variations

  38. Steps in procedure: • Pick some set of h, J values • Plug these into the energy formula • Calculate the energy of each configuration • Calculate the probability of each configuration (Boltzmann distribution) • Compare the model’s probability distribution with the data’s • Adjust the h and J values to produce a better fit with the data • Repeat loop until satisfactory fit is obtained

  39. Outline • Correlated states • Maximum entropy • Spatial correlations • Temporal correlations

  40. Assuming that this can be done, how well does the model perform? Schneidman, et al, 2006

  41. Assuming that this can be done, how well does the model perform? Data from our lab and colleagues

  42. Assuming that this can be done, how well does the model perform? Works for cortical tissue, not just retina Works for local field potentials

  43. Information from model Information from data While this may “look” good, is there any objective way to quantify performance?

  44. In their studies, the model accounted for ~90% of the available information Schneidman, et al, 2006

  45. In our studies, it did about the same on a variety of tissues:

  46. Outline • Correlated states • Maximum entropy • Spatial correlations • Temporal correlations

  47. Can the model predict sequences? Is activity in these five time bins correlated over time? Schneidman, et al, 2006

  48. Concatenated states from the model compared to data 9/13 length distributions significantly differ

  49. The model can predict spatial, but not temporal, correlations. Could the model be extended to account for temporal correlations?

More Related