Statistics of natural sensory signals: A key to brain function and efficient signal processing

1. Statistics of natural sensory signals: A key to brain function and efficient signal processing Lecture 1: Early auditory and visual system Lecture 2: Statistics in natural images Lecture 3: Ecological theories of sensory processing Lecture 4: Learning of image representations Lecture 5: Learning of sound representations

2. Lecture 1: Early audition and early vision

3. Three major points: ambient air pressure low pressure low density high pressure high density • Translating F into Y is an issue. • Psychophysics tries to quantify that translation. • For the sense of touch, the translation is faulty. These will recur—with more detail—in senses that have been studied more. Hearing What is the relevant F energy? Sound pressure. air molecules pushed away by string leave a gap and bunch up High & low pressure Sound Wave

4. Frequency (# per second) high pressure ambient low pressure amplitude U time l Tuning Fork  single frequency Dropped pot complex combo of frequencies Y loudness pitch timbre Sound Pressure Levelis a measure of force  relative to static ambient atmospheric pressure F amplitude frequency complexity

5. Loudness Amplitude • a cone for collecting sound • a cone for collecting sound • parts that vibrate to transmit the wave • some way to note amplitude and frequency outer ear outer ear middle ear inner ear cochlea As with touch, Y experience does not map F variables strictly. • Not 1:1 • Different functions for different ƒs. (solid line is 1,000 Hz) What kind of anatomy underlies such functions? What kind of code does it (can it) use? Consider the ear as a chamber for capturing sound pressure waves. What properties should it have?

6. inside is the basilar membrane Uncoil cochlea: • increases in thickness • decreases in stiffness Receptors sit on membrane and bend according to place and rate of bulges. • bulged by waves in fluid C • Wave travels via bending surface, levers, & fluid. • The inner ear translatesF to neural. Large Scale Movie a neural code for F properties: Different stiffnesses have different resonant frequencies Place of maximal wave depends on frequency of sound: Place Theory Shape of wave envelope depends on amplitude of sound Sound wave properties are copied in neural code. Y“looks like” F

7. As the basilar membrane moves, the hair cells move and the hairs are deflected, creating a nerve impulse. Place of maximal wave depends on frequency of sound: Place Theory: different cells, different pitches At low frequencies, basilar membrane moves as a unit Frequency Theory: Frequency tone=Frequency impulses

8. Review: • We transduce the presence of a variety of properties • The air pressure waves created by vibrating objects • Gathered by outer ear • Mechanically transferred by Ear Drum, Malleus, Incus, and Stapes • Motion of Basilar Membrane • Deflects hairs, simulates hair cells (receptors) • Two means of coding: • Place theory • Frequency theory • Now Vision

9. source reflected scattered absorbed Some light gets to eye surfaces, substances • First Contact in Vision : energy-->neural impulses-->sensation • Anatomy suited to F properties • Anatomy influences coding—copies F properties Light: The stimulus for vision • Light travels straight • good for image-production • Light travels fast • we can know them immediately • Light travels far • we can know about far objects What properties should the eye have?

10. fovea lens cornea optic nerve pupil retina iris Image Production Image Production What properties should the eye have? Optical parts Structures for gathering and focusing light Translating parts Structures for copying light and sending signals Inverted image is projected on the retina. Light scatters in many directions Some passes through pupil, lens. Lens changes shape to accommodate distance of object to size of eye.

11. threshold • 1st acts fast, adapts less. • 2nd is slow but adapts more. minutes in dark So does shape: rods and cones Transducing light How do we know what’s in the eye, what does what? Before we had techniques to see cells, we had behavioral data: Go from bright light into dark room—can’t see at first. Improves for 5 min., levels off… improves again for 15–20 min. Kink in function is clue: There are 2 functions 2 functions  2 jobs  2 types of photoreceptors Location, number, sensitivity, connections differ.

12. Rods • many:1 with later cells • greater sensitivity Cones • 1:1 with later cells •  greater acuity • more plentiful • throughout retina • fewer in number • fovea only has cones [No receptors where optic nerve leaves eye: blind spotgap in image] * Where light hits affects whether it’s noticed

13. Blind spot: Cover left eye; look at cross with right eye +

14. N L W Z X H P F J Q D S Y K C H G V M B R T X E Fovea and the distribution of acuity

15. Rods • many:1 with later cells • greater sensitivity Cones • 1:1 with later cells •  greater acuity • more plentiful • throughout retina • fewer in number • fovea only has cones [No receptors where optic nerve leaves eye: blind spotgap in image] * Where light hits affects whether it’s noticed • Two types of receptors allow eyes to: • work in dim and bright light • provide sensitivity and clarity • work in B&W and Color Receptors outnumber cells in the next layer pooling of information, editing, altering before signals are passed along

16. Review: • Visual system: Contact with light. • Image formation • Bending light by cornea • Limiting by iris • Focusing by lens • Image on the retina • Fovea: acuity • Transduction: 2 receptor types • Rods: dim illumination; no color • Cones: color vision • Next: The issue of Color Vision

17. THREE KINDS OF CONES: “BLUE”“GREEN” or “RED” SHORT MIDDLE LONG l l l 100% 0% SENSITIVITY CURVES MAX ABSORPTION (millionth of a meter) 400 500 600 nm Wavelength (l) Young-Helmholtz Trichromatic Theory of 19th Century ANY COLOR = someblue, somegreen, somered QUESTION: YELLOW?

18. (1) Color blindness comes in pairs (2) Some mixes of light yield gray R-G & B-Y complementary colors: Y Trichromatic theory not the whole story (3) Color afterimages

19. l l G R G B B R Y Y Y Y Opponent Process Theory: Perhaps outputs of cones are re-coded somewhere into pairs whose members are antagonists (Hurvich & Jameson, 20th Century)

20. optic nerve RODS CONES fovea retina BIPOLARS GANGLIONS light LIGHT DESIGN OF RETINA To Brain TO OPTIC NERVE

21. CONES CONES – – + GANGLION GANGLION FOR BOTH OPPONENT PROCESS SYSTEMS: IF + = –, THEN “GRAY” (ACHROMATIC) – + + A NEURAL SYSTEM OF OPPONENT PROCESSES IF + > –, THEN “BLUE” IF – > +,THEN “YELLOW” Most common color blindness: No red versus green IF + > –, THEN “RED” IF – > +,THEN “GREEN”

22. F: Central squares reflect same amount of light. Y: The darker the surround, the lighter they look. INTERACTION OF CELLS CODE COLOR, ALSO… Brightness Contrast

23. B ALEFTlooks darker than ARIGHT B A A • Subsequent connections • end-to-end • sideways If signal from B exceeds threshold of laterally connecting cells, signal from A will be reduced Signal from BRIGHT does not affect ARIGHT Consequently, ALEFT < ARIGHT Implies interaction in connections between neighboring cells: some signals boosted, some signals reduced Initial “strength” of signals (registered by rods) ALEFT = ARIGHT BLEFT > BRIGHT • excitatoryor inhibitory Signal from BLEFT inhibits signal from ALEFT: lateral inhibition

24. Mechanism distorts Y relative to F: Illusion In less contrived circumstances, this same mechanism enhances the detection of an important feature of the world  edges

25. Part I: Solving the problem of F Y • Receptors copy properties as best they can (omissions, distortions, errors) • Signals travel to specialized brain mechanisms (broadly: language, space; sensory, motor) More specific copies or representations are needed. Regions of left & right eyes correspond in cortex • cortical cells form retinotopic or topographic maps of LVF and RVF • spatially distorted, reflecting importance of receptor region *receptor signals do not remain separate* many:1 (or a few:1) from receptors to next cells pooling, editing informationconstruction of representations

26. side view front view • many:1 with later cells • greater sensitivity • 1:1 or few:1 with later cells • greater acuity sends excitatory signal when stimulated sends inhibitory signal when stimulated

27. maximum rate: stimulus “fits” field modest rate: stimulus smaller than field …but not orientation. reduced rate: stimulus hits both excitatory and inhibitory cells ” Lateral inhibition-type mechanisms • Separate receptors are connected • pool info to ganglion cells • some excite; some inhibit •  collection is ganglion’s receptive field Receptive fields “care about” size & shape… But since orientation influences what objects mean  Pool some more.

28. Some prefer vertical, others prefer horizontal or oblique: simple cells response rate • maximal response to stimuli of a particular orientation ±15°. response rate Receptive fields overlap Across a collection of receptive fields, orientation matters to cells in the cortex. They have receptive fields too Record from 3 cortical cells

29. response rate orientation direction Some prefer movement of those features in a particular direction: complex cells • Complex response (to orientation & size & motion direction) Cortical Cells as Feature Detectors? • Provide some where, what, and what’s it doing. • ambiguity: response is reduced if orientation or size or motion is not exact  which is it? rate of firing is only vocabulary

30. each map extracts some property • topography preserves spatial arrangement Multiple representations of the retina in cortex in excess of 100,000,000 cells again • Hierarchical organization (simple to complex) of visual system builds up ever-better representations of world. • Form is not meaning, however.

31. Review • Coding of properties of Visual Stimulation • Color: Trichromatic and Opponent Processes • Contrast: Lateral Inhibition yields Brightness Contrast • Feature Detectors of various sorts • Not just in CATS! Motion aftereffects • The eye does not send the brain a picture.