How Not to Present Multivariate Data

1. How Not to Present Multivariate Data Harry R. Erwin, PhD School of Computing and Technology University of Sunderland

2. Background • This lecture is intended to provide some good bad examples of data visualization and animation, and discuss why they are bad. • It does depend on some understanding how we see and understand what we see. • So, no math, but some cognitive science.

3. Resources • Everitt, BS, and G Dunn (2001) Applied Multivariate Data Analysis, London: Arnold. • Everitt, BS (2005) An R and S-PLUS® Companion to Multivariate Analysis, London: Springer • Tukey’s seminal paper: • http://www.edwardtufte.com/tufte/tukey • Tufte’s work: • http://en.wikipedia.org/wiki/Edward_Tufte • Murrell, P, (2006) R Graphics, Florida: Chapman & Hall/CRC. • Cleveland, William S. (1993) Visualizing Data, Hobart Press. • Notes from INFT 875/CSI 803, Spring 1995, George Mason University.

4. Human Understanding • People perceive in various ways: • Visual/spatially • Auditory iconically • Emotionally • Verbally • Symbolically • Operationally • Transformations are a key idea.

5. Neuroscience Resources • Nicholls, Martin, Wallace, and Fuchs, 2001, From Neuron to Brain, 4rd edition,Sinauer. (Good for references, unless otherwise indicated, the primary reference for this lecture) • Kandel, Schwartz, and Jessell, 2000, Principles of Neural Science, 4th edition, McGraw Hill. Covers the vestibular-ocular reflex. • Dowling, 1992, Neurons and Networks, Belknap Harvard.

6. A Few Points from Kandel • Vision is a creative process. • Visual information is processed in parallel by multiple cortical areas • Motion, depth, form, and color are handled separately. • Two major pathways (dorsal or parietal for spatial/color and ventral or temporal for object recognition) • One minor pathway (LGN—Superior Colliculus) • Conversion between frames of reference is necessary, especially in depth perception. • Role of visual attention is probably important, particularly in proposing possible matches.

7. The Retina and Laternal Geniculate Nucleus (LGN) • A portion of the CNS exposed to direct experimental observation. • Multiple layers • Five main classes of neurons • Uses both electrical and chemical synapses. • Action potentials are used to communicate down the optic nerve to the lateral geniculate nucleus (LGN) of the thalamus.

8. Retinal Anatomy From <http://thalamus.wustl.edu/course/eyeret.html>

9. Structure of the Retina Back Front From <http://thalamus.wustl.edu/course/eyeret.html>

10. The retina has seven layers, from inside to outside: Optic nerve fibers Ganglion cells Inner plexiform layer Horizontal, bipolar, and amacrine cells Outer plexiform layer Photoreceptors Choroid To reach the photoreceptors, light must pass through five of these layers! Details

11. Photoreceptors From <http://www.cis.rit.edu/people/faculty/montag/vandplite/pages/chap_9/ch9p1.html>

12. Receptor organization • Outer segment traps the light using visual pigment. This is a modified cilium. Membrane potentials of about -40 mV due to the ‘dark current’. • Inner segment contains the nucleus and organelles

13. Rods • Rods are black/white receptors. About 100,000,000 proteins (rhodopsin) per rod, stored on several hundred disks. Rhodopsin is most sensitive to blue-green. Changes configuration in about 10-12 seconds. Regenerates over minutes. • Cyclic GMP binding to cell membrane mediates the amplification of the signal. Produces constant output using ligand-activated channels that pass most cations. Light blocks cGMP binding. • A single photon can be detected. The activation of seven rods by a photon can be consciously perceived. • Rods become inactive during the day.

14. Cones • The color receptors. The disklikeinfoldings holding the visual pigments are portions of the cell membrane. All three color pigments are closely related to rhodopsin. On X-chromosome, and red is the usual pigment involved in colorblindness. • Mostly red (64%), some green (34%), and a few blue (2%), the three primary colors. Some girls can see four colors. • Concentrated in the fovea. • Comparison of the outputs of different types of cones produces color vision (Devalois and Devalois).

15. Structure of the Retina Back Front From <http://thalamus.wustl.edu/course/eyeret.html>

16. Retinal Processing • Without light, receptors release glutamate (the “Dark Current”). • The receptors hyperpolarize in response to light at a rate that reflects the rate at which photons are received. Cones are much less sensitive than rods. That stops glutamate release. • Horizontal cells produce GABA and interact directly with the receptors in a feedback relationship. Also release GABA to bipolar cells and communicate with other horizontal cells by gap junctions. • Bipolar cells receive input from the receptors (Glu) as long as there is no light. • Amacrine cells synapse on bipolar and ganglion cells. • Ganglion cells receive inputs from bipolar and amacrine cells and generate action potentials that travel to the LGN.

17. Receptive Field • Key concept (Sherrington and later Hartline) • Also applies in the cortex • “The receptive field of a neuron is the area on the retina from which the activity of the neuron can be influenced by light”. • There are neurons in the auditory cortex with visual receptive fields.

18. Structure of the Retina Back Front From <http://thalamus.wustl.edu/course/eyeret.html>

19. Bipolar Cells • Produce graded sustained changes in polarization based on the Glu input. Some depolarize and some hyperpolarize based on their receptor types. Report small spots of darkness in light or light surrounded by darkness. Output via chemical synapses. • Glutamate is inhibitory for on-center (H) bipolar cells (metabotropicGlu receptors), excitatory (normal) for off-center (D) bipolar cells. • Rod bipolar cells listen to 15-45 rods. Detect large spots of light (D). • Midget bipolars listen to a single cone and are concentrated in the fovea. Both H and D. • Other cone bipolars listen to 5-20 adjacent cones.

20. Horizontal Cells • Release GABA continuously if not activated. Activation by receptors causes them to cease GABA release, preventing the cone and rod receptors from signaling light detection. • Play a role in ‘center/surround’ detection. • Best stimulus is illumination of a large area of the retina.

21. Amacrine (‘no-axon’) Cells • Rod bipolars do not connect directly to ganglion cells, but rather indirectly via amacrine cells. • In response to light, the rod bipolar depolarizes and releases Glu onto an amacrine cell. • The amacrine cell generates an action potential, but has no axon. They output (+) via gap junctions (electrical synapses) to depolarizing cone bipolar cells and via Gly release (-) to ‘off’ ganglion cells, producing their ‘off’ responses. • The depolarizing cone bipolars then trigger the ‘on’ ganglion cells. • Hence the rod and cone systems trigger the same ganglion cells.

22. Ganglion Cells • Output of the retina: ‘on’ and ‘off’-center receptive cells • Two main categories of each (M & P) • M cells project to the magnocellular division of the LGN, have large receptive field centers, low spatial resolution, are not color sensitive, and handle low contrast. • P cells project to the parvocellular division of the LGN, have small receptive field centers, high spatial resolution, and are color sensitive. Require high contrast. • The system elegantly deals with background light intensity variation. • The eye adjusts automatically to a change in the light background, if necessary, switching between rods and cones.

23. Visual Cortex • 6-layered neocortex, moderately specialized, which appears to consist of general-purpose computational elements. • The connections and functions have been mapped out, but… • We don’t yet understand in any detail how neocortex performs its computational functions • We know M and P cell inputs are kept separate in vision. This is an important structural constraint. • Lesions in early stages of processing result in gaps in the visual field—’neglect’. The mind pretends they don’t exist. • Perceptive illusions appear to reflect the processing of the individual areas.

24. Primary Visual Cortex • Primary visual cortex, V1, striate cortex, or area 17. V2, which surrounds V1, is also visual. • Most (80%) cells are already binocular. • Receptive fields result in simple and complex cells • Simple cells detect a number of patterns, but an important one is a short bar based on: position, area, and angle • Edge detectors are also present, with length constraints. • Diffuse background illumination is ignored.

25. Complex Cells in V1 • Abundant in layers 2, 3 and 5 • Specific field axis orientation of a dark/light boundary • Diffuse illumination is ignored. • Accept freely positioned stimuli • Detects orientation without strict reference to position. • Respond best to moving edges or slits.

26. Role of Left and Right Visual Cortices • Each hemicortex handles half of the visual world, but using inputs from both eyes. • Right/left connections exist between the hemicortices at the border. • V1 communication is via the corpus callosum and involves cells in layer 3.

27. Two Visual Pathways • Dorsal or parietal for spatial/color—“Over the top”. Parvocellular • Ventral or temporal for object recognition —“Down the side”. Magnocellular • There is evidence for a third pathway (“blindsight”) and perhaps of a fourth.

28. Motion, Depth, and Form • Motion is a dorsal function • The middle temporal region solves the aperture problem (partially hidden motion). • Depth makes use of cues (LR) and binocular disparity (SR) • Combined in V1 • Object vision is a ventral function • V2 detects contours including illusions • V4 detects form • Complex forms (faces) in the inferior temporal cortex

29. Color Vision • Captures properties of surfaces • Poor at capturing spatial detail • Imagine a Dalmatian dog. Now count its spots. • Color transformations are early in visual processing. P cells respond to: • Opposed signals from red and green-sensitive photoreceptors • Opposed signals from blue-sensitive photoreceptors and some combination of red and green. • There are multiple color pathways in the cortex.

30. What Does This All Mean • How well you perceive something depends on the details of how it is processed in the brain. • A lot of our intuition reflects automatic processing that we’re not even aware of.

31. Cleveland’s Recommendations Based on Cognitive Science • From Best • Position data on a common scale • Position data on non-aligned identical scales • Use line length as the cue • Use angle-slope as the cue (pie charts) • Use area as the cue • Use volume or density as the cue • Use a colour cue • To Worst

32. Problem areas • We have particular problems with slopes, so transform them to something else for visualisation. • The brain also doesn’t recognise differences among vertical lines—avoid.

33. The Process of Understanding Data (Cleveland) • Pattern perception • Detection (of data points) • Assembly (grouping of common objects) • Estimation (identification of relationships) • Table lookup • From a value to a label • Readout • Interpolation

34. So Let’s Look at Some Examples • http://www.math.yorku.ca/SCS/Gallery/