File 1 Topic: Evidence-based decisions • Evidence can be used to improve your life and world • Scientific method is the most powerful form of evidence creation • It can help us calibrate our intuition and discover more about ourselves • Let's start with our own education
Student predictions over time • Everyone knows that the most common grade is generally in the C range, yet the average prediction for ourselves is a full grade higher. • Why are we biased toward optimistic predictions for ourselves?
Evidenced-based decisions • Can we do things to improve our grades? • How can we know what to do? • What I read at Facebook? • Data? • Thoughtfully collected and evaluated data?
Hypothesis: Attention is essential to significant learning and creative thought. • Let's do a little science on attention: • Hypothesis: Multitasking reduces attention and performance on each component task • Test case: driving while using cell phone
Indirect Evidence • fMRI: measures change in blood flow related to neural activity • Part of brain used when driving shown. Parts used in speech/listening not shown Adam, Kellera and Cynkara, Brain Research,1205, 2008, pp 70-80.
Multitasking Direct Evidence: a driving simulator to test performance. Results: "Driving performance while cell-phoning, is equivalent to that of being legally drunk." http://www.npr.org/templates/story/story.php?storyId=95702512 Strayer, HUMAN FACTORS, Vol. 48, No. 2, Summer 2006, pp. 381–391.
Your GPA • Experiment involving 62 undergraduate students, half of the cohort was allowed to text during a lecture and half had their phones turned off. After the lecture both groups took the same quiz and the students who did not text scored significantly higher on the quiz. • Ellis, Y. et al., (2010). The effect of multitasking on the grade performance of business students. Research in Higher Education Journal, 8. http://www.aabri.com/manuscripts/10498.pdf
Multitasking • Conclusions from various studies: • When switching between activities the brain losses some information • Much of this loss is hidden from the performer • Multitaskers overestimate their ability to perform • Psychiatrist Richard Hallowell, 2007: "...Multitasking is a mythical activity in which people believe they can perform two or more tasks simultaneously.” [without loss of performance]
Attention • Why are you here? • To get a degree • to make more money? • to earn a meaningful job? • to learn skills? • to learn information? • Skills (practical, intellectual, emotional), not diplomas, determine success. The diploma is useful to the extent that it represents your achievement in these areas. • Attention and mindfulness lead to all significant skill
We are not objective or fully conscious • We unconsciously hide and distort information for a variety of reasons: • Side effect of some other trait (e.g., blind spot) • Minimize distractions (e.g., blind spot) • Self Deceit:to carry out a self-promoting agenda • How often have you seen people unaware of their selfishness even when it is obvious to everyone else?
Experiment • Dunning and Kruger, 1999 • Tested people in some area of knowledge like logical reasoning, knowledge about STDs and how to avoid them, emotional intelligence, etc. • Then asked them how well they think they've done: "what percentile will your performance fall in?"
Perceived ability V. Performance (Dunning and Kruger, 1999)
Experiment • Close/cover left eye • Stare at black cross with right eye • Can you see all of the red rectangle w/ your peripheral vision? • Our brains hide from us the fact that their is lost information • Something analogous to this happens when we multitask
Example of how our brains hide information about ourselves Our literal blind spot VertebrateOctopus nerves > retina nerves < retina Vertebrate: 1) rods/cones detect light. (2) Nerves convey that information to brain. (3) Optic nerve. What's wrong with vertebrate anatomy?
Which gray square looks darker? The one on the LEFT is darker
Our brains try to maximize contrast of adjacent colors by perceiving them as more different than they actually are
Summary • Our brains do not process information in a fully objective way • Our senses do not provide objective information • Therefore, we must be very careful to use methods of generating knowledge that help overcome our limitations
Outline: Scientific literacy and evidence-based decision making • The scientific method • Evaluate claims • Evolution of knowledge • Public policy • What and why evidence is ignored
(1) An observation ↓ (2) Propose a hypothesis (explanation) ↓ (3) Test the hypothesis through methods that make predictions ("if H is true then...") ↓ (4) Results (new observations) & Conclusions: hypothesis supported, or, not The Scientific Method (SM)
Hypotheses are constructed to be falsified. • For example, (H1) "All humans are mortal" is unfalsifiable. One would have to observe a human living forever to falsify that claim. In contrast, • (H2) "All humans are immortal" is simpler to falsify: one dead human proves the statement wrong. • After many observations, we can provisionally accept the alternative, H1
Science Is Empirical Information gained by observation or experiment. It must be rational, testable, & repeatable. The empirical nature of the scientific approach makes it self-correcting: Incorrect ideas are eventually discarded in favor of more accurate explanations. Truth current belief Time
Self-Correcting What happens when something you believe is not supported by evidence? If science proves some belief of Buddhism wrong, then Buddhism will have to change. Dalai Lama, 2005 Scientists strive to be as rational as the Dalai Lama, but like all humans, we have our own prejudices about nature.
Examples of the SM The scientific method is a general strategy for learning, not just about nature. We will look closely at how it works by looking at a specific questionthat has recently begun to benefit from analysis using the scientific method: How reliable is eyewitness testimony in criminal courts?
Eyewitness Testimony For more than 200 years, courts in the United States have viewed eyewitness testimony as unassailable. 1)Make observations. Justice Department reviewed 28 criminal convictions that had been overturned by DNA evidence. The strongest evidence against the defendants had been eyewitness ID.
Which below is a hypothesis regarding the use of eyewitness testimony? Is eyewitness testimony always accurate? Eyewitness testimony is always accurate. If eyewitness testimony is always accurate, then DNA and physical evidence should support the story of the eyewitness. Eyewitness testimony is accurate at least 99% of the time. None of the above. • 2) Formulate a hypothesis
The goal is to Propose a situation that will give a particular outcome if your hypothesis is true… …but that will give a different outcome if your hypothesis is not true. 3) Devise an experiment that makes a prediction.
3) Devise an experiment that makes a prediction. Prediction:Individuals who have witnessed a crime, will correctly identify the criminal > 99% of the time.
Ex, Mock crime with witnesses. Ask the witnesses to identify the criminal. (Perform multiple experiments to test prediction in different ways) Create two groups, those who: View 6 suspects simultaneously View them sequentially. If eye witness testimony is correct 99% of the time, our predictions should come true: 3) Devise an experiment that makes a prediction.
Results When suspects were viewed together (line up), the witnesses identified the wrong person about 1/3 of the time. When suspects were viewed one at a time, the observers identified the wrong person less than 10% of the time.
Conclusions The results demonstrated that our hypotheses – eyewitness testimony is always (or 99%) accurate – is not supported by the data. Our experiment suggests that the accuracy of an eyewitness’ testimony: has a substantial error rate depends on the method used to present suspects
New Cycle of Experiments Further research found that when viewed together, witnesses tend to pick the suspect who most resembles the person they saw. This led to the higher false positive rate (falsely believing something is true, e.g., falsely identifying someone who did not commit the crime )
Hypotheses & Theories General public: Theory thought of as a hunch, guess, or speculation (~hypothesis to scientists). Scientists: Theoryis a hypothesis that has withstood the test of time (many experiments, lines of evidence, etc.), and has been provisionally accepted as fact. Theories are never proven.
Scientific Theories • Well known scientific theories… • Atomic Theory: Matter is composed of atoms • Cell Theory: Organisms are composed of cells • Evolutionary Theory: populations of organisms change through successive generations. Species present today arose from a common ancestor
Scientific Method Example • An example of hypothesis testing is Louis Pasteur’s experiment regarding the spontaneous generation of life (mid 1800’s)…
Scientific Method Example • Observation: Life seems to sprout from non-living things. time
Scientific Method Example • Hypotheses (falsifiable) or… 1) Life arises spontaneously 2) Life arises from microscopic material outside the flask
Scientific Method Example Predictions: • If "life arises spontaneously", then a sterilized flask of broth should produce life • If "life can arise from microscopic material outside the flask", then a sterilized flask of broth should produce life only if exposed to microbes.
Scientific Method Example Bacteria can't move themselves Experiment 1 time Sterile broth/flask with particle trap No growth
Scientific Method Example Experiment 2 time Particle trap removed Growth occurred
Scientific Method Example Experiment 2b (alternative design) time Flask tipped, mixing dust into broth Growth occurred
Scientific Method Example Conclusion • Hypothesis 1 rejected, hypothesis 2 not rejected. • Conclusion: No growth occurs in the sterile broth unless something is introduced from the outside. • Over the last 150 years no one has found life to spontaneously arise from non-living material. • 'Life comes from life' is a theory that has yet to be falsified
Tools of Experimentation • Replication: experiments are repeated to verify and allow statistical analysis time time
Tools of Experimentation • A treatment group and a control group. • The control group is identical to the treatment group, except for the singlevariable of interest whose effect is being tested, which is only applied to the treatment group • Negative control: no change expected. Or baseline measure. • e.g., effect of fertilizer. • treatment: fertilizer: growth= 2"/day • What is the effect of fertilizer? • neg control: no fertilizer: growth= 1"/day
Tools of Human Experimentation • Blind: test subjects do not know which is treatment and which is control (not always possible). • Single blind: test subjects don't know if they are in the treatment or control group. • Doubleblind: researcher also doesn't know which group the test subjects are in. • Purpose? • Placebo: a sham treatment to keep the participants 'blind', and other reasons.