File 1 Topic: Evidence-based decisions

1. File 1 Topic: Evidence-based decisions • Evidence can be used to improve your life and world • Scientific method is the most powerful form of evidence creation • It can help us calibrate our intuition and discover more about ourselves • Let's start with our own education

2. Student predictions over time

3. Student predictions over time • Everyone knows that the most common grade is generally in the C range, yet the average prediction for ourselves is a full grade higher. • Why are we biased toward optimistic predictions for ourselves?

4. Evidenced-based decisions • Can we do things to improve our grades? • How can we know what to do? • What I read at Facebook? • Data? • Thoughtfully collected and evaluated data?

5. Scores diverge over time

6. Attendance

7. 2012) +

8. Did you print and use the pdf copies of the instructor's slides? 

9. Hypothesis: Attention is essential to significant learning and creative thought. • Let's do a little science on attention: • Hypothesis: Multitasking reduces attention and performance on each component task • Test case: driving while using cell phone

10. Indirect Evidence • fMRI: measures change in blood flow related to neural activity • Part of brain used when driving shown. Parts used in speech/listening not shown Adam, Kellera and Cynkara, Brain Research,1205, 2008, pp 70-80.

11. Multitasking Direct Evidence: a driving simulator to test performance. Results: "Driving performance while cell-phoning, is equivalent to that of being legally drunk." http://www.npr.org/templates/story/story.php?storyId=95702512 Strayer, HUMAN FACTORS, Vol. 48, No. 2, Summer 2006, pp. 381–391.

12. Your GPA • Experiment involving 62 undergraduate students, half of the cohort was allowed to text during a lecture and half had their phones turned off. After the lecture both groups took the same quiz and the students who did not text scored significantly higher on the quiz. • Ellis, Y. et al., (2010). The effect of multitasking on the grade performance of business students. Research in Higher Education Journal, 8. http://www.aabri.com/manuscripts/10498.pdf

13. Multitasking • Conclusions from various studies: • When switching between activities the brain losses some information • Much of this loss is hidden from the performer • Multitaskers overestimate their ability to perform • Psychiatrist Richard Hallowell, 2007: "...Multitasking is a mythical activity in which people believe they can perform two or more tasks simultaneously.” [without loss of performance]

14. Attention • Why are you here? • To get a degree • to make more money? • to earn a meaningful job? • to learn skills? • to learn information? • Skills (practical, intellectual, emotional), not diplomas, determine success. The diploma is useful to the extent that it represents your achievement in these areas. • Attention and mindfulness lead to all significant skill

15. We are not objective or fully conscious • We unconsciously hide and distort information for a variety of reasons: • Side effect of some other trait (e.g., blind spot) • Minimize distractions (e.g., blind spot) • Self Deceit:to carry out a self-promoting agenda • How often have you seen people unaware of their selfishness even when it is obvious to everyone else?

16. Experiment • Dunning and Kruger, 1999 • Tested people in some area of knowledge like logical reasoning, knowledge about STDs and how to avoid them, emotional intelligence, etc. • Then asked them how well they think they've done: "what percentile will your performance fall in?"

17. Perceived ability V. Performance (Dunning and Kruger, 1999)

18. Experiment • Close/cover left eye • Stare at black cross with right eye • Can you see all of the red rectangle w/ your peripheral vision? • Our brains hide from us the fact that their is lost information • Something analogous to this happens when we multitask

19. Example of how our brains hide information about ourselves Our literal blind spot VertebrateOctopus nerves > retina nerves < retina Vertebrate: 1) rods/cones detect light. (2) Nerves convey that information to brain. (3) Optic nerve. What's wrong with vertebrate anatomy?

20. Which gray square looks darker? The one on the LEFT is darker

21. Our brains try to maximize contrast of adjacent colors by perceiving them as more different than they actually are

22. Stare at the white circle and wait to see the persons “aura”

23. Summary • Our brains do not process information in a fully objective way • Our senses do not provide objective information • Therefore, we must be very careful to use methods of generating knowledge that help overcome our limitations

24. Outline: Scientific literacy and evidence-based decision making • The scientific method • Evaluate claims • Evolution of knowledge • Public policy • What and why evidence is ignored

25. (1) An observation ↓ (2) Propose a hypothesis (explanation) ↓ (3) Test the hypothesis through methods that make predictions ("if H is true then...") ↓ (4) Results (new observations) & Conclusions: hypothesis supported, or, not The Scientific Method (SM)

26. Hypotheses are constructed to be falsified. • For example, (H1) "All humans are mortal" is unfalsifiable. One would have to observe a human living forever to falsify that claim. In contrast, • (H2) "All humans are immortal" is simpler to falsify: one dead human proves the statement wrong. • After many observations, we can provisionally accept the alternative, H1

27. Science Is Empirical Information gained by observation or experiment. It must be rational, testable, & repeatable. The empirical nature of the scientific approach makes it self-correcting: Incorrect ideas are eventually discarded in favor of more accurate explanations. Truth current belief Time

28. Self-Correcting What happens when something you believe is not supported by evidence? If science proves some belief of Buddhism wrong, then Buddhism will have to change. Dalai Lama, 2005 Scientists strive to be as rational as the Dalai Lama, but like all humans, we have our own prejudices about nature.

29. Examples of the SM The scientific method is a general strategy for learning, not just about nature. We will look closely at how it works by looking at a specific questionthat has recently begun to benefit from analysis using the scientific method: How reliable is eyewitness testimony in criminal courts?

30. Eyewitness Testimony For more than 200 years, courts in the United States have viewed eyewitness testimony as unassailable. 1)Make observations. Justice Department reviewed 28 criminal convictions that had been overturned by DNA evidence. The strongest evidence against the defendants had been eyewitness ID.

31. Which below is a hypothesis regarding the use of eyewitness testimony? Is eyewitness testimony always accurate? Eyewitness testimony is always accurate. If eyewitness testimony is always accurate, then DNA and physical evidence should support the story of the eyewitness. Eyewitness testimony is accurate at least 99% of the time. None of the above. • 2) Formulate a hypothesis

32. The goal is to Propose a situation that will give a particular outcome if your hypothesis is true… …but that will give a different outcome if your hypothesis is not true. 3) Devise an experiment that makes a prediction.

33. 3) Devise an experiment that makes a prediction. Prediction:Individuals who have witnessed a crime, will correctly identify the criminal > 99% of the time.

34. Ex, Mock crime with witnesses. Ask the witnesses to identify the criminal. (Perform multiple experiments to test prediction in different ways) Create two groups, those who: View 6 suspects simultaneously View them sequentially. If eye witness testimony is correct 99% of the time, our predictions should come true: 3) Devise an experiment that makes a prediction.

35. Results When suspects were viewed together (line up), the witnesses identified the wrong person about 1/3 of the time. When suspects were viewed one at a time, the observers identified the wrong person less than 10% of the time.

36. Conclusions The results demonstrated that our hypotheses – eyewitness testimony is always (or 99%) accurate – is not supported by the data. Our experiment suggests that the accuracy of an eyewitness’ testimony: has a substantial error rate depends on the method used to present suspects

37. New Cycle of Experiments Further research found that when viewed together, witnesses tend to pick the suspect who most resembles the person they saw. This led to the higher false positive rate (falsely believing something is true, e.g., falsely identifying someone who did not commit the crime )

38. Hypotheses & Theories General public: Theory thought of as a hunch, guess, or speculation (~hypothesis to scientists). Scientists: Theoryis a hypothesis that has withstood the test of time (many experiments, lines of evidence, etc.), and has been provisionally accepted as fact. Theories are never proven.

39. Scientific Theories • Well known scientific theories… • Atomic Theory: Matter is composed of atoms • Cell Theory: Organisms are composed of cells • Evolutionary Theory: populations of organisms change through successive generations. Species present today arose from a common ancestor

40. Scientific Method Example • An example of hypothesis testing is Louis Pasteur’s experiment regarding the spontaneous generation of life (mid 1800’s)…

41. Scientific Method Example • Observation: Life seems to sprout from non-living things. time

42. Scientific Method Example • Hypotheses (falsifiable) or… 1) Life arises spontaneously 2) Life arises from microscopic material outside the flask

43. Scientific Method Example Predictions: • If "life arises spontaneously", then a sterilized flask of broth should produce life • If "life can arise from microscopic material outside the flask", then a sterilized flask of broth should produce life only if exposed to microbes.

44. Scientific Method Example Bacteria can't move themselves Experiment 1 time Sterile broth/flask with particle trap No growth

45. Scientific Method Example Experiment 2 time Particle trap removed Growth occurred

46. Scientific Method Example Experiment 2b (alternative design) time Flask tipped, mixing dust into broth Growth occurred

47. Scientific Method Example Conclusion • Hypothesis 1 rejected, hypothesis 2 not rejected. • Conclusion: No growth occurs in the sterile broth unless something is introduced from the outside. • Over the last 150 years no one has found life to spontaneously arise from non-living material. • 'Life comes from life' is a theory that has yet to be falsified

48. Tools of Experimentation • Replication: experiments are repeated to verify and allow statistical analysis time time

49. Tools of Experimentation • A treatment group and a control group. • The control group is identical to the treatment group, except for the singlevariable of interest whose effect is being tested, which is only applied to the treatment group • Negative control: no change expected. Or baseline measure. • e.g., effect of fertilizer. • treatment: fertilizer: growth= 2"/day • What is the effect of fertilizer? • neg control: no fertilizer: growth= 1"/day

50. Tools of Human Experimentation • Blind: test subjects do not know which is treatment and which is control (not always possible). • Single blind: test subjects don't know if they are in the treatment or control group. • Doubleblind: researcher also doesn't know which group the test subjects are in. • Purpose? • Placebo: a sham treatment to keep the participants 'blind', and other reasons.