Evaluation of Statistical Reports

1. Evaluation of Statistical Reports AS 3.12, Externally assessed, 4 credits Experiments and Observational Studies

2. Claims made in Newspapers every day – do you believe them?

3. Does listening to music help students study? Do drugs have an adverse effect on teenage brains? What information would you need to answer these questions? What acne treatments are the most effective?

4. Article Structure Introduction – Broad summary of story or main finding. Slightly more detail than provided in the heading. Main angle developed – further details provided More on main angle Strongest Quote More details such as results of other studies or quotes from other experts or witnesses Background information Who, What, Where, Why and How ? Identify in an article

5. What is an Experiment? • Participants randomly allocated to treatments • Variable of interest is measured and results compared between treatment groups

6. Identify which are EXPERIMENTS ? • Study which compares pulse rates of students with and without exercise • Study that compares performance in an IQ test with and without caffeinated drinks • Study of teenage binge-drinkers’ brain activity • Impact of smoking during pregnancy on infant performance ability • Effect of birth weight on adult academic performance • Effect of acupuncture or physiotherapy on back pain • Effect of exercises on age babies start walking • Study of breakfast habits and study results • Study of massage as a cure for migraines • Are food colours linked to hyperactivity in children? • Are defensive drivers better drivers? • Effect of sleep deprivation on driving ( in a simulator)

7. KEY TERMINOLOGY • Response Variables – the variable measured as the outcome of an experiment. Measure blood pressure to assess effect of new drug. • Explanatory Variables – the variable that attempts to explain differences in the response variable • Treatment- one or a combination of explanatory variables assigned by the researcher • Confounding Variables – variables connected to the explanatory variable that may be the actual cause of the differences in the response variable.

8. CONFOUNDING VARIABLES “ Imported lemons reduces highway fatalities – study reveals”

9. Identify Explanatory, Response Variables and Treatment in following studies • Study which compares pulse rates of students with and without exercise • Study that compares performance in an IQ test with and without caffeinated drinks • Study of teenage binge-drinkers’ brain activity • Impact of smoking during pregnancy on infant performance ability • Effect of birth weight on adult academic performance • Effect of acupuncture or physiotherapy on back pain • Effect of exercises on age babies start walking • Study of breakfast habits and study results • Study of massage as a cure for migraines • Are food colours linked to hyperactivity in children? • Are defensive drivers better drivers? • Effect of sleep deprivation on driving ( in a simulator)

10. How to design a good experiment • Random Allocation – participants randomly allocated to treatments and random order of treatment • Use of a Control Group – use to establish base line effect • Use of a Placebo – dummy, existing or no treatment • Use of Blinding – single and double blinding

11. What is an Observational Study? A study which observes differences in the explanatory variable and then assesses whether these differences are related to differences in the response variable Famous NZ Observational Study is the Dunedin Multidisciplinary Health and Development Study (DMHDS) .A study of 1037 children who were born in Dunedin, in 1972-73. Where they now live shown below.

12. Potential Problems with Observational Studies Confounding Variables “Happy People Live Longer” Possible confounding variables? Why is this an observational study? PROBLEM – Causal Claim cannot be made by Observational Study

13. Potential Problems with Observational Studies Extending Results inappropriately “WOMEN HIT HARDER BY HANGOVERS” The study observed only undergraduate students at one University so results cannot be extended to cover ALL WOMEN

14. Potential Problems with Observational Studies Using the past as a source of data “LEFT HANDED PEOPLE DIE YOUNGER” In early part of twentieth century children forced to write with right hands, hence greater numbers of right-handers. Confounding variables can change over time. Recalled information often unreliable.

15. COMPARISON OF EXPERIMENTS AND OBSERVATIONAL STUDIES

16. EVALUATION OF STATISTICALLY BASED REPORTS Step One – Is report based on sound research? Step Two – Identify type of study – experiment or observational? Step Three – Who funded research? Did they have vested interest in outcome? Step Four – Where reliable data collection methods used? Step Five – Who or what was studied? How were they selected?

17. EVALUATION OF STATISTICALLY BASED REPORTS Step Six – Are measurements provided? Step Seven – Familiar or unfamiliar settings? Step Eight – Size of observed effect quantified or just described? Step Nine – Confounding variables? Step Ten – Results extended appropriately? Step Eleven – Any missing data? Step Twelve – Do findings make sense?

18. EVALUATION OF STATISTICALLY BASED REPORTS Does your evaluation persuade you or someone you know to modify their behaviour? What is your overall conclusion?