1 / 28

Statistical Argument and Safe Designs for Researchers

Learn the essentials of statistical argument and safe designs for research. Understand inference, prediction, hypothesis testing, and implications of statistical findings. Explore different statistical tests, effect sizes, and multivariate analysis. Avoid common pitfalls in statistical analysis. Gain insights on choosing the right test and interpreting results effectively.

peri
Download Presentation

Statistical Argument and Safe Designs for Researchers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quantitative Methods for Researchers Paul Cairns paul.cairns@york.ac.uk

  2. Objectives • Statistical argument • Safe designs • A whizz through some stats • Time for questions

  3. Statistical Argument • Inference is an argument form • Prediction is essential • Alternative hypothesis • “X causes Y” • No prediction – measuring noise

  4. Gold standard argument • Collect data • Data variation could be chance (null) • Predictthe variations (alternative) • Statistics give probabilities • Unlikely predictions “prove” your case

  5. Implications • Must have an alt (testable) hyp • No multiple testing • No post hoc analysis • Need multiple experiments

  6. Silver standard argument • Collect data • Data variations could be chance (null) • Are there “real” patterns in the data? • Use statistics to suggest (unlikely) patterns • Follow up findings with gold standard work

  7. Fishing: This is bad science • Collect lots of data • DVs and IVs • Data variations could be chance • Test until a significant result appears • Report the tests that were significant • Claim the result is important

  8. Statistical pit… • … is bottomless! • Safe designs • One (or two) IV • Two (or three) conditions • One primary DV • Other stuff is not severely tested

  9. Choosing a test • What’s the data type? • Do you know the distribution? • Within or between • What are you looking for?

  10. Seeing location • Boxplots • Median, IQR, • “Range” • Outliers

  11. Distributions • Theoretical stance • Must have this! • Not inferred from samples

  12. Parametric tests • Normal distribution • Two parameters • Null = one underlying normal distribution • Differences in location (mean)

  13. t-test: null vs alternate

  14. t-test • Two samples • Two means • Are means showing natural variation? • Compare difference to natural variation

  15. Effect size • How interesting is the difference? • 2s difference in timings • Significance is not same as importance • Cohen’s d

  16. ANOVA • Parametric • Multiple groups • Why not do pairwise comparison? • Get an F value • Follow up tests

  17. ANOVA++ • Multiple IV • So more F values! • Within and between • Effect size, η2 • Amount of variance predicted by IV

  18. Non-parametric tests • Unknown underlying distribution • Heterogeneity of variance • Non-interval data • Usually test location • Effect size is tricky!

  19. Basic tests • Mann-Whitney • Wilcoxon • Kruskal-Wallis • Friedman • No accepted two-way tests

  20. Choosing a test For your fantasy abstract, what test would you choose? Why? Would you change your design?

  21. Questions • Specific problems • Specific tests • Other tests?

  22. Useful Reading • Cairns, Cox, Research Methods for HCI: chaps 6 • Rowntree, Statistics Without Tears • Howell, Fundamental Statistics for the Behavioural Sciences, 6thedn. • Abelson, Statistics as Principled Argument • Silver, The Signal and the Noise

  23. Multivariate • Multiple DV • Multivariate normal distribution • Normal no matter how you slice • MANOVA • Null = one underlying (mv) normal distribution

  24. Issues • Sample size • Assumptions • Interpretation • Communication

  25. Monte Carlo • Process but not distribution • Generate a really large sample • Compare to your sample • Still theoretically driven!

  26. Example • Event = 4 heads in a row from a set of 20 flips of a coin • You have sample of 30 sets • 18 events • How likely? • Get flipping!

More Related