slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Locating Variance: Post-Hoc Tests PowerPoint Presentation
Download Presentation
Locating Variance: Post-Hoc Tests

Loading in 2 Seconds...

play fullscreen
1 / 23

Locating Variance: Post-Hoc Tests - PowerPoint PPT Presentation


  • 153 Views
  • Uploaded on

Locating Variance: Post-Hoc Tests. Developing Study Skills and Research Methods (HL20107). Dr James Betts. It is easy (i.e. data in  P value out) It provides the ‘Illusion of Scientific Objectivity’ Everybody else does it. Why do we use Hypothesis Testing?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Locating Variance: Post-Hoc Tests' - ban


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Locating Variance: Post-Hoc Tests

Developing Study Skills and Research Methods (HL20107)

Dr James Betts

why do we use hypothesis testing
It is easy (i.e. data in  P value out)

It provides the ‘Illusion of Scientific Objectivity’

Everybody else does it.

Why do we use Hypothesis Testing?
problems with hypothesis testing
P<0.05 is an arbitrary probability (P<0.06?)

The size of the effect is not expressed

The variability of this effect is not expressed

Overall, hypothesis testing ignores ‘judgement’.

Problems with Hypothesis Testing?
lecture outline
Lecture Outline:
  • Influence of multiple comparisons on P
  • Tukey’s HSD test
  • Bonferroni Corrections
  • Ryan-Holm-Bonferroni Adjustments.
slide5

Placebo

Placebo

Lucozade

Lucozade

Gatorade

Gatorade

Powerade

Powerade

slide6

Placebo

Placebo

Lucozade

Lucozade

Gatorade

Gatorade

Powerade

Powerade

why not multiple t tests
i.e.

Placebo vs Lucozade

Placebo vs Gatorade

Placebo vs Powerade

Lucozade vs Gatorade

Lucozade vs Powerade

Gatorade vs Powerade

We accept ‘significance’ and reject the null hypothesis at P0.05 (i.e. a 5% chance that we are wrong)

Performing multiple tests therefore means that our overall chance of committing a type I error is >5%.

Why not multiple t-tests?
slide8

Post-hoc Tests

  • A popular solution is the Tukey HSD (Honestly Significant Difference) test
  • This uses the omnibus error term from the ANOVA to determine which means are significantly different
  • T = (q)

Error Variance

n

slide14

Tukey Test Critique

  • As you learnt last week, the omnibus error term is not reflective of all contrasts if sphericity is violated

Placebo

Lucozade

  • So Tukey tests commit many type I errors with even a slight degree of asphericity.

Gatorade

Powerade

slide15

Solution for Aspherical Data

  • There are alternatives to the Tukey HSD test which use specific error terms for each contrast
    • Fisher’s LSD (Least Significant Difference)
    • Sidak
    • Bonferroni
    • Many others…
    • e.g. Newman-Kewls, Scheffe, Duncan, Dunnett, Gabriel, R-E-G-W, etc.
slide18

Trial 2

Trial 4

Trial 1

Fisher’s LSD

Bonferroni

Trial 3

slide19

Bonferroni Correction Critique

  • Correction of LSD values successfully controls for type I errors following a 1-way ANOVA
  • However, factorial designs often involve a larger number of contrasts, many of which may not be relevant.

Recovery Supp. 1

Recovery Supp. 2

See also Perneger (1998) BMJ 316: 1236

slide20

Solution for Factorial Designs

  • An adjustment to the standard Bonferroni correction can be applied for factorial designs
  • This ‘Ryan-Holm-Bonferroni’ or ‘stepwise’ method involves returning to the P values of interest from our LSD test
  • These P values are placed in numerical order and the most significant is Bonferroni corrected (i.e. P x m)
  • However, all subsequent P values are multplied by m minus the number of contrasts already corrected.
summary post hoc tests
A Tukey test may be appropriate when sphericity can be assumed

Multiple t-tests with a Bonferroni correction are more appropriate for aspherical data

Stepwise correction of standard Bonferroni procedures maintain power with factorial designs

Best option is to keep your study simple:

Pre-planned contrast at a specific time point

Summary statistics (e.g. rate of change, area under curve)

Just make an informed based on the data available.

Summary Post-Hoc Tests
slide22

Further reading from this lecture…

  • Atkinson, G. (2001) Analysis of repeated measurements in physical therapy research Physical Therapy in Sport 2: p. 194-208
  • Atkinson, G. (2002) Analysis of repeated measurements in physical therapy research: multiple comparisons amongst level means and multi-factorial designs Physical Therapy in Sport 3: p. 191-203
slide23

Compulsory reading for next week’s lecture…

  • Batterham A. M. & Atkinson, G. (2005) How Big Does My Sample Need to Be? A primer on the Murky World of Sample Size Estimation Physical Therapy in Sport 6: p. 153-163.