Sample size considerations for answering quantitative research questions
Sponsored Links
This presentation is the property of its rightful owner.
1 / 25

Sample Size Considerations for Answering Quantitative Research Questions PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on
  • Presentation posted in: General

Sample Size Considerations for Answering Quantitative Research Questions. Lunch & Learn May 15, 2013 M Boyle. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21. Planning Costs 2000-2006: $54.7M Implementation Costs 2007-2011: $744.6. Sample Size Justification: ?.

Download Presentation

Sample Size Considerations for Answering Quantitative Research Questions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Sample Size Considerations for Answering Quantitative Research Questions

Lunch & Learn May 15, 2013

M Boyle


National Children’s Study in the US

Proposed Birth Cohort 100,000 to age 21

Planning Costs 2000-2006: $54.7M

Implementation Costs 2007-2011: $744.6

Sample Size Justification: ?


What is Statistical Power?

  • The statistical power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a statistically significant effect when one exists


Types of Power Analysis

A priori: Used to plan a study (Question: What sample size is needed to obtain a certain level of power)?

Post hoc: Used to evaluate a study faced with a constrained sample size (Question: Do you have a large enough sample to detect a meaningful effect)?

[Types of constraints: (1) a completed study; (2) a proposed study with limited number of eligible subjects; (3) a proposed study faced with limited resources]


Elements of Power Calculations

  • Effect size ∆

  • Measurement variability SD

  • Type I error Alpha (α)

    typically specified at p=0.05, 2-tailed

  • Type II error Beta (β)

    typically specified at p=0.20

  • Power = 1-β; typically 0.80

  • Sample Size


Hypothesized distributions, effect sizes and error rates

Effect Size ∆

Type I

Type II

Measurement Variability +/- 1 SD


Decisions

Disease Status

Present Absent

Medical Diagnosis

+ve

Test Result

-ve

correct false +ve

false -ve correct

Population Status

H0 false

H0 true[H1 true]

Hypothesis Testing

correct Type II

1-αβ

Accept H0

Decision

Reject H0

[Accept H1]

Type I correct

α 1-β (power)


Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ =

SD =

Alpha (α) =

Beta (β) =

Power =

Sample Size ?


Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ = 5

SD = 15

Alpha (α) = 0.05 2-tailed

Beta (β) = 0.20

Power = 80

Sample Size 146 per group


n=292


FACTORS THAT INFLUENCE SAMPLE SIZE PLANNING AND STATISTICAL POWER


Sample Size Planningand Power

1. Error rates

Type I (α)

-smaller α requires larger sample sizes

-2-sided tests requires larger sample sizes

Type II (β) Statistical power:

-smaller β (more power) requires larger

sample sizes

[Use conventional levels & worry about the trade-

offs between effect size and sample size]


Sample Size Planningand Power

2. Effect Size ∆

“What is the minimally important effect based on clinical, biological or social implications of the findings?”


Sample Size Planningand Power

  • Effect size ∆

  • What do you know about the nature of the effect – its scale of measurement and its perceived importance to practice, policy, resource allocation (e.g., infant mortality; dollars; self-esteem)?

  • What do previous empirical studies tell you about achievable effects?


Sample Size Planningand Power

1.Effect Size ∆

  • Can you generate a consensus among your investigative team on a minimally important effect?

  • Is it reasonable to use conventional estimates of small, medium and large?

  • Are you limited by the dollar amount you can request?


Sample Size Planningand Power

  • The measurement scale of the dependent variable: discrete, ordinal, interval

    -interval level measurements require

    smaller samples

  • The variability of the dependent variable in the general population (SD, Variance)

    -lower variability requires smaller

    sample sizes


Sample Size Planningand Power

  • The statistical test

    -simple estimation; differences between groups; correlation and prediction. The test must be appropriate for the question and data. A key element in sample size planning

    5. Sample distribution, for example, exposed versus not exposed)

    -balanced is the most powerful


Sample Size Planningand Power

6. Attrition loss of subjects

-higher attrition leads to lower power

7. Measurement reliability

-complicated: if true variance is constant and error variance is reduced statistical power will increase


Sample Size Planningand Power

8.Study costs – what the market will bear

9. Analytical complexity – what to do when your models require much more information than you can get?


Adding Complexity

  • Multilevel Model

yij = β0j + β1z0j+(u0j + eij)

y

H0 The association between neighbourhood affluence measured on resident 4-16 year olds in 1983 and years of education assessed in 2001 will be = 0.00 standard units

x

H0∆ = β1z0j > 0.20

Neigh Affluence


Estimates

  • 2-level balanced data, nested model

  • Significance level = 0.025 (to get 0.05 2-tailed)

  • Number of simulations per setting = 100

  • Response variable = normal

  • Estimation method = IGLS

  • Fixed intercept = yes

  • Random intercept = yes

  • Number of explanatory variables = 1

  • Type of predictor = continuous


Estimates

  • Mean of the predictor = 0.0

  • Variance of the predictor at level 1 = 0.0

  • Variance of the predictor at level 2 = 1.0

  • Smallest/Largest # units at L1 (increment)

  • Smallest/Largest # units at L2 (increment)

  • Estimate β0 = 0

  • Estimateβ1= 0.15

  • Estimate L2 variance 0.05

  • Estimate L1 variance 0.95


Comments

  • Ask specific, quantifiable research questions

  • Consult with colleagues about clinical, biological and social importance of your outcomes

  • Move from simple to complex hypotheses. Complex models – SEM, Multilevel – can require you to provide an enormous number of parameters.

  • When estimating sample size requirements for complex models, you will inevitably use standardized variables


Comments

  • Estimating sample size requirements is part game, subject to practical constraints (limited resources and subjects) and convincing reviewers that you know what your doing

  • Take a ‘reasoned’ approach – most reviewers will have no clue what you are going on about

  • The hardest part of the process is acquiring the information you need.


  • Login