Sample size considerations for answering quantitative research questions
This presentation is the property of its rightful owner.
Sponsored Links
1 / 25

Sample Size Considerations for Answering Quantitative Research Questions PowerPoint PPT Presentation


  • 56 Views
  • Uploaded on
  • Presentation posted in: General

Sample Size Considerations for Answering Quantitative Research Questions. Lunch & Learn May 15, 2013 M Boyle. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21. Planning Costs 2000-2006: $54.7M Implementation Costs 2007-2011: $744.6. Sample Size Justification: ?.

Download Presentation

Sample Size Considerations for Answering Quantitative Research Questions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Sample size considerations for answering quantitative research questions

Sample Size Considerations for Answering Quantitative Research Questions

Lunch & Learn May 15, 2013

M Boyle


Sample size considerations for answering quantitative research questions

National Children’s Study in the US

Proposed Birth Cohort 100,000 to age 21

Planning Costs 2000-2006: $54.7M

Implementation Costs 2007-2011: $744.6

Sample Size Justification: ?


What is statistical power

What is Statistical Power?

  • The statistical power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a statistically significant effect when one exists


Types of power analysis

Types of Power Analysis

A priori: Used to plan a study (Question: What sample size is needed to obtain a certain level of power)?

Post hoc: Used to evaluate a study faced with a constrained sample size (Question: Do you have a large enough sample to detect a meaningful effect)?

[Types of constraints: (1) a completed study; (2) a proposed study with limited number of eligible subjects; (3) a proposed study faced with limited resources]


Elements of power calculations

Elements of Power Calculations

  • Effect size ∆

  • Measurement variability SD

  • Type I error Alpha (α)

    typically specified at p=0.05, 2-tailed

  • Type II error Beta (β)

    typically specified at p=0.20

  • Power = 1-β; typically 0.80

  • Sample Size


Hypothesized distributions effect sizes and error rates

Hypothesized distributions, effect sizes and error rates

Effect Size ∆

Type I

Type II

Measurement Variability +/- 1 SD


Decisions

Decisions

Disease Status

Present Absent

Medical Diagnosis

+ve

Test Result

-ve

correct false +ve

false -ve correct

Population Status

H0 false

H0 true[H1 true]

Hypothesis Testing

correct Type II

1-αβ

Accept H0

Decision

Reject H0

[Accept H1]

Type I correct

α 1-β (power)


Example power calculation

Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ =

SD =

Alpha (α) =

Beta (β) =

Power =

Sample Size ?


Example power calculation1

Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ = 5

SD = 15

Alpha (α) = 0.05 2-tailed

Beta (β) = 0.20

Power = 80

Sample Size 146 per group


Sample size considerations for answering quantitative research questions

n=292


Sample size considerations for answering quantitative research questions

FACTORS THAT INFLUENCE SAMPLE SIZE PLANNING AND STATISTICAL POWER


Sample size planning and power

Sample Size Planningand Power

1. Error rates

Type I (α)

-smaller α requires larger sample sizes

-2-sided tests requires larger sample sizes

Type II (β) Statistical power:

-smaller β (more power) requires larger

sample sizes

[Use conventional levels & worry about the trade-

offs between effect size and sample size]


Sample size planning and power1

Sample Size Planningand Power

2. Effect Size ∆

“What is the minimally important effect based on clinical, biological or social implications of the findings?”


Sample size planning and power2

Sample Size Planningand Power

  • Effect size ∆

  • What do you know about the nature of the effect – its scale of measurement and its perceived importance to practice, policy, resource allocation (e.g., infant mortality; dollars; self-esteem)?

  • What do previous empirical studies tell you about achievable effects?


Sample size planning and power3

Sample Size Planningand Power

1.Effect Size ∆

  • Can you generate a consensus among your investigative team on a minimally important effect?

  • Is it reasonable to use conventional estimates of small, medium and large?

  • Are you limited by the dollar amount you can request?


Sample size planning and power4

Sample Size Planningand Power

  • The measurement scale of the dependent variable: discrete, ordinal, interval

    -interval level measurements require

    smaller samples

  • The variability of the dependent variable in the general population (SD, Variance)

    -lower variability requires smaller

    sample sizes


Sample size planning and power5

Sample Size Planningand Power

  • The statistical test

    -simple estimation; differences between groups; correlation and prediction. The test must be appropriate for the question and data. A key element in sample size planning

    5. Sample distribution, for example, exposed versus not exposed)

    -balanced is the most powerful


Sample size planning and power6

Sample Size Planningand Power

6. Attrition loss of subjects

-higher attrition leads to lower power

7. Measurement reliability

-complicated: if true variance is constant and error variance is reduced statistical power will increase


Sample size planning and power7

Sample Size Planningand Power

8.Study costs – what the market will bear

9. Analytical complexity – what to do when your models require much more information than you can get?


Adding complexity

Adding Complexity

  • Multilevel Model

yij = β0j + β1z0j+(u0j + eij)

y

H0 The association between neighbourhood affluence measured on resident 4-16 year olds in 1983 and years of education assessed in 2001 will be = 0.00 standard units

x

H0∆ = β1z0j > 0.20

Neigh Affluence


Estimates

Estimates

  • 2-level balanced data, nested model

  • Significance level = 0.025 (to get 0.05 2-tailed)

  • Number of simulations per setting = 100

  • Response variable = normal

  • Estimation method = IGLS

  • Fixed intercept = yes

  • Random intercept = yes

  • Number of explanatory variables = 1

  • Type of predictor = continuous


Estimates1

Estimates

  • Mean of the predictor = 0.0

  • Variance of the predictor at level 1 = 0.0

  • Variance of the predictor at level 2 = 1.0

  • Smallest/Largest # units at L1 (increment)

  • Smallest/Largest # units at L2 (increment)

  • Estimate β0 = 0

  • Estimateβ1= 0.15

  • Estimate L2 variance 0.05

  • Estimate L1 variance 0.95


Comments

Comments

  • Ask specific, quantifiable research questions

  • Consult with colleagues about clinical, biological and social importance of your outcomes

  • Move from simple to complex hypotheses. Complex models – SEM, Multilevel – can require you to provide an enormous number of parameters.

  • When estimating sample size requirements for complex models, you will inevitably use standardized variables


Comments1

Comments

  • Estimating sample size requirements is part game, subject to practical constraints (limited resources and subjects) and convincing reviewers that you know what your doing

  • Take a ‘reasoned’ approach – most reviewers will have no clue what you are going on about

  • The hardest part of the process is acquiring the information you need.


  • Login