sample size considerations for answering quantitative research questions
Download
Skip this Video
Download Presentation
Sample Size Considerations for Answering Quantitative Research Questions

Loading in 2 Seconds...

play fullscreen
1 / 25

Sample Size Considerations for Answering Quantitative Research Questions - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

Sample Size Considerations for Answering Quantitative Research Questions. Lunch & Learn May 15, 2013 M Boyle. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21. Planning Costs 2000-2006: $54.7M Implementation Costs 2007-2011: $744.6. Sample Size Justification: ?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Sample Size Considerations for Answering Quantitative Research Questions' - daphne-burris


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sample size considerations for answering quantitative research questions

Sample Size Considerations for Answering Quantitative Research Questions

Lunch & Learn May 15, 2013

M Boyle

slide2

National Children’s Study in the US

Proposed Birth Cohort 100,000 to age 21

Planning Costs 2000-2006: $54.7M

Implementation Costs 2007-2011: $744.6

Sample Size Justification: ?

what is statistical power
What is Statistical Power?
  • The statistical power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a statistically significant effect when one exists
types of power analysis
Types of Power Analysis

A priori: Used to plan a study (Question: What sample size is needed to obtain a certain level of power)?

Post hoc: Used to evaluate a study faced with a constrained sample size (Question: Do you have a large enough sample to detect a meaningful effect)?

[Types of constraints: (1) a completed study; (2) a proposed study with limited number of eligible subjects; (3) a proposed study faced with limited resources]

elements of power calculations
Elements of Power Calculations
  • Effect size ∆
  • Measurement variability SD
  • Type I error Alpha (α)

typically specified at p=0.05, 2-tailed

  • Type II error Beta (β)

typically specified at p=0.20

  • Power = 1-β; typically 0.80
  • Sample Size
hypothesized distributions effect sizes and error rates
Hypothesized distributions, effect sizes and error rates

Effect Size ∆

Type I

Type II

Measurement Variability +/- 1 SD

decisions
Decisions

Disease Status

Present Absent

Medical Diagnosis

+ve

Test Result

-ve

correct false +ve

false -ve correct

Population Status

H0 false

H0 true[H1 true]

Hypothesis Testing

correct Type II

1-αβ

Accept H0

Decision

Reject H0

[Accept H1]

Type I correct

α 1-β (power)

example power calculation
Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ =

SD =

Alpha (α) =

Beta (β) =

Power =

Sample Size ?

example power calculation1
Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ = 5

SD = 15

Alpha (α) = 0.05 2-tailed

Beta (β) = 0.20

Power = 80

Sample Size 146 per group

sample size planning and power
Sample Size Planningand Power

1. Error rates

Type I (α)

-smaller α requires larger sample sizes

-2-sided tests requires larger sample sizes

Type II (β) Statistical power:

-smaller β (more power) requires larger

sample sizes

[Use conventional levels & worry about the trade-

offs between effect size and sample size]

sample size planning and power1
Sample Size Planningand Power

2. Effect Size ∆

“What is the minimally important effect based on clinical, biological or social implications of the findings?”

sample size planning and power2
Sample Size Planningand Power
  • Effect size ∆
  • What do you know about the nature of the effect – its scale of measurement and its perceived importance to practice, policy, resource allocation (e.g., infant mortality; dollars; self-esteem)?
  • What do previous empirical studies tell you about achievable effects?
sample size planning and power3
Sample Size Planningand Power

1.Effect Size ∆

  • Can you generate a consensus among your investigative team on a minimally important effect?
  • Is it reasonable to use conventional estimates of small, medium and large?
  • Are you limited by the dollar amount you can request?
sample size planning and power4
Sample Size Planningand Power
  • The measurement scale of the dependent variable: discrete, ordinal, interval

-interval level measurements require

smaller samples

  • The variability of the dependent variable in the general population (SD, Variance)

-lower variability requires smaller

sample sizes

sample size planning and power5
Sample Size Planningand Power
  • The statistical test

-simple estimation; differences between groups; correlation and prediction. The test must be appropriate for the question and data. A key element in sample size planning

5. Sample distribution, for example, exposed versus not exposed)

-balanced is the most powerful

sample size planning and power6
Sample Size Planningand Power

6. Attrition loss of subjects

-higher attrition leads to lower power

7. Measurement reliability

-complicated: if true variance is constant and error variance is reduced statistical power will increase

sample size planning and power7
Sample Size Planningand Power

8.Study costs – what the market will bear

9. Analytical complexity – what to do when your models require much more information than you can get?

adding complexity
Adding Complexity
  • Multilevel Model

yij = β0j + β1z0j+(u0j + eij)

y

H0 The association between neighbourhood affluence measured on resident 4-16 year olds in 1983 and years of education assessed in 2001 will be = 0.00 standard units

x

H0∆ = β1z0j > 0.20

Neigh Affluence

estimates
Estimates
  • 2-level balanced data, nested model
  • Significance level = 0.025 (to get 0.05 2-tailed)
  • Number of simulations per setting = 100
  • Response variable = normal
  • Estimation method = IGLS
  • Fixed intercept = yes
  • Random intercept = yes
  • Number of explanatory variables = 1
  • Type of predictor = continuous
estimates1
Estimates
  • Mean of the predictor = 0.0
  • Variance of the predictor at level 1 = 0.0
  • Variance of the predictor at level 2 = 1.0
  • Smallest/Largest # units at L1 (increment)
  • Smallest/Largest # units at L2 (increment)
  • Estimate β0 = 0
  • Estimateβ1= 0.15
  • Estimate L2 variance 0.05
  • Estimate L1 variance 0.95
comments
Comments
  • Ask specific, quantifiable research questions
  • Consult with colleagues about clinical, biological and social importance of your outcomes
  • Move from simple to complex hypotheses. Complex models – SEM, Multilevel – can require you to provide an enormous number of parameters.
  • When estimating sample size requirements for complex models, you will inevitably use standardized variables
comments1
Comments
  • Estimating sample size requirements is part game, subject to practical constraints (limited resources and subjects) and convincing reviewers that you know what your doing
  • Take a ‘reasoned’ approach – most reviewers will have no clue what you are going on about
  • The hardest part of the process is acquiring the information you need.
ad