Sample Size Considerations for Answering Quantitative Research Questions

1 / 25

# Sample Size Considerations for Answering Quantitative Research Questions - PowerPoint PPT Presentation

Sample Size Considerations for Answering Quantitative Research Questions. Lunch &amp; Learn May 15, 2013 M Boyle. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21. Planning Costs 2000-2006: \$54.7M Implementation Costs 2007-2011: \$744.6. Sample Size Justification: ?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Sample Size Considerations for Answering Quantitative Research Questions' - daphne-burris

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Sample Size Considerations for Answering Quantitative Research Questions

Lunch & Learn May 15, 2013

M Boyle

National Children’s Study in the US

Proposed Birth Cohort 100,000 to age 21

Planning Costs 2000-2006: \$54.7M

Implementation Costs 2007-2011: \$744.6

Sample Size Justification: ?

What is Statistical Power?
• The statistical power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a statistically significant effect when one exists
Types of Power Analysis

A priori: Used to plan a study (Question: What sample size is needed to obtain a certain level of power)?

Post hoc: Used to evaluate a study faced with a constrained sample size (Question: Do you have a large enough sample to detect a meaningful effect)?

[Types of constraints: (1) a completed study; (2) a proposed study with limited number of eligible subjects; (3) a proposed study faced with limited resources]

Elements of Power Calculations
• Effect size ∆
• Measurement variability SD
• Type I error Alpha (α)

typically specified at p=0.05, 2-tailed

• Type II error Beta (β)

typically specified at p=0.20

• Power = 1-β; typically 0.80
• Sample Size
Hypothesized distributions, effect sizes and error rates

Effect Size ∆

Type I

Type II

Measurement Variability +/- 1 SD

Decisions

Disease Status

Present Absent

Medical Diagnosis

+ve

Test Result

-ve

correct false +ve

false -ve correct

Population Status

H0 false

H0 true[H1 true]

Hypothesis Testing

correct Type II

1-αβ

Accept H0

Decision

Reject H0

[Accept H1]

Type I correct

α 1-β (power)

Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ =

SD =

Alpha (α) =

Beta (β) =

Power =

Sample Size ?

Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ = 5

SD = 15

Alpha (α) = 0.05 2-tailed

Beta (β) = 0.20

Power = 80

Sample Size 146 per group

Sample Size Planningand Power

1. Error rates

Type I (α)

-smaller α requires larger sample sizes

-2-sided tests requires larger sample sizes

Type II (β) Statistical power:

-smaller β (more power) requires larger

sample sizes

offs between effect size and sample size]

Sample Size Planningand Power

2. Effect Size ∆

“What is the minimally important effect based on clinical, biological or social implications of the findings?”

Sample Size Planningand Power
• Effect size ∆
• What do you know about the nature of the effect – its scale of measurement and its perceived importance to practice, policy, resource allocation (e.g., infant mortality; dollars; self-esteem)?
• What do previous empirical studies tell you about achievable effects?
Sample Size Planningand Power

1.Effect Size ∆

• Can you generate a consensus among your investigative team on a minimally important effect?
• Is it reasonable to use conventional estimates of small, medium and large?
• Are you limited by the dollar amount you can request?
Sample Size Planningand Power
• The measurement scale of the dependent variable: discrete, ordinal, interval

-interval level measurements require

smaller samples

• The variability of the dependent variable in the general population (SD, Variance)

-lower variability requires smaller

sample sizes

Sample Size Planningand Power
• The statistical test

-simple estimation; differences between groups; correlation and prediction. The test must be appropriate for the question and data. A key element in sample size planning

5. Sample distribution, for example, exposed versus not exposed)

-balanced is the most powerful

Sample Size Planningand Power

6. Attrition loss of subjects

-higher attrition leads to lower power

7. Measurement reliability

-complicated: if true variance is constant and error variance is reduced statistical power will increase

Sample Size Planningand Power

8.Study costs – what the market will bear

9. Analytical complexity – what to do when your models require much more information than you can get?

• Multilevel Model

yij = β0j + β1z0j+(u0j + eij)

y

H0 The association between neighbourhood affluence measured on resident 4-16 year olds in 1983 and years of education assessed in 2001 will be = 0.00 standard units

x

H0∆ = β1z0j > 0.20

Neigh Affluence

Estimates
• 2-level balanced data, nested model
• Significance level = 0.025 (to get 0.05 2-tailed)
• Number of simulations per setting = 100
• Response variable = normal
• Estimation method = IGLS
• Fixed intercept = yes
• Random intercept = yes
• Number of explanatory variables = 1
• Type of predictor = continuous
Estimates
• Mean of the predictor = 0.0
• Variance of the predictor at level 1 = 0.0
• Variance of the predictor at level 2 = 1.0
• Smallest/Largest # units at L1 (increment)
• Smallest/Largest # units at L2 (increment)
• Estimate β0 = 0
• Estimateβ1= 0.15
• Estimate L2 variance 0.05
• Estimate L1 variance 0.95