# Sample Size Considerations for Answering Quantitative Research Questions - PowerPoint PPT Presentation

1 / 25

Sample Size Considerations for Answering Quantitative Research Questions. Lunch & Learn May 15, 2013 M Boyle. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21. Planning Costs 2000-2006: \$54.7M Implementation Costs 2007-2011: \$744.6. Sample Size Justification: ?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Sample Size Considerations for Answering Quantitative Research Questions

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Sample Size Considerations for Answering Quantitative Research Questions

Lunch & Learn May 15, 2013

M Boyle

National Children’s Study in the US

Proposed Birth Cohort 100,000 to age 21

Planning Costs 2000-2006: \$54.7M

Implementation Costs 2007-2011: \$744.6

Sample Size Justification: ?

### What is Statistical Power?

• The statistical power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a statistically significant effect when one exists

### Types of Power Analysis

A priori: Used to plan a study (Question: What sample size is needed to obtain a certain level of power)?

Post hoc: Used to evaluate a study faced with a constrained sample size (Question: Do you have a large enough sample to detect a meaningful effect)?

[Types of constraints: (1) a completed study; (2) a proposed study with limited number of eligible subjects; (3) a proposed study faced with limited resources]

### Elements of Power Calculations

• Effect size ∆

• Measurement variability SD

• Type I error Alpha (α)

typically specified at p=0.05, 2-tailed

• Type II error Beta (β)

typically specified at p=0.20

• Power = 1-β; typically 0.80

• Sample Size

### Hypothesized distributions, effect sizes and error rates

Effect Size ∆

Type I

Type II

Measurement Variability +/- 1 SD

### Decisions

Disease Status

Present Absent

Medical Diagnosis

+ve

Test Result

-ve

correct false +ve

false -ve correct

Population Status

H0 false

H0 true[H1 true]

Hypothesis Testing

correct Type II

1-αβ

Accept H0

Decision

Reject H0

[Accept H1]

Type I correct

α 1-β (power)

### Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ =

SD =

Alpha (α) =

Beta (β) =

Power =

Sample Size ?

### Example Power Calculation

H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care.

H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher.

Effect size ∆ = 5

SD = 15

Alpha (α) = 0.05 2-tailed

Beta (β) = 0.20

Power = 80

Sample Size 146 per group

n=292

FACTORS THAT INFLUENCE SAMPLE SIZE PLANNING AND STATISTICAL POWER

### Sample Size Planningand Power

1. Error rates

Type I (α)

-smaller α requires larger sample sizes

-2-sided tests requires larger sample sizes

Type II (β) Statistical power:

-smaller β (more power) requires larger

sample sizes

offs between effect size and sample size]

### Sample Size Planningand Power

2. Effect Size ∆

“What is the minimally important effect based on clinical, biological or social implications of the findings?”

### Sample Size Planningand Power

• Effect size ∆

• What do you know about the nature of the effect – its scale of measurement and its perceived importance to practice, policy, resource allocation (e.g., infant mortality; dollars; self-esteem)?

• What do previous empirical studies tell you about achievable effects?

### Sample Size Planningand Power

1.Effect Size ∆

• Can you generate a consensus among your investigative team on a minimally important effect?

• Is it reasonable to use conventional estimates of small, medium and large?

• Are you limited by the dollar amount you can request?

### Sample Size Planningand Power

• The measurement scale of the dependent variable: discrete, ordinal, interval

-interval level measurements require

smaller samples

• The variability of the dependent variable in the general population (SD, Variance)

-lower variability requires smaller

sample sizes

### Sample Size Planningand Power

• The statistical test

-simple estimation; differences between groups; correlation and prediction. The test must be appropriate for the question and data. A key element in sample size planning

5. Sample distribution, for example, exposed versus not exposed)

-balanced is the most powerful

### Sample Size Planningand Power

6. Attrition loss of subjects

-higher attrition leads to lower power

7. Measurement reliability

-complicated: if true variance is constant and error variance is reduced statistical power will increase

### Sample Size Planningand Power

8.Study costs – what the market will bear

9. Analytical complexity – what to do when your models require much more information than you can get?

• Multilevel Model

yij = β0j + β1z0j+(u0j + eij)

y

H0 The association between neighbourhood affluence measured on resident 4-16 year olds in 1983 and years of education assessed in 2001 will be = 0.00 standard units

x

H0∆ = β1z0j > 0.20

Neigh Affluence

### Estimates

• 2-level balanced data, nested model

• Significance level = 0.025 (to get 0.05 2-tailed)

• Number of simulations per setting = 100

• Response variable = normal

• Estimation method = IGLS

• Fixed intercept = yes

• Random intercept = yes

• Number of explanatory variables = 1

• Type of predictor = continuous

### Estimates

• Mean of the predictor = 0.0

• Variance of the predictor at level 1 = 0.0

• Variance of the predictor at level 2 = 1.0

• Smallest/Largest # units at L1 (increment)

• Smallest/Largest # units at L2 (increment)

• Estimate β0 = 0

• Estimateβ1= 0.15

• Estimate L2 variance 0.05

• Estimate L1 variance 0.95

• Ask specific, quantifiable research questions

• Consult with colleagues about clinical, biological and social importance of your outcomes

• Move from simple to complex hypotheses. Complex models – SEM, Multilevel – can require you to provide an enormous number of parameters.

• When estimating sample size requirements for complex models, you will inevitably use standardized variables

• Estimating sample size requirements is part game, subject to practical constraints (limited resources and subjects) and convincing reviewers that you know what your doing

• Take a ‘reasoned’ approach – most reviewers will have no clue what you are going on about

• The hardest part of the process is acquiring the information you need.