Loading in 2 Seconds...
Loading in 2 Seconds...
Sample Size calculations for multilevel models (Part II). William Browne and Mousa Golalizadeh Department of Clinical Veterinary Sciences University of Bristol. Summary. Introduction to sample size calculations. Simulation-based approaches. The MLPOWSIM software. Two level normal examples.
William Browne and Mousa Golalizadeh
Department of Clinical Veterinary Sciences
University of Bristol
Here RHS is sum of cases H0 true and H0 false.
Here we have looked at two examples with effect sizes 5 and 1 respectively. Assume σ takes the value 5 and so let us suppose we take a sample of size 25 Welshmen.
Case 1: 5/(5/√25)=1.645+z1-β,z1-β=3.355
Case 2: 1/(5/ √25)=1.645+z1-β,z1-β=-0.645
So here a sample of 25 Welshmen from a population with mean 180cms would almost always result in rejecting H0,
but if the population mean is 176cms then only 26% of such samples would be rejected.
We can plot curves of how power increases with sample size as shown in the next slide.
Here we see the two power curves for the two scenarios:
be used in many situations and hypothesis tests.
Construct research question -> Form null hypothesis that we believe false -> Collect appropriate data -> Reject hypothesis therefore proving our research question.
Note simulation curve is a good approximation of the theoretical curve although there are some minor (Monte Carlo) errors even with 5000 simulations per sample size.
We will here mainly consider 2-level models and take as our application area education, so we have students nested within schools.
When deciding on a sampling scheme we have many choices:
Our decision will depend on which parameter we wish to estimate in the model.
The design effect formula:
Design effect = 1 + (n-1)ρ
suggests that if we are to sample a fixed (balanced) number of pupils n*N then our best power results when n is smallest i.e. sampling one pupil each from 100 schools is better than sampling 100 pupils from the same school.
The effect of sampling policy is most important in scenarios where ρ is large e.g. repeated measures designs.
The simulation procedure gives approximately the same power curve and so in this simple example we have an easy to use formula.
The reason in practice for sampling several pupils from each school is purely the additional cost incurred in visiting additional schools.
Here we see the results of the 0/1 approach in blue and the much smoother red line corresponds to the standard error approach for calculating power.
Here we see that the SE method (red) gives higher power than the 0/1 method (blue) but this is due to the bias in the estimates and hence the SEs. Here the 0/1 method is preferable.
Here we see less difference between the SE method (red) and the 0/1 method (blue) as PQL is far less biased than MQL.
We will look at perhaps the simplest model that accounts for the structure. Here our response is a mark out of 10 from which we subtract 5 which we treat as a pass mark. This response is then assumed to be affected by both the primary and secondary school attended
Approximate estimates from the actual data that we will use for our power calculations are β0 = 0.5, variances for primary, secondary and residual are 1.2, 0.4 and 5 respectively. We are interested in sample sizes to detect an average greater than 5.