Rerandomization in Randomized Experiments

1. Rerandomization in Randomized Experiments Kari Lock and Don Rubin Harvard University JSM 2010

2. The “Gold Standard” Why are randomized experiments so good? • They yield unbiased estimates of the treatment effect • They eliminate (?) confounding factors… • … ON AVERAGE. For any particular experiment, covariate imbalance is possible (and likely)

3. Rerandomization • Suppose you are doing a randomized experiment and have covariate information available before conducting the experiment • You randomize to treatment and control, but get a “bad” randomization • Can you rerandomize? • Yes, but you first need to specify a concrete definition of “bad”

4. Collect covariate data Specify a criteria determining when a randomization is unacceptable; based on covariate balance 1) Randomize subjects to treated and control (Re)randomize subjects to treated and control 2) Check covariate balance acceptable unacceptable Conduct experiment 3) Analyze results with a Fisher randomization test 4)

5. Unbiased • To maintain an unbiased estimate of the treatment effect, the decision to rerandomize or not must be • automatic and specified in advance • blind to which group is treated • Theorem: If the treated and control groups are the same size, and if for every unacceptable randomization the exact opposite randomization is also unacceptable, then rerandomization yields an unbiased estimate of the treatment effect.

6. Mahalanobis Distance Dj: Standardized difference between treated and control covariate means for covariate j k = number of covariates D = (D1, …, Dk) r = covariate correlation matrix = cov(D) Define overall covariate distance by M = D’r-1D Choose a and rerandomize when M > a

7. Rerandomization Based on M • Since M follows a known distribution, easy to specify the proportion of rejected randomizations • M is affinely invariant • Correlations between covariates are maintained • The variance reduction on each covariateis the same (and known) • The variance reduction for any linear combination of the covariates is known

8. Rerandomization Theorem: If nT = nC and rerandomization occurs when M > a, then and

9. Difference in Outcome Means Difference in Covariate Means

10. (theoretical va = .16)

11. (theory = .58) Equivalent to increasing the sample size by a factor of 1.7 Difference in Outcome Means Under Null

12. Conclusion • Rerandomization improves covariate balance between the treated and control means, and increases precision in estimating the treatment effect if the covariates are correlated with the response • Rerandomization gives the researcher more power to detect a significant result, and more faith that an observed effect is really due to the treatment lock@stat.harvard.edu