Richard K Burdick Elion Labs MBSW Meetings May 2016

1. Using the Confidence Interval on Effect Size to Demonstrate Analytical Similarity BetweenReference and Biosimilar Products Richard K Burdick Elion Labs MBSW Meetings May 2016

2. Collaborators • Harry Yang-MedImmune, LLC • Steven Novick-MedImmune, LLC • Neal Thomas, Pfizer, Statistical Research and Consulting Center • Aili Cheng, Pfizer, Pharmaceutical Sciences and Manufacturing Statistics

3. Introduction • The FDA Briefing Document for the Oncologic Drugs Advisory Committee (ODAC) Meeting held on January 7, 2015, as well as recent presentations and information provided by Chow (2014) describes the statistical procedure recommended for demonstration of analytical similarity for biosimilar applications. • This approach uses a three-tier system based on attribute criticality with a different statistical procedure applied for each tier.

4. Tier 1 Testing • Tier 1 testing is based on a statistical test of equivalence. • A statistical test of equivalence must “prove” the difference in averages between the reference product (RP) and test product (TP) is less than a pre-defined equivalence margin. (Burden of proof is on sponsor of TP) • The equivalence margin is 1.5sRP where sRP is the true standard deviation of the RP.

5. Variation Inherent in Tier 1 Testing • Tier 1 testing occurs under two possible conditions that impact test size: • Condition A: The value of sRP is unknown and must be estimated with sampled RP lots. • Condition B: RP lots may be correlated if sourced from the same drug substance or formulated bulk (an unknown condition for TP sponsors) • A test size exceeding the desired level, typically 5%, is problematic because it shifts a greater than desired risk to patient. • That is, the risk of passing a TP product that is not analytically similar to the RP is greater than the desired level of 5%.

6. Estimation of Effect Size • One approach to mitigate these conditions is to construct a confidence interval on the effect size directly. Effect Size

7. Estimation of Effect Size • By changing hypotheses in this manner, sRP is now contained in the parameter of interest, and the uncertainty in its true value is properly accounted for in the confidence interval. • The estimated parameter is now compared to a constant rather than another estimate. • This format also allows appropriate adjustment for correlation when the correlation structure can be identified.

8. Performing Calculations • Equal variances • Exact closed-form solution is based on the inversion confidence interval principle and the non-central t-distribution (see, e.g., Hedges (1981)). • The general approach for constructing a confidence interval on the effect size is to first form a confidence interval on the non-centrality parameter of the t-distribution associated with the test statistic • This non-centrality parameter is

9. Performing Calculations • To determine a two-sided 90% confidence interval on l, determine the lower bound lL such that • and the upper bound lU by such that • Here is the SAS code using the function tnonct to compute the confidence interval: Converts from l to effect size

10. Performing Calculations • Unequal variances • An “exact” procedure can be performed using a generalized confidence interval (GCI). • The solution is simulation based, requiring simulation of independent normal and chi-squared random variables. • SAS code is available from the author, but can be written in any other software package including Excel. • Power is impacted by relative sizes of sRP and sTP, but test size is “exact”.

11. Unequal Variances • Using a recipe provided in Appendix B.2 of Burdick, Borror, and Montgomery (2005) perform the following steps:

12. Unequal Variances

13. Example

14. 90% CI

15. Unequal Variances With Correlation • Reference drug product lots may be correlated if sourced from the same drug substance or formulated bulk. • As noted by Ramirez and Burdick (2015), correlation among RP lots can lead to an inflated Type I error rate for the TOST. • Since a sponsor has information concerning TP lots, sourcing of these lots will be known so that independently sourced lots can be selected in the analysis. • In contrast, the lack of sponsor knowledge concerning sourcing of the RP lots makes this risk problematic.

16. Unequal Variances With Correlation • If the correlation structure can be identified, then a GCI can be constructed. • For example, assume the correlation structure generated by a one-factor random model where there are RP lots created by sourcing r DP lots each from n DS lots.

17. One-Factor Random Model • is a chi-squared random variable with n-1 df • is a chi-squared random variable with n(r-1) df • Here, two DP lots sourced from the same DS lot have a correlation of

18. Unequal Variances With Correlation • Replace ESsim in previous recipe with the quantity

19. Simulation Study • TOST1=Unequal variance CI on difference • TOST2=Pooled variance CI on difference • GCI1=Unequal variance CI on effect size (ESsim) • GCI2=Pooled variance CI on effect size • GCI3=Unequal variance CI on effect size with correlation adjustment (ES2sim) • Number of DP lots per DS lot (r=1,2,5) • NRP=10,20 (NTP=10) • 10,000 simulations with GCI based on 2,000 iterations. • Correlation is 0.8

24. Identification of Correlated Lots Repeated measures made on RP lots Fisher’s LSD Test

25. Recommendations to Sponsors • Purchase RP lots over an extended time frame to decrease the likelihood of obtaining DP lots sourced with same DS. This strategy will also provide an opportunity for the lot-to-lot variation to fully manifest and provide a better estimate of the RP standard deviation. • It might be possible for the sponsor to identify the source material of RP lots using a stable isotope profile. (e.g. Apostol et al. (2001)). • An empirical examination of repeated measurements of RP DP lots can be useful in identifying lots sourced with the same DS. • If identifiable, do not use DP lots sourced from the same DS. Although the data can be modeled using the GCI approach, it will decrease power.

26. References • Apostol, I., Brooks, P. D., and A. J. Mathews. (2001). “Application of high-precision isotope ratio monitoring mass spectrometry to identify the biosynthetic origins of proteins”, Protein Science , July 10(7), pages 1466-1469. • Burdick, R. K., Borror, C.M., and Montgomery, D. C. (2005). Design and analysis of gauge R&R studies: Making decisions with confidence intervals in random and mixed ANOVA models, SIAM, PA. • Burdick, R. K. and Ramírez, J. G. (2015) “Statistical issues in biosimilar analytical assessment: Perspectives on FDA ODAC analysis, DIA Conference, Washington, D. C., April. • Chow, S.-C. (2014). “On assessment of analytical similarity in biosimilar studies”, Drug Designing 3: 119. doi:10.4172/2169-0138.1000e124 (accessed March 2, 2015). • Hedges, L. V. (1981). “Distribution theory for Glass’s estimator of effect size and related estimators”, Journal of Educational Statistics, Vol. 6 (2), pages 107-128. • FDA Briefing Document for the Oncologic Drugs Advisory Committee (ODAC) Meeting held on January 7, 2015 (http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/OncologicDrugsAdvisoryCommittee/UCM428781.pdf)