Jeff Gardner Principal Consultant

1. Mitigating Risk of Out-of-Specification Results During Stability Testing of Biopharmaceutical Products 36th Annual Midwest Biopharmaceutical Statistics Workshop May 21, 2013 Jeff Gardner Principal Consultant

2. Outline What Is an OOS Result? Business Consequences of OOS Results - Regulatory requirements - Marketing/business impact Risks of OOS Associated with Statistical Approaches to Estimating Shelf Life - ICH Q1E Guidance - Mixed model (random slopes) estimation Recommendations for Managing OOS Risk

3. What Is an OOS Result? From 21 CFR 314.81(b)(1)(ii): “Any failure of a distributed batch to meet [i.e. conform to] any of the specifications established in an application.” From ICH Q6A/Q6B: “’Conformance to specifications’ means that the drug substance and/or drug product, when tested according to the listed analytical procedures, will meet the listed acceptance criteria.” Expectation is that any test result, not just the average of results or majority of results, meets the specified acceptance criterion.

4. Business Impact of OOS Results Regulatory Requirements Field alert required within 72 hours of identification Commercial Impact Potential product recall Possible reduction of shelf-life

5. Illustration As seen here, a portion of the population of results extends into the OOS region, meaning that some risk exists of seeing OOS results.

6. ICH Q1E Guidance “An appropriate approach to… shelf life estimation is to analyze a quantitative attribute… by determining the earliest time at which the 95 percent confidence limit for the mean intersects the proposed acceptance criterion.” The above applies in either of two cases (contingent on assumption of common slope using “poolability” test): Linear regression analysis is performed on data from a single batch In this case, mean = batch mean Linear regression analysis is performed on data from multiple batches In this case, mean = mean of all batches in analysis This mean is then taken to be an estimate of the product mean Numerous critiques exist in literature which address “flaws” or limitations of the statistical approach outlined in ICH Q1E.

7. ICH Q1E Limitations Methodology treats batch as a fixed effect, thereby limiting inference to only the sample of batches being evaluated. Calculation of a (1- α)% prediction interval about the mean is only useful in defining the region of likely results from the batch(es) being evaluated. This is despite the stated aim of the guidance which is to provide a “high degree of confidence” that all future batches will remain within acceptance criteria. A statistical methodology is needed that more closely aligns with stated objectives of the ICH guidance documents.

8. Mixed Effects Model Consider the model A = Population intercept (mfg. process mean) B = Mean product degradation rate ai= Initial value for batch i bi = Degradation rate for batch i xj= Stability test interval j eij= Random error when measuring batch iat test interval j , , Inclusion of random effects aiand bi in the model allows for calculation of prediction limits which more closely align with stated objective of ICH guidance.

9. Prediction Intervals Using Mixed Model Reflects test result (i.e. analytical method) variability When the model parameters are known, (1-α)% prediction intervals can be constructed at each xj such that and Constructing the intervals in this way assumes that , and are all independent. The first equation above is “philosophically” closest to ICH Q1E in that it governs the distribution of individual batch means at each stability testing interval.

10. Prediction Intervals Using Mixed Model When the model parameters are unknown (which is pretty much always the case), and For sake of discussion, we will use the first equation as the basis for determining shelf life since it aligns closest with ICH Q1E.

11. Determination of Shelf Life Using α=0.05, shelf life = maximum xjsuch that Upper Acceptance Limit (UAL) or Lower Acceptance Limit (LAL) Once shelf life has been determined, calculate Pj(OOS) = P(LAL | xj = Shelf Life) + P(OOS) = P(UAL | xj = Shelf Life) Shelf life and Pj(OOS) were determined as described abovefor n = 25 = 32 experimental settings in order to examine the relationship between the two.

12. Experimental Settings 𝜑batch and 𝜑indiv were calculated for each combination of settings for A, B, , , and as specified below. All numbers are expressed as multiples of unit scale where 1 = (two-sided) or 1 = UAL (one-sided) A:0, -0.4 B:-0.015, -0.04 0.15, 0.33 0.008, 0.024 0.05, 0.15

13. Example 1: P(OOS) at 24-Month Expiry Since parameters are pre-defined (known), z statistics were used in lieu of t for these calculations. batch and indiv were calculated in this manner at 0 through 24 months for each of the 32 experimental settings.

14. Probability of OOS at Expiry

15. Determining Cumulative P(OOS) Note: this probability significantly increases if testing is performed more frequently later in shelf life Therefore the cumulative probability is most useful when calculated only for those intervals that are specified when establishing the stability testing protocol. The cumulative P(OOS) at xj is given by Determining cumulative probability of OOS provides a direct measure of business risk of field alert/product recall.

16. Example 2: Cumulative P(OOS) Note: same model parameters used here as in Example 1.

17. Cumulative Probability of OOS This chart illustrates that determining shelf life based on mean alone (x-axis) does not address the risk of OOS results.

18. Application

19. Example 3: 12 Month Assay Data Consider the following:

20. Example 3 cont’d: Q1E Approach Per ICH Q1E, p-value for the month*lot effect (<0.25) requires determination of individual confidence limit for each lot. Shelf life is based on the earliest time at which any lot’s confidence limit intersects the lower acceptance limit (shown in previous graph). In this case, the data supports a maximum shelf life of 15 months which per ICH Q1E would have to be verified with real-time data.

21. Example 3 cont’d: Mixed Model Approach The point estimates for A, B, , , and can be used to determine Pj(OOS). (Note: Output is from SAS® PROC MIXED) A B

22. Example 3 cont’d Based on a nominal α=0.05, the cumulative P(OOS) suggests a shelf life of 12 months. Using = ,

23. Summary

24. Summary of ICH Guidance Estimation of shelf-life is based on the earliest point at which a batch’s true mean is likely to intersect a specification limit. Substantial risk exists for observing individual OOS analytical results during the expiry period OOS risk is greatest toward the end of product shelf-life

25. Estimating Expiry Using Mixed Model A prediction interval based on a mixed effects model can be constructed with either of two aims: philosophical alignment with ICH guidance (i.e. including only variance components for batch intercept & slope variability) minimizing risk of OOS results (by including residual variance component as well as batch variance components) Either of the above methods are an improvement upon the statistical approaches outlined in ICH guidance.

26. Impact of Analytical Method Variability It is recommended that both prediction interval calculations are employed for preliminary shelf life estimation. Difference in shelf life estimates = impact of analytical method variability on shelf-life Additional analysis may be considered for evaluating how this impact can be lessened via increased replication Such understanding may provide guidance toward optimal method validation

27. Formally Assessing Risk of OOS It’s recommended that a cumulative probability of OOS occurrence accompanies each preliminary shelf life estimate calculate P(OOS) at each scheduled stability testing interval calculate cumulative P(OOS) across all stability testing intervals It may be worth considering whether the final proposed shelf life should be based on first determining a maximum cumulative P(OOS) rather than by focusing on lower bound on the population of batch slopes.

28. Future Research Using experimental settings defined earlier, examine the sampling distributions of model parameter estimates via Monte Carlo simulations: How does sampling error contribute to over-estimation or under-estimation of P(OOS)? What impact does analytical method variability have on shelf life estimates if P(OOS) is used instead of the distribution of batch means at a given testing interval?

29. End of Presentation