The equivalence trial

1. NATIONAL VETERINARY S C H O O L T O U L O U S E The equivalence trial Didier Concordet d.concordet@envt.fr

2. Comparison of two treatments Aim of all trials : to compare treatments on the population of individuals Population of animals Treatment effect R = 17.8 T =16.8 Impossible in practice

3. A two-steps method Sampling Sample of animals Population of animals

4. Effect of sampling Sample of animals Treatment effect XR = 16.2 XT =17.8

5. A two-steps method Inference Population of animals Sample of animals

6. Effect of inference Truth in the population Observed on the samples R = 17.8 XR = 16.2 T =16.8 XT =17.8 Lead to a wrong conclusion Ref > New Treatment T New Treatment T > Ref

7. A good trial Minimize the risk of bias in sampling Minimize the risk of a wrong conclusion in inference - All Randomised Study Animals - Per Protocol Set of Study Animals - Response Variable - Experimental (study) design - Consumer Risk - Producer Risk - Relevant difference

8. Tree kinds of study R - D1 T Non inferiority R + D1 T Superiority D1T- R D2 Equivalence study

9. Non inferiority R - D1 Values of T R R - D1 T Unacceptable for primary efficacy variable in clinical trial Does not prove that the treatment T works

10. Superiority R + D1 Values of T R R + D1 T Primary efficacy variable in clinical trials

11. Equivalence Values of T R + D2 R - D1 R Equivalence range Does not prove that the treatment T works For secondary efficacy variables in clinical trials

12. Equivalence Equivalence range Clinical effect Values of T R R - D1 R + D2

13. Equivalence Clinical effect Values of T R R - D1 R + D2

14. Equivalence Clinical effect Values of T R R - D1 R + D2

15. Even with a good question, a poor design leads to poor conclusions Superiority clinical trials REFERENCE New TRT Cure rate = 79% N = 2100 Cure rate = 83% N = 2400 Reference < New TRT (P<0.001)

16. Even with a good question, a poor design leads to poor conclusions Superiority trials Clinical trial 1 Clinical trial 2 REFERENCE New TRT REFERENCE New TRT Cure rate = 90% N = 2000 Cure rate = 96% N = 1000 Cure rate = 50% N = 400 Cure rate = 63% N = 1100 New TRT < Ref P<0.001 New TRT< Ref P<0.001 Conclusion : Reference > New TRT

17. Even with a good question, a poor design leads to poor conclusions Superiority clinical trials REFERENCE New TRT X = 39 N = 100 SD = 1 X = 37 N = 100 SD = 1 Reference < New TRT (P<0.001)

18. Even with a good question, a poor design leads to poor conclusions Superiority trials Clinical trial 1 Clinical trial 2 REFERENCE New TRT REFERENCE New TRT X = 30 N = 10 SD = 1 X = 32 N = 50 SD = 1 X = 40 N = 90 SD = 1 X = 42 N = 50 SD = 1 New TRT < Ref P<0.001 New TRT< Ref P<0.001 Conclusion : Reference > New TRT

19. N 3 3 Usual statistical tests are not intended to answer to useful questions Efficacy variable on two groups of dogs Student t-test Ref Test P = 0.23 20.0 Mean 15.4 2.6 2.4 SD Conclusion : “EQUIVALENCE” In the population R = 14.5 ; T =19.7 this difference is clinically important

20. N 15 15 Comparison of two treatments Efficacy variable on two groups of dogs Student t-test Ref Test P = 0.03 Study 1 18.1 Mean 16.0 2.6 2.4 SD Conclusion : NO EQUIVALENCE In the population R = 16.8 ; T =17.8 This difference is not clinically important

21. N 15 15 Comparison of two formulations Efficacy variable on two groups of dogs Ref Test Student t-test P = 0.26 Study 2 18.1 Mean 16.0 5.1 4.9 SD Conclusion : EQUIVALENCE In the population R = 16.8 ; T =17.8 This difference is not clinically important

22. Consequences Small sample size Large variability "Equivalence" Large samples size Small variability Penalty for companies to show equivalence An ill-posed problem that encourages poor trials A bad answer to a wrong question

23. Too restrictive and not relevant Tand R are close A wrong question? Classical hypotheses for student t-test T = population mean for test treatment R =population mean for reference treatment H 0 : T= R Treatments areequivalent H 1 : T R Treatments arenotequivalent

24. A bad answer ? Classical test of null hypothesis (student t-test) H 0 : T= R Treatments areequivalent H 1 : T R Treatments arenotequivalent The controlled risk a = risk to wrongly reject H0 = risk to declare not equivalent formulations that are equivalent = risk for drug companies Not important from a regulatory point of view The consumer risk is uncontrolled

25. Bioequivalence : objectives Check that T and R are close with regard to clinical relevance Control the consumer risk risk to declare equivalent treatments that are not

26. Check that T andR are close Close in an absolute way D1T- R D2 bioequivalence T- R< D1 or D2 < T- R bioinequivalence Close in a relative way bioequivalence bioinequivalence [D1 ; D2] = equivalence range (to be discussed)

27. Bioinequivalence H0 Bioinequivalence H0 D1 D2 T- R Possible values of Equivalence range Control the consumer risk A test controls the risk to wrongly choose the H1 hypothesis Consumer risk : the risk to wrongly conclude to bioequivalence Bioequivalence H1

28. H0 : H1 : Hypotheses of a bioequivalence study Additive hypotheses H0 : T- R< D1 or D2 < T- R bioinequivalence H1 : D1T- R D2 bioequivalence Multiplicative hypotheses bioinequivalence bioequivalence [D1 ; D2] = equivalence range