DRCR

1. DRCR.net Visual Acuity Outcomes DRCR.net Clinical Trials Workshop Tampa, FL February 1, 2019

2. Pre-Specified VA Outcomes From Recent DRCR.net Protocols *VA over time from longitudinal analysis **AUC = Area under the curve

3. Pre-Specified VA Outcomes From Recent DRCR.net Protocols ***VA outcome to be tested only if PDR/DME outcome at same time point is statistically significant (p<0.025)

4. Outline • Review frequently reported VA outcomes from clinical trials • Illustrate various outcomes using Protocol I data* • Interpretation • Advantages and disadvantages of each • How to choose for a new study *Data/analyses are intended for illustration only, and have not been verified. Analyses shown here excluded data from one site with a large proportion of ineligible patients.** **The primary publication included these patients, so results may differ slightly.

5. VA Outcomes – Many Choices • Mean ± SD (or Median and percentiles) at single time point • Proportion (P) or % with dichotomized VA outcome at single time point • E.g. % gaining 10+ letters; % with VA 20/40 or better • Area under the curve (AUC) for a defined time period • Time-to-event (cumulative proportion or % with event by time) • E.g. cumulative % gaining 10+ letters at each visit • All of these outcomes can be for change in VA from baseline or VA level

6. Primary Outcome: Protocol IMean Change in VA at 52 Weeks • “All groups improved on average, but ranibizumab groups improved more” • Considerable individual variation (high SD) • Assuming data are normally-distributed: • Mean ± 1 SD ≈ 68% of distribution; Mean ± 2 SD ≈ 95% of distribution • “Individual responses may vary”

7. Primary Outcome: Protocol IMean Change in VA at 52 Weeks • SD (applies to distribution/individuals) vs SE (applies to mean/group) • 95% confidence interval applies to mean, not to individuals • 50% of observed differences fell between the tabled 25th & 75th percentiles • Caution: all these are estimates; not precise when sample size is small!

8. Mean at Single Time Point:Advantages and Disadvantages

9. Median at Single Time Point:Advantages and Disadvantages **All that is needed is ordered rank of each value and value of the midpoint

10. VA Proportion Outcomes: Protocol I • Supports primary results: • Higher % with large gains in VA in ranibizumab groups • Smaller % with large losses in VA • “Nearly ½ of all patients receiving Rb gained 2 or more lines of VA” • “Only about ¼ of patients receiving the competing brand gained ≥2 lines”

11. Proportions: Advantages & Disadvantages

12. Proportions: Disadvantages

13. Information: Mean vs. Proportion • Minimum data to calculate the statistic in question: • Mean VA: each individual’s (change) value • Proportion with 10 letter gain: whether each individual’s change was 10 or more (yes or no) • Calculate proportion with data needed to calculate mean? YES! • Calculate mean with data needed to calculate proportion? NO!

14. Misclassification • May occur in ANY situation where a cut point is applied, as long as: • There is measurement error; AND/OR, • There is moment-to-moment variation in the quantity being measured whose causes are not being captured • The greater the number of outcomes close to the cut point, the greater the misclassification

15. Distribution of VA Change: Groups A and B A: Laser + Sham B: Ranibizumab + Prompt Laser Gained 10 ± 2 letters

16. Mean Change in VA over Time by Treatment

17. VA Measurement Over Time • Area Under the Curve (AUC) • Time to VA event (e.g. gain of 10+ letters) • Mean VA over 52 weeks • Mean proportion with gain of 10 letters over 52 weeks

18. Area Under the Curve: Sample Subject Total AUC = 690 letter-weeks; Average VA = 690 letter-weeks / 52 weeks = 13.3 letters AUC for each subject is approximated by the sum of a series of geometric figures.

19. AUC with Positive and Negative VA Changes Area above the curve (due to negative values) is subtracted from area under the curve (due to positive values).

20. AUC: Protocol I Results and Interpretation • Provides an average VA over a time period for each subject • Sample subject had +13.3 letter gain on average through year 1 • Can calculate AUC summary statistics for treatment groups • E.g. Mean, SD, 95% CI for use in describing/comparing groups • “Ranibizumab groups had larger improvement in VA throughout 1 year” • “Individual results may vary”

21. Additional Comments on AUC • AUC can be a useful measure for assessing outcome over study duration (or any other defined period of time) • AUC divided by the corresponding time period gives an average value over the time period, i.e. it’s a time-averaged value • Mean AUC for a treatment group is an average of an average: • 1. Calculate average over time for each individual → individual AUCs • 2. Calculate average of individual AUCs for each treatment • Usual AUC weights all time periods by their length alone • Future results weighted equally to present results • Can easily be modified if desired • E.g. to discount far future relative to present and near future

22. AUC: Advantages and Disadvantages

23. AUC: Advantages and Disadvantages

24. Time-to-Event: Gain 10+ Letters by 1 Year • “Vision in over 70% of patients improved by 2 or more lines in 1st year with ranibizumab”

25. Time to Gain of 10+ Letters • B and C appear fairly similar • D is similar to B and C early, but appears to diverge around 12 weeks • Strengthens impression D does relatively better on this measure

26. Time to Gain of 10+ Letters • General profile similar by treatment: most 10+ gain events are early (within 16-20 weeks), but additional events continue to be seen up to 52 weeks with all 4 treatments • Partly an artifact of the method • Only 1st event counts per subject • Ceiling at 100% (or less if there is loss to follow up)

27. Time-to-Event: Advantages and Disadvantages

28. Time-to-Event: Disadvantages

29. Mean Change in VA over Time by Treatment

30. Longitudinal Analysis • Alternative to AUC and time-to-event analysis • Allows analysis of mean VA over time or proportions over time • E.g. Model individual profiles of VA over time & calculate average slope (or intercept) for each treatment group • Retains information lost in AUC and time-to-event analyses • Can have very high statistical power • Require smaller sample sizes than other methods (sometimes much smaller)

31. Longitudinal Analysis • More complex to apply and interpret • Involves additional assumptions about data • Frequently, no or insufficient data for sample size calculation • Not always possible to implement (especially for proportions) • Requires advanced statistical software • Poorly fitting model(s) • Algorithm convergence problems • Consider for secondary papers (or in observational studies)

32. Treatment Group Comparisons:*How do the VA measures stack up? • *Confidence intervals and p-values are adjusted for baseline VA, correlation between eyes, and multiple comparisons.

33. Treatment Group Comparisons:How do the VA measures stack up? • Results are generally consistent across all VA outcomes for this protocol. • Ranibizumabis better than laser + sham on all measures • For triamcinolone + prompt laser: • Not different from Laser + sham at 52 weeks • Somewhat better than Laser + sham if consider entire 1st year • +3 letter difference in average VA over 1st year • 40% more likely to gain 10+ letters or more during 1st year

34. Choosing the Primary VA Outcome • Primary VA outcome • Will form basis for conclusions and clinical care recommendations • Basis for sample size calculation (usually) • What do you and your patients care about most? • What do you hope to show? • Over what time frame? • Is there prior data to support your choice? • Consider mean change in VA or VA level at single time first • Are there good reasons to deviate from this choice? • In prevention trials, dichotomous loss of VA outcome not uncommon(Protocol V, CAPT)

35. Choosing Secondary VA Outcomes • Secondary VA outcomes • Add additional information, e.g. other time points • Bolster primary outcome results, e.g. by showing consistency • Aid in clinical interpretation / application of primary results

36. VA Outcomes: Protocol V • What did we want to show? • Which of 3 treatment strategies results in best vision outcome in those with CI-DME and good VA (20/25 or better)? • Eyes are unlikely to improve as have good VA already • Over what time frame? • May take some time for CI-DME to affect vision • Need to also allow time for CI-DME that develops with observation or laser to be treated with anti-VEGF • Prior data? • Nothing recent. ETDRS suggested substantial risk of 5+ letter loss by 36 months. Current rate projected to be lower.

37. Summary • Ultimately, VA outcomes are different facets of same phenomenon • Results across different VA measures generally consistent • However, choice of primary VA outcome affects: • Sample size • Primary conclusion(s) regarding significance of treatment effect(s) • Recommended for 1o: Mean VA at single time point • Choose 2o to aid with interpretation & clinical application • Dichotomized outcomes to look at chances of large gain/loss • Time-to-event to look at timing of gains/losses • AUC to get a time-averaged VA change (or VA level)