Comparison of Outcomes from a Phase 3 Study of Age-Related Macular Degeneration with a Matched, Observational Cohort Mark C. Gillies, MBBS, PhD,1 Richard J. Walton, MSc,1 Jennifer J. Arnold, MBBS(Hons),2 Ian L. McAllister, MBBS,3 Judy M. Simpson, PhD,4 Alex P. Hunyor, MBBS,5 Robyn Guymer, MBBS, PhD,6 Rohan W. Essex, MBBS,7 Nigel Morlet, MBBS,8 Daniel Barthelmes, MD, PhD1,9 Objective: To compare outcomes of intravitreal therapy from an observational study cohort with those of participants receiving treatment in the Minimally Classic/Occult Trial of the Anti-VEGF Antibody Ranibizumab (MARINA) for the treatment of neovascular age-related macular degeneration (wet AMD). Design: Database observational study. Participants in the observational cohort were chosen to match demographic features and entry criteria of the treatment group from MARINA. Outcomes over 12 months were compared. Participants: Eight hundred twenty-one antievascular endothelial growth factor (anti-VEGF)-naïve eyes treated with ranibizumab with 12 months or more of follow-up were included in the total Fight Retinal Blindness! (FRB-All) cohort, whereas a subset of this cohort of 401 eyes who were matched to the MARINA treatment group were included as the FRB-MARINA cohort. Intervention: Intravitreal ranibizumab therapy of 0.5 mg for wet AMD. Methods: Visual acuity (VA) in logarithm of the minimum angle of resolution (logMAR) letters and treatments given were recorded continuously and anonymously in an electronic database for 12 months. Locally weighted scatterplot smoothing (LOESS) regression was used to plot change in visual acuity data over the course of 12 months for both the FRB-All cohort and the FRB-MARINA cohort, whereas results from the MARINA trial were taken from the published study report. Main Outcome Measures: Change in VA in logMAR letters over 12 months, treatment, and visit intensity. Results: Mean visual acuity improvement after 12 months in FRB-MARINA (þ5.5 letters) was similar to that of the 0.5-mg group from MARINA (þ7.2 letters). Improvement in FRB-ALL was slightly less (þ4.9 letters). Mean treatment effect compared with the MARINA control group was similar for the MARINA treated group (þ17.6 letters) and the FRB-MARINA cohort (þ15.9 letters). A mean of 7.3 injections in 12 months was received by the observational cohorts. Conclusions: Similarity of mean VA improvement in the matched observational cohort with that of the phase 3 clinical trial suggests that these results can be achieved in real-world clinical practice with a modified treatment regimen. Ophthalmology 2014;121:676-681 ª 2014 by the American Academy of Ophthalmology. Rather than studying random samples of the general population with a disease, interventional clinical trials seek to enrich the study population to maximize the chance of demonstrating a treatment effect.1 The pivotal studies of ranibizumab for neovascular age-related macular degeneration (wet AMD), the Minimally Classic/Occult Trial of the Anti-VEGF Antibody Ranibizumab (MARINA)2 and the Anti-VEGF Antibody for the Treatment of Predominantly Classic Choroidal Neovascularization in Age-Related Macular Degeneration (ANCHOR) trial,3 applied exclusions such as lesion size, lesion type, visual acuity, presence of structural damage, and whether there was recent evidence of disease progression. Although phase 3 clinical trials such as these test whether a treatment can work under ideal and standardized conditions, observational studies can measure how a treatment actually works in the general population. A number of recent observational studies that examined outcomes in the general population generally have reported a positive effect of intravitreal therapy with ranibizumab for
676
2014 by the American Academy of Ophthalmology Published by Elsevier Inc.
wet AMD, but the effect generally has been less pronounced than was reported by the phase 3 studies of ranibizumab.4e8 In some, there was a good initial gain in visual acuity that tailed off after 12 months.9e11 Although different treatment regimens used in clinical trials and observational studies undoubtedly play a role, there may be other sources of bias that may influence the outcomes reported by the 2 types of study. Since observational studies have much wider inclusion criteria than randomized controlled trials, their results may be influenced by ceiling and floor effects because limitations on visual acuity, lesion size, previous treatments, and disease activity are not imposed. Other effects, such as a patient’s fitness and willingness to participate in a clinical study, also may conspire to make the outcomes of phase 3 studies unattainable in cohorts from routine practice, even when treatment is being given satisfactorily. We conducted the present study to investigate the hypothesis that outcomes of patients selected from an observational study cohort with occult and minimally classic ISSN 0161-6420/14/$ - see front matter http://dx.doi.org/10.1016/j.ophtha.2013.09.050
Gillies et al
Observational and Phase III Studies for Wet AMD
choroidal neovascularization (CNV) receiving intravitreal therapy for wet AMD are similar to those of the pivotal phase 3 clinical trial MARINA when the patient groups from the observational study are closely matched with those of MARINA. Because we already have reported that visual improvements for classic CNV are similar to those of minimally classic and occult CNV, as have the Comparison of AMD Treatment Trials and the Phase IIIb, Multicenter, Randomized, Double-Masked, Sham Injection-Controlled Study of the Efficacy and Safety of Ranibizumab in Subjects with Subfoveal CNV with or without Classic CNV Secondary to AMD studies,12,13 we did not match and compare results of our observational cohort with classic CNV with those of the ANCHOR study treatment groups, because the latter’s improvement in visual acuity over 12 months was substantially greater than that of MARINA.
Methods Ethics Each of the 3 academic core centers from the Universities of Sydney, Melbourne, and Western Australia obtained approval from their respective human research ethics committees to conduct the project as a quality assurance activity. Overarching ethical approval for the other centers was obtained from the Royal Australian and New Zealand College of Ophthalmologists’ Human Research Ethics Committee. All ethics committees approved the use of opt-out patient consent. The described research adhered to the tenets of the Declaration of Helsinki.
Fight Retinal Blindness Database and Creation of Matched Cohort Data were used from the Fight Retinal Blindness! (FRB) database wet AMD audit. Design, including quality assurance features, of the FRB project’s web-based tool that prospectively collects data on people receiving treatment for wet AMD have been published previously.14 All investigators administering intravitreal therapy in the FRB project used some form of variable dosing regimen in consultation with their patients according to real-world clinical practice. Investigators were asked to enter whichever visual acuity reading was best: uncorrected, corrected, or pinhole. The best visual acuity score achieved during each visit was used for analysis. At the time of analysis, 31 practices located in Australia, New Zealand, and Switzerland contributed to the database with a median of 32 eyes per practice. All treatment-naïve eyes with occult or minimally classic subretinal neovascularization resulting from AMD entering the FRB database from January 2006 through December 2011 that were treated with ranibizumab and had at least 12 months of follow-up were considered for analysis. If eyes were paired, one was deleted randomly, ensuring that only 1 eye per patient was included in any analysis.15 Two cohorts were formed: all eligible eyes after random deletion of fellow eyes (the FRB-All cohort) and the matched FRBMARINA cohort, which was formed by applying the relevant inclusion and exclusion criteria stipulated by the MARINA protocol2 as well as with the maximum and minimum values reported in the summaries of baseline characteristics.
Treatment Protocol No treatment protocols were specified for this observational study. At a recent meeting of the Australian and New Zealand Retinal
Society (June 2, 2013, Sydney) that was attended by most FRB project investigators, a poll of 140 delegates’ personal treatment strategies revealed the following proportions: 74% inject and extend; 13% pro re nata with less than monthly follow-up; 9% pro re nata with strict monthly follow-up; and 4% strict monthly treatment.
Study End Points and Analyses The clinical end points of interest were the following: the proportion of eyes losing fewer than 15 logarithm of the minimum angle of resolution letters, change from baseline in visual acuity at 12 months, the estimated treatment effect at 12 months (estimated using the sham injection group of the MARINA trial), as well as treatment and visit frequency. The primary analysis compared outcomes of the FRBMARINA matched cohort with the reported outcomes of the associated 0.5-mg ranibizumab group in the MARINA trial. Secondary analyses compared MARINA outcomes with the larger FRB-All cohort.
Statistical Analyses Continuous data are summarized as mean and standard deviation or median and twenty-fifth and seventy-fifth percentiles and categorical variables as proportions. Because the FRB data are gathered in an observational setting, they do not have the prespecified, fixed-visit structure that is imposed by clinical trial protocols. To summarize the group change from baseline visual acuity at 12 months, the mean change was calculated using data from an eye’s nearest visit to 12 months after baseline. Proportions responding to treatment were compared using the chisquare test. Differences in mean changes in VA at 12 months were not tested formally because this would require eye-level data from the phase 3 trial database, which we do not have. Locally weighted scatterplot smoothing (LOESS) regression performed using R (R: A Language and Environment for Statistical Computing; R Core Team, R Foundation for Statistical Computing, Vienna, Austria) was used to fit a smooth curve to the change in visual acuity data over the course of 12 months of follow-up to elucidate the overall treatment response profile. Results from the MARINA trial were taken from the published study report.2
Results Available Data before Trimming Data from 821 eyes naïve to antievascular endothelial growth factor treatment with occult or minimally classic lesions subsequently treated with Lucentis with 12 months or more follow-up were available from the FRB wet AMD audit database. One hundred seventy-five fellow eyes were removed randomly to form the FRB-All cohort. This cohort then was refined further to form the FRB-MARINA cohort as follows: Eyes that previously had been treated with laser or photodynamic therapy were excluded, as were eyes that did not have visual acuity between 25 and 70 letters, along with any patients not between 52 and 93 years of age. This selection yielded 401 eyes to form the matched FRB-MARINA cohort.
Baseline Data Although the inclusion and exclusion criteria specify who can possibly enter a trial, they do not necessarily characterize which patients actually did enter the trial. However, a well-matched
677
Ophthalmology Volume 121, Number 3, March 2014 Table 1. Baseline Characteristics Reported in MARINA Cohort, the Matched Fight Retinal Blindness-MARINA Cohort, and the Fight Retinal Blindness-All Cohort
No. VA (Q1 and Q3), logMAR VA
676
2014 by the American Academy of Ophthalmology Published by Elsevier Inc.
wet AMD, but the effect generally has been less pronounced than was reported by the phase 3 studies of ranibizumab.4e8 In some, there was a good initial gain in visual acuity that tailed off after 12 months.9e11 Although different treatment regimens used in clinical trials and observational studies undoubtedly play a role, there may be other sources of bias that may influence the outcomes reported by the 2 types of study. Since observational studies have much wider inclusion criteria than randomized controlled trials, their results may be influenced by ceiling and floor effects because limitations on visual acuity, lesion size, previous treatments, and disease activity are not imposed. Other effects, such as a patient’s fitness and willingness to participate in a clinical study, also may conspire to make the outcomes of phase 3 studies unattainable in cohorts from routine practice, even when treatment is being given satisfactorily. We conducted the present study to investigate the hypothesis that outcomes of patients selected from an observational study cohort with occult and minimally classic ISSN 0161-6420/14/$ - see front matter http://dx.doi.org/10.1016/j.ophtha.2013.09.050
Gillies et al
Observational and Phase III Studies for Wet AMD
choroidal neovascularization (CNV) receiving intravitreal therapy for wet AMD are similar to those of the pivotal phase 3 clinical trial MARINA when the patient groups from the observational study are closely matched with those of MARINA. Because we already have reported that visual improvements for classic CNV are similar to those of minimally classic and occult CNV, as have the Comparison of AMD Treatment Trials and the Phase IIIb, Multicenter, Randomized, Double-Masked, Sham Injection-Controlled Study of the Efficacy and Safety of Ranibizumab in Subjects with Subfoveal CNV with or without Classic CNV Secondary to AMD studies,12,13 we did not match and compare results of our observational cohort with classic CNV with those of the ANCHOR study treatment groups, because the latter’s improvement in visual acuity over 12 months was substantially greater than that of MARINA.
Methods Ethics Each of the 3 academic core centers from the Universities of Sydney, Melbourne, and Western Australia obtained approval from their respective human research ethics committees to conduct the project as a quality assurance activity. Overarching ethical approval for the other centers was obtained from the Royal Australian and New Zealand College of Ophthalmologists’ Human Research Ethics Committee. All ethics committees approved the use of opt-out patient consent. The described research adhered to the tenets of the Declaration of Helsinki.
Fight Retinal Blindness Database and Creation of Matched Cohort Data were used from the Fight Retinal Blindness! (FRB) database wet AMD audit. Design, including quality assurance features, of the FRB project’s web-based tool that prospectively collects data on people receiving treatment for wet AMD have been published previously.14 All investigators administering intravitreal therapy in the FRB project used some form of variable dosing regimen in consultation with their patients according to real-world clinical practice. Investigators were asked to enter whichever visual acuity reading was best: uncorrected, corrected, or pinhole. The best visual acuity score achieved during each visit was used for analysis. At the time of analysis, 31 practices located in Australia, New Zealand, and Switzerland contributed to the database with a median of 32 eyes per practice. All treatment-naïve eyes with occult or minimally classic subretinal neovascularization resulting from AMD entering the FRB database from January 2006 through December 2011 that were treated with ranibizumab and had at least 12 months of follow-up were considered for analysis. If eyes were paired, one was deleted randomly, ensuring that only 1 eye per patient was included in any analysis.15 Two cohorts were formed: all eligible eyes after random deletion of fellow eyes (the FRB-All cohort) and the matched FRBMARINA cohort, which was formed by applying the relevant inclusion and exclusion criteria stipulated by the MARINA protocol2 as well as with the maximum and minimum values reported in the summaries of baseline characteristics.
Treatment Protocol No treatment protocols were specified for this observational study. At a recent meeting of the Australian and New Zealand Retinal
Society (June 2, 2013, Sydney) that was attended by most FRB project investigators, a poll of 140 delegates’ personal treatment strategies revealed the following proportions: 74% inject and extend; 13% pro re nata with less than monthly follow-up; 9% pro re nata with strict monthly follow-up; and 4% strict monthly treatment.
Study End Points and Analyses The clinical end points of interest were the following: the proportion of eyes losing fewer than 15 logarithm of the minimum angle of resolution letters, change from baseline in visual acuity at 12 months, the estimated treatment effect at 12 months (estimated using the sham injection group of the MARINA trial), as well as treatment and visit frequency. The primary analysis compared outcomes of the FRBMARINA matched cohort with the reported outcomes of the associated 0.5-mg ranibizumab group in the MARINA trial. Secondary analyses compared MARINA outcomes with the larger FRB-All cohort.
Statistical Analyses Continuous data are summarized as mean and standard deviation or median and twenty-fifth and seventy-fifth percentiles and categorical variables as proportions. Because the FRB data are gathered in an observational setting, they do not have the prespecified, fixed-visit structure that is imposed by clinical trial protocols. To summarize the group change from baseline visual acuity at 12 months, the mean change was calculated using data from an eye’s nearest visit to 12 months after baseline. Proportions responding to treatment were compared using the chisquare test. Differences in mean changes in VA at 12 months were not tested formally because this would require eye-level data from the phase 3 trial database, which we do not have. Locally weighted scatterplot smoothing (LOESS) regression performed using R (R: A Language and Environment for Statistical Computing; R Core Team, R Foundation for Statistical Computing, Vienna, Austria) was used to fit a smooth curve to the change in visual acuity data over the course of 12 months of follow-up to elucidate the overall treatment response profile. Results from the MARINA trial were taken from the published study report.2
Results Available Data before Trimming Data from 821 eyes naïve to antievascular endothelial growth factor treatment with occult or minimally classic lesions subsequently treated with Lucentis with 12 months or more follow-up were available from the FRB wet AMD audit database. One hundred seventy-five fellow eyes were removed randomly to form the FRB-All cohort. This cohort then was refined further to form the FRB-MARINA cohort as follows: Eyes that previously had been treated with laser or photodynamic therapy were excluded, as were eyes that did not have visual acuity between 25 and 70 letters, along with any patients not between 52 and 93 years of age. This selection yielded 401 eyes to form the matched FRB-MARINA cohort.
Baseline Data Although the inclusion and exclusion criteria specify who can possibly enter a trial, they do not necessarily characterize which patients actually did enter the trial. However, a well-matched
677
Ophthalmology Volume 121, Number 3, March 2014 Table 1. Baseline Characteristics Reported in MARINA Cohort, the Matched Fight Retinal Blindness-MARINA Cohort, and the Fight Retinal Blindness-All Cohort
No. VA (Q1 and Q3), logMAR VA
