509509 2013
MSJ20610.1177/1352458513509509Multiple Sclerosis JournalCarruthers et al.
MULTIPLE SCLEROSIS MSJ JOURNAL
Personal Viewpoint
Modeling probability of additional cases of natalizumab-associated JCV sero-negative progressive multifocal leukoencephalopathy
Multiple Sclerosis Journal 2014, Vol. 20(6) 757–760 © The Author(s) 2013 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1352458513509509 msj.sagepub.com
Robert L Carruthers1,2, Tanuja Chitnis1,2 and Brian C Healy1,2,3
Abstract JCV serologic status is used to determine PML risk in natalizumab-treated patients. Given two cases of natalizumabassociated PML in JCV sero-negative patients and two publications that question the false negative rate of the JCV serologic test, clinicians may question whether our understanding of PML risk is adequate. Given that there is no gold standard for diagnosing previous JCV exposure, the test characteristics of the JCV serologic test are unknowable. We propose a model of PML risk in JCV sero-negative natalizumab patients. Using the numbers of JCV sero-positive and -negative patients from a study of PML risk by JCV serologic status (sero-positive: 13,950 and sero-negative: 11,414), we apply a range of sensitivities and specificities in order calculate the number of JCV-exposed but JCV sero-negative patients (false negatives). We then apply a range of rates of developing PML in sero-negative patients to calculate the expected number of PML cases. By using the binomial function, we calculate the probability of a given number of JCV sero-negative PML cases. With this model, one has a means to establish a threshold number of JCV sero-negative natalizumab-associated PML cases at which it is improbable that our understanding of PML risk in JCV sero-negative patients is adequate. Keywords disease modifying therapies, JC Virus, multiple sclerosis, natalizumab, progressive multifocal leukoencephalopathy Date received: 3 September 2013; accepted: 23 September 2013
Introduction A model of PML risk that includes JCV serology results has enabled many patients to benefit from the efficacy of natalizumab. Since data regarding PML risk by JCV serology status in natalizumab-treated multiple sclerosis patients was reviewed,1 there have been two cases of PML in JCV sero-negative patients tested 8 and 9 months prior to diagnosis.2 Also, two publications3,4 suggest that the false negative rate of the serologic test may be higher than 2.2% originally reported.5 The relevance of diverging false negative rates in the previously mentioned reports is unclear as the serologic test was developed to stratify risk of PML, not to test primary exposure. Furthermore, the sensitivity and specificity of the JCV serologic test are unknowable as there is no gold standard test for JCV exposure. Patients with positive urinary or serum JCV DNA PCR have been used to assess the false negative rate of the JCV test. Since JCV sero-negative natalizumab-associated PML is a rare but anticipated event, we raise two important questions: 1) is there a threshold rate of JCV sero-negative PML cases at which our working model of PML risk is inadequate, and 2)
how do JCV serology sensitivity/specificity, and the rate of PML in falsely JCV sero-negative patients each impact this threshold?
Methods A schematic of the study design appears in Figure 1. Using the numbers of JCV sero-positive and sero-negative patients from the Bloomgren study, we applied a range of sensitivities and specificities of the JCV serologic test (80%, 90%, 1Department
of Neurology, Harvard Medical School, Boston, MA, USA. Multiple Sclerosis Center, Brigham and Women’s Hospital, Boston, MA, USA. 3Biostatistics Center, Massachusetts General Hospital, Boston, MA, USA. 2Partners
Corresponding author: Dr Robert Carruthers, Partners MS Center, 1 Brookline Place Suite 225, Brookline, MA 02446, USA. Email: [email protected]
758
Multiple Sclerosis Journal 20(6)
Figure 1. Using Bloomgren et al’s number of JCV sero-positive and sero-negative patients, we applied an assumed sensitivity and specificity of the JCV serologic test in order to calculate the number of patients with falsely negative JCV serologic tests. Assuming a given rate of PML in JCV exposed but sero-negative patients, the expected number of PML cases can be calculated. The binomial function can be used to determine the probability of a given outcome.
95%, and 97.5%) which enable one to calculate the number of patients with falsely negative JCV serology tests. In addition, we assessed a range of potential rates of developing PML in the false negatives bracketing (1.02, 2.04, and 8.16 PML cases per 1000 patients) and equal to that reported in the JCV sero-positive group (4.08 PML cases per 1000 patients)1 to calculate the predicted number of cases of JCV sero-negative PML arising from the false negative group. We then used the binomial function to calculate the probability of a given number of JCV sero-negative PML cases.
would argue against there being a continuum of PML risk that extends below the cutpoint of the JCV assay. Our model suggests that more than five or ten cases of JCV sero-negative PML in a group of 25,364 patients would be highly improbable except if the test sensitivity is poor and the probability of PML is high. The rate of PML in falsely-negative JCV patients and the assay sensitivity are key factors in determining the probability of specific numbers of events.
Results
This model does establish a threshold number of JCV seronegative PML cases arising in group of 25,364 JCV serology-tested natalizumab-treated patients that would be highly improbable given our assumptions. Implicit in our model is the assumption that JCV serology functions a determinant of primary exposure rather than PML risk. Recent data regarding JCV index and PML risk might suggest that the serologic test does predict PML risk in the range that the assay is considered positive. Whether this applies in the JCV sero-negative range is unclear but further cases of JCV sero-negative PML might be informative. It is important to note that our model does not address the effects of serial testing, as mandated by Biogen’s prudent label change requiring JCV serology testing every 6 months while receiving natalizumab. Depending on the inter-test correlation, the safety benefit of serial testing could range from being multiplicative (enhancing safety) to null (no effect). Simply stated, if JCV-exposed patients remain sero-negative, then there will be no benefit to serial testing. Since the effect of serial testing is not accounted for in our model, we feel that additional cases of JCV seronegative cases where serial testing was performed would suggest that inter-test correlation is high and could indicate that serial testing does not eliminate risk of PML in JCV sero-negative patients.
Table 1 presents several potential sensitivities and specificities of the JCV test and the corresponding number of patients with falsely negative JCV serology assuming previously published JCV serology results. Based on these values and a range of PML risk in JCV sero-negative, the probability of observing a specific number of cases of PML among the false negatives can be calculated using the binomial distribution. Based on our calculations for Bloomgren et al’s group of 25,364 patients, there is a fair (11%–17%) to high (89%–95%) chance of at least three cases of PML in the false negative group assuming that the rate of developing PML in the JCV sero-negative group is the same as the JCV sero-positive group. Therefore, observed cases of PML could result from falsely negative tests. Given the range of assay sensitivities and specificities shown in Table 1, it would be improbable to observe 5 or more cases of PML in Bloomgren’s group of 25,364 JCV serology tested patients1 if the rate of PML was substantially lower in the JCV sero-negative group than in the seropositive group. This would be of interest particularly when one considers recently published data regarding PML risk in JCV sero-positive patients can be further refined by JCV index.6 Five or more cases of JCV sero-negative PML
Discussion
759
Carruthers et al.
Table 1. Using published JCV serology data, (sero-positive: 13,950 and sero-negative: 11,414), a range of JCV serology test sensitivities and specificities, the number of false negatives is equal to (JCV sero neg − (Spec/(1−Spec) × JCV sero pos))/((1−Sens) − (Spec/(1−Spec) × Sens)) × (1−Sens). Using the binomial function with the probability of PML and number of false negatives as the parameters, the probability of several specific numbers of cases of PML can be calculated. Sensitivity
Specificity
Number with false negative JCV serology
Probability of PML
Probability of 3 or more PML cases
Probability of 5 or more
Probability of 10 or more
0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9 0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9 0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9 0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9
0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8
350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2 350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2 350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2 350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2
0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102
0.547 0.499 0.415 0.933 0.911 0.861 1.000 0.999 0.998 0.174 0.151 0.114 0.563 0.517 0.435 0.947 0.930 0.890 0.036 0.030 0.022 0.183 0.159 0.122 0.600 0.557 0.479 0.006 0.005 0.003 0.038 0.032 0.023 0.204 0.180 0.142
0.162 0.132 0.088 0.699 0.641 0.530 0.994 0.990 0.977 0.015 0.012 0.007 0.174 0.143 0.097 0.743 0.691 0.590 0.001 0.001 0.000 0.017 0.013 0.008 0.202 0.170 0.121 3.55E-05 2.54E-05 1.36E-05 9.61E-04 7.08E-04 4.01E-04 0.021 0.017 0.010
0.001 0.000 0.000 0.075 0.052 0.026 0.793 0.726 0.586 2.49E-06 1.33E-06 4.09E-07 9.15E-04 5.44E-04 2.03E-04 9.87E-02 7.18E-02 3.84E-02 4.56E-09 2.33E-09 6.66E-10 3.26E-06 1.78E-06 5.75E-07 1.40E-03 8.69E-04 3.56E-04 6.11E-12 3.05E-12 8.41E-13 6.12E-09 3.20E-09 9.60E-10 5.40E-06 3.08E-06 1.10E-06
Physicians must remain vigilant for JCV sero-negative PML as additional cases could force the MS community to re-evaluate the working model of PML risk in JCV seronegative patients and even prompt use of ancillary JCV testing.5,7 While there are limitations in our approach, we hope that clinicians find this to be a helpful first step in setting an a priori threshold at which we re-think our understanding of PML risk in natalizumab-treated JCV sero-negative patients. Acknowledgements RC and BH conceived of the study design and drafted the manuscript. TC aided in study design and drafted the manuscript.
Conflict of interest RC received consulting fees from Genzyme. TC received personal compensation from Biogen Idec, Novartis, Sanofi Aventis, EMDSerono, and Teva Neurosciences for consulting services. BH received grant support from Merck-Serono.
Funding RC received a Clinical Fellowship Training Grant from the National Multiple Sclerosis Society.
References 1. Bloomgren G, Richman S, Hotermans C, et al. Risk of natalizumab-associated progressive multifocal leukoencephalopathy. N Engl J Med 2012; 366: 1870–80.
760 2. PML incidence in patients receiving Tysabri (natalizumab). Biogen Idec; 2013 [updated 7/14/2013 ]; Available from: http://medinfo.biogenidec.com/ 3. Berger JR, Houff SA, Gurwell J, et al. JC virus antibody status underestimates infection rates. Ann Neurol 2013. Jul; 74(1): 84–90. 4. Major EO, Frohman E and Douek D. JC viremia in natalizumab-treated patients with multiple sclerosis. N Engl J Med 2013; 368: 2240–1.
Multiple Sclerosis Journal 20(6) 5. Gorelik L, Lerner M, Bixler S, et al. Anti-JC virus antibodies: implications for PML risk stratification. Ann Neurol 2010; 68: 295–303. 6. Plavina T. Anti-JCV antibody index further defines PML risk in natalizumab-treated MS patients. Consortium of Multiple Sclerosis Centers. Orlando, FL, 2013. 7. Laroni A, Giacomazzi CG, Grimaldi L, et al. Urinary JCVDNA testing during natalizumab treatment may increase accuracy of PML risk stratification. J Neuroimmune Pharmacol 2012; 7: 665–72.
MSJ20610.1177/1352458513509509Multiple Sclerosis JournalCarruthers et al.
MULTIPLE SCLEROSIS MSJ JOURNAL
Personal Viewpoint
Modeling probability of additional cases of natalizumab-associated JCV sero-negative progressive multifocal leukoencephalopathy
Multiple Sclerosis Journal 2014, Vol. 20(6) 757–760 © The Author(s) 2013 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1352458513509509 msj.sagepub.com
Robert L Carruthers1,2, Tanuja Chitnis1,2 and Brian C Healy1,2,3
Abstract JCV serologic status is used to determine PML risk in natalizumab-treated patients. Given two cases of natalizumabassociated PML in JCV sero-negative patients and two publications that question the false negative rate of the JCV serologic test, clinicians may question whether our understanding of PML risk is adequate. Given that there is no gold standard for diagnosing previous JCV exposure, the test characteristics of the JCV serologic test are unknowable. We propose a model of PML risk in JCV sero-negative natalizumab patients. Using the numbers of JCV sero-positive and -negative patients from a study of PML risk by JCV serologic status (sero-positive: 13,950 and sero-negative: 11,414), we apply a range of sensitivities and specificities in order calculate the number of JCV-exposed but JCV sero-negative patients (false negatives). We then apply a range of rates of developing PML in sero-negative patients to calculate the expected number of PML cases. By using the binomial function, we calculate the probability of a given number of JCV sero-negative PML cases. With this model, one has a means to establish a threshold number of JCV sero-negative natalizumab-associated PML cases at which it is improbable that our understanding of PML risk in JCV sero-negative patients is adequate. Keywords disease modifying therapies, JC Virus, multiple sclerosis, natalizumab, progressive multifocal leukoencephalopathy Date received: 3 September 2013; accepted: 23 September 2013
Introduction A model of PML risk that includes JCV serology results has enabled many patients to benefit from the efficacy of natalizumab. Since data regarding PML risk by JCV serology status in natalizumab-treated multiple sclerosis patients was reviewed,1 there have been two cases of PML in JCV sero-negative patients tested 8 and 9 months prior to diagnosis.2 Also, two publications3,4 suggest that the false negative rate of the serologic test may be higher than 2.2% originally reported.5 The relevance of diverging false negative rates in the previously mentioned reports is unclear as the serologic test was developed to stratify risk of PML, not to test primary exposure. Furthermore, the sensitivity and specificity of the JCV serologic test are unknowable as there is no gold standard test for JCV exposure. Patients with positive urinary or serum JCV DNA PCR have been used to assess the false negative rate of the JCV test. Since JCV sero-negative natalizumab-associated PML is a rare but anticipated event, we raise two important questions: 1) is there a threshold rate of JCV sero-negative PML cases at which our working model of PML risk is inadequate, and 2)
how do JCV serology sensitivity/specificity, and the rate of PML in falsely JCV sero-negative patients each impact this threshold?
Methods A schematic of the study design appears in Figure 1. Using the numbers of JCV sero-positive and sero-negative patients from the Bloomgren study, we applied a range of sensitivities and specificities of the JCV serologic test (80%, 90%, 1Department
of Neurology, Harvard Medical School, Boston, MA, USA. Multiple Sclerosis Center, Brigham and Women’s Hospital, Boston, MA, USA. 3Biostatistics Center, Massachusetts General Hospital, Boston, MA, USA. 2Partners
Corresponding author: Dr Robert Carruthers, Partners MS Center, 1 Brookline Place Suite 225, Brookline, MA 02446, USA. Email: [email protected]
758
Multiple Sclerosis Journal 20(6)
Figure 1. Using Bloomgren et al’s number of JCV sero-positive and sero-negative patients, we applied an assumed sensitivity and specificity of the JCV serologic test in order to calculate the number of patients with falsely negative JCV serologic tests. Assuming a given rate of PML in JCV exposed but sero-negative patients, the expected number of PML cases can be calculated. The binomial function can be used to determine the probability of a given outcome.
95%, and 97.5%) which enable one to calculate the number of patients with falsely negative JCV serology tests. In addition, we assessed a range of potential rates of developing PML in the false negatives bracketing (1.02, 2.04, and 8.16 PML cases per 1000 patients) and equal to that reported in the JCV sero-positive group (4.08 PML cases per 1000 patients)1 to calculate the predicted number of cases of JCV sero-negative PML arising from the false negative group. We then used the binomial function to calculate the probability of a given number of JCV sero-negative PML cases.
would argue against there being a continuum of PML risk that extends below the cutpoint of the JCV assay. Our model suggests that more than five or ten cases of JCV sero-negative PML in a group of 25,364 patients would be highly improbable except if the test sensitivity is poor and the probability of PML is high. The rate of PML in falsely-negative JCV patients and the assay sensitivity are key factors in determining the probability of specific numbers of events.
Results
This model does establish a threshold number of JCV seronegative PML cases arising in group of 25,364 JCV serology-tested natalizumab-treated patients that would be highly improbable given our assumptions. Implicit in our model is the assumption that JCV serology functions a determinant of primary exposure rather than PML risk. Recent data regarding JCV index and PML risk might suggest that the serologic test does predict PML risk in the range that the assay is considered positive. Whether this applies in the JCV sero-negative range is unclear but further cases of JCV sero-negative PML might be informative. It is important to note that our model does not address the effects of serial testing, as mandated by Biogen’s prudent label change requiring JCV serology testing every 6 months while receiving natalizumab. Depending on the inter-test correlation, the safety benefit of serial testing could range from being multiplicative (enhancing safety) to null (no effect). Simply stated, if JCV-exposed patients remain sero-negative, then there will be no benefit to serial testing. Since the effect of serial testing is not accounted for in our model, we feel that additional cases of JCV seronegative cases where serial testing was performed would suggest that inter-test correlation is high and could indicate that serial testing does not eliminate risk of PML in JCV sero-negative patients.
Table 1 presents several potential sensitivities and specificities of the JCV test and the corresponding number of patients with falsely negative JCV serology assuming previously published JCV serology results. Based on these values and a range of PML risk in JCV sero-negative, the probability of observing a specific number of cases of PML among the false negatives can be calculated using the binomial distribution. Based on our calculations for Bloomgren et al’s group of 25,364 patients, there is a fair (11%–17%) to high (89%–95%) chance of at least three cases of PML in the false negative group assuming that the rate of developing PML in the JCV sero-negative group is the same as the JCV sero-positive group. Therefore, observed cases of PML could result from falsely negative tests. Given the range of assay sensitivities and specificities shown in Table 1, it would be improbable to observe 5 or more cases of PML in Bloomgren’s group of 25,364 JCV serology tested patients1 if the rate of PML was substantially lower in the JCV sero-negative group than in the seropositive group. This would be of interest particularly when one considers recently published data regarding PML risk in JCV sero-positive patients can be further refined by JCV index.6 Five or more cases of JCV sero-negative PML
Discussion
759
Carruthers et al.
Table 1. Using published JCV serology data, (sero-positive: 13,950 and sero-negative: 11,414), a range of JCV serology test sensitivities and specificities, the number of false negatives is equal to (JCV sero neg − (Spec/(1−Spec) × JCV sero pos))/((1−Sens) − (Spec/(1−Spec) × Sens)) × (1−Sens). Using the binomial function with the probability of PML and number of false negatives as the parameters, the probability of several specific numbers of cases of PML can be calculated. Sensitivity
Specificity
Number with false negative JCV serology
Probability of PML
Probability of 3 or more PML cases
Probability of 5 or more
Probability of 10 or more
0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9 0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9 0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9 0.975 0.975 0.975 0.95 0.95 0.95 0.9 0.9 0.9
0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8 0.975 0.9 0.8
350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2 350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2 350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2 350.4 326.1 286.4 719.8 671.4 591.8 1521.8 1426.7 1268.2
0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00816 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00408 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00204 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102 0.00102
0.547 0.499 0.415 0.933 0.911 0.861 1.000 0.999 0.998 0.174 0.151 0.114 0.563 0.517 0.435 0.947 0.930 0.890 0.036 0.030 0.022 0.183 0.159 0.122 0.600 0.557 0.479 0.006 0.005 0.003 0.038 0.032 0.023 0.204 0.180 0.142
0.162 0.132 0.088 0.699 0.641 0.530 0.994 0.990 0.977 0.015 0.012 0.007 0.174 0.143 0.097 0.743 0.691 0.590 0.001 0.001 0.000 0.017 0.013 0.008 0.202 0.170 0.121 3.55E-05 2.54E-05 1.36E-05 9.61E-04 7.08E-04 4.01E-04 0.021 0.017 0.010
0.001 0.000 0.000 0.075 0.052 0.026 0.793 0.726 0.586 2.49E-06 1.33E-06 4.09E-07 9.15E-04 5.44E-04 2.03E-04 9.87E-02 7.18E-02 3.84E-02 4.56E-09 2.33E-09 6.66E-10 3.26E-06 1.78E-06 5.75E-07 1.40E-03 8.69E-04 3.56E-04 6.11E-12 3.05E-12 8.41E-13 6.12E-09 3.20E-09 9.60E-10 5.40E-06 3.08E-06 1.10E-06
Physicians must remain vigilant for JCV sero-negative PML as additional cases could force the MS community to re-evaluate the working model of PML risk in JCV seronegative patients and even prompt use of ancillary JCV testing.5,7 While there are limitations in our approach, we hope that clinicians find this to be a helpful first step in setting an a priori threshold at which we re-think our understanding of PML risk in natalizumab-treated JCV sero-negative patients. Acknowledgements RC and BH conceived of the study design and drafted the manuscript. TC aided in study design and drafted the manuscript.
Conflict of interest RC received consulting fees from Genzyme. TC received personal compensation from Biogen Idec, Novartis, Sanofi Aventis, EMDSerono, and Teva Neurosciences for consulting services. BH received grant support from Merck-Serono.
Funding RC received a Clinical Fellowship Training Grant from the National Multiple Sclerosis Society.
References 1. Bloomgren G, Richman S, Hotermans C, et al. Risk of natalizumab-associated progressive multifocal leukoencephalopathy. N Engl J Med 2012; 366: 1870–80.
760 2. PML incidence in patients receiving Tysabri (natalizumab). Biogen Idec; 2013 [updated 7/14/2013 ]; Available from: http://medinfo.biogenidec.com/ 3. Berger JR, Houff SA, Gurwell J, et al. JC virus antibody status underestimates infection rates. Ann Neurol 2013. Jul; 74(1): 84–90. 4. Major EO, Frohman E and Douek D. JC viremia in natalizumab-treated patients with multiple sclerosis. N Engl J Med 2013; 368: 2240–1.
Multiple Sclerosis Journal 20(6) 5. Gorelik L, Lerner M, Bixler S, et al. Anti-JC virus antibodies: implications for PML risk stratification. Ann Neurol 2010; 68: 295–303. 6. Plavina T. Anti-JCV antibody index further defines PML risk in natalizumab-treated MS patients. Consortium of Multiple Sclerosis Centers. Orlando, FL, 2013. 7. Laroni A, Giacomazzi CG, Grimaldi L, et al. Urinary JCVDNA testing during natalizumab treatment may increase accuracy of PML risk stratification. J Neuroimmune Pharmacol 2012; 7: 665–72.
