ORIGINAL ARTICLE
Transplant Patient Classification and Tacrolimus Assays: More Evidence of the Need for Assay Standardization Yash Pal Agrawal, MBBS, PhD,* Maria Cid, BS,* Sten Westgard, MS,† Thomas S. Parker, PhD,*‡ Ryan Jaikaran, BS,‡ and Daniel M. Levine, PhD*‡
Background: A global tacrolimus proficiency study recently showed clinically significant variability between laboratories, the inability of a common calibrator to harmonize methods, and differences in patient classification depending on the test method. The authors evaluated (1) the effect of a change in methodology on patient classification based on tacrolimus blood concentration and (2) the ability of 2 methods to position the concentration in a given specimen within the correct range.
Methods: A total of 839 consecutive samples were analyzed at The Rogosin Institute and New York Presbyterian Hospital for routine tacrolimus monitoring over 30 days. Concordance analysis between the methods was performed covering dosage target ranges of 8–10, 6–8, 4–6 ng/mL currently used at our center. Six Sigma Metrics were applied to statistically evaluate the discordance rate. Results: Deming regression comparing liquid chromatography– tandem mass spectrometry and immunoassay yielded y = 0.927x 2 0.24; 95% confidence interval, 0.903–0.951; R2 = 0.875; n = 839. There were 310 pairs (37%) discordant by 1, 21 (2.5%) discordant by 2, and 4 (0.5%) discordant by 3 therapeutic ranges. Surprisingly, 40% of patient samples were discordant when therapeutic ranges were 2 ng/mL wide. This discordant rate is equivalent to 1.7 Sigma and falls far below the minimum acceptable threshold of 3 Sigma.
Conclusions: Both methods are capable of measuring tacrolimus in the clinically relevant range between 1 and 10 ng/mL, yet 40% of the samples were discordant with an unacceptable Sigma level. Standardization of tacrolimus assays will mitigate this issue. Key Words: tacrolimus, LC-MS, immunoassay, therapeutic drug monitoring, assay standardization (Ther Drug Monit 2014;36:706–709)
Received for publication January 31, 2014; accepted April 12, 2014. From the *New York Presbyterian Hospital-Weill Cornell Medical College, New York, NY; †Westgard QC, Inc., Madison, Wisconsin; and ‡The Rogosin Institute, New York, NY. D. M. Levine has received honoraria for lectures and has served on an advisory board for Waters. S. Westgard is a consultant for Abbott through his company Westgard QC. The remaining authors declare no conflict of interest. Supported by The Rogosin Institute and the New York Presbyterian HospitalWeill Cornell Medical College. Correspondence: Daniel M. Levine, PhD, The Rogosin Institute, 310 E. 67th St, New York, NY 10065 (e-mail: [email protected]). Copyright © 2014 by Lippincott Williams & Wilkins
706
INTRODUCTION Tacrolimus is an immunosuppressive agent that is used for prophylaxis against organ and tissue rejection after allogeneic transplantation. The Efficacy Limiting Toxicity Elimination (ELiTE)-Symphony study1,2 and the European Consensus Conference on Tacrolimus3 have recommended that dose of tacrolimus is minimized to prevent rejection of a transplanted organ while reducing long-term nephrotoxicity of the drug. Patients with tacrolimus blood concentrations of ,10 ng/mL in the 3-year ELiTE-Symphony follow-up study had the best kidney graft survival.2 At our center, patients receiving a transplanted kidney are targeted to 8–10 ng/mL for the first 3 months after transplant, 6–8 ng/mL from 3 months to 1 year, and 4–6 ng/mL thereafter. A future goal may be to reduce the tacrolimus whole blood concentration even further, to around 2–4 ng/mL, on a patient by patient basis to minimize nephrotoxic effects and still maintain adequate immunosuppression.4 Tacrolimus blood levels of 3–7 ng/mL in renal transplant patients from the early posttransplantation period forward were associated with the lowest risk for acute rejection and the highest glomerular filtration rate at 12 months.5 The Consensus Conference recommends a limit of quantitation of 1 ng/mL to ensure that tacrolimus test methods are both accurate and precise to measure these lower concentrations of tacrolimus. Recently, a global tacrolimus proficiency study assessed the comparability of liquid chromatography–tandem mass spectrometry (LC-MS) and immunoassay measurements and demonstrated the need for assay standardization.6 Exactmatching isotope dilution mass spectrometry served as the candidate reference method. The study showed clinically significant variability between laboratories and that a common calibrator did not harmonize methods. The study also demonstrated that patients would be classified differently for dosing by highlighting the proficiency results from 2 laboratories at the extreme low and high range of the study. Selection of the best method or the method producing the “true” tacrolimus blood concentration cannot be determined in the absence of assay standardization. Given that current guidelines recommend tacrolimus blood concentrations of ,10 ng/mL for patients receiving solid organ transplantation, it is essential to understand the analytical limits of methods used to monitor levels between 1 and 10 ng/mL. In the absence of standardization, the lack of comparability between methods may result in a change in dose classification, which may in turn, lead to under dosing a patient (allograft rejection) or providing an unnecessarily high dose that may result in nephrotoxicity. Ther Drug Monit Volume 36, Number 6, December 2014
Ther Drug Monit Volume 36, Number 6, December 2014
Tacrolimus Assay Method and Patient Classification
The Rogosin Institute (RI) is located at the Cornell campus of the New York Presbyterian Hospital (NYP) in New York City and co-administers the kidney transplant program with the Hospital. The tacrolimus testing service recently moved from RI, running an Abbott Architect immunoassay (IA), to the NYP, where the test is run using the Waters TQD Acquity mass spectrometer with Mass Trak kit (LC-MS). This change in method provides an opportunity to test the predictions of the proficiency study under conditions of standard clinical practice.6
MATERIALS AND METHODS A total of 839 consecutive whole blood samples sent to the RI Clinical Laboratory for routine tacrolimus monitoring over a 30-day period were used for a method comparison study. Samples were measured the day they arrived at RI, were sent to the NYP laboratory, and were analyzed the same day or stored at 48C for no more than 2 days. Tacrolimus was measured using the Abbott Architect i2000 IA (Abbott Park, IL) at RI and at the NYP laboratory using the Waters TQD Acquity mass spectrometer with the Mass Trak LC-MS kit (Milford, MA) for tacrolimus. The Abbott method has a reportable range of 2–30 ng/mL, limits of quantitation of 0.8–30 ng/mL, and within-run and total imprecision of ,6% and ,7%.7 The Waters method has a reportable range of 0.68–31.7 ng/mL, limits of quantitation of 0.38–63.4 ng/mL, and within-run and total imprecision of ,6% and ,8%, respectively.8 Method comparison was evaluated by Deming linear regression analysis and a Bland–Altman plot. Method concordance was determined across dosage target ranges of 8– 10, 6–8, 4–6, and 2–4 ng/mL (SAS JMP Pro version 10; SAS, Cary, NC and SigmaPlot version 12; Systat Software, Inc., San Jose, CA). These ranges were defined as greater than the lower value and less than or equal to higher value (eg, 2–4, 2 , n # 4). To evaluate the patient sample concordant rate, the Six Sigma Metric benchmarking technique was applied. To implement this benchmark, the error rate was first converted into a defects-per-million (dpm) rate. Then, a standard table of Six Sigma values was consulted to convert the dpm rate into a Six Sigma Metric on the short-term scale.9 A range of values from 3 Sigma (approximately 67,000 dpm) to 6 Sigma (3.4 dpm) is generally expected. The higher the Sigma Metric, the more reliable the process, the lower the operating costs, and the more satisfaction is expected from clinician and patient with the outcome. As the Sigma Metric lowers, more defects and waste occur, more rework is required, and less optimal outcomes are expected because of patient misclassification.
FIGURE 1. Deming regression analysis comparing tacrolimus concentrations measured by LC-MS and IA (y = 0.927x 20.24; 95% confidence interval, 0.903–0.951; R2 = 0.875; n = 839). Perfect agreement is indicated by the dashed line. The solid line is the best fit to a linear Deming regression.
expressed as a percent (coefficient of variation) was not significantly related to tacrolimus level (CV% = 212.6 + 0.18 · tacro; R2 = 0.001; P . 0.31). The effect of bias on concordance was evaluated by counting the number of discordant pairs observed across the
RESULTS Comparison by Deming regression of the Waters Mass Trak LC-MS assay and the Abbott Architect Tacrolimus IA is shown in Figure 1. The average bias was 0.81 ng/mL (211%). A Bland–Altman plot of bias (LC-MS–IA) by increasing tacrolimus concentration is shown in Figure 2. Bias, expressed as the difference between measurements seems to increase with tacrolimus levels. However, bias 2014 Lippincott Williams & Wilkins
FIGURE 2. Bland-Altman plot of bias (LC-MS–IA) by increasing tacrolimus concentration. Samples with identical results lie on the dashed line. The solid line indicates the mean bias (0.81 ng/mL).
707
Ther Drug Monit Volume 36, Number 6, December 2014
Agrawal et al
TABLE 1. Target Therapeutic Ranges for Tacrolimus, Degree of Concordance, and Sigma Value as a Function of Method IA to LC-MS
LC-MS to IA
Count
Count
Target Range for Tacrolimus (ng/mL)
Low
High
% (N/Total) (Sigma)
Low
High
% (N/Total) (Sigma)
,2 2–4 4–6 6–8 8–10 .10 Total
0 6 39 91 76 77 289
2 15 10 14 5 0 46
22 (2/9) (2.2) 24 (21/89) (2.2) 24 (49/208) (2.2) 48 (105/220) (1.5) 63 (81/128) (1.1) 42 (77/185) (1.7) 40 (335/839) (1.7)
0 2 15 10 13 6 46
9 41 93 82 64 0 289
56 (9/16) (1.3) 39 (43/111) (1.7) 40 (108/267) (1.7) 44 (92/207) (1.6) 62 (77/124) (1.2) 5 (6/114) (3.2) 40 (335/839) (1.7)
Data are counts of concordant pairs: patients (paired samples) who would fall into a different therapeutic range if measured first by IA then by LC-MS or the reverse. Concordance is given as the percent and (total concordant/total pairs) in the therapeutic range by the method listed first. Low and high are the counts of concordant pairs that move from a higher to a lower range or the reverse.
boundaries between therapeutically targeted ranges (Table 1 and Fig. 3). There were 310 pairs (37%) discordant by 1, 21 (2.5%) discordant by 2, and 4 (0.5%) discordant by 3 therapeutic ranges. Overall, a surprising 40% of patient
samples were discordant when therapeutic ranges were 2 ng/mL wide. As would be predicted from the bias, more samples were discordant low when going from the IA to LC-MS and more were discordant high in the reverse direction (Fig. 3).
DISCUSSION
FIGURE 3. Total degree of concordance and discordance (low and high) measured by counts and percentage as a function of assay method. Upper panel: Measured first by IA then by LC-MS; lower panel: measured first by LC-MS and then by IA. See Table 1 for counts by therapeutic range.
708
In our previous study, we hypothesized that the bias between different tacrolimus test methods was large enough to cause clinically relevant changes in dosing classifications. We concluded that tacrolimus assay standardization will be necessary (1) to compare patient results between clinical laboratories and (2) to be able to provide optimized drug dosing and consistent care across transplant centers.6 The recent transition of the tacrolimus testing service from RI, running an Abbott Architect IA, to NYP, where the test is run using the Waters TQD Acquity mass spectrometer with the Mass Trak kit LC-MS has provided a real-world test for the need for tacrolimus assay standardization. The relatively modest bias of 0.81 ng/mL between methods caused a 40% discordance rate between samples measured by the 2 methods when the interval between therapeutic dose ranges was 2 ng/mL. Increasing the targeted therapeutic ranges to 4 ng/mL lowers the discordance rate to 3%. It should be noted that 6% (46/839) of the samples shifted discordant in the direction against the bias when using the 2 ng/mL interval. This shift against the bias is probably because of assay imprecision, which suggests that 94% of the discordance observed in our study is likely because of bias and not assay imprecision. Only through assay standardization can assay bias be mitigated and narrower ranges implemented for monitoring allograft recipients. This realworld test of the effect of a lack of standardization of tacrolimus methods shows that differences between assay methods preclude the clinical implementation of narrower therapeutic ranges with multiple assays. It is worth emphasizing the importance of this 40% discordance rate. In the absence of a context or a benchmark, clinicians and laboratories are likely to accept this rate of error without realizing the severity of outcomes being generated. If instead, a widely used scale of process performance is used to 2014 Lippincott Williams & Wilkins
Ther Drug Monit Volume 36, Number 6, December 2014
assess this defect rate, the discordance is more clearly understood. Using Six Sigma Metrics,9 a quality management framework widely accepted in both industry and health care, we can rank the discordance on a benchmark scale of 0–6, where Six Sigma is considered world class performance, and Three Sigma is considered (outside of health care) as the minimum threshold for efficient process operation. A 40% discordance rate, unfortunately, falls far below the minimum Sigma threshold, coming in at only 1.7 Sigma. Nevelainen et al10 studied common laboratory processes more than a decade ago and found that Sigma Metrics ranged between 2.2 (therapeutic drug timing) and 5.4 (specimen acceptability for transport). In other words, according to conventional industrial standards and medical laboratory standards, the tacrolimus discordance rate is wholly unacceptable. Because of the 3-year follow-up results of the ELiTESymphony study, where patients were maintained at tacrolimus blood concentrations of ,10 ng/mL and largely without complications, much attention is being given to develop tacrolimus minimizing strategies.3 Both methods used in this study were capable of measuring tacrolimus in the range between 1 and 10 ng/mL. Additionally, both correlate well to the LGC (Teddington, Middlesex, United Kingdom) candidate reference method and the LC-MS method at the Analytical Unit, St. Georges, University of London.6,11 Even though both methods used in this study correlate well to these test methods, 40% of the sample results were discordant across the targeted therapeutic ranges used at our center. Laboratories changing from IA to LC-MS should expect discordantly lower values, whereas laboratories changing from LC-MS to IA should anticipate the reverse. Laboratories must make clinicians aware of the potential consequences a change in method may have on the classification of patients and the resulting dosing decisions that may be made. Re-baseline of patients when changing tacrolimus test methods is indicated in the absence of assay standardization.
2014 Lippincott Williams & Wilkins
Tacrolimus Assay Method and Patient Classification
CONCLUSIONS The IA and LC-MS methods studied here are capable of measuring tacrolimus in the clinically relevant range between 1 and 10 ng/mL, yet 40% of the samples were discordant with an unacceptable Sigma level. Discordancy arises not from differences in method sensitivity, but from a lack of tacrolimus assay standardization. The effect of a change in methodology in the absence of standardization on patient classification based on tacrolimus blood concentration and the ability of a test method to position the concentration in a given specimen within the correct range directly impacts the proper dosing of transplant patients with immunosuppressive drugs. The quality of life and long-term survival of transplant patients could be greatly improved by standardization of tacrolimus assays. REFERENCES 1. Ekberg H, Tedesco-Silva H, Demirbas A, et al. Reduced exposure to calcineurin inhibitors in renal transplantation. N Engl J Med. 2007;357:2562–2575. 2. Ekberg H, Bernasconi C, Tedesco-Silva H, et al. Calcineurin inhibitor minimization in the symphony study: observational results 3 years after transplantation. Am J Transplant. 2009;9:1876–1885. 3. Wallemacq P, Armstrong VW, Brunet M, et al. Opportunities to optimize tacrolimus therapy in solid organ transplantation: report of the European consensus conference. Ther Drug Monit. 2009;31:139–152. 4. Amann S, Parker TS, Levine DM. Evaluation of 2 immunoassays for monitoring low blood levels of tacrolimus. Ther Drug Monit. 2009;31:273–276. 5. Schiff J, Cole E, Cantarovich M. Therapeutic monitoring of calcineurin inhibitors for the nephrologist. Clin J Am Soc Nephrol. 2007;2:374–384. 6. Levine DM, Maine GT, Armbruster DA, et al. The need for standardization of tacrolimus assays. Clin Chem. 2011;57:1739–1747. 7. Tacrolimus Architect System [Package Insert]. Abbott Park, IL: Abbott Laboratories; 2009. 8. Napoli KL, Hammett-Stabler C, Taylor PJ, et al. Multi-center evaluation of a commercial kit for tacrolimus determination by LC/MS/MS. Clin Biochem. 2010;43:910–920. 9. Westgard JO. Six Sigma Quality Design and Control. 2nd ed. Madison, WI: Westgard QC Inc; 2006. 10. Nevelainen D, Berte L, Kraft C, et al. Evaluating laboratory performance on quality indicators with the six sigma scale. Arch Pathol Lab Med. 2000;124:516–519. 11. Annesley TM, McKeown DA, Holt DW, et al. Standardization of LC-MS for therapeutic drug monitoring of tacrolimus. Clin Chem. 2013;59:1630–1637.
709
Transplant Patient Classification and Tacrolimus Assays: More Evidence of the Need for Assay Standardization Yash Pal Agrawal, MBBS, PhD,* Maria Cid, BS,* Sten Westgard, MS,† Thomas S. Parker, PhD,*‡ Ryan Jaikaran, BS,‡ and Daniel M. Levine, PhD*‡
Background: A global tacrolimus proficiency study recently showed clinically significant variability between laboratories, the inability of a common calibrator to harmonize methods, and differences in patient classification depending on the test method. The authors evaluated (1) the effect of a change in methodology on patient classification based on tacrolimus blood concentration and (2) the ability of 2 methods to position the concentration in a given specimen within the correct range.
Methods: A total of 839 consecutive samples were analyzed at The Rogosin Institute and New York Presbyterian Hospital for routine tacrolimus monitoring over 30 days. Concordance analysis between the methods was performed covering dosage target ranges of 8–10, 6–8, 4–6 ng/mL currently used at our center. Six Sigma Metrics were applied to statistically evaluate the discordance rate. Results: Deming regression comparing liquid chromatography– tandem mass spectrometry and immunoassay yielded y = 0.927x 2 0.24; 95% confidence interval, 0.903–0.951; R2 = 0.875; n = 839. There were 310 pairs (37%) discordant by 1, 21 (2.5%) discordant by 2, and 4 (0.5%) discordant by 3 therapeutic ranges. Surprisingly, 40% of patient samples were discordant when therapeutic ranges were 2 ng/mL wide. This discordant rate is equivalent to 1.7 Sigma and falls far below the minimum acceptable threshold of 3 Sigma.
Conclusions: Both methods are capable of measuring tacrolimus in the clinically relevant range between 1 and 10 ng/mL, yet 40% of the samples were discordant with an unacceptable Sigma level. Standardization of tacrolimus assays will mitigate this issue. Key Words: tacrolimus, LC-MS, immunoassay, therapeutic drug monitoring, assay standardization (Ther Drug Monit 2014;36:706–709)
Received for publication January 31, 2014; accepted April 12, 2014. From the *New York Presbyterian Hospital-Weill Cornell Medical College, New York, NY; †Westgard QC, Inc., Madison, Wisconsin; and ‡The Rogosin Institute, New York, NY. D. M. Levine has received honoraria for lectures and has served on an advisory board for Waters. S. Westgard is a consultant for Abbott through his company Westgard QC. The remaining authors declare no conflict of interest. Supported by The Rogosin Institute and the New York Presbyterian HospitalWeill Cornell Medical College. Correspondence: Daniel M. Levine, PhD, The Rogosin Institute, 310 E. 67th St, New York, NY 10065 (e-mail: [email protected]). Copyright © 2014 by Lippincott Williams & Wilkins
706
INTRODUCTION Tacrolimus is an immunosuppressive agent that is used for prophylaxis against organ and tissue rejection after allogeneic transplantation. The Efficacy Limiting Toxicity Elimination (ELiTE)-Symphony study1,2 and the European Consensus Conference on Tacrolimus3 have recommended that dose of tacrolimus is minimized to prevent rejection of a transplanted organ while reducing long-term nephrotoxicity of the drug. Patients with tacrolimus blood concentrations of ,10 ng/mL in the 3-year ELiTE-Symphony follow-up study had the best kidney graft survival.2 At our center, patients receiving a transplanted kidney are targeted to 8–10 ng/mL for the first 3 months after transplant, 6–8 ng/mL from 3 months to 1 year, and 4–6 ng/mL thereafter. A future goal may be to reduce the tacrolimus whole blood concentration even further, to around 2–4 ng/mL, on a patient by patient basis to minimize nephrotoxic effects and still maintain adequate immunosuppression.4 Tacrolimus blood levels of 3–7 ng/mL in renal transplant patients from the early posttransplantation period forward were associated with the lowest risk for acute rejection and the highest glomerular filtration rate at 12 months.5 The Consensus Conference recommends a limit of quantitation of 1 ng/mL to ensure that tacrolimus test methods are both accurate and precise to measure these lower concentrations of tacrolimus. Recently, a global tacrolimus proficiency study assessed the comparability of liquid chromatography–tandem mass spectrometry (LC-MS) and immunoassay measurements and demonstrated the need for assay standardization.6 Exactmatching isotope dilution mass spectrometry served as the candidate reference method. The study showed clinically significant variability between laboratories and that a common calibrator did not harmonize methods. The study also demonstrated that patients would be classified differently for dosing by highlighting the proficiency results from 2 laboratories at the extreme low and high range of the study. Selection of the best method or the method producing the “true” tacrolimus blood concentration cannot be determined in the absence of assay standardization. Given that current guidelines recommend tacrolimus blood concentrations of ,10 ng/mL for patients receiving solid organ transplantation, it is essential to understand the analytical limits of methods used to monitor levels between 1 and 10 ng/mL. In the absence of standardization, the lack of comparability between methods may result in a change in dose classification, which may in turn, lead to under dosing a patient (allograft rejection) or providing an unnecessarily high dose that may result in nephrotoxicity. Ther Drug Monit Volume 36, Number 6, December 2014
Ther Drug Monit Volume 36, Number 6, December 2014
Tacrolimus Assay Method and Patient Classification
The Rogosin Institute (RI) is located at the Cornell campus of the New York Presbyterian Hospital (NYP) in New York City and co-administers the kidney transplant program with the Hospital. The tacrolimus testing service recently moved from RI, running an Abbott Architect immunoassay (IA), to the NYP, where the test is run using the Waters TQD Acquity mass spectrometer with Mass Trak kit (LC-MS). This change in method provides an opportunity to test the predictions of the proficiency study under conditions of standard clinical practice.6
MATERIALS AND METHODS A total of 839 consecutive whole blood samples sent to the RI Clinical Laboratory for routine tacrolimus monitoring over a 30-day period were used for a method comparison study. Samples were measured the day they arrived at RI, were sent to the NYP laboratory, and were analyzed the same day or stored at 48C for no more than 2 days. Tacrolimus was measured using the Abbott Architect i2000 IA (Abbott Park, IL) at RI and at the NYP laboratory using the Waters TQD Acquity mass spectrometer with the Mass Trak LC-MS kit (Milford, MA) for tacrolimus. The Abbott method has a reportable range of 2–30 ng/mL, limits of quantitation of 0.8–30 ng/mL, and within-run and total imprecision of ,6% and ,7%.7 The Waters method has a reportable range of 0.68–31.7 ng/mL, limits of quantitation of 0.38–63.4 ng/mL, and within-run and total imprecision of ,6% and ,8%, respectively.8 Method comparison was evaluated by Deming linear regression analysis and a Bland–Altman plot. Method concordance was determined across dosage target ranges of 8– 10, 6–8, 4–6, and 2–4 ng/mL (SAS JMP Pro version 10; SAS, Cary, NC and SigmaPlot version 12; Systat Software, Inc., San Jose, CA). These ranges were defined as greater than the lower value and less than or equal to higher value (eg, 2–4, 2 , n # 4). To evaluate the patient sample concordant rate, the Six Sigma Metric benchmarking technique was applied. To implement this benchmark, the error rate was first converted into a defects-per-million (dpm) rate. Then, a standard table of Six Sigma values was consulted to convert the dpm rate into a Six Sigma Metric on the short-term scale.9 A range of values from 3 Sigma (approximately 67,000 dpm) to 6 Sigma (3.4 dpm) is generally expected. The higher the Sigma Metric, the more reliable the process, the lower the operating costs, and the more satisfaction is expected from clinician and patient with the outcome. As the Sigma Metric lowers, more defects and waste occur, more rework is required, and less optimal outcomes are expected because of patient misclassification.
FIGURE 1. Deming regression analysis comparing tacrolimus concentrations measured by LC-MS and IA (y = 0.927x 20.24; 95% confidence interval, 0.903–0.951; R2 = 0.875; n = 839). Perfect agreement is indicated by the dashed line. The solid line is the best fit to a linear Deming regression.
expressed as a percent (coefficient of variation) was not significantly related to tacrolimus level (CV% = 212.6 + 0.18 · tacro; R2 = 0.001; P . 0.31). The effect of bias on concordance was evaluated by counting the number of discordant pairs observed across the
RESULTS Comparison by Deming regression of the Waters Mass Trak LC-MS assay and the Abbott Architect Tacrolimus IA is shown in Figure 1. The average bias was 0.81 ng/mL (211%). A Bland–Altman plot of bias (LC-MS–IA) by increasing tacrolimus concentration is shown in Figure 2. Bias, expressed as the difference between measurements seems to increase with tacrolimus levels. However, bias 2014 Lippincott Williams & Wilkins
FIGURE 2. Bland-Altman plot of bias (LC-MS–IA) by increasing tacrolimus concentration. Samples with identical results lie on the dashed line. The solid line indicates the mean bias (0.81 ng/mL).
707
Ther Drug Monit Volume 36, Number 6, December 2014
Agrawal et al
TABLE 1. Target Therapeutic Ranges for Tacrolimus, Degree of Concordance, and Sigma Value as a Function of Method IA to LC-MS
LC-MS to IA
Count
Count
Target Range for Tacrolimus (ng/mL)
Low
High
% (N/Total) (Sigma)
Low
High
% (N/Total) (Sigma)
,2 2–4 4–6 6–8 8–10 .10 Total
0 6 39 91 76 77 289
2 15 10 14 5 0 46
22 (2/9) (2.2) 24 (21/89) (2.2) 24 (49/208) (2.2) 48 (105/220) (1.5) 63 (81/128) (1.1) 42 (77/185) (1.7) 40 (335/839) (1.7)
0 2 15 10 13 6 46
9 41 93 82 64 0 289
56 (9/16) (1.3) 39 (43/111) (1.7) 40 (108/267) (1.7) 44 (92/207) (1.6) 62 (77/124) (1.2) 5 (6/114) (3.2) 40 (335/839) (1.7)
Data are counts of concordant pairs: patients (paired samples) who would fall into a different therapeutic range if measured first by IA then by LC-MS or the reverse. Concordance is given as the percent and (total concordant/total pairs) in the therapeutic range by the method listed first. Low and high are the counts of concordant pairs that move from a higher to a lower range or the reverse.
boundaries between therapeutically targeted ranges (Table 1 and Fig. 3). There were 310 pairs (37%) discordant by 1, 21 (2.5%) discordant by 2, and 4 (0.5%) discordant by 3 therapeutic ranges. Overall, a surprising 40% of patient
samples were discordant when therapeutic ranges were 2 ng/mL wide. As would be predicted from the bias, more samples were discordant low when going from the IA to LC-MS and more were discordant high in the reverse direction (Fig. 3).
DISCUSSION
FIGURE 3. Total degree of concordance and discordance (low and high) measured by counts and percentage as a function of assay method. Upper panel: Measured first by IA then by LC-MS; lower panel: measured first by LC-MS and then by IA. See Table 1 for counts by therapeutic range.
708
In our previous study, we hypothesized that the bias between different tacrolimus test methods was large enough to cause clinically relevant changes in dosing classifications. We concluded that tacrolimus assay standardization will be necessary (1) to compare patient results between clinical laboratories and (2) to be able to provide optimized drug dosing and consistent care across transplant centers.6 The recent transition of the tacrolimus testing service from RI, running an Abbott Architect IA, to NYP, where the test is run using the Waters TQD Acquity mass spectrometer with the Mass Trak kit LC-MS has provided a real-world test for the need for tacrolimus assay standardization. The relatively modest bias of 0.81 ng/mL between methods caused a 40% discordance rate between samples measured by the 2 methods when the interval between therapeutic dose ranges was 2 ng/mL. Increasing the targeted therapeutic ranges to 4 ng/mL lowers the discordance rate to 3%. It should be noted that 6% (46/839) of the samples shifted discordant in the direction against the bias when using the 2 ng/mL interval. This shift against the bias is probably because of assay imprecision, which suggests that 94% of the discordance observed in our study is likely because of bias and not assay imprecision. Only through assay standardization can assay bias be mitigated and narrower ranges implemented for monitoring allograft recipients. This realworld test of the effect of a lack of standardization of tacrolimus methods shows that differences between assay methods preclude the clinical implementation of narrower therapeutic ranges with multiple assays. It is worth emphasizing the importance of this 40% discordance rate. In the absence of a context or a benchmark, clinicians and laboratories are likely to accept this rate of error without realizing the severity of outcomes being generated. If instead, a widely used scale of process performance is used to 2014 Lippincott Williams & Wilkins
Ther Drug Monit Volume 36, Number 6, December 2014
assess this defect rate, the discordance is more clearly understood. Using Six Sigma Metrics,9 a quality management framework widely accepted in both industry and health care, we can rank the discordance on a benchmark scale of 0–6, where Six Sigma is considered world class performance, and Three Sigma is considered (outside of health care) as the minimum threshold for efficient process operation. A 40% discordance rate, unfortunately, falls far below the minimum Sigma threshold, coming in at only 1.7 Sigma. Nevelainen et al10 studied common laboratory processes more than a decade ago and found that Sigma Metrics ranged between 2.2 (therapeutic drug timing) and 5.4 (specimen acceptability for transport). In other words, according to conventional industrial standards and medical laboratory standards, the tacrolimus discordance rate is wholly unacceptable. Because of the 3-year follow-up results of the ELiTESymphony study, where patients were maintained at tacrolimus blood concentrations of ,10 ng/mL and largely without complications, much attention is being given to develop tacrolimus minimizing strategies.3 Both methods used in this study were capable of measuring tacrolimus in the range between 1 and 10 ng/mL. Additionally, both correlate well to the LGC (Teddington, Middlesex, United Kingdom) candidate reference method and the LC-MS method at the Analytical Unit, St. Georges, University of London.6,11 Even though both methods used in this study correlate well to these test methods, 40% of the sample results were discordant across the targeted therapeutic ranges used at our center. Laboratories changing from IA to LC-MS should expect discordantly lower values, whereas laboratories changing from LC-MS to IA should anticipate the reverse. Laboratories must make clinicians aware of the potential consequences a change in method may have on the classification of patients and the resulting dosing decisions that may be made. Re-baseline of patients when changing tacrolimus test methods is indicated in the absence of assay standardization.
2014 Lippincott Williams & Wilkins
Tacrolimus Assay Method and Patient Classification
CONCLUSIONS The IA and LC-MS methods studied here are capable of measuring tacrolimus in the clinically relevant range between 1 and 10 ng/mL, yet 40% of the samples were discordant with an unacceptable Sigma level. Discordancy arises not from differences in method sensitivity, but from a lack of tacrolimus assay standardization. The effect of a change in methodology in the absence of standardization on patient classification based on tacrolimus blood concentration and the ability of a test method to position the concentration in a given specimen within the correct range directly impacts the proper dosing of transplant patients with immunosuppressive drugs. The quality of life and long-term survival of transplant patients could be greatly improved by standardization of tacrolimus assays. REFERENCES 1. Ekberg H, Tedesco-Silva H, Demirbas A, et al. Reduced exposure to calcineurin inhibitors in renal transplantation. N Engl J Med. 2007;357:2562–2575. 2. Ekberg H, Bernasconi C, Tedesco-Silva H, et al. Calcineurin inhibitor minimization in the symphony study: observational results 3 years after transplantation. Am J Transplant. 2009;9:1876–1885. 3. Wallemacq P, Armstrong VW, Brunet M, et al. Opportunities to optimize tacrolimus therapy in solid organ transplantation: report of the European consensus conference. Ther Drug Monit. 2009;31:139–152. 4. Amann S, Parker TS, Levine DM. Evaluation of 2 immunoassays for monitoring low blood levels of tacrolimus. Ther Drug Monit. 2009;31:273–276. 5. Schiff J, Cole E, Cantarovich M. Therapeutic monitoring of calcineurin inhibitors for the nephrologist. Clin J Am Soc Nephrol. 2007;2:374–384. 6. Levine DM, Maine GT, Armbruster DA, et al. The need for standardization of tacrolimus assays. Clin Chem. 2011;57:1739–1747. 7. Tacrolimus Architect System [Package Insert]. Abbott Park, IL: Abbott Laboratories; 2009. 8. Napoli KL, Hammett-Stabler C, Taylor PJ, et al. Multi-center evaluation of a commercial kit for tacrolimus determination by LC/MS/MS. Clin Biochem. 2010;43:910–920. 9. Westgard JO. Six Sigma Quality Design and Control. 2nd ed. Madison, WI: Westgard QC Inc; 2006. 10. Nevelainen D, Berte L, Kraft C, et al. Evaluating laboratory performance on quality indicators with the six sigma scale. Arch Pathol Lab Med. 2000;124:516–519. 11. Annesley TM, McKeown DA, Holt DW, et al. Standardization of LC-MS for therapeutic drug monitoring of tacrolimus. Clin Chem. 2013;59:1630–1637.
709
