Scandinavian Journal of Clinical and Laboratory Investigation
ISSN: 0036-5513 (Print) 1502-7686 (Online) Journal homepage: http://www.tandfonline.com/loi/iclb20
Confirmation of analytical performance characteristics required for the reference change value applied in patient monitoring Per Hyltoft Petersen, Callum G. Fraser, Flemming Lund & György Sölétormos To cite this article: Per Hyltoft Petersen, Callum G. Fraser, Flemming Lund & György Sölétormos (2015) Confirmation of analytical performance characteristics required for the reference change value applied in patient monitoring, Scandinavian Journal of Clinical and Laboratory Investigation, 75:7, 628-630 To link to this article: http://dx.doi.org/10.3109/00365513.2015.1057897
Published online: 13 Aug 2015.
Submit your article to this journal
Article views: 81
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=iclb20 Download by: [University of California, San Diego]
Date: 15 December 2015, At: 23:32
Scandinavian Journal of Clinical & Laboratory Investigation, 2015; 75: 628–630
LETTER TO THE EDITOR
Confirmation of analytical performance characteristics required for the reference change value applied in patient monitoring
Downloaded by [University of California, San Diego] at 23:32 15 December 2015
PER HYLTOFT PETERSEN1,2, CALLUM G. FRASER3, FLEMMING LUND1 & GYÖRGY SÖLÉTORMOS1 1Department
of Clinical Biochemistry, North Zealand Hospital, University of Copenhagen, Hillerød, Denmark, Quality Improvement of Primary Care Laboratories (NOKLUS), Section for General Practice, University of Bergen, Bergen, Norway, and 3Centre for Research into Cancer Prevention and Screening, University of Dundee, Ninewells Hospital and Medical School, Dundee, Scotland, UK 2Norwegian
SIR: Although the results of examinations in laboratory medicine are used for many purposes, monitoring of acute and chronic disease is the major application. In monitoring, serial results must be considered and objective interpretation of differences facilitates patient care. In 1991, a proposal for analytical goals (also known as analytical quality specifications or performance specifications) for combining analytical bias and imprecision for examinations used in monitoring of patients was published, using HbA1c as an example [1]. It was based on the assessment of the difference between two consecutive measurements for significance at p 0.05 according to the model first published by Harris and Yasaka [2] and now termed the ‘Reference Change Value’, RCV 21/2*1.96*{CVA2 CVI2}1/2, where CVA is the analytical imprecision and CVI is the withinsubject biological variation. The basic specification for allowable performance is that imprecision should be less than half of the within-subject biological variation (CVA 0.5*CVI) as originally proposed by Cotlove et al. [3] and widely accepted since then [4]. However, no proposal for acceptable systematic deviation between the two measurements in the form of delta bias, ΔB, had been made to that time. In consequence, it was suggested that ΔB should be such that the combined effects of ΔB and CVA should not increase the RCV by more than the maximum allowable CVA 0.5*CVI alone. Further calculation demonstrated that ΔB should be 0.33*CVI when CVA was considered negligible. In a further more general study, specifications for examinations on two different instruments were investigated [4] and it was
restated that, if the examinations have negligible imprecision, then the maximum allowable bias between two methods used for monitoring is 0.33*CVI. Although, it is now an accepted fundamental principle of metrology that known bias should be eliminated, changes in bias due, for example, to changes of lots of reagents and calibrators do occur and are difficult and time consuming to correct: thus, an objective quality specification for ΔBias must be available to ensure smooth ongoing laboratory operation and timely delivery of test results. Recently, Åsberg et al. proposed that the allowable ΔB should be larger under the same conditions; however, this concept was elaborated using a different model with formulae for different frequencies of examination on two instruments [5]. Their model is a type of addition of variance model, as also Harris applied for single-point diagnostic testing in relation to reference intervals [6], but his application was based on use of standard deviations, s, as applied by Åsberg et al. [5]. Harris’ model included betweensubject biological variation, sG, so the formula was sA2 B2 0.25*(sI2 sG2), which makes bias, B, a variable and not a constant although it is, by definition (measurement bias: estimate of a systematic measurement error (point 2.18 in BIPM (VIM 2008)) and in laboratory medicine in practice. Consequently, a variance model is not useful for estimating a bias or ΔB. Åsberg et al. [5] estimate an allowable difference between two instruments of 1.0*sI and even larger for a special combination of sA values for the two instruments. If the addition of variance model of
Correspondence: Per Hyltoft Petersen, Norwegian quality improvement of primary care laboratories, Division for General Practice, University of Bergen Box 6165, 5892 Bergen, Norway. Tel: 45 6596 2565. E-mail: [email protected] (Received 20 February 2015 ; accepted 31 May 2015) ISSN 0036-5513 print/ISSN 1502-7686 online © 2015 Informa Healthcare DOI: 10.3109/00365513.2015.1057897
Downloaded by [University of California, San Diego] at 23:32 15 December 2015
Letter to the Editor Harris [6] is applied, it becomes sA2 ΔB2 0.25*sI2, and, in consequence, the maximum ΔB can never exceed 0.5*sI, even when sA 0.0. Thus, with the variance model, the advocated ΔB of 1.0*sI cannot be correct. Amongst the reasons for this error is the use of the variance model and/or the postulate in beginning of their calculations that ‘…. the variance of the results would simply be (ds/2)2, where ds is the systematic difference between the two instruments …’. If it is wished to calculate the variance on the constant ds, then the formula should be: spooled2 (Σds2)/(2*n), where n is the number of differences, which is equal to (n*ds2)/(2*n) ds2/2, indicating that the conclusions reached by Åsberg et al. are incorrect. The optimal way to estimate the ΔB is to find the linear distance between RCV calculated between the error free value: RCVEF 1.96*21/2*sI 2.77*sI, and with the maximal allowable imprecision RCVE 1.96*21/2*(sI2 (0.5*sI)2)1/2 3.10*sI, so allowable maximal distance |ΔBMAX| when sA 0.0 must be: |ΔBMAX| 3.10*sI 2.77*sI 0.33*sI.
(1).
Thus, the general formula for ΔB when sA is between 0.0 and 0.5*sI, as examinations should be: |ΔB| 3.10*sI 2.77*(sI2 sA2)1/2
(2),
which is appropriate for sA between 0.0 and 0.5*sI, and |ΔBMAX| cannot exceed 0.33*sI. This applies even when two instruments are used, because each of them must attain the specification of sA 0.5*sI: in consequence, the formulae derived in publications from 1991 [1] and 1992 [4] are still correct. The trueness of the formula can further be documented by computer simulation, in which we performed 100,000 simulations of RCV for varying combinations of ΔB and sA, both within the condi-
629
Figure 2. Three situations of sA2 sA1 (green – as in Figure 1), sA2 0.0 (brown) and sA2 0.5*sI (blue) with ΔB/sI as function of sA/sI. The percentages of measurements above RCV 3.10*sI are documented for the sA2-values 0.0, 0.25 and 0.50*sI.
tions according to the specifications, and for the variance model according to Harris [2]. The results are shown in Figure 1, in which, in both curves, the percentage of values measured above 3.10*sI are plotted for sA 0.0, 0.25 and 0.50*sI. The percentages for our model are all accurate with values indistinguishable from 2.500 %, whereas the variance model has values up to approximately 5% for sA 0.0 and only when ΔB 0.0 is the percentage accurate and equal to 2.5%. For two instruments with different imprecisions, sA1 and sA2 (Figure 2), the formula becomes: |ΔB| 3.10*sI 1.96*(2*sI2 sA12 sA22)1/2 (3) and, both sA1 and sA2 must be below 0.5*sI. If one of them has sA 0.0, the |ΔB| lies between 0.16 and 0.33*sI, and, if sA1 is constantly equal to 0.5*sI the |ΔB| lies between 0.0 and 0.16*sI. For the three curves for RCV 3.10*sI with sA2 sA1, sA2 0.0 and sA2 0.5*sI in Figure 2, the curves for ΔB/sI as function of sA/sI are different, but all have ΔB/sI 0.33 and with the percentage equal to 2.5 %, so that |ΔBMAX| 0.33*sI. Therefore, the quality specifications for combination of ΔB and sA for RCV [1,4] are still correct and useful for RCV with z 1.96. If other probabilities for RCV with different z-values are to be applied, however, then other values for |ΔBMAX| must be used, as shown in Table I. The |ΔBMAX| increases with increasing |RCVMAX| for decreasing probabilities. Table I. Relations between elements of reference change values for varying probabilities.
Figure 1. Percentage of results exceeding RCV 3.10*sI with ΔB/sI as function of sA/sI for the two models: Our correct model (green) and the variance model (purple). The percentages of measurements above RCV 3.10*sI are documented for sA-values of 0.0, 0.25 and 0.50*sI.
Probability
z-value
|RCVMAX|
|ΔBMAX|
0.100 0.050 0.020 0.010 0.005 0.001
1.645 1.960 2.326 2.576 2.807 3.291
2.601 3.099 3.678 4.073 4.438 5.203
0.275 0.327 0.388 0.430 0.469 0.549
630
P. H. Petersen et al.
In conclusion, the appropriate model that is ubiquitously applicable throughout laboratory medicine, is that allowable difference in bias between two examination systems is less than 0.33 CVI. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. References
Downloaded by [University of California, San Diego] at 23:32 15 December 2015
[1] Lytken Larsen M, Fraser CG, Hyltoft Petersen P. A comparison of analytical goals for haemoglobin A1c assays derived using different strategies. Ann Clin Biochem 1991;28:272–8.
[2] Harris EG, Yasaka T. On the calculation of a “reference change” for comparing two consecutive measurements. Clin Chem 1983;29:25–30. [3] Cotlove E, Harris EG, Williams GZ. Biological and analytic components of variation in long-term studies of serum constituents in normal subjects. Clin Chem 1970;16: 1028–32. [4] Hyltoft Petersen P, Fraser CG, Westgard JO, Lytken Larsen M. Analytical goal-setting for monitoring patients when two analytical methods are used. Clin Chem 1992;38: 2256–60. [5] Åsberg A, Solem KB, Mikkelsen G. Allowable systematic difference between two instruments measuring the same analyte. Scand J Clin Lab Invest 2014;74:588–90. [6] Harris EK. Proposed goals for analytical precision and accuracy in single-point diagnostic testing. Arch Pathol Lab Med 1988;112:416–20.
ISSN: 0036-5513 (Print) 1502-7686 (Online) Journal homepage: http://www.tandfonline.com/loi/iclb20
Confirmation of analytical performance characteristics required for the reference change value applied in patient monitoring Per Hyltoft Petersen, Callum G. Fraser, Flemming Lund & György Sölétormos To cite this article: Per Hyltoft Petersen, Callum G. Fraser, Flemming Lund & György Sölétormos (2015) Confirmation of analytical performance characteristics required for the reference change value applied in patient monitoring, Scandinavian Journal of Clinical and Laboratory Investigation, 75:7, 628-630 To link to this article: http://dx.doi.org/10.3109/00365513.2015.1057897
Published online: 13 Aug 2015.
Submit your article to this journal
Article views: 81
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=iclb20 Download by: [University of California, San Diego]
Date: 15 December 2015, At: 23:32
Scandinavian Journal of Clinical & Laboratory Investigation, 2015; 75: 628–630
LETTER TO THE EDITOR
Confirmation of analytical performance characteristics required for the reference change value applied in patient monitoring
Downloaded by [University of California, San Diego] at 23:32 15 December 2015
PER HYLTOFT PETERSEN1,2, CALLUM G. FRASER3, FLEMMING LUND1 & GYÖRGY SÖLÉTORMOS1 1Department
of Clinical Biochemistry, North Zealand Hospital, University of Copenhagen, Hillerød, Denmark, Quality Improvement of Primary Care Laboratories (NOKLUS), Section for General Practice, University of Bergen, Bergen, Norway, and 3Centre for Research into Cancer Prevention and Screening, University of Dundee, Ninewells Hospital and Medical School, Dundee, Scotland, UK 2Norwegian
SIR: Although the results of examinations in laboratory medicine are used for many purposes, monitoring of acute and chronic disease is the major application. In monitoring, serial results must be considered and objective interpretation of differences facilitates patient care. In 1991, a proposal for analytical goals (also known as analytical quality specifications or performance specifications) for combining analytical bias and imprecision for examinations used in monitoring of patients was published, using HbA1c as an example [1]. It was based on the assessment of the difference between two consecutive measurements for significance at p 0.05 according to the model first published by Harris and Yasaka [2] and now termed the ‘Reference Change Value’, RCV 21/2*1.96*{CVA2 CVI2}1/2, where CVA is the analytical imprecision and CVI is the withinsubject biological variation. The basic specification for allowable performance is that imprecision should be less than half of the within-subject biological variation (CVA 0.5*CVI) as originally proposed by Cotlove et al. [3] and widely accepted since then [4]. However, no proposal for acceptable systematic deviation between the two measurements in the form of delta bias, ΔB, had been made to that time. In consequence, it was suggested that ΔB should be such that the combined effects of ΔB and CVA should not increase the RCV by more than the maximum allowable CVA 0.5*CVI alone. Further calculation demonstrated that ΔB should be 0.33*CVI when CVA was considered negligible. In a further more general study, specifications for examinations on two different instruments were investigated [4] and it was
restated that, if the examinations have negligible imprecision, then the maximum allowable bias between two methods used for monitoring is 0.33*CVI. Although, it is now an accepted fundamental principle of metrology that known bias should be eliminated, changes in bias due, for example, to changes of lots of reagents and calibrators do occur and are difficult and time consuming to correct: thus, an objective quality specification for ΔBias must be available to ensure smooth ongoing laboratory operation and timely delivery of test results. Recently, Åsberg et al. proposed that the allowable ΔB should be larger under the same conditions; however, this concept was elaborated using a different model with formulae for different frequencies of examination on two instruments [5]. Their model is a type of addition of variance model, as also Harris applied for single-point diagnostic testing in relation to reference intervals [6], but his application was based on use of standard deviations, s, as applied by Åsberg et al. [5]. Harris’ model included betweensubject biological variation, sG, so the formula was sA2 B2 0.25*(sI2 sG2), which makes bias, B, a variable and not a constant although it is, by definition (measurement bias: estimate of a systematic measurement error (point 2.18 in BIPM (VIM 2008)) and in laboratory medicine in practice. Consequently, a variance model is not useful for estimating a bias or ΔB. Åsberg et al. [5] estimate an allowable difference between two instruments of 1.0*sI and even larger for a special combination of sA values for the two instruments. If the addition of variance model of
Correspondence: Per Hyltoft Petersen, Norwegian quality improvement of primary care laboratories, Division for General Practice, University of Bergen Box 6165, 5892 Bergen, Norway. Tel: 45 6596 2565. E-mail: [email protected] (Received 20 February 2015 ; accepted 31 May 2015) ISSN 0036-5513 print/ISSN 1502-7686 online © 2015 Informa Healthcare DOI: 10.3109/00365513.2015.1057897
Downloaded by [University of California, San Diego] at 23:32 15 December 2015
Letter to the Editor Harris [6] is applied, it becomes sA2 ΔB2 0.25*sI2, and, in consequence, the maximum ΔB can never exceed 0.5*sI, even when sA 0.0. Thus, with the variance model, the advocated ΔB of 1.0*sI cannot be correct. Amongst the reasons for this error is the use of the variance model and/or the postulate in beginning of their calculations that ‘…. the variance of the results would simply be (ds/2)2, where ds is the systematic difference between the two instruments …’. If it is wished to calculate the variance on the constant ds, then the formula should be: spooled2 (Σds2)/(2*n), where n is the number of differences, which is equal to (n*ds2)/(2*n) ds2/2, indicating that the conclusions reached by Åsberg et al. are incorrect. The optimal way to estimate the ΔB is to find the linear distance between RCV calculated between the error free value: RCVEF 1.96*21/2*sI 2.77*sI, and with the maximal allowable imprecision RCVE 1.96*21/2*(sI2 (0.5*sI)2)1/2 3.10*sI, so allowable maximal distance |ΔBMAX| when sA 0.0 must be: |ΔBMAX| 3.10*sI 2.77*sI 0.33*sI.
(1).
Thus, the general formula for ΔB when sA is between 0.0 and 0.5*sI, as examinations should be: |ΔB| 3.10*sI 2.77*(sI2 sA2)1/2
(2),
which is appropriate for sA between 0.0 and 0.5*sI, and |ΔBMAX| cannot exceed 0.33*sI. This applies even when two instruments are used, because each of them must attain the specification of sA 0.5*sI: in consequence, the formulae derived in publications from 1991 [1] and 1992 [4] are still correct. The trueness of the formula can further be documented by computer simulation, in which we performed 100,000 simulations of RCV for varying combinations of ΔB and sA, both within the condi-
629
Figure 2. Three situations of sA2 sA1 (green – as in Figure 1), sA2 0.0 (brown) and sA2 0.5*sI (blue) with ΔB/sI as function of sA/sI. The percentages of measurements above RCV 3.10*sI are documented for the sA2-values 0.0, 0.25 and 0.50*sI.
tions according to the specifications, and for the variance model according to Harris [2]. The results are shown in Figure 1, in which, in both curves, the percentage of values measured above 3.10*sI are plotted for sA 0.0, 0.25 and 0.50*sI. The percentages for our model are all accurate with values indistinguishable from 2.500 %, whereas the variance model has values up to approximately 5% for sA 0.0 and only when ΔB 0.0 is the percentage accurate and equal to 2.5%. For two instruments with different imprecisions, sA1 and sA2 (Figure 2), the formula becomes: |ΔB| 3.10*sI 1.96*(2*sI2 sA12 sA22)1/2 (3) and, both sA1 and sA2 must be below 0.5*sI. If one of them has sA 0.0, the |ΔB| lies between 0.16 and 0.33*sI, and, if sA1 is constantly equal to 0.5*sI the |ΔB| lies between 0.0 and 0.16*sI. For the three curves for RCV 3.10*sI with sA2 sA1, sA2 0.0 and sA2 0.5*sI in Figure 2, the curves for ΔB/sI as function of sA/sI are different, but all have ΔB/sI 0.33 and with the percentage equal to 2.5 %, so that |ΔBMAX| 0.33*sI. Therefore, the quality specifications for combination of ΔB and sA for RCV [1,4] are still correct and useful for RCV with z 1.96. If other probabilities for RCV with different z-values are to be applied, however, then other values for |ΔBMAX| must be used, as shown in Table I. The |ΔBMAX| increases with increasing |RCVMAX| for decreasing probabilities. Table I. Relations between elements of reference change values for varying probabilities.
Figure 1. Percentage of results exceeding RCV 3.10*sI with ΔB/sI as function of sA/sI for the two models: Our correct model (green) and the variance model (purple). The percentages of measurements above RCV 3.10*sI are documented for sA-values of 0.0, 0.25 and 0.50*sI.
Probability
z-value
|RCVMAX|
|ΔBMAX|
0.100 0.050 0.020 0.010 0.005 0.001
1.645 1.960 2.326 2.576 2.807 3.291
2.601 3.099 3.678 4.073 4.438 5.203
0.275 0.327 0.388 0.430 0.469 0.549
630
P. H. Petersen et al.
In conclusion, the appropriate model that is ubiquitously applicable throughout laboratory medicine, is that allowable difference in bias between two examination systems is less than 0.33 CVI. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. References
Downloaded by [University of California, San Diego] at 23:32 15 December 2015
[1] Lytken Larsen M, Fraser CG, Hyltoft Petersen P. A comparison of analytical goals for haemoglobin A1c assays derived using different strategies. Ann Clin Biochem 1991;28:272–8.
[2] Harris EG, Yasaka T. On the calculation of a “reference change” for comparing two consecutive measurements. Clin Chem 1983;29:25–30. [3] Cotlove E, Harris EG, Williams GZ. Biological and analytic components of variation in long-term studies of serum constituents in normal subjects. Clin Chem 1970;16: 1028–32. [4] Hyltoft Petersen P, Fraser CG, Westgard JO, Lytken Larsen M. Analytical goal-setting for monitoring patients when two analytical methods are used. Clin Chem 1992;38: 2256–60. [5] Åsberg A, Solem KB, Mikkelsen G. Allowable systematic difference between two instruments measuring the same analyte. Scand J Clin Lab Invest 2014;74:588–90. [6] Harris EK. Proposed goals for analytical precision and accuracy in single-point diagnostic testing. Arch Pathol Lab Med 1988;112:416–20.
