Clin Chem Lab Med 2014; 52(12): e295–e296
Letter to the Editor Siamak Sabour*
Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults; statistical issues in validity and reliability analysis Keywords: reliability; statistical issues; validity. DOI 10.1515/cclm-2014-0453 Received April 28, 2014; accepted May 19, 2014; previously published online June 14, 2014
To the Editor,
I was interested to read the paper by Maldonado-Hernández and colleagues published in the May 2014 issue of Clinical Chemistry and Laboratory Medicine [1]. The authors aimed to evaluate the validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia (HHcy). As the authors mentioned, they performed a methionine loading test (MLT) as gold standard to characterize the HHcy status of 75 healthy adult subjects. For the breath test protocol, they selected a subsample and assigned to the control group 17 subjects with fasting and post-loading homocysteine (Hcy) ≤ 12 and 42.3 μmol/L after loading. Selected subjects were requested to have a second visit to perform a breath test within 1–4 weeks following the MLT test and received an oral dose of 2.5 mg/kg of 1-13C-methionine dissolved in water. Breath samples were collected at basal, 20, 40, 60, 80, 100 and 120 min (test 1). The same procedure was repeated within 1 week (test 2). The coefficient of variation *Corresponding author: Siamak Sabour, MD, MSc, DSc, PhD, Postdoc, Safety Promotion and Injury Prevention Research Centre, Department of Clinical Epidemiology, Shahid Beheshti University of Medical Sciences, Tehran, I. R. Iran, Phone: +98 21 22421814, E-mail: [email protected]
(CV) between breath test 1 and breath test 2 was 9.0 ± 5.4% [1]. Reliability (repeatability or reproducibility) is being assessed by different statistical tests. Briefly, for quantitative variable intra class correlation coefficient (ICCC) and for qualitative variables weighted κ should be used [1]. Regarding reliability or agreement, it is crucial to know that an individual-based approach instead of groupbased should be considered [2, 3]. Reliability assessment is the reason why we should consider individual results and not global average. Therefore, ICCC single measure instead of average measure should be reported to correctly assess reliability. In other words, the possibility of obtaining exactly the same average of a variable between two methods with no reliability at all is high. The same reasoning is also true for CV [2, 3]. These authors also reported that the 13C-methionine breath test has a sensitivity of 81.3% and specificity of 64.7% against the MLT [1]. Sensitivity (percentage with the disease tested positive, True Positives/[True Positives+False Negative]), specificity (percentage healthy tested negative, True Negatives/[True Negatives+False Positive]), positive predictive value (PPV), (percentage of positive tests in subjects who actually are diseased, True Positives/[True Positives+False Positive]), negative predictive value (NPV) (percentage of negative tests in subjects who are healthy, True Negatives/[True Negatives+False Negative]), likelihood ratio positive and likelihood ratio negative as well as diagnostic accuracy [(both true positive and true negative results/total) × 100] and odds ratio (true results/false results) preferably more than 50, are among the tests to evaluate the validity of a single test compared to a gold standard [2–7]. Accuracy (validity) is a completely different concept with precision (reliability) and should not be confused with each other [2, 3]. As the authors pointed out in their conclusions, the 13 C-methionine breath test is a valid and reliable method
e296 Sabour: Validity and reliability analysis for identifying subjects with moderate HHcy. Such a conclusion can be a misleading message due to inappropriate use of statistical tests to assess reliability and also incomplete evaluation of validity. Conflict of interest statement Authors’ conflict of interest disclosure: The authors stated that there are no conflicts of interest regarding the publication of this article. Research funding: None declared. Employment or leadership: None declared. Honorarium: None declared.
References 1. Maldonado-Hernández J, Prina-Ojeda LV, Montalvo-Velarde I, Del Prado-Manríquez M, de Lourdes Barbosa-Cortés M, Repetto-
Madrid M. Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults. Clin Chem Lab Med 2014;52:687–92. 2. Rothman KJ, Greenland S, Lash TL. Modern epidemiology, 4th ed. Baltimore, US: Lippincott Williams & Wilkins, 2010. 3. Sabour S. A quantitative assessment of the accuracy and reliability of O-arm images for deep brain stimulation surgery. Neurosurgery 2013;72:E696. 4. Sabour S, Ghassemi F. Interrater reliability of intensive care unit electroencephalogram revised terminology: pitfalls and challenges of using kappa value. J Clin Neurophysiol 2013;30:210. 5. Sabour S. A quantitative assessment of the accuracy and reliability of O-arm images for deep brain stimulation surgery. Neurosurgery 2013;72:E696. 6. Sabour S, Dastjerdi EV. Reliability of assessment of nasal flow rate for nostril selection during nasotracheal intubation: common mistakes in reliability analysis. J Clin Anesth 2013;25:162. 7. Sabour S, Ghassemi F. The reproducibility of measurements of differential renal function in paediatric 99mTc-MAG3 renography: is this correct? Nucl Med Commun 2012;33:1311; author reply 1311–2.
Copyright of Clinical Chemistry & Laboratory Medicine is the property of De Gruyter and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Letter to the Editor Siamak Sabour*
Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults; statistical issues in validity and reliability analysis Keywords: reliability; statistical issues; validity. DOI 10.1515/cclm-2014-0453 Received April 28, 2014; accepted May 19, 2014; previously published online June 14, 2014
To the Editor,
I was interested to read the paper by Maldonado-Hernández and colleagues published in the May 2014 issue of Clinical Chemistry and Laboratory Medicine [1]. The authors aimed to evaluate the validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia (HHcy). As the authors mentioned, they performed a methionine loading test (MLT) as gold standard to characterize the HHcy status of 75 healthy adult subjects. For the breath test protocol, they selected a subsample and assigned to the control group 17 subjects with fasting and post-loading homocysteine (Hcy) ≤ 12 and 42.3 μmol/L after loading. Selected subjects were requested to have a second visit to perform a breath test within 1–4 weeks following the MLT test and received an oral dose of 2.5 mg/kg of 1-13C-methionine dissolved in water. Breath samples were collected at basal, 20, 40, 60, 80, 100 and 120 min (test 1). The same procedure was repeated within 1 week (test 2). The coefficient of variation *Corresponding author: Siamak Sabour, MD, MSc, DSc, PhD, Postdoc, Safety Promotion and Injury Prevention Research Centre, Department of Clinical Epidemiology, Shahid Beheshti University of Medical Sciences, Tehran, I. R. Iran, Phone: +98 21 22421814, E-mail: [email protected]
(CV) between breath test 1 and breath test 2 was 9.0 ± 5.4% [1]. Reliability (repeatability or reproducibility) is being assessed by different statistical tests. Briefly, for quantitative variable intra class correlation coefficient (ICCC) and for qualitative variables weighted κ should be used [1]. Regarding reliability or agreement, it is crucial to know that an individual-based approach instead of groupbased should be considered [2, 3]. Reliability assessment is the reason why we should consider individual results and not global average. Therefore, ICCC single measure instead of average measure should be reported to correctly assess reliability. In other words, the possibility of obtaining exactly the same average of a variable between two methods with no reliability at all is high. The same reasoning is also true for CV [2, 3]. These authors also reported that the 13C-methionine breath test has a sensitivity of 81.3% and specificity of 64.7% against the MLT [1]. Sensitivity (percentage with the disease tested positive, True Positives/[True Positives+False Negative]), specificity (percentage healthy tested negative, True Negatives/[True Negatives+False Positive]), positive predictive value (PPV), (percentage of positive tests in subjects who actually are diseased, True Positives/[True Positives+False Positive]), negative predictive value (NPV) (percentage of negative tests in subjects who are healthy, True Negatives/[True Negatives+False Negative]), likelihood ratio positive and likelihood ratio negative as well as diagnostic accuracy [(both true positive and true negative results/total) × 100] and odds ratio (true results/false results) preferably more than 50, are among the tests to evaluate the validity of a single test compared to a gold standard [2–7]. Accuracy (validity) is a completely different concept with precision (reliability) and should not be confused with each other [2, 3]. As the authors pointed out in their conclusions, the 13 C-methionine breath test is a valid and reliable method
e296 Sabour: Validity and reliability analysis for identifying subjects with moderate HHcy. Such a conclusion can be a misleading message due to inappropriate use of statistical tests to assess reliability and also incomplete evaluation of validity. Conflict of interest statement Authors’ conflict of interest disclosure: The authors stated that there are no conflicts of interest regarding the publication of this article. Research funding: None declared. Employment or leadership: None declared. Honorarium: None declared.
References 1. Maldonado-Hernández J, Prina-Ojeda LV, Montalvo-Velarde I, Del Prado-Manríquez M, de Lourdes Barbosa-Cortés M, Repetto-
Madrid M. Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults. Clin Chem Lab Med 2014;52:687–92. 2. Rothman KJ, Greenland S, Lash TL. Modern epidemiology, 4th ed. Baltimore, US: Lippincott Williams & Wilkins, 2010. 3. Sabour S. A quantitative assessment of the accuracy and reliability of O-arm images for deep brain stimulation surgery. Neurosurgery 2013;72:E696. 4. Sabour S, Ghassemi F. Interrater reliability of intensive care unit electroencephalogram revised terminology: pitfalls and challenges of using kappa value. J Clin Neurophysiol 2013;30:210. 5. Sabour S. A quantitative assessment of the accuracy and reliability of O-arm images for deep brain stimulation surgery. Neurosurgery 2013;72:E696. 6. Sabour S, Dastjerdi EV. Reliability of assessment of nasal flow rate for nostril selection during nasotracheal intubation: common mistakes in reliability analysis. J Clin Anesth 2013;25:162. 7. Sabour S, Ghassemi F. The reproducibility of measurements of differential renal function in paediatric 99mTc-MAG3 renography: is this correct? Nucl Med Commun 2012;33:1311; author reply 1311–2.
Copyright of Clinical Chemistry & Laboratory Medicine is the property of De Gruyter and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
