Downloaded from http://veterinaryrecord.bmj.com/ on February 18, 2016 - Published by group.bmj.com
Paper
Paper Comparison of a lateral flow milk progesterone test with enzyme immunoassay as an aid for reproductive status determination in cows A. Waldmann, A. Raud The lateral flow test (LFT) is an immunochromatographic method that utilises an immunostrip for non-laboratory diagnostic purposes. The present study evaluated a milk progesterone LFT against the enzyme immunoassay (EIA) to confirm oestrus and a nonpregnancy diagnosis. In total, 277 milk samples from 70 cows were analysed, collected on the day of artificial insemination and at 19 days, 21 days and 24 days post insemination. The level of accuracy of the LFT compared with the EIA was 95.0 per cent for milk samples containing 10 ng/ml progesterone. The validation of oestrus by the LFT was 98.6 per cent accurate using 2 ng/ml progesterone as the EIA estimate for oestrus. The test performance for a non-pregnancy diagnosis was subject to the day of milk sampling, showing the highest accuracy on day 24 post insemination for both tests. When optimised for maximum specificity, and compared with rectal palpation, the LFT had a sensitivity and specificity for non-pregnancy diagnosis on day 24 post insemination of 75.0 per cent and 100.0 per cent, respectively, with an overall accuracy of 84.4 per cent. The corresponding characteristics for the quantitative EIA were 85.0 per cent, 100.0 per cent and 90.6 per cent, respectively. The LFT results compared favourably with the quantitative milk progesterone EIA. Introduction Milk progesterone testing has a wide range of applications in animal production and veterinary requirements, including the identification of which cows are not pregnant 19–23 days post insemination (Burke and others 2012), confirmation of oestrous signs and pregnancy (Rajamahendran and others 1993, Romagnolo and Nebel 1993), as well as measuring the efficiency and accuracy of the detection of oestrus (Heersche and Nebel 1994), differentiation of ovarian cysts (Booth 1988; Sprecher and others 1988) and follow-up endocrine therapy (Sprecher and others 1990), screening of embryo donors and embryo transfer recipients (Britt and Holt 1988, Herrler and others 1990) and assessment of ovulation detection performance (Morton and Wynn 2010). Following the development of enzyme immunoassay (EIA) technology at the beginning of the 1980s for milk progesterone measurement (Arnstadt and Cleere 1981, Sauer and others 1981, van de Wiel and Koops 1982), on-farm milk progesterone tests quickly became commercially available (Nebel Veterinary Record (2016) A. Waldmann, DVM, MSc, DVetSc, A. Raud, MSc, Institute of Veterinary Medicine and Animal Sciences, Estonian University of Life Sciences, Kreutzwaldi 62, Tartu, 51014 Estonia A. Raud’s current address is Rauni Põllumajanduse OÜ, Saaremaa, Orissaare vald, Tagavere küla, 94601 Estonia
doi: 10.1136/vr.103605 E-mail for correspondence: [email protected] Part of this study was presented as a research abstract at 19th Annual ESDAR Conference, Albena, September 16–19, 2015 Provenance: Not commissioned; externally peer reviewed Accepted January 18, 2016
1988). Consequently, progesterone measurements could for the first time be performed directly on farms, thus abrogating the requirement to transport milk samples to specialised laboratories. Recently, a new milk progesterone on-farm test based on lateral flow technology has become commercially available. Lateral flow technology is considered to be particularly handy for use outside the laboratory (Posthuma-Trumpie and others 2009). The milk progesterone lateral flow test (LFT) is an immunochromatographic method in the form of an immunostrip, which is simply dipped into the milk sample. In the immunostrip, the specific interaction of progesterone in milk and an antiprogesterone antibody is responsible for the response. The assay uses dry reagents and constitutes a one-step reaction. When compared with EIA-based on-farm milk progesterone tests, which require the addition of several reagents, plus washing and incubation steps, the one-step LFT offers a big advantage in terms of ease of handling and speed. Moreover, the immunostrips do not need refrigeration for long-term storage. The aim of the present study was to evaluate the on-farm milk progesterone LFT against a quantitative EIA for assessing the accuracy in determining progesterone concentration in milk, to confirm oestrus on the day of artificial insemination (AI), and for a nonpregnancy diagnosis on days 19, 21 and 24 post AI in dairy cows.
Materials and methods The study was carried out using animals from a herd of 530 loose-housed commercial dairy cattle with an average 305-day milk yield of 11,000 kg per cow. Cows were milked three times per day, at 5:00, 13:00 and 20:00 hours. On the day of AI, and on days 19, 21 and 24 post AI, 277 milk samples were collected 10.1136/vr.103605 | Veterinary Record | 1 of 7
Downloaded from http://veterinaryrecord.bmj.com/ on February 18, 2016 - Published by group.bmj.com
Paper from 40 Estonian Red and 30 Estonian Holstein dairy cows, which formed a consecutive series. Milk sampling was carried out from July to September 2014 by one of the authors (AR) at the milking parlour during the midday milking. About 15–20 ml of milk was drawn by hand-stripping into plastic tubes after milking, after which 0.5 ml of the milk was used for the on-farm progesterone analysis. The remaining milk portion was preserved by the addition of 0.1 per cent potassium dichromate and stored at −18°C until milk progesterone analysis using EIA was performed.
Progesterone analysis On-farm test The LFT (Ridgeway Science, Gloucestershire, UK) was performed by AR approximately 10–15 minutes post collection according to the manufacturer’s instructions. Each sample was mixed and 0.5 ml of milk was added to plastic test vials using a disposable plastic dropper. A test strip, labelled with the cow number and date, was inserted into the milk in the test vial. Care was always taken to pipette the exact quantity of milk needed to perform the analysis, such that the milk level was always below the red line of the test strip. According to the manufacturer, too much milk can saturate the test and render it invalid. Colour development was recorded after 10 minutes’ incubation. The colour intensity of the test line was inversely proportional to the progesterone concentration within the sample. The colour intensities of the test line and the reference line were subjectively compared by naked eye and were recorded as: (A) test line darker than the reference line, (B) test line matched the reference line, (C) test line lighter than the reference line or (D) no visible test line (Fig 1). According to the manufacturer’s instructions, a low progesterone level is indicated by the test line being dark or darker than the reference line, while a high progesterone level is indicated by the test line being fainter than the reference line up to being invisible.
Enzyme immunoassay The EIA assay conditions have been described previously, together with the specificity (SP) of the monoclonal antibody used (Waldmann 1999). Milk samples were assayed by EIA in duplicate using eight 96-well microtitre plates. The interassay coefficients of variation for milk progesterone concentrations of
A
B
C
D
Reference line
Test line
2.67 ng/ml, 6.47 ng/ml and 23.38 ng/ml were 13.18 per cent, 4.25 per cent and 4.92 per cent, respectively. The intraassay coefficients of variation for milk progesterone concentrations of 3.02 ng/ml, 6.66 ng/ml and 24.76 ng/ml were 3.40 per cent, 2.91 per cent and 2.76 per cent, respectively (six replicates in duplicate). The calibration curve of the EIA assay ranged from 0 ng/ml to 60 ng/ml. The limit of detection of the assay, using a 20 ml sample of milk, was lower than 0.5 ng/ml.
Pregnancy diagnosis Pregnancy diagnoses were performed by a veterinarian by rectal palpation of the reproductive organs between 60 days and 70 days post AI. Rectal palpation diagnoses were used as a gold standard to estimate the accuracy of LFT and EIA in the diagnosis of non-pregnancy on days 19, 21 and 24 post AI.
Statistical analyses Differences in milk progesterone concentrations between the LFT colour classes were examined using the Kruskal-Wallis one-way analysis of variance. Pairwise comparison of the subgroups was performed according to the method described by Conover (1999). The Spearman’s rank correlation coefficient (r) was calculated to assess the agreement between LFT colour classes (semiquantitative analysis) and quantitative measurements of progesterone. To evaluate the predictive capability of LFT to discriminate between low and high progesterone concentrations and to determine the cut-off points, the colour intensity categories were converted to a binary scale as follows: (A) test line darker than the reference line=low concentration; test line matches or is lighter than the reference line or there is no visible test line=high concentration; (B) test line darker than or matches the reference line=low concentration; test line lighter than the reference line or there is no visible test line=high concentration. A receiveroperating characteristic (ROC) curve analysis was performed to evaluate the ability of LFT to discriminate between low and high progesterone concentrations and to estimate the cut-off values. To estimate the accuracy of LFT and EIA in the diagnosis of non-pregnancy, the areas under the ROC curves were calculated for LFT colour classes and progesterone concentrations as measured by EIA on days 19, 21 and 24 post AI. The areas under the ROC curves for the LFT colour classes and the EIA progesterone concentrations were compared according to the method described by DeLong and others (1988). Sensitivity (SE), SP, positive predictive values (PPVs), negative predictive values (NPVs) and accuracy were calculated using the following equations: SE=TP/(TP+FN); SP=TN/(TN+FP); PPV=TP/(TP+FP); NPV=TN/(TN+FN); and Accuracy=(TP +TN)/(TP+FP+TN+FN), where TPs are the true positives, TNs are the true negatives, FPs are the false positives and FNs are the false negatives. Statistical significance was set at P10 ng/ml
>5 – £10 ng/ml
>1 0
£1 0 –
– 0 >2
–
B
Lateral flow test colour class
>5
£5
0
A
n = 147
n = 98 A
0
0
n = 21
0.8
£2
10
n = 11
1.0
c
rho = 0.85 P< 0.0001 Progesterone (ng/ml)
n = 147
d
50
Progesterone (ng/ml)
FIG 4: Proportion of milk samples by concentration of progesterone from 70 cows, collected on the day of artificial insemination (AI) and 19 days and 21 days post AI, and from 67 cows, collected on day 24 post AI, as determined by the enzyme immunoassay (EIA), after classification by the lateral flow test colour class. A=test line darker than the reference line; B=test line matching the reference line; C=test line lighter than the reference line; D=no visible test line
milk samples with a progesterone concentration ≤2 ng/ml, 95.0 per cent gave rise to a test line darker than the reference line. For milk samples with progesterone concentrations >10 ng/ml, 97.0 per cent had a test line lighter than the reference line or no visible test line. Of the milk samples with progesterone concentrations >2 ng/ml and ≤10 ng/ml, larger variations between different LFT colour classes were observed.
Determining the cut-off values and predictive capability The ability of LFT to discriminate between low and high progesterone concentrations was tested by converting the LFT colour intensities into a binary scale. Two different colour class combinations were tested and the corresponding cut-offs determined. First: samples with a test line darker than the reference line were classified as low concentration, while samples with test lines matching or fainter than the reference line, including no visible test line, were classified as high concentration. For such sample classification, ROC plots for SE versus 100 − SP, at the point of the greatest sum of SE and SP, yielded an SE equal to 97.3 per cent (95% confidence interval (CI) 93.2 to 99.3) and an SP equal to 94.6 per cent (95% CI 90.8 to 97.8), corresponding to a cut-off value of 2.3 ng/ml (Fig 5a). Second: when samples with a test line darker than or matching the reference line were classified as low concentration, while samples with a test line fainter than the reference line, (including no visible test line) were classified as high concentration, the ROC plot yielded an SE equal to 95.2 per cent (95% CI 90.8 to 97.8) and a SP of 95.5 per cent (95% CI 89.7 to 98.5), corresponding to a cut-off value of 6.9 ng/ ml (Fig 5b). Figs 5c and 5d depict the corresponding SE and SP values as a function of progesterone concentrations.
Progesterone in milk samples on the day of AI
0.4 0.2 0 A
B
C
D
Lateral flow test colour class
FIG 3: Distributions of milk progesterone concentrations from 70 cows collected on the day of artificial insemination (AI) and 19 days and 21 days post AI, and from 67 cows, collected on day 24 post AI, as determined by the enzyme immunoassay (EIA) for each lateral flow test colour class. A=test line darker than the reference line; B=test line matching the reference line; C=test line lighter than the reference line; D=no visible test line
Progesterone concentrations for 70 cows were measured in milk samples collected on the day of AI. Sixty-seven cows considered to have low progesterone were found to have progesterone concentrations 5.0 ng/ml and were thus considered to have high progesterone. Of the 67 milk samples classified as low progesterone by EIA, all had a test line darker than the reference line on the LFT and were hence classified as low by LFT. For the three milk samples classified as high by EIA, two samples were classified as high (test line lighter than the reference line) and one sample was classified as low (test line darker than the reference line) by LFT. The test accuracy was 69/70, that is 98.6 per cent. 10.1136/vr.103605 | Veterinary Record | 3 of 7
Downloaded from http://veterinaryrecord.bmj.com/ on February 18, 2016 - Published by group.bmj.com
Paper (b) 100
(a) 100
Sensitivity (%)
Sensitivity (%)
AUC = 0.975 95% CI = 0.953–0.997 P< 0.0001
80
AUC = 0.981 95% CI = 0.965–0.998 P< 0.0001
80
60
40
60
40
20
20
0
0 0
20
40
80
60
0
100
20
60
100
80
(d) 100 Sensitivity and specificity (%)
(c) 100 Sensitivity and specificity (%)
40
100-Specificity (%)
100-Specificity (%)
Sensitivity
80
Specificity 60
40
20
Sensitivity
80
Specificity
60
40
20
0
0 0
10
20
30
40
50
0
10
20
30
40
50
Progesterne (ng/ml)
Progesterne (ng/ml)
FIG 5: Receiver operating characteristic (ROC) curves (upper figures) illustrating the sensitivity and specificity of the milk progesterone lateral flow test from 70 cows on the day of artificial insemination (AI) and 19 days and 21 days post AI, and from 67 cows on day 24 post AI, for differentiating between samples with low and high progesterone concentrations. (a) Low=test line darker than the reference line (n=147); high=test line matching the reference line/test line lighter than the reference line/no visible test line (n=130). (b) Low=test line darker than the reference line/test line matching the reference line (n=167); high=test line lighter than the reference line/no visible test line (n=110). The optimal curve deviates upwards towards the upper left corner of the graph. The lower figures (c and d) depict sensitivity and specificity values as a function of progesterone concentration. AUC, area under the curve.
Non-pregnancy diagnosis The three cows with high milk progesterone concentrations on the day of AI were not included in the non-pregnancy analysis, neither were cows that did not produce a complete set of milk samples for days 19, 21 and 24 post insemination. There were 40 non-pregnant (63 per cent) and 24 pregnant (37 per cent) cows, confirmed by rectal palpation. The ROC plots for non-pregnancy diagnosis showed that for LFT (Fig 6a) and EIA (Fig 6b), day 24 post AI had the most
(a)
favourable AUC (mean±SE) (0.91±0.036 and 0.99±0.010, respectively), followed by day 21 (0.86±0.040 and 0.92±0.035) and then day 19 (0.70±057 and 0.79±0.056). However, AUC for LFT between days 21 and 24 did not differ (P=0.19). For all milk sampling days, AUC of EIA was larger (P
Paper
Paper Comparison of a lateral flow milk progesterone test with enzyme immunoassay as an aid for reproductive status determination in cows A. Waldmann, A. Raud The lateral flow test (LFT) is an immunochromatographic method that utilises an immunostrip for non-laboratory diagnostic purposes. The present study evaluated a milk progesterone LFT against the enzyme immunoassay (EIA) to confirm oestrus and a nonpregnancy diagnosis. In total, 277 milk samples from 70 cows were analysed, collected on the day of artificial insemination and at 19 days, 21 days and 24 days post insemination. The level of accuracy of the LFT compared with the EIA was 95.0 per cent for milk samples containing 10 ng/ml progesterone. The validation of oestrus by the LFT was 98.6 per cent accurate using 2 ng/ml progesterone as the EIA estimate for oestrus. The test performance for a non-pregnancy diagnosis was subject to the day of milk sampling, showing the highest accuracy on day 24 post insemination for both tests. When optimised for maximum specificity, and compared with rectal palpation, the LFT had a sensitivity and specificity for non-pregnancy diagnosis on day 24 post insemination of 75.0 per cent and 100.0 per cent, respectively, with an overall accuracy of 84.4 per cent. The corresponding characteristics for the quantitative EIA were 85.0 per cent, 100.0 per cent and 90.6 per cent, respectively. The LFT results compared favourably with the quantitative milk progesterone EIA. Introduction Milk progesterone testing has a wide range of applications in animal production and veterinary requirements, including the identification of which cows are not pregnant 19–23 days post insemination (Burke and others 2012), confirmation of oestrous signs and pregnancy (Rajamahendran and others 1993, Romagnolo and Nebel 1993), as well as measuring the efficiency and accuracy of the detection of oestrus (Heersche and Nebel 1994), differentiation of ovarian cysts (Booth 1988; Sprecher and others 1988) and follow-up endocrine therapy (Sprecher and others 1990), screening of embryo donors and embryo transfer recipients (Britt and Holt 1988, Herrler and others 1990) and assessment of ovulation detection performance (Morton and Wynn 2010). Following the development of enzyme immunoassay (EIA) technology at the beginning of the 1980s for milk progesterone measurement (Arnstadt and Cleere 1981, Sauer and others 1981, van de Wiel and Koops 1982), on-farm milk progesterone tests quickly became commercially available (Nebel Veterinary Record (2016) A. Waldmann, DVM, MSc, DVetSc, A. Raud, MSc, Institute of Veterinary Medicine and Animal Sciences, Estonian University of Life Sciences, Kreutzwaldi 62, Tartu, 51014 Estonia A. Raud’s current address is Rauni Põllumajanduse OÜ, Saaremaa, Orissaare vald, Tagavere küla, 94601 Estonia
doi: 10.1136/vr.103605 E-mail for correspondence: [email protected] Part of this study was presented as a research abstract at 19th Annual ESDAR Conference, Albena, September 16–19, 2015 Provenance: Not commissioned; externally peer reviewed Accepted January 18, 2016
1988). Consequently, progesterone measurements could for the first time be performed directly on farms, thus abrogating the requirement to transport milk samples to specialised laboratories. Recently, a new milk progesterone on-farm test based on lateral flow technology has become commercially available. Lateral flow technology is considered to be particularly handy for use outside the laboratory (Posthuma-Trumpie and others 2009). The milk progesterone lateral flow test (LFT) is an immunochromatographic method in the form of an immunostrip, which is simply dipped into the milk sample. In the immunostrip, the specific interaction of progesterone in milk and an antiprogesterone antibody is responsible for the response. The assay uses dry reagents and constitutes a one-step reaction. When compared with EIA-based on-farm milk progesterone tests, which require the addition of several reagents, plus washing and incubation steps, the one-step LFT offers a big advantage in terms of ease of handling and speed. Moreover, the immunostrips do not need refrigeration for long-term storage. The aim of the present study was to evaluate the on-farm milk progesterone LFT against a quantitative EIA for assessing the accuracy in determining progesterone concentration in milk, to confirm oestrus on the day of artificial insemination (AI), and for a nonpregnancy diagnosis on days 19, 21 and 24 post AI in dairy cows.
Materials and methods The study was carried out using animals from a herd of 530 loose-housed commercial dairy cattle with an average 305-day milk yield of 11,000 kg per cow. Cows were milked three times per day, at 5:00, 13:00 and 20:00 hours. On the day of AI, and on days 19, 21 and 24 post AI, 277 milk samples were collected 10.1136/vr.103605 | Veterinary Record | 1 of 7
Downloaded from http://veterinaryrecord.bmj.com/ on February 18, 2016 - Published by group.bmj.com
Paper from 40 Estonian Red and 30 Estonian Holstein dairy cows, which formed a consecutive series. Milk sampling was carried out from July to September 2014 by one of the authors (AR) at the milking parlour during the midday milking. About 15–20 ml of milk was drawn by hand-stripping into plastic tubes after milking, after which 0.5 ml of the milk was used for the on-farm progesterone analysis. The remaining milk portion was preserved by the addition of 0.1 per cent potassium dichromate and stored at −18°C until milk progesterone analysis using EIA was performed.
Progesterone analysis On-farm test The LFT (Ridgeway Science, Gloucestershire, UK) was performed by AR approximately 10–15 minutes post collection according to the manufacturer’s instructions. Each sample was mixed and 0.5 ml of milk was added to plastic test vials using a disposable plastic dropper. A test strip, labelled with the cow number and date, was inserted into the milk in the test vial. Care was always taken to pipette the exact quantity of milk needed to perform the analysis, such that the milk level was always below the red line of the test strip. According to the manufacturer, too much milk can saturate the test and render it invalid. Colour development was recorded after 10 minutes’ incubation. The colour intensity of the test line was inversely proportional to the progesterone concentration within the sample. The colour intensities of the test line and the reference line were subjectively compared by naked eye and were recorded as: (A) test line darker than the reference line, (B) test line matched the reference line, (C) test line lighter than the reference line or (D) no visible test line (Fig 1). According to the manufacturer’s instructions, a low progesterone level is indicated by the test line being dark or darker than the reference line, while a high progesterone level is indicated by the test line being fainter than the reference line up to being invisible.
Enzyme immunoassay The EIA assay conditions have been described previously, together with the specificity (SP) of the monoclonal antibody used (Waldmann 1999). Milk samples were assayed by EIA in duplicate using eight 96-well microtitre plates. The interassay coefficients of variation for milk progesterone concentrations of
A
B
C
D
Reference line
Test line
2.67 ng/ml, 6.47 ng/ml and 23.38 ng/ml were 13.18 per cent, 4.25 per cent and 4.92 per cent, respectively. The intraassay coefficients of variation for milk progesterone concentrations of 3.02 ng/ml, 6.66 ng/ml and 24.76 ng/ml were 3.40 per cent, 2.91 per cent and 2.76 per cent, respectively (six replicates in duplicate). The calibration curve of the EIA assay ranged from 0 ng/ml to 60 ng/ml. The limit of detection of the assay, using a 20 ml sample of milk, was lower than 0.5 ng/ml.
Pregnancy diagnosis Pregnancy diagnoses were performed by a veterinarian by rectal palpation of the reproductive organs between 60 days and 70 days post AI. Rectal palpation diagnoses were used as a gold standard to estimate the accuracy of LFT and EIA in the diagnosis of non-pregnancy on days 19, 21 and 24 post AI.
Statistical analyses Differences in milk progesterone concentrations between the LFT colour classes were examined using the Kruskal-Wallis one-way analysis of variance. Pairwise comparison of the subgroups was performed according to the method described by Conover (1999). The Spearman’s rank correlation coefficient (r) was calculated to assess the agreement between LFT colour classes (semiquantitative analysis) and quantitative measurements of progesterone. To evaluate the predictive capability of LFT to discriminate between low and high progesterone concentrations and to determine the cut-off points, the colour intensity categories were converted to a binary scale as follows: (A) test line darker than the reference line=low concentration; test line matches or is lighter than the reference line or there is no visible test line=high concentration; (B) test line darker than or matches the reference line=low concentration; test line lighter than the reference line or there is no visible test line=high concentration. A receiveroperating characteristic (ROC) curve analysis was performed to evaluate the ability of LFT to discriminate between low and high progesterone concentrations and to estimate the cut-off values. To estimate the accuracy of LFT and EIA in the diagnosis of non-pregnancy, the areas under the ROC curves were calculated for LFT colour classes and progesterone concentrations as measured by EIA on days 19, 21 and 24 post AI. The areas under the ROC curves for the LFT colour classes and the EIA progesterone concentrations were compared according to the method described by DeLong and others (1988). Sensitivity (SE), SP, positive predictive values (PPVs), negative predictive values (NPVs) and accuracy were calculated using the following equations: SE=TP/(TP+FN); SP=TN/(TN+FP); PPV=TP/(TP+FP); NPV=TN/(TN+FN); and Accuracy=(TP +TN)/(TP+FP+TN+FN), where TPs are the true positives, TNs are the true negatives, FPs are the false positives and FNs are the false negatives. Statistical significance was set at P10 ng/ml
>5 – £10 ng/ml
>1 0
£1 0 –
– 0 >2
–
B
Lateral flow test colour class
>5
£5
0
A
n = 147
n = 98 A
0
0
n = 21
0.8
£2
10
n = 11
1.0
c
rho = 0.85 P< 0.0001 Progesterone (ng/ml)
n = 147
d
50
Progesterone (ng/ml)
FIG 4: Proportion of milk samples by concentration of progesterone from 70 cows, collected on the day of artificial insemination (AI) and 19 days and 21 days post AI, and from 67 cows, collected on day 24 post AI, as determined by the enzyme immunoassay (EIA), after classification by the lateral flow test colour class. A=test line darker than the reference line; B=test line matching the reference line; C=test line lighter than the reference line; D=no visible test line
milk samples with a progesterone concentration ≤2 ng/ml, 95.0 per cent gave rise to a test line darker than the reference line. For milk samples with progesterone concentrations >10 ng/ml, 97.0 per cent had a test line lighter than the reference line or no visible test line. Of the milk samples with progesterone concentrations >2 ng/ml and ≤10 ng/ml, larger variations between different LFT colour classes were observed.
Determining the cut-off values and predictive capability The ability of LFT to discriminate between low and high progesterone concentrations was tested by converting the LFT colour intensities into a binary scale. Two different colour class combinations were tested and the corresponding cut-offs determined. First: samples with a test line darker than the reference line were classified as low concentration, while samples with test lines matching or fainter than the reference line, including no visible test line, were classified as high concentration. For such sample classification, ROC plots for SE versus 100 − SP, at the point of the greatest sum of SE and SP, yielded an SE equal to 97.3 per cent (95% confidence interval (CI) 93.2 to 99.3) and an SP equal to 94.6 per cent (95% CI 90.8 to 97.8), corresponding to a cut-off value of 2.3 ng/ml (Fig 5a). Second: when samples with a test line darker than or matching the reference line were classified as low concentration, while samples with a test line fainter than the reference line, (including no visible test line) were classified as high concentration, the ROC plot yielded an SE equal to 95.2 per cent (95% CI 90.8 to 97.8) and a SP of 95.5 per cent (95% CI 89.7 to 98.5), corresponding to a cut-off value of 6.9 ng/ ml (Fig 5b). Figs 5c and 5d depict the corresponding SE and SP values as a function of progesterone concentrations.
Progesterone in milk samples on the day of AI
0.4 0.2 0 A
B
C
D
Lateral flow test colour class
FIG 3: Distributions of milk progesterone concentrations from 70 cows collected on the day of artificial insemination (AI) and 19 days and 21 days post AI, and from 67 cows, collected on day 24 post AI, as determined by the enzyme immunoassay (EIA) for each lateral flow test colour class. A=test line darker than the reference line; B=test line matching the reference line; C=test line lighter than the reference line; D=no visible test line
Progesterone concentrations for 70 cows were measured in milk samples collected on the day of AI. Sixty-seven cows considered to have low progesterone were found to have progesterone concentrations 5.0 ng/ml and were thus considered to have high progesterone. Of the 67 milk samples classified as low progesterone by EIA, all had a test line darker than the reference line on the LFT and were hence classified as low by LFT. For the three milk samples classified as high by EIA, two samples were classified as high (test line lighter than the reference line) and one sample was classified as low (test line darker than the reference line) by LFT. The test accuracy was 69/70, that is 98.6 per cent. 10.1136/vr.103605 | Veterinary Record | 3 of 7
Downloaded from http://veterinaryrecord.bmj.com/ on February 18, 2016 - Published by group.bmj.com
Paper (b) 100
(a) 100
Sensitivity (%)
Sensitivity (%)
AUC = 0.975 95% CI = 0.953–0.997 P< 0.0001
80
AUC = 0.981 95% CI = 0.965–0.998 P< 0.0001
80
60
40
60
40
20
20
0
0 0
20
40
80
60
0
100
20
60
100
80
(d) 100 Sensitivity and specificity (%)
(c) 100 Sensitivity and specificity (%)
40
100-Specificity (%)
100-Specificity (%)
Sensitivity
80
Specificity 60
40
20
Sensitivity
80
Specificity
60
40
20
0
0 0
10
20
30
40
50
0
10
20
30
40
50
Progesterne (ng/ml)
Progesterne (ng/ml)
FIG 5: Receiver operating characteristic (ROC) curves (upper figures) illustrating the sensitivity and specificity of the milk progesterone lateral flow test from 70 cows on the day of artificial insemination (AI) and 19 days and 21 days post AI, and from 67 cows on day 24 post AI, for differentiating between samples with low and high progesterone concentrations. (a) Low=test line darker than the reference line (n=147); high=test line matching the reference line/test line lighter than the reference line/no visible test line (n=130). (b) Low=test line darker than the reference line/test line matching the reference line (n=167); high=test line lighter than the reference line/no visible test line (n=110). The optimal curve deviates upwards towards the upper left corner of the graph. The lower figures (c and d) depict sensitivity and specificity values as a function of progesterone concentration. AUC, area under the curve.
Non-pregnancy diagnosis The three cows with high milk progesterone concentrations on the day of AI were not included in the non-pregnancy analysis, neither were cows that did not produce a complete set of milk samples for days 19, 21 and 24 post insemination. There were 40 non-pregnant (63 per cent) and 24 pregnant (37 per cent) cows, confirmed by rectal palpation. The ROC plots for non-pregnancy diagnosis showed that for LFT (Fig 6a) and EIA (Fig 6b), day 24 post AI had the most
(a)
favourable AUC (mean±SE) (0.91±0.036 and 0.99±0.010, respectively), followed by day 21 (0.86±0.040 and 0.92±0.035) and then day 19 (0.70±057 and 0.79±0.056). However, AUC for LFT between days 21 and 24 did not differ (P=0.19). For all milk sampling days, AUC of EIA was larger (P
