BAYESIAN ANALYSIS TO EVALUATE TESTS FOR THE DETECTION OF MYCOBACTERIUM BOVIS INFECTION IN FREE-RANGING WILD BISON (BISON BISON ATHABASCAE) IN THE ABSENCE OF A GOLD STANDARD Author(s): Núria Chapinal, Brant A. Schumaker, Damien O. Joly, Brett T. Elkin, and Craig Stephen Source: Journal of Wildlife Diseases, 51(3):619-625. Published By: Wildlife Disease Association DOI: http://dx.doi.org/10.7589/2013-09-254 URL: http://www.bioone.org/doi/full/10.7589/2013-09-254
BioOne (www.bioone.org) is a nonprofit, online aggregation of core research in the biological, ecological, and environmental sciences. BioOne provides a sustainable online platform for over 170 journals and books published by nonprofit societies, associations, museums, institutions, and presses. Your use of this PDF, the BioOne Web site, and all posted and associated content indicates your acceptance of BioOne’s Terms of Use, available at www.bioone.org/page/ terms_of_use. Usage of BioOne content is strictly limited to personal, educational, and non-commercial use. Commercial inquiries or rights and permissions requests should be directed to the individual publisher as copyright holder.
BioOne sees sustainable scholarly publishing as an inherently collaborative enterprise connecting authors, nonprofit publishers, academic institutions, research libraries, and research funders in the common goal of maximizing access to critical research.
DOI: 10.7589/2013-09-254
Journal of Wildlife Diseases, 51(3), 2015, pp. 619–625 # Wildlife Disease Association 2015
BAYESIAN ANALYSIS TO EVALUATE TESTS FOR THE DETECTION OF MYCOBACTERIUM BOVIS INFECTION IN FREE-RANGING WILD BISON (BISON BISON ATHABASCAE) IN THE ABSENCE OF A GOLD STANDARD Nu´ria Chapinal,1 Brant A. Schumaker,2,6 Damien O. Joly,3 Brett T. Elkin,4 and Craig Stephen5 1 Animal Welfare Program, University of British Columbia, 2357 Main Mall, Vancouver, British Columbia V6T 1Z4, Canada 2 Department of Veterinary Sciences, University of Wyoming, 1000 E University Avenue, Laramie, Wyoming 82070, USA 3 Metabiota, Unit 7, 1611 Bowen Road, Nanaimo, British Columbia V9S 1G5, Canada 4 Wildlife Division, Department of Environment and Natural Resources, Government of the Northwest Territories, 600, 5102 50th Avenue, Yellowknife, Northwest Territories X1A 3S8, Canada 5 Canadian Wildlife Health Cooperative, 52 Campus Drive, Saskatoon, Saskatchewan S7N 5B4, Canada 6 Corresponding author (email: [email protected])
ABSTRACT: We estimated the sensitivity and specificity of the caudal-fold skin test (CFT), the fluorescent polarization assay (FPA), and the rapid lateral-flow test (RT) for the detection of Mycobacterium bovis in free-ranging wild wood bison (Bison bison athabascae), in the absence of a gold standard, by using Bayesian analysis, and then used those estimates to forecast the performance of a pairwise combination of tests in parallel. In 1998–99, 212 wood bison from Wood Buffalo National Park (Canada) were tested for M. bovis infection using CFT and two serologic tests (FPA and RT). The sensitivity and specificity of each test were estimated using a three-test, one-population, Bayesian model allowing for conditional dependence between FPA and RT. The sensitivity and specificity of the combination of CFT and each serologic test in parallel were calculated assuming conditional independence. The test performance estimates were influenced by the prior values chosen. However, the rank of tests and combinations of tests based on those estimates remained constant. The CFT was the most sensitive test and the FPA was the least sensitive, whereas RT was the most specific test and CFT was the least specific. In conclusion, given the fact that gold standards for the detection of M. bovis are imperfect and difficult to obtain in the field, Bayesian analysis holds promise as a tool to rank tests and combinations of tests based on their performance. Combining a skin test with an animal-side serologic test, such as RT, increases sensitivity in the detection of M. bovis and is a good approach to enhance disease eradication or control in wild bison. Key words: Bayesian analysis, diagnostic tests, serological tests, tuberculin skin tests, wildlife disease, zoonosis.
infection from a population, particularly in free-ranging wildlife where handling individuals is difficult. Disease-eradication programs often involve regular testing of the population and subsequent removal of positive animals. Therefore, a detection method with high sensitivity is necessary to avoid releasing infected (false-negative) animals. In other situations, less-radical measures might be chosen to control, rather than eliminate, the disease to support conservation objectives. Managers may want to remove only animals that are clinically affected and more likely to be actively shedding bacteria, rather than all animals positive on serologic tests. The test or combination of tests selected should reflect
INTRODUCTION
Bovine tuberculosis, caused by Mycobacterium bovis, is a reemerging disease that affects wildlife, livestock, and humans (Thoen et al. 2009). It is endemic in free-ranging wood bison (Bison bison athabascae) populations in and around Wood Buffalo National Park (WBNP), Canada (Joly and Messier 2004), and these populations act as a reservoir of the pathogen (Nishi et al. 2006). Reliable detection methods (based on one or a combination of validated diagnostic tests) for tuberculosis in bison are essential to effectively implement wildlife and disease-management programs to control or eradicate 619
620
JOURNAL OF WILDLIFE DISEASES, VOL. 51, NO. 3, JULY 2015
the desired goals and should be customized to the particular management context. Bovine tuberculosis is commonly diagnosed in domestic and wild bovids with a single intradermal test (skin test), which measures a cell-mediated immune response to the intradermal injection of a purified protein derivative (PPD or tuberculin) prepared from M. bovis. Skin tests have been reported to have low sensitivity in bison (Himsworth et al. 2010). Different serologic tests, with varied sensitivity and specificity, have been developed for the detection of M. bovis in bison (Himsworth et al. 2010). These tests show poor agreement with the skin test (Himsworth et al. 2010; Chapinal et al. 2012), partly because the tests measure different immune responses (humoral vs. cell-mediated), which are predominant at different stages of the infection. Some studies in different wildlife species (Himsworth et al. 2010; Waters et al. 2011) have suggested combining a skin test with a serologic test in parallel (where one positive test would result in disease-positive classification) as a way to increase sensitivity in the detection of M. bovis. Assessing the sensitivity and specificity of the available diagnostic tests is necessary to determine what combination of tests would be more accurate. However, there is a dearth of well-validated data to assess accuracy in diagnostic tests for tuberculosis in bison. Culture of M. bovis continues to be the gold standard for evaluation of tests, despite its imperfect sensitivity and the difficulty in obtaining samples for culture from wild populations (Cousins and Florisson 2005). Other methods have been developed to assess sensitivity and specificity of diagnostic tests in the absence of a gold standard, such as maximum likelihood and Bayesian estimations, which merit further research. Bayesian estimations have recently been used for the evaluation of diagnostic tests for brucellosis in bison (Schumaker et al. 2010). These methods are flexible and can account for
factors such as dependence among test results, which is to be expected if tests measure similar biologic processes, such as serum antibody responses to infectious agents (Dohoo et al. 2009). Herein, we estimate the sensitivity and specificity of the caudal-fold skin test (CFT) and serologic tests for the detection of M. bovis in free-ranging wild wood bison in the absence of a gold standard by Bayesian analysis, and we use those estimates to forecast the performance of a pairwise combination of tests in parallel for the detection of M. bovis infection in free-ranging wild wood bison under different testing goals. MATERIALS AND METHODS Animals
This study is a retrospective analysis of historical test results derived from other research projects. We used 212 free-ranging wild wood bison (157 females and 55 males; 17% calves, 9% yearlings, 15% subadults, and 59% adults) from WBNP. These are the same animals used by Chapinal et al. (2012) and a subset of the 346 animals used by Joly and Messier (2004), but the results presented here include additional serologic tests that were not available at the time of those studies. Animals were captured and sampled in February 1998 and in February and March 1999. Details on animal handling are described by Joly and Messier (2004). Animals had not been vaccinated against any disease. Sampling and testing
The CFT was conducted as described by Joly and Messier (2004). Briefly, bison were injected with 0.1 mL of PPD tuberculin intradermally in the caudal fold and the injection site was inspected 72 h later (Monaghan et al. 1994). Bison were classified as positive if there was any visible or palpable change in the tissue at the point of injection. Blood samples were taken from each bison from the caudal or jugular vein, collected in serum separator tubes to facilitate clotting. Serum was removed by centrifuge within 12 h, and serum samples were frozen until serologic tests for M. bovis were conducted. Four different serologic tests were performed as described by Chapinal et al. (2012): 1) the fluorescent polarization assay (FPA), 2) the
CHAPINAL ET AL.—DIAGNOSIS OF M. BOVIS IN FREE-RANGING BISON
multiantigen print immunoassay (MAPIA), 3) the rapid lateral-flow test (RT; VetTB STAT-PAK, Chembio Diagnostic Systems, Inc., Melford, New York, USA), and 4) the dual-path platform test (DPP; DPP VetTB, Chembio). The MAPIA, RT, and DPP measure a very similar immune response (they share three antigens) and they show almost perfect agreement (Chapinal et al. 2012). This suggests a large conditional dependence among them and therefore, limited benefit in combining them for the diagnosis of M. bovis in bison. Therefore, of these three tests, only RT was considered for further analysis because it is the only animal-side test currently available.
621
T ABLE 1. Informative prior distributions for a three-test, one-population, Bayesian model of a caudal fold test (CFT), fluorescent polarization assay (FPA), and rapid test (RT) for the detection of Mycobacterium bovis in 212 free-ranging wild wood bison (Bison bison athabascae) in Wood Buffalo National Park, Canada.
Parameter
b Distribution
Expected values (%)
Prevalence CFT sensitivity CFT specificity RT30 specificity FPA specificity
b(42.75, 42.75) b(30.25, 30.25) b(53.2, 7.3) b(225, 4) b(105, 29)
50 50 87.9 98.3 78.4
Data analysis
Bayesian models allow for the probabilistic estimation of test performance characteristics by incorporating previous information about the diagnostic tests and the prevalence of variable stages of infection in the study population. In biologic terms, animals under different stages of infection were considered ‘‘positive’’ for M. bovis if they showed a sufficient immune response to be consistent with that particular infection stage. The sensitivity and specificity of CFT, FPA, and RT were estimated using a three-test, one-population, Bayesian model allowing for conditional dependence between FPA and RT, as described in section 3.4 of Branscum et al. (2005). The CFT measures a different immune response and shows no agreement with serologic tests (Himsworth et al. 2010; Chapinal et al. 2012). Therefore, CFT was considered independent from FPA and RT in the Bayesian models. Beta prior distributions for the specificity and sensitivity of CFT were calculated from the tests results of Himsworth et al. (2010; Table 1). Beta prior distributions for the specificity of FPA and RT were calculated from the tests results of 227 animals sampled from 1990 to 2007 in the tuberculosis-free Mackenzie Bison Sanctuary herd (Canada; Table 1). The beta prior distribution of the prevalence of infected animals was calculated from the test results of Joly and Messier (2004), who estimated an apparent prevalence of infected animals at 50% when interpreting CFT and FPA in parallel (i.e., animals positive to either test were considered infected with M. bovis; Table 1). Posterior inferences were based on 50,000 iterations of each model and were estimated using freeware WinBugs (Spiegelhalter et al. 2003). Five initial chains were used and convergence of the model was assessed through sample trace, history, and Gelman-Rubin plots.
Additionally, the joint sensitivities and specificities of CFT with each of the two serologic tests was calculated from section 3.1 of Gardner et al. (2000) and iterated to determine a point estimate and 95% probability interval. A complete sensitivity analysis was performed, increasing and decreasing the informative prior distributions by 5% and 10% and determining their influence on the posterior distributions of the prevalence in addition to all sensitivity and specificity values. Additionally, we used prevalence estimates of 25% and 5% to calculate the prevalence of the prior distribution to determine the importance of the prevalence prior on the posterior distributions. Finally, diffuse (b[1,1]) priors were applied individually and together to determine the most frequent outcome of our analyses. For all analyses, the model was iterated 50,000 times and all posterior point and interval estimates were obtained. Before convergence, 5,000 iterations were used for burn-in and then discarded. RESULTS
Table 2 shows the cross-classified results for M. bovis from CFT, FPA, and RT. TABLE 2. Cross-classified results from caudal fold test (CFT), fluorescent polarization assay (FPA), and rapid test (RT) for the detection of Mycobacterium bovis in 212 free-ranging wild wood bison (Bison bison athabascae) in Wood Buffalo National Park, Canada. CFT+ (n5100) RT
FPA+ FPA2
6 9
+
RT
CFT2 (n5112) 2
3 82
RT+
RT2
2 9
2 99
622
JOURNAL OF WILDLIFE DISEASES, VOL. 51, NO. 3, JULY 2015
TABLE 3. Summary of a three-test, one-population, Bayesian model for Mycobacterium bovis infection in 212 free-ranging wild wood bison (Bison bison athabascae) in Wood Buffalo National Park, Canada, assuming conditional independence between the caudal fold test (CFT) and the two serologic tests (fluorescent polarization assay [FPA] and rapid test [RT]), and conditional dependence between FPA and RT. Parameter
Estimate
a
CFT sensitivity % (95% PI) CFT specificity % (95% PI)a FPA sensitivity % (95% PI)a FPA specificity % (95% PI)a RT sensitivity % (95% PI)a RT specificity % (95% PI)a Prevalence in the sample % (95% PI)a Sensitivity covariance of FPA and RT Specificity covariance of FPA and RT % iterations CFT sensitivity was higher than FPA sensitivity % iterations CFT sensitivity was higher than RT sensitivity % iterations RT sensitivity was higher than FPA sensitivity % iterations CFT specificity was higher than FPA specificity % iterations CFT specificity was higher than RT specificity % iterations RT specificity was higher than FPA specificity a
57.6 80.3 4.4 83.2 12.7 98.4 57.5 2.0 25.5 100 100 97.6 31.9 0 100
(48.2, 67.0) (68.7, 90.2) (0.4, 10.6) (77.2, 88.2) (4.2, 21.9) (96.2, 99.5) (47.4, 67.0) (20.1, 6.4) (217.9, 1.1)
Expressed as median with 95% probability interval (PI) in parentheses.
Based on the three-test, one-population, Bayesian model (Table 3), CFT was the most sensitive test and FPA was the least sensitive, whereas RT was the most specific test and CFT was the least specific. Table 4 summarizes the sensitivity and specificity of combinations of CFT with either FPA or RT in parallel. The posterior point and interval estimates for test performance and prevalence were most sensitive to changes in the specificity priors for FPA and CFT, followed by changes to the prevalence prior. The model results were comparatively insensitive to changes to the prior estimates for specificity of RT and sensitivity of CFT. However, even using all diffuse prior distributions, the rankings of test sensitivities and specificities did not change.
DISCUSSION
The assessment of the sensitivity and specificity of diagnostics tests for bovine tuberculosis is challenging because a perfect gold standard for live-animal testing does not exist. Not surprisingly, a consensus on a definition of a ‘‘disease-positive’’ animal to use in the field has yet to be achieved. Culture, histology, and even CFT have been use as (pseudo)-gold standards for the evaluation of diagnostic tests for bovine tuberculosis in wild bison and other wildlife species (Cousins and Florisson 2005) but, to our knowledge, Bayesian methods have not been used for that purpose. Bayesian analyses have limitations or biases because prior information about the prevalence of the disease in the population and about the
TABLE 4. Sensitivity and specificity of combinations in parallel of the caudal fold test (CFT) and two serologic tests (fluorescent polarization assay [FPA] and rapid test [RT]). Conditional independence between CFT and the two serological tests was assumed. Tests
% Sensitivity (95% PI)a
% Specificity (95% PI)a
CFT and FPA CFT and RT
59.6 (49.8, 69.0) 63.1 (53.1, 72.1)
66.6 (56.0, 76.6) 78.9 (67.4, 88.8)
1
Expressed as median with and 95% probability interval (PI) in parentheses.
CHAPINAL ET AL.—DIAGNOSIS OF M. BOVIS IN FREE-RANGING BISON
sensitivity and specificity of the tests are necessary for evaluation, even though that information is usually scarce or of unknown accuracy. We focused on three tests (CFT and two serologic tests: FPA and RT) that differ in the immunologic response measured (cellular vs. humoral response and response to different antigens in the case of the two serologic tests), thus, representing most types of diagnostic tests currently available for this disease. Although the values of our prevalence and two specificity priors (FPA and CFT) had a large influence on the sensitivities of FPA and RT estimated by the Bayesian models, the rankings of test performance did not change regardless of the prior used. Therefore, we suggest that the sensitivity and specificity estimated in Bayesian models should serve the purpose of ranking different diagnostic tests to decide what test or combination of tests should be used in the field. According to the Bayesian analysis, the ranking of tests was CFT . RT . FPA based on sensitivity and RT . FPA . CFT based on specificity. Although results vary widely across studies and species, CFT often has equal or higher sensitivity than serologic tests have (Cousins and Florisson 2005). Himsworth et al. (2010) evaluated CFT, RT, and FPA in a captive M. bovis–infected bison herd and ranked the tests according to their sensitivity in the same order as in our study when histopathology was used as a gold standard. However, when culture was used as a gold standard, CFT was less sensitive than both FPA and RTA (the latter two were tied). In agreement with our study, Himsworth et al. (2010) found that RT was the test with the greatest specificity, regardless of the gold standard used. However, they found CFT was more specific than FPA, whereas CFT was the least specific test in our study. Some disagreement between the two studies was expected because the methodologies used were different and likely subject to different sources of bias. In turn, each
623
source of bias might affect each test to a different extent. Himsworth et al. (2010) used imperfectly sensitive tests as the gold standard in an infected population, factors that are known to underestimate specificity. In addition, individual bison in their study herd were tested by CFT repeatedly over time. The inoculation of PPD may boost antibody responses in other species, such as cattle (Waters et al. 2006) or llamas (Lama glama; Dean et al. 2009), which could result in an overestimation of the sensitivity and an underestimation of the specificity of serologic tests. According to both studies, RT outperformed FPA in sensitivity and specificity. This is not surprising because FPA measures the response to only one antigen (MPB70), which was less recognized by infected cattle than the three antigens (ESAT-6, CFP-10, and MPB83) used in RT (Waters et al. 2006). Furthermore, RT is an animal-side test that can be easily performed in the field, obtaining immediate results with little cost. Therefore, RT has some distinct advantages over FPA for use in wild bison. In agreement with Himsworth et al. (2010), the specificity of CFT and RT was greater than their sensitivity. For efficient pathogen management, diagnostic methods need to have high sensitivity and reasonable specificity. Sensitivity can be increased by combining two tests in parallel (Dohoo et al. 2009). In our study, the combination in parallel of CFT and RT ranked higher in sensitivity than did CFT alone. Considering the inverse relationship between cell-mediated and humoral responses to M. bovis (Plackett et al. 1989; Neill et al. 1994), it might be impossible to develop one serologic test that outperforms a combination of CFT and a serologic test in detecting animals at different stages of infection. Our sample size did not allow for assessment of changes in sensitivity and specificity related to age. Joly and Messier (2004) found that the apparent prevalence of bovine tuberculosis in bison increased
624
JOURNAL OF WILDLIFE DISEASES, VOL. 51, NO. 3, JULY 2015
with age. Further research is needed to investigate changes in the immune response to M. bovis over the lifetime of bison. Treanor et al. (2011) described Bayesian models to estimate the probability of brucellosis infection in bison based on the age of the animal and the serologic tests used. Decisions on what animals to cull can be made based on these probabilities. This type of model could have an important application to disease management, increasing the effectiveness of disease-control programs while supporting long-term conservation efforts. Given that gold standards for the detection of M. bovis are imperfect and difficult to obtain in the field, the use of Bayesian models to estimate sensitivity and specificity in the absence of a gold standard holds promise as a tool to rank tests and combinations of tests based on their performance. Combining a skin test with an animal-side serologic test, such as RT, increases sensitivity in the detection of M. bovis and thus, is a good approach to enhance the eradication or control of M. bovis in wild bison. ACKNOWLEDGMENTS
We thank the Canadian Food Inspection Agency (Nepean, Ontario, Canada), Konstantin Lyaschenko, and Todd Shury for conducting the diagnostic tests. We thank Amanda Salb, Jane Harms, Tim Evans, Trevor Evans, Michelle MacDonald, and Amanda Plante for technical assistance. This research was funded by the Natural Sciences and Engineering Research Council (Canada). D.O.J. was supported by the Dunemere Foundation. LITERATURE CITED Branscum AJ, Gardner IA, Johnson WO. 2005. Estimation of diagnostic-test sensitivity and specificity through Bayesian modeling. Prev Vet Med 68:145–163. Chapinal N, Elkin BT, Joly DO, Schumaker BA, Stephen C. 2012. Agreement between the caudal fold test and serological tests for the detection of Mycobacterium bovis infection in bison. Prev Vet Med 105:326–330. Cousins DV, Florisson N. 2005. A review of tests available for use in the diagnosis of tuberculosis
in non-bovine species. Rev Sci Tech Off Int Epizoot 24:1039–1059. Dean GS, Crawshaw TR, de la Rua-Domenech R, Farrant L, Greenwald R, Higgins RJ, Lyashchenko K, Vordermeier HM, Twomey DF. 2009. Use of serological techniques for diagnosis of Mycobacterium bovis infection in a llama herd. Vet Rec 165:323–324. Dohoo I, Martin W, Stryhn H. 2009. Veterinary epidemiologic research. 2nd Ed. VER Inc., Charlottetown, Prince Edward Island, Canada, 865 pp. Gardner IA, Stryhn H, Lind P, Collins MT. 2000. Conditional dependence between tests affects the diagnosis and surveillance of animal diseases. Prev Vet Med 45:107–122. Himsworth CG, Elkin BT, Nishi JS, Tasha E, Lyashchenko KP, Surujballi O, Turcotte C, Esfandiari J, Greenwald R, Leighton FA. 2010. Comparison of test performance and evaluation of novel immunoassays for tuberculosis in a captive herd of wood bison naturally infected with Mycobacterium bovis. J Wildl Dis 46:78–86. Joly DO, Messier F. 2004. Factors affecting apparent prevalence of tuberculosis and brucellosis in wood bison. J Anim Ecol 73:623–631. Monaghan ML, Doherty ML, Collins JD, Kazda JF, Quinn PJ. 1994. The tuberculin test. Vet Microbiol 40:111–124. Neill SD, Pollock JM, Bryson DB, Hanna J. 1994. Pathogenesis of Mycobacterium bovis infection in cattle. Vet Microbiol 40:41–52. Nishi JS, Shury T, Elkin BT. 2006. Wildlife reservoirs for bovine tuberculosis (Mycobacterium bovis) in Canada: Strategies for management and research. Vet Microbiol 112:325–338. Plackett P, Ripper J, Comer LA, Small K, deWitte K, Melville L, Hides S, Wood PR. 1989. An ELISA for the detection of anergic tuberculous cattle. Aust Vet J 66:15–19. Schumaker BA, Corso BA, Rhyan JC, Philo LM, Salman MD, Gardner IA. 2010. Evaluation of the fluorescence polarization assay for the detection of Brucella abortus antibodies in bison in a natural setting. Comp Immunol Microbiol Infect Dis 33:e119–e125. Spiegelhalter D, Thomas A, Best N, Lunn D. 2003. WinBUGS user manual, Version 1.4. http://www. mrc-bsu.cam.ac.uk/software/bugs. Accessed March 2015. Thoen CO, Lobue PA, Enarson DA, Kaneene JB, de Kantor IN. 2009. Tuberculosis: A re-emerging disease in animals and humans. Vet Ital 45: 135–81. Treanor J, Geremia C, Crowley P, Cox J, White P, Wallen R, Blanton D. 2011. Estimating probabilities of active brucellosis infection in Yellowstone bison through quantitative serology and tissue culture. J Appl Ecol 48:1324–1332. Waters WR, Palmer MV, Thacker TC, Bannantine JP, Vordermeier HM, Hewinson RG, Greenwald
CHAPINAL ET AL.—DIAGNOSIS OF M. BOVIS IN FREE-RANGING BISON
R, Esfandiari J, McNair J, Pollock JM, et al. 2006. Early antibody responses to experimental Mycobacterium bovis infection of cattle. Clin Vaccine Immunol 13:648–654. Waters W, Stevens G, Schoenbaum M, Orloski K, Robbe-Austerman S, Harris N, Hall S, Thomsen B, Wilson A, Brannian R, et al. 2011. Bovine
625
tuberculosis in a Nebraska herd of farmed elk and fallow deer: A failure of the tuberculin skin test and opportunities for serodiagnosis. Vet Med Int 2011:953985. Submitted for publication 27 September 2013. Accepted 30 January 2015.
BioOne (www.bioone.org) is a nonprofit, online aggregation of core research in the biological, ecological, and environmental sciences. BioOne provides a sustainable online platform for over 170 journals and books published by nonprofit societies, associations, museums, institutions, and presses. Your use of this PDF, the BioOne Web site, and all posted and associated content indicates your acceptance of BioOne’s Terms of Use, available at www.bioone.org/page/ terms_of_use. Usage of BioOne content is strictly limited to personal, educational, and non-commercial use. Commercial inquiries or rights and permissions requests should be directed to the individual publisher as copyright holder.
BioOne sees sustainable scholarly publishing as an inherently collaborative enterprise connecting authors, nonprofit publishers, academic institutions, research libraries, and research funders in the common goal of maximizing access to critical research.
DOI: 10.7589/2013-09-254
Journal of Wildlife Diseases, 51(3), 2015, pp. 619–625 # Wildlife Disease Association 2015
BAYESIAN ANALYSIS TO EVALUATE TESTS FOR THE DETECTION OF MYCOBACTERIUM BOVIS INFECTION IN FREE-RANGING WILD BISON (BISON BISON ATHABASCAE) IN THE ABSENCE OF A GOLD STANDARD Nu´ria Chapinal,1 Brant A. Schumaker,2,6 Damien O. Joly,3 Brett T. Elkin,4 and Craig Stephen5 1 Animal Welfare Program, University of British Columbia, 2357 Main Mall, Vancouver, British Columbia V6T 1Z4, Canada 2 Department of Veterinary Sciences, University of Wyoming, 1000 E University Avenue, Laramie, Wyoming 82070, USA 3 Metabiota, Unit 7, 1611 Bowen Road, Nanaimo, British Columbia V9S 1G5, Canada 4 Wildlife Division, Department of Environment and Natural Resources, Government of the Northwest Territories, 600, 5102 50th Avenue, Yellowknife, Northwest Territories X1A 3S8, Canada 5 Canadian Wildlife Health Cooperative, 52 Campus Drive, Saskatoon, Saskatchewan S7N 5B4, Canada 6 Corresponding author (email: [email protected])
ABSTRACT: We estimated the sensitivity and specificity of the caudal-fold skin test (CFT), the fluorescent polarization assay (FPA), and the rapid lateral-flow test (RT) for the detection of Mycobacterium bovis in free-ranging wild wood bison (Bison bison athabascae), in the absence of a gold standard, by using Bayesian analysis, and then used those estimates to forecast the performance of a pairwise combination of tests in parallel. In 1998–99, 212 wood bison from Wood Buffalo National Park (Canada) were tested for M. bovis infection using CFT and two serologic tests (FPA and RT). The sensitivity and specificity of each test were estimated using a three-test, one-population, Bayesian model allowing for conditional dependence between FPA and RT. The sensitivity and specificity of the combination of CFT and each serologic test in parallel were calculated assuming conditional independence. The test performance estimates were influenced by the prior values chosen. However, the rank of tests and combinations of tests based on those estimates remained constant. The CFT was the most sensitive test and the FPA was the least sensitive, whereas RT was the most specific test and CFT was the least specific. In conclusion, given the fact that gold standards for the detection of M. bovis are imperfect and difficult to obtain in the field, Bayesian analysis holds promise as a tool to rank tests and combinations of tests based on their performance. Combining a skin test with an animal-side serologic test, such as RT, increases sensitivity in the detection of M. bovis and is a good approach to enhance disease eradication or control in wild bison. Key words: Bayesian analysis, diagnostic tests, serological tests, tuberculin skin tests, wildlife disease, zoonosis.
infection from a population, particularly in free-ranging wildlife where handling individuals is difficult. Disease-eradication programs often involve regular testing of the population and subsequent removal of positive animals. Therefore, a detection method with high sensitivity is necessary to avoid releasing infected (false-negative) animals. In other situations, less-radical measures might be chosen to control, rather than eliminate, the disease to support conservation objectives. Managers may want to remove only animals that are clinically affected and more likely to be actively shedding bacteria, rather than all animals positive on serologic tests. The test or combination of tests selected should reflect
INTRODUCTION
Bovine tuberculosis, caused by Mycobacterium bovis, is a reemerging disease that affects wildlife, livestock, and humans (Thoen et al. 2009). It is endemic in free-ranging wood bison (Bison bison athabascae) populations in and around Wood Buffalo National Park (WBNP), Canada (Joly and Messier 2004), and these populations act as a reservoir of the pathogen (Nishi et al. 2006). Reliable detection methods (based on one or a combination of validated diagnostic tests) for tuberculosis in bison are essential to effectively implement wildlife and disease-management programs to control or eradicate 619
620
JOURNAL OF WILDLIFE DISEASES, VOL. 51, NO. 3, JULY 2015
the desired goals and should be customized to the particular management context. Bovine tuberculosis is commonly diagnosed in domestic and wild bovids with a single intradermal test (skin test), which measures a cell-mediated immune response to the intradermal injection of a purified protein derivative (PPD or tuberculin) prepared from M. bovis. Skin tests have been reported to have low sensitivity in bison (Himsworth et al. 2010). Different serologic tests, with varied sensitivity and specificity, have been developed for the detection of M. bovis in bison (Himsworth et al. 2010). These tests show poor agreement with the skin test (Himsworth et al. 2010; Chapinal et al. 2012), partly because the tests measure different immune responses (humoral vs. cell-mediated), which are predominant at different stages of the infection. Some studies in different wildlife species (Himsworth et al. 2010; Waters et al. 2011) have suggested combining a skin test with a serologic test in parallel (where one positive test would result in disease-positive classification) as a way to increase sensitivity in the detection of M. bovis. Assessing the sensitivity and specificity of the available diagnostic tests is necessary to determine what combination of tests would be more accurate. However, there is a dearth of well-validated data to assess accuracy in diagnostic tests for tuberculosis in bison. Culture of M. bovis continues to be the gold standard for evaluation of tests, despite its imperfect sensitivity and the difficulty in obtaining samples for culture from wild populations (Cousins and Florisson 2005). Other methods have been developed to assess sensitivity and specificity of diagnostic tests in the absence of a gold standard, such as maximum likelihood and Bayesian estimations, which merit further research. Bayesian estimations have recently been used for the evaluation of diagnostic tests for brucellosis in bison (Schumaker et al. 2010). These methods are flexible and can account for
factors such as dependence among test results, which is to be expected if tests measure similar biologic processes, such as serum antibody responses to infectious agents (Dohoo et al. 2009). Herein, we estimate the sensitivity and specificity of the caudal-fold skin test (CFT) and serologic tests for the detection of M. bovis in free-ranging wild wood bison in the absence of a gold standard by Bayesian analysis, and we use those estimates to forecast the performance of a pairwise combination of tests in parallel for the detection of M. bovis infection in free-ranging wild wood bison under different testing goals. MATERIALS AND METHODS Animals
This study is a retrospective analysis of historical test results derived from other research projects. We used 212 free-ranging wild wood bison (157 females and 55 males; 17% calves, 9% yearlings, 15% subadults, and 59% adults) from WBNP. These are the same animals used by Chapinal et al. (2012) and a subset of the 346 animals used by Joly and Messier (2004), but the results presented here include additional serologic tests that were not available at the time of those studies. Animals were captured and sampled in February 1998 and in February and March 1999. Details on animal handling are described by Joly and Messier (2004). Animals had not been vaccinated against any disease. Sampling and testing
The CFT was conducted as described by Joly and Messier (2004). Briefly, bison were injected with 0.1 mL of PPD tuberculin intradermally in the caudal fold and the injection site was inspected 72 h later (Monaghan et al. 1994). Bison were classified as positive if there was any visible or palpable change in the tissue at the point of injection. Blood samples were taken from each bison from the caudal or jugular vein, collected in serum separator tubes to facilitate clotting. Serum was removed by centrifuge within 12 h, and serum samples were frozen until serologic tests for M. bovis were conducted. Four different serologic tests were performed as described by Chapinal et al. (2012): 1) the fluorescent polarization assay (FPA), 2) the
CHAPINAL ET AL.—DIAGNOSIS OF M. BOVIS IN FREE-RANGING BISON
multiantigen print immunoassay (MAPIA), 3) the rapid lateral-flow test (RT; VetTB STAT-PAK, Chembio Diagnostic Systems, Inc., Melford, New York, USA), and 4) the dual-path platform test (DPP; DPP VetTB, Chembio). The MAPIA, RT, and DPP measure a very similar immune response (they share three antigens) and they show almost perfect agreement (Chapinal et al. 2012). This suggests a large conditional dependence among them and therefore, limited benefit in combining them for the diagnosis of M. bovis in bison. Therefore, of these three tests, only RT was considered for further analysis because it is the only animal-side test currently available.
621
T ABLE 1. Informative prior distributions for a three-test, one-population, Bayesian model of a caudal fold test (CFT), fluorescent polarization assay (FPA), and rapid test (RT) for the detection of Mycobacterium bovis in 212 free-ranging wild wood bison (Bison bison athabascae) in Wood Buffalo National Park, Canada.
Parameter
b Distribution
Expected values (%)
Prevalence CFT sensitivity CFT specificity RT30 specificity FPA specificity
b(42.75, 42.75) b(30.25, 30.25) b(53.2, 7.3) b(225, 4) b(105, 29)
50 50 87.9 98.3 78.4
Data analysis
Bayesian models allow for the probabilistic estimation of test performance characteristics by incorporating previous information about the diagnostic tests and the prevalence of variable stages of infection in the study population. In biologic terms, animals under different stages of infection were considered ‘‘positive’’ for M. bovis if they showed a sufficient immune response to be consistent with that particular infection stage. The sensitivity and specificity of CFT, FPA, and RT were estimated using a three-test, one-population, Bayesian model allowing for conditional dependence between FPA and RT, as described in section 3.4 of Branscum et al. (2005). The CFT measures a different immune response and shows no agreement with serologic tests (Himsworth et al. 2010; Chapinal et al. 2012). Therefore, CFT was considered independent from FPA and RT in the Bayesian models. Beta prior distributions for the specificity and sensitivity of CFT were calculated from the tests results of Himsworth et al. (2010; Table 1). Beta prior distributions for the specificity of FPA and RT were calculated from the tests results of 227 animals sampled from 1990 to 2007 in the tuberculosis-free Mackenzie Bison Sanctuary herd (Canada; Table 1). The beta prior distribution of the prevalence of infected animals was calculated from the test results of Joly and Messier (2004), who estimated an apparent prevalence of infected animals at 50% when interpreting CFT and FPA in parallel (i.e., animals positive to either test were considered infected with M. bovis; Table 1). Posterior inferences were based on 50,000 iterations of each model and were estimated using freeware WinBugs (Spiegelhalter et al. 2003). Five initial chains were used and convergence of the model was assessed through sample trace, history, and Gelman-Rubin plots.
Additionally, the joint sensitivities and specificities of CFT with each of the two serologic tests was calculated from section 3.1 of Gardner et al. (2000) and iterated to determine a point estimate and 95% probability interval. A complete sensitivity analysis was performed, increasing and decreasing the informative prior distributions by 5% and 10% and determining their influence on the posterior distributions of the prevalence in addition to all sensitivity and specificity values. Additionally, we used prevalence estimates of 25% and 5% to calculate the prevalence of the prior distribution to determine the importance of the prevalence prior on the posterior distributions. Finally, diffuse (b[1,1]) priors were applied individually and together to determine the most frequent outcome of our analyses. For all analyses, the model was iterated 50,000 times and all posterior point and interval estimates were obtained. Before convergence, 5,000 iterations were used for burn-in and then discarded. RESULTS
Table 2 shows the cross-classified results for M. bovis from CFT, FPA, and RT. TABLE 2. Cross-classified results from caudal fold test (CFT), fluorescent polarization assay (FPA), and rapid test (RT) for the detection of Mycobacterium bovis in 212 free-ranging wild wood bison (Bison bison athabascae) in Wood Buffalo National Park, Canada. CFT+ (n5100) RT
FPA+ FPA2
6 9
+
RT
CFT2 (n5112) 2
3 82
RT+
RT2
2 9
2 99
622
JOURNAL OF WILDLIFE DISEASES, VOL. 51, NO. 3, JULY 2015
TABLE 3. Summary of a three-test, one-population, Bayesian model for Mycobacterium bovis infection in 212 free-ranging wild wood bison (Bison bison athabascae) in Wood Buffalo National Park, Canada, assuming conditional independence between the caudal fold test (CFT) and the two serologic tests (fluorescent polarization assay [FPA] and rapid test [RT]), and conditional dependence between FPA and RT. Parameter
Estimate
a
CFT sensitivity % (95% PI) CFT specificity % (95% PI)a FPA sensitivity % (95% PI)a FPA specificity % (95% PI)a RT sensitivity % (95% PI)a RT specificity % (95% PI)a Prevalence in the sample % (95% PI)a Sensitivity covariance of FPA and RT Specificity covariance of FPA and RT % iterations CFT sensitivity was higher than FPA sensitivity % iterations CFT sensitivity was higher than RT sensitivity % iterations RT sensitivity was higher than FPA sensitivity % iterations CFT specificity was higher than FPA specificity % iterations CFT specificity was higher than RT specificity % iterations RT specificity was higher than FPA specificity a
57.6 80.3 4.4 83.2 12.7 98.4 57.5 2.0 25.5 100 100 97.6 31.9 0 100
(48.2, 67.0) (68.7, 90.2) (0.4, 10.6) (77.2, 88.2) (4.2, 21.9) (96.2, 99.5) (47.4, 67.0) (20.1, 6.4) (217.9, 1.1)
Expressed as median with 95% probability interval (PI) in parentheses.
Based on the three-test, one-population, Bayesian model (Table 3), CFT was the most sensitive test and FPA was the least sensitive, whereas RT was the most specific test and CFT was the least specific. Table 4 summarizes the sensitivity and specificity of combinations of CFT with either FPA or RT in parallel. The posterior point and interval estimates for test performance and prevalence were most sensitive to changes in the specificity priors for FPA and CFT, followed by changes to the prevalence prior. The model results were comparatively insensitive to changes to the prior estimates for specificity of RT and sensitivity of CFT. However, even using all diffuse prior distributions, the rankings of test sensitivities and specificities did not change.
DISCUSSION
The assessment of the sensitivity and specificity of diagnostics tests for bovine tuberculosis is challenging because a perfect gold standard for live-animal testing does not exist. Not surprisingly, a consensus on a definition of a ‘‘disease-positive’’ animal to use in the field has yet to be achieved. Culture, histology, and even CFT have been use as (pseudo)-gold standards for the evaluation of diagnostic tests for bovine tuberculosis in wild bison and other wildlife species (Cousins and Florisson 2005) but, to our knowledge, Bayesian methods have not been used for that purpose. Bayesian analyses have limitations or biases because prior information about the prevalence of the disease in the population and about the
TABLE 4. Sensitivity and specificity of combinations in parallel of the caudal fold test (CFT) and two serologic tests (fluorescent polarization assay [FPA] and rapid test [RT]). Conditional independence between CFT and the two serological tests was assumed. Tests
% Sensitivity (95% PI)a
% Specificity (95% PI)a
CFT and FPA CFT and RT
59.6 (49.8, 69.0) 63.1 (53.1, 72.1)
66.6 (56.0, 76.6) 78.9 (67.4, 88.8)
1
Expressed as median with and 95% probability interval (PI) in parentheses.
CHAPINAL ET AL.—DIAGNOSIS OF M. BOVIS IN FREE-RANGING BISON
sensitivity and specificity of the tests are necessary for evaluation, even though that information is usually scarce or of unknown accuracy. We focused on three tests (CFT and two serologic tests: FPA and RT) that differ in the immunologic response measured (cellular vs. humoral response and response to different antigens in the case of the two serologic tests), thus, representing most types of diagnostic tests currently available for this disease. Although the values of our prevalence and two specificity priors (FPA and CFT) had a large influence on the sensitivities of FPA and RT estimated by the Bayesian models, the rankings of test performance did not change regardless of the prior used. Therefore, we suggest that the sensitivity and specificity estimated in Bayesian models should serve the purpose of ranking different diagnostic tests to decide what test or combination of tests should be used in the field. According to the Bayesian analysis, the ranking of tests was CFT . RT . FPA based on sensitivity and RT . FPA . CFT based on specificity. Although results vary widely across studies and species, CFT often has equal or higher sensitivity than serologic tests have (Cousins and Florisson 2005). Himsworth et al. (2010) evaluated CFT, RT, and FPA in a captive M. bovis–infected bison herd and ranked the tests according to their sensitivity in the same order as in our study when histopathology was used as a gold standard. However, when culture was used as a gold standard, CFT was less sensitive than both FPA and RTA (the latter two were tied). In agreement with our study, Himsworth et al. (2010) found that RT was the test with the greatest specificity, regardless of the gold standard used. However, they found CFT was more specific than FPA, whereas CFT was the least specific test in our study. Some disagreement between the two studies was expected because the methodologies used were different and likely subject to different sources of bias. In turn, each
623
source of bias might affect each test to a different extent. Himsworth et al. (2010) used imperfectly sensitive tests as the gold standard in an infected population, factors that are known to underestimate specificity. In addition, individual bison in their study herd were tested by CFT repeatedly over time. The inoculation of PPD may boost antibody responses in other species, such as cattle (Waters et al. 2006) or llamas (Lama glama; Dean et al. 2009), which could result in an overestimation of the sensitivity and an underestimation of the specificity of serologic tests. According to both studies, RT outperformed FPA in sensitivity and specificity. This is not surprising because FPA measures the response to only one antigen (MPB70), which was less recognized by infected cattle than the three antigens (ESAT-6, CFP-10, and MPB83) used in RT (Waters et al. 2006). Furthermore, RT is an animal-side test that can be easily performed in the field, obtaining immediate results with little cost. Therefore, RT has some distinct advantages over FPA for use in wild bison. In agreement with Himsworth et al. (2010), the specificity of CFT and RT was greater than their sensitivity. For efficient pathogen management, diagnostic methods need to have high sensitivity and reasonable specificity. Sensitivity can be increased by combining two tests in parallel (Dohoo et al. 2009). In our study, the combination in parallel of CFT and RT ranked higher in sensitivity than did CFT alone. Considering the inverse relationship between cell-mediated and humoral responses to M. bovis (Plackett et al. 1989; Neill et al. 1994), it might be impossible to develop one serologic test that outperforms a combination of CFT and a serologic test in detecting animals at different stages of infection. Our sample size did not allow for assessment of changes in sensitivity and specificity related to age. Joly and Messier (2004) found that the apparent prevalence of bovine tuberculosis in bison increased
624
JOURNAL OF WILDLIFE DISEASES, VOL. 51, NO. 3, JULY 2015
with age. Further research is needed to investigate changes in the immune response to M. bovis over the lifetime of bison. Treanor et al. (2011) described Bayesian models to estimate the probability of brucellosis infection in bison based on the age of the animal and the serologic tests used. Decisions on what animals to cull can be made based on these probabilities. This type of model could have an important application to disease management, increasing the effectiveness of disease-control programs while supporting long-term conservation efforts. Given that gold standards for the detection of M. bovis are imperfect and difficult to obtain in the field, the use of Bayesian models to estimate sensitivity and specificity in the absence of a gold standard holds promise as a tool to rank tests and combinations of tests based on their performance. Combining a skin test with an animal-side serologic test, such as RT, increases sensitivity in the detection of M. bovis and thus, is a good approach to enhance the eradication or control of M. bovis in wild bison. ACKNOWLEDGMENTS
We thank the Canadian Food Inspection Agency (Nepean, Ontario, Canada), Konstantin Lyaschenko, and Todd Shury for conducting the diagnostic tests. We thank Amanda Salb, Jane Harms, Tim Evans, Trevor Evans, Michelle MacDonald, and Amanda Plante for technical assistance. This research was funded by the Natural Sciences and Engineering Research Council (Canada). D.O.J. was supported by the Dunemere Foundation. LITERATURE CITED Branscum AJ, Gardner IA, Johnson WO. 2005. Estimation of diagnostic-test sensitivity and specificity through Bayesian modeling. Prev Vet Med 68:145–163. Chapinal N, Elkin BT, Joly DO, Schumaker BA, Stephen C. 2012. Agreement between the caudal fold test and serological tests for the detection of Mycobacterium bovis infection in bison. Prev Vet Med 105:326–330. Cousins DV, Florisson N. 2005. A review of tests available for use in the diagnosis of tuberculosis
in non-bovine species. Rev Sci Tech Off Int Epizoot 24:1039–1059. Dean GS, Crawshaw TR, de la Rua-Domenech R, Farrant L, Greenwald R, Higgins RJ, Lyashchenko K, Vordermeier HM, Twomey DF. 2009. Use of serological techniques for diagnosis of Mycobacterium bovis infection in a llama herd. Vet Rec 165:323–324. Dohoo I, Martin W, Stryhn H. 2009. Veterinary epidemiologic research. 2nd Ed. VER Inc., Charlottetown, Prince Edward Island, Canada, 865 pp. Gardner IA, Stryhn H, Lind P, Collins MT. 2000. Conditional dependence between tests affects the diagnosis and surveillance of animal diseases. Prev Vet Med 45:107–122. Himsworth CG, Elkin BT, Nishi JS, Tasha E, Lyashchenko KP, Surujballi O, Turcotte C, Esfandiari J, Greenwald R, Leighton FA. 2010. Comparison of test performance and evaluation of novel immunoassays for tuberculosis in a captive herd of wood bison naturally infected with Mycobacterium bovis. J Wildl Dis 46:78–86. Joly DO, Messier F. 2004. Factors affecting apparent prevalence of tuberculosis and brucellosis in wood bison. J Anim Ecol 73:623–631. Monaghan ML, Doherty ML, Collins JD, Kazda JF, Quinn PJ. 1994. The tuberculin test. Vet Microbiol 40:111–124. Neill SD, Pollock JM, Bryson DB, Hanna J. 1994. Pathogenesis of Mycobacterium bovis infection in cattle. Vet Microbiol 40:41–52. Nishi JS, Shury T, Elkin BT. 2006. Wildlife reservoirs for bovine tuberculosis (Mycobacterium bovis) in Canada: Strategies for management and research. Vet Microbiol 112:325–338. Plackett P, Ripper J, Comer LA, Small K, deWitte K, Melville L, Hides S, Wood PR. 1989. An ELISA for the detection of anergic tuberculous cattle. Aust Vet J 66:15–19. Schumaker BA, Corso BA, Rhyan JC, Philo LM, Salman MD, Gardner IA. 2010. Evaluation of the fluorescence polarization assay for the detection of Brucella abortus antibodies in bison in a natural setting. Comp Immunol Microbiol Infect Dis 33:e119–e125. Spiegelhalter D, Thomas A, Best N, Lunn D. 2003. WinBUGS user manual, Version 1.4. http://www. mrc-bsu.cam.ac.uk/software/bugs. Accessed March 2015. Thoen CO, Lobue PA, Enarson DA, Kaneene JB, de Kantor IN. 2009. Tuberculosis: A re-emerging disease in animals and humans. Vet Ital 45: 135–81. Treanor J, Geremia C, Crowley P, Cox J, White P, Wallen R, Blanton D. 2011. Estimating probabilities of active brucellosis infection in Yellowstone bison through quantitative serology and tissue culture. J Appl Ecol 48:1324–1332. Waters WR, Palmer MV, Thacker TC, Bannantine JP, Vordermeier HM, Hewinson RG, Greenwald
CHAPINAL ET AL.—DIAGNOSIS OF M. BOVIS IN FREE-RANGING BISON
R, Esfandiari J, McNair J, Pollock JM, et al. 2006. Early antibody responses to experimental Mycobacterium bovis infection of cattle. Clin Vaccine Immunol 13:648–654. Waters W, Stevens G, Schoenbaum M, Orloski K, Robbe-Austerman S, Harris N, Hall S, Thomsen B, Wilson A, Brannian R, et al. 2011. Bovine
625
tuberculosis in a Nebraska herd of farmed elk and fallow deer: A failure of the tuberculin skin test and opportunities for serodiagnosis. Vet Med Int 2011:953985. Submitted for publication 27 September 2013. Accepted 30 January 2015.
