G Model
ARTICLE IN PRESS
JVAC-15351; No. of Pages 4
Vaccine xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
Brief report
An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty Deepa Joshi ∗ , Emily Alsentzer, Kathryn Edwards, Allison Norton, Sarah Elizabeth Williams Vanderbilt University Medical Center, Light Hall, 2215 Garland Avenue, Mailbox #43, Nashville, TN 37232, USA
a r t i c l e
i n f o
Article history: Received 22 January 2014 Received in revised form 21 March 2014 Accepted 22 April 2014 Available online xxx Keywords: Brighton Collaboration Adverse events following immunization Vaccine safety monitoring
a b s t r a c t The Brighton Collaboration is a global research network focused on vaccine safety. The Collaboration has created case definitions to determine diagnostic certainty for several adverse events. Currently nested within multi-page publications, these definitions can be cumbersome for use. We report the results of a randomized trial in which the case definition for anaphylaxis was converted into a user-friendly algorithm and compared the algorithm with the standard case definition. The primary outcomes were efficiency and accuracy. Forty medical students determined the Brighton Level of diagnostic certainty of a sample case of anaphylaxis using either the algorithm or the original case definition. Most participants in both groups selected the correct Brighton Level. Participants using the algorithm required significantly less time to review the case and determine the level of diagnostic certainty [mean difference = 107 s (95% CI: 13–200; p = 0.026)], supporting that the algorithm was more efficient without impacting accuracy. © 2014 Elsevier Ltd. All rights reserved.
1. Introduction Immunizations are powerful public health interventions that have been very effective in reducing global disease burden [1]. Although vaccines are generally safe, there are certain risks associated with their administration. While adverse events following immunization (AEFI), such as anaphylaxis, are rare, efficient systems to monitor AEFIs are essential to systematically assess vaccine safety. The Brighton Collaboration is a non-profit, international research network that provides standardized, validated, and objective case definitions for monitoring vaccine safety [2]. The definitions provide clinical and diagnostic criteria to allow AEFIs to be assigned to one of three levels of “diagnostic certainty”; a Level 1 indicates the highest level of confidence that an AEFI meets the corresponding diagnosis. The use of these standardized case definitions in research and clinical settings will more precisely
Abbreviations: AEFI, adverse events following immunization; LMICs, low to middle income countries; GVSI, Global Vaccine Safety Initiative. ∗ Corresponding author. Tel.: +1 402 305 8135. E-mail addresses: [email protected] (D. Joshi), [email protected] (E. Alsentzer), [email protected] (K. Edwards), [email protected] (A. Norton), [email protected] (S.E. Williams).
characterize events, leading to a better understanding of the true risk of AEFIs [3]. Because the case definition format is generally a footnoted table nested within a 10–20 page journal article, the goal of this study was to convert one Brighton case definition into an algorithm, and evaluate the efficiency and accuracy of the algorithm compared to the original case definition. 2. Materials and methods 2.1. Algorithm development In July 2012 the Brighton Collaboration case definition of anaphylaxis [4] was reviewed. Key clinical criteria that distinguished the levels of diagnostic certainty were abstracted and using SmartDraw software [5] were transposed into a step-wise algorithm that guided users to the appropriate level (Fig. 1). The algorithm was tested by applying it to cases of anaphylaxis identified through a Pubmed search. The algorithm was reviewed by one pediatrician and one allergist to verify that the criteria matched the original case definition. 2.2. Sample case The sample case was selected from clinical cases presented to the Clinical Immunization Safety Assessment network (CISA)
http://dx.doi.org/10.1016/j.vaccine.2014.04.070 0264-410X/© 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
G Model JVAC-15351; No. of Pages 4 2
ARTICLE IN PRESS D. Joshi et al. / Vaccine xxx (2014) xxx–xxx
Fig. 1. Algorithm for anaphylaxis developed from the Brighton Collaboration case definition.
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
G Model
ARTICLE IN PRESS
JVAC-15351; No. of Pages 4
D. Joshi et al. / Vaccine xxx (2014) xxx–xxx
Baby B is a 12-month-old male with a history of atopic dermas, egg allergy (prick +), and report of hives aer eang carrots. The paent visited the allergist’s office to receive his 1st influenza vaccine. Due to the paent’s history of egg allergy, the Influenza, Inacvated Vaccine (TIV) was administered per 2-part protocol to right quadricep: 1. TIV 0.05cc Intramuscularly: The paent was noted to have full-strength, which was monitored for 30 minutes. No other adverse reacon was noted. 2. TIV 0.20cc Intramuscularly: Within 30 minutes of the 2nd dose, the paent experienced urcaria of neck, anterior torso, and both arms. He also became fussy and started crying. Baby B had increased work of breathing, coughing, stridor, retracons, mild erythema, and angioedema of uvula. The following lab values were noted:
Before First Vaccine Aer Second Vaccine
Respiratory Rate
O2 Saturaon
Pulse
24 bpm
97%
124bpm
Systolic Blood Pressure 110
42 bpm
93%
137bpm
176
The paent was treated with epinephrine intramuscularly along with cerizine, prednisolone, and albuterol. The child went home, and the night aer the clinic visit, the child experienced diarrhea, mild wheezing, and a blotchy rash. He was treated in the clinic with albuterol and Benadryl. The family reports that the child had not eaten any eggs, nuts, or carrots, and there were no changes in the home. Fig. 2. Sample case.
(Fig. 2) [6,7] and reviewed by a board-certified allergist using both the standard case definition and the algorithm to determine the level of diagnostic certainty; the case was determined to meet Brighton Level 1 criteria. 2.3. Study design Second year Vanderbilt medical students were recruited for participation. The students had no prior experience reading a Brighton case definition. Students signed up electronically for time slots after receiving an electronic mailing introducing the study. Participants who signed up received a reminder electronic mailing 1 h prior to their scheduled time. Written, informed consent was obtained. Participants were provided a small financial compensation. The Vanderbilt Institutional Review Board approved the study. Using block randomization, participants were randomized to receive 1 of 2 resources: the original case definition (nested in the 10 page journal article [8]) or the algorithm (Fig. 1). Prior to study onset, students received identical introduction sheets explaining (1) the purpose of the study and (2) the goal to determine the level of diagnostic certainty for the sample case using the provided resource. For the original case definition [8], participants were directed on the first page of the printed article to turn to a table on a later page that summarized the criteria. The second resource, the algorithm, was a two-page document that led the user through a series of questions until the appropriate level of diagnostic certainty was reached (Fig. 1). Additional instructions were not provided. Participant identification numbers were assigned to each study participant sequentially. Both participants and study staff were blinded to the assigned resource until the participants began the study. Participants were asked to use the provided resource to select the level of diagnostic certainty (1, 2, 3 or insufficient information) for the sample case. Each participant was given an electronic timer. The study staff read a standard instruction script when all participants arrived and advised participants to start their timers at the end of the verbal instructions and to stop their timers immediately after marking their choice. Each participant’s time was recorded by study staff, and all study documents were collected at the end of each session.
3
Table 1 Assignment of Brighton Level of diagnostic certainty for sample case by resource used. Brighton Level
Case definition
Algorithm
Total
1 2 3
16 (80%) 2 (10%) 2 (10%)
15 (75%) 5 (25%) 0
31 (77.5%) 7 (17.5%) 2 (5%)
Total
20
20
40
Fisher’s exact; p = 0.246.
2.4. Statistical analysis A sample size of 17 participants in each study arm was estimated to provide 80% power to detect a significant difference in the time to complete the study between the two resources, assuming a standard deviation of 10 min and an alpha of 0.05. To evaluate the efficiency of the algorithm compared to the standard case definition, a comparison of the average time required to determine the level of diagnostic certainty for the sample case was achieved using a two group, two sided t-test. The algorithm’s ability to achieve the same level of diagnostic certainty for the sample case as the case definition was analyzed using Fisher’s exact test. 3. Results Forty participants enrolled in the study; 20 in each study arm. The average age of participants was 23.7 years overall [24.1 years for the case definition group, 23.7 years for the algorithm (p = 0.48)]. All participants completed the study and selected a level of diagnostic certainty (Table 1). The majority of participants in both study arms selected the correct Brighton Level (80% using the case definition and 75% using the algorithm; p = 0.246). Among all participants who incorrectly identified the case as not meeting Level 1, those using the algorithm all chose Level 2 (n = 5) whereas those using the case definition chose Level 3 (n = 2) as well as Level 2 (n = 2). No participants selected “Insufficient Information” as a level. The mean time for participants to review the sample case and select a Brighton Level was 415 s (95% CI: 341–490) using the case definition, and 309 s (95% CI: 253–364) using the algorithm, representing a significant difference in the mean time to review the case and determine the level by resource used [mean difference = 107 s (95% CI: 13–200; p = 0.026)]. 4. Discussion This study supports the hypothesis that an algorithm developed using the Brighton Collaboration case definition for an AEFI can result in the assignment of an appropriate level of diagnostic certainty in a significantly shorter period of time. Because our study design directed participants to the criteria table within the case definition journal article, the time required to locate the criteria for a researcher unfamiliar with the current journal format for Brighton definitions was eliminated. If we had not included this instruction, the difference in time to assign a level between resources may have been much greater. Given that most people who refer to the case definitions are familiar with the structure, this design best replicates the use of the Brighton case definitions and most accurately demonstrates the time difference between the two resources. The development of more efficient, practical tools for assessing AEFI is needed. New vaccine products used globally lead to a greater demand for vaccine safety surveillance, particularly in low to middle income countries (LMICs) [9]. This is supported by the World Health Organization’s Global Vaccine Safety Initiative (GVSI)
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
G Model JVAC-15351; No. of Pages 4
ARTICLE IN PRESS D. Joshi et al. / Vaccine xxx (2014) xxx–xxx
4
in 2012, whose objectives include the development of internationally harmonized tools to support vaccine surveillance in LMICs [10]. Aggregating data across multiple surveillance systems would allow for more statistical power and a greater probability of identifying rare AEFI; however, assembling of data is only possible if AEFI are based on similar criteria [10]. Brighton case definitions allow for standardization of AEFI reporting criteria, and more efficient methods for AEFI definition would enhance this effort, both in LMICs and in countries with strong vaccine surveillance systems. Our study has several limitations. The sample size is small, and the study may not be generalizable as all participants were medical students. However, many users of Brighton case definitions will be healthcare providers who have completed medical school training. Another potential limitation is that the participants completed the study in the same room as up to four other participants. When a participant stopped their timer, it beeped, signifying to all other participants in the room that the individual had completed the study. Thus, some participants may have rushed to finish the study after hearing another participant’s timer beep. Additionally, the inaccurate assignment of the level by a few participants using the algorithm may be a result of its format. In the algorithm’s listing of minor respiratory criteria, “Stridor” appears on its own line (Fig. 1). Participants may not have read the complete statement (“difficulty breathing without wheeze or stridor”) and incorrectly assigned the case a Brighton Level 2. Future research should include testing the efficiency and accuracy of the algorithm by application to multiple cases with different levels of diagnostic certainty and with participants with various training. Furthermore, since all participants who incorrectly identified the case as not meeting Level 1 using the algorithm all chose Level 2 (n = 5) whereas those using the case definition table chose Level 3 (n = 2) and Level 2 (n = 2), it would be interesting to elucidate the rationale for this inaccurate determination and thereafter correct the algorithm to avoid this error. 5. Summary An algorithm developed from the Brighton Collaboration case definition for anaphylaxis was more efficient in determining the level of diagnostic certainty for a case of anaphylaxis, without impacting accuracy. Easily disseminated methods for identifying
AEFIs and systematic methods for analysis would allow for enhanced comparability of vaccine safety data. Conflicts of interest The authors have no conflicts of interest relevant to this article to disclose. The authors have no financial relationships relevant to this article to disclose. Acknowledgements The Vanderbilt Institute for Clinical and Translational Research grant supported this project (UL1 TR000445 from NCATS/NIH). Study data were collected and managed using REDCap electronic data capture tools hosted at Vanderbilt University. References [1] CDC. Impact of vaccines universally recommended for children – United States, 1990–1998. MMWR Morb Mortal Wkly Rep 1999;48(April (12)):243–8. PubMed PMID: 10220251. [2] The Brighton Collaboration. Available from: https://brightoncollaboration.org/ internet/en/index.html [23.10.10]. [3] Bonhoeffer J, Kohl K, Chen R, Duclos P, Heijbel H, Heininger U, et al. The Brighton Collaboration: addressing the need for standardized case definitions of adverse events following immunization (AEFI). Vaccine 2002;21(December (3–4)):298–302. PubMed PMID: 12450705. [4] Ruggeberg JU, Gold MS, Bayas JM, Blum MD, Bonhoeffer J, Friedlander S, et al. Anaphylaxis: case definition and guidelines for data collection, analysis, and presentation of immunization safety data. Vaccine 2007;25(August (31)):5675–84. PubMed PMID: 17448577. [5] SmartDraw; 2013. Available from: http://www.smartdraw.com/ [10.10.13]. [6] LaRussa P, Edwards K, Dekker C, Klein N, Halsey N, Marchant C, et al. Understanding the role of human variation in vaccine adverse events: the Clinical Immunization Safety Assessment (CISA) network. Pediatrics 2011;127n3(March (Vaccine Safety Supplement)). [7] CDC. Vaccine safety – Clinical Immunization Safety Assessment (CISA) network. Available from: http://www.cdc.gov/vaccinesafety/Activities/cisa.html [updated 02/25/201008/04/2010]. [8] Gold MS, Gidudu J, Erlewyn-Lajeunesse M, Law B, Brighton Collaboration Working Group on A. Can the Brighton Collaboration case definitions be used to improve the quality of Adverse Event Following Immunization (AEFI) reporting? Anaphylaxis as a case study. Vaccine 2010;28(June (28)):4487–98. PubMed PMID: 20434547. [9] Amarasinghe A, Black S, Bonhoeffer J, Carvalho SM, Dodoo A, Eskola J, et al. Effective vaccine safety systems in all countries: a challenge for more equitable access to immunization. Vaccine 2013;31(April (Suppl. 2)):B108–14. PubMed PMID: 23598471. [10] WHO Global Vaccine Safety Blueprint; 2012. Available from: http://www.who. int/vaccine safety/en/ [10.10.2013].
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
ARTICLE IN PRESS
JVAC-15351; No. of Pages 4
Vaccine xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
Brief report
An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty Deepa Joshi ∗ , Emily Alsentzer, Kathryn Edwards, Allison Norton, Sarah Elizabeth Williams Vanderbilt University Medical Center, Light Hall, 2215 Garland Avenue, Mailbox #43, Nashville, TN 37232, USA
a r t i c l e
i n f o
Article history: Received 22 January 2014 Received in revised form 21 March 2014 Accepted 22 April 2014 Available online xxx Keywords: Brighton Collaboration Adverse events following immunization Vaccine safety monitoring
a b s t r a c t The Brighton Collaboration is a global research network focused on vaccine safety. The Collaboration has created case definitions to determine diagnostic certainty for several adverse events. Currently nested within multi-page publications, these definitions can be cumbersome for use. We report the results of a randomized trial in which the case definition for anaphylaxis was converted into a user-friendly algorithm and compared the algorithm with the standard case definition. The primary outcomes were efficiency and accuracy. Forty medical students determined the Brighton Level of diagnostic certainty of a sample case of anaphylaxis using either the algorithm or the original case definition. Most participants in both groups selected the correct Brighton Level. Participants using the algorithm required significantly less time to review the case and determine the level of diagnostic certainty [mean difference = 107 s (95% CI: 13–200; p = 0.026)], supporting that the algorithm was more efficient without impacting accuracy. © 2014 Elsevier Ltd. All rights reserved.
1. Introduction Immunizations are powerful public health interventions that have been very effective in reducing global disease burden [1]. Although vaccines are generally safe, there are certain risks associated with their administration. While adverse events following immunization (AEFI), such as anaphylaxis, are rare, efficient systems to monitor AEFIs are essential to systematically assess vaccine safety. The Brighton Collaboration is a non-profit, international research network that provides standardized, validated, and objective case definitions for monitoring vaccine safety [2]. The definitions provide clinical and diagnostic criteria to allow AEFIs to be assigned to one of three levels of “diagnostic certainty”; a Level 1 indicates the highest level of confidence that an AEFI meets the corresponding diagnosis. The use of these standardized case definitions in research and clinical settings will more precisely
Abbreviations: AEFI, adverse events following immunization; LMICs, low to middle income countries; GVSI, Global Vaccine Safety Initiative. ∗ Corresponding author. Tel.: +1 402 305 8135. E-mail addresses: [email protected] (D. Joshi), [email protected] (E. Alsentzer), [email protected] (K. Edwards), [email protected] (A. Norton), [email protected] (S.E. Williams).
characterize events, leading to a better understanding of the true risk of AEFIs [3]. Because the case definition format is generally a footnoted table nested within a 10–20 page journal article, the goal of this study was to convert one Brighton case definition into an algorithm, and evaluate the efficiency and accuracy of the algorithm compared to the original case definition. 2. Materials and methods 2.1. Algorithm development In July 2012 the Brighton Collaboration case definition of anaphylaxis [4] was reviewed. Key clinical criteria that distinguished the levels of diagnostic certainty were abstracted and using SmartDraw software [5] were transposed into a step-wise algorithm that guided users to the appropriate level (Fig. 1). The algorithm was tested by applying it to cases of anaphylaxis identified through a Pubmed search. The algorithm was reviewed by one pediatrician and one allergist to verify that the criteria matched the original case definition. 2.2. Sample case The sample case was selected from clinical cases presented to the Clinical Immunization Safety Assessment network (CISA)
http://dx.doi.org/10.1016/j.vaccine.2014.04.070 0264-410X/© 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
G Model JVAC-15351; No. of Pages 4 2
ARTICLE IN PRESS D. Joshi et al. / Vaccine xxx (2014) xxx–xxx
Fig. 1. Algorithm for anaphylaxis developed from the Brighton Collaboration case definition.
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
G Model
ARTICLE IN PRESS
JVAC-15351; No. of Pages 4
D. Joshi et al. / Vaccine xxx (2014) xxx–xxx
Baby B is a 12-month-old male with a history of atopic dermas, egg allergy (prick +), and report of hives aer eang carrots. The paent visited the allergist’s office to receive his 1st influenza vaccine. Due to the paent’s history of egg allergy, the Influenza, Inacvated Vaccine (TIV) was administered per 2-part protocol to right quadricep: 1. TIV 0.05cc Intramuscularly: The paent was noted to have full-strength, which was monitored for 30 minutes. No other adverse reacon was noted. 2. TIV 0.20cc Intramuscularly: Within 30 minutes of the 2nd dose, the paent experienced urcaria of neck, anterior torso, and both arms. He also became fussy and started crying. Baby B had increased work of breathing, coughing, stridor, retracons, mild erythema, and angioedema of uvula. The following lab values were noted:
Before First Vaccine Aer Second Vaccine
Respiratory Rate
O2 Saturaon
Pulse
24 bpm
97%
124bpm
Systolic Blood Pressure 110
42 bpm
93%
137bpm
176
The paent was treated with epinephrine intramuscularly along with cerizine, prednisolone, and albuterol. The child went home, and the night aer the clinic visit, the child experienced diarrhea, mild wheezing, and a blotchy rash. He was treated in the clinic with albuterol and Benadryl. The family reports that the child had not eaten any eggs, nuts, or carrots, and there were no changes in the home. Fig. 2. Sample case.
(Fig. 2) [6,7] and reviewed by a board-certified allergist using both the standard case definition and the algorithm to determine the level of diagnostic certainty; the case was determined to meet Brighton Level 1 criteria. 2.3. Study design Second year Vanderbilt medical students were recruited for participation. The students had no prior experience reading a Brighton case definition. Students signed up electronically for time slots after receiving an electronic mailing introducing the study. Participants who signed up received a reminder electronic mailing 1 h prior to their scheduled time. Written, informed consent was obtained. Participants were provided a small financial compensation. The Vanderbilt Institutional Review Board approved the study. Using block randomization, participants were randomized to receive 1 of 2 resources: the original case definition (nested in the 10 page journal article [8]) or the algorithm (Fig. 1). Prior to study onset, students received identical introduction sheets explaining (1) the purpose of the study and (2) the goal to determine the level of diagnostic certainty for the sample case using the provided resource. For the original case definition [8], participants were directed on the first page of the printed article to turn to a table on a later page that summarized the criteria. The second resource, the algorithm, was a two-page document that led the user through a series of questions until the appropriate level of diagnostic certainty was reached (Fig. 1). Additional instructions were not provided. Participant identification numbers were assigned to each study participant sequentially. Both participants and study staff were blinded to the assigned resource until the participants began the study. Participants were asked to use the provided resource to select the level of diagnostic certainty (1, 2, 3 or insufficient information) for the sample case. Each participant was given an electronic timer. The study staff read a standard instruction script when all participants arrived and advised participants to start their timers at the end of the verbal instructions and to stop their timers immediately after marking their choice. Each participant’s time was recorded by study staff, and all study documents were collected at the end of each session.
3
Table 1 Assignment of Brighton Level of diagnostic certainty for sample case by resource used. Brighton Level
Case definition
Algorithm
Total
1 2 3
16 (80%) 2 (10%) 2 (10%)
15 (75%) 5 (25%) 0
31 (77.5%) 7 (17.5%) 2 (5%)
Total
20
20
40
Fisher’s exact; p = 0.246.
2.4. Statistical analysis A sample size of 17 participants in each study arm was estimated to provide 80% power to detect a significant difference in the time to complete the study between the two resources, assuming a standard deviation of 10 min and an alpha of 0.05. To evaluate the efficiency of the algorithm compared to the standard case definition, a comparison of the average time required to determine the level of diagnostic certainty for the sample case was achieved using a two group, two sided t-test. The algorithm’s ability to achieve the same level of diagnostic certainty for the sample case as the case definition was analyzed using Fisher’s exact test. 3. Results Forty participants enrolled in the study; 20 in each study arm. The average age of participants was 23.7 years overall [24.1 years for the case definition group, 23.7 years for the algorithm (p = 0.48)]. All participants completed the study and selected a level of diagnostic certainty (Table 1). The majority of participants in both study arms selected the correct Brighton Level (80% using the case definition and 75% using the algorithm; p = 0.246). Among all participants who incorrectly identified the case as not meeting Level 1, those using the algorithm all chose Level 2 (n = 5) whereas those using the case definition chose Level 3 (n = 2) as well as Level 2 (n = 2). No participants selected “Insufficient Information” as a level. The mean time for participants to review the sample case and select a Brighton Level was 415 s (95% CI: 341–490) using the case definition, and 309 s (95% CI: 253–364) using the algorithm, representing a significant difference in the mean time to review the case and determine the level by resource used [mean difference = 107 s (95% CI: 13–200; p = 0.026)]. 4. Discussion This study supports the hypothesis that an algorithm developed using the Brighton Collaboration case definition for an AEFI can result in the assignment of an appropriate level of diagnostic certainty in a significantly shorter period of time. Because our study design directed participants to the criteria table within the case definition journal article, the time required to locate the criteria for a researcher unfamiliar with the current journal format for Brighton definitions was eliminated. If we had not included this instruction, the difference in time to assign a level between resources may have been much greater. Given that most people who refer to the case definitions are familiar with the structure, this design best replicates the use of the Brighton case definitions and most accurately demonstrates the time difference between the two resources. The development of more efficient, practical tools for assessing AEFI is needed. New vaccine products used globally lead to a greater demand for vaccine safety surveillance, particularly in low to middle income countries (LMICs) [9]. This is supported by the World Health Organization’s Global Vaccine Safety Initiative (GVSI)
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
G Model JVAC-15351; No. of Pages 4
ARTICLE IN PRESS D. Joshi et al. / Vaccine xxx (2014) xxx–xxx
4
in 2012, whose objectives include the development of internationally harmonized tools to support vaccine surveillance in LMICs [10]. Aggregating data across multiple surveillance systems would allow for more statistical power and a greater probability of identifying rare AEFI; however, assembling of data is only possible if AEFI are based on similar criteria [10]. Brighton case definitions allow for standardization of AEFI reporting criteria, and more efficient methods for AEFI definition would enhance this effort, both in LMICs and in countries with strong vaccine surveillance systems. Our study has several limitations. The sample size is small, and the study may not be generalizable as all participants were medical students. However, many users of Brighton case definitions will be healthcare providers who have completed medical school training. Another potential limitation is that the participants completed the study in the same room as up to four other participants. When a participant stopped their timer, it beeped, signifying to all other participants in the room that the individual had completed the study. Thus, some participants may have rushed to finish the study after hearing another participant’s timer beep. Additionally, the inaccurate assignment of the level by a few participants using the algorithm may be a result of its format. In the algorithm’s listing of minor respiratory criteria, “Stridor” appears on its own line (Fig. 1). Participants may not have read the complete statement (“difficulty breathing without wheeze or stridor”) and incorrectly assigned the case a Brighton Level 2. Future research should include testing the efficiency and accuracy of the algorithm by application to multiple cases with different levels of diagnostic certainty and with participants with various training. Furthermore, since all participants who incorrectly identified the case as not meeting Level 1 using the algorithm all chose Level 2 (n = 5) whereas those using the case definition table chose Level 3 (n = 2) and Level 2 (n = 2), it would be interesting to elucidate the rationale for this inaccurate determination and thereafter correct the algorithm to avoid this error. 5. Summary An algorithm developed from the Brighton Collaboration case definition for anaphylaxis was more efficient in determining the level of diagnostic certainty for a case of anaphylaxis, without impacting accuracy. Easily disseminated methods for identifying
AEFIs and systematic methods for analysis would allow for enhanced comparability of vaccine safety data. Conflicts of interest The authors have no conflicts of interest relevant to this article to disclose. The authors have no financial relationships relevant to this article to disclose. Acknowledgements The Vanderbilt Institute for Clinical and Translational Research grant supported this project (UL1 TR000445 from NCATS/NIH). Study data were collected and managed using REDCap electronic data capture tools hosted at Vanderbilt University. References [1] CDC. Impact of vaccines universally recommended for children – United States, 1990–1998. MMWR Morb Mortal Wkly Rep 1999;48(April (12)):243–8. PubMed PMID: 10220251. [2] The Brighton Collaboration. Available from: https://brightoncollaboration.org/ internet/en/index.html [23.10.10]. [3] Bonhoeffer J, Kohl K, Chen R, Duclos P, Heijbel H, Heininger U, et al. The Brighton Collaboration: addressing the need for standardized case definitions of adverse events following immunization (AEFI). Vaccine 2002;21(December (3–4)):298–302. PubMed PMID: 12450705. [4] Ruggeberg JU, Gold MS, Bayas JM, Blum MD, Bonhoeffer J, Friedlander S, et al. Anaphylaxis: case definition and guidelines for data collection, analysis, and presentation of immunization safety data. Vaccine 2007;25(August (31)):5675–84. PubMed PMID: 17448577. [5] SmartDraw; 2013. Available from: http://www.smartdraw.com/ [10.10.13]. [6] LaRussa P, Edwards K, Dekker C, Klein N, Halsey N, Marchant C, et al. Understanding the role of human variation in vaccine adverse events: the Clinical Immunization Safety Assessment (CISA) network. Pediatrics 2011;127n3(March (Vaccine Safety Supplement)). [7] CDC. Vaccine safety – Clinical Immunization Safety Assessment (CISA) network. Available from: http://www.cdc.gov/vaccinesafety/Activities/cisa.html [updated 02/25/201008/04/2010]. [8] Gold MS, Gidudu J, Erlewyn-Lajeunesse M, Law B, Brighton Collaboration Working Group on A. Can the Brighton Collaboration case definitions be used to improve the quality of Adverse Event Following Immunization (AEFI) reporting? Anaphylaxis as a case study. Vaccine 2010;28(June (28)):4487–98. PubMed PMID: 20434547. [9] Amarasinghe A, Black S, Bonhoeffer J, Carvalho SM, Dodoo A, Eskola J, et al. Effective vaccine safety systems in all countries: a challenge for more equitable access to immunization. Vaccine 2013;31(April (Suppl. 2)):B108–14. PubMed PMID: 23598471. [10] WHO Global Vaccine Safety Blueprint; 2012. Available from: http://www.who. int/vaccine safety/en/ [10.10.2013].
Please cite this article in press as: Joshi D, et al. An algorithm developed using the Brighton Collaboration case definitions is more efficient for determining diagnostic certainty. Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.04.070
