Vaccine 31S (2013) K83–K87
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
Review
A systematic review of validated methods for identifying transverse myelitis using administrative or claims data S. Elizabeth Williams a,∗ , Ryan Carnahan b , Shanthi Krishnaswami c , Melissa L. McPheeters c,d a
Vanderbilt Vaccine Research Program, Vanderbilt University Medical Center, USA Department of Epidemiology, University of Iowa College of Public Health, S437 CPHB University of Iowa, 105 River Street, Iowa City, IA 52242, USA Vanderbilt Evidence-based Practice Center, Institute for Medicine and Public Health, Vanderbilt University Medical Center, Suite 600, 2525 West End Avenue, Nashville, TN 372031738, USA d Department of Obstetrics and Gynecology, Vanderbilt University Medical Center, Suite 600, 2525 West End Avenue, Nashville, TN 37203-1738, USA b c
a r t i c l e
i n f o
Article history: Received 28 November 2012 Received in revised form 7 February 2013 Accepted 26 March 2013
Keywords: Transverse myelitis Administrative database ICD-9 Positive predictive value
a b s t r a c t Purpose: To identify and assess billing, procedural, or diagnostic code algorithms used to identify transverse myelitis in administrative databases. Methods: We searched the MEDLINE database from 1991 to September 2012 using controlled vocabulary and key terms related to transverse myelitis. We also searched the reference lists of included studies. Two investigators independently assessed the full text of studies against pre-determined inclusion criteria. Two reviewers independently extracted data regarding participant and algorithm characteristics. Results: Three studies met criteria for inclusion in this review. The only algorithm based solely on administrative claims data with a reported positive predictive value included five ICD-9 codes (codes 341.20, 341.21, 341.22, 323.8, 323.9). The positive predictive value for physician-diagnosed acute transverse myelitis was 62%. Conclusions: More research is needed to establish an accurate algorithm to identify transverse myelitis in large administrative databases using diagnosis and/or procedure codes. Use of standardized consensus definitions, clear description for algorithm selection, and reporting of validation procedure and results would be most beneficial. © 2013 Elsevier Ltd. All rights reserved.
Contents 1. 2. 3.
4.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1. Transverse myelitis as a health outcome of interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Materials and methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1. Study population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2. Study design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3. Validation of cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A. Supplementary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
K84 K84 K84 K84 K84 K86 K86 K86 K87 K87 K87
Abbreviations: ATM, acute transverse myelitis; CSF, cerebrospinal fluid; ICD, International Classification of Diseases; KPNC, Kaiser Permanente Northern California; MRI, magnetic resonance imaging; N, number; NR, not reported; PPV, positive predictive value; TM, transverse myelitis. ∗ Corresponding author at: 1161 21st Avenue S., CCC 5326 Medical Center North, Nashville, TN 37232-0012, USA. Tel.: +1 615 322 8344; fax: +1 615 322 2733. E-mail addresses: [email protected] (S.E. Williams), [email protected] (R. Carnahan), [email protected] (S. Krishnaswami), [email protected] (M.L. McPheeters). 0264-410X/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.vaccine.2013.03.074
K84
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
1. Introduction Mini-Sentinel, a pilot project sponsored by the United States Food and Drug Administration (FDA), aims to inform and facilitate the development of an active surveillance system, the Sentinel System, for monitoring the safety of FDA-regulated medical products. Mini-Sentinel is one facet of the Sentinel Initiative, an FDA effort to develop a national electronic system that will complement existing methods of safety surveillance. A first step in developing the Sentinel system is an attempt at understanding the validity of algorithms (i.e., combinations of billing, procedural, pharmacy, or diagnosis codes) for identifying health outcomes of interest in administrative and claims data, hereafter administrative data. As part of Mini-Sentinel, the Post-Licensure Rapid Immunization Safety Monitoring (PRISM) program intends to establish mechanisms for conducting vaccine safety research using healthcare claims data [1,2]. In order to conduct vaccine research in administrative data effectively, accurate methods of identifying events of interest need to be developed. This may include using multiple codes sequentially or simultaneously as indicators that a clinical event has occurred. Therefore, this project aims to identify existing studies in which specific codes or sets of codes typically used for administrative purposes (e.g., ICD-9 codes) are able to capture clinical events (health outcomes of interest) accurately. PRISM program collaborators selected health outcomes of interest using an expert elicitation process through which investigators developed a list of candidate outcomes based on input from global vaccine safety experts. A panel of five vaccine experts then prioritized the list via an iterative process using criteria including clinical severity, public health importance, incidence, and relevance [3]. The particular health event of interest in this paper is transverse myelitis. 1.1. Transverse myelitis as a health outcome of interest Transverse myelitis is a severe neurological disorder where one third of affected individuals acquire a permanent severe disability [4,5]. Clinically, the disorder is characterized by acute or subacute onset of sensory, motor or autonomic dysfunction due to demyelination of the spinal cord. Symptoms are usually bilateral; however atypical presentations can also occur. The causes are highly variable and often not identified (30% are idiopathic). Infection precedes approximately 50% of cases [4]. The condition may also occur as the presenting sign of an underlying systemic autoimmune disorder, such as multiple sclerosis. Case reports have implicated a variety of vaccines, with the condition developing subsequent to vaccination [4]. All age groups are affected, but there tends to be a bimodal increased incidence during the second and fourth decade of life. There does not appear to be any predisposition based on geographic location, familial history, ethnicity or gender. Reported incidence rates have varied from 1.34 to 4.6 cases per million per year [4]. Confirmation of the diagnosis is made with clinical history and examination, usually including the identification of a clear sensory level below which sensory changes occur, along with findings on magnetic resonance imaging (MRI). Spinal cord lesions typically enhance with intravenous gadolinium and often span at least two vertebrae. MRI is also important in ruling out other causes of cord compression that may need acute surgical intervention [5]. In the 2011 Institute of Medicine review of adverse effects of vaccines, the potential for a causal relationship between transverse myelitis and eight different vaccines (measles, mumps, and rubella; varicella; influenza; Hepatitis A and B; human papillomavirus; diphtheria toxoid, tetanus toxoid, and acellular pertussis-containing vaccines; meningococcal vaccine) was
assessed via literature review [6]. The Committee’s epidemiologic assessment (assessment of the weight of evidence from the epidemiologic literature) of the evidence for cases of transverse myelitis associated with each vaccine was “weak.” Similarly, the Committee’s mechanistic assessment (assessment of the weight of evidence from the biological and clinical literature) was “weak” or “lacking.” The report’s consensus was that the current evidence was inadequate to accept or reject a causal relationship. Case reports of transverse myelitis occurring after vaccination have been reported [6], yet whether these were truly causally related to the vaccine or merely coincidental occurrences is uncertain. Thus, further epidemiologic evidence is needed to properly assess the potential association with vaccination. 2. Materials and methods As described fully in the accompanying methods paper by McPheeters et al. [7], we developed a search strategy over a period of several months. Building on prior Mini-Sentinel approaches to searching [1], we expanded those approaches and tested the need to assess gray literature, including via Google Scholar, which did not yield any citations beyond the traditional search. Therefore, the final search strategy was executed in MEDLINE via the PubMed interface. The strategy is outlined in Appendix A. We limited searches to the last 21 years (1991 to September 2012) and required that included studies address transverse myelitis; use an administrative database reporting data from the United States or Canada; and clearly define an algorithm to identify cases of transverse myelitis. We also noted whether studies reported validation of the algorithm (e.g., via chart review or independent diagnosis). We searched the reference lists of included studies. Two investigators independently assessed the full text of each study against our inclusion criteria with disagreements resolved via a third reviewer or discussion to reach consensus. These investigators included a pediatrician with fellowship training in vaccine safety and an epidemiologist with research training. One investigator extracted data regarding the study population, outcome studied, algorithms used, validation procedure, and validity statistics. A second reviewer and lead author of the review independently verified the accuracy of the data extraction. The lead author also contacted study investigators to request unpublished data for those studies indicating additional case-finding methods that were not fully detailed. The lead author also independently considered methodologic elements in included studies by assessing whether: (1) all case data were validated or only a random sample; (2) authors reported the percentage of records that were sought but not obtained; (3) authors using multiple codes to identify the outcome of interest validated individual codes; (4) authors reported predictive or sensitivity measures or presented enough data that such measures could be calculated; (5) authors presented the representativeness of their sample and therefore the generalizability of the results. We summarized results of included studies qualitatively and report key characteristics below. 3. Results 3.1. Study population Our searches identified 47 potential citations of which 3 met our inclusion criteria (Fig. 1). Table 1 summarizes study characteristics, and Appendix B includes a list of studies not meeting our review criteria with reasons for exclusion. All studies were conducted in the United States. One study by Schulz et al. reports data collected from the Thomas Jefferson University Hospital
Table 1 Summary of study characteristics and diagnostic accuracy by algorithm for transverse myelitis. Data source/population Population N
Sample/case characteristicsa
Clinical event
Algorithm
Validation/adjudication procedure operational definition
Validation statistics
Schulz et al., 2011 [8], United States, 1994–2007
Thomas Jefferson University Hospital medical record system NR
87 inpatients including 76% women and 26% African Americans, overall median age of 47 (range: 22–78) years
Incident cases of acute transverse myelitis – incident cases identified by excluding non-acute cases of transverse myelitis or previous event of transverse myelitis. No diagnosis-free baseline time period specified for incidence determination
An inpatient admitting diagnosis ICD-9 – code of any of the following: Acute TM: 341.20 Acute TM in conditions classified elsewhere: 341.21 Idiopathic TM: 341.22 Other causes of encephalitis and encephalomyelitis: 323.8 Unspecified causes of encephalitis, myelitis, and encephalomyelitis: 323.9
Physician diagnosis of idiopathic or systemic lupus erythematosus-related ATM and MRI/CSF evidence of TM ATM defined as development of sensory, motor or autonomic dysfunction attributable to spinal cord lesions and progressing over hours or up to 3 weeks before admission to hospital
PPV: 62.1% 95% CI = 51.6–71.5% 54 of 87 ICD-identified cases were confirmed
Klein et al., 2010 [9], United States, 1/1998–12/2004
Kaiser Permanente Northern California, administrative database 3.2 million members
153 new TM cases who were KPNC members for at least 77 months during the study period and AID-free and between 10–62 years; 71% female
Incident cases of transverse myelitis – incident cases identified by excluding anyone with a prior diagnosis of transverse myelitis reported in the Kaiser Permanente database in inception period (1/1996–12/1997)
ICD-9 (inpatient, outpatient, emergency department) 323.9 or “text write in diagnosis” or other unspecified methods
Chart review by neurologist NR
PPV: 75.7% 95% CI: 68.5–81.9%b Not generalizable due to use of text write in diagnoses.
Langer-Gould et al., 2011 [10], United States, 1/2004–12/2009
Kaiser Permanente Southern California/multi-ethnic children ≤18 years of age 3.2 million members with over 900,000 ≤18 years of age
10 children newly diagnosed with TM, mean age of 14.4 (range: 7.25–18) years, 60% females, with an average observation period of 3.2 years and 60% Hispanics
Incident cases of transverse myelitis and other acquired demyelinating syndromes–incident cases confirmed by medical record review. No asymptomatic baseline time period specified for incidence determination
ICD-9 (inpatient and outpatient) Acute TM: 341.20 Idiopathic TM: 341.22
Medical record review, including inpatient, outpatient records, MRI scans and diagnostic test results by specialists Idiopathic TM defined according to proposed consensus definitions after exclusion of infectious, vascular, and other inflammatory causes of myelopathy
NR
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
Author, year, country, time period
ATM, acute transverse myelitis; CSF, cerebrospinal fluid; ICD, International Classification of Diseases; KPNC, Kaiser Permanente Northern California; MRI, magnetic resonance imaging; N, number; NR, not reported; PPV, positive predictive value; TM, transverse myelitis. a Data reported in this column are either the # of confirmed cases (where reported) or the number of cases identified, but not necessarily confirmed, using the search algorithm. b Calculated based on assumed number of cases given the PPV.
K85
K86
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
Nonduplicate articles identified in searches and reviewed N=47
• •
Literature search: N=40 Hand search: N=7
Articles excluded a N=44
•
Did not address health outcome of interest N=18
•
Did not use an administrative database N= 37
3.3. Validation of cases
• Included studies n = 3b
•
Describing validation of algorithm or case confirmation methods: n=3
determination. The Langer-Gould study included only children less than 18 years of age, increasing the likelihood that the cases were incident. This study also reported that all cases were confirmed with full medical record abstraction. A total of 250 cases were identified across all three studies using the ICD-9 codes outlined above.
Data source not from U.S. or Canada N= 36
•
Did not report algorithm N=38
a
Note: Numbers do not tally as articles were excluded for multiple reasons
b
Two of three included studies located via handsearch.
Fig. 1. Flow of studies identified for review. (a) Note: Numbers do not tally as articles were excluded for multiple reasons. (b) Two of three included studies located via handsearch.
medical system [8]. Two studies, Klein et al. and Langer-Gould et al., reported data collected from Kaiser Permanente databases (Northern and Southern California) [9,10]. Kaiser Permanente members are described as ethnically, racially, socioeconomically heterogeneous and reflective of the local population, with the exception that very high and very low incomes are underrepresented [11,12]. One of the Kaiser studies included 900,000 children less than 18 years of age [10] and one included adults and children ages 10–55 years of age from an overall cohort of 3.2 million subscribers [9]. Although a description of the population served by Thomas Jefferson University Hospital was not provided in the Schulz paper, we assume that the population is heterogeneous group and a fair representation of the local surrounding population as the institution is an academic medical center serving patients in Philadelphia and the surrounding communities in the Delaware Valley [8]. Transverse myelitis occurred in older patients in the studies reported by Klein et al., and Schulz et al.; more cases were identified in the age range of 26–62 in the Klein study [9], and a mean age for cases was reported as 50.4 years in the Schulz study [8]. All cases of transverse myelitis in the report by Langer-Gould et al. occurred in children less than 18 years of age since this was the study population [10]. In all three included studies, the condition was more common in females than males (range of patients who were female: 60–76%). 3.2. Study design All three studies were conducted as retrospective reviews of large administrative databases and used specified ICD-9 codes: 341.20 [8,10] (acute transverse myelitis), 341.21 [8] (acute transverse myelitis in conditions classified elsewhere), 341.22 [8,10] (idiopathic transverse myelitis), 323.8 [8] (other causes of encephalitis and encephalomyelitis) and 323.9 [8,9] (unspecified causes of encephalitis, myelitis, and encephalomyelitis). All studies reported incident cases of transverse myelitis, yet only two, Klein et al. and Schulz et al., specifically described how they determined all identified cases were incident by excluding non-acute causes of transverse myelitis. The Klein et al. study also reported excluding anyone with a prior diagnosis of transverse myelitis in the Kaiser Permanente database in the study inception period (1/1996–12/1997). Neither the Schulz et al. nor Langer-Gould et al. study reported asymptomatic baseline time periods for incidence
Validation was conducted by chart review in the three studies via the following methods: clearly stated diagnosis of transverse myelitis by a neurologist [9], use of a reported consensus definition [13] after exclusion of infectious, vascular and other inflammatory causes of myelopathy [10], and including those with either magnetic resonance imaging or cerebral spinal fluid evidence of transverse myelitis and/or patients who were diagnosed by a physician as having transverse myelitis [8]. Although all three described a validation approach, only two reported the results allowing calculation of the proportion of cases verified after the validation process. The study by Schulz et al. included the positive predictive value of the algorithm; 54 of 87 ICD-9 identified cases were confirmed upon review, resulting in a predictive value of 62.1% (95% CI = 51.6–71.5%) using a combination of ICD-9 codes (323.8, 323.9, 341.20, 341.22, 341.21) [8]. Each code was not evaluated separately; therefore, the validity of any individual code is unknown as the results did not describe whether any of the five ICD-9 codes was more likely to identify cases or the relative yield for any individual codes. Two of the codes used to identify cases in this study were also not specific for the outcome of interest, transverse myelitis, but represented broad categories of “other causes of encephalitis and encephalomyelitis” (323.8) and “unspecified causes of encephalitis, myelitis, and encephalomyelitis” (323.9). Although not stated in the report, we contacted study investigators who informed us that these ICD codes were used to capture potential cases that were misclassified under similar ICD neurological codes, with the plan to review cases individually afterwards to make the correct determination [14]. The study by Klein et al. also included the results of the validation using the ICD-9 code 323.9 in addition to other methods, such as text write-in diagnosis, for identifying cases [15]. This study reported that of 153 cases initially identified using this combination of methods, 75.7% (95% CI = 68.5–81.9%) were verified after medical record review [9]. However, the validity of this individual code for identifying cases of transverse myelitis, separate from the addition of other methods used to identify cases, was not reported. Further, it is unlikely that this code alone would be highly accurate considering that it represents a broad range of potential diagnoses (“unspecified causes of encephalitis, myelitis, and encephalomyelitis”) compared with other diagnosis codes available. The third included study by Langer-Gould et al. [10] did not report validation statistics to evaluate the predictive value of the combination of codes used to identify transverse myelitis; namely, 341.20 and 341.22. Investigators initially searched under 323.x codes as well, but these did not identify any additional cases of transverse myelitis than were found using just 341.20 and 341.22. The author noted that some children had received both 323.x and 341.2x codes for the disorder [16]. 4. Discussion Few studies report algorithms to identify incident cases of transverse myelitis in large, health-related administrative databases. Of the reports identified, only one presented results allowing
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
the true predictive value of an algorithm to be determined. This study from Schulz and colleagues used the ICD-9 code combination of 323.8, 323.9, 341.20, 341.21, and 341.22, which had a positive predictive value of 62.1% [8]. In future algorithm-based administrative database studies it would be valuable to potential stakeholders for the investigators to include the rationale for ICD-9 code selection and to clearly state the results of any validation process. One might expect that the 341.2x codes, which are codes defined as acute transverse myelitis of various types, would have a higher PPV than other more general codes used in the reviewed studies. However, this has not been confirmed with any validation evidence. It is also probable that sensitivity might be reduced by restricting the algorithm to these codes, given that other codes for myelitis and related conditions exist, and the diagnosis of transverse myelitis is not straightforward. For vaccine adverse event research, the ICD-9 code 323.52 (myelitis following immunization procedures) might be of particular interest. Researchers should keep in mind that all cases with this code would assumedly be attributed to a vaccine exposure, depending on the duration of the risk window specified in the study design. Thus, restricting an algorithm to only this code would not be recommended, as it would produce a type of coding bias akin to diagnostic suspicion bias. The baseline rate of myelitis would need to be included to evaluate whether the relationship between the exposure and outcome was coincidental and not causative, so other codes for these conditions would also need to be included in the algorithm. One primary limitation identified in the included studies is the lack of reported experts in multiple sclerosis, as opposed to general neurologists, in differentiating cases of transverse myelitis from early onset cases of multiple sclerosis. The diagnostic criteria for multiple sclerosis have very recently changed, and it may be possible that a significant number of cases classified as transverse myelitis in the included studies would now be classified as multiple sclerosis. Therefore, in future studies, if the goal is to identify PPV and NPV of algorithms in identifying transverse myelitis, confirmation of cases might best be determined by an expert in multiple sclerosis since the clinical syndromes are so similar and are often misdiagnosed. The evidence is currently insufficient to recommend a specific algorithm for accurately identifying transverse myelitis. Further studies to evaluate the sequential or simultaneous use of the ICD-9 codes shown in this review would be beneficial, as well as clarification of which specific algorithms are the most accurate and why specific ICD-9 codes were selected. Given that this disorder is primarily diagnosed by MRI, one consideration for future studies would be including an MRI procedural code (e.g., ICD-9 procedure codes 72156 for cervical MRI with and without contrast) for case identification. Further, use of standardized consensus definitions for cases, such as those published in 2002 [13] and used by LangerGould and colleagues [10] may provide an improved strategy for case validation which could be more easily replicated by other investigators. Consensus definitions may also be useful as a gold standard to validate future algorithms. Because transverse myelitis is a rare disorder, additional large database studies clarifying the incidence of this condition in conjunction with recent vaccination would be valuable for vaccine safety knowledge and clarification of risk.
K87
Acknowledgements Conflict of interest: The authors have no conflict to declare. Contributors: All authors declare that they have participated in: (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, (3) final approval of the version submitted. Funding source: Mini-Sentinel is funded by the Food and Drug Administration (FDA) through Department of Health and Human Services (HHS) Contract Number HHSF223200910006I. The views expressed in this document do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. government. Role of the funding source: FDA staff reviewed articles prior to publication but had no role in study design or conduct. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/ j.vaccine.2013.03.074. References [1] Carnahan RM, Moores KG. Mini-Sentinel’s systematic reviews of validated methods for identifying health outcomes using administrative and claims data: methods and lessons learned. Pharmacoepidemiol Drug Saf 2012;21(January (Suppl. 1)):82–9. [2] Nguyen M, Ball R, Midthun K, Lieu TA. The Food and Drug Administration’s Post-Licensure Rapid Immunization Safety Monitoring program: strengthening the federal vaccine safety enterprise. Pharmacoepidemiol Drug Saf 2012;21(January (Suppl. 1)):291–7. [3] Lieu TA, Nguyen MD, Ball R, Martin DB. Health outcomes of interest for evaluation in the Post-Licensure Rapid Immunization Safety Monitoring Program. Vaccine 2012;30(April (18)):2824–30. [4] Bhat A, Naguwa S, Cheema G, Gershwin ME. The epidemiology of transverse myelitis. Autoimmun Rev 2010;9(March (5)):A395–9. [5] Frohman EM, Wingerchuk DM. Clinical practice. Transverse myelitis. N Engl J Med 2010;363(August (6)):564–72. [6] Institute of Medicine. Adverse Effects of Vaccines: Evidence and Causality. Washington, DC: The National Academies Press; 2011. [7] McPheeters M, Sathe N, Jerome R, Carnahan R. Methods for systematic reviews of administrative database studies capturing health outcomes of interest vaccine, submitted for publication. [8] Schulz SW, Shenin M, Mehta A, Kebede A, Fluerant M, Derk CT. Initial presentation of acute transverse myelitis in systemic lupus erythematosus: demographics, diagnosis, management and comparison to idiopathic cases. Rheumatol Int 2011;(July). [9] Klein NP, Ray P, Carpenter D, Hansen J, Lewis E, Fireman B, et al. Rates of autoimmune diseases in Kaiser Permanente for use in vaccine adverse event safety studies. Vaccine 2010;28(January (4)):1062–8. [10] Langer-Gould A, Zhang JL, Chung J, Yeung Y, Waubant E, Yao J. Incidence of acquired CNS demyelinating syndromes in a multiethnic cohort of children. Neurology 2011;77(September (12)):1143–8. [11] Krieger N. Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health 1992;82(May (5)):703–10. [12] Koebnick C, Smith N, Coleman KJ, Getahun D, Reynolds K, Quinn VP, et al. Prevalence of extreme obesity in a multiethnic cohort of children and adolescents. J Pediatr 2010;157(July (1)):26–31, e2. [13] Proposed diagnostic criteria and nosology of acute transverse myelitis. Neurology 2002;59(August (4)):499–505. [14] Derk CT. Personal communication, 6 March 2012. [15] Klein NP. Personal communication, 9 December 2011. [16] Langer-Gould A. Personal communication, 6 March 2012.
Contents lists available at ScienceDirect
Vaccine journal homepage: www.elsevier.com/locate/vaccine
Review
A systematic review of validated methods for identifying transverse myelitis using administrative or claims data S. Elizabeth Williams a,∗ , Ryan Carnahan b , Shanthi Krishnaswami c , Melissa L. McPheeters c,d a
Vanderbilt Vaccine Research Program, Vanderbilt University Medical Center, USA Department of Epidemiology, University of Iowa College of Public Health, S437 CPHB University of Iowa, 105 River Street, Iowa City, IA 52242, USA Vanderbilt Evidence-based Practice Center, Institute for Medicine and Public Health, Vanderbilt University Medical Center, Suite 600, 2525 West End Avenue, Nashville, TN 372031738, USA d Department of Obstetrics and Gynecology, Vanderbilt University Medical Center, Suite 600, 2525 West End Avenue, Nashville, TN 37203-1738, USA b c
a r t i c l e
i n f o
Article history: Received 28 November 2012 Received in revised form 7 February 2013 Accepted 26 March 2013
Keywords: Transverse myelitis Administrative database ICD-9 Positive predictive value
a b s t r a c t Purpose: To identify and assess billing, procedural, or diagnostic code algorithms used to identify transverse myelitis in administrative databases. Methods: We searched the MEDLINE database from 1991 to September 2012 using controlled vocabulary and key terms related to transverse myelitis. We also searched the reference lists of included studies. Two investigators independently assessed the full text of studies against pre-determined inclusion criteria. Two reviewers independently extracted data regarding participant and algorithm characteristics. Results: Three studies met criteria for inclusion in this review. The only algorithm based solely on administrative claims data with a reported positive predictive value included five ICD-9 codes (codes 341.20, 341.21, 341.22, 323.8, 323.9). The positive predictive value for physician-diagnosed acute transverse myelitis was 62%. Conclusions: More research is needed to establish an accurate algorithm to identify transverse myelitis in large administrative databases using diagnosis and/or procedure codes. Use of standardized consensus definitions, clear description for algorithm selection, and reporting of validation procedure and results would be most beneficial. © 2013 Elsevier Ltd. All rights reserved.
Contents 1. 2. 3.
4.
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1. Transverse myelitis as a health outcome of interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Materials and methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1. Study population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2. Study design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3. Validation of cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A. Supplementary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
K84 K84 K84 K84 K84 K86 K86 K86 K87 K87 K87
Abbreviations: ATM, acute transverse myelitis; CSF, cerebrospinal fluid; ICD, International Classification of Diseases; KPNC, Kaiser Permanente Northern California; MRI, magnetic resonance imaging; N, number; NR, not reported; PPV, positive predictive value; TM, transverse myelitis. ∗ Corresponding author at: 1161 21st Avenue S., CCC 5326 Medical Center North, Nashville, TN 37232-0012, USA. Tel.: +1 615 322 8344; fax: +1 615 322 2733. E-mail addresses: [email protected] (S.E. Williams), [email protected] (R. Carnahan), [email protected] (S. Krishnaswami), [email protected] (M.L. McPheeters). 0264-410X/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.vaccine.2013.03.074
K84
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
1. Introduction Mini-Sentinel, a pilot project sponsored by the United States Food and Drug Administration (FDA), aims to inform and facilitate the development of an active surveillance system, the Sentinel System, for monitoring the safety of FDA-regulated medical products. Mini-Sentinel is one facet of the Sentinel Initiative, an FDA effort to develop a national electronic system that will complement existing methods of safety surveillance. A first step in developing the Sentinel system is an attempt at understanding the validity of algorithms (i.e., combinations of billing, procedural, pharmacy, or diagnosis codes) for identifying health outcomes of interest in administrative and claims data, hereafter administrative data. As part of Mini-Sentinel, the Post-Licensure Rapid Immunization Safety Monitoring (PRISM) program intends to establish mechanisms for conducting vaccine safety research using healthcare claims data [1,2]. In order to conduct vaccine research in administrative data effectively, accurate methods of identifying events of interest need to be developed. This may include using multiple codes sequentially or simultaneously as indicators that a clinical event has occurred. Therefore, this project aims to identify existing studies in which specific codes or sets of codes typically used for administrative purposes (e.g., ICD-9 codes) are able to capture clinical events (health outcomes of interest) accurately. PRISM program collaborators selected health outcomes of interest using an expert elicitation process through which investigators developed a list of candidate outcomes based on input from global vaccine safety experts. A panel of five vaccine experts then prioritized the list via an iterative process using criteria including clinical severity, public health importance, incidence, and relevance [3]. The particular health event of interest in this paper is transverse myelitis. 1.1. Transverse myelitis as a health outcome of interest Transverse myelitis is a severe neurological disorder where one third of affected individuals acquire a permanent severe disability [4,5]. Clinically, the disorder is characterized by acute or subacute onset of sensory, motor or autonomic dysfunction due to demyelination of the spinal cord. Symptoms are usually bilateral; however atypical presentations can also occur. The causes are highly variable and often not identified (30% are idiopathic). Infection precedes approximately 50% of cases [4]. The condition may also occur as the presenting sign of an underlying systemic autoimmune disorder, such as multiple sclerosis. Case reports have implicated a variety of vaccines, with the condition developing subsequent to vaccination [4]. All age groups are affected, but there tends to be a bimodal increased incidence during the second and fourth decade of life. There does not appear to be any predisposition based on geographic location, familial history, ethnicity or gender. Reported incidence rates have varied from 1.34 to 4.6 cases per million per year [4]. Confirmation of the diagnosis is made with clinical history and examination, usually including the identification of a clear sensory level below which sensory changes occur, along with findings on magnetic resonance imaging (MRI). Spinal cord lesions typically enhance with intravenous gadolinium and often span at least two vertebrae. MRI is also important in ruling out other causes of cord compression that may need acute surgical intervention [5]. In the 2011 Institute of Medicine review of adverse effects of vaccines, the potential for a causal relationship between transverse myelitis and eight different vaccines (measles, mumps, and rubella; varicella; influenza; Hepatitis A and B; human papillomavirus; diphtheria toxoid, tetanus toxoid, and acellular pertussis-containing vaccines; meningococcal vaccine) was
assessed via literature review [6]. The Committee’s epidemiologic assessment (assessment of the weight of evidence from the epidemiologic literature) of the evidence for cases of transverse myelitis associated with each vaccine was “weak.” Similarly, the Committee’s mechanistic assessment (assessment of the weight of evidence from the biological and clinical literature) was “weak” or “lacking.” The report’s consensus was that the current evidence was inadequate to accept or reject a causal relationship. Case reports of transverse myelitis occurring after vaccination have been reported [6], yet whether these were truly causally related to the vaccine or merely coincidental occurrences is uncertain. Thus, further epidemiologic evidence is needed to properly assess the potential association with vaccination. 2. Materials and methods As described fully in the accompanying methods paper by McPheeters et al. [7], we developed a search strategy over a period of several months. Building on prior Mini-Sentinel approaches to searching [1], we expanded those approaches and tested the need to assess gray literature, including via Google Scholar, which did not yield any citations beyond the traditional search. Therefore, the final search strategy was executed in MEDLINE via the PubMed interface. The strategy is outlined in Appendix A. We limited searches to the last 21 years (1991 to September 2012) and required that included studies address transverse myelitis; use an administrative database reporting data from the United States or Canada; and clearly define an algorithm to identify cases of transverse myelitis. We also noted whether studies reported validation of the algorithm (e.g., via chart review or independent diagnosis). We searched the reference lists of included studies. Two investigators independently assessed the full text of each study against our inclusion criteria with disagreements resolved via a third reviewer or discussion to reach consensus. These investigators included a pediatrician with fellowship training in vaccine safety and an epidemiologist with research training. One investigator extracted data regarding the study population, outcome studied, algorithms used, validation procedure, and validity statistics. A second reviewer and lead author of the review independently verified the accuracy of the data extraction. The lead author also contacted study investigators to request unpublished data for those studies indicating additional case-finding methods that were not fully detailed. The lead author also independently considered methodologic elements in included studies by assessing whether: (1) all case data were validated or only a random sample; (2) authors reported the percentage of records that were sought but not obtained; (3) authors using multiple codes to identify the outcome of interest validated individual codes; (4) authors reported predictive or sensitivity measures or presented enough data that such measures could be calculated; (5) authors presented the representativeness of their sample and therefore the generalizability of the results. We summarized results of included studies qualitatively and report key characteristics below. 3. Results 3.1. Study population Our searches identified 47 potential citations of which 3 met our inclusion criteria (Fig. 1). Table 1 summarizes study characteristics, and Appendix B includes a list of studies not meeting our review criteria with reasons for exclusion. All studies were conducted in the United States. One study by Schulz et al. reports data collected from the Thomas Jefferson University Hospital
Table 1 Summary of study characteristics and diagnostic accuracy by algorithm for transverse myelitis. Data source/population Population N
Sample/case characteristicsa
Clinical event
Algorithm
Validation/adjudication procedure operational definition
Validation statistics
Schulz et al., 2011 [8], United States, 1994–2007
Thomas Jefferson University Hospital medical record system NR
87 inpatients including 76% women and 26% African Americans, overall median age of 47 (range: 22–78) years
Incident cases of acute transverse myelitis – incident cases identified by excluding non-acute cases of transverse myelitis or previous event of transverse myelitis. No diagnosis-free baseline time period specified for incidence determination
An inpatient admitting diagnosis ICD-9 – code of any of the following: Acute TM: 341.20 Acute TM in conditions classified elsewhere: 341.21 Idiopathic TM: 341.22 Other causes of encephalitis and encephalomyelitis: 323.8 Unspecified causes of encephalitis, myelitis, and encephalomyelitis: 323.9
Physician diagnosis of idiopathic or systemic lupus erythematosus-related ATM and MRI/CSF evidence of TM ATM defined as development of sensory, motor or autonomic dysfunction attributable to spinal cord lesions and progressing over hours or up to 3 weeks before admission to hospital
PPV: 62.1% 95% CI = 51.6–71.5% 54 of 87 ICD-identified cases were confirmed
Klein et al., 2010 [9], United States, 1/1998–12/2004
Kaiser Permanente Northern California, administrative database 3.2 million members
153 new TM cases who were KPNC members for at least 77 months during the study period and AID-free and between 10–62 years; 71% female
Incident cases of transverse myelitis – incident cases identified by excluding anyone with a prior diagnosis of transverse myelitis reported in the Kaiser Permanente database in inception period (1/1996–12/1997)
ICD-9 (inpatient, outpatient, emergency department) 323.9 or “text write in diagnosis” or other unspecified methods
Chart review by neurologist NR
PPV: 75.7% 95% CI: 68.5–81.9%b Not generalizable due to use of text write in diagnoses.
Langer-Gould et al., 2011 [10], United States, 1/2004–12/2009
Kaiser Permanente Southern California/multi-ethnic children ≤18 years of age 3.2 million members with over 900,000 ≤18 years of age
10 children newly diagnosed with TM, mean age of 14.4 (range: 7.25–18) years, 60% females, with an average observation period of 3.2 years and 60% Hispanics
Incident cases of transverse myelitis and other acquired demyelinating syndromes–incident cases confirmed by medical record review. No asymptomatic baseline time period specified for incidence determination
ICD-9 (inpatient and outpatient) Acute TM: 341.20 Idiopathic TM: 341.22
Medical record review, including inpatient, outpatient records, MRI scans and diagnostic test results by specialists Idiopathic TM defined according to proposed consensus definitions after exclusion of infectious, vascular, and other inflammatory causes of myelopathy
NR
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
Author, year, country, time period
ATM, acute transverse myelitis; CSF, cerebrospinal fluid; ICD, International Classification of Diseases; KPNC, Kaiser Permanente Northern California; MRI, magnetic resonance imaging; N, number; NR, not reported; PPV, positive predictive value; TM, transverse myelitis. a Data reported in this column are either the # of confirmed cases (where reported) or the number of cases identified, but not necessarily confirmed, using the search algorithm. b Calculated based on assumed number of cases given the PPV.
K85
K86
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
Nonduplicate articles identified in searches and reviewed N=47
• •
Literature search: N=40 Hand search: N=7
Articles excluded a N=44
•
Did not address health outcome of interest N=18
•
Did not use an administrative database N= 37
3.3. Validation of cases
• Included studies n = 3b
•
Describing validation of algorithm or case confirmation methods: n=3
determination. The Langer-Gould study included only children less than 18 years of age, increasing the likelihood that the cases were incident. This study also reported that all cases were confirmed with full medical record abstraction. A total of 250 cases were identified across all three studies using the ICD-9 codes outlined above.
Data source not from U.S. or Canada N= 36
•
Did not report algorithm N=38
a
Note: Numbers do not tally as articles were excluded for multiple reasons
b
Two of three included studies located via handsearch.
Fig. 1. Flow of studies identified for review. (a) Note: Numbers do not tally as articles were excluded for multiple reasons. (b) Two of three included studies located via handsearch.
medical system [8]. Two studies, Klein et al. and Langer-Gould et al., reported data collected from Kaiser Permanente databases (Northern and Southern California) [9,10]. Kaiser Permanente members are described as ethnically, racially, socioeconomically heterogeneous and reflective of the local population, with the exception that very high and very low incomes are underrepresented [11,12]. One of the Kaiser studies included 900,000 children less than 18 years of age [10] and one included adults and children ages 10–55 years of age from an overall cohort of 3.2 million subscribers [9]. Although a description of the population served by Thomas Jefferson University Hospital was not provided in the Schulz paper, we assume that the population is heterogeneous group and a fair representation of the local surrounding population as the institution is an academic medical center serving patients in Philadelphia and the surrounding communities in the Delaware Valley [8]. Transverse myelitis occurred in older patients in the studies reported by Klein et al., and Schulz et al.; more cases were identified in the age range of 26–62 in the Klein study [9], and a mean age for cases was reported as 50.4 years in the Schulz study [8]. All cases of transverse myelitis in the report by Langer-Gould et al. occurred in children less than 18 years of age since this was the study population [10]. In all three included studies, the condition was more common in females than males (range of patients who were female: 60–76%). 3.2. Study design All three studies were conducted as retrospective reviews of large administrative databases and used specified ICD-9 codes: 341.20 [8,10] (acute transverse myelitis), 341.21 [8] (acute transverse myelitis in conditions classified elsewhere), 341.22 [8,10] (idiopathic transverse myelitis), 323.8 [8] (other causes of encephalitis and encephalomyelitis) and 323.9 [8,9] (unspecified causes of encephalitis, myelitis, and encephalomyelitis). All studies reported incident cases of transverse myelitis, yet only two, Klein et al. and Schulz et al., specifically described how they determined all identified cases were incident by excluding non-acute causes of transverse myelitis. The Klein et al. study also reported excluding anyone with a prior diagnosis of transverse myelitis in the Kaiser Permanente database in the study inception period (1/1996–12/1997). Neither the Schulz et al. nor Langer-Gould et al. study reported asymptomatic baseline time periods for incidence
Validation was conducted by chart review in the three studies via the following methods: clearly stated diagnosis of transverse myelitis by a neurologist [9], use of a reported consensus definition [13] after exclusion of infectious, vascular and other inflammatory causes of myelopathy [10], and including those with either magnetic resonance imaging or cerebral spinal fluid evidence of transverse myelitis and/or patients who were diagnosed by a physician as having transverse myelitis [8]. Although all three described a validation approach, only two reported the results allowing calculation of the proportion of cases verified after the validation process. The study by Schulz et al. included the positive predictive value of the algorithm; 54 of 87 ICD-9 identified cases were confirmed upon review, resulting in a predictive value of 62.1% (95% CI = 51.6–71.5%) using a combination of ICD-9 codes (323.8, 323.9, 341.20, 341.22, 341.21) [8]. Each code was not evaluated separately; therefore, the validity of any individual code is unknown as the results did not describe whether any of the five ICD-9 codes was more likely to identify cases or the relative yield for any individual codes. Two of the codes used to identify cases in this study were also not specific for the outcome of interest, transverse myelitis, but represented broad categories of “other causes of encephalitis and encephalomyelitis” (323.8) and “unspecified causes of encephalitis, myelitis, and encephalomyelitis” (323.9). Although not stated in the report, we contacted study investigators who informed us that these ICD codes were used to capture potential cases that were misclassified under similar ICD neurological codes, with the plan to review cases individually afterwards to make the correct determination [14]. The study by Klein et al. also included the results of the validation using the ICD-9 code 323.9 in addition to other methods, such as text write-in diagnosis, for identifying cases [15]. This study reported that of 153 cases initially identified using this combination of methods, 75.7% (95% CI = 68.5–81.9%) were verified after medical record review [9]. However, the validity of this individual code for identifying cases of transverse myelitis, separate from the addition of other methods used to identify cases, was not reported. Further, it is unlikely that this code alone would be highly accurate considering that it represents a broad range of potential diagnoses (“unspecified causes of encephalitis, myelitis, and encephalomyelitis”) compared with other diagnosis codes available. The third included study by Langer-Gould et al. [10] did not report validation statistics to evaluate the predictive value of the combination of codes used to identify transverse myelitis; namely, 341.20 and 341.22. Investigators initially searched under 323.x codes as well, but these did not identify any additional cases of transverse myelitis than were found using just 341.20 and 341.22. The author noted that some children had received both 323.x and 341.2x codes for the disorder [16]. 4. Discussion Few studies report algorithms to identify incident cases of transverse myelitis in large, health-related administrative databases. Of the reports identified, only one presented results allowing
S.E. Williams et al. / Vaccine 31S (2013) K83–K87
the true predictive value of an algorithm to be determined. This study from Schulz and colleagues used the ICD-9 code combination of 323.8, 323.9, 341.20, 341.21, and 341.22, which had a positive predictive value of 62.1% [8]. In future algorithm-based administrative database studies it would be valuable to potential stakeholders for the investigators to include the rationale for ICD-9 code selection and to clearly state the results of any validation process. One might expect that the 341.2x codes, which are codes defined as acute transverse myelitis of various types, would have a higher PPV than other more general codes used in the reviewed studies. However, this has not been confirmed with any validation evidence. It is also probable that sensitivity might be reduced by restricting the algorithm to these codes, given that other codes for myelitis and related conditions exist, and the diagnosis of transverse myelitis is not straightforward. For vaccine adverse event research, the ICD-9 code 323.52 (myelitis following immunization procedures) might be of particular interest. Researchers should keep in mind that all cases with this code would assumedly be attributed to a vaccine exposure, depending on the duration of the risk window specified in the study design. Thus, restricting an algorithm to only this code would not be recommended, as it would produce a type of coding bias akin to diagnostic suspicion bias. The baseline rate of myelitis would need to be included to evaluate whether the relationship between the exposure and outcome was coincidental and not causative, so other codes for these conditions would also need to be included in the algorithm. One primary limitation identified in the included studies is the lack of reported experts in multiple sclerosis, as opposed to general neurologists, in differentiating cases of transverse myelitis from early onset cases of multiple sclerosis. The diagnostic criteria for multiple sclerosis have very recently changed, and it may be possible that a significant number of cases classified as transverse myelitis in the included studies would now be classified as multiple sclerosis. Therefore, in future studies, if the goal is to identify PPV and NPV of algorithms in identifying transverse myelitis, confirmation of cases might best be determined by an expert in multiple sclerosis since the clinical syndromes are so similar and are often misdiagnosed. The evidence is currently insufficient to recommend a specific algorithm for accurately identifying transverse myelitis. Further studies to evaluate the sequential or simultaneous use of the ICD-9 codes shown in this review would be beneficial, as well as clarification of which specific algorithms are the most accurate and why specific ICD-9 codes were selected. Given that this disorder is primarily diagnosed by MRI, one consideration for future studies would be including an MRI procedural code (e.g., ICD-9 procedure codes 72156 for cervical MRI with and without contrast) for case identification. Further, use of standardized consensus definitions for cases, such as those published in 2002 [13] and used by LangerGould and colleagues [10] may provide an improved strategy for case validation which could be more easily replicated by other investigators. Consensus definitions may also be useful as a gold standard to validate future algorithms. Because transverse myelitis is a rare disorder, additional large database studies clarifying the incidence of this condition in conjunction with recent vaccination would be valuable for vaccine safety knowledge and clarification of risk.
K87
Acknowledgements Conflict of interest: The authors have no conflict to declare. Contributors: All authors declare that they have participated in: (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, (3) final approval of the version submitted. Funding source: Mini-Sentinel is funded by the Food and Drug Administration (FDA) through Department of Health and Human Services (HHS) Contract Number HHSF223200910006I. The views expressed in this document do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. government. Role of the funding source: FDA staff reviewed articles prior to publication but had no role in study design or conduct. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/ j.vaccine.2013.03.074. References [1] Carnahan RM, Moores KG. Mini-Sentinel’s systematic reviews of validated methods for identifying health outcomes using administrative and claims data: methods and lessons learned. Pharmacoepidemiol Drug Saf 2012;21(January (Suppl. 1)):82–9. [2] Nguyen M, Ball R, Midthun K, Lieu TA. The Food and Drug Administration’s Post-Licensure Rapid Immunization Safety Monitoring program: strengthening the federal vaccine safety enterprise. Pharmacoepidemiol Drug Saf 2012;21(January (Suppl. 1)):291–7. [3] Lieu TA, Nguyen MD, Ball R, Martin DB. Health outcomes of interest for evaluation in the Post-Licensure Rapid Immunization Safety Monitoring Program. Vaccine 2012;30(April (18)):2824–30. [4] Bhat A, Naguwa S, Cheema G, Gershwin ME. The epidemiology of transverse myelitis. Autoimmun Rev 2010;9(March (5)):A395–9. [5] Frohman EM, Wingerchuk DM. Clinical practice. Transverse myelitis. N Engl J Med 2010;363(August (6)):564–72. [6] Institute of Medicine. Adverse Effects of Vaccines: Evidence and Causality. Washington, DC: The National Academies Press; 2011. [7] McPheeters M, Sathe N, Jerome R, Carnahan R. Methods for systematic reviews of administrative database studies capturing health outcomes of interest vaccine, submitted for publication. [8] Schulz SW, Shenin M, Mehta A, Kebede A, Fluerant M, Derk CT. Initial presentation of acute transverse myelitis in systemic lupus erythematosus: demographics, diagnosis, management and comparison to idiopathic cases. Rheumatol Int 2011;(July). [9] Klein NP, Ray P, Carpenter D, Hansen J, Lewis E, Fireman B, et al. Rates of autoimmune diseases in Kaiser Permanente for use in vaccine adverse event safety studies. Vaccine 2010;28(January (4)):1062–8. [10] Langer-Gould A, Zhang JL, Chung J, Yeung Y, Waubant E, Yao J. Incidence of acquired CNS demyelinating syndromes in a multiethnic cohort of children. Neurology 2011;77(September (12)):1143–8. [11] Krieger N. Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health 1992;82(May (5)):703–10. [12] Koebnick C, Smith N, Coleman KJ, Getahun D, Reynolds K, Quinn VP, et al. Prevalence of extreme obesity in a multiethnic cohort of children and adolescents. J Pediatr 2010;157(July (1)):26–31, e2. [13] Proposed diagnostic criteria and nosology of acute transverse myelitis. Neurology 2002;59(August (4)):499–505. [14] Derk CT. Personal communication, 6 March 2012. [15] Klein NP. Personal communication, 9 December 2011. [16] Langer-Gould A. Personal communication, 6 March 2012.
