Intern Emerg Med DOI 10.1007/s11739-015-1213-9
CE - COCHRANE’S CORNER
Use of administrative data in healthcare research Cristina Mazzali • Piergiorgio Duca
Received: 24 September 2014 / Accepted: 12 February 2015 Ó SIMI 2015
Abstract Health research based on administrative data and the availability of regional or national administrative databases has been increasing in recent years. We will discuss the general characteristics of administrative data and specific aspects of their use for health research purposes, indicating their advantages and disadvantages. Some fields of application will be discussed and described through examples. Keywords Administrative data Utilisation databases Healthcare research Research methods
Introduction Healthcare administrative databases are large repositories of data on healthcare systems that are routinely collected by healthcare providers and other institutions (e.g. civil registry). They provide a variety of already stored data with an ongoing collection process [1]. They may contain information on hospitalisations, outpatient care, drug prescriptions, rehabilitation services, implanted endoprostheses, psychiatric service, etc. The advantages and disadvantages of the use of administrative databases in epidemiological, clinical and healthcare research are well known [1–4]. The advantages include large sample size, population coverage and heterogeneity, which allow researchers to reflect the ‘‘real world’’ practice [2]. Other advantages are the absence of
C. Mazzali (&) P. Duca Statistics and Biometrics Unit-’L. Sacco’ Department of Biomedical and Clinical Sciences, University of Milan, Via G.B. Grassi, 74, 20157 Milan, Italy e-mail: [email protected]
additional costs for gathering data, long observation periods, and the possibility of linking different databases that contain information on the patient. Moreover, data are usually available for quite long periods, allowing trend analysis [5, 6]. Furthermore, thanks to the ongoing collection process, the data are up to date. The disadvantages include the variable quality of collected data, which will be discussed in the following section, and the absence of specific information of interest for clinical research (i.e., information that is usually recorded in medical records). Other disadvantages are the difficulties in drawing causal conclusions due to the presence of bias or confounding influences and the possible misclassification of outcome or exposure. For example, Ioannidis [7] states the possibility of the differential misclassification of control variables when comparing groups of hospitals. In this article, we will discuss the general characteristics of administrative databases that the researcher must consider to perform a correct analysis. Subsequently, we will describe the main issues of a study that uses administrative databases and some fields of application.
General aspects in the use of administrative databases Administrative data are collected with various purposes, often with the main aim of obtaining a reimbursement. Because these databases are not planned for clinical research, they should be carefully used for research purposes because the data of interest for clinical research may not be accurately captured [2, 3]. For example, for chronic conditions such as diabetes, which are usually managed on an outpatient basis, inpatient data may not be capable of capturing treatment patterns [8]. Therefore, a better way to detect these conditions may be to use databases other than
123
Intern Emerg Med
those of discharge abstracts, such as the database of outpatient care (the database of medical care and treatments on an outpatient basis), or of drug prescriptions. Moreover, information that is controlled for reimbursement purposes may be of better quality than other information [9]. The researcher should know which information can be considered of good quality and which should be excluded from the analysis. In general, it is important to know a database’s architecture, rules for ‘‘upload,’’ stored information, units of observation, etc. This type of knowledge is best achieved if different experts are involved in an interdisciplinary work. The research group should include experts of the particular data set, experts of coding rules, clinicians and statisticians. Countries differ in the administration of financing for public and private healthcare facilities. As a consequence, administrative databases may contain data from both public and private facilities or from public facilities only. In the latter case, only a portion of the patient population is captured. Depending on the country, the population that is covered by these administrative data sets may be limited. For example, in the US, the Medicare system covers the population aged 65 years or older. However, in several countries, such as Italy, Denmark, and Sweden, the total resident population is represented [8]. However, by sampling within the population of interest, administrative databases allow a large sample size, which reduces sampling errors and increases external validity. Indeed, the heterogeneity of the population is often well represented, allowing studies to reflect ‘‘real world’’ practice [2]. It is clear that the database characteristics, and the population to which they refer, affect the results of the study in terms of the population involved, detection of cases of interest, facilities to which the data refer, etc. Several authors (see, for example, [9]), have stated the importance of clearly describing the database characteristics by answering questions such as ‘‘why’’ and ‘‘how’’ the data set was created. Moreover, as we will discuss in the following section, to perform an analysis with these data sets, several decisions are taken into account, including the treatment of missing data, criteria for case selection, evaluation of comorbidities, selection of observation units, etc.
Issues concerning studies involving administrative databases Data quality The Manitoba Centre for Health Policy suggests an interesting framework to assess data quality, which can be used
123
as a guideline [10]. In the phase of data acquisition, the centre evaluates, among other characteristics, completeness (rate of missing values), correctness (invalid data, out of range data, outliers), and internal and external validity. Some characteristics of the internal validity are internal consistency, which is the agreement between fields, and temporal consistency, which evaluates changes in data over time. For example, in a project on patients who were hospitalised for heart failure in the Lombardia region (Italy), we find that, for patients who were hospitalised in other Italian regions, abnormal mortality rate values are due to a change in coding discharge status that occurred in the study period [11]. An important issue concerning data quality is cases that have a missing patient personal identifier (ID). To allow privacy protection, a data set identifier is assigned to each patient. This ID replaces the usual identifier that is used in the ‘‘real world.’’ The patient’s ID can be used to identify different records in the data set that refer to the same subject (for example, to detect re-hospitalisations or death). Therefore, the inability to identify the patient may eventually lead to the exclusion of the specific record from the analysis. For this reason, the number of records with missing IDs should be evaluated, and other information in these records, such as age, gender and diagnoses, should be analysed. Case selection When dealing with administrative data instead of the customary observational studies, a change of perspective occurs. The data are already available, whereas the patients of interest must be found. Patient selection is often made on the basis of specific codes of disease. Several countries use the ICD9 and ICD10 disease classification systems to code diseases treated during a hospital stay in discharge abstracts. Using these codes and their position in the discharge abstract, which distinguishes between primary and secondary diagnoses, the researchers can define extraction criteria to select the patients of interest. For example, a possible extraction criterion to detect patients who are affected by heart failure is the presence of a 428.x code as the primary diagnosis. Using these codes to select patients, a question arises about the association between the codes and the real disease that is documented in the health records. An additional question arises about the best combination of codes for selecting patients for a particular study. Several studies have addressed the issue of the best algorithms with which to identify patients who are affected by a specific disease by comparing the results obtained using different codes or different data sets (for example, hospital discharge
Intern Emerg Med
abstracts or outpatient care data sets). Examples of these algorithms can be found in [12] and references therein for the detection of patients who are affected by heart failure. In [13], six methods for the detection of drug adverse events from hospital abstracts are discussed. To validate the detection algorithms, selected cases are compared with the clinical standard of chart review, and the results are expressed in terms of sensitivity (SE), specificity (SP) and positive predictive value (PPV). Systematic reviews of different methods for identifying patients who are affected by specific diseases are also available (see [12] for an example on heart failure). Comorbidities The control of confounding influences when using administrative data to predict risk or evaluate performance is a key topic. In this sense, it is critical to take into account in the statistical analysis the patients’ comorbidities. Several measures have been proposed to control for patient conditions using administrative data, which lack detailed clinical information. Specifically, comorbidity scores have become increasingly important because they are easy to apply and widely accepted [14, 15]. When dealing with hospital discharge abstracts, secondary diagnoses are usually used to identify comorbidities, and comorbidity scores are usually constructed by assigning weights to relevant comorbidities. Researchers have proposed several comorbidity scores (see [16] for a systematic review of comorbidity indices for administrative data). The authors of the systematic review find that most of the indices are better able to predict long-term mortality than short-term mortality, inpatient mortality or any mortality within 30 days from admission. This might be explained by considering that comorbidities are under-reported in patients with severe acute conditions, as stated in the literature [17]. They also find that of the comorbidity scores, the Romano version of Charlson [18] and Elixhauser [17] measures are best for predicting long-term mortality. Following [15], several factors affect the performance of a comorbidity score, including the clinical conditions included in the score and their associated weight, the distribution of comorbidities in the population under study, the outcomes of interest, and the accuracy and completeness of the data. Another factor that affects comorbidity scores is the socalled look-back period (i.e., the time before the index event), which can be considered to detect patients’ possible comorbidities. It is quite easy to use administrative data, which usually cover several years, to collect information on a patient’s history from the same or from other data sets. Several authors have suggested that a look-back period of
1 year before the index event is suitable to improve mortality prediction [16]. Statistical analysis The statistical methods used to analyse the data of administrative data sets are the same as those used for observational studies. However, with administrative data sets, specific aspects of the methods must be considered due to the usually very large sample size and the clustered nature of sampling. When dealing with administrative databases, a sample size of tens or hundreds of thousands of units of observation is quite usual. As a consequence, the associations between outcome variables and exposures or patient characteristics are quite easily statistically significant, even if the clinical significance is negligible. The exposure variables or patient characteristics that are clinically known to influence the outcomes should be considered in the interpretation of results. Some authors have suggested that they should also be considered in model selection [1]. The issue of ‘‘time-dependent’’ variables is common in observational studies, but it deserves particular attention in analysing routinely collected data. Time-dependent variables refer to variables that influence the outcome, and display changing values during the observation period. Van Walraven et al. [9] suggest the example of elderly patients who were discharged from an Ontario hospital with a primary diagnosis of hip fracture. They linked the database of drug prescriptions to determine if these patients received a prescription for laxatives or anti-flatulence medication. A survival analysis, which simply divided patients into two groups (those with prescriptions and those without), leads to the conclusion that receiving a prescription is associated with a significantly decreased risk of death. This result, which is implausible from a clinical point of view, occurs because some patients die before having a chance to receive a prescription for the considered drugs. The authors suggest that a correct way to analyse these data is to treat drug prescription as a time-dependent variable in the model. Through such an analysis, the researchers find an increased risk of death associated with a prescription for laxatives or anti-flatulence medication. The clustered nature of data is another issue to consider when dealing with administrative data sets [9]. For example, data on hospitalisations often concern events that happen in patients who are discharged from several hospitals. Therefore, patients who are discharged from the same hospital are likely to have more similar outcomes than those who are discharged from another hospital. Treating patients as independent subjects may result in overestimating the precision of estimates and misleading or
123
Intern Emerg Med
incorrect conclusions. Multilevel models, which can account for the hierarchical nature of data, must be applied. As in other observational studies, the causal relationship between exposure factors and outcomes is difficult to prove using administrative data because of confounding events and bias. Some methods can be adopted to evaluate the presence of confounding factors. For example, the Cochrane developed a tool to evaluate the risk of bias in the results of non-randomised studies that compare the effect of two or more interventions [19]. Furthermore, ‘‘negative controls’’ can be used to detect confounding events or bias in arbitrary observational studies. Negative controls are used in biology to rule out possible non-causal interpretations of results, but they can also be used in other observational studies, as discussed in [20].
Examples In the following sections, we will present some sparse, but suggestive, examples of studies based on administrative data. Population-based epidemiological studies As previously stated, in several countries, the use of administrative databases allows population-based studies to evaluate, for example, the incidence, prevalence, and temporal trends of specific diseases and health conditions and the associated mortality. Such an evaluation is possible when a data set covers the entire population of a region in terms of an important healthcare event such as hospital admission. Zarrinkoub et al. [21] studied the epidemiology of heart failure based on data of the 2.1 million inhabitants of Sweden. The authors determined the current prevalence, incidence, mortality, and 5-year survival rate of congestive heart failure (CHF) and possible temporal changes in Sweden. The selection of patients who are affected by the disease of interest may also be performed through linking different data sets. Lipscombe and Hux [5] enrolled patients who were affected by diabetes using the hospital discharge abstracts and physician service claims. They constructed a validated diabetes registry with a high sensitivity (86 %) and specificity (97 %) for the identification of patients with diabetes recorded in primary care records. According to the disease of interest, an event is considered to be an incident if it is preceded by a suitably longtime period that is free from events of the same nature. In this way, the onset of clinical conditions can be recognised. Lipscombe and Hux provide trends in diabetes incidence, prevalence and mortality for a time period of 10 years, i.e.,
123
from 1995 to 2005. They find a large increase in the prevalence of diabetes in Ontario, due to a rising incidence and declining mortality, and request a public health intervention. In fact, population-based studies may be good instruments for policy makers to consider when designing public interventions. Outcomes studies The outcomes that can be investigated using administrative data include mortality, both in terms of short- and long-term mortality, hospital readmission and length of stay in the hospital. Such outcomes are quite easily determined from administrative data sets, and provide information on patients’ health conditions. In particular, long-term mortality can be studied thanks to the linkage with residents’ personal data sets, which give information on the vital status of patients during the period under investigation. In clinical trials, the prognosis of patients who are affected by specific diseases is studied with more clinical details. However, these studies are frequently criticised because they often enroll patients who are unrepresentative of the general population. Long et al. highlight that clinical trials that study the prognosis of patients who are affected by heart failure often exclude older women and individuals with significant comorbidities. Studies that are based on administrative data analyse outcomes in the general population. Although they lack clinical data, which are available in clinical research, these investigations allow to study the effect of demographic characteristics, e.g. sex and age, and major medical conditions, e.g. comorbidities, over a short or long period of time. Obviously, this characteristic of administrative data (compared to clinical trials) holds in general and in particular when studying outcomes such as mortality. Patients’ low socioeconomic status is considered to be a risk factor, or is associated with risk factors, of the onset and development of several diseases. Indicators of residents’ socioeconomic status are usually evaluated by institutions for social research. When calculated at the area level, such indicators can be included in the analysis on the basis of the patients’ postal codes, preserving their privacy [22]. The knowledge of the effects of socioeconomic status on outcomes may be useful in evaluating the effectiveness of primary and secondary prevention. In a 10-year study on the incidence and short-term outcomes of acute myocardial infarction (AMI) in The Netherlands, Koopman et al. [23] find that although substantial improvements are observed in the incidence and outcomes, young and female groups show smaller improvements. They also find that socioeconomic inequalities persist in AMI incidence and do not narrow over time.
Intern Emerg Med
Quality of care studies Administrative data can be used in the evaluation of quality of healthcare services and systems, as will be discussed below. A systematic review on the use of English administrative databases for the assessment of healthcare outcomes conclude that the volume of published studies on quality of care, the length of the study period and the number of outcomes assessed per study has increased over the years [24]. Moreover, the authors conclude that the importance of these studies is likely to continue to increase. Common topics in the field of quality of care are inequalities of outcomes and treatments and their trends, the association between hospital volume and processes and outcomes, and avoidable hospitalisations. Let us consider Joynt et al.’s [25] work on patients affected by CHF. They used Medicare claims to examine the association between hospital volumes, outcomes and costs of treatment. The starting point of this study is that outcomes remain suboptimal, despite pharmacologic and technical advances in the diagnosis and management of these patients. They find that some hospitals perform better than others, and, sometimes, with lower costs. Moreover, they find that hospitals with higher volumes, a proxy of the expertise in managing CHF, perform better in terms of quality of care and outcomes but have higher costs. An important achievement in the field of quality of care evaluation using administrative data is that proposed by the Agency for Healthcare Research and Quality (AHRQ). The agency proposed a set of measures, called Quality Indicators (QIs), based on in-hospital discharge abstracts. Hospital managers, policy makers, and researchers can use QIs to highlight potential quality concerns, identify areas that need further investigation, and track changes over time. QIs are divided into the following four groups: inpatient (IQIs), safety (SQIs), prevention (PQIs), and paediatric. Inpatient and safety QIs reflect the quality of care inside hospitals. In particular, IQIs focus on in-hospital mortality for selected procedures and medical conditions, and on the utilisation and volume of procedures. Conversely, PQIs use hospital discharge data to identify conditions for which good outpatient care can prevent hospitalisations, i.e. ambulatory care sensitive conditions (ACSCs). Gao et al. [26] used PQIs to develop a predictive model to identify high-risk patients, for which early interventions could reduce ACSCs hospitalisations. The model shows a high predictive ability. Thus, primary care providers may be able to use this instrument to reduce such hospitalisations. Another issue that is related to quality of care and the use of administrative data is the comparison of the performance of hospitals or groups of hospitals, e.g. teaching vs. non-teaching hospitals. Moreover, the utility of public
reporting in improving quality of care is frequently discussed. The importance of evaluating potential benefits and preventing unexpected consequences (for example, the tendency to develop risk adverse behaviour in care delivery), is stressed [27]. Ioannidis highlights several drawbacks in the use of administrative data to compare groups of hospitals [7]. For instance, the differential misclassification of important covariates in different groups cannot be excluded (e.g. comorbidities coding). Moreover, several confounders are not recorded, and cannot be accounted for in the models (e.g. smoking status and body mass index). In conclusion, caution is needed when comparing hospitals using administrative data, especially for public reporting or financing. Adverse drug events studies Adverse drug events (ADEs) are of major concern in healthcare, and a safety problem for patients. They are responsible for increased morbidity, mortality and costs for healthcare systems. When a medication is approved for use, pharmacological surveillance must be introduced to detect ADEs. Several approaches can be applied, from spontaneous reporting to focused observational studies, including those based on administrative data. In particular, the databases of hospitalisations, outpatient care, and drug dispensations from pharmacies are useful for research. The advantages and disadvantages in the use of administrative data for the detection of ADEs are well described in Suissa [28]. Some advantages are typical of studies based on this type of data, e.g. population coverage. A very large sample size and long observation period allow the possibility of detecting rare drug-related events. Moreover, they allow the observation of ADEs in a ‘‘realworld’’ setting for long periods and overcome the broad exclusion criteria and short follow-up of randomised clinical trials (RCTs). Some disadvantages are also typical of the use of administrative data, including the lack of information on important covariates (e.g. smoking status, alcohol use and any other patient characteristic that influences the healthcare outcome), and the need for validation studies on diagnostic codes. Moreover, patients’ compliances with prescribed pharmacological treatment are unknown. An example of the use of administrative data for the detection of adverse drug reactions (ADRs), which are a subset of ADEs, can be found in Miguel et al. [13]. The authors selected a group of ICD-9-CM codes, including ‘E’ codes, for detecting ADRs. A chart review was performed on a sample to assess the predictive positive value (PPV) of every ICD code; PPVs vary from 60 to 100 %. For the 9271,122 hospitalisations from 2000 to 2009, they find 116,720 ADRs. In the same period, the number of
123
Intern Emerg Med Table 1 Advantages and disadvantages in the use of administrative data Advantages Population coverage Very large sample size Heterogeneity of covered population (‘‘real-world’’ perspective) Long observation periods Up to date information No additional costs for gathering data Possibility to link several sources of information (e.g. hospitalisations, outpatient care, drug prescriptions, vital status, etc.) Disadvantages Expertise on the specific database is needed to ensure the correct use of data Not planned for health research, they lack of specific clinical information Quality of data is affected by their administrative use, e.g. for reimbursement Case selection based on diagnosis codes; validation studies are needed Possible misclassification of outcomes or exposure Difficulties to control confounding factors and to draw causal relations Statistical significance is easily achieved, clinical relevance has to be taken into account in statistical analysis and in discussing results
Table 2 A checklist to read a work based on administrative data What information an article describing a research based on administrative data should contain Target population and population covered by the data source considered (e.g. is the health coverage universal?) Characteristics of the underlying health care system (e.g. are there data on both public and private structures?) Which data sets were used in the study (e.g. hospitalisations, outpatient, drug prescriptions, socioeconomic status) and which variables are taken into account Criteria for cases selection: which diagnosis or procedure codes and, in case, in which position (i.e. principal or secondary) Study supporting the use of these codes Which methods was chosen to detect comorbidities Consistency of statistical methods with the characteristics and structure of data (e.g. use of models for clustered data, time-dependent variables correctly handled, evaluation of clinical significance, etc.)
spontaneous reports was 13,562. Although the real number of ADRs is likely to be smaller than that found using administrative data, given the PPVs, it should certainly be higher than that from spontaneous reports. Using data from all public and private hospitals in Western Australia during the period of 1980–2000, Zhang et al. [29] report the results of a study that aimed to identify factors that predict repeated admission to the hospital due to ADRs. ADRs were identified by specific codes in the ICD-9, ICD-9CM, and ICD-10-AM systems, according to the classification system in use. The presence of specific comorbidities, but not advancing age, was found to be associated with repeat admissions due to ADRs. The comorbidities that were often managed in the community were especially associated with such admissions. However, the authors emphasise that this association may have been magnified, as the identification of ADRs would have been more likely because the presence of comorbidities increases the patient’s contact with the healthcare system.
123
As previously mentioned and briefly shown in the above examples, administrative data can complement information found in RCTs and in post-marketing surveillance. However, careful attention must be put into the design, case selection, and analysis of studies based on administrative data. Detection codes need to be validated. Multiple sources of data, such as pharmacy data, should be used to overcome, if possible, the lack of important information. Furthermore, the cause–effect relationship of statistical associations with ‘‘statistically significant’’ predictive factors has to be deeply considered, discussed and understood.
Conclusions The number of studies based on administrative data (for different purposes) and the availability of such databases have been increasing in recent years. We discussed the advantages and disadvantages of the use of these data (see
Intern Emerg Med
Table 1 for a review). According to the type of regional or national healthcare system, they allow population-based studies that evaluate changes and trends of outcomes over time, and observe the effects of treatments from a ‘‘realworld’’ perspective, including their adverse effects. However, they rely heavily on coded diagnoses to select patients of interest, potentially resulting in selection bias, especially when codes do not selectively represent the condition under study. In fact, validation studies of selected codes are often needed. Moreover, administrative data lack detailed clinical information, making it difficult to adjust for relevant confounding factors in the comparison of different groups of treatments (confounding by indication). In conclusion, administrative data can be very useful in studying health-related issues, but their use requires care. To avoid the pitfalls that are related to their use and to ensure the correct use, an in-depth understanding of the underlying healthcare and data registration and coding systems is needed. An interdisciplinary collaboration of clinicians, coding experts, epidemiologists, and statisticians is necessary to define the appropriate study design, case selection, and data analysis. The reader has to be aware of the represented population, particularly considering the reported selection strategy, the eligibility criteria, and the comorbidity detection method applied (see Table 2). In the future, the increasing availability of routinely collected data will offer more information that can be used to refine statistical analysis. The patients’ socioeconomic status, even if at an aggregated level and exposed to the ecological fallacy, can be used in several countries. We imagine that, at least at the hospital level, other clinical information (e.g. clinical laboratory test results), will be automatically linkable to single patients. The use of record linking, text mining, network analysis, and machine learning techniques will likely offer a new development in the use of routinely collected data. Electronic medical records are already collected in several regions and countries and represent a comprehensive set of information on patient health status, disease progression, treatment effectiveness, and healthcare quality, including the continuity of care [30–32]. Conflict of interest of interest.
The authors declare that they have no conflict
References 1. Gavrielov-Yusim N, Friger M (2014) Use of administrative medical databases in population-based research. J Epidemiol Commun Health, to appear. doi:10.1136/jech-2013-202744 2. Nguyen LL, Barshes NR (2010) Analysis of large databases in vascular surgery. J Vasc Surg 52(3):768–774
3. Grimes DA (2010) Epidemiologic research using administrative databases—garbage in, garbage out. Obstet Gynecol 116(5): 1018–1019 4. Hoover KW, Tao G, Kent CK, Aral SO (2011) Epidemiologic research using administrative databases: garbage in, garbage out. Letter to the editor. Obstet Gynecol 117(3):729–730 5. Lipscombe LL, Hux JE (2007) Trends in diabetes prevalence, incidence, and mortality in Ontario, Canada 1995–2005: a population-based study. Lancet 369:750–756 6. Corrada E, Ferrante G, Mazzali C, Barbieri P, Merlino L, Merlini P, Presbiterio P (2014) Eleven-year trends in gender differences of treatments and mortality in ST-elevation acute myocardial infarction in northern Italy, 2000 to 2010 Am J Cardiol 114(3):336–341 7. Ioannidis IP (2013) Are mortality differences detected by administrative data reliable and actionable? JAMA 309(13):1410–1411 8. Moise P (2001) Using hospital administrative databases for a disease-based approach to studying health care systems. OECD Ageing related disease study. http://www.oecd.org/sweden/ 1889879.pdf. Accessed 8 Sep 2014 9. Van Walraven C, Austin P (2012) Administrative database research has unique characteristics that can risk biased results. J Clin Epidemiol 65:126–131 10. Azimaee M, Smith M, Lix L, Ostapyk T, Burchill C, Pham Hong S. (2014) Manitoba centre for health policy. MCHP data quality framework. http://umanitoba.ca/faculties/medicine/units/commu nity_health_sciences/departmental_units/mchp/protocol/media/ Data_Quality_Framework.pdf. Accessed 8 Sep 2014 11. Mazzali C, Maistrello M, Ieva F, Barbieri P (2015) Methodological issues in the use of administrative databases to study heart failure. In: Paganoni AM, Secchi P (eds) Advances in complex data modeling and computational methods in statistics. Contributions to Statistics, Springer, pp 149–160 12. Saczynski JS, Andrade SE, Harrold LR, Tjia J, Cutrona SL, Dodd KS, Goldberg RJ, Gurwitz JH (2012) A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf 21(S1):129–140 13. Miguel A, Marques B, Freitas A, Lopes F, Azevedo L, Costa Pereira A (2013) Detection of adverse drug reactions using hospital databases—a nationwide study in Portugal. Pharmacoepidemiol Drug Saf 22:907–913 14. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S (2011) A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol 64:749–759 15. Schneeweiss S, Wang PS, Avorn J, Glynn RJ (2003) Improved comorbidity adjustment for predicting mortality in medicare populations. Health Serv Res 38(4):1103–1120 16. Sharabiani MTA, Aylin P, Bottle A (2012) Systematic review of comorbidity indices for administrative data. Med Care 50:1109–1118 17. Elixhauser A, Steiner C, Harris DR, Coffey RM (1998) Comorbidity measures for use with administrative data. Med Care 36:8–12 18. Romano PS, Roos LL, Jollis JG (1993) Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol 46:1075–1079 19. Sterne JAC, Higgins JPT, Reeves BC on behalf of the development group for ACROBAT-NRSI. A cochrane risk of bias assessment tool: for non-randomized studies of interventions (ACROBAT-NRSI), version 1.0.0, 24 September 2014. http:// www.riskofbias.info. Accessed 24 Sep 2014 20. Lipsitich M, Tchetgen ET, Cohen T (2010) Negative controls. A tool for detecting confounding and bias in observational studies. Epidemiol 21(3):383–388 21. Zarrinkoub R, Wettermark B, Wandell P, Mejhert M, Szulkin R, Ljunggren G, Kahan T (2013) The epidemiology of heart failure,
123
Intern Emerg Med
22.
23.
24.
25.
26. 27.
based on data for 2.1 million inhabitants in Sweden. Eur J Heart Fail 15:995–1002 Jong P, Vowinckel E, Liu PP, Gong YTuJV (2002) Prognosis and determinants of survival in patients newly hospitalized for heart failure—a population-based study. Arch Intern Med 162:1689–1694 Koopman C, Bots ML, van Oeffelen AAM, van Dis I, Verschuren WMM, Engelfriet PM, Capewell S, Vaartjes I (2013) Population trends and inequalities in incidence and short-term outcome of acute myocardial infarction between 1998 and 2007. Int J Cardiol 168:993–998 Sinha S, Peach G, Poloniecki JD, Thompson MM, Holt PJ (2012) Studies using english administrative data (hospital episode statistics) to assess health-care outcomes—systematic review and recommendations for reporting. Eur J Public Health 23(1):86–92 Joynt KE, Orav EJ, Jha AK (2011) The association between hospital volume and processes, outcomes, and costs of care for congestive heart failure. Ann Intern Med 154(2):94–102 Gao J, Moran E, Li Y, Almenoff PL (2014) Predicting potentially avoidable hospitalizations. Med Care 52:164–171 Dehmer GJ, Drozda JP, Brindis RG, Masoudi FA, Rumsfeld JS, Slattery LE, Oetgen WJ (2014) Public reporting of clinical data— an update for cardiovascular specialists. J Am Coll Cardiol 63(13):1239–1245
123
28. Suissa S, Garbe E (2007) Primer: administrative health databases on observational studies of drug effects—advantages and disadvantages. Nat Clin Pract Rheumatol 3(12):725–732 29. Zhang M, Holman CDJ, Price SD, Sanfilippo FM, Preen DB, Bulsara MK (2009) Comorbidity and repeat admission to hospital for adverse drug reactions in older adults: retrospective cohort study. Br Med J. doi:10.1136/bmj.a2752 30. ASSR (2006) Gli Indicatori per la qualita`: strumenti, metodi, risultati. Supplemento a Monitor 15. http://www.agenas.it/ima ges/agenas/monitor/quaderno/pdf/1_INDICATORI_QUALITA. pdf. Accessed 24 Sep 2014 31. Mazzali C, Severgnini B, Maistrello M, Barbieri P, Marzegalli M (2013) Heart diseases registries based on healthcare databases. In: Grieco N., Marzegalli M, Paganoni AM (eds) New diagnostic, therapeutic and organizational strategies for acute coronary syndromes patients. Springer-Verlag Italia and Physica-Verlag Heidelberg, Germany, pp 25–46 32. Ballerio S, Cerizza D (2013) Using text mining to validate diagnosis of acute myocardial infarction. In Grieco N, Marzegalli M, Paganoni AM (eds) New diagnostic, therapeutic and organizational strategies for acute coronary syndromes patients. Springer-Verlag Italia and Physica-Verlag Heidelberg, Germany, pp 69–82
CE - COCHRANE’S CORNER
Use of administrative data in healthcare research Cristina Mazzali • Piergiorgio Duca
Received: 24 September 2014 / Accepted: 12 February 2015 Ó SIMI 2015
Abstract Health research based on administrative data and the availability of regional or national administrative databases has been increasing in recent years. We will discuss the general characteristics of administrative data and specific aspects of their use for health research purposes, indicating their advantages and disadvantages. Some fields of application will be discussed and described through examples. Keywords Administrative data Utilisation databases Healthcare research Research methods
Introduction Healthcare administrative databases are large repositories of data on healthcare systems that are routinely collected by healthcare providers and other institutions (e.g. civil registry). They provide a variety of already stored data with an ongoing collection process [1]. They may contain information on hospitalisations, outpatient care, drug prescriptions, rehabilitation services, implanted endoprostheses, psychiatric service, etc. The advantages and disadvantages of the use of administrative databases in epidemiological, clinical and healthcare research are well known [1–4]. The advantages include large sample size, population coverage and heterogeneity, which allow researchers to reflect the ‘‘real world’’ practice [2]. Other advantages are the absence of
C. Mazzali (&) P. Duca Statistics and Biometrics Unit-’L. Sacco’ Department of Biomedical and Clinical Sciences, University of Milan, Via G.B. Grassi, 74, 20157 Milan, Italy e-mail: [email protected]
additional costs for gathering data, long observation periods, and the possibility of linking different databases that contain information on the patient. Moreover, data are usually available for quite long periods, allowing trend analysis [5, 6]. Furthermore, thanks to the ongoing collection process, the data are up to date. The disadvantages include the variable quality of collected data, which will be discussed in the following section, and the absence of specific information of interest for clinical research (i.e., information that is usually recorded in medical records). Other disadvantages are the difficulties in drawing causal conclusions due to the presence of bias or confounding influences and the possible misclassification of outcome or exposure. For example, Ioannidis [7] states the possibility of the differential misclassification of control variables when comparing groups of hospitals. In this article, we will discuss the general characteristics of administrative databases that the researcher must consider to perform a correct analysis. Subsequently, we will describe the main issues of a study that uses administrative databases and some fields of application.
General aspects in the use of administrative databases Administrative data are collected with various purposes, often with the main aim of obtaining a reimbursement. Because these databases are not planned for clinical research, they should be carefully used for research purposes because the data of interest for clinical research may not be accurately captured [2, 3]. For example, for chronic conditions such as diabetes, which are usually managed on an outpatient basis, inpatient data may not be capable of capturing treatment patterns [8]. Therefore, a better way to detect these conditions may be to use databases other than
123
Intern Emerg Med
those of discharge abstracts, such as the database of outpatient care (the database of medical care and treatments on an outpatient basis), or of drug prescriptions. Moreover, information that is controlled for reimbursement purposes may be of better quality than other information [9]. The researcher should know which information can be considered of good quality and which should be excluded from the analysis. In general, it is important to know a database’s architecture, rules for ‘‘upload,’’ stored information, units of observation, etc. This type of knowledge is best achieved if different experts are involved in an interdisciplinary work. The research group should include experts of the particular data set, experts of coding rules, clinicians and statisticians. Countries differ in the administration of financing for public and private healthcare facilities. As a consequence, administrative databases may contain data from both public and private facilities or from public facilities only. In the latter case, only a portion of the patient population is captured. Depending on the country, the population that is covered by these administrative data sets may be limited. For example, in the US, the Medicare system covers the population aged 65 years or older. However, in several countries, such as Italy, Denmark, and Sweden, the total resident population is represented [8]. However, by sampling within the population of interest, administrative databases allow a large sample size, which reduces sampling errors and increases external validity. Indeed, the heterogeneity of the population is often well represented, allowing studies to reflect ‘‘real world’’ practice [2]. It is clear that the database characteristics, and the population to which they refer, affect the results of the study in terms of the population involved, detection of cases of interest, facilities to which the data refer, etc. Several authors (see, for example, [9]), have stated the importance of clearly describing the database characteristics by answering questions such as ‘‘why’’ and ‘‘how’’ the data set was created. Moreover, as we will discuss in the following section, to perform an analysis with these data sets, several decisions are taken into account, including the treatment of missing data, criteria for case selection, evaluation of comorbidities, selection of observation units, etc.
Issues concerning studies involving administrative databases Data quality The Manitoba Centre for Health Policy suggests an interesting framework to assess data quality, which can be used
123
as a guideline [10]. In the phase of data acquisition, the centre evaluates, among other characteristics, completeness (rate of missing values), correctness (invalid data, out of range data, outliers), and internal and external validity. Some characteristics of the internal validity are internal consistency, which is the agreement between fields, and temporal consistency, which evaluates changes in data over time. For example, in a project on patients who were hospitalised for heart failure in the Lombardia region (Italy), we find that, for patients who were hospitalised in other Italian regions, abnormal mortality rate values are due to a change in coding discharge status that occurred in the study period [11]. An important issue concerning data quality is cases that have a missing patient personal identifier (ID). To allow privacy protection, a data set identifier is assigned to each patient. This ID replaces the usual identifier that is used in the ‘‘real world.’’ The patient’s ID can be used to identify different records in the data set that refer to the same subject (for example, to detect re-hospitalisations or death). Therefore, the inability to identify the patient may eventually lead to the exclusion of the specific record from the analysis. For this reason, the number of records with missing IDs should be evaluated, and other information in these records, such as age, gender and diagnoses, should be analysed. Case selection When dealing with administrative data instead of the customary observational studies, a change of perspective occurs. The data are already available, whereas the patients of interest must be found. Patient selection is often made on the basis of specific codes of disease. Several countries use the ICD9 and ICD10 disease classification systems to code diseases treated during a hospital stay in discharge abstracts. Using these codes and their position in the discharge abstract, which distinguishes between primary and secondary diagnoses, the researchers can define extraction criteria to select the patients of interest. For example, a possible extraction criterion to detect patients who are affected by heart failure is the presence of a 428.x code as the primary diagnosis. Using these codes to select patients, a question arises about the association between the codes and the real disease that is documented in the health records. An additional question arises about the best combination of codes for selecting patients for a particular study. Several studies have addressed the issue of the best algorithms with which to identify patients who are affected by a specific disease by comparing the results obtained using different codes or different data sets (for example, hospital discharge
Intern Emerg Med
abstracts or outpatient care data sets). Examples of these algorithms can be found in [12] and references therein for the detection of patients who are affected by heart failure. In [13], six methods for the detection of drug adverse events from hospital abstracts are discussed. To validate the detection algorithms, selected cases are compared with the clinical standard of chart review, and the results are expressed in terms of sensitivity (SE), specificity (SP) and positive predictive value (PPV). Systematic reviews of different methods for identifying patients who are affected by specific diseases are also available (see [12] for an example on heart failure). Comorbidities The control of confounding influences when using administrative data to predict risk or evaluate performance is a key topic. In this sense, it is critical to take into account in the statistical analysis the patients’ comorbidities. Several measures have been proposed to control for patient conditions using administrative data, which lack detailed clinical information. Specifically, comorbidity scores have become increasingly important because they are easy to apply and widely accepted [14, 15]. When dealing with hospital discharge abstracts, secondary diagnoses are usually used to identify comorbidities, and comorbidity scores are usually constructed by assigning weights to relevant comorbidities. Researchers have proposed several comorbidity scores (see [16] for a systematic review of comorbidity indices for administrative data). The authors of the systematic review find that most of the indices are better able to predict long-term mortality than short-term mortality, inpatient mortality or any mortality within 30 days from admission. This might be explained by considering that comorbidities are under-reported in patients with severe acute conditions, as stated in the literature [17]. They also find that of the comorbidity scores, the Romano version of Charlson [18] and Elixhauser [17] measures are best for predicting long-term mortality. Following [15], several factors affect the performance of a comorbidity score, including the clinical conditions included in the score and their associated weight, the distribution of comorbidities in the population under study, the outcomes of interest, and the accuracy and completeness of the data. Another factor that affects comorbidity scores is the socalled look-back period (i.e., the time before the index event), which can be considered to detect patients’ possible comorbidities. It is quite easy to use administrative data, which usually cover several years, to collect information on a patient’s history from the same or from other data sets. Several authors have suggested that a look-back period of
1 year before the index event is suitable to improve mortality prediction [16]. Statistical analysis The statistical methods used to analyse the data of administrative data sets are the same as those used for observational studies. However, with administrative data sets, specific aspects of the methods must be considered due to the usually very large sample size and the clustered nature of sampling. When dealing with administrative databases, a sample size of tens or hundreds of thousands of units of observation is quite usual. As a consequence, the associations between outcome variables and exposures or patient characteristics are quite easily statistically significant, even if the clinical significance is negligible. The exposure variables or patient characteristics that are clinically known to influence the outcomes should be considered in the interpretation of results. Some authors have suggested that they should also be considered in model selection [1]. The issue of ‘‘time-dependent’’ variables is common in observational studies, but it deserves particular attention in analysing routinely collected data. Time-dependent variables refer to variables that influence the outcome, and display changing values during the observation period. Van Walraven et al. [9] suggest the example of elderly patients who were discharged from an Ontario hospital with a primary diagnosis of hip fracture. They linked the database of drug prescriptions to determine if these patients received a prescription for laxatives or anti-flatulence medication. A survival analysis, which simply divided patients into two groups (those with prescriptions and those without), leads to the conclusion that receiving a prescription is associated with a significantly decreased risk of death. This result, which is implausible from a clinical point of view, occurs because some patients die before having a chance to receive a prescription for the considered drugs. The authors suggest that a correct way to analyse these data is to treat drug prescription as a time-dependent variable in the model. Through such an analysis, the researchers find an increased risk of death associated with a prescription for laxatives or anti-flatulence medication. The clustered nature of data is another issue to consider when dealing with administrative data sets [9]. For example, data on hospitalisations often concern events that happen in patients who are discharged from several hospitals. Therefore, patients who are discharged from the same hospital are likely to have more similar outcomes than those who are discharged from another hospital. Treating patients as independent subjects may result in overestimating the precision of estimates and misleading or
123
Intern Emerg Med
incorrect conclusions. Multilevel models, which can account for the hierarchical nature of data, must be applied. As in other observational studies, the causal relationship between exposure factors and outcomes is difficult to prove using administrative data because of confounding events and bias. Some methods can be adopted to evaluate the presence of confounding factors. For example, the Cochrane developed a tool to evaluate the risk of bias in the results of non-randomised studies that compare the effect of two or more interventions [19]. Furthermore, ‘‘negative controls’’ can be used to detect confounding events or bias in arbitrary observational studies. Negative controls are used in biology to rule out possible non-causal interpretations of results, but they can also be used in other observational studies, as discussed in [20].
Examples In the following sections, we will present some sparse, but suggestive, examples of studies based on administrative data. Population-based epidemiological studies As previously stated, in several countries, the use of administrative databases allows population-based studies to evaluate, for example, the incidence, prevalence, and temporal trends of specific diseases and health conditions and the associated mortality. Such an evaluation is possible when a data set covers the entire population of a region in terms of an important healthcare event such as hospital admission. Zarrinkoub et al. [21] studied the epidemiology of heart failure based on data of the 2.1 million inhabitants of Sweden. The authors determined the current prevalence, incidence, mortality, and 5-year survival rate of congestive heart failure (CHF) and possible temporal changes in Sweden. The selection of patients who are affected by the disease of interest may also be performed through linking different data sets. Lipscombe and Hux [5] enrolled patients who were affected by diabetes using the hospital discharge abstracts and physician service claims. They constructed a validated diabetes registry with a high sensitivity (86 %) and specificity (97 %) for the identification of patients with diabetes recorded in primary care records. According to the disease of interest, an event is considered to be an incident if it is preceded by a suitably longtime period that is free from events of the same nature. In this way, the onset of clinical conditions can be recognised. Lipscombe and Hux provide trends in diabetes incidence, prevalence and mortality for a time period of 10 years, i.e.,
123
from 1995 to 2005. They find a large increase in the prevalence of diabetes in Ontario, due to a rising incidence and declining mortality, and request a public health intervention. In fact, population-based studies may be good instruments for policy makers to consider when designing public interventions. Outcomes studies The outcomes that can be investigated using administrative data include mortality, both in terms of short- and long-term mortality, hospital readmission and length of stay in the hospital. Such outcomes are quite easily determined from administrative data sets, and provide information on patients’ health conditions. In particular, long-term mortality can be studied thanks to the linkage with residents’ personal data sets, which give information on the vital status of patients during the period under investigation. In clinical trials, the prognosis of patients who are affected by specific diseases is studied with more clinical details. However, these studies are frequently criticised because they often enroll patients who are unrepresentative of the general population. Long et al. highlight that clinical trials that study the prognosis of patients who are affected by heart failure often exclude older women and individuals with significant comorbidities. Studies that are based on administrative data analyse outcomes in the general population. Although they lack clinical data, which are available in clinical research, these investigations allow to study the effect of demographic characteristics, e.g. sex and age, and major medical conditions, e.g. comorbidities, over a short or long period of time. Obviously, this characteristic of administrative data (compared to clinical trials) holds in general and in particular when studying outcomes such as mortality. Patients’ low socioeconomic status is considered to be a risk factor, or is associated with risk factors, of the onset and development of several diseases. Indicators of residents’ socioeconomic status are usually evaluated by institutions for social research. When calculated at the area level, such indicators can be included in the analysis on the basis of the patients’ postal codes, preserving their privacy [22]. The knowledge of the effects of socioeconomic status on outcomes may be useful in evaluating the effectiveness of primary and secondary prevention. In a 10-year study on the incidence and short-term outcomes of acute myocardial infarction (AMI) in The Netherlands, Koopman et al. [23] find that although substantial improvements are observed in the incidence and outcomes, young and female groups show smaller improvements. They also find that socioeconomic inequalities persist in AMI incidence and do not narrow over time.
Intern Emerg Med
Quality of care studies Administrative data can be used in the evaluation of quality of healthcare services and systems, as will be discussed below. A systematic review on the use of English administrative databases for the assessment of healthcare outcomes conclude that the volume of published studies on quality of care, the length of the study period and the number of outcomes assessed per study has increased over the years [24]. Moreover, the authors conclude that the importance of these studies is likely to continue to increase. Common topics in the field of quality of care are inequalities of outcomes and treatments and their trends, the association between hospital volume and processes and outcomes, and avoidable hospitalisations. Let us consider Joynt et al.’s [25] work on patients affected by CHF. They used Medicare claims to examine the association between hospital volumes, outcomes and costs of treatment. The starting point of this study is that outcomes remain suboptimal, despite pharmacologic and technical advances in the diagnosis and management of these patients. They find that some hospitals perform better than others, and, sometimes, with lower costs. Moreover, they find that hospitals with higher volumes, a proxy of the expertise in managing CHF, perform better in terms of quality of care and outcomes but have higher costs. An important achievement in the field of quality of care evaluation using administrative data is that proposed by the Agency for Healthcare Research and Quality (AHRQ). The agency proposed a set of measures, called Quality Indicators (QIs), based on in-hospital discharge abstracts. Hospital managers, policy makers, and researchers can use QIs to highlight potential quality concerns, identify areas that need further investigation, and track changes over time. QIs are divided into the following four groups: inpatient (IQIs), safety (SQIs), prevention (PQIs), and paediatric. Inpatient and safety QIs reflect the quality of care inside hospitals. In particular, IQIs focus on in-hospital mortality for selected procedures and medical conditions, and on the utilisation and volume of procedures. Conversely, PQIs use hospital discharge data to identify conditions for which good outpatient care can prevent hospitalisations, i.e. ambulatory care sensitive conditions (ACSCs). Gao et al. [26] used PQIs to develop a predictive model to identify high-risk patients, for which early interventions could reduce ACSCs hospitalisations. The model shows a high predictive ability. Thus, primary care providers may be able to use this instrument to reduce such hospitalisations. Another issue that is related to quality of care and the use of administrative data is the comparison of the performance of hospitals or groups of hospitals, e.g. teaching vs. non-teaching hospitals. Moreover, the utility of public
reporting in improving quality of care is frequently discussed. The importance of evaluating potential benefits and preventing unexpected consequences (for example, the tendency to develop risk adverse behaviour in care delivery), is stressed [27]. Ioannidis highlights several drawbacks in the use of administrative data to compare groups of hospitals [7]. For instance, the differential misclassification of important covariates in different groups cannot be excluded (e.g. comorbidities coding). Moreover, several confounders are not recorded, and cannot be accounted for in the models (e.g. smoking status and body mass index). In conclusion, caution is needed when comparing hospitals using administrative data, especially for public reporting or financing. Adverse drug events studies Adverse drug events (ADEs) are of major concern in healthcare, and a safety problem for patients. They are responsible for increased morbidity, mortality and costs for healthcare systems. When a medication is approved for use, pharmacological surveillance must be introduced to detect ADEs. Several approaches can be applied, from spontaneous reporting to focused observational studies, including those based on administrative data. In particular, the databases of hospitalisations, outpatient care, and drug dispensations from pharmacies are useful for research. The advantages and disadvantages in the use of administrative data for the detection of ADEs are well described in Suissa [28]. Some advantages are typical of studies based on this type of data, e.g. population coverage. A very large sample size and long observation period allow the possibility of detecting rare drug-related events. Moreover, they allow the observation of ADEs in a ‘‘realworld’’ setting for long periods and overcome the broad exclusion criteria and short follow-up of randomised clinical trials (RCTs). Some disadvantages are also typical of the use of administrative data, including the lack of information on important covariates (e.g. smoking status, alcohol use and any other patient characteristic that influences the healthcare outcome), and the need for validation studies on diagnostic codes. Moreover, patients’ compliances with prescribed pharmacological treatment are unknown. An example of the use of administrative data for the detection of adverse drug reactions (ADRs), which are a subset of ADEs, can be found in Miguel et al. [13]. The authors selected a group of ICD-9-CM codes, including ‘E’ codes, for detecting ADRs. A chart review was performed on a sample to assess the predictive positive value (PPV) of every ICD code; PPVs vary from 60 to 100 %. For the 9271,122 hospitalisations from 2000 to 2009, they find 116,720 ADRs. In the same period, the number of
123
Intern Emerg Med Table 1 Advantages and disadvantages in the use of administrative data Advantages Population coverage Very large sample size Heterogeneity of covered population (‘‘real-world’’ perspective) Long observation periods Up to date information No additional costs for gathering data Possibility to link several sources of information (e.g. hospitalisations, outpatient care, drug prescriptions, vital status, etc.) Disadvantages Expertise on the specific database is needed to ensure the correct use of data Not planned for health research, they lack of specific clinical information Quality of data is affected by their administrative use, e.g. for reimbursement Case selection based on diagnosis codes; validation studies are needed Possible misclassification of outcomes or exposure Difficulties to control confounding factors and to draw causal relations Statistical significance is easily achieved, clinical relevance has to be taken into account in statistical analysis and in discussing results
Table 2 A checklist to read a work based on administrative data What information an article describing a research based on administrative data should contain Target population and population covered by the data source considered (e.g. is the health coverage universal?) Characteristics of the underlying health care system (e.g. are there data on both public and private structures?) Which data sets were used in the study (e.g. hospitalisations, outpatient, drug prescriptions, socioeconomic status) and which variables are taken into account Criteria for cases selection: which diagnosis or procedure codes and, in case, in which position (i.e. principal or secondary) Study supporting the use of these codes Which methods was chosen to detect comorbidities Consistency of statistical methods with the characteristics and structure of data (e.g. use of models for clustered data, time-dependent variables correctly handled, evaluation of clinical significance, etc.)
spontaneous reports was 13,562. Although the real number of ADRs is likely to be smaller than that found using administrative data, given the PPVs, it should certainly be higher than that from spontaneous reports. Using data from all public and private hospitals in Western Australia during the period of 1980–2000, Zhang et al. [29] report the results of a study that aimed to identify factors that predict repeated admission to the hospital due to ADRs. ADRs were identified by specific codes in the ICD-9, ICD-9CM, and ICD-10-AM systems, according to the classification system in use. The presence of specific comorbidities, but not advancing age, was found to be associated with repeat admissions due to ADRs. The comorbidities that were often managed in the community were especially associated with such admissions. However, the authors emphasise that this association may have been magnified, as the identification of ADRs would have been more likely because the presence of comorbidities increases the patient’s contact with the healthcare system.
123
As previously mentioned and briefly shown in the above examples, administrative data can complement information found in RCTs and in post-marketing surveillance. However, careful attention must be put into the design, case selection, and analysis of studies based on administrative data. Detection codes need to be validated. Multiple sources of data, such as pharmacy data, should be used to overcome, if possible, the lack of important information. Furthermore, the cause–effect relationship of statistical associations with ‘‘statistically significant’’ predictive factors has to be deeply considered, discussed and understood.
Conclusions The number of studies based on administrative data (for different purposes) and the availability of such databases have been increasing in recent years. We discussed the advantages and disadvantages of the use of these data (see
Intern Emerg Med
Table 1 for a review). According to the type of regional or national healthcare system, they allow population-based studies that evaluate changes and trends of outcomes over time, and observe the effects of treatments from a ‘‘realworld’’ perspective, including their adverse effects. However, they rely heavily on coded diagnoses to select patients of interest, potentially resulting in selection bias, especially when codes do not selectively represent the condition under study. In fact, validation studies of selected codes are often needed. Moreover, administrative data lack detailed clinical information, making it difficult to adjust for relevant confounding factors in the comparison of different groups of treatments (confounding by indication). In conclusion, administrative data can be very useful in studying health-related issues, but their use requires care. To avoid the pitfalls that are related to their use and to ensure the correct use, an in-depth understanding of the underlying healthcare and data registration and coding systems is needed. An interdisciplinary collaboration of clinicians, coding experts, epidemiologists, and statisticians is necessary to define the appropriate study design, case selection, and data analysis. The reader has to be aware of the represented population, particularly considering the reported selection strategy, the eligibility criteria, and the comorbidity detection method applied (see Table 2). In the future, the increasing availability of routinely collected data will offer more information that can be used to refine statistical analysis. The patients’ socioeconomic status, even if at an aggregated level and exposed to the ecological fallacy, can be used in several countries. We imagine that, at least at the hospital level, other clinical information (e.g. clinical laboratory test results), will be automatically linkable to single patients. The use of record linking, text mining, network analysis, and machine learning techniques will likely offer a new development in the use of routinely collected data. Electronic medical records are already collected in several regions and countries and represent a comprehensive set of information on patient health status, disease progression, treatment effectiveness, and healthcare quality, including the continuity of care [30–32]. Conflict of interest of interest.
The authors declare that they have no conflict
References 1. Gavrielov-Yusim N, Friger M (2014) Use of administrative medical databases in population-based research. J Epidemiol Commun Health, to appear. doi:10.1136/jech-2013-202744 2. Nguyen LL, Barshes NR (2010) Analysis of large databases in vascular surgery. J Vasc Surg 52(3):768–774
3. Grimes DA (2010) Epidemiologic research using administrative databases—garbage in, garbage out. Obstet Gynecol 116(5): 1018–1019 4. Hoover KW, Tao G, Kent CK, Aral SO (2011) Epidemiologic research using administrative databases: garbage in, garbage out. Letter to the editor. Obstet Gynecol 117(3):729–730 5. Lipscombe LL, Hux JE (2007) Trends in diabetes prevalence, incidence, and mortality in Ontario, Canada 1995–2005: a population-based study. Lancet 369:750–756 6. Corrada E, Ferrante G, Mazzali C, Barbieri P, Merlino L, Merlini P, Presbiterio P (2014) Eleven-year trends in gender differences of treatments and mortality in ST-elevation acute myocardial infarction in northern Italy, 2000 to 2010 Am J Cardiol 114(3):336–341 7. Ioannidis IP (2013) Are mortality differences detected by administrative data reliable and actionable? JAMA 309(13):1410–1411 8. Moise P (2001) Using hospital administrative databases for a disease-based approach to studying health care systems. OECD Ageing related disease study. http://www.oecd.org/sweden/ 1889879.pdf. Accessed 8 Sep 2014 9. Van Walraven C, Austin P (2012) Administrative database research has unique characteristics that can risk biased results. J Clin Epidemiol 65:126–131 10. Azimaee M, Smith M, Lix L, Ostapyk T, Burchill C, Pham Hong S. (2014) Manitoba centre for health policy. MCHP data quality framework. http://umanitoba.ca/faculties/medicine/units/commu nity_health_sciences/departmental_units/mchp/protocol/media/ Data_Quality_Framework.pdf. Accessed 8 Sep 2014 11. Mazzali C, Maistrello M, Ieva F, Barbieri P (2015) Methodological issues in the use of administrative databases to study heart failure. In: Paganoni AM, Secchi P (eds) Advances in complex data modeling and computational methods in statistics. Contributions to Statistics, Springer, pp 149–160 12. Saczynski JS, Andrade SE, Harrold LR, Tjia J, Cutrona SL, Dodd KS, Goldberg RJ, Gurwitz JH (2012) A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf 21(S1):129–140 13. Miguel A, Marques B, Freitas A, Lopes F, Azevedo L, Costa Pereira A (2013) Detection of adverse drug reactions using hospital databases—a nationwide study in Portugal. Pharmacoepidemiol Drug Saf 22:907–913 14. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S (2011) A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol 64:749–759 15. Schneeweiss S, Wang PS, Avorn J, Glynn RJ (2003) Improved comorbidity adjustment for predicting mortality in medicare populations. Health Serv Res 38(4):1103–1120 16. Sharabiani MTA, Aylin P, Bottle A (2012) Systematic review of comorbidity indices for administrative data. Med Care 50:1109–1118 17. Elixhauser A, Steiner C, Harris DR, Coffey RM (1998) Comorbidity measures for use with administrative data. Med Care 36:8–12 18. Romano PS, Roos LL, Jollis JG (1993) Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol 46:1075–1079 19. Sterne JAC, Higgins JPT, Reeves BC on behalf of the development group for ACROBAT-NRSI. A cochrane risk of bias assessment tool: for non-randomized studies of interventions (ACROBAT-NRSI), version 1.0.0, 24 September 2014. http:// www.riskofbias.info. Accessed 24 Sep 2014 20. Lipsitich M, Tchetgen ET, Cohen T (2010) Negative controls. A tool for detecting confounding and bias in observational studies. Epidemiol 21(3):383–388 21. Zarrinkoub R, Wettermark B, Wandell P, Mejhert M, Szulkin R, Ljunggren G, Kahan T (2013) The epidemiology of heart failure,
123
Intern Emerg Med
22.
23.
24.
25.
26. 27.
based on data for 2.1 million inhabitants in Sweden. Eur J Heart Fail 15:995–1002 Jong P, Vowinckel E, Liu PP, Gong YTuJV (2002) Prognosis and determinants of survival in patients newly hospitalized for heart failure—a population-based study. Arch Intern Med 162:1689–1694 Koopman C, Bots ML, van Oeffelen AAM, van Dis I, Verschuren WMM, Engelfriet PM, Capewell S, Vaartjes I (2013) Population trends and inequalities in incidence and short-term outcome of acute myocardial infarction between 1998 and 2007. Int J Cardiol 168:993–998 Sinha S, Peach G, Poloniecki JD, Thompson MM, Holt PJ (2012) Studies using english administrative data (hospital episode statistics) to assess health-care outcomes—systematic review and recommendations for reporting. Eur J Public Health 23(1):86–92 Joynt KE, Orav EJ, Jha AK (2011) The association between hospital volume and processes, outcomes, and costs of care for congestive heart failure. Ann Intern Med 154(2):94–102 Gao J, Moran E, Li Y, Almenoff PL (2014) Predicting potentially avoidable hospitalizations. Med Care 52:164–171 Dehmer GJ, Drozda JP, Brindis RG, Masoudi FA, Rumsfeld JS, Slattery LE, Oetgen WJ (2014) Public reporting of clinical data— an update for cardiovascular specialists. J Am Coll Cardiol 63(13):1239–1245
123
28. Suissa S, Garbe E (2007) Primer: administrative health databases on observational studies of drug effects—advantages and disadvantages. Nat Clin Pract Rheumatol 3(12):725–732 29. Zhang M, Holman CDJ, Price SD, Sanfilippo FM, Preen DB, Bulsara MK (2009) Comorbidity and repeat admission to hospital for adverse drug reactions in older adults: retrospective cohort study. Br Med J. doi:10.1136/bmj.a2752 30. ASSR (2006) Gli Indicatori per la qualita`: strumenti, metodi, risultati. Supplemento a Monitor 15. http://www.agenas.it/ima ges/agenas/monitor/quaderno/pdf/1_INDICATORI_QUALITA. pdf. Accessed 24 Sep 2014 31. Mazzali C, Severgnini B, Maistrello M, Barbieri P, Marzegalli M (2013) Heart diseases registries based on healthcare databases. In: Grieco N., Marzegalli M, Paganoni AM (eds) New diagnostic, therapeutic and organizational strategies for acute coronary syndromes patients. Springer-Verlag Italia and Physica-Verlag Heidelberg, Germany, pp 25–46 32. Ballerio S, Cerizza D (2013) Using text mining to validate diagnosis of acute myocardial infarction. In Grieco N, Marzegalli M, Paganoni AM (eds) New diagnostic, therapeutic and organizational strategies for acute coronary syndromes patients. Springer-Verlag Italia and Physica-Verlag Heidelberg, Germany, pp 69–82
