CORRESPONDENCE CORRESPONDENCE
TOPIC
Response to Journal Club: The Epidemiology of Admissions of Nontraumatic Subarachnoid Hemorrhage in the United States Fred Rincon, MD, MSc, MBE Departments of Neurology, Neurosurgery, and Divisions of Critical Care and Neurotrauma, Thomas Jefferson University, Philadelphia, Pennsylvania Correspondence: Fred Rincon, MD, MSc, MBE, FACP, FCCP, FCCM, Division of Critical Care and Neurotrauma, 909 Walnut Street, 3rd Floor, Philadelphia, PA 19107. E-mail: [email protected]
Copyright © 2013 by the Congress of Neurological Surgeons.
C
urrently, there are limited data regarding long-term epidemiological trends of aneurysmal subarachnoid hemorrhage (SAH) in the United States. Much of the available data on SAH are derived from population-based studies, but these results may not be generalizable based on the different incidence rates of SAH based on geographic location.1,2 Population-based studies in SAH populations may require considerable infrastructure, participation of subjects, and financial resources. To this end, large administrative databases are an essential tool to allow for analysis of long-term data with a robust sample size representative of the actual population.3 Although administrative databases are not primarily designed for research purposes, they accrue a significant amount of information, providing a robust alternative to traditional research used by observational studies. The quality of the data in administrative databases depends on specific incentives for data reporting, which the most prevalent is financial.4 As a result, diagnoses and pertinent therapies with high financial burden for health care systems, such as SAH,5-8 are documented better than less costly diagnoses or medical interventions.4 Conversely, less costly diseases may require pilot-scaled field investigations to support hypothesis testing.4 Significant disadvantages exist when performing research based on administrative databases. In general, accrued data from administrative databases suffer from the same biases as observational data. First, exposure and outcome misclassification.4 This term is related to measurement error and is probably the most common form of bias in epidemiological research. Misclassification occurs frequently when broad definitions of exposure or outcome occur. Imperative for accuracy in exposure and outcome interpretation is adequate validation of an identifiable algorithm for appropriate ascertainment. In reference to SAH, the ICD9CM code has been shown to adequately represent the disease.9 Confounding is another limitation of estimates obtained from administrative databases. Based on
230 | VOLUME 74 | NUMBER 2 | FEBRUARY 2014
the robustness of data sampling, point estimates and 95% confidence intervals (CIs) are extremely precise. However, clinical relevance on the basis of sole statistical significance could be problematic, impractical, and has been criticized in the past.10 Investigators can bypass this limitation by using or “creating” surrogate variables associated with the outcome as evidence by prior studies. The lack of important variables such as radiographic, physiological, and in-hospital complications is often identified as a major limitation in studies from these databases. Sample size is often recognized as an advantage when using administrative databases. However, there are limitations in relation to statistical inference. Because the distribution of the population represented by administrative databases vary, appropriate statistical techniques must be selected to analyze the data. The National Hospital Discharge Survey (NHDS), for example, (http://www.cdc. gov/nchs/nhds.htm), uses a sample that is representative of the US Hospital population. The NHDS uses a modified three-stage design. Units selected at the first stage consist of either hospitals or geographic areas such as counties, groups of counties, or metropolitan statistical areas. Next, within a sampled geographic area, hospitals are selected. At the last stage, systematic random sampling is used to select discharges within sampled hospitals. Because of the complex multistage design of NHDS, the survey data must be inflated or weighted in order to produce national estimates.11 Generation of NHDS-based estimates relies primarily on the appropriate evaluation of the relative standard error (RSE), which are measurements generated from statistical extrapolation.12 The standard error is primarily a measure of sampling variability that occurs by chance because only a sample rather than the entire universe is surveyed.13 While bias could be present in the NHDS, estimates are less likely to be biased because the hospital sampling frame is far less restricted than that from other datasets such as the Nationwide Inpatient Sample (NIS).12 Though the idea that clinical relevance on the basis of statistical
www.neurosurgery-online.com
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.
CORRESPONDENCE
significance is a prevalent requirement to satisfy the validity of results in the research community, the applicability of this approach may be questionable, particularly because standard statistical techniques such as regression or even trend analysis may not be suitable to analyze estimates generated from statistical extrapolation.13 Finally, generalizability is a common claim made against the use of administrative databases for research purposes.4 Epidemiologists believe that results from observational studies are meaningful only if the are applicable to large geographic definitions. The premise of generalizability may be misleading particularly because other variables beyond subject or case geographic area of residence such as age, race/ethnicity, and gender, are important for the definition of the cohort.4 In order to interpret the results, investigators should optimize their study’s internal validity by recognizing and providing estimates based on these additional factors. To this end, the description by Bogason et al14 in reference to the methodology of the recently published study entitled “The Epidemiology of Admissions of Nontraumatic Subarachnoid Hemorrhage in the United States” by Rincon et al,3 is both valid and accurate. Results from epidemiological estimates based upon administrative databases, which are not designed to test hypothesis as in prospective studies, should be interpreted with caution, but provide sufficient support to future hypothesis testing, resource allocation for research, and other important public health initiatives. Disclosure Dr Rincon has received salary support from the American Heart Association (AHA 12CRP12050342). The author has no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.
NEUROSURGERY
REFERENCES 1. de Rooij NK, Linn FH, van der Plas JA, Algra A, Rinkel GJ. Incidence of subarachnoid haemorrhage: a systematic review with emphasis on region, age, gender and time trends. J Neurol Neurosurg Psychiatry. 2007;78(12):1365–1372. 2. Nieuwkamp DJ, Setz LE, Algra A, Linn FH, de Rooij NK, Rinkel GJ. Changes in case fatality of aneurysmal subarachnoid haemorrhage over time, according to age, sex, and region: a meta-analysis. Lancet Neurol. 2009;8(7):635–642. 3. Rincon F, Rossenwasser RH, Dumont A. The epidemiology of admissions of nontraumatic subarachnoid hemorrhage in the United States. Neurosurgery. 2013; 73(2):217-222; discussion 212-213. 4. Gavrielov-Yusim N, Friger M. Use of administrative medical databases in population-based research. J Epidemiol Community Health. 2013. doi: 10.1136/ jech-2013-202744. [Epub ahead of print] 5. Dodel R, Winter Y, Ringel F, et al. Cost of illness in subarachnoid hemorrhage: a German longitudinal study. Stroke. 2010;41(12):2918–2923. 6. Rivero-Arias O, Gray A, Wolstenholme J. Burden of disease and costs of aneurysmal subarachnoid haemorrhage (aSAH) in the United Kingdom. Cost Eff Resour Allc. 2010;8:6. 7. Roos YB, Dijkgraaf MG, Albrecht KW, et al. Direct costs of modern treatment of aneurysmal subarachnoid hemorrhage in the first year after diagnosis. Stroke. 2002; 33(6):1595–1599. 8. Yundt KD, Dacey RG Jr, Diringer MN. Hospital resource utilization in the treatment of cerebral aneurysms. J Neurosurg. 1996;85(3):403–409. 9. Tirschwell DL, Longstreth WT Jr. Validating administrative data in stroke research. Stroke. 2002;33(10):2465–2470. 10. Borenstein M. The case for confidence intervals in controlled clinical trials. Control Clin Trials. 1994;15(5):411–428. 11. Hall MJ, Levant S, DeFrances CJ. Hospitalization for Stroke in U.S. Hospital, 1989-2009. NCHS Data Brief. Hyattsville, MD: Statistics NCfH; 2012. 12. Comparative analysis of HCUP and NHDS inpatient discharge data. Available at: http://www.ahrq.gov/research/data/hcup/nhds/niscomp.html. Accessed December 1, 2013. 13. National Hospital Discharge Survey. Hyasttsville, MD: Public Health Service, Statistics NCfH; 2010. 14. Bogason ET, Anderson B, Brandmeir NJ, et al. The epidemiology of admissions of nontraumatic subarachnoid hemorrhage in the United States. Neurosurgery. 2013; 73(2):212-223.
VOLUME 74 | NUMBER 2 | FEBRUARY 2014 | 231
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.
TOPIC
Response to Journal Club: The Epidemiology of Admissions of Nontraumatic Subarachnoid Hemorrhage in the United States Fred Rincon, MD, MSc, MBE Departments of Neurology, Neurosurgery, and Divisions of Critical Care and Neurotrauma, Thomas Jefferson University, Philadelphia, Pennsylvania Correspondence: Fred Rincon, MD, MSc, MBE, FACP, FCCP, FCCM, Division of Critical Care and Neurotrauma, 909 Walnut Street, 3rd Floor, Philadelphia, PA 19107. E-mail: [email protected]
Copyright © 2013 by the Congress of Neurological Surgeons.
C
urrently, there are limited data regarding long-term epidemiological trends of aneurysmal subarachnoid hemorrhage (SAH) in the United States. Much of the available data on SAH are derived from population-based studies, but these results may not be generalizable based on the different incidence rates of SAH based on geographic location.1,2 Population-based studies in SAH populations may require considerable infrastructure, participation of subjects, and financial resources. To this end, large administrative databases are an essential tool to allow for analysis of long-term data with a robust sample size representative of the actual population.3 Although administrative databases are not primarily designed for research purposes, they accrue a significant amount of information, providing a robust alternative to traditional research used by observational studies. The quality of the data in administrative databases depends on specific incentives for data reporting, which the most prevalent is financial.4 As a result, diagnoses and pertinent therapies with high financial burden for health care systems, such as SAH,5-8 are documented better than less costly diagnoses or medical interventions.4 Conversely, less costly diseases may require pilot-scaled field investigations to support hypothesis testing.4 Significant disadvantages exist when performing research based on administrative databases. In general, accrued data from administrative databases suffer from the same biases as observational data. First, exposure and outcome misclassification.4 This term is related to measurement error and is probably the most common form of bias in epidemiological research. Misclassification occurs frequently when broad definitions of exposure or outcome occur. Imperative for accuracy in exposure and outcome interpretation is adequate validation of an identifiable algorithm for appropriate ascertainment. In reference to SAH, the ICD9CM code has been shown to adequately represent the disease.9 Confounding is another limitation of estimates obtained from administrative databases. Based on
230 | VOLUME 74 | NUMBER 2 | FEBRUARY 2014
the robustness of data sampling, point estimates and 95% confidence intervals (CIs) are extremely precise. However, clinical relevance on the basis of sole statistical significance could be problematic, impractical, and has been criticized in the past.10 Investigators can bypass this limitation by using or “creating” surrogate variables associated with the outcome as evidence by prior studies. The lack of important variables such as radiographic, physiological, and in-hospital complications is often identified as a major limitation in studies from these databases. Sample size is often recognized as an advantage when using administrative databases. However, there are limitations in relation to statistical inference. Because the distribution of the population represented by administrative databases vary, appropriate statistical techniques must be selected to analyze the data. The National Hospital Discharge Survey (NHDS), for example, (http://www.cdc. gov/nchs/nhds.htm), uses a sample that is representative of the US Hospital population. The NHDS uses a modified three-stage design. Units selected at the first stage consist of either hospitals or geographic areas such as counties, groups of counties, or metropolitan statistical areas. Next, within a sampled geographic area, hospitals are selected. At the last stage, systematic random sampling is used to select discharges within sampled hospitals. Because of the complex multistage design of NHDS, the survey data must be inflated or weighted in order to produce national estimates.11 Generation of NHDS-based estimates relies primarily on the appropriate evaluation of the relative standard error (RSE), which are measurements generated from statistical extrapolation.12 The standard error is primarily a measure of sampling variability that occurs by chance because only a sample rather than the entire universe is surveyed.13 While bias could be present in the NHDS, estimates are less likely to be biased because the hospital sampling frame is far less restricted than that from other datasets such as the Nationwide Inpatient Sample (NIS).12 Though the idea that clinical relevance on the basis of statistical
www.neurosurgery-online.com
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.
CORRESPONDENCE
significance is a prevalent requirement to satisfy the validity of results in the research community, the applicability of this approach may be questionable, particularly because standard statistical techniques such as regression or even trend analysis may not be suitable to analyze estimates generated from statistical extrapolation.13 Finally, generalizability is a common claim made against the use of administrative databases for research purposes.4 Epidemiologists believe that results from observational studies are meaningful only if the are applicable to large geographic definitions. The premise of generalizability may be misleading particularly because other variables beyond subject or case geographic area of residence such as age, race/ethnicity, and gender, are important for the definition of the cohort.4 In order to interpret the results, investigators should optimize their study’s internal validity by recognizing and providing estimates based on these additional factors. To this end, the description by Bogason et al14 in reference to the methodology of the recently published study entitled “The Epidemiology of Admissions of Nontraumatic Subarachnoid Hemorrhage in the United States” by Rincon et al,3 is both valid and accurate. Results from epidemiological estimates based upon administrative databases, which are not designed to test hypothesis as in prospective studies, should be interpreted with caution, but provide sufficient support to future hypothesis testing, resource allocation for research, and other important public health initiatives. Disclosure Dr Rincon has received salary support from the American Heart Association (AHA 12CRP12050342). The author has no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.
NEUROSURGERY
REFERENCES 1. de Rooij NK, Linn FH, van der Plas JA, Algra A, Rinkel GJ. Incidence of subarachnoid haemorrhage: a systematic review with emphasis on region, age, gender and time trends. J Neurol Neurosurg Psychiatry. 2007;78(12):1365–1372. 2. Nieuwkamp DJ, Setz LE, Algra A, Linn FH, de Rooij NK, Rinkel GJ. Changes in case fatality of aneurysmal subarachnoid haemorrhage over time, according to age, sex, and region: a meta-analysis. Lancet Neurol. 2009;8(7):635–642. 3. Rincon F, Rossenwasser RH, Dumont A. The epidemiology of admissions of nontraumatic subarachnoid hemorrhage in the United States. Neurosurgery. 2013; 73(2):217-222; discussion 212-213. 4. Gavrielov-Yusim N, Friger M. Use of administrative medical databases in population-based research. J Epidemiol Community Health. 2013. doi: 10.1136/ jech-2013-202744. [Epub ahead of print] 5. Dodel R, Winter Y, Ringel F, et al. Cost of illness in subarachnoid hemorrhage: a German longitudinal study. Stroke. 2010;41(12):2918–2923. 6. Rivero-Arias O, Gray A, Wolstenholme J. Burden of disease and costs of aneurysmal subarachnoid haemorrhage (aSAH) in the United Kingdom. Cost Eff Resour Allc. 2010;8:6. 7. Roos YB, Dijkgraaf MG, Albrecht KW, et al. Direct costs of modern treatment of aneurysmal subarachnoid hemorrhage in the first year after diagnosis. Stroke. 2002; 33(6):1595–1599. 8. Yundt KD, Dacey RG Jr, Diringer MN. Hospital resource utilization in the treatment of cerebral aneurysms. J Neurosurg. 1996;85(3):403–409. 9. Tirschwell DL, Longstreth WT Jr. Validating administrative data in stroke research. Stroke. 2002;33(10):2465–2470. 10. Borenstein M. The case for confidence intervals in controlled clinical trials. Control Clin Trials. 1994;15(5):411–428. 11. Hall MJ, Levant S, DeFrances CJ. Hospitalization for Stroke in U.S. Hospital, 1989-2009. NCHS Data Brief. Hyattsville, MD: Statistics NCfH; 2012. 12. Comparative analysis of HCUP and NHDS inpatient discharge data. Available at: http://www.ahrq.gov/research/data/hcup/nhds/niscomp.html. Accessed December 1, 2013. 13. National Hospital Discharge Survey. Hyasttsville, MD: Public Health Service, Statistics NCfH; 2010. 14. Bogason ET, Anderson B, Brandmeir NJ, et al. The epidemiology of admissions of nontraumatic subarachnoid hemorrhage in the United States. Neurosurgery. 2013; 73(2):212-223.
VOLUME 74 | NUMBER 2 | FEBRUARY 2014 | 231
Copyright © Congress of Neurological Surgeons. Unauthorized reproduction of this article is prohibited.
