ORIGINAL ARTICLE
Systematic Review of Risk Adjustment Models of Hospital Length of Stay (LOS) Mingshan Lu, PhD,* Tolulope Sajobi, PhD,w Kelsey Lucyk, MSc,w Diane Lorenzetti, PhD,w and Hude Quan, MD, PhDw
Background: Policy decisions in health care, such as hospital performance evaluation and performance-based budgeting, require an accurate prediction of hospital length of stay (LOS). This paper provides a systematic review of risk adjustment models for hospital LOS, and focuses primarily on studies that use administrative data. Methods: MEDLINE, EMBASE, Cochrane, PubMed, and EconLit were searched for studies that tested the performance of risk adjustment models in predicting hospital LOS. We included studies that tested models developed for the general inpatient population, and excluded those that analyzed risk factors only correlated with LOS, impact analyses, or those that used disease-specific scales and indexes to predict LOS. Results: Our search yielded 3973 abstracts, of which 37 were included. These studies used various disease groupers and severity/ morbidity indexes to predict LOS. Few models were developed specifically for explaining hospital LOS; most focused primarily on explaining resource spending and the costs associated with hospital LOS, and applied these models to hospital LOS. We found a large variation in predictive power across different LOS predictive models. The best model performance for most studies fell in the range of 0.30–0.60, approximately. Conclusions: The current risk adjustment methodologies for predicting LOS are still limited in terms of models, predictors, and predictive power. One possible approach to improving the performance of LOS risk adjustment models is to include more diseasespecific variables, such as disease-specific or condition-specific measures, and functional measures. For this approach, however, more comprehensive and standardized data are urgently needed. In addition, statistical methods and evaluation tools more appropriate to LOS should be tested and adopted. Key Words: length of stay, risk adjustment, systematic review (Med Care 2015;53: 355–365)
C
ost containment and improved efficiency are primary issues in health care policy as countries face budgetary constraints in health care globally. The existing literature suggests that hospital length of stay (LOS) is a key indicator of inpatient resource use and hospital efficiency.1–6 As a result, research and policy efforts have focused on reducing unnecessary hospital LOS.7–10 The literature suggests that LOS exhibits large variation across hospitals in different samples. For example, hospital LOS for acute myocardial infarction patients may vary from 4.17 to 41.0 days and from 2.7 to 16.0 days for C-section patients.11–14 Important policy decisions, such as hospital budgeting and performance evaluation, therefore must account for differences in characteristics of the underlying population. This calls for the development of LOS risk adjustment models. The risk adjustment literature has grown significantly during the past few decades, which has been driven mostly by countries adopting methods for prospective hospital payment.15 However, few studies report risk adjustors or risk adjustment models that have been specifically developed for LOS. Instead, much of the existing literature focuses on predicting hospital spending and applies these methods to predict LOS.16–18 It remains unclear which risk adjustors and risk adjustment models best control for patient severity when analyzing hospital LOS, and whether the predictive power of existing models are satisfactory. To fill this gap, this paper provides a systematic review of the performance of risk adjustment models for hospital LOS. Our review focuses on methods that can be applied to general inpatient populations. We summarize existing LOS risk adjustors, study populations and data sources, statistical methods, and model specifications for each study. We present and compare the performance of LOS risk adjustment models, and provide recommendations for future research based on our discussion of results.
METHODS From the *Department of Economics and Community Health Sciences; and wDepartment of Community Health Sciences and Centre for Health and Policy Studies, University of Calgary, Calgary, AB, Canada. T.S.’s research was supported by the MSI Foundation Grant; H.Q.’s salary was supported by Alberta Innovates Health Solutions; and K.L. was supported by an Achievers in Medical Sciences scholarship at the time of writing. The authors declare no conflict of interest. Reprints: Mingshan Lu, PhD, Department of Economics, University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4. Email: [email protected]. Copyright r 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0025-7079/15/5304-0355
Medical Care
Volume 53, Number 4, April 2015
Data Sources and Searches We searched a total of 5 peer-reviewed databases: MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, PubMed, and EconLit. We implemented a search strategy that combined text words and MeSH subject headings (using the Boolean and positional Operators AND, OR, and “adj”) that represented the concepts relevant to our research question: Concept 1: length of stay or LOS or hospital days or days adj3 care; Concept 2: risk assessment or regression analys* or model or models or logistic regression or www.lww-medicalcare.com |
355
Medical Care
Lu et al
risk adjustment* or theoretical model or computer simulation or predictive or predicted. We compiled all references into an Excel dataset that we used to select studies for inclusion or exclusion.
Criteria for Study Selection We searched studies published between January 1993 and June 2013. We limited our search to studies that contained English language abstracts and used data from nonexperimental hospital settings to test the statistical performance of risk adjustment models in predicting hospital LOS. Articles were included for review if they: (1) measured LOS as an outcome/ dependent variable; (2) were original research; or, (3) were a full-text empirical study that used data from nonexperimental hospital settings to test risk adjustment model performance on predicting LOS. We excluded studies if they: (1) analyzed risk factors or determinants correlated with LOS without testing the statistical performance of risk adjustment models; (2) used disease-specific scales and indexes to predict LOS; (3) performed impact analysis with LOS as a performance measure; (4) were randomized control trials or theoretical papers; or, (5) focused on LOS for special inpatient populations. We did not impose any age limitation on the target population. We excluded studies that used disease-specific risk adjustment models, for the following reasons. First, there is an accompanying literature for many diseases whereby disease-specific factors are tested to determine which should be included in risk adjustment models. Including these studies would have made the scope and volume of our review unmanageable. Second, it is difficult to apply generalizations from one disease to another when predicting hospital LOS, which detracts from our interest in models for general inpatient populations. Finally, while disease-specific models may address questions related to disease or clinical management, they are not suitable for our interests in policy issues that relate to hospital performance evaluation or payment design. Reference lists of included studies were scanned to identify additional relevant papers, which led to the inclusion of an additional 7 articles. Following the title and abstract review, the authors discussed additional and potentially relevant risk adjustment models that might have been missed. As such, we conducted a keyword search (ie, ACG, Truven, Symmetry, Cave) of all title and abstracts, which identified an additional 19 full-text articles. A complete search strategy is available from the authors.
Volume 53, Number 4, April 2015
One of the major challenges of using a risk adjustment model is deciding which risk adjustors to use. Therefore, in synthesizing our results, we focus on 2 types of risk adjustors for patient characteristics: those that adjust for disease type using diagnostic information (ie, disease groupers) or patient case mix (ie, disease severity/morbidity indexes). We synthesized data in 3 ways. First, we summarized risk adjustors used in studies by name, type, and test population. Next, we classified studies by type of patient group: general inpatient population or disease-specific inpatients. Finally, within each patient group we presented the study population and data source including sample size, statistical method, model specification, and model performance.
RESULTS As indicated in the PRISMA flow diagram of studies in Figure 1, 7016 records were identified through database searching and 7 through reference-list searching for a total of 3973 unique citations, with duplicates removed. Title and abstract screening was conducted by 3 investigators, and 225 articles were selected for full-text review. After full-text review, 37 studies were included in this review.
LOS Risk Adjustors Table 1 presents a summary of the risk adjustors used, classified into disease groupers, disease severity/morbidity indexes, and their test populations.
Disease Groupers
The results of our review are reported in accordance with the PRISMA Statement (http://www.prisma-statement.org). Abstracts were divided into 3 groups, which were each independently reviewed by 2 of the 3 investigators (M.L., T.S., H.Q.) to decide whether the full-text article should be reviewed. Full-text articles were also divided for review in this way. Although we did conduct pretesting of inclusion and exclusion criteria, we did not calculate interrater reliability. Disagreements were resolved through discussion and consensus. Reviewers were not blinded to study author, institution, or journal.
Disease groupers or “diagnosis-related groups” (DRGs), refer to the various methods of classifying inpatients by main diagnosis or procedure.51 Within a disease group, institutions may also classify patients by the severity of their condition using a severity index. For example, “DRG” refers to an American grouper that uses the ICD-9, whereas the All Patient Refined Diagnosis-related Groups (APR-DRG) is another form of DRG that uses ICD-10-CM and includes severity of diagnosis. A comprehensive list of disease groupers can be found online.52 The most commonly used and widely tested grouper for LOS risk adjustment is DRG, an inpatient episode grouper. In total, we identified 22 studies that used disease groupers. DRGs were tested for their predictive power on LOS for general inpatient populations,19,20,23 and also acute care,24 appendectomy,25 coronary artery bypass graft (CABG) surgery,16 child delivery,26 cholecystectomy,27 elderly,28,29 hip fracture,21 inguinal hernia,30 knee and hip replacement,31,32 pneumonia,17 psychiatric,33–36 stroke,37 trauma,53 and vascular patients.38 Other commonly used disease groupers included DRG variations, such as APR-DRGs and Refined Diagnosisrelated Groups. APR-DRGs were used by 4 studies for general inpatient populations,19,20 hip fracture21 and pneumonia patients.17 Refined Diagnosis-related Groups were tested on general inpatient populations,19,42 pneumonia17 and hip fracture patients.21 Major Diagnostic Categories (MDC) were also used by 4 studies and tested on general inpatient populations,39 acute care24 and psychiatric patients.40,41 One study tested the predictive power of Case Mix Groups for LOS on psychiatric patients.22
356 | www.lww-medicalcare.com
Copyright
Synthesis of Results and Quality Assessment
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
Volume 53, Number 4, April 2015
Identification
Medical Care
7016 records identified through database searching
Review of Risk Adjustment Models of Hospital LOS
7 records identified through other sources
Included
Eligibility
Screening
3973 records remained after removal of duplicates
3050 duplicate records excluded
3973 articles screened using title and abstract
538 records excluded for not reporting LOS and risk adjustment
3435 articles assessed for eligibility based on title and abstract
3206 records excluded because eligibility criteria not met
225 full-text articles reviewed
187 records excluded because eligibility criteria not met
37 articles included in the final review
8 articles focus on general population
29 articles focus on nongeneral population
FIGURE 1. PRISMA flow diagram of study selection. LOS indicates length of stay.
Disease Severity/Comorbidity Indexes Among the large varieties of disease severity/comorbidity indexes, Charlson Index (CI) was most commonly used. Among the 37 studies reviewed, CI was used by 14 to explain LOS for general inpatient populations,43,46,47 medical, procedural, and psychiatric inpatients,47 as well as pneumonia,17,33 appendectomy,25 chest pain,45 inguinal hernia,30 child delivery,26 cholecystectomy,27 carotid endarterectomy,18 spinal cord injury,48 stroke,49 and total knee replacement inpatients.50 Patient Management Categories Relative Intensity Score was used in 3 studies and tested on inpatients with hip fracture,21 total knee replacement,50 and pneumonia.17 Acuity Index Method was used by 2 studies for pneumonia17 and hip fracture inpatients.21 Admission Severity Groups were used by 2 studies and tested on general44 and pneumonia inpatients.17 One study used MedisGroups (Atlas MQ) on hip fracture inpatients.21 Two studies used Disease Staging Mortality Probability to explain LOS for inpatients with pneumonia17 and hip fracture.21 Another 2 Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
used ICD-9-based Illness Severity Score to explain LOS for general18 and trauma inpatients.20 Physiology Score and Body Systems Count were used in 2 studies for pneumonia17 and hip fracture inpatients.21 In addition, studies included in our review also used the following indexes: Acute Physiology Score (general population),43 Chronic Disease Score (general population),46 Comorbidity Index (hip fracture),21 Comorbidity Point Score (general population),23 Computerized Severity Index (psychiatric patients),22 Count of ICD-9 Diagnoses (spinal cord injury),48 Cumulative Illness Rating Scale (spinal cord injury),48 Difficulty of Clinical Management (acute care inpatients),24 Functional-linked Variables (elderly patients),28 Laboratory Acute Physiology Score (general population),23 Modified Severity of Illness Index (elderly inpatients),29 Multipurpose Australian Comorbidity Scoring System (medical, procedural, and psychiatric inpatients),47 Nursing Diagnosis Terminology (general population),19 number of diagnoses (total knee replacement),50 patient-reported www.lww-medicalcare.com |
357
Medical Care
Lu et al
TABLE 1. List of Risk Adjustors Used in Length of Stay Studies Risk Adjustor Name
Test Population
Groupers APR-DRG
General population Hip fracture Pneumonia patients CMG Psychiatric patients DRG General population Acute care patients Appendectomy CABG surgery Child delivery Cholecystectomy Elderly Hip fracture Inguinal hernia Knee replacement patients Pneumonia patients Psychiatric patients Stroke Trauma Vascular patients MDC General population Acute care patients Psychiatric patients RDRG General population Hip fracture Pneumonia patients Severity/comorbidity measures AIM Pneumonia patients Hip fracture APS General population ASG General patients Pneumonia patients Body systems count Pneumonia patients Hip fracture Charlson Index Appendectomy Chest pain Child delivery Cholecystectomy General population Inguinal hernia Medical, procedural, and psychiatric inpatients Pneumonia patients Spinal cord injury Stroke Total knee replacement CDS General population Comorbidity Index Hip fracture COPS General population CSI Psychiatric patients Count of ICD-9 Spinal cord injury diagnoses Cumulative illness rating Spinal cord injury scale Difficulty of clinical Acute care patients management Disease staging mortality Pneumonia patients probability Hip fracture Function-linked variables Elderly patients ICISS Trauma General population LAPS General population Hip fracture
References
17 21 43 44 17 17 21 25 45 26 27 43,46 30 47 17,49 48 49 50 46 21 23 22 48 48 24 17 21 28 20 53 23 21 (Continued )
358 | www.lww-medicalcare.com
Volume 53, Number 4, April 2015
TABLE 1. List of Risk Adjustors Used in Length of Stay Studies (continued) Risk Adjustor Name
19,20 21 17 22 19,20,23 24 25 16 26 27 28,29 21 30 31,32 17 33–36 37 18 38 39 24 40,41 19,42 21 17
Test Population
MedisGroups (Atlas MQ) MSII MACSS
Elderly patients Medical, procedural, and psychiatric inpatients NDX General population No. diagnoses Total knee replacement PMC RIS Hip fracture Total knee replacement Pneumonia patients Patient-reported severity CABG surgery measures PSL Total knee replacement Physiology Score Pneumonia patients Hip fracture
References
29 47 19 50 21 50 17 16 50 17 21
ACG indicates adjusted clinical groups; AIM, Acuity Index Method; APR-DRG, all patient refined diagnosis-related groups; APS, acute physiology score; ASG, admission severity groups; CABG, coronary artery bypass graft; CDPS, chronic illness and disability payment system; CDS, chronic disease score; CMG, case mix groups; COPS, comorbidity point score; CSI, computerized severity index; DRG, diagnosisrelated groups; ICD-9, is International Classification of Diseases, 9th revision; ICISS, ICD-9-based Illness Severity Score; LAPS, Laboratory Acute Physiology Score; MACSS, multipurpose Australian comorbidity scoring system; MDC, major diagnostic categories; MSII, Modified Severity of Illness Index; NDX, nursing diagnosis terminology; PMC, patient management categories; PSL, patient severity level; RDRG, is redefined diagnosis-related groups; RIS, relative intensity score.
severity measures (CABG surgery),16 and Patient Severity Level (total knee replacement).50
LOS Risk Adjustment Models Table 2 presents the study population, data source, statistical method, model specification, and the performance of LOS risk adjustment models used for each of the 37 studies included in our review. They are separated into 2 groups: the first includes 8 studies (22%) that analyze LOS for general inpatient populations, and the second includes 29 studies (78%) that analyze LOS for inpatient populations with specific diseases.
Study Population and Data Source The primary data sources used by studies were hospitalization data (eg, admission and/or discharge), medical records, and clinical data. In some studies, primary data were supplemented by pharmacy data,23,44,46 laboratory data,23,39 demographic data,39 and patient-reported data.29 Study populations included patients from 1 or more hospital or health insurance plan, from multiple countries, over a period of 1–6 years.
Statistical Method Four types of statistical methods were adopted by the studies we reviewed: ordinary least squares (OLS), generalized linear regression models, hierarchical regression models, and data mining models. Among studies included in our review, 29 (78%) used OLS to develop a hospital LOS risk adjustment model. Seven (18%) adopted some form of a Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
Medical Care
Volume 53, Number 4, April 2015
Review of Risk Adjustment Models of Hospital LOS
TABLE 2. Performance of Length of Stay Risk Adjustment Models Study Population and Data Source
References General population Halpine and Ashworth42 Hicks and Kammerling44
Liu et al23
Liu et al54
Mozes et al39
Parker et al46
Rutledge and Osler20 Welton and Halloran19
Abstracts reported to the Hospital Medical Records Institute, New Brunswick, and Alberta coded in ICD-9-CM, from January to March 1989 (N = 282,459) Information abstracted from clinical records of general medical patients discharged from an English (UK) teaching hospital in a 15-mo period, using MedisGroups system (N = 2279) Automated hospital, laboratory, administrative, and pharmacy data from hospitalizations at 17 Northern California Kaiser-Permanente hospitals between January 1, 2002 and June 30, 2005 (N = 155,474) Hospital data from Hospital Morbidity Database System (HMDS) from a large teaching hospital in Western Australia, where numbers of episodes were at least 1% of all discharges, from July 1995 to June 1996 (N = 4589) Laboratory, hospital admission, ansd demographic data from the Combined Patient Experience (COPE) database on patients from the University of California San Francisco Medical Center and Stanford University Medical Center discharged in 1981, 1982, and 1986 (N = 73,117) Automated hospital and pharmacy data from Kaiser-Permanente of acute hospitalizations in Southern California, from April 1993 to February 1995 (N = 6721) Every patient admitted to all acute care hospitals in North Carolina during 1996 whose records contained complete data (N = 821,455) Hospitalization data on patients admitted to a university hospital in a Midwestern City, US from 1986 to 1989 (N = 123,241)
Nongeneral population Brock and Brown33 Admissions to an adult inpatient psychiatric unit of Wilford Hall Medical Center in the US, from September 1, 1987 to October 1, 1989 (N = 1040) 43 Patient-reported symptoms for patients Clark et al admitted to the general medicine service of a teaching hospital affiliated with the Indiana University School of Medicine (US) from July 1, 1996 to June 30, 1997 (N = 2126) Cost data from 10 EU countriesw from 1365 Cots et al31 hospitals in 2008 (N = 190,000) Admissions data from all patients 18–65 y Creed et al34 admitted to Manchester Royal Infirmary Psychiatric Day Hospital during a 9-mo period (N = 115) 22 Discharge abstract from adult psychiatric Durbin et al patients during 1994–1995 fiscal year in 3 acute care inpatient settings in Greater Toronto (teaching hospital; academic research, treatment, and training center; provincial psychiatry facility) (N = 355)
Statistical Method
Model Specification
Adjusted R2/R2 [Min, Max]
OLS
CMG/RDRG
[0.44, 0.46]
OLS
ASG/DRG/Age
[0.14, 0.31]
OLS LOS log-transformed
Age/LAPS/COPS/Primary Condition/ Sex/Admission shift, day, month/ Length of stay/Physiological Scores/ In-hospital mortality
[0.14, 0.15]*
OLS LOS log-transformed
Sociodemographic/Source of Referral/ Payment classification/DRG/ Specialty of doctor at time of discharge
[0.30, 0.38]
OLS
Age/Sex/Lab Data/MDC
[0.23, 0.37]*
OLS
Age/Sex/Race/CDS/Charlson CI
[0.26, 0.27]
OLS
ICISS
OLS
Age/Sex/DRG/APR-DRG/NDX
[0.19, 0.34]
OLS
Sociodemographic/Diagnostic/Clinical
[0.10, 0.31]
OLS LOS log-transformed
Age/Sex/Race/APS
[0.08, 0.14]
Poisson
Age/Sex/DRG/Charlson index/ Transfers/Complications Age/Sex/DRGs/PSE/SBS
[0.13, 0.51] [0.15, 0.37]
Sociodemographic/CMG/CSI
[0.02, 0.23]
OLS LOS log-transformed OLS/Hierarchical regression
0.38*
(Continued )
Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
www.lww-medicalcare.com |
359
Medical Care
Lu et al
Volume 53, Number 4, April 2015
TABLE 2. Performance of Length of Stay Risk Adjustment Models (continued) References Faulkner et al41
Geissler et al32
Ghali et al16 Hartz et al49
Holman et al47
Iezzoni et al17
Kelleher24
Kieszak et al18
Kuczynski et al38 Mason et al25 Matsui et al45
McCrone and Phelan40
Melfi et al50
O’Reilly et al30
Or et al26 Paat-Ahi et al27
Study Population and Data Source Data from clinical records for all patients admitted to the acute psychiatric inpatient unit of the Grampians Region Psychiatric Services in Australia, from 1989 to 1990 (N = 236) Routine data (diagnosis, procedure, comorbidities) collected for hip replacement patients in 2008 from 10 EU countriesw (N = 70,112) Discharge data from Massachusetts Health Data Consortium for patients who had CABG surgery in 1992 (N = 6791) Health Care Financing Administration data for pneumonia and stroke patients admitted into Froedtert Memorial Lutheran Hospital in Wisconsin, US between 1987 and 1990, linked with national Uniform Clinical Data Set System (N = 1093) Linked administrative health data for medical, procedural, and psychiatric inpatients admitted to Western Australia hospitals from July 1, 29889 to December 31, 1996 (N = 1,118,989) Pneumonia patients in 105 US acute care hospitals in 1991, from MedisGroups Comparative Database (N = 18,016) Acute care inpatients admitted to Baltimore US Public Health Service Hospital between April 20, 1981 and October 2, 1981 (N = 661) Administrative billing data for Medicare beneficiaries who underwent carotid endarterectomy in Georgia (US) hospitals from January 1, 1993 to December 31, 1993 (N = 1980) Vascular patients hospitalized from June 1989 to December 1993 at the University of Michigan hospital (N = 1930) Data from appendectomy patients from 939 hospitals in 10 EU countriesw in 2008 (N = 106,929) Health records of patients > 30 y admitted to emergency department of Brigham and Women’s Hospital between October 1990 and May 1992 with chest pain (N = 1261) Data on psychiatric inpatients discharged from the Bethlem Royal and Maudsley teaching hospitals in south London (UK) between April 1990 and March 1992, from Patient Administration System (N = 5482) Administrative claims data for Medicare patients reimbursed for total knee replacement surgery between 1985 and 1989 in the US (N = 249,744) Routine data (diagnosis, procedure, comorbidity) collected for inguinal hernia inpatients in 10 EU countriesw between 2007 and 2009 (N = 173,968) Hospitalization data for child delivery in 10 EU countriesw in 2008 (N = 1,331,944) Cost data for cholecystectomy in 10 EU countriesw for 2008 (N = 161,956)
Statistical Method
Adjusted R2/R2 [Min, Max]
Model Specification
Classification and Age/Sociodemographic/DRG regression tree (CART) model
[0.03, 0.17]
Negative binomials
Age/Sex/DRGs/Charlson index
[0.26, 0.53]
OLS
Sociodemographic/Comorbidities/DRG
[0.16, 0.22]
OLS Age/Sex/Insurance/ICD-9-CM/ LOS truncated to the 90th Charlson index percentile for patients staying longer than 90th percentile
[0.01, 0.27]
OLS
MACSS/Charlson index
OLS LOS log-transformed
Age/Sex/DRG/Severity measures
OLS/Hierarchical regression LOS log-transformed
DRG/MDC/severity measures/difficulty in clinical management measures
Logistic Regression (LOS < 10 d = 0) (LOSZ10 d = 1) OR
Age/Sex/Neurological and medical risk factors/Charlson index
[0.04, 0.05]
OLS
Age/Sex/Insurance/Surgeon/Source of admission/DRG
[0.01, 0.22]
OLS
Age/Sex/DRGs/Charlson index/Patientlevel and appendectomy-specific variables Age/Sex/Charlson index/Clinical characteristics
[0.18, 0.40]
OLS/Hierarchical regression LOS log-transformed
[0.13, 0.33] MACSS [0.03, 0.17] Charlson [0.08, 0.14]* untrimmed data; [0.10, 0.17]* trimmed data [0.09, 0.34]*
[0.00, 0.51]
OLS
Age/Sex/DCs
OLS LOS log-transformed
Sociodemographic/PSL/RIS/D-CI/No. diagnoses
[0.17, 0.22]
Poisson regression
Age/Sex/DRGs/Charlson index/Other clinical controls
[0.16, 0.55]
Poisson regression
Age/Sex/DRGs/Charlson index/Other clinical controls Age/Sex/DRGs/Charlson index/Other clinical controls
[0.26, 0.41]
Poisson regression
0.03*
[0.25, 0.63] (Continued )
360 | www.lww-medicalcare.com
Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
Medical Care
Volume 53, Number 4, April 2015
Review of Risk Adjustment Models of Hospital LOS
TABLE 2. Performance of Length of Stay Risk Adjustment Models (continued) Study Population and Data Source
References Peltola37
Pertile et al35
Rochon et al48
Rutledge et al53
Sahadevan et al28
Shwartz et al21
Warnke and Ro¨ssler36 Wong et al29
Locally available hospitalization data for stroke inpatients above 1 y of age in 10 EU countriesw in 2008 [exc. England (2007/ 2008) and Poland (2009)] (N = 322,484) Discharge abstract data from the regional health information system for patients in general hospital psychiatric units in Veneto Region, in Italy for 2007 (N = 8789) Discharge data, demographic and comorbidity data derived from discharge abstracts for patients discharged from the spinal cord injury Service of the Brockton/ West Roxbury Veterans Affairs Medical Center, between January 1989 and December 1990, from aging with a longterm disability database (N = 330) Hospitalization data derived from discharge abstracts for trauma patients at University of North Carolina Hospitals from 1990 to 1996, from Health Care Utilization Project Database (N = 9483) Hospitalization data on Geriatric Medicine and General Medicine patients aged 65+ years who were admitted to an acute-care hospital in Singapore for 1999 (N = 232) Patients hospitalized for surgical repair of hip fracture in 80 US hospitals for 1991 calendar year, from MedicGroups Comparative Database (N = 5664) Data on inpatients referred to psychiatric hospitals of an area in Switzerland between 1997 and 2003, from regional psychiatric register (N = 30,616) Data from interviews with primary caregivers of Geriatric Medicine and General Medicine patients age 65+ years admitted to an acute general hospital in Singapore between 2003 and 2004 (N = 397)
Statistical Method
Model Specification
Adjusted R2/R2 [Min, Max]
Negative Binomial
Age/Sex/DRGs/Charlson index/Other clinical controls
[0.14, 0.63]
OLS/Hierarchical regression LOS log-transformed
Age/Sex/DRGs/Other patient, service, and area-level variables
[0.06, 0.20]
OLS
Age/Sex/CIRS/the Charlson index/Ct. of ICD-9-CM Diagnoses Codes
[0.02, 0.06]*
OLS
DRG/APR-DRG/ICISS
[0.31, 0.59]
OLS LOS log-transformed
Age/Sex/Functional status and changes/ [0.08, 0.28] Referrals/DRG untrimmed data [0.24, 0.31] trimmed data Age/Sex/DRG/APR-DRG/Severity [0.05, 0.17]* Score untrimmed data [0.04, 0.17]* trimmed data Sociodemographic/Clinical and [0.18, 0.09] admission-specific variables/DRG
OLS LOS log-transformed OLS LOS log-transformed OLS
Age/Sex/Functional and cognitive status/DRG/Overall illness severity score/No. referrals
[0.21, 0.31]
*R2, or weighted average R2, or cross-validated R2, is reported instead of adjusted R2. w Austria, France, Ireland, Poland, Spain, England, Estonia, Finland, Germany, Sweden. APR-DRG indicates All-Payer Revised DRG; APS, Acute Physiology Score; ASG, Admission Severity Group; CDS, Chronic Disease Score; Charlson CI, Charlson Comorbidity Indicator; CIRS, Cumulative Illness Rating Scale; CMG, Case Mix Groups; COPS, comorbidity point score; CSI, Computerized Severity Index; Ct., count; exc. is except; DCG, Diagnostic Cost Group; DCs, Diagnostic Categories; DRG, diagnosis-related groups; EU, European Union; HCC, Hierarchical Condition Categories; ICD-9-CM, indicated International Classification of Diseases, 9th revision, clinical modification; ICISS, International Classification of Diseases 9th Revision Based Illness Severity Score; LAPS, Laboratory Acute Physiology Score; LOS, Length of Stay; MACSS, Multipurpose Australian Comorbidity Scoring System; MDC, Major Diagnostic Categories; Min, minimum; Max, maximum; NDX, Nursing Diagnosis Terminology; OLS, Ordinary Least Squares; OR, Odds Ratio; PSE, Present State Examination; PSL, Patient Severity Level/RIS is Relative Intensity Score/D-CI is Deyo-adapted Charlson index; RDRG, Refined Diagnostic Groups; SBS, Social Behaviour Scale; UK, United Kingdom; US, United States.
generalized linear regression model: 4 (11%) adopted a Poisson regression model, 2 (5%) a negative binomial regression model, and 1 (3%) a logistic regression model. Four studies (11%) adopted a hierarchical regression model, and 1 (3%) adopted a classification and regression tree (CART) model, which is a data mining classification model. Although the highly skewed distribution of the LOS variable is well acknowledged, only 12 (32%) studies used a log transformation of LOS.
Model Specification It should be noted that very few studies used just 1 risk adjustor. Most studies attempted to include as many adjustCopyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
ors as possible; many tested various model specifications that included different groupers and/or disease severity/morbidity indexes. Age and sex were commonly included. The most commonly adopted model specification in these studies included sociodemographic controls, groupers, and disease severity/comorbidity indexes.
Model Performance (Adjusted R2/R2) The conventional measure for risk adjustment model performance is adjusted R2. Six studies (16%) did not report adjusted R2, in which case unadjusted R2 is presented. The range of different models tested in each study measured by www.lww-medicalcare.com |
361
Medical Care
Lu et al
minimum and maximum adjusted R2/R2 is reported in the last column of Table 2. As shown in Table 2, there were large disparities in performance among studies using different models. For those that used OLS, the highest adjusted R2/R2 ranged from 0.31 to 0.59.53 Among studies that used generalized linear regression models, the highest range was 0.25–0.63,27 and for hierarchical regression models it was 0.09–0.34.24 Finally, data mining classification models ranged highest from 0.03 to 0.17.41 In general, models that used only basic demographic variables (eg, age and sex) as risk adjustors performed most poorly. The performance of LOS risk adjustment models improved significantly once one or more disease groupers and/or disease severity/comorbidity indexes were added. The best model performance for most studies fell in the range of 0.03–0.60, approximately. The results for model performance varied between studies among different populations, as shown in Table 2. For example, the highest model performance in general inpatient populations ranged from 0.44 to 0.46.42 All studies for general inpatient populations used OLS as at least 1 statistical method. For nongeneral populations (eg, psychiatric, teaching, or private hospital patients), the highest model performance ranged from 0.25 to 0.63.27
Volume 53, Number 4, April 2015
The main findings from our systematic review are 3fold. First, among the vast risk adjustment literature, few models were developed specifically for LOS. Most studies sought to explain hospital spending and used these models in an attempt to explain LOS. This is likely due to the lack of information available for factors other than cost that affect LOS, such as characteristics of the social/family environment. Future work is needed to develop risk adjustment methodologies and models that best suit LOS. Second, our review suggests that OLS regression remains a popular choice for methods of LOS prediction, with reasonable performance. In general inpatient populations, Halpine and Ashworth42 achieved the highest performance using OLS with a range of 0.44–0.46, whereas Rutledge et al53 achieved a range of 0.31–0.59 in the nongeneral population. The use of OLS by nearly 80% of included studies, combined with its relatively high performances of 0.46 and 0.59 in general and nongeneral populations, suggest that this remains one of the best estimation methods for predicting LOS. In addition, among the various disease groupers and disease severity indexes, those most commonly used (ie, DRGs and CI) also performed well. Third, the studies reviewed showed large variation in performance across different risk adjustment models, with the lowest performing model explaining
Systematic Review of Risk Adjustment Models of Hospital Length of Stay (LOS) Mingshan Lu, PhD,* Tolulope Sajobi, PhD,w Kelsey Lucyk, MSc,w Diane Lorenzetti, PhD,w and Hude Quan, MD, PhDw
Background: Policy decisions in health care, such as hospital performance evaluation and performance-based budgeting, require an accurate prediction of hospital length of stay (LOS). This paper provides a systematic review of risk adjustment models for hospital LOS, and focuses primarily on studies that use administrative data. Methods: MEDLINE, EMBASE, Cochrane, PubMed, and EconLit were searched for studies that tested the performance of risk adjustment models in predicting hospital LOS. We included studies that tested models developed for the general inpatient population, and excluded those that analyzed risk factors only correlated with LOS, impact analyses, or those that used disease-specific scales and indexes to predict LOS. Results: Our search yielded 3973 abstracts, of which 37 were included. These studies used various disease groupers and severity/ morbidity indexes to predict LOS. Few models were developed specifically for explaining hospital LOS; most focused primarily on explaining resource spending and the costs associated with hospital LOS, and applied these models to hospital LOS. We found a large variation in predictive power across different LOS predictive models. The best model performance for most studies fell in the range of 0.30–0.60, approximately. Conclusions: The current risk adjustment methodologies for predicting LOS are still limited in terms of models, predictors, and predictive power. One possible approach to improving the performance of LOS risk adjustment models is to include more diseasespecific variables, such as disease-specific or condition-specific measures, and functional measures. For this approach, however, more comprehensive and standardized data are urgently needed. In addition, statistical methods and evaluation tools more appropriate to LOS should be tested and adopted. Key Words: length of stay, risk adjustment, systematic review (Med Care 2015;53: 355–365)
C
ost containment and improved efficiency are primary issues in health care policy as countries face budgetary constraints in health care globally. The existing literature suggests that hospital length of stay (LOS) is a key indicator of inpatient resource use and hospital efficiency.1–6 As a result, research and policy efforts have focused on reducing unnecessary hospital LOS.7–10 The literature suggests that LOS exhibits large variation across hospitals in different samples. For example, hospital LOS for acute myocardial infarction patients may vary from 4.17 to 41.0 days and from 2.7 to 16.0 days for C-section patients.11–14 Important policy decisions, such as hospital budgeting and performance evaluation, therefore must account for differences in characteristics of the underlying population. This calls for the development of LOS risk adjustment models. The risk adjustment literature has grown significantly during the past few decades, which has been driven mostly by countries adopting methods for prospective hospital payment.15 However, few studies report risk adjustors or risk adjustment models that have been specifically developed for LOS. Instead, much of the existing literature focuses on predicting hospital spending and applies these methods to predict LOS.16–18 It remains unclear which risk adjustors and risk adjustment models best control for patient severity when analyzing hospital LOS, and whether the predictive power of existing models are satisfactory. To fill this gap, this paper provides a systematic review of the performance of risk adjustment models for hospital LOS. Our review focuses on methods that can be applied to general inpatient populations. We summarize existing LOS risk adjustors, study populations and data sources, statistical methods, and model specifications for each study. We present and compare the performance of LOS risk adjustment models, and provide recommendations for future research based on our discussion of results.
METHODS From the *Department of Economics and Community Health Sciences; and wDepartment of Community Health Sciences and Centre for Health and Policy Studies, University of Calgary, Calgary, AB, Canada. T.S.’s research was supported by the MSI Foundation Grant; H.Q.’s salary was supported by Alberta Innovates Health Solutions; and K.L. was supported by an Achievers in Medical Sciences scholarship at the time of writing. The authors declare no conflict of interest. Reprints: Mingshan Lu, PhD, Department of Economics, University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4. Email: [email protected]. Copyright r 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0025-7079/15/5304-0355
Medical Care
Volume 53, Number 4, April 2015
Data Sources and Searches We searched a total of 5 peer-reviewed databases: MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, PubMed, and EconLit. We implemented a search strategy that combined text words and MeSH subject headings (using the Boolean and positional Operators AND, OR, and “adj”) that represented the concepts relevant to our research question: Concept 1: length of stay or LOS or hospital days or days adj3 care; Concept 2: risk assessment or regression analys* or model or models or logistic regression or www.lww-medicalcare.com |
355
Medical Care
Lu et al
risk adjustment* or theoretical model or computer simulation or predictive or predicted. We compiled all references into an Excel dataset that we used to select studies for inclusion or exclusion.
Criteria for Study Selection We searched studies published between January 1993 and June 2013. We limited our search to studies that contained English language abstracts and used data from nonexperimental hospital settings to test the statistical performance of risk adjustment models in predicting hospital LOS. Articles were included for review if they: (1) measured LOS as an outcome/ dependent variable; (2) were original research; or, (3) were a full-text empirical study that used data from nonexperimental hospital settings to test risk adjustment model performance on predicting LOS. We excluded studies if they: (1) analyzed risk factors or determinants correlated with LOS without testing the statistical performance of risk adjustment models; (2) used disease-specific scales and indexes to predict LOS; (3) performed impact analysis with LOS as a performance measure; (4) were randomized control trials or theoretical papers; or, (5) focused on LOS for special inpatient populations. We did not impose any age limitation on the target population. We excluded studies that used disease-specific risk adjustment models, for the following reasons. First, there is an accompanying literature for many diseases whereby disease-specific factors are tested to determine which should be included in risk adjustment models. Including these studies would have made the scope and volume of our review unmanageable. Second, it is difficult to apply generalizations from one disease to another when predicting hospital LOS, which detracts from our interest in models for general inpatient populations. Finally, while disease-specific models may address questions related to disease or clinical management, they are not suitable for our interests in policy issues that relate to hospital performance evaluation or payment design. Reference lists of included studies were scanned to identify additional relevant papers, which led to the inclusion of an additional 7 articles. Following the title and abstract review, the authors discussed additional and potentially relevant risk adjustment models that might have been missed. As such, we conducted a keyword search (ie, ACG, Truven, Symmetry, Cave) of all title and abstracts, which identified an additional 19 full-text articles. A complete search strategy is available from the authors.
Volume 53, Number 4, April 2015
One of the major challenges of using a risk adjustment model is deciding which risk adjustors to use. Therefore, in synthesizing our results, we focus on 2 types of risk adjustors for patient characteristics: those that adjust for disease type using diagnostic information (ie, disease groupers) or patient case mix (ie, disease severity/morbidity indexes). We synthesized data in 3 ways. First, we summarized risk adjustors used in studies by name, type, and test population. Next, we classified studies by type of patient group: general inpatient population or disease-specific inpatients. Finally, within each patient group we presented the study population and data source including sample size, statistical method, model specification, and model performance.
RESULTS As indicated in the PRISMA flow diagram of studies in Figure 1, 7016 records were identified through database searching and 7 through reference-list searching for a total of 3973 unique citations, with duplicates removed. Title and abstract screening was conducted by 3 investigators, and 225 articles were selected for full-text review. After full-text review, 37 studies were included in this review.
LOS Risk Adjustors Table 1 presents a summary of the risk adjustors used, classified into disease groupers, disease severity/morbidity indexes, and their test populations.
Disease Groupers
The results of our review are reported in accordance with the PRISMA Statement (http://www.prisma-statement.org). Abstracts were divided into 3 groups, which were each independently reviewed by 2 of the 3 investigators (M.L., T.S., H.Q.) to decide whether the full-text article should be reviewed. Full-text articles were also divided for review in this way. Although we did conduct pretesting of inclusion and exclusion criteria, we did not calculate interrater reliability. Disagreements were resolved through discussion and consensus. Reviewers were not blinded to study author, institution, or journal.
Disease groupers or “diagnosis-related groups” (DRGs), refer to the various methods of classifying inpatients by main diagnosis or procedure.51 Within a disease group, institutions may also classify patients by the severity of their condition using a severity index. For example, “DRG” refers to an American grouper that uses the ICD-9, whereas the All Patient Refined Diagnosis-related Groups (APR-DRG) is another form of DRG that uses ICD-10-CM and includes severity of diagnosis. A comprehensive list of disease groupers can be found online.52 The most commonly used and widely tested grouper for LOS risk adjustment is DRG, an inpatient episode grouper. In total, we identified 22 studies that used disease groupers. DRGs were tested for their predictive power on LOS for general inpatient populations,19,20,23 and also acute care,24 appendectomy,25 coronary artery bypass graft (CABG) surgery,16 child delivery,26 cholecystectomy,27 elderly,28,29 hip fracture,21 inguinal hernia,30 knee and hip replacement,31,32 pneumonia,17 psychiatric,33–36 stroke,37 trauma,53 and vascular patients.38 Other commonly used disease groupers included DRG variations, such as APR-DRGs and Refined Diagnosisrelated Groups. APR-DRGs were used by 4 studies for general inpatient populations,19,20 hip fracture21 and pneumonia patients.17 Refined Diagnosis-related Groups were tested on general inpatient populations,19,42 pneumonia17 and hip fracture patients.21 Major Diagnostic Categories (MDC) were also used by 4 studies and tested on general inpatient populations,39 acute care24 and psychiatric patients.40,41 One study tested the predictive power of Case Mix Groups for LOS on psychiatric patients.22
356 | www.lww-medicalcare.com
Copyright
Synthesis of Results and Quality Assessment
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
Volume 53, Number 4, April 2015
Identification
Medical Care
7016 records identified through database searching
Review of Risk Adjustment Models of Hospital LOS
7 records identified through other sources
Included
Eligibility
Screening
3973 records remained after removal of duplicates
3050 duplicate records excluded
3973 articles screened using title and abstract
538 records excluded for not reporting LOS and risk adjustment
3435 articles assessed for eligibility based on title and abstract
3206 records excluded because eligibility criteria not met
225 full-text articles reviewed
187 records excluded because eligibility criteria not met
37 articles included in the final review
8 articles focus on general population
29 articles focus on nongeneral population
FIGURE 1. PRISMA flow diagram of study selection. LOS indicates length of stay.
Disease Severity/Comorbidity Indexes Among the large varieties of disease severity/comorbidity indexes, Charlson Index (CI) was most commonly used. Among the 37 studies reviewed, CI was used by 14 to explain LOS for general inpatient populations,43,46,47 medical, procedural, and psychiatric inpatients,47 as well as pneumonia,17,33 appendectomy,25 chest pain,45 inguinal hernia,30 child delivery,26 cholecystectomy,27 carotid endarterectomy,18 spinal cord injury,48 stroke,49 and total knee replacement inpatients.50 Patient Management Categories Relative Intensity Score was used in 3 studies and tested on inpatients with hip fracture,21 total knee replacement,50 and pneumonia.17 Acuity Index Method was used by 2 studies for pneumonia17 and hip fracture inpatients.21 Admission Severity Groups were used by 2 studies and tested on general44 and pneumonia inpatients.17 One study used MedisGroups (Atlas MQ) on hip fracture inpatients.21 Two studies used Disease Staging Mortality Probability to explain LOS for inpatients with pneumonia17 and hip fracture.21 Another 2 Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
used ICD-9-based Illness Severity Score to explain LOS for general18 and trauma inpatients.20 Physiology Score and Body Systems Count were used in 2 studies for pneumonia17 and hip fracture inpatients.21 In addition, studies included in our review also used the following indexes: Acute Physiology Score (general population),43 Chronic Disease Score (general population),46 Comorbidity Index (hip fracture),21 Comorbidity Point Score (general population),23 Computerized Severity Index (psychiatric patients),22 Count of ICD-9 Diagnoses (spinal cord injury),48 Cumulative Illness Rating Scale (spinal cord injury),48 Difficulty of Clinical Management (acute care inpatients),24 Functional-linked Variables (elderly patients),28 Laboratory Acute Physiology Score (general population),23 Modified Severity of Illness Index (elderly inpatients),29 Multipurpose Australian Comorbidity Scoring System (medical, procedural, and psychiatric inpatients),47 Nursing Diagnosis Terminology (general population),19 number of diagnoses (total knee replacement),50 patient-reported www.lww-medicalcare.com |
357
Medical Care
Lu et al
TABLE 1. List of Risk Adjustors Used in Length of Stay Studies Risk Adjustor Name
Test Population
Groupers APR-DRG
General population Hip fracture Pneumonia patients CMG Psychiatric patients DRG General population Acute care patients Appendectomy CABG surgery Child delivery Cholecystectomy Elderly Hip fracture Inguinal hernia Knee replacement patients Pneumonia patients Psychiatric patients Stroke Trauma Vascular patients MDC General population Acute care patients Psychiatric patients RDRG General population Hip fracture Pneumonia patients Severity/comorbidity measures AIM Pneumonia patients Hip fracture APS General population ASG General patients Pneumonia patients Body systems count Pneumonia patients Hip fracture Charlson Index Appendectomy Chest pain Child delivery Cholecystectomy General population Inguinal hernia Medical, procedural, and psychiatric inpatients Pneumonia patients Spinal cord injury Stroke Total knee replacement CDS General population Comorbidity Index Hip fracture COPS General population CSI Psychiatric patients Count of ICD-9 Spinal cord injury diagnoses Cumulative illness rating Spinal cord injury scale Difficulty of clinical Acute care patients management Disease staging mortality Pneumonia patients probability Hip fracture Function-linked variables Elderly patients ICISS Trauma General population LAPS General population Hip fracture
References
17 21 43 44 17 17 21 25 45 26 27 43,46 30 47 17,49 48 49 50 46 21 23 22 48 48 24 17 21 28 20 53 23 21 (Continued )
358 | www.lww-medicalcare.com
Volume 53, Number 4, April 2015
TABLE 1. List of Risk Adjustors Used in Length of Stay Studies (continued) Risk Adjustor Name
19,20 21 17 22 19,20,23 24 25 16 26 27 28,29 21 30 31,32 17 33–36 37 18 38 39 24 40,41 19,42 21 17
Test Population
MedisGroups (Atlas MQ) MSII MACSS
Elderly patients Medical, procedural, and psychiatric inpatients NDX General population No. diagnoses Total knee replacement PMC RIS Hip fracture Total knee replacement Pneumonia patients Patient-reported severity CABG surgery measures PSL Total knee replacement Physiology Score Pneumonia patients Hip fracture
References
29 47 19 50 21 50 17 16 50 17 21
ACG indicates adjusted clinical groups; AIM, Acuity Index Method; APR-DRG, all patient refined diagnosis-related groups; APS, acute physiology score; ASG, admission severity groups; CABG, coronary artery bypass graft; CDPS, chronic illness and disability payment system; CDS, chronic disease score; CMG, case mix groups; COPS, comorbidity point score; CSI, computerized severity index; DRG, diagnosisrelated groups; ICD-9, is International Classification of Diseases, 9th revision; ICISS, ICD-9-based Illness Severity Score; LAPS, Laboratory Acute Physiology Score; MACSS, multipurpose Australian comorbidity scoring system; MDC, major diagnostic categories; MSII, Modified Severity of Illness Index; NDX, nursing diagnosis terminology; PMC, patient management categories; PSL, patient severity level; RDRG, is redefined diagnosis-related groups; RIS, relative intensity score.
severity measures (CABG surgery),16 and Patient Severity Level (total knee replacement).50
LOS Risk Adjustment Models Table 2 presents the study population, data source, statistical method, model specification, and the performance of LOS risk adjustment models used for each of the 37 studies included in our review. They are separated into 2 groups: the first includes 8 studies (22%) that analyze LOS for general inpatient populations, and the second includes 29 studies (78%) that analyze LOS for inpatient populations with specific diseases.
Study Population and Data Source The primary data sources used by studies were hospitalization data (eg, admission and/or discharge), medical records, and clinical data. In some studies, primary data were supplemented by pharmacy data,23,44,46 laboratory data,23,39 demographic data,39 and patient-reported data.29 Study populations included patients from 1 or more hospital or health insurance plan, from multiple countries, over a period of 1–6 years.
Statistical Method Four types of statistical methods were adopted by the studies we reviewed: ordinary least squares (OLS), generalized linear regression models, hierarchical regression models, and data mining models. Among studies included in our review, 29 (78%) used OLS to develop a hospital LOS risk adjustment model. Seven (18%) adopted some form of a Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
Medical Care
Volume 53, Number 4, April 2015
Review of Risk Adjustment Models of Hospital LOS
TABLE 2. Performance of Length of Stay Risk Adjustment Models Study Population and Data Source
References General population Halpine and Ashworth42 Hicks and Kammerling44
Liu et al23
Liu et al54
Mozes et al39
Parker et al46
Rutledge and Osler20 Welton and Halloran19
Abstracts reported to the Hospital Medical Records Institute, New Brunswick, and Alberta coded in ICD-9-CM, from January to March 1989 (N = 282,459) Information abstracted from clinical records of general medical patients discharged from an English (UK) teaching hospital in a 15-mo period, using MedisGroups system (N = 2279) Automated hospital, laboratory, administrative, and pharmacy data from hospitalizations at 17 Northern California Kaiser-Permanente hospitals between January 1, 2002 and June 30, 2005 (N = 155,474) Hospital data from Hospital Morbidity Database System (HMDS) from a large teaching hospital in Western Australia, where numbers of episodes were at least 1% of all discharges, from July 1995 to June 1996 (N = 4589) Laboratory, hospital admission, ansd demographic data from the Combined Patient Experience (COPE) database on patients from the University of California San Francisco Medical Center and Stanford University Medical Center discharged in 1981, 1982, and 1986 (N = 73,117) Automated hospital and pharmacy data from Kaiser-Permanente of acute hospitalizations in Southern California, from April 1993 to February 1995 (N = 6721) Every patient admitted to all acute care hospitals in North Carolina during 1996 whose records contained complete data (N = 821,455) Hospitalization data on patients admitted to a university hospital in a Midwestern City, US from 1986 to 1989 (N = 123,241)
Nongeneral population Brock and Brown33 Admissions to an adult inpatient psychiatric unit of Wilford Hall Medical Center in the US, from September 1, 1987 to October 1, 1989 (N = 1040) 43 Patient-reported symptoms for patients Clark et al admitted to the general medicine service of a teaching hospital affiliated with the Indiana University School of Medicine (US) from July 1, 1996 to June 30, 1997 (N = 2126) Cost data from 10 EU countriesw from 1365 Cots et al31 hospitals in 2008 (N = 190,000) Admissions data from all patients 18–65 y Creed et al34 admitted to Manchester Royal Infirmary Psychiatric Day Hospital during a 9-mo period (N = 115) 22 Discharge abstract from adult psychiatric Durbin et al patients during 1994–1995 fiscal year in 3 acute care inpatient settings in Greater Toronto (teaching hospital; academic research, treatment, and training center; provincial psychiatry facility) (N = 355)
Statistical Method
Model Specification
Adjusted R2/R2 [Min, Max]
OLS
CMG/RDRG
[0.44, 0.46]
OLS
ASG/DRG/Age
[0.14, 0.31]
OLS LOS log-transformed
Age/LAPS/COPS/Primary Condition/ Sex/Admission shift, day, month/ Length of stay/Physiological Scores/ In-hospital mortality
[0.14, 0.15]*
OLS LOS log-transformed
Sociodemographic/Source of Referral/ Payment classification/DRG/ Specialty of doctor at time of discharge
[0.30, 0.38]
OLS
Age/Sex/Lab Data/MDC
[0.23, 0.37]*
OLS
Age/Sex/Race/CDS/Charlson CI
[0.26, 0.27]
OLS
ICISS
OLS
Age/Sex/DRG/APR-DRG/NDX
[0.19, 0.34]
OLS
Sociodemographic/Diagnostic/Clinical
[0.10, 0.31]
OLS LOS log-transformed
Age/Sex/Race/APS
[0.08, 0.14]
Poisson
Age/Sex/DRG/Charlson index/ Transfers/Complications Age/Sex/DRGs/PSE/SBS
[0.13, 0.51] [0.15, 0.37]
Sociodemographic/CMG/CSI
[0.02, 0.23]
OLS LOS log-transformed OLS/Hierarchical regression
0.38*
(Continued )
Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
www.lww-medicalcare.com |
359
Medical Care
Lu et al
Volume 53, Number 4, April 2015
TABLE 2. Performance of Length of Stay Risk Adjustment Models (continued) References Faulkner et al41
Geissler et al32
Ghali et al16 Hartz et al49
Holman et al47
Iezzoni et al17
Kelleher24
Kieszak et al18
Kuczynski et al38 Mason et al25 Matsui et al45
McCrone and Phelan40
Melfi et al50
O’Reilly et al30
Or et al26 Paat-Ahi et al27
Study Population and Data Source Data from clinical records for all patients admitted to the acute psychiatric inpatient unit of the Grampians Region Psychiatric Services in Australia, from 1989 to 1990 (N = 236) Routine data (diagnosis, procedure, comorbidities) collected for hip replacement patients in 2008 from 10 EU countriesw (N = 70,112) Discharge data from Massachusetts Health Data Consortium for patients who had CABG surgery in 1992 (N = 6791) Health Care Financing Administration data for pneumonia and stroke patients admitted into Froedtert Memorial Lutheran Hospital in Wisconsin, US between 1987 and 1990, linked with national Uniform Clinical Data Set System (N = 1093) Linked administrative health data for medical, procedural, and psychiatric inpatients admitted to Western Australia hospitals from July 1, 29889 to December 31, 1996 (N = 1,118,989) Pneumonia patients in 105 US acute care hospitals in 1991, from MedisGroups Comparative Database (N = 18,016) Acute care inpatients admitted to Baltimore US Public Health Service Hospital between April 20, 1981 and October 2, 1981 (N = 661) Administrative billing data for Medicare beneficiaries who underwent carotid endarterectomy in Georgia (US) hospitals from January 1, 1993 to December 31, 1993 (N = 1980) Vascular patients hospitalized from June 1989 to December 1993 at the University of Michigan hospital (N = 1930) Data from appendectomy patients from 939 hospitals in 10 EU countriesw in 2008 (N = 106,929) Health records of patients > 30 y admitted to emergency department of Brigham and Women’s Hospital between October 1990 and May 1992 with chest pain (N = 1261) Data on psychiatric inpatients discharged from the Bethlem Royal and Maudsley teaching hospitals in south London (UK) between April 1990 and March 1992, from Patient Administration System (N = 5482) Administrative claims data for Medicare patients reimbursed for total knee replacement surgery between 1985 and 1989 in the US (N = 249,744) Routine data (diagnosis, procedure, comorbidity) collected for inguinal hernia inpatients in 10 EU countriesw between 2007 and 2009 (N = 173,968) Hospitalization data for child delivery in 10 EU countriesw in 2008 (N = 1,331,944) Cost data for cholecystectomy in 10 EU countriesw for 2008 (N = 161,956)
Statistical Method
Adjusted R2/R2 [Min, Max]
Model Specification
Classification and Age/Sociodemographic/DRG regression tree (CART) model
[0.03, 0.17]
Negative binomials
Age/Sex/DRGs/Charlson index
[0.26, 0.53]
OLS
Sociodemographic/Comorbidities/DRG
[0.16, 0.22]
OLS Age/Sex/Insurance/ICD-9-CM/ LOS truncated to the 90th Charlson index percentile for patients staying longer than 90th percentile
[0.01, 0.27]
OLS
MACSS/Charlson index
OLS LOS log-transformed
Age/Sex/DRG/Severity measures
OLS/Hierarchical regression LOS log-transformed
DRG/MDC/severity measures/difficulty in clinical management measures
Logistic Regression (LOS < 10 d = 0) (LOSZ10 d = 1) OR
Age/Sex/Neurological and medical risk factors/Charlson index
[0.04, 0.05]
OLS
Age/Sex/Insurance/Surgeon/Source of admission/DRG
[0.01, 0.22]
OLS
Age/Sex/DRGs/Charlson index/Patientlevel and appendectomy-specific variables Age/Sex/Charlson index/Clinical characteristics
[0.18, 0.40]
OLS/Hierarchical regression LOS log-transformed
[0.13, 0.33] MACSS [0.03, 0.17] Charlson [0.08, 0.14]* untrimmed data; [0.10, 0.17]* trimmed data [0.09, 0.34]*
[0.00, 0.51]
OLS
Age/Sex/DCs
OLS LOS log-transformed
Sociodemographic/PSL/RIS/D-CI/No. diagnoses
[0.17, 0.22]
Poisson regression
Age/Sex/DRGs/Charlson index/Other clinical controls
[0.16, 0.55]
Poisson regression
Age/Sex/DRGs/Charlson index/Other clinical controls Age/Sex/DRGs/Charlson index/Other clinical controls
[0.26, 0.41]
Poisson regression
0.03*
[0.25, 0.63] (Continued )
360 | www.lww-medicalcare.com
Copyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
Medical Care
Volume 53, Number 4, April 2015
Review of Risk Adjustment Models of Hospital LOS
TABLE 2. Performance of Length of Stay Risk Adjustment Models (continued) Study Population and Data Source
References Peltola37
Pertile et al35
Rochon et al48
Rutledge et al53
Sahadevan et al28
Shwartz et al21
Warnke and Ro¨ssler36 Wong et al29
Locally available hospitalization data for stroke inpatients above 1 y of age in 10 EU countriesw in 2008 [exc. England (2007/ 2008) and Poland (2009)] (N = 322,484) Discharge abstract data from the regional health information system for patients in general hospital psychiatric units in Veneto Region, in Italy for 2007 (N = 8789) Discharge data, demographic and comorbidity data derived from discharge abstracts for patients discharged from the spinal cord injury Service of the Brockton/ West Roxbury Veterans Affairs Medical Center, between January 1989 and December 1990, from aging with a longterm disability database (N = 330) Hospitalization data derived from discharge abstracts for trauma patients at University of North Carolina Hospitals from 1990 to 1996, from Health Care Utilization Project Database (N = 9483) Hospitalization data on Geriatric Medicine and General Medicine patients aged 65+ years who were admitted to an acute-care hospital in Singapore for 1999 (N = 232) Patients hospitalized for surgical repair of hip fracture in 80 US hospitals for 1991 calendar year, from MedicGroups Comparative Database (N = 5664) Data on inpatients referred to psychiatric hospitals of an area in Switzerland between 1997 and 2003, from regional psychiatric register (N = 30,616) Data from interviews with primary caregivers of Geriatric Medicine and General Medicine patients age 65+ years admitted to an acute general hospital in Singapore between 2003 and 2004 (N = 397)
Statistical Method
Model Specification
Adjusted R2/R2 [Min, Max]
Negative Binomial
Age/Sex/DRGs/Charlson index/Other clinical controls
[0.14, 0.63]
OLS/Hierarchical regression LOS log-transformed
Age/Sex/DRGs/Other patient, service, and area-level variables
[0.06, 0.20]
OLS
Age/Sex/CIRS/the Charlson index/Ct. of ICD-9-CM Diagnoses Codes
[0.02, 0.06]*
OLS
DRG/APR-DRG/ICISS
[0.31, 0.59]
OLS LOS log-transformed
Age/Sex/Functional status and changes/ [0.08, 0.28] Referrals/DRG untrimmed data [0.24, 0.31] trimmed data Age/Sex/DRG/APR-DRG/Severity [0.05, 0.17]* Score untrimmed data [0.04, 0.17]* trimmed data Sociodemographic/Clinical and [0.18, 0.09] admission-specific variables/DRG
OLS LOS log-transformed OLS LOS log-transformed OLS
Age/Sex/Functional and cognitive status/DRG/Overall illness severity score/No. referrals
[0.21, 0.31]
*R2, or weighted average R2, or cross-validated R2, is reported instead of adjusted R2. w Austria, France, Ireland, Poland, Spain, England, Estonia, Finland, Germany, Sweden. APR-DRG indicates All-Payer Revised DRG; APS, Acute Physiology Score; ASG, Admission Severity Group; CDS, Chronic Disease Score; Charlson CI, Charlson Comorbidity Indicator; CIRS, Cumulative Illness Rating Scale; CMG, Case Mix Groups; COPS, comorbidity point score; CSI, Computerized Severity Index; Ct., count; exc. is except; DCG, Diagnostic Cost Group; DCs, Diagnostic Categories; DRG, diagnosis-related groups; EU, European Union; HCC, Hierarchical Condition Categories; ICD-9-CM, indicated International Classification of Diseases, 9th revision, clinical modification; ICISS, International Classification of Diseases 9th Revision Based Illness Severity Score; LAPS, Laboratory Acute Physiology Score; LOS, Length of Stay; MACSS, Multipurpose Australian Comorbidity Scoring System; MDC, Major Diagnostic Categories; Min, minimum; Max, maximum; NDX, Nursing Diagnosis Terminology; OLS, Ordinary Least Squares; OR, Odds Ratio; PSE, Present State Examination; PSL, Patient Severity Level/RIS is Relative Intensity Score/D-CI is Deyo-adapted Charlson index; RDRG, Refined Diagnostic Groups; SBS, Social Behaviour Scale; UK, United Kingdom; US, United States.
generalized linear regression model: 4 (11%) adopted a Poisson regression model, 2 (5%) a negative binomial regression model, and 1 (3%) a logistic regression model. Four studies (11%) adopted a hierarchical regression model, and 1 (3%) adopted a classification and regression tree (CART) model, which is a data mining classification model. Although the highly skewed distribution of the LOS variable is well acknowledged, only 12 (32%) studies used a log transformation of LOS.
Model Specification It should be noted that very few studies used just 1 risk adjustor. Most studies attempted to include as many adjustCopyright
r
2015 Wolters Kluwer Health, Inc. All rights reserved.
ors as possible; many tested various model specifications that included different groupers and/or disease severity/morbidity indexes. Age and sex were commonly included. The most commonly adopted model specification in these studies included sociodemographic controls, groupers, and disease severity/comorbidity indexes.
Model Performance (Adjusted R2/R2) The conventional measure for risk adjustment model performance is adjusted R2. Six studies (16%) did not report adjusted R2, in which case unadjusted R2 is presented. The range of different models tested in each study measured by www.lww-medicalcare.com |
361
Medical Care
Lu et al
minimum and maximum adjusted R2/R2 is reported in the last column of Table 2. As shown in Table 2, there were large disparities in performance among studies using different models. For those that used OLS, the highest adjusted R2/R2 ranged from 0.31 to 0.59.53 Among studies that used generalized linear regression models, the highest range was 0.25–0.63,27 and for hierarchical regression models it was 0.09–0.34.24 Finally, data mining classification models ranged highest from 0.03 to 0.17.41 In general, models that used only basic demographic variables (eg, age and sex) as risk adjustors performed most poorly. The performance of LOS risk adjustment models improved significantly once one or more disease groupers and/or disease severity/comorbidity indexes were added. The best model performance for most studies fell in the range of 0.03–0.60, approximately. The results for model performance varied between studies among different populations, as shown in Table 2. For example, the highest model performance in general inpatient populations ranged from 0.44 to 0.46.42 All studies for general inpatient populations used OLS as at least 1 statistical method. For nongeneral populations (eg, psychiatric, teaching, or private hospital patients), the highest model performance ranged from 0.25 to 0.63.27
Volume 53, Number 4, April 2015
The main findings from our systematic review are 3fold. First, among the vast risk adjustment literature, few models were developed specifically for LOS. Most studies sought to explain hospital spending and used these models in an attempt to explain LOS. This is likely due to the lack of information available for factors other than cost that affect LOS, such as characteristics of the social/family environment. Future work is needed to develop risk adjustment methodologies and models that best suit LOS. Second, our review suggests that OLS regression remains a popular choice for methods of LOS prediction, with reasonable performance. In general inpatient populations, Halpine and Ashworth42 achieved the highest performance using OLS with a range of 0.44–0.46, whereas Rutledge et al53 achieved a range of 0.31–0.59 in the nongeneral population. The use of OLS by nearly 80% of included studies, combined with its relatively high performances of 0.46 and 0.59 in general and nongeneral populations, suggest that this remains one of the best estimation methods for predicting LOS. In addition, among the various disease groupers and disease severity indexes, those most commonly used (ie, DRGs and CI) also performed well. Third, the studies reviewed showed large variation in performance across different risk adjustment models, with the lowest performing model explaining
