Psychological Medicine, 1991, 21, 237-243 Printed in Great Britain
BRIEF COMMUNICATION
The Endstage Renal Disease Severity Index (ESRD-SI) J. CRAVEN, 1 C. LITTLEFIELD, G. RODIN AND M. MURRAY* From the Departments of Psychiatry and Nephrology, Toronto Hospital, University of Toronto, Canada
Measures of disease severity are required as experimental, control and outcome variables in studies of the psychosocial aspects of physical illness. The Endstage Renal Disease Severity Index (ESRD-SI) was developed for these purposes during a prospective study of patient adjustment to dialysis and endstage renal disease (ESRD). It was found to have adequate inter-rater reliability, test-retest reliability, and construct and predictive validity in two subsamples (total N = 233) of dialysis patients. The ESRD-SI is suitable as a sensitive indicator of disease severity in these patients.
SYNOPSIS
INTRODUCTION The term 'disease severity' has been used to refer to the degree of physiological or morphological disturbance, the impact of the disease process on the activities of the afflicted person (functional severity) or the impact of illness on family or society (burden of illness) (Stein et al. 1987). Measurement of disease severity is necessary to describe and compare patient samples (Charlson et al. 1986), to assess and predict quality of care (Green et al. 1990), and to investigate factors which determine clinical course (Cassileth et al. 1985). However, although there is increasing agreement that the accurate measurement of disease severity is a pre-requisite for such endeavours, references to disease severity are still, as Stein et al. (1987) have noted, inconsistent and vague in many published reports. We have argued elsewhere (Rodin et al. 1991) that this deficiency is a particular problem in studies designed to distinguish psychosocial from organic determinants of physical illness. Sensitive measures of physical disease severity are vital for such distinctions to be made. Some measures of physiological and morphological severity are available for specific • Also, the Toronto Hospital Nephrology Group. 1 Address for correspondence: Dr J. Craven, Psychosomatic Medicine Program, Department of Psychiatry, The Toronto Hospital: General Division 8EN-2I2, 200 Elizabeth Street, Toronto, Ontario, Canada M5G 2C4.
illnesses (National Diabetes Data Group, 1979; American Rheumatism Association, 1981) and approaches have been devised to determine disease severity in critical care settings (Knaus et al. 1985; Pollack et al. 1987). Others have developed measures of the functional severity or burden of illness (Katz et al. 1963; Bergner et al. 1976). These measures each have broad applications in research. However, reliable and valid measures of disease severity which are not directly affected by psychologically-mediated causes of disability are unavailable for most medical conditions. Patients with endstage renal disease have been widely studied with respect to the psychosocial complications of this condition (Kutner et al. 1985; Craven et al. 1987; Hinrichsen et al. 1989; Petrie, 1989) and the impact of psychosocial factors on morbidity and mortality (Foster et al. 1973; Farmer et al. 1979; Wai et al. 1981; Burton et al. 1986; Shulman et al. 1989). The most commonly used indices of disease activity in ESRD have been the metabolic and physiological (e.g. blood urea nitrogen, haemoglobin, weight fluctuation) parameters, and non-specific indicators of physical disease severity (e.g. days in hospital, number of medications) (Foster et al. 1973; Farmer et al. 1979; Wai et al. 1981; Burton et al. 1986; Craven et al. 1987; Shulman et al. 1989). While the reporting of these indices helps to characterize the study samples, the
237
238
J. Craven and others
relationship between such parameters and the overall severity of the patient's physical condition has not been established. Also, inconsistency in the use of these disease severity parameters has complicated the comparison of results from different studies. Some indices (e.g. physical complaints, days in hospital) are confounded not only by the influence of medical factors, but also by psychosocial ones such as depression, somatization and abnormal illness behaviour (Benjamin et al. 1984). This overlap renders such variables less appropriate for use in studies which attempt to differentiate psychosocial from physical causes of disability. In addition, physiological indices which measure the severity of renal disease or the adequacy of dialysis, do not necessarily take into account the severity of the many other illnesses which may affect dialysis patients (e.g. diabetes mellitus, cardiovascular disease, etc.). Diabetes, in particular, is a well recognized risk factor for morbidity and mortality in dialysis patients (Hellerstedt et al. 1984; Garcia-Garcia et al. 1985; Silins et al. 1989). In fact, death in dialysis patients most commonly results from concurrent illness (Santiago & Chazan, 1989) not directly measured by the physiological indices of ESRD. The Endstage Renal Disease Severity Index (ESRD-SI) was devised for use as a control variable in a longitudinal study investigating the interactions of social support and depressive symptoms in dialysis patients. For this purpose, it was desirable to assess illness severity in a manner which minimized artefactual overlap with psychosocial variables. This report will describe the development of the instrument, preliminary studies of its use, reliability and validity testing, and the general applicability of the measure. METHODS AND SUBJECTS A series of meetings of the Toronto Hospital Nephrology Group were held to determine appropriate items for inclusion in the index. Ten illness categories which are common in patients with ESRD were included in the measure. These categories are: cardiovascular; cerebrovascular; peripheral vascular; peripheral neuropathy; bone disease; respiratory disease; visual impairment; autonomic neuropathy and gastro-
intestinal disease; access and dialysis events; diabetes mellitus; and a single 'other' category. These illnesses were chosen for inclusion in the index as they are the most common complications and concurrent conditions found in ESRD patients and are each major determinants of morbidity and mortality. Consistent with the overall goals of the index, physical conditions were included in the index. Psychiatric disorders were specifically not included. The index is designed for use by either an independent clinician investigator with full access to clinical information or the patient's own internist. Guided by examples in each category, the person completing the index rates the items on separate scales. Each scale contains a range of severity which corresponds to descriptions which are provided in each category (see Appendix). Both the instructions and examples provided on the ESRD-SI direct raters to disregard the apparent burden of illness on that patient or family and to rate only indicators of biological severity. Examples are provided to correspond to scores which are defined in the following ranges: 1-3 for biochemical or mild indications of disease; 4-6 for moderately severe physical signs, handicap or prognosis; 7-8 for severe physical handicap or prognosis; and 9 10 for an imminently lethal condition. The scoring system gives a different range of scores for each disease item and the additive sum of item scores provides the total severity index. Other methods of scoring the index, including the use of a power function to increase the scores for severe or multiple disease states, did not substantially alter the measure's performance. To assess inter-rater reliability, each of four nephrologists were asked to present two case summaries from their own clinical practices. For each of the eight cases presented, all four of the nephrologists individually completed an ESRDSI. Further psychometric testing of the ESRDSI was undertaken in two distinct samples. Test-retest reliability was studied in subjects (N = 79) receiving either centre haemodialysis (85 %) or intermittent peritoneal dialysis (15 %). This sample included men (56%) and women (44 %) with a mean age of 58 years (S.D. = 16-79) and represented all persons receiving these types of dialysis at our centre during the time the study was undertaken. Approximately onequarter of the subjects in this group were also
The ESRD-SI included in the validity testing described below. For the purpose of assessing test-retest reliability, each subject's internist was asked to complete an ESRD-SI on two occasions, one week apart. Construct and predictive validity were examined in a sample (TV = 154) consisting of men (53%) and women (47%) with a mean age of 50 years (S.D. = 16-29) and receiving either haemodialysis (21%), intermittent peritoneal dialysis (19%), or continuous ambulatory peritoneal dialysis (CAPD) (60%). Subjects for this sample had been enrolled in a longitudinal study of adjustment to dialysis and represented a consecutive series of persons beginning any type of dialysis at our centre over a two-year period. These subjects also completed a measure of functional disability, the Sickness Impact Profile (Bergner et al. 1976). Measures were re-administered at 6 and 12 months following the initial assessments. The validity of the ESRD-SI was tested with two-tailed / tests and one-way analysis of variance to determine its ability to discriminate subgroups which would be expected to demonstrate a higher overall disease severity (e.g. subjects on medical disability, with diabetes mellitus or those who subsequently died during the follow-up period). A multiple regression was performed to determine the unique ability of the ESRD-SI to predict mortality. RESULTS The overall inter-rater reliability for the ESRDSI was r = 092 (P < 001). Table 1 summarizes the test-retest correlation and the mean difference between ratings for each physician. The overall test-retest correlation for the index was 0-923 (P < 0001) with a mean difference between ratings of 0-34 (P = 0-470). Raters found that for most of their patients, the ESRD-SI could be easily completed in one to three minutes. Further analyses were undertaken on data from the second sample (N = 154), for which the mean ESRD-SI was 10-2 (S.D. = 9-59). Table 2 provides the / test and one-way analysis of variance results in this sample. For each variable found to be significant with analysis of variance, the Tukey Multiple Comparison Test was performed (at 0-05 probability level) to identify the variable groups responsible for significant F values. Significantly higher scores were found
239
for subjects who were older (P = 0003) or who were unemployed by reason of medical disability or retirement (P = 0-03). There was no difference according to sex or marital status. Significantly higher scores were also found for subjects on intermittent peritoneal dialysis (P = 0-002), with diabetes mellitus (P = 0001) or who died within six months of initial rating (P = 0002). To determine the sensitivity of the ESRD-SI to detect an imminently lethal clinical condition, the ESRD-SI rating most proximal to the time of death was used both for correlations and for multiple regression analysis. The ESRD-SI had a zero-order correlation with the variable' death' (coded0 = no, 1 = yes)ofr = 0-374(P < 0001). This was higher than the correlation of death with type of dialysis (r = 0-193; P = 0008), age (r = 0-170; P = 0018), or diabetes status (r = 0-097; P = 0115). A multiple regression analysis was run with death as the criterion variable and the ESRD-SI score, age, sex, diabetes status, and the type of dialysis as the predictor variables. The ESRD-SI had a unique R2 of 0-0996 (/(147) = 4-2; P< 00001). The total model explained an R2 of 01698 (F(6,147) = 5 0 1 ; P = 00001). Correlations were also found between the ESRD-SI and the Sickness Impact Profile total score (r = 0-359; P < 0-001), the physical item subscale (r = 0-462; P < 0-001) and the psychosocial item subscale (r = 0149, P < 0001) of this measure. DISCUSSION These findings demonstrate that the ESRD-SI is a suitable measure of illness severity in patients with endstage renal disease. Adequate interrater reliability suggests that the index may be reliably completed by clinician investigators who have full access to clinical information. Alternatively, as we have done in our own research using the ESRD-SI, research workers may request that each specific internist of study subjects complete the index on their own patients. Our experience is that the measure may be easily completed by these persons in less than three minutes. When used in this manner, the instrument has adequate test-retest reliability and is sensitive to factors associated with increased medical morbidity and mortality in dialysis patients. These factors include advanced
240
J. Craven and others
Table 1. ESRD-SI test-re test parameters (N = 79) Difference
Correlation Doctor
N
r
a b c d e All
15 21 19 13 11 79
0-781 0-968 0-943 0-929 0-916 0-923
Two-tailed probability /> = P< P< P< P< P
BRIEF COMMUNICATION
The Endstage Renal Disease Severity Index (ESRD-SI) J. CRAVEN, 1 C. LITTLEFIELD, G. RODIN AND M. MURRAY* From the Departments of Psychiatry and Nephrology, Toronto Hospital, University of Toronto, Canada
Measures of disease severity are required as experimental, control and outcome variables in studies of the psychosocial aspects of physical illness. The Endstage Renal Disease Severity Index (ESRD-SI) was developed for these purposes during a prospective study of patient adjustment to dialysis and endstage renal disease (ESRD). It was found to have adequate inter-rater reliability, test-retest reliability, and construct and predictive validity in two subsamples (total N = 233) of dialysis patients. The ESRD-SI is suitable as a sensitive indicator of disease severity in these patients.
SYNOPSIS
INTRODUCTION The term 'disease severity' has been used to refer to the degree of physiological or morphological disturbance, the impact of the disease process on the activities of the afflicted person (functional severity) or the impact of illness on family or society (burden of illness) (Stein et al. 1987). Measurement of disease severity is necessary to describe and compare patient samples (Charlson et al. 1986), to assess and predict quality of care (Green et al. 1990), and to investigate factors which determine clinical course (Cassileth et al. 1985). However, although there is increasing agreement that the accurate measurement of disease severity is a pre-requisite for such endeavours, references to disease severity are still, as Stein et al. (1987) have noted, inconsistent and vague in many published reports. We have argued elsewhere (Rodin et al. 1991) that this deficiency is a particular problem in studies designed to distinguish psychosocial from organic determinants of physical illness. Sensitive measures of physical disease severity are vital for such distinctions to be made. Some measures of physiological and morphological severity are available for specific • Also, the Toronto Hospital Nephrology Group. 1 Address for correspondence: Dr J. Craven, Psychosomatic Medicine Program, Department of Psychiatry, The Toronto Hospital: General Division 8EN-2I2, 200 Elizabeth Street, Toronto, Ontario, Canada M5G 2C4.
illnesses (National Diabetes Data Group, 1979; American Rheumatism Association, 1981) and approaches have been devised to determine disease severity in critical care settings (Knaus et al. 1985; Pollack et al. 1987). Others have developed measures of the functional severity or burden of illness (Katz et al. 1963; Bergner et al. 1976). These measures each have broad applications in research. However, reliable and valid measures of disease severity which are not directly affected by psychologically-mediated causes of disability are unavailable for most medical conditions. Patients with endstage renal disease have been widely studied with respect to the psychosocial complications of this condition (Kutner et al. 1985; Craven et al. 1987; Hinrichsen et al. 1989; Petrie, 1989) and the impact of psychosocial factors on morbidity and mortality (Foster et al. 1973; Farmer et al. 1979; Wai et al. 1981; Burton et al. 1986; Shulman et al. 1989). The most commonly used indices of disease activity in ESRD have been the metabolic and physiological (e.g. blood urea nitrogen, haemoglobin, weight fluctuation) parameters, and non-specific indicators of physical disease severity (e.g. days in hospital, number of medications) (Foster et al. 1973; Farmer et al. 1979; Wai et al. 1981; Burton et al. 1986; Craven et al. 1987; Shulman et al. 1989). While the reporting of these indices helps to characterize the study samples, the
237
238
J. Craven and others
relationship between such parameters and the overall severity of the patient's physical condition has not been established. Also, inconsistency in the use of these disease severity parameters has complicated the comparison of results from different studies. Some indices (e.g. physical complaints, days in hospital) are confounded not only by the influence of medical factors, but also by psychosocial ones such as depression, somatization and abnormal illness behaviour (Benjamin et al. 1984). This overlap renders such variables less appropriate for use in studies which attempt to differentiate psychosocial from physical causes of disability. In addition, physiological indices which measure the severity of renal disease or the adequacy of dialysis, do not necessarily take into account the severity of the many other illnesses which may affect dialysis patients (e.g. diabetes mellitus, cardiovascular disease, etc.). Diabetes, in particular, is a well recognized risk factor for morbidity and mortality in dialysis patients (Hellerstedt et al. 1984; Garcia-Garcia et al. 1985; Silins et al. 1989). In fact, death in dialysis patients most commonly results from concurrent illness (Santiago & Chazan, 1989) not directly measured by the physiological indices of ESRD. The Endstage Renal Disease Severity Index (ESRD-SI) was devised for use as a control variable in a longitudinal study investigating the interactions of social support and depressive symptoms in dialysis patients. For this purpose, it was desirable to assess illness severity in a manner which minimized artefactual overlap with psychosocial variables. This report will describe the development of the instrument, preliminary studies of its use, reliability and validity testing, and the general applicability of the measure. METHODS AND SUBJECTS A series of meetings of the Toronto Hospital Nephrology Group were held to determine appropriate items for inclusion in the index. Ten illness categories which are common in patients with ESRD were included in the measure. These categories are: cardiovascular; cerebrovascular; peripheral vascular; peripheral neuropathy; bone disease; respiratory disease; visual impairment; autonomic neuropathy and gastro-
intestinal disease; access and dialysis events; diabetes mellitus; and a single 'other' category. These illnesses were chosen for inclusion in the index as they are the most common complications and concurrent conditions found in ESRD patients and are each major determinants of morbidity and mortality. Consistent with the overall goals of the index, physical conditions were included in the index. Psychiatric disorders were specifically not included. The index is designed for use by either an independent clinician investigator with full access to clinical information or the patient's own internist. Guided by examples in each category, the person completing the index rates the items on separate scales. Each scale contains a range of severity which corresponds to descriptions which are provided in each category (see Appendix). Both the instructions and examples provided on the ESRD-SI direct raters to disregard the apparent burden of illness on that patient or family and to rate only indicators of biological severity. Examples are provided to correspond to scores which are defined in the following ranges: 1-3 for biochemical or mild indications of disease; 4-6 for moderately severe physical signs, handicap or prognosis; 7-8 for severe physical handicap or prognosis; and 9 10 for an imminently lethal condition. The scoring system gives a different range of scores for each disease item and the additive sum of item scores provides the total severity index. Other methods of scoring the index, including the use of a power function to increase the scores for severe or multiple disease states, did not substantially alter the measure's performance. To assess inter-rater reliability, each of four nephrologists were asked to present two case summaries from their own clinical practices. For each of the eight cases presented, all four of the nephrologists individually completed an ESRDSI. Further psychometric testing of the ESRDSI was undertaken in two distinct samples. Test-retest reliability was studied in subjects (N = 79) receiving either centre haemodialysis (85 %) or intermittent peritoneal dialysis (15 %). This sample included men (56%) and women (44 %) with a mean age of 58 years (S.D. = 16-79) and represented all persons receiving these types of dialysis at our centre during the time the study was undertaken. Approximately onequarter of the subjects in this group were also
The ESRD-SI included in the validity testing described below. For the purpose of assessing test-retest reliability, each subject's internist was asked to complete an ESRD-SI on two occasions, one week apart. Construct and predictive validity were examined in a sample (TV = 154) consisting of men (53%) and women (47%) with a mean age of 50 years (S.D. = 16-29) and receiving either haemodialysis (21%), intermittent peritoneal dialysis (19%), or continuous ambulatory peritoneal dialysis (CAPD) (60%). Subjects for this sample had been enrolled in a longitudinal study of adjustment to dialysis and represented a consecutive series of persons beginning any type of dialysis at our centre over a two-year period. These subjects also completed a measure of functional disability, the Sickness Impact Profile (Bergner et al. 1976). Measures were re-administered at 6 and 12 months following the initial assessments. The validity of the ESRD-SI was tested with two-tailed / tests and one-way analysis of variance to determine its ability to discriminate subgroups which would be expected to demonstrate a higher overall disease severity (e.g. subjects on medical disability, with diabetes mellitus or those who subsequently died during the follow-up period). A multiple regression was performed to determine the unique ability of the ESRD-SI to predict mortality. RESULTS The overall inter-rater reliability for the ESRDSI was r = 092 (P < 001). Table 1 summarizes the test-retest correlation and the mean difference between ratings for each physician. The overall test-retest correlation for the index was 0-923 (P < 0001) with a mean difference between ratings of 0-34 (P = 0-470). Raters found that for most of their patients, the ESRD-SI could be easily completed in one to three minutes. Further analyses were undertaken on data from the second sample (N = 154), for which the mean ESRD-SI was 10-2 (S.D. = 9-59). Table 2 provides the / test and one-way analysis of variance results in this sample. For each variable found to be significant with analysis of variance, the Tukey Multiple Comparison Test was performed (at 0-05 probability level) to identify the variable groups responsible for significant F values. Significantly higher scores were found
239
for subjects who were older (P = 0003) or who were unemployed by reason of medical disability or retirement (P = 0-03). There was no difference according to sex or marital status. Significantly higher scores were also found for subjects on intermittent peritoneal dialysis (P = 0-002), with diabetes mellitus (P = 0001) or who died within six months of initial rating (P = 0002). To determine the sensitivity of the ESRD-SI to detect an imminently lethal clinical condition, the ESRD-SI rating most proximal to the time of death was used both for correlations and for multiple regression analysis. The ESRD-SI had a zero-order correlation with the variable' death' (coded0 = no, 1 = yes)ofr = 0-374(P < 0001). This was higher than the correlation of death with type of dialysis (r = 0-193; P = 0008), age (r = 0-170; P = 0018), or diabetes status (r = 0-097; P = 0115). A multiple regression analysis was run with death as the criterion variable and the ESRD-SI score, age, sex, diabetes status, and the type of dialysis as the predictor variables. The ESRD-SI had a unique R2 of 0-0996 (/(147) = 4-2; P< 00001). The total model explained an R2 of 01698 (F(6,147) = 5 0 1 ; P = 00001). Correlations were also found between the ESRD-SI and the Sickness Impact Profile total score (r = 0-359; P < 0-001), the physical item subscale (r = 0-462; P < 0-001) and the psychosocial item subscale (r = 0149, P < 0001) of this measure. DISCUSSION These findings demonstrate that the ESRD-SI is a suitable measure of illness severity in patients with endstage renal disease. Adequate interrater reliability suggests that the index may be reliably completed by clinician investigators who have full access to clinical information. Alternatively, as we have done in our own research using the ESRD-SI, research workers may request that each specific internist of study subjects complete the index on their own patients. Our experience is that the measure may be easily completed by these persons in less than three minutes. When used in this manner, the instrument has adequate test-retest reliability and is sensitive to factors associated with increased medical morbidity and mortality in dialysis patients. These factors include advanced
240
J. Craven and others
Table 1. ESRD-SI test-re test parameters (N = 79) Difference
Correlation Doctor
N
r
a b c d e All
15 21 19 13 11 79
0-781 0-968 0-943 0-929 0-916 0-923
Two-tailed probability /> = P< P< P< P< P
