QuafityofLifiResearch,
1, pp. 63-72
Reliability and validity of the Katz Adjustment Scales iti an epilepsy sample B. G. Vickrey,’ R. D. Hays, R. H. Brook and R. Rausch Departments of Medicine and Health Services (R. H. Brook), Department of Psychiatry (Ft. Rausch), Department of Neurology (B. G. Vickrey, R. Rausch), University of California, C-128 Reed Building, 710 Westwood Plaza, Los Angeles, California 90024-l 789, USA; RAND, Santa Monica, California, USA (R. H. Brook, R. D. Hays).
Development of quality of life measures da now0 is time-consuming and expensive, and a number of instruments are already available for general use. Reevaluation and refinement of quality of life tools are needed to Improve the existlng pool of measures. In this study, data from a sample of 336 epilepsy patients were used to revise a measure of social adjustment and emotional status developed in the W6Os, the 127-item form Rl of the Katx Adjustment Scales (KAS-Rl). Using a comprehensive item analysis procedure, we increased the number of Items used in scoring the KAS-I?1 from 76 to 113 and rubstsntially improved the reliability of scales in both the original sample of 326 epilepsy patients and in a second administration to a ‘cross-validation’ sample of lg3 epilepsy patients. Support for the validity of the revised KAS-Rl scoring system was obtained in a known groups analysis of patients who hsd prevlourrly undergone surgery for epilepsy: mean scores were signlfkzmtly higher (p < 0.05) on 11 out of 14 scale0 in the revised KAS-Rl for patients who were completely seizure-free compared to patients who were having seizures with loss of consciousness, after adjusting for age, gender, and anticonvulsant medication use. Key words: Epilepsy, epilepsy surgery, Katz Adjustment Scales, mental health, multiirait scaling analysis, quality of life, social health.
This project was supported in part by The Robert Wood Johnson Clinical scholars Program, the Agency for Health Care Policy and Research (fellow&p suppor$ &d by BRSG SO7 RR5756 awarded bv the Biomedical Research Su~uort Grant Program, Division df Research Resources, Nationalxkitutes of Health. Opinions are those of the authors and do not necessarily reflect the views of the sponsoring institutions, RAND, or the University of CaLifomia, Los Angeles. l
To whom correspondence should be addressed.
@ 1992 Rapid Communications of Oxfird Ltd
The objectives of medical care include not only prevention of death and physical disability but also restoration and promotion of emotional well-being and integration of the individual into societal roles, or social adjustment. Increasingly, comprehensive evaluation of health-related quality of life, which includes physical, mental, and social health, has been advocated for evaluating the effectiveness of treatments for a spectrum of medical conditions. For example, the recent NIH Consensus Conference Panel on surgery for epilepsy recommended the development of reliable and valid quality of life measures for studies assessing the effectiveness of surgical intervention for epilepsy.’ Previously reported measures of psychosocial functioning in epilepsy surgery patients are limited by lack of information about reliability or validity or both. In addition, many of these measures are intervieweradministered, making them impractical for use in large-scale or multicentre studies. These measures are more extensively reviewed elsewhere.’ For patients with neurological conditions, emotional status and social functioning may be the most critical areas of impairment and among the most important outcomes to measure.3 Measures of these can be obtained either by self-report or by a trained observer. If a patient’s cognition is compromised, he or she may be unable to provide self-report information. Psychopathology or a desire to bias responses to please the health care provider or other recipient of the information may also make the patient an unreliable informant. On the other hand, use of even trained specialists to obtain these measures may introduce systematic observer bias. Because of the above problems, many investigators of chronic neurologic problems have used a Quality of Life Research . Vol 1 .I992
63
B. G. Vickrey et al. relative or friend’s report on a standardized instrument, the Katz Adjustment Scales (KAS), to measure emotional and social functioning.4-8 The KAS was originally developed in the early 1960s to measure social adjustment and behaviour of schizophrenic patients discharged from the hospital into the community .9 At the time of its development, the inventory was unique in that it included both self-report ratings and ratings from a close friend or relative who assessed the patient’s psychiatric symptoms, social behaviour, and daily activities. The KAS consists of multiple forms. In this paper we will discuss only the KAS-Rl, which has 127 items and 12 scales. The KAS-RI is the relative or friends ratings of the patient’s psychiatric symptoms and social behaviour . Although Katz and Lyerly presented data on the reliability and validity of the KAS-RI in schizophrenic patients, its validity in other patient populations has not been clearly demonstrated. Internal consistency reliability coefficients were less than 0.70 (the minimally acceptable level”) for six of 11 scales for which these coefficients were originally reported. 9 In addition, 50 of the 127 items in the KAS-Rl are not used in any of the 12 scales. Based on these observations, McDowell and Newell” recommended use of the KAS-Rl only if none of several other measures of social health are applicable. In this paper we have reevaluated item scoring procedures for the KAS-Rl in patients with epilepsy. Our goals were (1) to determine whether scale reliability and the number of items used in scoring could be increased, (2) to evaluate whether the improved psychometric properties of the revised scale scoring ‘cross-validated in a second administration of the instrument, and (3) to assess the construct validity of the revised measure. We began our analysis with the Katz and Lyerly scoring of the KAS-Rl and revised it using a combination of both clinical and statistical considerations.
Samples
To revise the KAS-RI, we used data from a sample of 328 patients who were seen between 1974 and 1989 at one west coast epilepsy centre. Over 90% of these patients had been referred to the epilepsy centre for evaluation of medicationresistant epilepsy; another 21 patients (6.4%) had 64
Quality of Life Research . Vol 1 . 1992
previously undergone surgery for epilepsy. The mean age was 30 years, ranging from 12 to 63 years of age; 92% of patients were age 18 years or older. The majority (56%) of the patients were male. As part of a comprehensive neuropsychological evaluation that all 328 patients underwent prior to surgery, patients were given the KAS-Rl and were asked to have it filled out by a relative or close friend and returned by mail. The KAS-Rls were completed by a spouse or fiance (30%), mother (41%), father (9%), sibling (5%), some other relative (1.5%), or friend (6%). For 8% of the cases, the relationship between the respondent and the patient was not reported. After revising the KAS-Rl on the above sample, we ‘cross-validated it using a mail survey of 193 adults with epilepsy who had been consecutively evaluated by an epilepsy surgery protocol at the same west coast epilepsy centre; 83% of them had undergone epilepsy surgery prior to collecting these data. The severity of epilepsy in this predominantly post-operative sample ranged from bein completely seizure-free to having daily seizures. 5 The majority (144) of the 193 patients in the second administration of the KAS-RI also participated in the first administration. The second administration occurred in 1990, at least 1 year after the first administration and either following epilepsy surgery or following a protocol evaluated without surgery. The 1990 mail survey included the KAS-Rl and questions for the patient about seizure occurrence over the preceding year and use of anticonvulsant medications. Patients were asked to give the KAS-RI to a close friend or relative, and these completed forms were mailed directly to the study investigators. (The possibility that a close friend or relative’s report would be biased by direct interaction with the patient was minimized by requesting that the KAS-Rl form be returned to the investigators in a separate envelope and by assuring the patient in the cover letter of the confidentiality of these data). A total of 224 patients were eligible for the survey in 1990, yielding a response rate from the close friend or relative of 86% (193 respondents). Forty-seven per cent (105) of the 193 epilepsy patients were male. The mean age of this cohort was 34 years, ranging from 18 to 66 years of age. The inventories were completed by a spouse or fiance (34%), mother (33%), father (6%), sibling (7%), some other relative (7%), or friend (10%); the relationship was not reported in 4% of cases. A test of the construct validity of the revised
Katz Adjustment Scalesand epilepsy measure was evaluated in the subset of 136 of the 193 participants in the 1990 mail survey who had undergone surgery for epilepsy more than 12 months prior to responding to the questionnaire. This subgroup was selected so that construct validity could be evaluated in a homogeneous (post-surgical) sample whose quality of life was less likely to be influenced by the acuteness of the surgical intervention. All studies described above were reviewed and approved by the UCLA Human Subjects Protection Committee.
Measures Katz Adjustment Scales firm Rl WAS-RI). For each of the 127 items in the KAS-Rl, the relative or friend is asked to rate the patient according to ‘how your relative or friend has looked to you during the past few weeks on these things’. For each item there are four response choices: almost never, sometimes, often, and almost always. Original psychometric analyses of the KAS-Rl by Katz and Lyerly led to the use of 76 of the 127 items to score 12 scales: General psychopathology, Suspiciousness, Anxiety, Negativism, Confusion, Belligerence, Withdrawal, Bizarreness, Hyperactivity, Helplessness, Verbal expansiveness, and Nervousness (Table 1). We evaluated and modified a 126item version of the KAS-Rl (one item from the original Withdrawal scale was accidentally omitted; Appendix A).
Analysis plan We used multitrait scaling analysis to evaluate hypothesized item clusters. Multitrait scaling is an analytic approach that applies the logic of convergent and disaiminant validity to the evaluation of multiple constructs when each is measured using the same method. l2 Multitrait scaling is based directly on scale development procedures advocated several decades ago.‘3*‘4 In multitrait scaling, items are examined with respect to how well they correlate with the sum of other items designed to measure the same construct versus how they correlate with items that measure different constructs. Ideally, an item should be most highly correlated with its hypothesized scale and display lower correlations with the other scales. Internal consistency reliability for each scale was estimated by Cronbach’s alpha.15 We evaluated item convergence and discrimination by inspecting item-scale correlations (corrected for item overlap) produced by the Multitrait Analysis Program, a microcomputer program that generates an itemscale correlation matrix. lzJ6 Use of this program to revise the KAS-Rl scoring procedure involved two types of searches. One search focused on determining whether items in hypothesized scales should be retained or moved out of their hypothesized scale. We looked for items with poor convergence with their hypothesized scale and/or lack of discrimination across scales, moving items as appropriate. The second type of search concentrated on whether items not previously scored in scales should be moved and included in a scale.
Table 1. Number of items and internal consistency reliability for the Katz Adjustment Scales Rl form (KAS-Rl) epilepsy patient samples: original scoring of scales Original KAS-Rl scales
Alpha reliability
Number of items
First administration General psychopathology Suspiciousness Anxiety Negativism Confusion Belligerence Withdrawal Bizarreness Hyperactivity Helplessness Verbal expansiveness Nervousness Miscellaneous (items not used)
24 4 ii 3 4 5 5 3 4 5 4 50
in
(n = 328)
Second administration (n = 193)
0.89 0.81 0.81 0.78 0.70 0.87 0.88 0.88 0.85 0.84 0.58 0.57 NR
0.89 0.78 0.74 0.78 0.89 0.88 0.74 0.57 0.88 0.87 0.73 0.84 NR
NR = not relevant Quality
of Life Research . Vol I .I992
65
B. G. Vickrey et al.
An item was moved into a new scale if the correlation of the item with the new scale exceeded the item’s correlations with other scales (corrected for item overlap) and the move was clinically reasonable. If none of the item-scale correlations for an item exceeded 0.30 or if the highest item-scale correlations for an item were approximately equal (i.e., the item did not discriminate between scales), we put the item in a residual, miscellaneous (‘holding’) scale. The Multitrait Analysis Program was then rerun to evaluate the revised scales, and the item-scale correlation matrix was reviewed once again. Iterations were discontinued when moving items no longer improved the multitrait scaling properties of the instrument. Each run of the program is summarized in terms of ‘scaling successes’, ‘scaling failures’, and ‘probable scaling failures’. A scaling success is defined as a correlation of an item with its hypothesized scale exceeding the correlation of this item with another scale by two standard errors, and a scaling failure occurs when the correlation of an item with its hypothesized scale falls below a correlation of the item with another scale by two standard errors. A probable scaling failure is counted if the correlation of an item with another scale lies within two standard errors of the correlation of the item with its hypothesized scale. We evaluated the improvement in item discrimination across scales by comparing results of multitrait scaling analyses based on the original scoring of the KAS-Rl with our revised scoring. Specifically, the percentages of scaling successes, scaling failures, and probable scaling failures were compared for the original and revised scoring of the instrument. To develop names for the revised scales that reflected the item content, a panel of three professionals (two epilepsy behaviour specialists and a neurologist) reviewed lists of the revised item clusters and suggested names based on the content of the lists. Using data from the second administration of the KAS-RI, we evaluated the construct validity of the revised KAS-Rl by analysing scale scores by seizure status. We postulated that LAS-R1 scores would be higher among seizure-free patients and lower among patients having severe seizures. We used data on the 136 patients in the cohort who had undergone epilepsy surgery more than 12 months prior to the study. Seizure status was determined based on self-reported seizure occurrence over the preceding 12 months. Patients were classified as having ‘full seizures’ if they had had 66
Quality of Life Research . Vol 1 .I992
one or more seizures with alteration or loss of consciousness, ‘simple partial seizures’ if they had had one or more seizures without alteration or loss of consciousness, or ‘seizure-free’ if they had had no seizures of any kind, over the 12 months prior to the second KAS-Rl administration. Scores on the KAS-Rl scales were transformed linearly to O-100 point scales for ease of comparison, with higher values on a scale indicating better functioning (less impairment). Mean scale scores were calculated for each of these three seizure classification groups. The significance of differences in revised KAS-Rl mean scores between seizure classification groups was assessed by upriori ANOVA-based t-statistics. We used relative efficiency analysis to compare different scales in the revised KAS-Rl in terms of their sensitivity to seizure occurrence.‘7 One-way ANOVA linear trend F-ratios were calculated for each of the revised KAS-Rl scales and compared across scales to evaluate their relative sensitivity to seizure classification. (Relative efficiencies themselves are determined by dividing the F-ratio of each scale by the F-ratio of a scale selected to be the ‘reference’ scale. This ‘reference’ scale has a relative efficiency equal to 1, by definition). Age, gender, and medication use of the epilepsy patients may all affect scores on the KAS-Rl. Therefore, as a sensitivity analysis, ordinary least squares regression was used to estimate the unique impact of seizure status on revised KAS-Rl scales, adjusting for these variables. Medication use was dichotomized into taking no anticonvulsant medication vs. taking one or more anticonvulsant medications at the time of the study.
Results First administration (n = 328): modification of the KAS-Rl Nine iterations of multitrait scaling were performed. At least two items were moved after each iteration. The maximum number of items were moved in the first iteration, when a total of 43 items (primarily previously unscored items) were moved to a specific scale. As a result of this process, we increased the number of items scored from 76 to 113 out of 126 items, and the number of scales increased from 12 to 14 (Tables 1 and 2). The three members of the panel named all 14 of the revised scales based on
Katz Adjustment Scalesand epilepsy their content. The revised scales are Oversensitivity/fearfulness (18 items), Social (10 items), Irritability (nine items), Dependency (15 items), Acting out (12 items), Paranoia (five items), Abnormal thought process (five items), Withdrawal (11 items), Emotional lability (six items), Nervousness (five items), Sociopathy (four items), Bizarreness (four items), Hyperactivity (four items), and Disorientation (five items) (Table 2; see Appendix A). A comparison of these two sets of item g-roupings reveaIs that the Suspiciousness, Confusion, and Belligerence scales of the original version are subsets of the Paranoia, Disorientation, and Acting out scales, respectively, of the revised version. The revised item groupings comprising the Dependency and Social scales are derived primarily from miscellaneous, unused items in the original KAS-Rl, and item dusters forming the Irritability and Emotional lability scales by the revised scoring procedure are derived primarily from items formerly scored in the General psychopathology scale. The remainder of the revised item clusters are partial combinations of items from two or more of the original scales. Comparing the item-scale correlations based on the originaI scoring of the KAS-Rl to the revised scoring, we found that scaling successes increased from 73% to 88%, scaling failures decreased from 1% to 0%, and probable failures decreased from 27% to 12%. Internal consistency reliability coefficients (Cronbach’s alpha) for the original 12
KAS-Rl scales in our sample ranged from 0.57 for the four-item Nervousness scale to 0.89 for the 24item General psychopathology scale; alpha feII below the suggested reliability cutoff for group comparisons’0 on seven of the 12 scales. In contrast, internal consistency reliability coefficients for the 14 revised KAS-Rl scales ranged from 0.66 to 0.88, and the number of scales with reliability coefficients equaIIing or exceeding 0.70 improved from five out of 12 scales to 12 out of 14 scales.
Second administration the instrument
(n = 193): testing
k?rn analysis. Multitrait scaling analysis revealed that item dis crimination improved in our test sample of 193 patients. Scaling successes increased from 61% of aII item-scale correlations when the original 12 scales were used, to 74% when the 14 revised scales were used. The percentage of scaling failures decreased from 0.7% to 0.2%, and the percentage of probable failures fell from 38% to 26%. Reliability coefficients ranged from 0.63 to 0.88, and the number of scales with reliability coefficients equaIli.ng or exceeding 0.70 increased from six out of 12 to 12 out of 14 scales (Tables 1 and 2). Seizure-free patients scored significantly higher than patients classified as having full seizures on
Table 2. Number of items and internal consistency reliability for the KAS-Rl in epilepsy patient samples: new wring of scales Revised KAS-Rl scales
Alpha reliability
Number of items
First administration
(n= 328) Oversensitivity/fearfulness Social Irritability Dependency Acting out Paranoia Abnormal thought process Withdrawal-R Emotional lability Nervousness-R Sociopathy Bizarreness-R Hyperactivity-R Disorientation Miscellaneous (items not used)
18 10 9 15 12 2 11 6 5 4 4 4 5 13
Second administration (n= 193)
0.66 0.87 0.87 0.85 0.64 0.83 0.80 0.79 K
0.87 0.87 tE 0:85 0.79 0.64 0.64 0.78 0.78 0.63 0.69 0.71 0.75 NR
0:71 0.72 0.67 0.66 NR
NR = not relevant
Qualityof tifi
Research
- Vol 1 - 1992
67
B. G. Vickrey et al. nine of 14 scales before adjustment for age, gender, and medication use, and on 11 of 14 scales after adjustment for these variables (Table 3). (Differences between mean scores on the Irritability and Paranoia scales increased and were significant at p < 0.05 after adjustment). Patients having only simple partial seizures scored in-between the other groups on 12 scales and significantly higher than patients having full seizures on seven of these scales (p < 0.05; Table 3). The relative sensitivity to seizure differences as measured by linear trend F-ratios was greatest for the Nervousness (F = 32.6), Dependency (F = 22.6), Oversensitivity/fearfulness (F = 17.0), and Withdrawal (F = 13.9) scales of the revised KAS-Rl, while the Sociopathy and Hyperactivity scales were least sensitive (linear trend F-ratios less than 1; Table 3).
Discussion The Katz Adjustment Scales form Rl &AS-Rl) is a widely used measure of social adjustment. The KAS-Rl is unique in that social behaviour and adjustment are evaluated by a close friend or relative rather than by either self-report or trained expert. However, 50 of its 127 items are not used in
scoring, and the psychometric properties of several of the 12 scales are suspect. In the present study we evaluated the scoring procedure employed for the KAS-RI in patients with epilepsy. We produced a revised scoring scheme that uses 113 of the KAS-Rl items. The revised scoring yielded substantial improvements in scale internal consistency reliability and item discrimination across scales. These improvements were observed in two different administrations of the KAS-Rl. The content validity of the revised scales was supported in that three experts were independently able to apply similar or identical, clinically meaningful labels to the 14 item clusters. Supporting the construct validity of the inventory, mean scores on 11 of its 14 scales were significantly related to seizure status after adjusting for age, gender, and medication use. Nervousness, Dependency, Oversensitivity/ fearfulness, and Withdrawal scales of the KAS-Rl were the most sensitive to differences in seizure status, with linear trend F-ratios exceeding 13. Previous studies using different measures also support associations between ongoing seizure occurrence and impairment in these areas. For example, Arntson and colleagues3 administered the Hopkins Symptom Checklist to a sample of epilepsy patients (median reported seizure frequency of two per month). Symptoms similar in
Table 3. Mean revised KAS-Rl scale scores by seizure classification Seizure classification
Revised KAS-Rl scales Seizure-free Nervousness-R Dependency Oversensitivity/fearfulness Withdrawal-R Social Abnormal thought process Disorientation Emotional lability Acting out Bizarreness-R Irritability* Paranoia* Sociopathy Hyperactivity-R
Simple partial
(n=40)
(n = 42)
88.988 87.338 92.76a 87KIa 81.428 89.13a 9O.67a 93.8ga 92.36a 96.81 a& 78.148 92.83a 97.71 a 85.21 a
82.548 84.5Qa 91 .Ol a 83.70a 78.658 87.62a 86.968sb 93.65a 90.97aJJ 98.21 a 75.138 90.16a 98.41 a 81 .75a
Linear trend F-ratio Full seizures (n=!54)
68.92b 74.77b 84.9gb 76.74b 70.36b 79.80” 82.25b 88.1 7b 87.5ab 93.67b 71 .Oaa 88.52a 96.91 a 84.10a
32.58 22.60 16.98 1394 8.54 8.31 6.87 6.44 5.16 3.81 3.48 2.17 0.70 0.06
Note: Higher scores indicate better functioning (O-100 point scales). Seizure classification is based on self-reported seizure occurrence over the 12 months preceding completion of the KAS-Rl . Study patients who had undergone surgery more than 12 months previously were included. One-way ANOVA linear trend F-ratios of KAS-Rl scales by seizure classification are shown. Degrees of freedom are (1,134). a,bMeans within a row with different superscripts differ significantly (p c 0.05) from one another as determined by ANOVA-based t-statistics. *After adjustment for age, gender, and anticonvulsant medication use, the differences in predicted mean scores between seizure-free and full seizure patients, based on ordinary least squares regression, were also significant (p < 0.05) for these scales. 68
Quality of Life Research . Vol 1 .I992
Katz Adjustment Scalesand epilepsy content to the items on the KAS-RI Nervousness scale (“I feel nervous or shaky inside”, “I notice my hands trembling”), Oversensitivity/fearfulness scale (“I cry easily or feel like crying”, “I feel fearful or afraid’, “I feel downhearted or blue”, “I have thoughts of possibly ending my life”), Dependency scale (“I feel low in energy or slowed down”, “I have headaches or head pains”), and Withdrawal scale (“I feel bored or have little interest in doing things”) were endorsed from 1.5 to 15 times more frequently by their sample of patients with epilepsy than by a general population sample. In another study, neuropsychologists rated a group of 31 epilepsy patients on a scale of dependency before and after surgery for epilepsy. One year post-operative scores differed significantly between seizure-free patients and patients having little or no improvement in seizure control after surgery. l8 Not surprisingly, scores on the Sociopathy scale were not related to seizure occurrence. Mean Sociopathy scale scores exceeded 96 out of 100 possible points, suggesting that the content of this scale is not explained by epilepsy. Indeed, the two epilepsy behaviour specialists who helped identify content areas for the revised item clusters informally (and blind to the results of the analysis) rated the Sociopathy scale as being least relevant to these epilepsy patients. Our findings are also consistent with previously documented relationships between post-operative seizure occurrence in epilepsy patients and mental and social health as measured by the Epilepsy Surgery Inventory (ESI-55), a 55-item measure tapping 11 dimensions of health-related quality of life. Similar to what was found for many scales of the KAS-Rl, mean scores as a function of seizure classification on the ESI-55 emotional well-being scale and the social function scale were significantly higher among seizure-free patients compared to patients having seizures with loss of consciousness. 2 The need for proxy respondents and validity of their responses in assessing quality of life has only been preliminarily addressed in the literature. Certainly, proxy respondents are essential when patients are unable to respond due to disability and very useful when patients’ responses are potentially unreliable. In developing the Katz Adjustment Scales, Katz and Lyerly justified the supplementation of self-reported psychiatric symptoms and social behaviour with measures based on a friend or relative’s report by pointing out that use of the proxy protects against the
limitation of relying solely on the patient for information on social functioning.9 Further evaluation of the reliability and validity of our revised scoring of the KAS-Rl is needed. Of particular importance are studies examining the sensitivity of these scales to known differences between groups of patients at a point in time as well as their responsiveness to change over time. l9 In addition, the generalizability of these results to other diseases needs to be evaluated. The revised scales described here are recommended in the interim if an assessment of the social and emotional functioning of an epilepsy patient from a relative or friend’s perspective is desired.
Acknowledgements The authors are grateful to several people who helped with implementing this project, including Jerome Engel, Jr, Jason Graber, Maria Melendez, Juan Vasconez, and Cathy Ary. We thank Lucille Spiff for help in preparing the manuscript. References 1. National Institutes of Health Consensus Conference, Surgery for Epilepsy. JAMA 1990; 264: 729-733. 2. Vickrey BG, Hays RD, Graber J, et al. A healthrelated quality of Iife instrument for patients evaluated for epilepsy surgery. Med Cure 1992 (in press). 3. Amtson P, Droge D, Norton R, et al. The perceived psychosocial consequences of having epilepsy. In: Whitman S, Hermann BP, eds. Psyc/~yofltology in Epilepsy: So&Z Dirtzensions. New York: Oxford University Press, 1986: 143-161. 4. Rausch R. Psychological evaluation. In: Engel J, ed. Surgical Treatment offhe Epikpsies. New York: Raven Press, 1987: 181-195. 5. Hinheldez NS, Corrigan JD. The structure of headinjured patients’ neurobehavioural complaints: A preliminary study. Brain Injury 1990; 4(2): 115-133. 6. Fordyce DJ, Roueche JR, Prigatano GP. Enhanced emotional reactions in chronic head trauma patients. J Neural Neurosurg Psychiatry 1983; 46: 620-624. 7. Rao SM, Leo GJ, EIIington MS, et al. Cognitive dysfunction in multiple sclerosis. II. Impact on employment and social functioning. Neurology 1991; 41: 692-6%. 8. Dashieff, Richard. Personal communication, April 1991. 9. Katz MM, Lyerly SB. Methods for measuring adjustment and social behavior in the community: I. Rationale, description, discriminative validity and scale development. PsycholRep 1963; 13: 503-535. 10. Nurmally J. Psychometric Theoy, 2nd edition. New York: McGraw-I-W, 1978. Qualityof Lij+Research. Vol1 f 2992 69
B. G. Vicky
et al.
11. McDowell I, Newell C. The Katz Adjustment scales. In: Measuring Health: A Guide to Rating Scalesand Questionnaires. New York: Oxford University Press, 1987: 169-172. 12. Hays RD, Hayashi T. Beyond internal consistency reliability: Rationale and user’s guide for the Multitrait Analysis Program on the microcomputer. Behav ResMethods lnstrum Comput 1990; 22: 167-175. 13. Jackson DN. Multimethod factor analysis in the evaluation of convergent and discriminant validity. PsycholBull 1969; 72: 30-49. 14. Loevinger J, Gleser G, DuBois PH. Maximizing the discriminating power of a multiple-score test. Psychometrika 1953; 18: 309-317. 15. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika1951; 16: 297-334.
16. Hays RD, Hayashi T, Carson S, et al. User’sGuidefir the Multitrait Analysis Program (MAP). N-27%~RC. RAND, Santa Monica, CA, 1988. 17. Liang MI-I, Larson MG, Cullen KE, et al. Comparative measurement efficiency and sensitivity of five health status instruments for arthritis research. Arthritis Rheum 1985; 28: 542-547. 18. Rausch R, Crandall PH. Psychological status related to surgical control of temporal lobe seizures. Epikpsia 1982; 23: 191-202. 19. Guyatt GH, Deyo RA, Charlson M, et al. Responsiveness and validity in health status measurement: A clarification. J Clin Epidemiol 1989; 42(S): 403-408.
(Received 10 October 2991; accepted in revised form 30 October 1991)
Appendix A. Revised scales in the KAS-Rl Nervousness 16. Tries too hard 20. Gets nervous easily 21. Jittery 22. Worries or frets 63. Hands tremble Dependency 9. Acts as if he/she does not have much energy 10. Looks worn out 17. Needs to do things very slowly to do them right 41. Complains of headaches, stomach trouble, other physical ailments 72. Needs a lot of attention 73. Behaviour is childish 74. Acts helpless 75. Is independent (Note: item is reverse-coded in this scale) 76. Clumsy; keeps bumping into things or dropping things 61. Gets into peculiar positions 62. Makes peculiar movements 90. Acts as if he’s/she’s confused about things: in a daze 92. Acts as if he/she can’t concentrate on one thing 93. Acts as if he/she can’t make decisions 95. Hard to understand his/her words Oversensitivity/fearfulness 2. Gets very self-critical, starts to blame himself/herself for things 3. Cries easily 4. Feels lonely 11. Feelings get hurt easily 12. Feels that people don’t care about him/her 15. Gets very sad, blue 18. Has strange fears 19. Afraid something terrible is going to happen 23. Gets sudden fright for no reason 24. Has bad dreams 27. Attempts suicide 109. Talks as if he/she committed the worst sin 111. Talks about people or things he’s/she’s very afraid of 115. Says he/she is afraid that he/she will not be able to control himself/herself 116. Talks about strange things that are going on inside his/her body 117. Says how bad or useless he/she is 123. Says that something terrible is going to happen 125. Talks about suicide
70
Quality
of
Life Research . Vol1 ’ 1992
Katz Adjustment Scalesand epilepsy Appendix continued Withdrawal 5. Acts as if he/she has no interest in things 8. Just sits 67. Stays away from people 69. Shy 70. Quiet 71. Prefers to be alone 60. Very slow to react 84. Will stay in one position for a long period 96. Speaks so low you cannot hear him/her 102. Speaks very slowly 103. Acts as if he/she wants to speak but can’t Abnormal thought process 91. Acts as if he/she can’t get certain thoughts out of his/her mind 94. Talks without making sense 104. Keeps repeating the same idea 105. Keeps changing from one subject to another 119. Says the same thing over and over again Social 39. Generous 46. Is co-operative 49. Answers when talked to 54. Friendly 57. Pleasant 56. Gets along well with people 62. Is dependable 63. Is responsible 65. Obedient 66. Shows good judgement Emotional lability 30. Acts as if he/she has no control over his/her emotions 31. Laughs or cries at strange times 32. Has mood changes without reason 34. Gets very excited for no reason 35. Gets very happy for no reason 126. Talks about strange sexual ideas Bizarreness 13. Does the same thing over and over again without reason 25. Acts as if he/she sees people or things that aren’t there 26. Does strange things without reason 124. Believes in strange things Disorientation 65. Loses track of day, month, or year 66. Forgets his/her address or other places he/she knows well 87. Remembers the names of people he/she knows well (Note: item is reverse-coded in this scale) 88. Acts as if he/she doesn’t know where he/she is 89. Remembers important things (Note: item is reverse-coded in this scale) Acting out 28. Gets angry and breaks things 33. Has temper tantrums 36. Acts as if he/she doesn’t care about other people’s feelings 37. Thinks only of himself/herself 45. Gets into fights with people 47. Does the opposite of what he/she is asked 50. Curses at people 51. Deliberately upsets routine
Quality
of Lifi Research . Vol 1 .I992
71
B. G. Vichy
et al.
Appendix continued 99. Speaks very loudly 100. Shouts or yells for no reason 112. Threatens to injure certain people 113. Threatens to tell people off Irritability 42. Bossy 44. Argues 46. Stubborn 52. Resentful 55. Gets annoyed easily 56. Critical of other people 110. Talks about how angry he/she is at certain people 120. Complains about people and things in general 127. Gives advice without being asked Paranoia 40. Thinks people are talking about him/her 43. Acts as if he/she is suspicious of people 107. Says that people are talking about him/her 106. Says that people are trying to make him/her do or think things he/she doesn’t want to 122. Says or acts as if people are after him/her Socioparhy 59. Lies 60. Gets into trouble with the law 61. Gets drunk 66. Takes drugs other than recommended by hospital or clinic Hyperactivity 6. Is restless 7. Has periods when he/she can’t stop moving or doing something 77. Moves about in a hurried way 101. Speaks very fast Note: Item numbers refer to their position in the measure that was administered to our patients and are identical to the numbering of items listed in Katz and Lyerly (1963), pages 512-513.9 The item omitted from the original 127 item KAS-Rl was #76: “Moves about very slowly”.
72
Quality of Life Research . Vol 1 .1992
1, pp. 63-72
Reliability and validity of the Katz Adjustment Scales iti an epilepsy sample B. G. Vickrey,’ R. D. Hays, R. H. Brook and R. Rausch Departments of Medicine and Health Services (R. H. Brook), Department of Psychiatry (Ft. Rausch), Department of Neurology (B. G. Vickrey, R. Rausch), University of California, C-128 Reed Building, 710 Westwood Plaza, Los Angeles, California 90024-l 789, USA; RAND, Santa Monica, California, USA (R. H. Brook, R. D. Hays).
Development of quality of life measures da now0 is time-consuming and expensive, and a number of instruments are already available for general use. Reevaluation and refinement of quality of life tools are needed to Improve the existlng pool of measures. In this study, data from a sample of 336 epilepsy patients were used to revise a measure of social adjustment and emotional status developed in the W6Os, the 127-item form Rl of the Katx Adjustment Scales (KAS-Rl). Using a comprehensive item analysis procedure, we increased the number of Items used in scoring the KAS-I?1 from 76 to 113 and rubstsntially improved the reliability of scales in both the original sample of 326 epilepsy patients and in a second administration to a ‘cross-validation’ sample of lg3 epilepsy patients. Support for the validity of the revised KAS-Rl scoring system was obtained in a known groups analysis of patients who hsd prevlourrly undergone surgery for epilepsy: mean scores were signlfkzmtly higher (p < 0.05) on 11 out of 14 scale0 in the revised KAS-Rl for patients who were completely seizure-free compared to patients who were having seizures with loss of consciousness, after adjusting for age, gender, and anticonvulsant medication use. Key words: Epilepsy, epilepsy surgery, Katz Adjustment Scales, mental health, multiirait scaling analysis, quality of life, social health.
This project was supported in part by The Robert Wood Johnson Clinical scholars Program, the Agency for Health Care Policy and Research (fellow&p suppor$ &d by BRSG SO7 RR5756 awarded bv the Biomedical Research Su~uort Grant Program, Division df Research Resources, Nationalxkitutes of Health. Opinions are those of the authors and do not necessarily reflect the views of the sponsoring institutions, RAND, or the University of CaLifomia, Los Angeles. l
To whom correspondence should be addressed.
@ 1992 Rapid Communications of Oxfird Ltd
The objectives of medical care include not only prevention of death and physical disability but also restoration and promotion of emotional well-being and integration of the individual into societal roles, or social adjustment. Increasingly, comprehensive evaluation of health-related quality of life, which includes physical, mental, and social health, has been advocated for evaluating the effectiveness of treatments for a spectrum of medical conditions. For example, the recent NIH Consensus Conference Panel on surgery for epilepsy recommended the development of reliable and valid quality of life measures for studies assessing the effectiveness of surgical intervention for epilepsy.’ Previously reported measures of psychosocial functioning in epilepsy surgery patients are limited by lack of information about reliability or validity or both. In addition, many of these measures are intervieweradministered, making them impractical for use in large-scale or multicentre studies. These measures are more extensively reviewed elsewhere.’ For patients with neurological conditions, emotional status and social functioning may be the most critical areas of impairment and among the most important outcomes to measure.3 Measures of these can be obtained either by self-report or by a trained observer. If a patient’s cognition is compromised, he or she may be unable to provide self-report information. Psychopathology or a desire to bias responses to please the health care provider or other recipient of the information may also make the patient an unreliable informant. On the other hand, use of even trained specialists to obtain these measures may introduce systematic observer bias. Because of the above problems, many investigators of chronic neurologic problems have used a Quality of Life Research . Vol 1 .I992
63
B. G. Vickrey et al. relative or friend’s report on a standardized instrument, the Katz Adjustment Scales (KAS), to measure emotional and social functioning.4-8 The KAS was originally developed in the early 1960s to measure social adjustment and behaviour of schizophrenic patients discharged from the hospital into the community .9 At the time of its development, the inventory was unique in that it included both self-report ratings and ratings from a close friend or relative who assessed the patient’s psychiatric symptoms, social behaviour, and daily activities. The KAS consists of multiple forms. In this paper we will discuss only the KAS-Rl, which has 127 items and 12 scales. The KAS-RI is the relative or friends ratings of the patient’s psychiatric symptoms and social behaviour . Although Katz and Lyerly presented data on the reliability and validity of the KAS-RI in schizophrenic patients, its validity in other patient populations has not been clearly demonstrated. Internal consistency reliability coefficients were less than 0.70 (the minimally acceptable level”) for six of 11 scales for which these coefficients were originally reported. 9 In addition, 50 of the 127 items in the KAS-Rl are not used in any of the 12 scales. Based on these observations, McDowell and Newell” recommended use of the KAS-Rl only if none of several other measures of social health are applicable. In this paper we have reevaluated item scoring procedures for the KAS-Rl in patients with epilepsy. Our goals were (1) to determine whether scale reliability and the number of items used in scoring could be increased, (2) to evaluate whether the improved psychometric properties of the revised scale scoring ‘cross-validated in a second administration of the instrument, and (3) to assess the construct validity of the revised measure. We began our analysis with the Katz and Lyerly scoring of the KAS-Rl and revised it using a combination of both clinical and statistical considerations.
Samples
To revise the KAS-RI, we used data from a sample of 328 patients who were seen between 1974 and 1989 at one west coast epilepsy centre. Over 90% of these patients had been referred to the epilepsy centre for evaluation of medicationresistant epilepsy; another 21 patients (6.4%) had 64
Quality of Life Research . Vol 1 . 1992
previously undergone surgery for epilepsy. The mean age was 30 years, ranging from 12 to 63 years of age; 92% of patients were age 18 years or older. The majority (56%) of the patients were male. As part of a comprehensive neuropsychological evaluation that all 328 patients underwent prior to surgery, patients were given the KAS-Rl and were asked to have it filled out by a relative or close friend and returned by mail. The KAS-Rls were completed by a spouse or fiance (30%), mother (41%), father (9%), sibling (5%), some other relative (1.5%), or friend (6%). For 8% of the cases, the relationship between the respondent and the patient was not reported. After revising the KAS-Rl on the above sample, we ‘cross-validated it using a mail survey of 193 adults with epilepsy who had been consecutively evaluated by an epilepsy surgery protocol at the same west coast epilepsy centre; 83% of them had undergone epilepsy surgery prior to collecting these data. The severity of epilepsy in this predominantly post-operative sample ranged from bein completely seizure-free to having daily seizures. 5 The majority (144) of the 193 patients in the second administration of the KAS-RI also participated in the first administration. The second administration occurred in 1990, at least 1 year after the first administration and either following epilepsy surgery or following a protocol evaluated without surgery. The 1990 mail survey included the KAS-Rl and questions for the patient about seizure occurrence over the preceding year and use of anticonvulsant medications. Patients were asked to give the KAS-RI to a close friend or relative, and these completed forms were mailed directly to the study investigators. (The possibility that a close friend or relative’s report would be biased by direct interaction with the patient was minimized by requesting that the KAS-Rl form be returned to the investigators in a separate envelope and by assuring the patient in the cover letter of the confidentiality of these data). A total of 224 patients were eligible for the survey in 1990, yielding a response rate from the close friend or relative of 86% (193 respondents). Forty-seven per cent (105) of the 193 epilepsy patients were male. The mean age of this cohort was 34 years, ranging from 18 to 66 years of age. The inventories were completed by a spouse or fiance (34%), mother (33%), father (6%), sibling (7%), some other relative (7%), or friend (10%); the relationship was not reported in 4% of cases. A test of the construct validity of the revised
Katz Adjustment Scalesand epilepsy measure was evaluated in the subset of 136 of the 193 participants in the 1990 mail survey who had undergone surgery for epilepsy more than 12 months prior to responding to the questionnaire. This subgroup was selected so that construct validity could be evaluated in a homogeneous (post-surgical) sample whose quality of life was less likely to be influenced by the acuteness of the surgical intervention. All studies described above were reviewed and approved by the UCLA Human Subjects Protection Committee.
Measures Katz Adjustment Scales firm Rl WAS-RI). For each of the 127 items in the KAS-Rl, the relative or friend is asked to rate the patient according to ‘how your relative or friend has looked to you during the past few weeks on these things’. For each item there are four response choices: almost never, sometimes, often, and almost always. Original psychometric analyses of the KAS-Rl by Katz and Lyerly led to the use of 76 of the 127 items to score 12 scales: General psychopathology, Suspiciousness, Anxiety, Negativism, Confusion, Belligerence, Withdrawal, Bizarreness, Hyperactivity, Helplessness, Verbal expansiveness, and Nervousness (Table 1). We evaluated and modified a 126item version of the KAS-Rl (one item from the original Withdrawal scale was accidentally omitted; Appendix A).
Analysis plan We used multitrait scaling analysis to evaluate hypothesized item clusters. Multitrait scaling is an analytic approach that applies the logic of convergent and disaiminant validity to the evaluation of multiple constructs when each is measured using the same method. l2 Multitrait scaling is based directly on scale development procedures advocated several decades ago.‘3*‘4 In multitrait scaling, items are examined with respect to how well they correlate with the sum of other items designed to measure the same construct versus how they correlate with items that measure different constructs. Ideally, an item should be most highly correlated with its hypothesized scale and display lower correlations with the other scales. Internal consistency reliability for each scale was estimated by Cronbach’s alpha.15 We evaluated item convergence and discrimination by inspecting item-scale correlations (corrected for item overlap) produced by the Multitrait Analysis Program, a microcomputer program that generates an itemscale correlation matrix. lzJ6 Use of this program to revise the KAS-Rl scoring procedure involved two types of searches. One search focused on determining whether items in hypothesized scales should be retained or moved out of their hypothesized scale. We looked for items with poor convergence with their hypothesized scale and/or lack of discrimination across scales, moving items as appropriate. The second type of search concentrated on whether items not previously scored in scales should be moved and included in a scale.
Table 1. Number of items and internal consistency reliability for the Katz Adjustment Scales Rl form (KAS-Rl) epilepsy patient samples: original scoring of scales Original KAS-Rl scales
Alpha reliability
Number of items
First administration General psychopathology Suspiciousness Anxiety Negativism Confusion Belligerence Withdrawal Bizarreness Hyperactivity Helplessness Verbal expansiveness Nervousness Miscellaneous (items not used)
24 4 ii 3 4 5 5 3 4 5 4 50
in
(n = 328)
Second administration (n = 193)
0.89 0.81 0.81 0.78 0.70 0.87 0.88 0.88 0.85 0.84 0.58 0.57 NR
0.89 0.78 0.74 0.78 0.89 0.88 0.74 0.57 0.88 0.87 0.73 0.84 NR
NR = not relevant Quality
of Life Research . Vol I .I992
65
B. G. Vickrey et al.
An item was moved into a new scale if the correlation of the item with the new scale exceeded the item’s correlations with other scales (corrected for item overlap) and the move was clinically reasonable. If none of the item-scale correlations for an item exceeded 0.30 or if the highest item-scale correlations for an item were approximately equal (i.e., the item did not discriminate between scales), we put the item in a residual, miscellaneous (‘holding’) scale. The Multitrait Analysis Program was then rerun to evaluate the revised scales, and the item-scale correlation matrix was reviewed once again. Iterations were discontinued when moving items no longer improved the multitrait scaling properties of the instrument. Each run of the program is summarized in terms of ‘scaling successes’, ‘scaling failures’, and ‘probable scaling failures’. A scaling success is defined as a correlation of an item with its hypothesized scale exceeding the correlation of this item with another scale by two standard errors, and a scaling failure occurs when the correlation of an item with its hypothesized scale falls below a correlation of the item with another scale by two standard errors. A probable scaling failure is counted if the correlation of an item with another scale lies within two standard errors of the correlation of the item with its hypothesized scale. We evaluated the improvement in item discrimination across scales by comparing results of multitrait scaling analyses based on the original scoring of the KAS-Rl with our revised scoring. Specifically, the percentages of scaling successes, scaling failures, and probable scaling failures were compared for the original and revised scoring of the instrument. To develop names for the revised scales that reflected the item content, a panel of three professionals (two epilepsy behaviour specialists and a neurologist) reviewed lists of the revised item clusters and suggested names based on the content of the lists. Using data from the second administration of the KAS-RI, we evaluated the construct validity of the revised KAS-Rl by analysing scale scores by seizure status. We postulated that LAS-R1 scores would be higher among seizure-free patients and lower among patients having severe seizures. We used data on the 136 patients in the cohort who had undergone epilepsy surgery more than 12 months prior to the study. Seizure status was determined based on self-reported seizure occurrence over the preceding 12 months. Patients were classified as having ‘full seizures’ if they had had 66
Quality of Life Research . Vol 1 .I992
one or more seizures with alteration or loss of consciousness, ‘simple partial seizures’ if they had had one or more seizures without alteration or loss of consciousness, or ‘seizure-free’ if they had had no seizures of any kind, over the 12 months prior to the second KAS-Rl administration. Scores on the KAS-Rl scales were transformed linearly to O-100 point scales for ease of comparison, with higher values on a scale indicating better functioning (less impairment). Mean scale scores were calculated for each of these three seizure classification groups. The significance of differences in revised KAS-Rl mean scores between seizure classification groups was assessed by upriori ANOVA-based t-statistics. We used relative efficiency analysis to compare different scales in the revised KAS-Rl in terms of their sensitivity to seizure occurrence.‘7 One-way ANOVA linear trend F-ratios were calculated for each of the revised KAS-Rl scales and compared across scales to evaluate their relative sensitivity to seizure classification. (Relative efficiencies themselves are determined by dividing the F-ratio of each scale by the F-ratio of a scale selected to be the ‘reference’ scale. This ‘reference’ scale has a relative efficiency equal to 1, by definition). Age, gender, and medication use of the epilepsy patients may all affect scores on the KAS-Rl. Therefore, as a sensitivity analysis, ordinary least squares regression was used to estimate the unique impact of seizure status on revised KAS-Rl scales, adjusting for these variables. Medication use was dichotomized into taking no anticonvulsant medication vs. taking one or more anticonvulsant medications at the time of the study.
Results First administration (n = 328): modification of the KAS-Rl Nine iterations of multitrait scaling were performed. At least two items were moved after each iteration. The maximum number of items were moved in the first iteration, when a total of 43 items (primarily previously unscored items) were moved to a specific scale. As a result of this process, we increased the number of items scored from 76 to 113 out of 126 items, and the number of scales increased from 12 to 14 (Tables 1 and 2). The three members of the panel named all 14 of the revised scales based on
Katz Adjustment Scalesand epilepsy their content. The revised scales are Oversensitivity/fearfulness (18 items), Social (10 items), Irritability (nine items), Dependency (15 items), Acting out (12 items), Paranoia (five items), Abnormal thought process (five items), Withdrawal (11 items), Emotional lability (six items), Nervousness (five items), Sociopathy (four items), Bizarreness (four items), Hyperactivity (four items), and Disorientation (five items) (Table 2; see Appendix A). A comparison of these two sets of item g-roupings reveaIs that the Suspiciousness, Confusion, and Belligerence scales of the original version are subsets of the Paranoia, Disorientation, and Acting out scales, respectively, of the revised version. The revised item groupings comprising the Dependency and Social scales are derived primarily from miscellaneous, unused items in the original KAS-Rl, and item dusters forming the Irritability and Emotional lability scales by the revised scoring procedure are derived primarily from items formerly scored in the General psychopathology scale. The remainder of the revised item clusters are partial combinations of items from two or more of the original scales. Comparing the item-scale correlations based on the originaI scoring of the KAS-Rl to the revised scoring, we found that scaling successes increased from 73% to 88%, scaling failures decreased from 1% to 0%, and probable failures decreased from 27% to 12%. Internal consistency reliability coefficients (Cronbach’s alpha) for the original 12
KAS-Rl scales in our sample ranged from 0.57 for the four-item Nervousness scale to 0.89 for the 24item General psychopathology scale; alpha feII below the suggested reliability cutoff for group comparisons’0 on seven of the 12 scales. In contrast, internal consistency reliability coefficients for the 14 revised KAS-Rl scales ranged from 0.66 to 0.88, and the number of scales with reliability coefficients equaIIing or exceeding 0.70 improved from five out of 12 scales to 12 out of 14 scales.
Second administration the instrument
(n = 193): testing
k?rn analysis. Multitrait scaling analysis revealed that item dis crimination improved in our test sample of 193 patients. Scaling successes increased from 61% of aII item-scale correlations when the original 12 scales were used, to 74% when the 14 revised scales were used. The percentage of scaling failures decreased from 0.7% to 0.2%, and the percentage of probable failures fell from 38% to 26%. Reliability coefficients ranged from 0.63 to 0.88, and the number of scales with reliability coefficients equaIli.ng or exceeding 0.70 increased from six out of 12 to 12 out of 14 scales (Tables 1 and 2). Seizure-free patients scored significantly higher than patients classified as having full seizures on
Table 2. Number of items and internal consistency reliability for the KAS-Rl in epilepsy patient samples: new wring of scales Revised KAS-Rl scales
Alpha reliability
Number of items
First administration
(n= 328) Oversensitivity/fearfulness Social Irritability Dependency Acting out Paranoia Abnormal thought process Withdrawal-R Emotional lability Nervousness-R Sociopathy Bizarreness-R Hyperactivity-R Disorientation Miscellaneous (items not used)
18 10 9 15 12 2 11 6 5 4 4 4 5 13
Second administration (n= 193)
0.66 0.87 0.87 0.85 0.64 0.83 0.80 0.79 K
0.87 0.87 tE 0:85 0.79 0.64 0.64 0.78 0.78 0.63 0.69 0.71 0.75 NR
0:71 0.72 0.67 0.66 NR
NR = not relevant
Qualityof tifi
Research
- Vol 1 - 1992
67
B. G. Vickrey et al. nine of 14 scales before adjustment for age, gender, and medication use, and on 11 of 14 scales after adjustment for these variables (Table 3). (Differences between mean scores on the Irritability and Paranoia scales increased and were significant at p < 0.05 after adjustment). Patients having only simple partial seizures scored in-between the other groups on 12 scales and significantly higher than patients having full seizures on seven of these scales (p < 0.05; Table 3). The relative sensitivity to seizure differences as measured by linear trend F-ratios was greatest for the Nervousness (F = 32.6), Dependency (F = 22.6), Oversensitivity/fearfulness (F = 17.0), and Withdrawal (F = 13.9) scales of the revised KAS-Rl, while the Sociopathy and Hyperactivity scales were least sensitive (linear trend F-ratios less than 1; Table 3).
Discussion The Katz Adjustment Scales form Rl &AS-Rl) is a widely used measure of social adjustment. The KAS-Rl is unique in that social behaviour and adjustment are evaluated by a close friend or relative rather than by either self-report or trained expert. However, 50 of its 127 items are not used in
scoring, and the psychometric properties of several of the 12 scales are suspect. In the present study we evaluated the scoring procedure employed for the KAS-RI in patients with epilepsy. We produced a revised scoring scheme that uses 113 of the KAS-Rl items. The revised scoring yielded substantial improvements in scale internal consistency reliability and item discrimination across scales. These improvements were observed in two different administrations of the KAS-Rl. The content validity of the revised scales was supported in that three experts were independently able to apply similar or identical, clinically meaningful labels to the 14 item clusters. Supporting the construct validity of the inventory, mean scores on 11 of its 14 scales were significantly related to seizure status after adjusting for age, gender, and medication use. Nervousness, Dependency, Oversensitivity/ fearfulness, and Withdrawal scales of the KAS-Rl were the most sensitive to differences in seizure status, with linear trend F-ratios exceeding 13. Previous studies using different measures also support associations between ongoing seizure occurrence and impairment in these areas. For example, Arntson and colleagues3 administered the Hopkins Symptom Checklist to a sample of epilepsy patients (median reported seizure frequency of two per month). Symptoms similar in
Table 3. Mean revised KAS-Rl scale scores by seizure classification Seizure classification
Revised KAS-Rl scales Seizure-free Nervousness-R Dependency Oversensitivity/fearfulness Withdrawal-R Social Abnormal thought process Disorientation Emotional lability Acting out Bizarreness-R Irritability* Paranoia* Sociopathy Hyperactivity-R
Simple partial
(n=40)
(n = 42)
88.988 87.338 92.76a 87KIa 81.428 89.13a 9O.67a 93.8ga 92.36a 96.81 a& 78.148 92.83a 97.71 a 85.21 a
82.548 84.5Qa 91 .Ol a 83.70a 78.658 87.62a 86.968sb 93.65a 90.97aJJ 98.21 a 75.138 90.16a 98.41 a 81 .75a
Linear trend F-ratio Full seizures (n=!54)
68.92b 74.77b 84.9gb 76.74b 70.36b 79.80” 82.25b 88.1 7b 87.5ab 93.67b 71 .Oaa 88.52a 96.91 a 84.10a
32.58 22.60 16.98 1394 8.54 8.31 6.87 6.44 5.16 3.81 3.48 2.17 0.70 0.06
Note: Higher scores indicate better functioning (O-100 point scales). Seizure classification is based on self-reported seizure occurrence over the 12 months preceding completion of the KAS-Rl . Study patients who had undergone surgery more than 12 months previously were included. One-way ANOVA linear trend F-ratios of KAS-Rl scales by seizure classification are shown. Degrees of freedom are (1,134). a,bMeans within a row with different superscripts differ significantly (p c 0.05) from one another as determined by ANOVA-based t-statistics. *After adjustment for age, gender, and anticonvulsant medication use, the differences in predicted mean scores between seizure-free and full seizure patients, based on ordinary least squares regression, were also significant (p < 0.05) for these scales. 68
Quality of Life Research . Vol 1 .I992
Katz Adjustment Scalesand epilepsy content to the items on the KAS-RI Nervousness scale (“I feel nervous or shaky inside”, “I notice my hands trembling”), Oversensitivity/fearfulness scale (“I cry easily or feel like crying”, “I feel fearful or afraid’, “I feel downhearted or blue”, “I have thoughts of possibly ending my life”), Dependency scale (“I feel low in energy or slowed down”, “I have headaches or head pains”), and Withdrawal scale (“I feel bored or have little interest in doing things”) were endorsed from 1.5 to 15 times more frequently by their sample of patients with epilepsy than by a general population sample. In another study, neuropsychologists rated a group of 31 epilepsy patients on a scale of dependency before and after surgery for epilepsy. One year post-operative scores differed significantly between seizure-free patients and patients having little or no improvement in seizure control after surgery. l8 Not surprisingly, scores on the Sociopathy scale were not related to seizure occurrence. Mean Sociopathy scale scores exceeded 96 out of 100 possible points, suggesting that the content of this scale is not explained by epilepsy. Indeed, the two epilepsy behaviour specialists who helped identify content areas for the revised item clusters informally (and blind to the results of the analysis) rated the Sociopathy scale as being least relevant to these epilepsy patients. Our findings are also consistent with previously documented relationships between post-operative seizure occurrence in epilepsy patients and mental and social health as measured by the Epilepsy Surgery Inventory (ESI-55), a 55-item measure tapping 11 dimensions of health-related quality of life. Similar to what was found for many scales of the KAS-Rl, mean scores as a function of seizure classification on the ESI-55 emotional well-being scale and the social function scale were significantly higher among seizure-free patients compared to patients having seizures with loss of consciousness. 2 The need for proxy respondents and validity of their responses in assessing quality of life has only been preliminarily addressed in the literature. Certainly, proxy respondents are essential when patients are unable to respond due to disability and very useful when patients’ responses are potentially unreliable. In developing the Katz Adjustment Scales, Katz and Lyerly justified the supplementation of self-reported psychiatric symptoms and social behaviour with measures based on a friend or relative’s report by pointing out that use of the proxy protects against the
limitation of relying solely on the patient for information on social functioning.9 Further evaluation of the reliability and validity of our revised scoring of the KAS-Rl is needed. Of particular importance are studies examining the sensitivity of these scales to known differences between groups of patients at a point in time as well as their responsiveness to change over time. l9 In addition, the generalizability of these results to other diseases needs to be evaluated. The revised scales described here are recommended in the interim if an assessment of the social and emotional functioning of an epilepsy patient from a relative or friend’s perspective is desired.
Acknowledgements The authors are grateful to several people who helped with implementing this project, including Jerome Engel, Jr, Jason Graber, Maria Melendez, Juan Vasconez, and Cathy Ary. We thank Lucille Spiff for help in preparing the manuscript. References 1. National Institutes of Health Consensus Conference, Surgery for Epilepsy. JAMA 1990; 264: 729-733. 2. Vickrey BG, Hays RD, Graber J, et al. A healthrelated quality of Iife instrument for patients evaluated for epilepsy surgery. Med Cure 1992 (in press). 3. Amtson P, Droge D, Norton R, et al. The perceived psychosocial consequences of having epilepsy. In: Whitman S, Hermann BP, eds. Psyc/~yofltology in Epilepsy: So&Z Dirtzensions. New York: Oxford University Press, 1986: 143-161. 4. Rausch R. Psychological evaluation. In: Engel J, ed. Surgical Treatment offhe Epikpsies. New York: Raven Press, 1987: 181-195. 5. Hinheldez NS, Corrigan JD. The structure of headinjured patients’ neurobehavioural complaints: A preliminary study. Brain Injury 1990; 4(2): 115-133. 6. Fordyce DJ, Roueche JR, Prigatano GP. Enhanced emotional reactions in chronic head trauma patients. J Neural Neurosurg Psychiatry 1983; 46: 620-624. 7. Rao SM, Leo GJ, EIIington MS, et al. Cognitive dysfunction in multiple sclerosis. II. Impact on employment and social functioning. Neurology 1991; 41: 692-6%. 8. Dashieff, Richard. Personal communication, April 1991. 9. Katz MM, Lyerly SB. Methods for measuring adjustment and social behavior in the community: I. Rationale, description, discriminative validity and scale development. PsycholRep 1963; 13: 503-535. 10. Nurmally J. Psychometric Theoy, 2nd edition. New York: McGraw-I-W, 1978. Qualityof Lij+Research. Vol1 f 2992 69
B. G. Vicky
et al.
11. McDowell I, Newell C. The Katz Adjustment scales. In: Measuring Health: A Guide to Rating Scalesand Questionnaires. New York: Oxford University Press, 1987: 169-172. 12. Hays RD, Hayashi T. Beyond internal consistency reliability: Rationale and user’s guide for the Multitrait Analysis Program on the microcomputer. Behav ResMethods lnstrum Comput 1990; 22: 167-175. 13. Jackson DN. Multimethod factor analysis in the evaluation of convergent and discriminant validity. PsycholBull 1969; 72: 30-49. 14. Loevinger J, Gleser G, DuBois PH. Maximizing the discriminating power of a multiple-score test. Psychometrika 1953; 18: 309-317. 15. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika1951; 16: 297-334.
16. Hays RD, Hayashi T, Carson S, et al. User’sGuidefir the Multitrait Analysis Program (MAP). N-27%~RC. RAND, Santa Monica, CA, 1988. 17. Liang MI-I, Larson MG, Cullen KE, et al. Comparative measurement efficiency and sensitivity of five health status instruments for arthritis research. Arthritis Rheum 1985; 28: 542-547. 18. Rausch R, Crandall PH. Psychological status related to surgical control of temporal lobe seizures. Epikpsia 1982; 23: 191-202. 19. Guyatt GH, Deyo RA, Charlson M, et al. Responsiveness and validity in health status measurement: A clarification. J Clin Epidemiol 1989; 42(S): 403-408.
(Received 10 October 2991; accepted in revised form 30 October 1991)
Appendix A. Revised scales in the KAS-Rl Nervousness 16. Tries too hard 20. Gets nervous easily 21. Jittery 22. Worries or frets 63. Hands tremble Dependency 9. Acts as if he/she does not have much energy 10. Looks worn out 17. Needs to do things very slowly to do them right 41. Complains of headaches, stomach trouble, other physical ailments 72. Needs a lot of attention 73. Behaviour is childish 74. Acts helpless 75. Is independent (Note: item is reverse-coded in this scale) 76. Clumsy; keeps bumping into things or dropping things 61. Gets into peculiar positions 62. Makes peculiar movements 90. Acts as if he’s/she’s confused about things: in a daze 92. Acts as if he/she can’t concentrate on one thing 93. Acts as if he/she can’t make decisions 95. Hard to understand his/her words Oversensitivity/fearfulness 2. Gets very self-critical, starts to blame himself/herself for things 3. Cries easily 4. Feels lonely 11. Feelings get hurt easily 12. Feels that people don’t care about him/her 15. Gets very sad, blue 18. Has strange fears 19. Afraid something terrible is going to happen 23. Gets sudden fright for no reason 24. Has bad dreams 27. Attempts suicide 109. Talks as if he/she committed the worst sin 111. Talks about people or things he’s/she’s very afraid of 115. Says he/she is afraid that he/she will not be able to control himself/herself 116. Talks about strange things that are going on inside his/her body 117. Says how bad or useless he/she is 123. Says that something terrible is going to happen 125. Talks about suicide
70
Quality
of
Life Research . Vol1 ’ 1992
Katz Adjustment Scalesand epilepsy Appendix continued Withdrawal 5. Acts as if he/she has no interest in things 8. Just sits 67. Stays away from people 69. Shy 70. Quiet 71. Prefers to be alone 60. Very slow to react 84. Will stay in one position for a long period 96. Speaks so low you cannot hear him/her 102. Speaks very slowly 103. Acts as if he/she wants to speak but can’t Abnormal thought process 91. Acts as if he/she can’t get certain thoughts out of his/her mind 94. Talks without making sense 104. Keeps repeating the same idea 105. Keeps changing from one subject to another 119. Says the same thing over and over again Social 39. Generous 46. Is co-operative 49. Answers when talked to 54. Friendly 57. Pleasant 56. Gets along well with people 62. Is dependable 63. Is responsible 65. Obedient 66. Shows good judgement Emotional lability 30. Acts as if he/she has no control over his/her emotions 31. Laughs or cries at strange times 32. Has mood changes without reason 34. Gets very excited for no reason 35. Gets very happy for no reason 126. Talks about strange sexual ideas Bizarreness 13. Does the same thing over and over again without reason 25. Acts as if he/she sees people or things that aren’t there 26. Does strange things without reason 124. Believes in strange things Disorientation 65. Loses track of day, month, or year 66. Forgets his/her address or other places he/she knows well 87. Remembers the names of people he/she knows well (Note: item is reverse-coded in this scale) 88. Acts as if he/she doesn’t know where he/she is 89. Remembers important things (Note: item is reverse-coded in this scale) Acting out 28. Gets angry and breaks things 33. Has temper tantrums 36. Acts as if he/she doesn’t care about other people’s feelings 37. Thinks only of himself/herself 45. Gets into fights with people 47. Does the opposite of what he/she is asked 50. Curses at people 51. Deliberately upsets routine
Quality
of Lifi Research . Vol 1 .I992
71
B. G. Vichy
et al.
Appendix continued 99. Speaks very loudly 100. Shouts or yells for no reason 112. Threatens to injure certain people 113. Threatens to tell people off Irritability 42. Bossy 44. Argues 46. Stubborn 52. Resentful 55. Gets annoyed easily 56. Critical of other people 110. Talks about how angry he/she is at certain people 120. Complains about people and things in general 127. Gives advice without being asked Paranoia 40. Thinks people are talking about him/her 43. Acts as if he/she is suspicious of people 107. Says that people are talking about him/her 106. Says that people are trying to make him/her do or think things he/she doesn’t want to 122. Says or acts as if people are after him/her Socioparhy 59. Lies 60. Gets into trouble with the law 61. Gets drunk 66. Takes drugs other than recommended by hospital or clinic Hyperactivity 6. Is restless 7. Has periods when he/she can’t stop moving or doing something 77. Moves about in a hurried way 101. Speaks very fast Note: Item numbers refer to their position in the measure that was administered to our patients and are identical to the numbering of items listed in Katz and Lyerly (1963), pages 512-513.9 The item omitted from the original 127 item KAS-Rl was #76: “Moves about very slowly”.
72
Quality of Life Research . Vol 1 .1992
