A Comparison of Metacognitive Ratings of Persons with Traumatic Brain Injury and Their Family Members Janet Kay Anderson·Parente, PhD Psychology Department, Towson State University, Towson, MD Clients with traumatic brain injury (TBI) who were referred for neuropsychological testing were asked to provide self-ratings of their intellectual, social, memory, and academic skills. Their family members and relatives were asked to rate the clients on the same scales. The ratings were correlated with the client's neuropsychological test scores. The results indicated that clients tend to underestimate their cognitive deficits compared to their relatives or family members. The relatives' ratings correlated with a wider range of neuropsychological measures. Different sets of ratings for the relatives and clients correlated with the test scores. Keywords: Metamemory; brain injury; neuropsychological; self-ratings
Metamemory questionnaires are designed to subjectively assess a person's memory. They differ from conventional memory evaluations because there is no attempt to assess performance. Therefore, these questionnaires do not actually measure memory, but, they are instead, a subjective selfevaluation, or are based on the observations of another person.! At one time it was thought that these short, simple questionnaires might replace actual memory testing. Consequently, by 1989, over 30 metamemory surveys had been developed. l Unfortunately, researchers almost universally found that metamemory questionnaires were not valid, although they were extremely reliable. 2-4 That is to say, the questionnaires produced consistent ratings, but the results did not necessarily correlate with memory performance measures. This result was found when they were used to evaluate elderly populations, college students, and in a few cases, persons with TBI. Several hypotheses have been put forth to explain why a TBI survivor, or people in general, would not be capable of accurately rating his or
her memory. According to Sunderland et al., 5 a metamemory questionnaire is itself a test of memory, and people with poor memories will literally not be able to remember how well or poorly they function. Herrmann! labeled this phenomena the "memory introspection paradox." He found that persons with superior memories had more accurate self-ratings. He also suggested that a person's perception of his or her memory functioning was based not only on that person's firsthand experience with memory tasks, but also on secondhand reports from others, although these reports may be biased by their relationship with the person. Small 6 suggested that TBI survivors suffered from the dissolution of cogiIitive skills. Therefore, since the person's cognition was no longer functioning reliably, it was possible that their selfratings would be equally unreliable. There was also the possibility of "organic denial," a condition whereby the person simply denied any deficits related to cognitive losses. Cicerone7 suggested that denial might be a protective response in the face of gradually increasing awareness of the severity of a disability. Clearly, all of these factors NeuroRehabilI994; 4(3):168-173 Copyright © 1994 by Butterworth-Heinemann
Metacognitive Ratings After Brain Trauma
could combine to cloud the TBI survivor's selfawareness and attention to personal performance of any kind. Poon 8 found that in an elderly population, depression definitely lowered a person's estimation of their memory on self-report questionnaires, while Dementia raised their estimation of themselves. Miller9 suggested that premorbid paranoia combined with post-traumatic factors, such as a loss of cognitive flexibility, might greatly increase paranoia and denial in every situation following TBI. The correlation between metamemory scales and measured memory performance may therefore vary as a function of the level or type of personality pathology the client experiences after a TB I. Gentry and Herrmann 10 pointed out that people's status in a social context may be determined by how well they remember things. These authors proposed the notion of "memory contrivances" to describe the deliberate distortion of self-ratings and performance because of a desire to make a good impression. Finally, Poon 8 points out that many factors may influence memory functioning, such as: environmental stress; pre- and post-morbid personality differences, and different coping styles; overall health, intelligence, educational level, socioeconomic status and hormonal changes. Herrmann (this issue) describes a similar set of factors in his Multimodal Model of Cognition. The above discussion indicates that myriad factors may bias the TBI person's self-ratings of memory. However, when the same metamemory questionnaires are given to a relative of a TBI survivor who is then asked to rate that person, significant correlations are found between the relatives ratings and objective testing (Sunderland, et al.,5 Chaffin, et al.,ll Broadbent et al. 3). Unfortunately, most of the research that correlates self-ratings to performance test results is done with elderly populations and college students. Although a handful of studies with headinjured clients have been reported, most have assessed memory alone. There are none that have assessed the head-injured person's ability to rate a variety of cognitive and behavioral skills relative to objective test scores. There have been no studies
169
that have evaluated accuracy of a TBI survivor's self-ratings for a variety of cognitive skills such as intelligence, social functioning, academic ability and so forth, relative to family members' selfratings. 'The following research addressed this issue.
METHOD Subjects and Instruments Twenty-two traumatically brain-injured subjects participated. The clients were referred for neuropsychological evaluation and the testing was requested as a precursor for planning a treatment program. The subjects had a variety of different head injuries, however, each was classified as "severely limited" according to State of Maryland Vocational Rehabilitation Services guidelines. Specifically, one-half of the subjects had both frontal and temporal damage secondary to automobile and motorcycle accidents. Eight of the subjects had experienced falls resulting in global brain trauma with no obvious locus. The remainder of the subjects had a mixture of injuries including gunshot wounds, sports related traumas, and assaults. The subjects ranged in age from 17 to 51. Fifty-five percent were females and forty-five percent were males. The group ranged from 6 months to 7 years post injury. The length of coma ranged from 2 weeks to 14 months. The subjects were tested between 6 months and 7 years post injury. Of the family members who participated, 10 were mothers, and the remainder included 3 fathers, 1 daughter, 2 husbands, 1 son, 2 spouses, 1 step-parent, 1 teacher, and 1 wife. Each client received the same neuropsychological evaluation, including the Wechsler Adult Intelligence Scale,12 Revised Wechsler Memory Scale,13 Wide Range Achievement Test, 14 and the Symptom Checklist (90-R).15 Although the entire battery included many more tests, these were commonly administered to all clients and therefore, used here. As part of the battery, the client and the family member was asked to fill out a rating evaluation of cognitive limitation. This evaluation is called the CAMS survey.16 The name CAMS derives from the fact that the survey was intended to provide
170
NEUROREHABILITATION / JULY 1994
compensations, accommodations, modifications, and strategies for persons with learning disabilities. It was originally developed as part of a learning disability project at the University of Alabama in 1987. Each item on the survey is indexed with several suggestions that could be used to obviate or lessen the deficit described in the survey statement. CAMS was used as part of the neuropsychological evaluation in order to provide suggestions for rehabilitation. As described here, its main purpose was to provide a wide ranging survey of cognitive and behavioral dimensions. CAMS presents 100 survey items, divided unequally into the following categories derived from factor analysis of the items: (1) attention-deficit hyperactivity disorder (ATTN), (2) reasoning (REAS) , (3) processing (PROC), (4) memory (MEMO), (5) interpersonal skills/emotional maturity (ITPR), (6) coordination/motor functions (COOR), (7) reading (READ), (8) writing/spelling (WRIT), (9) math calculation/application (ARIT). The survey required the participant to provide a rating on a scale where 0 = never, 1 = infrequently, 2 = occasionally, 3 = quite often, 4 = all the time. The questions were worded so that each could be answered by the client, or by a family member who was rating the client.
side the vector of CAMS subscale scores. These scores included: (1) verbal intelligence quotient (IQ), (2) performance IQ, (3) full scale IQ, (4) verbal memory, (5) visual memory, (6) attention/ concentration, (7) reading grade level, (8) math grade level, and (9, 10, 11) three measures of global personality dysfunction from the SCL90-R.
Differences Between Relatives' and Clients CAMS Ratings The first analysis examined differences in rating levels for the family members versus the clients. Previous research indicated that family members usually rate clients as more severely impaired relative to the clients ratings. Figure 1 compares the CAMS ratings for the two groups. Although the general pattern of ratings is the same for both groups, the relatives' ratings indicate more severe disability across all the rating subscales. The fact that nine out of nine subscales were rated as more severe by the family members than by the clients was significant by Binomial Test (p < .01).
1.6
RESULTS
1.4 OJ
Each of the CAMS surveys were reduced to subscale scores by averaging the ratings for the various items within a subscale. In order to assess the internal consistency of the sub scale items, itemscale correlations were computed for each item within the various subscales. Only those items with significant item-scale correlations were used to construct the subscales. The procedure resulted in a loss of an average of one item from each scale. However, the procedure ensured that the subscales were internally consistent. In cases where there was missing data, the averages were computed excluding the missing data. This initial data reduction yielded nine sub scale scores, one for each of the dimensions listed above. Each subject's neuropsychological test scores were computed and added to the data base along-
c:
~ 1.2
a:
CD
~ CD
1
CD
0.8
~
iii 0
UI
.!l ::I
rJ) rJ)
0.6
:::i!: 0.4
«
u
0.2
o '2 ,() ~ ~ 4 ("\_ -qA -5>«,_ ~A ~cS' 7$1"0 ~4,0~/)- ~ -00-5>"'>7;- '"Va ~
Z
~I CAMS SUBSCALE
Figure 2. Correlation between ratings and
neuropsychological test scores.
171
DISCUSSION There are three major findings in these data. First, the results show that clients tend to underestimate their cognitive deficits, as compared to relatives or family members. Second, relatives' ratings are correlated with a wider range of neuropsychological measures. Third, different sets of ratings for the relatives and clients are correlated with the neuropsychological measures. With respect to the first issue, these data are generally consistent with earlier studies that showed that relatives' ratings of clients typically indicated a greater severity of impairment. These data indicate that although the general pattern of ratings is the same for both groups, the tendency for the clients to underestimate their deficits, compared to the relatives' perception, extends over a variety of different behaviors, not just memory ratings. It is therefore reasonable to suggest that therapists should interview family members and relatives in order to gain an accurate assessment of the severity of the clients' deficits. Family members provided ratings that significandy correlated with most of the neuropsychological test scores. This finding has not been generally reported in the literature. Although the clients' ratings produced only about one-half as many significant correlations with the neuropsychological test scores, the number of significant correlations was larger than expected from the earlier literature. This finding again suggests that the relatives' ratings are perhaps the more useful of the two sets of measures for therapists. The findings also suggests that clients' self-ratings are more accurate than has been assumed. A third finding was that the pattern of correlations between the CAMS subscales and the neuropsychological measures for the relatives differed markedly from those obtained with the clients. The relatives' ratings of abstract qualities such as reasoning, attention, processing, and memory were highly correlated with the test scores, whereas the clients' ratings of these same qualities had no relationship with the test scores. Clients' self-ratings of qualities such as writing skills and arithmetic were highly correlated with the neuropsychological measures. The difference suggests that clients focus not
172
NEUROREHABILITATION / JULY 1994
only on a narrower range of behaviors and skills but that they are more attune to the concrete aspects of their behavior. Family members perceive a wider range of behaviors and are attune to the clients' more abstract cognitive functions. What can the clinician glean from all of these findings? First, the data lead to the conclusion that neuropsychologists and cognitive rehabilitation therapists should use a multi-factor self-report instrument such as the CAMS as part of a comprehensive test battery. The author has found that clients and relatives appreciate the type of questioning that a well-constructed survey provides. It provides a chance to communicate problems in a way that is impossible to do in an unstructured interview or with standardized testing. In addition, the survey allows the clinician to assess a wide variety of different types of behaviors, many of which are impossible to measure with performance instruments. Most important, the survey assesses the nuances of daily behavior that a oneshot test battery cannot evaluate. The therapist should be aware that clients are likely to underestimate their cognitive deficits. The difference between the clients' and relatives' ratings may therefore be a useful index of underestimation and lack of acceptance of disability. The problem may be due to denial, poor memory, or some other form oflimited awareness. Regardless of the cause, at some point, the therapist will have to address this issue with the client. Clearly, little
progress can be made until the client realizes and accepts his or her deficits. Clients and relatives may have a difficult time filling out many survey instruments. In the author's experience, the survey must be easy to read (reading level of approximately 5th to 6th grade), and simple to fill out. Ratings of 0 = "no problem," through 4 = "severe problem," worked reasonably well with these clients. Ranking scales or more complex rating systems should be avoided. The scale must also be constructed so that the examiner can read the items to clients who cannot read or write. It is important to use a scale that measures a variety of different functional components and to avoid single attribute instruments. Although there was no formal analysis of the relationship between the number of raters and the accuracy of predictions of neuropsychological test scores, the author's anecdotal experience has been that the more relatives who rate the same client, the more accurate the assessment of everyday performance. Therapists are therefore advised to distribute rating surveys to as many different observers as possible. Not only the average rating of the client, but also the variability of the ratings provides important information about daily functioning. Those items that elicit discrepant ratings identifY behaviors and skills that are likely to be quite variable or volatile.
REFERENCES 1. Herrmann D. Self-perceptions of memory performance. In Erlbaum S, ed., Self-directedness. Sterling, VA: World Comprehension Services, 1989. 2. Bennett-Levy J, Powell GE. The subjective memory questionnaire (SMQ): An investigation into the self-reporting of "real-life" memory skills. Brit J Soc Clin Psych 1980; 19: 177-188. 3. Broadbent DE, Cooper BE, Fitzgerald P, Parkes KR. The cognitive failures questionnaire (CFQ) and its correlates. Brit J Soc Clin Psych 1982;21:1-16. 4. Herrmann, DJ. Questionnaires about memory. In Harris J, Morris P, eds., Everyday merrwry. London: Academic Press, 1983.
5. Sunderland A, Harris JE, Baddely A. Do labo-
6. 7. 8.
9. lO.
ratory tests predict everyday memory? A neuropsychological study. ] of Verb Learn and Verbal Beh 1983;22:341-357. Small L. Neuropsychodiagnosis in psychotherapy (revised). New York: Brunner/Mazel, 1980. Cicerone Ko. Psychotherapeutic interventions with traumatically brain-injured patients. Rehab Psych 1989;34: lO5-114. Poon L, ed. Handbook for clinical merrwry assessment of older adults. Washington DC: Am Psych Assn, 1986. Miller L. Psychotherapy of the brain-injured patient. New York: WW Norton & Co., 1993. Gentry M, Herrmann DJ. Memory contrivances
Metacognitive Ratings Mter Brain Trauma
in everyday life. Pers and Soc Psych Bull 1990; 16(2):241-253. 11. Chaffin, RJS, Deffenbacher, K, Herrmann DJ: Awareness and lack of awareness of memory function between spouses. Unpublished manuscript, Hamilton College, New York, 1980. 12. Wechsler Adult Intelligence Scale-Revised. The Psychological Corporation, New York: Harcourt Brace Jovanovitch Inc., 1981.
173
13. Wechsler Memory Scale-Revised. The Psychological Corporation, New York: Harcourt Brace Jovanovitch Inc., 1987. 14. Wide Range Achievement Test-Revised. Jastak Assessment Systems, Wilmington: DE, 1984. 15. Derogatis LR: The SCL90-R. Clinical Psychometric Research: Baltimore, 1975. 16. CAMS survey. University of Alabama. Unpublished manuscript, 1987.
Metamemory questionnaires are designed to subjectively assess a person's memory. They differ from conventional memory evaluations because there is no attempt to assess performance. Therefore, these questionnaires do not actually measure memory, but, they are instead, a subjective selfevaluation, or are based on the observations of another person.! At one time it was thought that these short, simple questionnaires might replace actual memory testing. Consequently, by 1989, over 30 metamemory surveys had been developed. l Unfortunately, researchers almost universally found that metamemory questionnaires were not valid, although they were extremely reliable. 2-4 That is to say, the questionnaires produced consistent ratings, but the results did not necessarily correlate with memory performance measures. This result was found when they were used to evaluate elderly populations, college students, and in a few cases, persons with TBI. Several hypotheses have been put forth to explain why a TBI survivor, or people in general, would not be capable of accurately rating his or
her memory. According to Sunderland et al., 5 a metamemory questionnaire is itself a test of memory, and people with poor memories will literally not be able to remember how well or poorly they function. Herrmann! labeled this phenomena the "memory introspection paradox." He found that persons with superior memories had more accurate self-ratings. He also suggested that a person's perception of his or her memory functioning was based not only on that person's firsthand experience with memory tasks, but also on secondhand reports from others, although these reports may be biased by their relationship with the person. Small 6 suggested that TBI survivors suffered from the dissolution of cogiIitive skills. Therefore, since the person's cognition was no longer functioning reliably, it was possible that their selfratings would be equally unreliable. There was also the possibility of "organic denial," a condition whereby the person simply denied any deficits related to cognitive losses. Cicerone7 suggested that denial might be a protective response in the face of gradually increasing awareness of the severity of a disability. Clearly, all of these factors NeuroRehabilI994; 4(3):168-173 Copyright © 1994 by Butterworth-Heinemann
Metacognitive Ratings After Brain Trauma
could combine to cloud the TBI survivor's selfawareness and attention to personal performance of any kind. Poon 8 found that in an elderly population, depression definitely lowered a person's estimation of their memory on self-report questionnaires, while Dementia raised their estimation of themselves. Miller9 suggested that premorbid paranoia combined with post-traumatic factors, such as a loss of cognitive flexibility, might greatly increase paranoia and denial in every situation following TBI. The correlation between metamemory scales and measured memory performance may therefore vary as a function of the level or type of personality pathology the client experiences after a TB I. Gentry and Herrmann 10 pointed out that people's status in a social context may be determined by how well they remember things. These authors proposed the notion of "memory contrivances" to describe the deliberate distortion of self-ratings and performance because of a desire to make a good impression. Finally, Poon 8 points out that many factors may influence memory functioning, such as: environmental stress; pre- and post-morbid personality differences, and different coping styles; overall health, intelligence, educational level, socioeconomic status and hormonal changes. Herrmann (this issue) describes a similar set of factors in his Multimodal Model of Cognition. The above discussion indicates that myriad factors may bias the TBI person's self-ratings of memory. However, when the same metamemory questionnaires are given to a relative of a TBI survivor who is then asked to rate that person, significant correlations are found between the relatives ratings and objective testing (Sunderland, et al.,5 Chaffin, et al.,ll Broadbent et al. 3). Unfortunately, most of the research that correlates self-ratings to performance test results is done with elderly populations and college students. Although a handful of studies with headinjured clients have been reported, most have assessed memory alone. There are none that have assessed the head-injured person's ability to rate a variety of cognitive and behavioral skills relative to objective test scores. There have been no studies
169
that have evaluated accuracy of a TBI survivor's self-ratings for a variety of cognitive skills such as intelligence, social functioning, academic ability and so forth, relative to family members' selfratings. 'The following research addressed this issue.
METHOD Subjects and Instruments Twenty-two traumatically brain-injured subjects participated. The clients were referred for neuropsychological evaluation and the testing was requested as a precursor for planning a treatment program. The subjects had a variety of different head injuries, however, each was classified as "severely limited" according to State of Maryland Vocational Rehabilitation Services guidelines. Specifically, one-half of the subjects had both frontal and temporal damage secondary to automobile and motorcycle accidents. Eight of the subjects had experienced falls resulting in global brain trauma with no obvious locus. The remainder of the subjects had a mixture of injuries including gunshot wounds, sports related traumas, and assaults. The subjects ranged in age from 17 to 51. Fifty-five percent were females and forty-five percent were males. The group ranged from 6 months to 7 years post injury. The length of coma ranged from 2 weeks to 14 months. The subjects were tested between 6 months and 7 years post injury. Of the family members who participated, 10 were mothers, and the remainder included 3 fathers, 1 daughter, 2 husbands, 1 son, 2 spouses, 1 step-parent, 1 teacher, and 1 wife. Each client received the same neuropsychological evaluation, including the Wechsler Adult Intelligence Scale,12 Revised Wechsler Memory Scale,13 Wide Range Achievement Test, 14 and the Symptom Checklist (90-R).15 Although the entire battery included many more tests, these were commonly administered to all clients and therefore, used here. As part of the battery, the client and the family member was asked to fill out a rating evaluation of cognitive limitation. This evaluation is called the CAMS survey.16 The name CAMS derives from the fact that the survey was intended to provide
170
NEUROREHABILITATION / JULY 1994
compensations, accommodations, modifications, and strategies for persons with learning disabilities. It was originally developed as part of a learning disability project at the University of Alabama in 1987. Each item on the survey is indexed with several suggestions that could be used to obviate or lessen the deficit described in the survey statement. CAMS was used as part of the neuropsychological evaluation in order to provide suggestions for rehabilitation. As described here, its main purpose was to provide a wide ranging survey of cognitive and behavioral dimensions. CAMS presents 100 survey items, divided unequally into the following categories derived from factor analysis of the items: (1) attention-deficit hyperactivity disorder (ATTN), (2) reasoning (REAS) , (3) processing (PROC), (4) memory (MEMO), (5) interpersonal skills/emotional maturity (ITPR), (6) coordination/motor functions (COOR), (7) reading (READ), (8) writing/spelling (WRIT), (9) math calculation/application (ARIT). The survey required the participant to provide a rating on a scale where 0 = never, 1 = infrequently, 2 = occasionally, 3 = quite often, 4 = all the time. The questions were worded so that each could be answered by the client, or by a family member who was rating the client.
side the vector of CAMS subscale scores. These scores included: (1) verbal intelligence quotient (IQ), (2) performance IQ, (3) full scale IQ, (4) verbal memory, (5) visual memory, (6) attention/ concentration, (7) reading grade level, (8) math grade level, and (9, 10, 11) three measures of global personality dysfunction from the SCL90-R.
Differences Between Relatives' and Clients CAMS Ratings The first analysis examined differences in rating levels for the family members versus the clients. Previous research indicated that family members usually rate clients as more severely impaired relative to the clients ratings. Figure 1 compares the CAMS ratings for the two groups. Although the general pattern of ratings is the same for both groups, the relatives' ratings indicate more severe disability across all the rating subscales. The fact that nine out of nine subscales were rated as more severe by the family members than by the clients was significant by Binomial Test (p < .01).
1.6
RESULTS
1.4 OJ
Each of the CAMS surveys were reduced to subscale scores by averaging the ratings for the various items within a subscale. In order to assess the internal consistency of the sub scale items, itemscale correlations were computed for each item within the various subscales. Only those items with significant item-scale correlations were used to construct the subscales. The procedure resulted in a loss of an average of one item from each scale. However, the procedure ensured that the subscales were internally consistent. In cases where there was missing data, the averages were computed excluding the missing data. This initial data reduction yielded nine sub scale scores, one for each of the dimensions listed above. Each subject's neuropsychological test scores were computed and added to the data base along-
c:
~ 1.2
a:
CD
~ CD
1
CD
0.8
~
iii 0
UI
.!l ::I
rJ) rJ)
0.6
:::i!: 0.4
«
u
0.2
o '2 ,() ~ ~ 4 ("\_ -qA -5>«,_ ~A ~cS' 7$1"0 ~4,0~/)- ~ -00-5>"'>7;- '"Va ~
Z
~I CAMS SUBSCALE
Figure 2. Correlation between ratings and
neuropsychological test scores.
171
DISCUSSION There are three major findings in these data. First, the results show that clients tend to underestimate their cognitive deficits, as compared to relatives or family members. Second, relatives' ratings are correlated with a wider range of neuropsychological measures. Third, different sets of ratings for the relatives and clients are correlated with the neuropsychological measures. With respect to the first issue, these data are generally consistent with earlier studies that showed that relatives' ratings of clients typically indicated a greater severity of impairment. These data indicate that although the general pattern of ratings is the same for both groups, the tendency for the clients to underestimate their deficits, compared to the relatives' perception, extends over a variety of different behaviors, not just memory ratings. It is therefore reasonable to suggest that therapists should interview family members and relatives in order to gain an accurate assessment of the severity of the clients' deficits. Family members provided ratings that significandy correlated with most of the neuropsychological test scores. This finding has not been generally reported in the literature. Although the clients' ratings produced only about one-half as many significant correlations with the neuropsychological test scores, the number of significant correlations was larger than expected from the earlier literature. This finding again suggests that the relatives' ratings are perhaps the more useful of the two sets of measures for therapists. The findings also suggests that clients' self-ratings are more accurate than has been assumed. A third finding was that the pattern of correlations between the CAMS subscales and the neuropsychological measures for the relatives differed markedly from those obtained with the clients. The relatives' ratings of abstract qualities such as reasoning, attention, processing, and memory were highly correlated with the test scores, whereas the clients' ratings of these same qualities had no relationship with the test scores. Clients' self-ratings of qualities such as writing skills and arithmetic were highly correlated with the neuropsychological measures. The difference suggests that clients focus not
172
NEUROREHABILITATION / JULY 1994
only on a narrower range of behaviors and skills but that they are more attune to the concrete aspects of their behavior. Family members perceive a wider range of behaviors and are attune to the clients' more abstract cognitive functions. What can the clinician glean from all of these findings? First, the data lead to the conclusion that neuropsychologists and cognitive rehabilitation therapists should use a multi-factor self-report instrument such as the CAMS as part of a comprehensive test battery. The author has found that clients and relatives appreciate the type of questioning that a well-constructed survey provides. It provides a chance to communicate problems in a way that is impossible to do in an unstructured interview or with standardized testing. In addition, the survey allows the clinician to assess a wide variety of different types of behaviors, many of which are impossible to measure with performance instruments. Most important, the survey assesses the nuances of daily behavior that a oneshot test battery cannot evaluate. The therapist should be aware that clients are likely to underestimate their cognitive deficits. The difference between the clients' and relatives' ratings may therefore be a useful index of underestimation and lack of acceptance of disability. The problem may be due to denial, poor memory, or some other form oflimited awareness. Regardless of the cause, at some point, the therapist will have to address this issue with the client. Clearly, little
progress can be made until the client realizes and accepts his or her deficits. Clients and relatives may have a difficult time filling out many survey instruments. In the author's experience, the survey must be easy to read (reading level of approximately 5th to 6th grade), and simple to fill out. Ratings of 0 = "no problem," through 4 = "severe problem," worked reasonably well with these clients. Ranking scales or more complex rating systems should be avoided. The scale must also be constructed so that the examiner can read the items to clients who cannot read or write. It is important to use a scale that measures a variety of different functional components and to avoid single attribute instruments. Although there was no formal analysis of the relationship between the number of raters and the accuracy of predictions of neuropsychological test scores, the author's anecdotal experience has been that the more relatives who rate the same client, the more accurate the assessment of everyday performance. Therapists are therefore advised to distribute rating surveys to as many different observers as possible. Not only the average rating of the client, but also the variability of the ratings provides important information about daily functioning. Those items that elicit discrepant ratings identifY behaviors and skills that are likely to be quite variable or volatile.
REFERENCES 1. Herrmann D. Self-perceptions of memory performance. In Erlbaum S, ed., Self-directedness. Sterling, VA: World Comprehension Services, 1989. 2. Bennett-Levy J, Powell GE. The subjective memory questionnaire (SMQ): An investigation into the self-reporting of "real-life" memory skills. Brit J Soc Clin Psych 1980; 19: 177-188. 3. Broadbent DE, Cooper BE, Fitzgerald P, Parkes KR. The cognitive failures questionnaire (CFQ) and its correlates. Brit J Soc Clin Psych 1982;21:1-16. 4. Herrmann, DJ. Questionnaires about memory. In Harris J, Morris P, eds., Everyday merrwry. London: Academic Press, 1983.
5. Sunderland A, Harris JE, Baddely A. Do labo-
6. 7. 8.
9. lO.
ratory tests predict everyday memory? A neuropsychological study. ] of Verb Learn and Verbal Beh 1983;22:341-357. Small L. Neuropsychodiagnosis in psychotherapy (revised). New York: Brunner/Mazel, 1980. Cicerone Ko. Psychotherapeutic interventions with traumatically brain-injured patients. Rehab Psych 1989;34: lO5-114. Poon L, ed. Handbook for clinical merrwry assessment of older adults. Washington DC: Am Psych Assn, 1986. Miller L. Psychotherapy of the brain-injured patient. New York: WW Norton & Co., 1993. Gentry M, Herrmann DJ. Memory contrivances
Metacognitive Ratings Mter Brain Trauma
in everyday life. Pers and Soc Psych Bull 1990; 16(2):241-253. 11. Chaffin, RJS, Deffenbacher, K, Herrmann DJ: Awareness and lack of awareness of memory function between spouses. Unpublished manuscript, Hamilton College, New York, 1980. 12. Wechsler Adult Intelligence Scale-Revised. The Psychological Corporation, New York: Harcourt Brace Jovanovitch Inc., 1981.
173
13. Wechsler Memory Scale-Revised. The Psychological Corporation, New York: Harcourt Brace Jovanovitch Inc., 1987. 14. Wide Range Achievement Test-Revised. Jastak Assessment Systems, Wilmington: DE, 1984. 15. Derogatis LR: The SCL90-R. Clinical Psychometric Research: Baltimore, 1975. 16. CAMS survey. University of Alabama. Unpublished manuscript, 1987.
