Neuropsychological Rehabilitation An International Journal
ISSN: 0960-2011 (Print) 1464-0694 (Online) Journal homepage: http://www.tandfonline.com/loi/pnrh20
Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke David W. K. Man, M. K. L. Chan & C. C. K. Yip To cite this article: David W. K. Man, M. K. L. Chan & C. C. K. Yip (2015) Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke, Neuropsychological Rehabilitation, 25:6, 895-912, DOI: 10.1080/09602011.2014.997253 To link to this article: http://dx.doi.org/10.1080/09602011.2014.997253
Published online: 03 Jan 2015.
Submit your article to this journal
Article views: 80
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=pnrh20 Download by: [University of Lethbridge]
Date: 21 October 2015, At: 03:37
Neuropsychological Rehabilitation, 2015 Vol. 25, No. 6, 895 – 912, http://dx.doi.org/10.1080/09602011.2014.997253
Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
David W. K. Man1, M. K. L. Chan2, and C. C. K. Yip1 1
Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong 2 Occupational Therapy Department, Kowloon Hospital, Hong Kong (Received 21 February 2014; accepted 8 December 2014)
This study aimed to develop and evaluate a Hong Kong Chinese version of the Cambridge Prospective Memory Test (CAMPROMPT-HKCV). Thirty-three subjects at least one year post-stroke participated in the study. They were simultaneously rated on version A of the CAMPROMPT-HKCV by two testers to establish its internal consistency and inter-rater reliability. Raters used the parallel versions of the test (A and B), in rating 10 patients within 2 weeks to establish the parallel form reliability. Another 10 were also assessed on the same day using both version A of the CAMPROMPT-HKCV and the Rivermead Behavioural Memory Test – Chinese version (RBMT-CV) to establish concurrent validity. A new group of 40 stroke patients and 44 healthy controls was recruited to establish its sensitivity and specificity. Results indicated that test – retest reliability on time-based, event-based and total scores, and inter-rater reliability for versions A and B of the test were high. Cronbach’s alpha of the event-based score was higher than that of the time-based score. The reliability and concurrent validity of the parallel forms were established. There was a significant difference in performance on CAMPROMPT-HKCV (version A) between the stroke group and the healthy control group. ROC Correspondence should be addressed to Prof David W. K. Man, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Yuk Choi Road, Hung Hom, Kowloon, Hong Kong. E-mail: [email protected] The authors would like to thank Professor Barbara Wilson and Pearson Assessment for permission to translate the CAMPROMPT into Hong Kong Chinese and to use it for research purposes. We thank the Hong Kong Self Help Group for Brain Damage, the Occupational Therapy Department of Kowloon Hospital, Hong Kong College of Technology, and all the stroke patients and health controls for participation in the study. Thanks also go to research assistants Miss Y. Z. Chan, S. Y. Ling, S. H. Hung and W. S. H. Yam for data collection. # 2014 Taylor & Francis
896
MAN, CHAN, AND YIP
analysis showed that the ability of the cut-off CAMPROMPT-HKCV (total score) to differentiate PM problems was 20.5 (out of 36) with sensitivity at 95.5% and specificity at 55.9%. Further study in developing stratified norms across different age groups in Chinese-speaking stroke patients is recommended.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Keywords: Stroke; Prospective memory; Cambridge Prospective Memory Test (CAMPROMPT); Reliability; Validity; Sensitivity; Specificity.
INTRODUCTION For years, the study of cognitive ageing has been focused on retrospective memory, or the recollection of past events (Henry, MacLeod, Philips, & Crawford, 2004). Currently, increasing interest is being shown in the investigation of prospective memory (PM), which refers to the ability to perform an intention at a particular time or in response to a particular cue in the future (Ellis & Kvavilashvili, 2000; McDaniel & Einstein, 2000). Prospective memory tasks can be mainly categorised as time-based and event-based (Einstein & McDaniel, 1990). Time-based tasks are those that must be carried out at a particular time or after a certain amount of time, while event-based tasks are those that must be carried out in association with a particular event. Everyday examples of PM tasks include remembering to take medication on time, checking the oven to see if a cake has finished baking and keeping a hospital appointment. PM affects one’s independence in living and it may often be a priority in the process of neuropsychological rehabilitation (Fleming, Shum, Strong, & Lightbody, 2005). PM is frequently impaired in patients with neurological disorders (Shum, Valentine, & Cutmore, 1999). Stroke patients have particular problems with both the content and the retrieval of PM tasks. PM deficits following stroke (Brooks, Rose, Potter, Jayawardena, & Morling, 2004; Kim, Craik, Luo, & Ween, 2009) have been studied much less frequently, despite the high prevalence of stroke in the adult population (Hachinski, 2007) and the significant impact of stroke on frontal executive function. Failures in PM could affect independent living, and thus, an accurate assessment of PM is a priority in the process of neuropsychological rehabilitation (Fleming et al., 2005). To date, there are various types of tests measuring PM, but there are few neuropsychological tests available for a large Chinese-speaking population, especially in the PM domain. It is important that neuropsychological tests developed in the West can be properly adapted and translated for use in Eastern cultures. Their psychometric properties should be established before application to clinical practice or research. Measures of PM ability may include self-reported questionnaires, such as the Prospective and
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
897
Retrospective Memory Questionnaire (PRMQ; Smith, Della Sala, Logie, & Maylor, 2000) and the Comprehensive Assessment of Prospective Memory (CAPM; Roche, Fleming, & Shum, 2002). However, validity in measuring PM using questionnaires has been questioned (Uttl & Libreab, 2011). They may also not reflect accurate PM ability if participants have limited insight. Available objective PM tests include the Royal Prince Alfred Prospective Memory Test (RPA-ProMem; Radford, Lah, Say, & Miller, 2011), the Memory for Intentions Screening Test (MIST; Raskin, 2009), and the Cambridge Test of Prospective Memory (CAMPROMPT; Wilson et al., 2005). MIST uses a real-world task and allows a 24-hour delay in testing PM function, but requires a lengthened assessment period and has no alternative form available to minimise practice effects. Similar to MIST, RPA-ProMem may be difficult to administer, especially the long-term PM task. The CAMPROMPT is another commercially available prospective memory test which has satisfactory convergent and construct validity, test –retest reliability, as well as normative data for healthy individuals (Wilson et al., 2005). It has two parallel forms (versions A and B) and each has event-based and three time-based PM tasks designed to simulate everyday activities. It takes only 25 minutes to complete the short-form PM task. Thus, this study aimed to develop a Hong Kong Chinese version of CAMPROMPT (CAMPROMPT-HKCV) and to examine its validity, reliability, sensitivity and specificity for people with stroke.
METHOD The present study of the CAMPROMPT-HKCV was divided into three parts: (1) development of a translated Hong Kong Chinese version and evaluation of its linguistic/cultural validity, (2) evaluation of its concurrent validity and reliability, and (3) its sensitivity and specificity.
Development of a translated version Participants. A review panel of four members was recruited. They included three occupational therapists and one clinical psychologist who had all had experience of working in stroke rehabilitation for more than five years. Measures. A questionnaire rating both the instructions and the paper-andpencil worksheets of versions A and B of the original CAMPROMPT and the Hong Kong Chinese version were developed. Procedures. Step 1: After gaining ethical approval from the Hong Kong Polytechnic University, invitation letters and information sheets were
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
898
MAN, CHAN, AND YIP
distributed by mail. Researchers then met the panel members in person to introduce the CAMPROMPT-HKCV and to explain the purposes of the study and the definitions of related terms, if necessary. Step 2: Panel members, whose native language was Chinese, initially translated both the instructions and the paper-and-pencil worksheets of the CAMPROMPT from English into Hong Kong Chinese (CAMPROMPTHKCV). The use of culturally relevant and equivalent materials and linguistic appropriateness were considered in this process. Step 3: Forward and back translations of the content and the instructions were reviewed by bilingual translators from the Department of Chinese and Bilingual Study of the Hong Kong Polytechnic University. The purposes of the study and the definitions of related terms for the test were explained to them in person. The word discrepancies between the translated version and the original were analysed and modifications were made. The data were analysed in order to determine the percentage of agreement. Answers to the openended portions were also analysed. Thus, initial versions A and B of the CAMPROMPT-HKCV were formulated. Step 4: A panel was invited to review again the clarity, relevance and representativeness of the instructions and worksheets in the CAMPROMPT-HKCV. Panel members had to assign ratings on a four-point Likert scale in the closed question section of the questionnaire. Suggestions or comments could be provided in the open-ended question portion. Step 5: After receiving all the questionnaires from the four panel members, the CAMPROMPT-HKCV was then revised. Thus, a 49-item questionnaire that was designed to evaluate the accuracy and equivalence of the translation in terms of clarity, cultural relevancy and representativeness was completed. Ratings on the four-point Likert scale were regrouped and dichotomised into agree (ratings of totally agree and agree) and disagree (ratings of totally disagree and disagree) ratings.
Reliability and validation study Participants. Thirty-three participants with stroke were recruited by a convenience sampling method from a local Kowloon Hospital (Hospital Authority of Hong Kong). The inclusion criteria were: (1) at least one year post-stroke; (2) Chinese speaking; (3) no dementia as indicated by the Chinese version of the Mini Mental State Examination (Chiu, Lee, Chung, & Kwong, 1994), with a cut-off score of 22 or above being used; (4) no severe language comprehension deficits or learning disabilities; (5) no previous neurological or psychiatric illness; (6) no history of substance abuse; and (7) no visual impairment. Of the 36 participants recruited, two were excluded due to not fulfilling the criteria and one additional subject
CAMPROMPT FOR STROKE
899
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
dropped out of the research. As a result, a sample of 33 participants was successfully recruited. Measures. The following measures were used. The Cambridge Prospective Memory Test (CAMPROMPT) drew on ideas developed over a number of years by Wilson and her colleagues (first mentioned in a paper by Kime, Lamb, & Wilson, 1996); extensive pilot studies having been conducted. This is the first standardised test to have been designed to assess prospective memory within an ecologically plausible context (Wilson et al., 2005). In the test, there are six prospective memory tasks for the examinee to carry out in a 25-minute period. The six prospective tasks are cued in two ways: three by time and three by events. The time intervals between being asked to do the task and responding appropriately at the right moment are balanced across cueing conditions. There are two parallel versions of the test, A and B. Examinees are asked to work on a number of distracting paper-and-pencil worksheets, such as a general knowledge quiz or a word-finder puzzle for a 20-minute period. While they are working on the worksheet, they are asked to remember six tasks (time- and eventbased), either during the 20-minute session or shortly after its conclusion. Examinees may use any strategy to remember the tasks, for example, they may write a reminder on the paper provided. Each of the six prospective memory tasks is given a score according to the pattern of response (the timing of the action and the action itself) that is described in the administrative manual. These responses were categorised using the letters A to H. These letters are later transformed into scores of 6, 4, 2, 1, or 0. For instance, if an examinee spontaneously carries out the correct action at the correct time (score A), this is awarded 6 marks; whereas, if an examinee fails to remember that there is something to be done even after being prompted twice, so that the correct task is still not carried out (score H), this is awarded 0 marks. Thus, the raw score for the six tasks can range from 0–36. By comparing a patient’s raw score within his/her IQ band and age group, one can classify the examinee’s prospective memory as impaired, borderline, poor, average, above average or very good. The original versions, A and B, including the on-going tasks (quizzes, exercises), were translated for validation in the first part of the present study. The Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975) was used in the present study to document stroke patients’ global cognitive status and it also served as a screening criterion for subject selection. This test is widely used for assessing cognitive functions both in clinical settings and in research. For the English version, the reported sensitivity was 97.5% and the specificity was 97.3%. The test–retest reliability is .78 and the inter-rater reliability .99. The validated Hong Kong Cantonese version of the MMSE was used for screening purposes (Chiu et al., 1994).
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
900
MAN, CHAN, AND YIP
A cut-off score of 19 or below out of the total possible score of 30 has been used in screening for dementia (Chiu et al., 1998). With reference to the educational level of older people, different cut-off scores are used in the Hong Kong Cantonese MMSE: a score of 22 or below for those with education of more than 2 years, a score of 20 for those with 1–2 years of education and a score of 18 or below for those with no education. The Test of Nonverbal Intelligence–3 (TONI-3; Brown, Sherbenou, & Johnson, 1997). In the present study, form A of the TONI-3 was adopted to test intelligence in stroke patients. It measures a specific component of intelligent behaviour by testing an individual’s ability to solve problems without overtly using language. Each of its two equivalent forms (versions A and B) contains 45 items that are arranged in order of difficulty (Brown et al., 1997). This test is not heavily loaded with linguistic, motoric and cultural elements and is easy to administer. The Rivermead Behavioural Memory Test –Chinese version (RBMT-CV; Li & Man, 2001; Wilson, Cockburn, & Baddeley, 1985) was suggested as a test to examine the concurrent validity of the CAMPROMPT-HKCV in this study. It is a commonly used standardised memory test. It includes nine subtests that have been designed to assess memory skills related to everyday situations. The test is useful to predict memory deficits in everyday life memory tasks for patients who have experienced brain damage through injury or illness (Brooks et al., 2004; Fish, Wilson, & Manly, 2010). One of the four parallel forms (version A) was used in this study to measure accurately memory change over time. In the RBMT-CV, three out of nine items measure eventbased prospective memory, i.e., to deliver a message, to ask for the return of a belonging and the date of an appointment. It is viewed as the most relevant test to assess everyday as well as event-based prospective memory. Procedures. All participants were given an information sheet describing the purpose of the study and signed a Chinese consent form prior to the study. The purposes of the study were explained to them before they were assessed. They were also randomly assigned into three different groups for reliability and validity testing. The administration of the CAMPROMPT-HKCV took place in a distraction-free room. All participants were rated by two testers simultaneously using version A of the test to establish the inter-rater reliability. Ten were assessed with both versions A and B within two weeks to establish parallel form reliability. Another 10 were rated by the same rater using both the CAMPROMPT-HKCV (version A) and the RBMT-CV (version A) on the same day. Ten extra participants were assessed by the CAMPROMPT-HKCV (version A) twice within a two week interval to study its test –retest reliability. The internal consistency of the CAMPROMPT-HKV was tested by analysing the consistency of the time-based and event-based tasks of the CAMPROMPT-HKCV (version A).
CAMPROMPT FOR STROKE
901
Sensitivity and specificity study
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Participants. Another group of 40 stroke patients who met similar inclusion criteria to the prior validity and reliability study were recruited by a convenience sampling method from the same local Kowloon Hospital. In addition, 44 healthy controls without a history of stroke, substance abuse, psychiatric, neurological or major medical disorders, who were approximately matched according to a self-report questionnaire collecting information of age, sex and education, were selected from a vocational training centre (Hong Kong College of Technology). Measures. The CAMPROMPT-HKCV (version A) developed in the validity and reliability study was adopted for testing the two groups’ PM abilities. Procedures. Ethical approval was granted before recruitment. Again both the stroke and healthy participants were given an information sheet about the study and written consent was obtained before testing.
RESULTS Translation of the CAMPROMPT-HKCV Overall, the percentage of agreement between the panel members in the dichotomised scale in terms of clarity, cultural relevancy, representativeness and overall comments were 91.8%, 89.8%, 100% and 100%, respectively. There were few disagreements on clarity (4 out of 49 questions) and cultural relevancy (5 out of 49 questions). The instructions were revised based on the comments of the panel members, which resulted in the finalisation of a revised CAMPROMPT-HKCV.
Validation and reliability of the study A total of 33 participants with stroke were tested: 22 males (67%) and 11 females (23%). The age range was 57 –65 years, with a mean of 53.48 years (SD ¼ 7.58). Total MMSE-CV scores ranged from 23–29 out of a total of 30 (mean ¼ 26.8, SD ¼ 2.40), and thus, participants did not have significant cognitive impairment. Participants’ TONI-III quotient (similar to IQ) scores ranged from 69 –102 (mean ¼ 84.9). Most scored in the ranges of 81– 90 (55.6%) and 71 –80 (22.2%). Their mean of RMBT-CV was 17.3 (SD ¼ 4.70; range 14–24). Table 1 shows the performance of the full sample on
902
MAN, CHAN, AND YIP TABLE 1 Item scores in Form A of CAMPROMPT-HKCV in reliability and validity study
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
(N ¼ 33) Time-based task Change task Take key Ring garage Total time-based Event-based task Book/map Give message Object locations Total event-based
Mean
SD
Range
1.35 2.09 2.48 5.91
1.191 1.905 2.192 3.655
0 –4 0 –6 0 –6 1 –16
3.17 3.96 3.83 10.96
1.992 2.286 1.775 4.791
0 –6 0 –6 0 –6 0 –18
Figure 1. Trend of mean subscores of CAMPROMPT-HKCV across time in reliability and validity study.
version A of the CAMPROMPT-HKCV. Figure 1 also shows the trend of mean scores of stroke patients taking CAMPROMPT-HKCV across time. The internal consistency, using the Cronbach’s alpha results, showed that the time-based score was lower (.493) than that for the event-based tasks (.693). The deletion of any items in the time-based or event-based scales, however, was found to cause negligible changes in the Cronbach’s alpha. It was also found that the mean score of the event-based tasks was higher, and nearly twice the mean score of the time-based tasks among the 33 participants. The test–retest reliability on time-based, event-based and total scores was considered acceptable and showed good reliability (ICC ¼ 0.809, p ¼ .047; ICC ¼ 0.890, p ¼ .015; and ICC ¼ 0.869, p ¼ .022, respectively). A lack of
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
903
variability in the test scores secondary to the homogeneous sample may reduce the test–retest reliability coefficient. In this study, the betweensubject variance in the test scores for time-based (F ¼ 5.231, p , .05), event-based (F ¼ 9.104, p , .05) and total score performance (F ¼ 7.651, p , .05) were statistically significant, which indicated that there was a great variability in the test scores. The high ICCs for the inter-rater reliability for versions A and B of the CAMPROMPT-HKCV were 0.959 (p , .001) and 0.983 (p , .001), respectively (Gerlinas, Gauthier, McIntyre, & Gauthier, 1999). The correlation between versions A and B (r ¼ .635, p ¼ .05) of the CAMPROMPT-HKCV was statistically significant. Paired samples t-test comparing the version A and version B scores of CAMPROMPT-HKCV showed that one performance was not significantly different from the other, t(8) ¼ 1.779, p ¼ .109.
Concurrent validity Only 10 participants were assessed with both version A of the CAMPROMPT-HKV and version A of the RBMT-CV on the same day due to difficulties in recruiting more clinical participants. The overall total score of each test was calculated and used to investigate the correlation between the two tests. In the RBMT-CV, the standardised profile score was used to analyse the results. The maximum of the standardised profile score in the RBMT-CV was 24 and the participants’ mean score was 17.6 (range 10– 23). On the other hand, the score of the CAMPROMPT-HKCV (maximum ¼ 36) ranged from 10–30 with a mean score of 19.1. The standardised profile score of the RBMT-CV and the overall total score of the CAMPROMPT-HKCV were compared and generated a correlation result. High to excellent correlation (Spearman rank-order correlation rho ¼ .868, p , .05) was found. Further analysis showed that there was significant correlation between the event-based CAMPROMPT-HKCV score and the RMBT-CV total score (Spearman’s rho correlation coefficient ¼ .751; p ¼ .03) and RMBT-CV event-based items scores (Spearman’s rho correlation coefficient ¼ .632; p ¼ .021), respectively. For the time-based CAMPROMPT-HKCV score, the Spearman’s rho correlation coefficient to RBMT total score and RBMT-CV event-base items score were .443 (p ¼ .13) and .59 (p ¼ .034), respectively. Thus, except CAMPROMPT-HKCV (time-based score), both the total and event-based scores of CAMPROMPT-HKCV significantly correlated with both RBMT-CV total and RBMT-CV event-based score. On one hand, CAMPROMPT-HKCV is equally valid in testing event-based PM as the RBMT-CV, on the other, CAMPROMPT-HKCV also tests time-based PM performance which is not tested in the RBMT-CV.
904
MAN, CHAN, AND YIP
PM performance on CAMPROMPT-HKCV As noted, scores for each of the PM tasks are provided in Table 1. It can be seen that overall, time-based tasks were not performed as well as event-based tasks. The lowest scoring task was the time-based task of asking participants to change task after seven minutes. The task on which participants scored highest was the event-based task of giving a message envelope when five minutes was left.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Sensitivity and specificity Forty-four healthy controls without a history of stroke, substance abuse or psychiatric, neurological or major medical disorders that were approximately matched according to age, sex and education were selected from a local vocational training centre (Hong Kong College of Technology). Forty patients who met similar inclusion criteria as the validation and reliability study were recruited by a convenience sample of patients from Kowloon Hospital (Hospital Authority of Hong Kong). Table 2 shows that the age of the control group (mean ¼ 41.86 years; SD ¼ 9.32) was significantly younger than the stroke group (mean ¼ 50.13, SD ¼ 10.29, p , .001). There was also a significant difference in the distribution of gender in the two groups (x2 ¼ 17.253, p , .001). To compare the differences in their performances on the total score of CAMPROMPT-HKCV, we adjusted for age and gender differences. ANCOVA showed that there was no significant interaction in the performance of PM between group and age, F(1, 78) ¼ 0.576, p ¼ .45; between group and gender, F(1, 78) ¼ 0.154, p ¼ .696; between age and gender, F(1, 78) ¼ 2.223, p ¼ 0.14; and among group, gender and age, F(1, 78) ¼ 0.128, p ¼ .722. After eliminating the interaction among the factors, there was a significant difference in the performance on the total score of CAMPROMPT-HKCV between the stroke group and the control group with adjustments for age and gender, F(1, 77) ¼ 32.9, p , .001. ROC analysis showed that total, event-based and time-based CAMPROMPT-HKCV cut-off scores could differentiate PM problems in stroke patents. Good trade-off between sensitivity and specificity was also shown (See Table 3). The area under curve (AUC) for total, event-based and timebased scores of CAMPROMPT-HKCV were 0.911, 0.843, and 0.894, respectively (see Figure 2).
DISCUSSION To date, this study is the first to develop a translated CAMPROMPT-HKCV and to investigate its psychometric properties for the stroke population. The process of translation is reported. In addition, the concurrent validity,
905
CAMPROMPT FOR STROKE TABLE 2 Demographics of stroke and healthy group in specificity and sensitivity study
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Stroke (N ¼ 40)
Healthy (N ¼ 44)
(SD)
Age Years of education CAMPROMPT-HKCV total score Event-based event total score Time-based event total score
50.13 10.85 20.74
(10.30) (4.35) (6.91)
41.86 10.55 29.61
(9.32) (1.45) (4.17)
.001 .675 .001
11.00
(3.52)
14.95
(2.23)
.001
7.55
(4.43)
14.66
(3.01)
.001
MMSE-CV1 Estimated IQ2
28.3 85.4
(2.20) (16.00)
90.68(14.19)
n
(%)
n
(%)
p-value∗∗
25.00 15.00 19.00 21.00 26.00 14.00
(62.50) (37.50) (47.50) (52.50) (65.00) (35.00)
8.00 36.00
(18.18) (81.82)
.001
Gender Type of stroke Paretic side
Male Female Ischaemic Haemorrhagic Left Right
Mean
p-value∗
Mean
(SD)
.113
∗
1 2
by independent t-test; ∗∗ by chi-square test. Mini Mental State Examination- Chinese version Estimated IQ based on Test of Non-verbal Intelligence-III (Toni-III; Brown et al., 1997)
reliability (internal consistency, test–retest, and parallel form), specificity and sensitivity in screening stroke patients with PM difficulties and healthy controls have been studied. The high percentage of agreement between the four panel members regarding the clarity, cultural relevancy and representativeness of the initial translation supported its quality. The CAMPROMPT-HKCV also demonstrated
TABLE 3 Cut-off scores, sensitivity and specificity of scores for CAMPROMPT-HKCV CAMPROMPT-HKCV Total score Event-based Time-based
Cut-off score
Sensitivity
Specificity
20.5 13 8.5
95.50% 86.40% 97.70%
63.20% 68.40% 63.20%
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
906
MAN, CHAN, AND YIP
Figure 2. ROC curve of COMPROMPT-HKCV in specific and sensitivity study.
a good reliability over time and consistency among raters and between the two versions. Internal consistency of the time-based score was much lower than that for event-based tasks within the CAMPROMPT-HKCV. It was suggested that the three time-based tasks might not be similar, thus producing different memory demands on the stroke patients. Reviewing and changing these three tasks might be necessary to further improve its internal consistency. Moreover, the observed higher mean score of the event-based than time-based tasks echoed Einstein, McDaniel, Richardson, Guynn, and Cunfer’s (1995) suggestion that time-based tasks may be more difficult to remember than event-based tasks because the passage of time must be monitored and the remembering is self-initiated, whereas event-based tasks were cued by an external stimulus. If the time-based PM was more dependent on internal
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
907
cues, it seemed to have a higher level of difficulty for stroke clients in the present study. Einstein et al. (1995) also suggested that event-based prospective remembering may actually rely on a more spontaneous or automatic component, at least compared with time-monitoring tasks. An event itself can act as an external cue in the environment and can make the event-based task easier. Although similar, good test –retest reliability of time-based, event-based and total scores of CAMPROMPT-HKCV are reported; the learning effect and the short 2-week test –retest interval were noted. It was found that participants were able to recall some testing procedures performed in the initial assessment, which indicated that a learning effect did occur. According to Gregory (1999), changes over time are the major source of error in test– retest reliability. The change may result from a real change in the traits of participants, unstable traits, testing effects, the test–retest interval, motivational factors of participants, or rater bias (Gregory, 1999; Portney & Watkins, 1993). As the participants in this study were stable with minimal cognitive fluctuation, the possibility of a dramatic change was unlikely during the test and retest period. The variation of scores in the test –retest may not be due to any real change in traits or unstable traits in the participants. The inter-rater reliability for versions A and B in our study was consistent with the original version of the CAMPROMPT (Wilson et al., 2005). This indicates that the scoring criteria of the CAMPROMPT-HKCV were clear to raters and well written for its administration. The moderate to good correlation between parallel versions A and B of the CAMPROMPT-HKCV suggested that they were not significantly different. Excellent correlation might have resulted from a larger sample size for parallel form reliability. With reference to the results of the original CAMPROMPT (Wilson et al., 2005), one of the CAMPROMPT-HKCV versions can be given to a subject without the results being affected. In establishing concurrent validity, the RBMT-CV was used as the gold standard by which the CAMPROMPT-HKCV was analysed. According to the results, the correlation in concurrent validity was reported to be high to excellent. Three out of the nine tasks in the RBMT-CV assessed PM function, the other six measured general memory skills such as delayed recall, immediate recall, working memory and orientation for date and place. The RBMTCV was not specifically designed to measure PM, but it might be one of the few objective assessments that has the most PM components to compare with the newly developed CAMPROMPT-HKCV. Moreover, the RBMT-CV is the only memory test that has been translated into a Hong Kong Chinese version and widely used in a Hong Kong clinical setting. Despite these limitations, the RBMT-CV was still chosen as the tool to determine the concurrent validity; however, it should be noted that the RBMT-CV was a better, but not perfect, standardised assessment to assess the
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
908
MAN, CHAN, AND YIP
participants’ PM function and pragmatically used to establish the CAMPROMPT-HKCV’s concurrent validity. On the other hand, the event-based score of CAMPROMPT-CV was found to correlate significantly with the total and event-based RBMT-CV scores, suggesting that it is valid in measuring PM, especially event-based PM. Although total and time-based scores of CAMPROMPT-HKCV only correlated significantly with event-based but not time-based RBMT-CV, it seems to hint that CAMPROMPT-HKCV has added value as a PM assessment in measuring comprehensively both eventbased and time-based PM functions while having similar merit to RBMTCV in being an ecologically valid memory assessment relevant to everyday functioning. Moreover, as both the Hong Kong Chinese and Chinese Mainland versions (Lou, Dou, Zheng & Man, 2009) have already been translated, including instructions and worksheets (as ongoing tasks during assessment), application to a larger Chinese-speaking stroke population becomes a possibility. The poorer performance of carrying out a PM task to change their task (in version A) or pen (in version B) after seven minutes might be explained by the fact that stroke participants needed to be aware of the instruction, “seven minutes later”. This instruction was an indirect indication of time. It seemed that the participants had difficulty in fully comprehending this instruction, became confused and performed the task badly. In addition, this task required more cognitive processing as the participants needed to calculate the approximate timing of seven minutes and, at the same time, check the timer through self-initiation. It is possible that this time-based task was more challenging for the participants. Therefore, this specific time-based task appeared to be the most difficult one of the six tasks. On the other hand, the event-based task of giving an envelope when five minutes was left scored the highest. A card with written instructions was given to the participants and could be used by them as a visual cue. In carrying out this assessment, several factors are worth noting as they might affect the participants’ performance, and thus, the scores and interpretation of the results. These factors might include the participants’ attention, interferences from the paper quiz, the verbal instructions given during the assessment, the level of difficulty of the tasks and the sequence in which the instructions were given. As reported earlier, time-based PM scores were lower than any of the event-based PM scores, thus time-based tasks may be more difficult than event-based tasks. Interruptions that engage participants in a new task for as little as 15 seconds have been shown to interfere with prospective memory (Einstein, McDaniel, Williford, Pagan, & Dismukes, 2003; see also Dodhia & Dismukes, 2009). Prospective memory might have suffered when the demands of the ongoing task were increased or when attention was divided with a task that engages central executive resources (Marsh & Hicks, 1998). Based on these findings, it seems that both interference
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
909
between tasks and interference with the participants’ attention may adversely affect performance, and thus, participants’ scores. The fact that interference was increasing and attention possibly decreasing while the assessment was in progress, suggests that participants’ scores might drop for a particular task, according to how late in time each task was to be executed. However, our findings contradicted this postulation. According to Figure 1, the scores varied instead of declining across the assessment period. This might be explained by the level of difficulty of the task itself having more influence on the scores than attention or interference. Based on the assessment results, the scores were rated by therapists on a 5-point scale (scores of 6, 4, 2, 1, and 0). The ratings depended on whether the participants spontaneously carried out the tasks and the number of times that they were prompted. As scoring was based on a prompting system, participants who made a guess regarding the task would receive higher marks, whereas participants who gave up would score zero. Correct guessing might lead to a random effect. It might create a deviation in the results that did not actually represent the prospective memory performance of the participants. During the assessment using the CAMPROMPTHKCV, the participants needed regularly to check a digital clock. Participants who were diagnosed with stroke may have visual problems that could lead to a poorer performance in the assessment. This assessment might not be easy to generalise to other stroke patients who have visuo-perceptual problems while reading the clock. The sensitivity and specificity findings showed that the CAMPROMPTHKCV had good discrimination ability for PM problems. It is suggested that the test can differentiate between stroke patients and healthy controls. In addition, among the three scores, the CAMPROMPT-HKCV total score was the most informative and is suggested for clinical screening use. The importance and implications of the above findings have been identified. The test items in CAMPROMPT-HKCV resemble long-term PM that is required in daily life. The test results are representative of daily life PM performance and increase the ecological validity of the CAMPROMPT-HKCV. In future, its predictive validity of daily functioning could be further examined to obtain more proven evidence. Both event-based and time-based PM can be assessed with the CAMPROMPT-HKCV, instead of relying only on RBMT-CV which mainly assesses the memory process and retrospective memory. CAMPROMPT-HKCV is also sensitive in picking up PM problems in stroke patients, especially in time-based tasks. Its ability to differentiate stroke patients from healthy controls using a cut-off score of 20.5/30 has additional merits in its development into a screening tool for early problem identification during acute or subacute stroke rehabilitation. As there are few standardised PM test batteries that have been translated and adopted in a Chinese context, the CAMPROMPT-HKCV is considered attractive to
910
MAN, CHAN, AND YIP
psychologists. Its satisfactory psychometric properties may also appeal to rehabilitation personnel in serving an increasing Chinese-speaking stroke population.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Limitations and suggestions There were several limitations in this study. The convenience sampling, the small sample size and the homogeneity of the sample reduced the generalisability of the results. Therefore, the recruitment of more participants from multi-centres by randomisation is suggested. Evidence from a number of studies indicates that older adults often perform poorly relative to younger adults on tasks that require prospective remembering (Maylor, 1993, 1996). It is assumed that there will be some decline in prospective memory function during healthy ageing; comparisons can be made between healthy elderly participants and elderly participants with stroke in future studies. Parallel form reliability was not assessed with a counterbalancing method, which can be a confounder, particularly when only 10 patients were used to assess reliability in this study. Some stroke patients had difficulties with clockchecking during testing. Their individual difficulties might be accommodated by a verbal request to the assessor. This may be another way of measuring of PM ability. In the original CAMPROMPT, normative data were calculated based on premorbid IQ and age. In future, normative data of CAMPROMPTP-HKCV in relation to age, gender and IQ should be developed for stroke and other disease populations.
CONCLUSION The Cambridge Prospective Memory Test (CAMPROMPT) was translated and validated in this study. The psychometric properties of the CAMPROMPT-HKCV were explored and, despite the methodological limitations, it was found that the translated version was reliable. This implies that the CAMPROMPT-HKCV has the potential to be a valid and reliable tool for screening and/or assessing prospective memory for Hong Kong and other Chinese-speaking patients with stroke. Further study in developing stratified norms across different age groups in stroke patients is recommended, so as to improve the measurement of the rate of change and decision making in clinical neuropsychological rehabilitation.
REFERENCES Brooks, B. M., Rose, F. D., Potter, J., Jayawardena, S., & Morling, A. (2004). Assessing stroke patients’ prospective memory using virtual reality. Brain Injury, 18(4), 391 – 401.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
911
Brown, L., Sherbenou, R. J., & Johnsen, S. K. (1997). Test of Nonverbal Intelligence: A language-free measure of cognitive ability (3rd ed.). Austin, TX: Pro-ed. Chiu, H. F. K., Lam, L. C. W., Chi, I., Leung, T., Li, S. W., Law, M. T., . . . & Lau, J. (1998). Prevalence of dementia in Chinese elderly in Hong Kong. Neurology, 50, 1002– 1009. Chiu, H. F. K., Lee, H. C., Chung, W. S., Kwong, P. K. (1994). Reliability and validity of the cantonese version of mini-mental state examination. Journal of Hong Kong College of Psychiatry, 4, 25 – 28. Dodhia, R. M., & Dismukes, R. K. (2009). Interruptions create prospective memory tasks. Applied Cognitive Psychology, 23(1), 73 –89. Einstein, G. O., & McDaniel, M. A. (1990). Normal aging and prospective memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 717 – 726. Einstein, G. O., McDaniel, M. A., Richardson, S. L., Guynn, M. J. & Cunfer, A. R. (1995). Aging and prospective memory: Examining the influences of self-initiated retrieval processes. Journal of Experimental Psychology: Learning Memory, and Cognition, 21(4), 996 – 1007. Einstein, G. O., McDaniel, M. A., Williford, C. L., Pagan, J. L., & Dismukes, R. K. (2003). Forgetting of intentions in demanding situations is rapid. Journal of Experimental Psychology: Applied, 9(3), 147 – 162. Ellis, J., & Kvavilashvili, L. (2000). Prospective memory in 2000: Past, present, and future directions. Applied Cognitive Psychology, 14, S1– S9. Fish, J., Wilson, B. A., & Manly, T. (2010). The assessment and rehabilitation of prospective memory problems in people with neurological disorders: A review. Neuropsychological Rehabilitation, 20(2), 161 – 179. Fleming, J. M., Shum, D., Strong, J., & Lightbody, S. (2005). Prospective memory rehabilitation for adults with traumatic brain injury: A compensatory training programme. Brain Injury, 19, 1 – 10. Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189 – 198. Gerlinas, I., Gauthier, L., McIntyre, M., & Gauthier, S. (1999). Development of a functional measure for persons with Alzheimer’s disease: The disability assessment for dementia. American Journal of Occupational Therapy, 53, 471 – 481. Gregory, R. J. (1999). Psychological testing: History, principles and applications (3rd ed.). Boston, MA: Allyn and Bacon. Hachinski, V. (2007). The 2005 Thomas Willis Lecture: Stroke and vascular cognitive impairment: A transdisciplinary, translational and transactional approach. Stroke, 38, 1396 – 1403. Henry, J. D., MacLeod, M. S., Philips, L. H., & Crawford, J. R. (2004). A meta-analytic review of prospective memory and aging. Psychology and Aging, 19(1), 27 – 39. Li, R., & Man, W. K. (2001). Assessing Chinese adult’s memory abilities: Validation of the Chinese version of the Rivermead Behavioural Memory Test. Clinical Gerontology, 24(3 – 4), 27 – 36. Lou, Z. L., Dou, Z. L., Zheng, J. L., Chen, Y. B., & Man, D. W. K. (2009). The study of the Chinese version of Cambridge Prospective Memory Test (CAMPROMPT) for traumatic brain injury (unpublished Master thesis). Guangzhou: Sun Yat Sen University. Kim, H. J., Craik, F. I. M., Luo, L., & Ween, J. E. (2009). Impairment in prospective and retrospective memory following stroke. Neurocase, 15(2), 145 – 156. Kime, S. K., Lamb, D. G., & Wilson, B. A. (1996). Use of a comprehensive program of external cuing to enhance procedural memory in a patient with dense amnesia. Brain Injury, 10, 17 – 25.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
912
MAN, CHAN, AND YIP
Marsh, R. L., & Hicks, J. L. (1998). Event-based prospective memory and executive control of working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(2), 336 – 349. Maylor, E. A. (1993). Aging and forgetting in prospective and retrospective memory tasks. Psychology & Aging, 8, 420 – 428. Maylor, E. A. (1996). Age-related impairment in an event-based prospective-memory task. Psychology and Aging, 11, 74 – 78. McDaniel, M. A., & Einstein, G. O. (2000). Strategic and automatic processes in prospective memory retrieval: A multiprocess framework. Applied Cognitive Psychology, 14, S127 – S144. Portney, L. G., & Watkins, M. P. (1993). Foundations of clinical research: Applications to practice. Norwalk, CT: Appleton & Lange. Radford, K. A., Lah, S., Say, M. J., & Miller, L. A. (2011). Validation of a new measure of prospective memory: The Royal Prince Alfred Prospective Memory Test. Clinical Neuropsychologist, 25(1), 127 – 140. Raskin, S. (2009). MIST: Psychometric properties and clinical evidence. Brain Impairment, 10(1), 23 – 33. Roche, N., Fleming, J., & Shum, D. (2002). Self-awareness of prospective memory failure in adults with traumatic brain injury. Brain Injury, 16, 931 – 946. Shum, D., Valentine, M., & Cutmore, T. (1999). Performance of individuals with severe longterm traumatic brain injury on time-, event-, and activity-based prospective memory tasks. Journal of Clinical & Experimental Neuropsychology, 21(1), 49 – 58. Smith, G., Della Sala, S., Logie, R.H., & Maylor, E. A. (2000). Prospective and retrospective memory in normal ageing and dementia: A questionnaire study. Memory, 8(5), 311 – 321. Uttl, B., & Libreab, K. (2011). Self-report measures of prospective memory are reliable but not valid. Canadian Journal of Experimental Psychology, 65(1), 57– 58. Wilson, B. A., Cockburn, J., & Baddeley, A. (1985). The Rivermead Behavioural Memory Test. Bury St. Edmunds, UK: Thames Valley Test Co. Wilson, B. A., Emslie, H., Foley, J., Shiel, A., Watson, P., & Hawkins, K. (2005). The Cambridge Prospective Memory Test. London: The Psychological Corporation.
ISSN: 0960-2011 (Print) 1464-0694 (Online) Journal homepage: http://www.tandfonline.com/loi/pnrh20
Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke David W. K. Man, M. K. L. Chan & C. C. K. Yip To cite this article: David W. K. Man, M. K. L. Chan & C. C. K. Yip (2015) Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke, Neuropsychological Rehabilitation, 25:6, 895-912, DOI: 10.1080/09602011.2014.997253 To link to this article: http://dx.doi.org/10.1080/09602011.2014.997253
Published online: 03 Jan 2015.
Submit your article to this journal
Article views: 80
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=pnrh20 Download by: [University of Lethbridge]
Date: 21 October 2015, At: 03:37
Neuropsychological Rehabilitation, 2015 Vol. 25, No. 6, 895 – 912, http://dx.doi.org/10.1080/09602011.2014.997253
Validation of the Cambridge Prospective Memory Test (Hong Kong Chinese version) for people with stroke
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
David W. K. Man1, M. K. L. Chan2, and C. C. K. Yip1 1
Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong 2 Occupational Therapy Department, Kowloon Hospital, Hong Kong (Received 21 February 2014; accepted 8 December 2014)
This study aimed to develop and evaluate a Hong Kong Chinese version of the Cambridge Prospective Memory Test (CAMPROMPT-HKCV). Thirty-three subjects at least one year post-stroke participated in the study. They were simultaneously rated on version A of the CAMPROMPT-HKCV by two testers to establish its internal consistency and inter-rater reliability. Raters used the parallel versions of the test (A and B), in rating 10 patients within 2 weeks to establish the parallel form reliability. Another 10 were also assessed on the same day using both version A of the CAMPROMPT-HKCV and the Rivermead Behavioural Memory Test – Chinese version (RBMT-CV) to establish concurrent validity. A new group of 40 stroke patients and 44 healthy controls was recruited to establish its sensitivity and specificity. Results indicated that test – retest reliability on time-based, event-based and total scores, and inter-rater reliability for versions A and B of the test were high. Cronbach’s alpha of the event-based score was higher than that of the time-based score. The reliability and concurrent validity of the parallel forms were established. There was a significant difference in performance on CAMPROMPT-HKCV (version A) between the stroke group and the healthy control group. ROC Correspondence should be addressed to Prof David W. K. Man, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Yuk Choi Road, Hung Hom, Kowloon, Hong Kong. E-mail: [email protected] The authors would like to thank Professor Barbara Wilson and Pearson Assessment for permission to translate the CAMPROMPT into Hong Kong Chinese and to use it for research purposes. We thank the Hong Kong Self Help Group for Brain Damage, the Occupational Therapy Department of Kowloon Hospital, Hong Kong College of Technology, and all the stroke patients and health controls for participation in the study. Thanks also go to research assistants Miss Y. Z. Chan, S. Y. Ling, S. H. Hung and W. S. H. Yam for data collection. # 2014 Taylor & Francis
896
MAN, CHAN, AND YIP
analysis showed that the ability of the cut-off CAMPROMPT-HKCV (total score) to differentiate PM problems was 20.5 (out of 36) with sensitivity at 95.5% and specificity at 55.9%. Further study in developing stratified norms across different age groups in Chinese-speaking stroke patients is recommended.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Keywords: Stroke; Prospective memory; Cambridge Prospective Memory Test (CAMPROMPT); Reliability; Validity; Sensitivity; Specificity.
INTRODUCTION For years, the study of cognitive ageing has been focused on retrospective memory, or the recollection of past events (Henry, MacLeod, Philips, & Crawford, 2004). Currently, increasing interest is being shown in the investigation of prospective memory (PM), which refers to the ability to perform an intention at a particular time or in response to a particular cue in the future (Ellis & Kvavilashvili, 2000; McDaniel & Einstein, 2000). Prospective memory tasks can be mainly categorised as time-based and event-based (Einstein & McDaniel, 1990). Time-based tasks are those that must be carried out at a particular time or after a certain amount of time, while event-based tasks are those that must be carried out in association with a particular event. Everyday examples of PM tasks include remembering to take medication on time, checking the oven to see if a cake has finished baking and keeping a hospital appointment. PM affects one’s independence in living and it may often be a priority in the process of neuropsychological rehabilitation (Fleming, Shum, Strong, & Lightbody, 2005). PM is frequently impaired in patients with neurological disorders (Shum, Valentine, & Cutmore, 1999). Stroke patients have particular problems with both the content and the retrieval of PM tasks. PM deficits following stroke (Brooks, Rose, Potter, Jayawardena, & Morling, 2004; Kim, Craik, Luo, & Ween, 2009) have been studied much less frequently, despite the high prevalence of stroke in the adult population (Hachinski, 2007) and the significant impact of stroke on frontal executive function. Failures in PM could affect independent living, and thus, an accurate assessment of PM is a priority in the process of neuropsychological rehabilitation (Fleming et al., 2005). To date, there are various types of tests measuring PM, but there are few neuropsychological tests available for a large Chinese-speaking population, especially in the PM domain. It is important that neuropsychological tests developed in the West can be properly adapted and translated for use in Eastern cultures. Their psychometric properties should be established before application to clinical practice or research. Measures of PM ability may include self-reported questionnaires, such as the Prospective and
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
897
Retrospective Memory Questionnaire (PRMQ; Smith, Della Sala, Logie, & Maylor, 2000) and the Comprehensive Assessment of Prospective Memory (CAPM; Roche, Fleming, & Shum, 2002). However, validity in measuring PM using questionnaires has been questioned (Uttl & Libreab, 2011). They may also not reflect accurate PM ability if participants have limited insight. Available objective PM tests include the Royal Prince Alfred Prospective Memory Test (RPA-ProMem; Radford, Lah, Say, & Miller, 2011), the Memory for Intentions Screening Test (MIST; Raskin, 2009), and the Cambridge Test of Prospective Memory (CAMPROMPT; Wilson et al., 2005). MIST uses a real-world task and allows a 24-hour delay in testing PM function, but requires a lengthened assessment period and has no alternative form available to minimise practice effects. Similar to MIST, RPA-ProMem may be difficult to administer, especially the long-term PM task. The CAMPROMPT is another commercially available prospective memory test which has satisfactory convergent and construct validity, test –retest reliability, as well as normative data for healthy individuals (Wilson et al., 2005). It has two parallel forms (versions A and B) and each has event-based and three time-based PM tasks designed to simulate everyday activities. It takes only 25 minutes to complete the short-form PM task. Thus, this study aimed to develop a Hong Kong Chinese version of CAMPROMPT (CAMPROMPT-HKCV) and to examine its validity, reliability, sensitivity and specificity for people with stroke.
METHOD The present study of the CAMPROMPT-HKCV was divided into three parts: (1) development of a translated Hong Kong Chinese version and evaluation of its linguistic/cultural validity, (2) evaluation of its concurrent validity and reliability, and (3) its sensitivity and specificity.
Development of a translated version Participants. A review panel of four members was recruited. They included three occupational therapists and one clinical psychologist who had all had experience of working in stroke rehabilitation for more than five years. Measures. A questionnaire rating both the instructions and the paper-andpencil worksheets of versions A and B of the original CAMPROMPT and the Hong Kong Chinese version were developed. Procedures. Step 1: After gaining ethical approval from the Hong Kong Polytechnic University, invitation letters and information sheets were
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
898
MAN, CHAN, AND YIP
distributed by mail. Researchers then met the panel members in person to introduce the CAMPROMPT-HKCV and to explain the purposes of the study and the definitions of related terms, if necessary. Step 2: Panel members, whose native language was Chinese, initially translated both the instructions and the paper-and-pencil worksheets of the CAMPROMPT from English into Hong Kong Chinese (CAMPROMPTHKCV). The use of culturally relevant and equivalent materials and linguistic appropriateness were considered in this process. Step 3: Forward and back translations of the content and the instructions were reviewed by bilingual translators from the Department of Chinese and Bilingual Study of the Hong Kong Polytechnic University. The purposes of the study and the definitions of related terms for the test were explained to them in person. The word discrepancies between the translated version and the original were analysed and modifications were made. The data were analysed in order to determine the percentage of agreement. Answers to the openended portions were also analysed. Thus, initial versions A and B of the CAMPROMPT-HKCV were formulated. Step 4: A panel was invited to review again the clarity, relevance and representativeness of the instructions and worksheets in the CAMPROMPT-HKCV. Panel members had to assign ratings on a four-point Likert scale in the closed question section of the questionnaire. Suggestions or comments could be provided in the open-ended question portion. Step 5: After receiving all the questionnaires from the four panel members, the CAMPROMPT-HKCV was then revised. Thus, a 49-item questionnaire that was designed to evaluate the accuracy and equivalence of the translation in terms of clarity, cultural relevancy and representativeness was completed. Ratings on the four-point Likert scale were regrouped and dichotomised into agree (ratings of totally agree and agree) and disagree (ratings of totally disagree and disagree) ratings.
Reliability and validation study Participants. Thirty-three participants with stroke were recruited by a convenience sampling method from a local Kowloon Hospital (Hospital Authority of Hong Kong). The inclusion criteria were: (1) at least one year post-stroke; (2) Chinese speaking; (3) no dementia as indicated by the Chinese version of the Mini Mental State Examination (Chiu, Lee, Chung, & Kwong, 1994), with a cut-off score of 22 or above being used; (4) no severe language comprehension deficits or learning disabilities; (5) no previous neurological or psychiatric illness; (6) no history of substance abuse; and (7) no visual impairment. Of the 36 participants recruited, two were excluded due to not fulfilling the criteria and one additional subject
CAMPROMPT FOR STROKE
899
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
dropped out of the research. As a result, a sample of 33 participants was successfully recruited. Measures. The following measures were used. The Cambridge Prospective Memory Test (CAMPROMPT) drew on ideas developed over a number of years by Wilson and her colleagues (first mentioned in a paper by Kime, Lamb, & Wilson, 1996); extensive pilot studies having been conducted. This is the first standardised test to have been designed to assess prospective memory within an ecologically plausible context (Wilson et al., 2005). In the test, there are six prospective memory tasks for the examinee to carry out in a 25-minute period. The six prospective tasks are cued in two ways: three by time and three by events. The time intervals between being asked to do the task and responding appropriately at the right moment are balanced across cueing conditions. There are two parallel versions of the test, A and B. Examinees are asked to work on a number of distracting paper-and-pencil worksheets, such as a general knowledge quiz or a word-finder puzzle for a 20-minute period. While they are working on the worksheet, they are asked to remember six tasks (time- and eventbased), either during the 20-minute session or shortly after its conclusion. Examinees may use any strategy to remember the tasks, for example, they may write a reminder on the paper provided. Each of the six prospective memory tasks is given a score according to the pattern of response (the timing of the action and the action itself) that is described in the administrative manual. These responses were categorised using the letters A to H. These letters are later transformed into scores of 6, 4, 2, 1, or 0. For instance, if an examinee spontaneously carries out the correct action at the correct time (score A), this is awarded 6 marks; whereas, if an examinee fails to remember that there is something to be done even after being prompted twice, so that the correct task is still not carried out (score H), this is awarded 0 marks. Thus, the raw score for the six tasks can range from 0–36. By comparing a patient’s raw score within his/her IQ band and age group, one can classify the examinee’s prospective memory as impaired, borderline, poor, average, above average or very good. The original versions, A and B, including the on-going tasks (quizzes, exercises), were translated for validation in the first part of the present study. The Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975) was used in the present study to document stroke patients’ global cognitive status and it also served as a screening criterion for subject selection. This test is widely used for assessing cognitive functions both in clinical settings and in research. For the English version, the reported sensitivity was 97.5% and the specificity was 97.3%. The test–retest reliability is .78 and the inter-rater reliability .99. The validated Hong Kong Cantonese version of the MMSE was used for screening purposes (Chiu et al., 1994).
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
900
MAN, CHAN, AND YIP
A cut-off score of 19 or below out of the total possible score of 30 has been used in screening for dementia (Chiu et al., 1998). With reference to the educational level of older people, different cut-off scores are used in the Hong Kong Cantonese MMSE: a score of 22 or below for those with education of more than 2 years, a score of 20 for those with 1–2 years of education and a score of 18 or below for those with no education. The Test of Nonverbal Intelligence–3 (TONI-3; Brown, Sherbenou, & Johnson, 1997). In the present study, form A of the TONI-3 was adopted to test intelligence in stroke patients. It measures a specific component of intelligent behaviour by testing an individual’s ability to solve problems without overtly using language. Each of its two equivalent forms (versions A and B) contains 45 items that are arranged in order of difficulty (Brown et al., 1997). This test is not heavily loaded with linguistic, motoric and cultural elements and is easy to administer. The Rivermead Behavioural Memory Test –Chinese version (RBMT-CV; Li & Man, 2001; Wilson, Cockburn, & Baddeley, 1985) was suggested as a test to examine the concurrent validity of the CAMPROMPT-HKCV in this study. It is a commonly used standardised memory test. It includes nine subtests that have been designed to assess memory skills related to everyday situations. The test is useful to predict memory deficits in everyday life memory tasks for patients who have experienced brain damage through injury or illness (Brooks et al., 2004; Fish, Wilson, & Manly, 2010). One of the four parallel forms (version A) was used in this study to measure accurately memory change over time. In the RBMT-CV, three out of nine items measure eventbased prospective memory, i.e., to deliver a message, to ask for the return of a belonging and the date of an appointment. It is viewed as the most relevant test to assess everyday as well as event-based prospective memory. Procedures. All participants were given an information sheet describing the purpose of the study and signed a Chinese consent form prior to the study. The purposes of the study were explained to them before they were assessed. They were also randomly assigned into three different groups for reliability and validity testing. The administration of the CAMPROMPT-HKCV took place in a distraction-free room. All participants were rated by two testers simultaneously using version A of the test to establish the inter-rater reliability. Ten were assessed with both versions A and B within two weeks to establish parallel form reliability. Another 10 were rated by the same rater using both the CAMPROMPT-HKCV (version A) and the RBMT-CV (version A) on the same day. Ten extra participants were assessed by the CAMPROMPT-HKCV (version A) twice within a two week interval to study its test –retest reliability. The internal consistency of the CAMPROMPT-HKV was tested by analysing the consistency of the time-based and event-based tasks of the CAMPROMPT-HKCV (version A).
CAMPROMPT FOR STROKE
901
Sensitivity and specificity study
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Participants. Another group of 40 stroke patients who met similar inclusion criteria to the prior validity and reliability study were recruited by a convenience sampling method from the same local Kowloon Hospital. In addition, 44 healthy controls without a history of stroke, substance abuse, psychiatric, neurological or major medical disorders, who were approximately matched according to a self-report questionnaire collecting information of age, sex and education, were selected from a vocational training centre (Hong Kong College of Technology). Measures. The CAMPROMPT-HKCV (version A) developed in the validity and reliability study was adopted for testing the two groups’ PM abilities. Procedures. Ethical approval was granted before recruitment. Again both the stroke and healthy participants were given an information sheet about the study and written consent was obtained before testing.
RESULTS Translation of the CAMPROMPT-HKCV Overall, the percentage of agreement between the panel members in the dichotomised scale in terms of clarity, cultural relevancy, representativeness and overall comments were 91.8%, 89.8%, 100% and 100%, respectively. There were few disagreements on clarity (4 out of 49 questions) and cultural relevancy (5 out of 49 questions). The instructions were revised based on the comments of the panel members, which resulted in the finalisation of a revised CAMPROMPT-HKCV.
Validation and reliability of the study A total of 33 participants with stroke were tested: 22 males (67%) and 11 females (23%). The age range was 57 –65 years, with a mean of 53.48 years (SD ¼ 7.58). Total MMSE-CV scores ranged from 23–29 out of a total of 30 (mean ¼ 26.8, SD ¼ 2.40), and thus, participants did not have significant cognitive impairment. Participants’ TONI-III quotient (similar to IQ) scores ranged from 69 –102 (mean ¼ 84.9). Most scored in the ranges of 81– 90 (55.6%) and 71 –80 (22.2%). Their mean of RMBT-CV was 17.3 (SD ¼ 4.70; range 14–24). Table 1 shows the performance of the full sample on
902
MAN, CHAN, AND YIP TABLE 1 Item scores in Form A of CAMPROMPT-HKCV in reliability and validity study
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
(N ¼ 33) Time-based task Change task Take key Ring garage Total time-based Event-based task Book/map Give message Object locations Total event-based
Mean
SD
Range
1.35 2.09 2.48 5.91
1.191 1.905 2.192 3.655
0 –4 0 –6 0 –6 1 –16
3.17 3.96 3.83 10.96
1.992 2.286 1.775 4.791
0 –6 0 –6 0 –6 0 –18
Figure 1. Trend of mean subscores of CAMPROMPT-HKCV across time in reliability and validity study.
version A of the CAMPROMPT-HKCV. Figure 1 also shows the trend of mean scores of stroke patients taking CAMPROMPT-HKCV across time. The internal consistency, using the Cronbach’s alpha results, showed that the time-based score was lower (.493) than that for the event-based tasks (.693). The deletion of any items in the time-based or event-based scales, however, was found to cause negligible changes in the Cronbach’s alpha. It was also found that the mean score of the event-based tasks was higher, and nearly twice the mean score of the time-based tasks among the 33 participants. The test–retest reliability on time-based, event-based and total scores was considered acceptable and showed good reliability (ICC ¼ 0.809, p ¼ .047; ICC ¼ 0.890, p ¼ .015; and ICC ¼ 0.869, p ¼ .022, respectively). A lack of
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
903
variability in the test scores secondary to the homogeneous sample may reduce the test–retest reliability coefficient. In this study, the betweensubject variance in the test scores for time-based (F ¼ 5.231, p , .05), event-based (F ¼ 9.104, p , .05) and total score performance (F ¼ 7.651, p , .05) were statistically significant, which indicated that there was a great variability in the test scores. The high ICCs for the inter-rater reliability for versions A and B of the CAMPROMPT-HKCV were 0.959 (p , .001) and 0.983 (p , .001), respectively (Gerlinas, Gauthier, McIntyre, & Gauthier, 1999). The correlation between versions A and B (r ¼ .635, p ¼ .05) of the CAMPROMPT-HKCV was statistically significant. Paired samples t-test comparing the version A and version B scores of CAMPROMPT-HKCV showed that one performance was not significantly different from the other, t(8) ¼ 1.779, p ¼ .109.
Concurrent validity Only 10 participants were assessed with both version A of the CAMPROMPT-HKV and version A of the RBMT-CV on the same day due to difficulties in recruiting more clinical participants. The overall total score of each test was calculated and used to investigate the correlation between the two tests. In the RBMT-CV, the standardised profile score was used to analyse the results. The maximum of the standardised profile score in the RBMT-CV was 24 and the participants’ mean score was 17.6 (range 10– 23). On the other hand, the score of the CAMPROMPT-HKCV (maximum ¼ 36) ranged from 10–30 with a mean score of 19.1. The standardised profile score of the RBMT-CV and the overall total score of the CAMPROMPT-HKCV were compared and generated a correlation result. High to excellent correlation (Spearman rank-order correlation rho ¼ .868, p , .05) was found. Further analysis showed that there was significant correlation between the event-based CAMPROMPT-HKCV score and the RMBT-CV total score (Spearman’s rho correlation coefficient ¼ .751; p ¼ .03) and RMBT-CV event-based items scores (Spearman’s rho correlation coefficient ¼ .632; p ¼ .021), respectively. For the time-based CAMPROMPT-HKCV score, the Spearman’s rho correlation coefficient to RBMT total score and RBMT-CV event-base items score were .443 (p ¼ .13) and .59 (p ¼ .034), respectively. Thus, except CAMPROMPT-HKCV (time-based score), both the total and event-based scores of CAMPROMPT-HKCV significantly correlated with both RBMT-CV total and RBMT-CV event-based score. On one hand, CAMPROMPT-HKCV is equally valid in testing event-based PM as the RBMT-CV, on the other, CAMPROMPT-HKCV also tests time-based PM performance which is not tested in the RBMT-CV.
904
MAN, CHAN, AND YIP
PM performance on CAMPROMPT-HKCV As noted, scores for each of the PM tasks are provided in Table 1. It can be seen that overall, time-based tasks were not performed as well as event-based tasks. The lowest scoring task was the time-based task of asking participants to change task after seven minutes. The task on which participants scored highest was the event-based task of giving a message envelope when five minutes was left.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Sensitivity and specificity Forty-four healthy controls without a history of stroke, substance abuse or psychiatric, neurological or major medical disorders that were approximately matched according to age, sex and education were selected from a local vocational training centre (Hong Kong College of Technology). Forty patients who met similar inclusion criteria as the validation and reliability study were recruited by a convenience sample of patients from Kowloon Hospital (Hospital Authority of Hong Kong). Table 2 shows that the age of the control group (mean ¼ 41.86 years; SD ¼ 9.32) was significantly younger than the stroke group (mean ¼ 50.13, SD ¼ 10.29, p , .001). There was also a significant difference in the distribution of gender in the two groups (x2 ¼ 17.253, p , .001). To compare the differences in their performances on the total score of CAMPROMPT-HKCV, we adjusted for age and gender differences. ANCOVA showed that there was no significant interaction in the performance of PM between group and age, F(1, 78) ¼ 0.576, p ¼ .45; between group and gender, F(1, 78) ¼ 0.154, p ¼ .696; between age and gender, F(1, 78) ¼ 2.223, p ¼ 0.14; and among group, gender and age, F(1, 78) ¼ 0.128, p ¼ .722. After eliminating the interaction among the factors, there was a significant difference in the performance on the total score of CAMPROMPT-HKCV between the stroke group and the control group with adjustments for age and gender, F(1, 77) ¼ 32.9, p , .001. ROC analysis showed that total, event-based and time-based CAMPROMPT-HKCV cut-off scores could differentiate PM problems in stroke patents. Good trade-off between sensitivity and specificity was also shown (See Table 3). The area under curve (AUC) for total, event-based and timebased scores of CAMPROMPT-HKCV were 0.911, 0.843, and 0.894, respectively (see Figure 2).
DISCUSSION To date, this study is the first to develop a translated CAMPROMPT-HKCV and to investigate its psychometric properties for the stroke population. The process of translation is reported. In addition, the concurrent validity,
905
CAMPROMPT FOR STROKE TABLE 2 Demographics of stroke and healthy group in specificity and sensitivity study
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Stroke (N ¼ 40)
Healthy (N ¼ 44)
(SD)
Age Years of education CAMPROMPT-HKCV total score Event-based event total score Time-based event total score
50.13 10.85 20.74
(10.30) (4.35) (6.91)
41.86 10.55 29.61
(9.32) (1.45) (4.17)
.001 .675 .001
11.00
(3.52)
14.95
(2.23)
.001
7.55
(4.43)
14.66
(3.01)
.001
MMSE-CV1 Estimated IQ2
28.3 85.4
(2.20) (16.00)
90.68(14.19)
n
(%)
n
(%)
p-value∗∗
25.00 15.00 19.00 21.00 26.00 14.00
(62.50) (37.50) (47.50) (52.50) (65.00) (35.00)
8.00 36.00
(18.18) (81.82)
.001
Gender Type of stroke Paretic side
Male Female Ischaemic Haemorrhagic Left Right
Mean
p-value∗
Mean
(SD)
.113
∗
1 2
by independent t-test; ∗∗ by chi-square test. Mini Mental State Examination- Chinese version Estimated IQ based on Test of Non-verbal Intelligence-III (Toni-III; Brown et al., 1997)
reliability (internal consistency, test–retest, and parallel form), specificity and sensitivity in screening stroke patients with PM difficulties and healthy controls have been studied. The high percentage of agreement between the four panel members regarding the clarity, cultural relevancy and representativeness of the initial translation supported its quality. The CAMPROMPT-HKCV also demonstrated
TABLE 3 Cut-off scores, sensitivity and specificity of scores for CAMPROMPT-HKCV CAMPROMPT-HKCV Total score Event-based Time-based
Cut-off score
Sensitivity
Specificity
20.5 13 8.5
95.50% 86.40% 97.70%
63.20% 68.40% 63.20%
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
906
MAN, CHAN, AND YIP
Figure 2. ROC curve of COMPROMPT-HKCV in specific and sensitivity study.
a good reliability over time and consistency among raters and between the two versions. Internal consistency of the time-based score was much lower than that for event-based tasks within the CAMPROMPT-HKCV. It was suggested that the three time-based tasks might not be similar, thus producing different memory demands on the stroke patients. Reviewing and changing these three tasks might be necessary to further improve its internal consistency. Moreover, the observed higher mean score of the event-based than time-based tasks echoed Einstein, McDaniel, Richardson, Guynn, and Cunfer’s (1995) suggestion that time-based tasks may be more difficult to remember than event-based tasks because the passage of time must be monitored and the remembering is self-initiated, whereas event-based tasks were cued by an external stimulus. If the time-based PM was more dependent on internal
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
907
cues, it seemed to have a higher level of difficulty for stroke clients in the present study. Einstein et al. (1995) also suggested that event-based prospective remembering may actually rely on a more spontaneous or automatic component, at least compared with time-monitoring tasks. An event itself can act as an external cue in the environment and can make the event-based task easier. Although similar, good test –retest reliability of time-based, event-based and total scores of CAMPROMPT-HKCV are reported; the learning effect and the short 2-week test –retest interval were noted. It was found that participants were able to recall some testing procedures performed in the initial assessment, which indicated that a learning effect did occur. According to Gregory (1999), changes over time are the major source of error in test– retest reliability. The change may result from a real change in the traits of participants, unstable traits, testing effects, the test–retest interval, motivational factors of participants, or rater bias (Gregory, 1999; Portney & Watkins, 1993). As the participants in this study were stable with minimal cognitive fluctuation, the possibility of a dramatic change was unlikely during the test and retest period. The variation of scores in the test –retest may not be due to any real change in traits or unstable traits in the participants. The inter-rater reliability for versions A and B in our study was consistent with the original version of the CAMPROMPT (Wilson et al., 2005). This indicates that the scoring criteria of the CAMPROMPT-HKCV were clear to raters and well written for its administration. The moderate to good correlation between parallel versions A and B of the CAMPROMPT-HKCV suggested that they were not significantly different. Excellent correlation might have resulted from a larger sample size for parallel form reliability. With reference to the results of the original CAMPROMPT (Wilson et al., 2005), one of the CAMPROMPT-HKCV versions can be given to a subject without the results being affected. In establishing concurrent validity, the RBMT-CV was used as the gold standard by which the CAMPROMPT-HKCV was analysed. According to the results, the correlation in concurrent validity was reported to be high to excellent. Three out of the nine tasks in the RBMT-CV assessed PM function, the other six measured general memory skills such as delayed recall, immediate recall, working memory and orientation for date and place. The RBMTCV was not specifically designed to measure PM, but it might be one of the few objective assessments that has the most PM components to compare with the newly developed CAMPROMPT-HKCV. Moreover, the RBMT-CV is the only memory test that has been translated into a Hong Kong Chinese version and widely used in a Hong Kong clinical setting. Despite these limitations, the RBMT-CV was still chosen as the tool to determine the concurrent validity; however, it should be noted that the RBMT-CV was a better, but not perfect, standardised assessment to assess the
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
908
MAN, CHAN, AND YIP
participants’ PM function and pragmatically used to establish the CAMPROMPT-HKCV’s concurrent validity. On the other hand, the event-based score of CAMPROMPT-CV was found to correlate significantly with the total and event-based RBMT-CV scores, suggesting that it is valid in measuring PM, especially event-based PM. Although total and time-based scores of CAMPROMPT-HKCV only correlated significantly with event-based but not time-based RBMT-CV, it seems to hint that CAMPROMPT-HKCV has added value as a PM assessment in measuring comprehensively both eventbased and time-based PM functions while having similar merit to RBMTCV in being an ecologically valid memory assessment relevant to everyday functioning. Moreover, as both the Hong Kong Chinese and Chinese Mainland versions (Lou, Dou, Zheng & Man, 2009) have already been translated, including instructions and worksheets (as ongoing tasks during assessment), application to a larger Chinese-speaking stroke population becomes a possibility. The poorer performance of carrying out a PM task to change their task (in version A) or pen (in version B) after seven minutes might be explained by the fact that stroke participants needed to be aware of the instruction, “seven minutes later”. This instruction was an indirect indication of time. It seemed that the participants had difficulty in fully comprehending this instruction, became confused and performed the task badly. In addition, this task required more cognitive processing as the participants needed to calculate the approximate timing of seven minutes and, at the same time, check the timer through self-initiation. It is possible that this time-based task was more challenging for the participants. Therefore, this specific time-based task appeared to be the most difficult one of the six tasks. On the other hand, the event-based task of giving an envelope when five minutes was left scored the highest. A card with written instructions was given to the participants and could be used by them as a visual cue. In carrying out this assessment, several factors are worth noting as they might affect the participants’ performance, and thus, the scores and interpretation of the results. These factors might include the participants’ attention, interferences from the paper quiz, the verbal instructions given during the assessment, the level of difficulty of the tasks and the sequence in which the instructions were given. As reported earlier, time-based PM scores were lower than any of the event-based PM scores, thus time-based tasks may be more difficult than event-based tasks. Interruptions that engage participants in a new task for as little as 15 seconds have been shown to interfere with prospective memory (Einstein, McDaniel, Williford, Pagan, & Dismukes, 2003; see also Dodhia & Dismukes, 2009). Prospective memory might have suffered when the demands of the ongoing task were increased or when attention was divided with a task that engages central executive resources (Marsh & Hicks, 1998). Based on these findings, it seems that both interference
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
909
between tasks and interference with the participants’ attention may adversely affect performance, and thus, participants’ scores. The fact that interference was increasing and attention possibly decreasing while the assessment was in progress, suggests that participants’ scores might drop for a particular task, according to how late in time each task was to be executed. However, our findings contradicted this postulation. According to Figure 1, the scores varied instead of declining across the assessment period. This might be explained by the level of difficulty of the task itself having more influence on the scores than attention or interference. Based on the assessment results, the scores were rated by therapists on a 5-point scale (scores of 6, 4, 2, 1, and 0). The ratings depended on whether the participants spontaneously carried out the tasks and the number of times that they were prompted. As scoring was based on a prompting system, participants who made a guess regarding the task would receive higher marks, whereas participants who gave up would score zero. Correct guessing might lead to a random effect. It might create a deviation in the results that did not actually represent the prospective memory performance of the participants. During the assessment using the CAMPROMPTHKCV, the participants needed regularly to check a digital clock. Participants who were diagnosed with stroke may have visual problems that could lead to a poorer performance in the assessment. This assessment might not be easy to generalise to other stroke patients who have visuo-perceptual problems while reading the clock. The sensitivity and specificity findings showed that the CAMPROMPTHKCV had good discrimination ability for PM problems. It is suggested that the test can differentiate between stroke patients and healthy controls. In addition, among the three scores, the CAMPROMPT-HKCV total score was the most informative and is suggested for clinical screening use. The importance and implications of the above findings have been identified. The test items in CAMPROMPT-HKCV resemble long-term PM that is required in daily life. The test results are representative of daily life PM performance and increase the ecological validity of the CAMPROMPT-HKCV. In future, its predictive validity of daily functioning could be further examined to obtain more proven evidence. Both event-based and time-based PM can be assessed with the CAMPROMPT-HKCV, instead of relying only on RBMT-CV which mainly assesses the memory process and retrospective memory. CAMPROMPT-HKCV is also sensitive in picking up PM problems in stroke patients, especially in time-based tasks. Its ability to differentiate stroke patients from healthy controls using a cut-off score of 20.5/30 has additional merits in its development into a screening tool for early problem identification during acute or subacute stroke rehabilitation. As there are few standardised PM test batteries that have been translated and adopted in a Chinese context, the CAMPROMPT-HKCV is considered attractive to
910
MAN, CHAN, AND YIP
psychologists. Its satisfactory psychometric properties may also appeal to rehabilitation personnel in serving an increasing Chinese-speaking stroke population.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
Limitations and suggestions There were several limitations in this study. The convenience sampling, the small sample size and the homogeneity of the sample reduced the generalisability of the results. Therefore, the recruitment of more participants from multi-centres by randomisation is suggested. Evidence from a number of studies indicates that older adults often perform poorly relative to younger adults on tasks that require prospective remembering (Maylor, 1993, 1996). It is assumed that there will be some decline in prospective memory function during healthy ageing; comparisons can be made between healthy elderly participants and elderly participants with stroke in future studies. Parallel form reliability was not assessed with a counterbalancing method, which can be a confounder, particularly when only 10 patients were used to assess reliability in this study. Some stroke patients had difficulties with clockchecking during testing. Their individual difficulties might be accommodated by a verbal request to the assessor. This may be another way of measuring of PM ability. In the original CAMPROMPT, normative data were calculated based on premorbid IQ and age. In future, normative data of CAMPROMPTP-HKCV in relation to age, gender and IQ should be developed for stroke and other disease populations.
CONCLUSION The Cambridge Prospective Memory Test (CAMPROMPT) was translated and validated in this study. The psychometric properties of the CAMPROMPT-HKCV were explored and, despite the methodological limitations, it was found that the translated version was reliable. This implies that the CAMPROMPT-HKCV has the potential to be a valid and reliable tool for screening and/or assessing prospective memory for Hong Kong and other Chinese-speaking patients with stroke. Further study in developing stratified norms across different age groups in stroke patients is recommended, so as to improve the measurement of the rate of change and decision making in clinical neuropsychological rehabilitation.
REFERENCES Brooks, B. M., Rose, F. D., Potter, J., Jayawardena, S., & Morling, A. (2004). Assessing stroke patients’ prospective memory using virtual reality. Brain Injury, 18(4), 391 – 401.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
CAMPROMPT FOR STROKE
911
Brown, L., Sherbenou, R. J., & Johnsen, S. K. (1997). Test of Nonverbal Intelligence: A language-free measure of cognitive ability (3rd ed.). Austin, TX: Pro-ed. Chiu, H. F. K., Lam, L. C. W., Chi, I., Leung, T., Li, S. W., Law, M. T., . . . & Lau, J. (1998). Prevalence of dementia in Chinese elderly in Hong Kong. Neurology, 50, 1002– 1009. Chiu, H. F. K., Lee, H. C., Chung, W. S., Kwong, P. K. (1994). Reliability and validity of the cantonese version of mini-mental state examination. Journal of Hong Kong College of Psychiatry, 4, 25 – 28. Dodhia, R. M., & Dismukes, R. K. (2009). Interruptions create prospective memory tasks. Applied Cognitive Psychology, 23(1), 73 –89. Einstein, G. O., & McDaniel, M. A. (1990). Normal aging and prospective memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 717 – 726. Einstein, G. O., McDaniel, M. A., Richardson, S. L., Guynn, M. J. & Cunfer, A. R. (1995). Aging and prospective memory: Examining the influences of self-initiated retrieval processes. Journal of Experimental Psychology: Learning Memory, and Cognition, 21(4), 996 – 1007. Einstein, G. O., McDaniel, M. A., Williford, C. L., Pagan, J. L., & Dismukes, R. K. (2003). Forgetting of intentions in demanding situations is rapid. Journal of Experimental Psychology: Applied, 9(3), 147 – 162. Ellis, J., & Kvavilashvili, L. (2000). Prospective memory in 2000: Past, present, and future directions. Applied Cognitive Psychology, 14, S1– S9. Fish, J., Wilson, B. A., & Manly, T. (2010). The assessment and rehabilitation of prospective memory problems in people with neurological disorders: A review. Neuropsychological Rehabilitation, 20(2), 161 – 179. Fleming, J. M., Shum, D., Strong, J., & Lightbody, S. (2005). Prospective memory rehabilitation for adults with traumatic brain injury: A compensatory training programme. Brain Injury, 19, 1 – 10. Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189 – 198. Gerlinas, I., Gauthier, L., McIntyre, M., & Gauthier, S. (1999). Development of a functional measure for persons with Alzheimer’s disease: The disability assessment for dementia. American Journal of Occupational Therapy, 53, 471 – 481. Gregory, R. J. (1999). Psychological testing: History, principles and applications (3rd ed.). Boston, MA: Allyn and Bacon. Hachinski, V. (2007). The 2005 Thomas Willis Lecture: Stroke and vascular cognitive impairment: A transdisciplinary, translational and transactional approach. Stroke, 38, 1396 – 1403. Henry, J. D., MacLeod, M. S., Philips, L. H., & Crawford, J. R. (2004). A meta-analytic review of prospective memory and aging. Psychology and Aging, 19(1), 27 – 39. Li, R., & Man, W. K. (2001). Assessing Chinese adult’s memory abilities: Validation of the Chinese version of the Rivermead Behavioural Memory Test. Clinical Gerontology, 24(3 – 4), 27 – 36. Lou, Z. L., Dou, Z. L., Zheng, J. L., Chen, Y. B., & Man, D. W. K. (2009). The study of the Chinese version of Cambridge Prospective Memory Test (CAMPROMPT) for traumatic brain injury (unpublished Master thesis). Guangzhou: Sun Yat Sen University. Kim, H. J., Craik, F. I. M., Luo, L., & Ween, J. E. (2009). Impairment in prospective and retrospective memory following stroke. Neurocase, 15(2), 145 – 156. Kime, S. K., Lamb, D. G., & Wilson, B. A. (1996). Use of a comprehensive program of external cuing to enhance procedural memory in a patient with dense amnesia. Brain Injury, 10, 17 – 25.
Downloaded by [University of Lethbridge] at 03:37 21 October 2015
912
MAN, CHAN, AND YIP
Marsh, R. L., & Hicks, J. L. (1998). Event-based prospective memory and executive control of working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24(2), 336 – 349. Maylor, E. A. (1993). Aging and forgetting in prospective and retrospective memory tasks. Psychology & Aging, 8, 420 – 428. Maylor, E. A. (1996). Age-related impairment in an event-based prospective-memory task. Psychology and Aging, 11, 74 – 78. McDaniel, M. A., & Einstein, G. O. (2000). Strategic and automatic processes in prospective memory retrieval: A multiprocess framework. Applied Cognitive Psychology, 14, S127 – S144. Portney, L. G., & Watkins, M. P. (1993). Foundations of clinical research: Applications to practice. Norwalk, CT: Appleton & Lange. Radford, K. A., Lah, S., Say, M. J., & Miller, L. A. (2011). Validation of a new measure of prospective memory: The Royal Prince Alfred Prospective Memory Test. Clinical Neuropsychologist, 25(1), 127 – 140. Raskin, S. (2009). MIST: Psychometric properties and clinical evidence. Brain Impairment, 10(1), 23 – 33. Roche, N., Fleming, J., & Shum, D. (2002). Self-awareness of prospective memory failure in adults with traumatic brain injury. Brain Injury, 16, 931 – 946. Shum, D., Valentine, M., & Cutmore, T. (1999). Performance of individuals with severe longterm traumatic brain injury on time-, event-, and activity-based prospective memory tasks. Journal of Clinical & Experimental Neuropsychology, 21(1), 49 – 58. Smith, G., Della Sala, S., Logie, R.H., & Maylor, E. A. (2000). Prospective and retrospective memory in normal ageing and dementia: A questionnaire study. Memory, 8(5), 311 – 321. Uttl, B., & Libreab, K. (2011). Self-report measures of prospective memory are reliable but not valid. Canadian Journal of Experimental Psychology, 65(1), 57– 58. Wilson, B. A., Cockburn, J., & Baddeley, A. (1985). The Rivermead Behavioural Memory Test. Bury St. Edmunds, UK: Thames Valley Test Co. Wilson, B. A., Emslie, H., Foley, J., Shiel, A., Watson, P., & Hawkins, K. (2005). The Cambridge Prospective Memory Test. London: The Psychological Corporation.
