530776
research-article2014
ASMXXX10.1177/1073191114530776AssessmentDonders and Strong
Article
Clinical Utility of the Wechsler Adult Intelligence Scale–Fourth Edition After Traumatic Brain Injury
Assessment 2015, Vol. 22(1) 17–22 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1073191114530776 asm.sagepub.com
Jacobus Donders1 and Carrie-Ann H. Strong1
Abstract The performance of 100 patients with traumatic brain injury (TBI) on the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) was compared with that of 100 demographically matched neurologically healthy controls. Processing Speed was the only WAIS-IV factor index that was able to discriminate between persons with moderate-severe TBI on the one hand and persons with either less severe TBI or neurologically healthy controls on the other hand. The Processing Speed index also had acceptable sensitivity and specificity when differentiating between patients with TBI who either did or did not have scores in the clinically significant range on the Trail Making Test. It is concluded that WAIS-IV Processing Speed has acceptable clinical utility in the evaluation of patients with moderate-severe TBI but that it should be supplemented with other measures to assure sufficient accuracy in the diagnostic process. Keywords assessment, processing speed, sensitivity, specificity Measurement of intellectual functioning is a routine component of neuropsychological assessment. The most commonly used intellectual test in neuropsychology is the Wechsler Adult Intelligence Scale (Loring & Bauer, 2010). The most recent revision of the test, the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV; Wechsler, 2008) retains some aspects of its predecessor, the Wechsler Adult Intelligence Scale–Third Edition (Wechsler, 1997). However, there have been considerable changes in the test’s content and administration. These include the removal and replacement of some subtests, a de-emphasis on speeded performance bonuses, and a reduction in all of the subtests’ discontinuation rules. Moreover, it eliminated the Verbal and Performance IQs of prior editions and focused interpretation on the four factor index scores, that is, Verbal Comprehension, Perceptual Reasoning, Working Memory, and Processing Speed. Previous research has established an empirical foundation for using the WAIS-IV’s predecessor in the evaluation of patients with neurological impairment, especially those with traumatic brain injury (TBI), which is one of the most common forms of acquired neurological impairment (Axelrod, Fichtenberg, Liethen, Czarnota, & Stucky, 2001; Donders, Tulsky, & Zhu, 2001; Martin, Donders, & Thomson, 2000). This research has consistently distinguished Processing Speed as the index that is most sensitive to TBI. However, because substantive changes have been made to the test, some have warned against assuming that
the WAIS-IV is analogous to the WAIS-III in terms of clinical utility with neurological populations (Loring & Bauer, 2010). There is a growing literature on the WAIS-IV relevant to neuropsychological practice. Several factor analytic studies have supported the four-factor model but with need for consideration of the overall impact of “g” or general intelligence (Canivez & Watkins, 2010; Holdnack, Zhou, Larrabee, Millis, & Salthouse, 2011; Nelson, Canivez, & Watkins, 2013), although some other studies have suggested some variations in factor structure along specific theoretical dimensions (Benson, Hulac, & Kranzler, 2010; Ward, Bergman, & Hebert, 2012). In addition, there have been several studies that have demonstrated that there is considerable WAIS-IV subtest variability and scatter in normal populations, often exceeding two standard deviations (Binder & Binder, 2011; Heyanka, Holster, & Golden, 2013). However, few independent studies specifically evaluating the clinical utility of the WAIS-IV in specific diagnostic groups have been published. The WAIS-IV performance of patients with multiple sclerosis (MS) was investigated in a recent study (Ryan, 1
Mary Free Bed Rehabilitation Hospital, Grand Rapids, MI, USA
Corresponding Author: Jacobus Donders, Psychology Service, Mary Free Bed Rehabilitation Hospital, 235 Wealthy, S.E., Grand Rapids, MI 49503, USA. Email: [email protected]
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
18
Assessment 22(1)
Gontkovsky, Kreiner, & Tree, 2012). There were statistically significant group differences between patients with MS and demographically matched controls, but Processing Speed was the only WAIS-IV index that was predictive of group membership. The cognitive performance of patients with schizophrenia has also been examined (Michel et al., 2013). Statistically significant group differences between patients with schizophrenia and demographically matched controls were found on the Working Memory and Processing Speed indices, with the greatest divergence on the latter score. To our knowledge, no independent studies have yet been published that investigated the clinical utility of the WAIS-IV in patients with TBI. The WAIS-IV manual does include a study of patients with TBI, though with a very small sample size (n = 22; Wechsler, 2008). Mean group differences with large effect sizes between the TBI and control groups were evident for Perceptual Reasoning, Processing Speed, and Full Scale IQ, with the greatest difference on Processing Speed. Brooks, Holdnack, and Iverson (2011) also used data from the TBI group of the standardization of the WAIS-IV to identify cognitive impairments, using base rate information. When applying base rates, the patients with moderate to severe TBI were more likely to have a “low cognitive profile” in comparison to normal healthy controls. The current study was undertaken to investigate the clinical utility of the WAIS-IV in the assessment of patients with TBI. We were particularly interested in the instrument’s sensitivity to injury severity and acquired impairment, which were considered to be important considerations in clinical evaluations of patients in a health care setting for differential diagnostic and treatment planning purposes. We focused primarily on the factor index scores because they are more reliable than subtest scores. However, one additional area of interest included exploration of the new Perceptual Reasoning subtest “Visual Puzzles,” because it places a greater emphasis on speed of performance than some of the other subtests. Based on prior studies, we hypothesized that the Processing Speed index would have the greatest sensitivity to TBI. It was decided a priori that in order to be clinically useful, the index scores WAIS-IV would have to meet the following criteria: (a) a statistically significant difference between persons with TBI and demographically matched controls and (b) a statistically significant correlation with length of coma. These criteria were chosen, consistent with previous research with different instruments (Donders & Janke, 2008; Donders & Levitt, 2012) and because length of coma is one of the most commonly used determinants of injury severity after TBI (Stucky, Kirkwood, & Donders, 2014). Furthermore, in order to have validation against an independent instrument, a third criterion was (c) sufficient classification accuracy of cognitive impairment, determined
by the WAIS-IV’s ability to distinguish between participants with TBI who did versus did not have scores in the clinically significant range on the Trail Making Test (TMT; Reitan & Wolfson, 1993). This test was chosen for validation purposes because it is a widely used instrument that is sensitive to acquired neuropsychological impairment in a wide range conditions, ranging from acute delirium (Pandharipande et al., 2013), to HIV-related cognitive impairment (uñoz-Moreno et al., 2013), to development of Alzheimer’s disease (Weissberger, Salmon, Bondi, & Gollan, 2013), and to sequelae of TBI in particular (Bagiella et al., 2010; Cullen, Krakowski, & Taggart, 2014; Martin, Hoffman, & Donders, 2003). We used a conventional minimum standard for clinical decision making, defined as a likelihood ratio (sensitivity/[1 − sensitivity]) ≥ 2 (Grimes & Schulz, 2005).
Method Research Participants Approval for this investigation was obtained from the Research Institutional Review Board at Mary Free Bed Rehabilitation Hospital, and all data included in this article were obtained and analyzed in compliance with the regulations of this board as well as in accordance with the Helsinki Declaration (http://www.wma.net/en/30publications/10policies/b3/). Two groups of participants were included in this study: a clinical group of patients with TBI and a group of demographically matched controls. The 100 clinical participants were selected from an approximately 4-year consecutive series of referrals for outpatient neuropsychological evaluation, according to the criteria listed below. Data collection continued until there were 100 patients who met all of the inclusion criteria. A control group (n = 100) was subsequently obtained, with permission from the publisher, from the standardization sample of the WAIS-IV. The control participants were matched to the clinical participants on age and education, and when possible also on gender and ethnicity. None of the control participants had any kind of neurological, psychiatric, or special education history. Demographic characteristics of both groups are presented in Table 1. The following criteria were used to select the clinical participants: (a) diagnosis of TBI, defined as an acute, external force to the head with alteration of consciousness; (b) age between 16 and 65 years; (c) absence of any prior neurological, psychiatric, special education, or substance abuse history; (d) evaluation with the WAIS-IV within 1 to 12 months after injury; and (e) acceptable performance on a stand-alone performance validity test, such as the Test of Memory Malingering (Tombaugh, 1996) or the Word Memory Test (Green, 2003). Alteration of consciousness was determined by means of review of medical records for
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
19
Donders and Strong Table 1. Demographic Characteristics and WAIS-IV Performances of Patients With TBI (n = 100) and Standardization Controls (n = 100). Variable Age (years; M, SD) Education (n) 15 years Gender (n) Female Male Ethnicity (n) Caucasian Latina/o Other
TBI
Controla
32.56 (14.78)
32.65 (14.75)
26 31 25 18
17 35 28 20
36 64
39 61
91 5 4
82 13 5
Note. WAIS-IV = Wechsler Adult Intelligence Scale–Fourth Edition; TBI = traumatic brain injury. a Standardization data derived from the WAIS-IV. Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved.
any documentation that the individual was dazed, confused, engaging in repetitive questioning, or not responding to commands at either the accident scene or in the emergency room. During the time period that these data were collected, the WAIS-IV and TMT had been included routinely in outpatient neuropsychological evaluations of patients with TBI, unless there were circumstances that would have compromised test validity (e.g., non-English speaking language background, severe orthopedic injury to the dominant hand). Only first evaluations, no repeat evaluations, were included. The participants with TBI were seen for evaluation with the WAIS-IV at a median of 122 days post injury (M = 143.23, SD = 87.73, range = 30-355). Injury circumstances reflected primarily motor vehicle collisions (64%), falls and recreational activities (28%), and various other causes (8%). Several measures of injury severity were considered. Exact initial Glasgow Coma Scale scores were not always available, and duration of posttraumatic amnesia would have required retrospective estimation in some cases, which was considered unreliable. However, information about the time to follow commands and with regard to the presence or absence of acute abnormalities on neuroimaging was available for all participants with TBI. In the complete group with TBI, median duration of time to follow commands was 0 days (M = 1.58, SD = 4.20, range = 0-28), with this duration being ≥24 hours in 29 participants. Fifty-eight of the participants with TBI had positive neuroimaging findings, including 57 with focal lesions (e.g., hemorrhagic contusion) and 22 with diffuse lesions (e.g., edema), with some participants having multifocal or a combination of focal and diffuse
lesions. A total of 42 participants had uncomplicated mild TBI, defined as duration of time to follow commands .53. There were also no statistically significant correlations between time span since injury and performance on any of the WAIS-IV factor indices (all ps > .06).
Procedure The WAIS-IV and TMT had been administered to clinical patients as part of outpatient neuropsychological evaluations, which were scheduled only when they were medically stable and could recall meaningful information from day to day. All evaluations had originally been completed for clinical reasons with informed consent. Only the 10 subtests that are necessary to compute the four WAIS-IV factor index scores were routinely included in these evaluations. Standard scores (M = 100, SD = 15) for the factor indices and scaled scores (M = 10, SD = 3) for the subtests were used in the statistical analyses. To balance the relative risks of Type I and Type II errors, the stepdown Bonferroni method (Holm, 1979) was used to adjust alpha in any post hoc comparisons. We also specified a priori a minimum acceptable effect size of .05 (η2 for group contrasts; ρ2 for correlations). Because of the skewed distribution of the days to follow commands, Spearman instead of Pearson correlations were used.
Results The average WAIS-IV factor index scores are presented in Table 2 for the three TBI severity groups and the demographically matched control group. For illustrative purposes, the subtest profiles are presented in Figure 1. Because we had predicted a priori that Processing Speed would stand out as the most sensitive index, a univariate instead of a multivariate approach to the data was used. Four separate analyses of variance were performed with in each, groups (n = 4) as the independent variable and one of the four WAIS-IV factor indices as the dependent variable. Inspection of Table 2 reflects that indicated that there was a
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
20
Assessment 22(1)
Table 2. WAIS-IV Performances of Patients With Various Severities of TBI and Standardization Controls. Uncomplicated TBI (n = 42) Variable Verbal Comprehension Perceptual Reasoning Working Memory Processing Speed
Complicated TBI (n = 29)
Moderate-severe TBI (n = 29)
Controla (n = 100)
M
SD
M
SD
M
SD
M
SD
F
p
η2
100.52
13.69
97.28
13.68
97.62
13.69
102.65
13.03
1.86
.14
0.03
102.05
12.66
105.14
12.36
95.41
12.07
101.14
15.37
2.48
.07
0.04
100.64 96.86
10.87 11.52
98.03 99.14
14.59 11.37
96.31 86.21
14.93 10.79
101.74 96.90
13.46 13.26
1.53 7.08
.21 .001
0.02 0.10
Note. WAIS-IV = Wechsler Adult Intelligence Scale–Fourth Edition; TBI = traumatic brain injury. a Standardization data derived from the WAIS-IV. Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved.
Figure 1. WAIS-IV subtest profiles for uncomplicated mild traumatic brain injury (Group 1), complicated mild traumatic brain injury (Group 2), moderate-severe traumatic brain injury (Group 3), and demographically matched standardization controls (Group 4).
Note. WAIS-IV = Wechsler Adult Intelligence Scale–Fourth Edition; BD = Block Design; SI = Similarities; DS = Digit Span; MR = Matrix Reasoning; VO = Vocabulary; AR = Arithmetic; SS = Symbol Search; VP = Visual Puzzles; IN = Information; CO = Coding. Standardization data derived from the WAIS-IV. Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved.
statistically significant main effect only for Processing Speed, which had a moderate overall effect size, whereas the group differences on the other factor indices were not statistically significant. Post hoc contrasts with the stepdown Bonferroni correction for multiple comparisons revealed that the moderate-severe TBI group had worse Processing Speed scores than the uncomplicated mild TBI, t(69) = 4.06, p < .0006, complicated mild TBI, t(56) = 4.64, p < .0006, and control, t(127) = 4.03, p < .0006, groups. All other group comparisons on this variable were not statistically significant. Inspection of Figure 1 suggests that Symbol Search and Coding were the only two WAIS-IV subtests on which the Moderate-Severe TBI group was clearly below that of all the other groups. The effect of Processing Speed can also be illustrated by considering the frequency of unusually large contrasts
between that index and one that was distinctly not sensitive to TBI, such as Verbal Comprehension. In the WAIS-IV standardization sample, discrepancies between Processing Speed and Verbal Comprehension (in favor of the latter) that exceed 20 points occur less than 10% of the time. In the current moderate-severe TBI group, such a discrepancy occurred more than twice as often; that is, 21% of the time (z = 2.10, p < .04). Because group means do not inform directly about classification accuracy, two sets of logistic regressions were used to determine the accuracy with which WAIS-IV Processing Speed could discriminate participants with moderate-severe TBI from, respectively, demographically matched controls and participants with any kind of mild TBI. The uncomplicated and complicated mild TBI were combined for this purpose because they did not differ significantly from each other on this index. For the distinction between the moderate-severe TBI and control groups, sensitivity was 72% and specificity was 65%, with an area under the curve of 0.73 and a likelihood ratio of 1.96. For the distinction between the moderate-severe TBI and combined mild TBI group, sensitivity was also 72% but with slightly better specificity (69%), resulting in an area under the curve of 0.79 and a likelihood ratio of 2.34. As another method of investigating the sensitivity of the WAIS-IV to injury severity, we computed Spearman correlations between each of the 10 subtests and length of time to follow commands. The only subtests that met the a priori specified criterion for minimal acceptable effect size (i.e., r2 ≥ .05) were Block Design (r = −0.29, p < .004), Symbol Search (r = −0.40, p < .0001), Visual Puzzles (r = −0.26, p < .009), and Coding (r = −0.40, p < .0001). We did not want to conflate injury severity and cognitive impairment. For this reason, we performed an additional analysis to determine to what degree the WAIS-IV Processing Speed index could discriminate between participants with TBI who did versus did not have scores in the clinically significant range on the TMT. Impairment on that instrument was defined a priori as scores falling below the
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
21
Donders and Strong 10th percentile (considering the median age and education of the clinical sample) on either Part A or Part B. In the complete sample, median TMT performance was 24 seconds on Part A (M = 27.44, SD = 13.96, range = 9-110) and 57 seconds on Part B (M = 65.91, SD = 29.41, range = 32-211). Both tasks were statistically significantly positively correlated with length to follow commands, suggesting that greater injury severity was associated with longer time to completion, although this relationship met the a priori specified criterion for effect size only for Part A (ρ = 0.26, p < .01) and fell just short of that for Part B (ρ = 0.21, p < .04). In the complete clinical sample there were 19 participants who had TMT Part A scores below the 10th percentile. A logistic regression analysis was used to determine to what extent WAIS-IV Processing speed could differentiate those participants with versus without such scores in the clinically significant range. Sensitivity and specificity were both 74%, with an area under the curve on 0.80 and a likelihood ratio of 2.85. Finally, we also explored how well the Visual Puzzles subtest was able to discriminate between patients with versus without impairment on TMT Part A. Although sensitivity was good (84%), specificity was poor (61%). Furthermore, although this subtest yielded a statistically significant difference between the moderate-severe TBI and control groups, F(1, 127) = 4.34, p < .04, this fell short of the a priori specified required level of effect size (η2 = .03).
Discussion This study was undertaken to ascertain the clinical utility of the WAIS-IV in the evaluation of participants with TBI. Consistent with our hypothesis, only the Processing Speed index met all the a priori specified criteria for clinical utility. Specifically, the Processing Speed scores of participants with moderate-severe TBI were significantly lower than those of demographically matched controls, as well as those of patients with milder forms of TBI, with a moderate effect size. Furthermore, the Processing Speed subtests (along with two of the PRI subtests that involve a timed component) showed sufficiently robust correlations with length to follow commands. Finally, Processing Speed showed adequate sensitivity and specificity to cognitive impairment, accurately classifying 74% of those with versus without cognitive impairment, as measured by the TMT. These findings are compatible with results from previous studies with the WAIS-III in patients with TBI (Axelrod et al., 2001; Donders et al., 2001) as well as with recent studies of the WAIS-IV in persons with MS (Ryan et al., 2012) and schizophrenia (Michel et al., 2013). As part of this study, we were also interested in determining the clinical usefulness of one of the new subtests, Visual Puzzles, particularly as this measure places a greater demand on rapid responding in comparison to some of its
PRI counterparts. Results revealed that Visual Puzzles has adequate sensitivity to cognitive impairment in TBI but poor specificity. In addition, whereas participants with moderate-severe TBI had significantly lower Visual Puzzle scores than controls, the effect size fell short of a priori criteria. In general, the current results suggest that the timed subtests of the WAIS-IV, but especially those forming the Processing Speed index, are most useful in the assessment of the cognitive sequelae of moderate-severe TBI. Our findings address the concerns expressed by Loring and Bauer (2010) and suggest that, at least when it comes to the evaluation of sequelae of moderate-severe TBI, there is sufficient evidence to insure the appropriate application of the WAIS-IV in clinical neuropsychological assessment. At the same time, we do not recommend isolated use of the WAIS-IV Processing Speed index in the evaluation of TBI. After all, in the discrimination of moderate-severe TBI from demographically matched controls, the likelihood ratio fell just below the a priori established minimum standard of 2. Instead, the WAIS-IV needs to be interpreted within the context of a more comprehensive neuropsychological battery, along with consideration of other clinical data. Potential limitations of this study include use of a primarily Caucasian sample, which limits the generalizability of these results to more diverse populations. In addition, we did not have the capability to more specifically analyze lesion location and/or volume and its relationship to WAIS-IV performance. These are both areas that can be addressed through replications in samples with greater proportions of ethnic minorities, and using more advanced neuroimaging techniques such as diffusion tensor and voxel-based morphometry applications. In conclusion, when used in the context of a comprehensive neuropsychological assessment, the WAIS-IV Processing Speed has adequate clinical utility in the evaluation of moderate-severe TBI. Moreover, these results are consistent with results from prior studies with the WAIS-III, so clinicians should feel more confident in drawing conclusions from the WAIS-IV that are similar to those from its predecessor. A specific goal for future research is the determination of the longer-term predictive utility of WAIS-IV Processing Speed, such as with regard to postacute rehabilitation outcomes. Authors’ Note This research was based in part on standardization data derived from the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV). Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved. The authors thank the publisher of the WAIS-IV for allowing access to these data.
Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
22
Assessment 22(1)
Funding The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Campbell Foundation is a private, charitable funding agency that provided financial support for this research through Grant No. MFB-JD-2013.
References Axelrod, B. N., Fichtenberg, N. L., Liethen, P. C., Czarnota, M. A., & Stucky, K. (2001). Performance characteristics of postacute traumatic brain injury patients on the WAIS-III and WMS-III. Clinical Neuropsychologist, 15, 516-520. Bagiella, E., Novack, T. A., Ansel, B., Diaz-Arrastia, R., Dikmen, S., Hart, T., & Temkin, N. (2010). Measuring outcome in traumatic brain injury treatment trials: Recommendations from the Traumatic Brain Injury Clinical Trials Network. Journal of Head Trauma Rehabilitation, 25, 375-382. Benson, N., Hulac, D. M., & Kranzler, J. H. (2010). Independent examination of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-V): What does the WAIS-IV measure? Psychological Assessment, 22, 121-130. Binder, L. M., & Binder, A. L. (2011). Relative subtest scatter in the WAIS-IV standardization sample. Clinical Neuropsychologist, 25, 62-71. Brooks, B. L., Holdnack, J. A., & Iverson, G. L. (2011). Advanced clinical interpretation of the WAIS-IV and WMS-IV: Prevalence of low scores varies by level of intelligence and years of education. Assessment, 18, 156-167. Canivez, G. L., & Watkins, M. W. (2010). Investigation of the factor structure of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV): Exploratory and higher order factor analyses. Psychological Assessment, 22, 827-836. Cullen, N., Krakowski, A., & Taggart, C. (2014). Early neuropsychological tests as correlates of return to driving after traumatic brain injury. Brain Injury, 28, 38-43. Donders, J., & Janke, K. (2008). Criterion validity of the Wechsler Intelligence Scale for Children–Fourth Edition after pediatric traumatic brain injury. Journal of the International Neuropsychological Society, 14, 651-655. Donders, J., & Levitt, T. (2012). Criterion validity of the Neuropsychological Assessment Battery after traumatic brain injury. Archives of Clinical Neuropsychology, 27, 440-445. Donders, J., Tulsky, D. S., & Zhu, J. (2001). Criterion validity of new WAIS-III subtest scores after traumatic brain injury. Journal of the International Neuropsychological Society, 7, 892-898. Grimes, D. A., & Schulz, K. F. (2005). Refining clinical diagnoses with likelihood ratios. Lancet, 365, 1500-1505. Green, P. (2003). Green’s Word Memory Test. Edmonton, AB: Green’s. Heyanka, D. J., Holster, J. L., & Golden, C. J. (2013). Intraindividual neuropsychological test variability in healthy individuals with high average intelligence and educational attainment. International Journal of Neuroscience, 123, 526-531.
Holdnack, J. A., Zhou, X., Larrabee, G. J., Millis, S. R., & Salthouse, T. A. (2011). Confirmatory factor analysis of the WAIS-IV/WMS-IV. Assessment, 18, 178-191. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65-70. Loring, D. W., & Bauer, R. M. (2010). Testing the limits: Cautions and concerns regarding the new Wechsler IQ and memory scales. Neurology, 74, 685-690. Martin, T. A., Donders, J., & Thompson, E. (2000). Potential of and problems with new measures of psychometric intelligence after traumatic brain injury. Rehabilitation Psychology, 45, 402-408. Martin, T. A., Hoffman, N. M., & Donders, J. (2003). Clinical utility of the Trail Making Test ratio score. Applied Neuropsychology, 10, 163-169. Michel, N. M., Goldberg, J. O., Heinrichs, R. W., Miles, A. A., Ammari, N., & McDermid Vaz, S. (2013). WAIS-IV profile of cognition in schizophrenia. Assessment, 20, 462-473. Muñoz-Moreno, J. A., Prats, A., Pérez-Álvarez, N., Fumaz, C. R., Garolera, M., & Doval, E., . . . The NEU Study Group. (2013). A brief and feasible paper-based method to screen for neurocognitive impairment in HIV-infected patients: The NEU screen. Journal of Acquired Immune Deficiency Syndromes, 63, 585-592. Nelson, J. M., Canivez, G. L., & Watkins, M. W. (2013). Structural and incremental validity of the Wechsler Adult Intelligence Scale–Fourth Edition with a clinical sample. Psychological Assessment, 25, 618-630. Pandharipande, P. P., Girard, T. D., Jackson, J. C., Morandi, A., Thompson, J. L., & Pun, B. T., . . . The BRAIN-ICU Study Investigators. (2013). Long-term cognitive impairment after critical illness. New England Journal of Medicine, 369, 13061316. Reitan, R. M., & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery (2nd ed.). Tucson, AZ: Neuropsychology Press. Ryan, J. J., Gontkovsky, S. T., Kreiner, D. S., & Tree, H. A. (2012). Wechsler Adult Intelligence Scale–Fourth Edition performance in relapsing-remitting multiple sclerosis. Journal of Clinical and Experimental Neuropsychology, 34, 571-579. Stucky, K. J., Kirkwood, M. W., & Donders, J. (2014). Traumatic brain injury. In K. J. Stucky, M. W. Kirkwood, & J. Donders (Eds.), Neuropsychology study guide & board review (pp. 432-447). New York, NY: Oxford. Tombaugh, T. N. (1996). Test of Memory Malingering. Toronto, Ontario, Canada: Multi Health Systems. Ward, C. L., Bergman, M. A., & Hebert, K. R. (2012). WAIS-IV subtest covariance structure: Conceptual and statistical considerations. Psychological Assessment, 24, 328-340. Wechsler, D. (1997). Wechsler Adult Intelligence Scale–Third Edition. San Antonio, TX: Psychological Corporation. Wechsler, D. (2008). Wechsler Adult Intelligence Scale–Fourth Edition. San Antonio, TX: Pearson. Weissberger, G. H., Salmon, D. P., Bondi, M. W., & Gollan, T. H. (2013). Which neuropsychological tests predict progression to Alzheimer’s disease in Hispanics? Neuropsychology, 27, 343-355.
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
research-article2014
ASMXXX10.1177/1073191114530776AssessmentDonders and Strong
Article
Clinical Utility of the Wechsler Adult Intelligence Scale–Fourth Edition After Traumatic Brain Injury
Assessment 2015, Vol. 22(1) 17–22 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1073191114530776 asm.sagepub.com
Jacobus Donders1 and Carrie-Ann H. Strong1
Abstract The performance of 100 patients with traumatic brain injury (TBI) on the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) was compared with that of 100 demographically matched neurologically healthy controls. Processing Speed was the only WAIS-IV factor index that was able to discriminate between persons with moderate-severe TBI on the one hand and persons with either less severe TBI or neurologically healthy controls on the other hand. The Processing Speed index also had acceptable sensitivity and specificity when differentiating between patients with TBI who either did or did not have scores in the clinically significant range on the Trail Making Test. It is concluded that WAIS-IV Processing Speed has acceptable clinical utility in the evaluation of patients with moderate-severe TBI but that it should be supplemented with other measures to assure sufficient accuracy in the diagnostic process. Keywords assessment, processing speed, sensitivity, specificity Measurement of intellectual functioning is a routine component of neuropsychological assessment. The most commonly used intellectual test in neuropsychology is the Wechsler Adult Intelligence Scale (Loring & Bauer, 2010). The most recent revision of the test, the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV; Wechsler, 2008) retains some aspects of its predecessor, the Wechsler Adult Intelligence Scale–Third Edition (Wechsler, 1997). However, there have been considerable changes in the test’s content and administration. These include the removal and replacement of some subtests, a de-emphasis on speeded performance bonuses, and a reduction in all of the subtests’ discontinuation rules. Moreover, it eliminated the Verbal and Performance IQs of prior editions and focused interpretation on the four factor index scores, that is, Verbal Comprehension, Perceptual Reasoning, Working Memory, and Processing Speed. Previous research has established an empirical foundation for using the WAIS-IV’s predecessor in the evaluation of patients with neurological impairment, especially those with traumatic brain injury (TBI), which is one of the most common forms of acquired neurological impairment (Axelrod, Fichtenberg, Liethen, Czarnota, & Stucky, 2001; Donders, Tulsky, & Zhu, 2001; Martin, Donders, & Thomson, 2000). This research has consistently distinguished Processing Speed as the index that is most sensitive to TBI. However, because substantive changes have been made to the test, some have warned against assuming that
the WAIS-IV is analogous to the WAIS-III in terms of clinical utility with neurological populations (Loring & Bauer, 2010). There is a growing literature on the WAIS-IV relevant to neuropsychological practice. Several factor analytic studies have supported the four-factor model but with need for consideration of the overall impact of “g” or general intelligence (Canivez & Watkins, 2010; Holdnack, Zhou, Larrabee, Millis, & Salthouse, 2011; Nelson, Canivez, & Watkins, 2013), although some other studies have suggested some variations in factor structure along specific theoretical dimensions (Benson, Hulac, & Kranzler, 2010; Ward, Bergman, & Hebert, 2012). In addition, there have been several studies that have demonstrated that there is considerable WAIS-IV subtest variability and scatter in normal populations, often exceeding two standard deviations (Binder & Binder, 2011; Heyanka, Holster, & Golden, 2013). However, few independent studies specifically evaluating the clinical utility of the WAIS-IV in specific diagnostic groups have been published. The WAIS-IV performance of patients with multiple sclerosis (MS) was investigated in a recent study (Ryan, 1
Mary Free Bed Rehabilitation Hospital, Grand Rapids, MI, USA
Corresponding Author: Jacobus Donders, Psychology Service, Mary Free Bed Rehabilitation Hospital, 235 Wealthy, S.E., Grand Rapids, MI 49503, USA. Email: [email protected]
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
18
Assessment 22(1)
Gontkovsky, Kreiner, & Tree, 2012). There were statistically significant group differences between patients with MS and demographically matched controls, but Processing Speed was the only WAIS-IV index that was predictive of group membership. The cognitive performance of patients with schizophrenia has also been examined (Michel et al., 2013). Statistically significant group differences between patients with schizophrenia and demographically matched controls were found on the Working Memory and Processing Speed indices, with the greatest divergence on the latter score. To our knowledge, no independent studies have yet been published that investigated the clinical utility of the WAIS-IV in patients with TBI. The WAIS-IV manual does include a study of patients with TBI, though with a very small sample size (n = 22; Wechsler, 2008). Mean group differences with large effect sizes between the TBI and control groups were evident for Perceptual Reasoning, Processing Speed, and Full Scale IQ, with the greatest difference on Processing Speed. Brooks, Holdnack, and Iverson (2011) also used data from the TBI group of the standardization of the WAIS-IV to identify cognitive impairments, using base rate information. When applying base rates, the patients with moderate to severe TBI were more likely to have a “low cognitive profile” in comparison to normal healthy controls. The current study was undertaken to investigate the clinical utility of the WAIS-IV in the assessment of patients with TBI. We were particularly interested in the instrument’s sensitivity to injury severity and acquired impairment, which were considered to be important considerations in clinical evaluations of patients in a health care setting for differential diagnostic and treatment planning purposes. We focused primarily on the factor index scores because they are more reliable than subtest scores. However, one additional area of interest included exploration of the new Perceptual Reasoning subtest “Visual Puzzles,” because it places a greater emphasis on speed of performance than some of the other subtests. Based on prior studies, we hypothesized that the Processing Speed index would have the greatest sensitivity to TBI. It was decided a priori that in order to be clinically useful, the index scores WAIS-IV would have to meet the following criteria: (a) a statistically significant difference between persons with TBI and demographically matched controls and (b) a statistically significant correlation with length of coma. These criteria were chosen, consistent with previous research with different instruments (Donders & Janke, 2008; Donders & Levitt, 2012) and because length of coma is one of the most commonly used determinants of injury severity after TBI (Stucky, Kirkwood, & Donders, 2014). Furthermore, in order to have validation against an independent instrument, a third criterion was (c) sufficient classification accuracy of cognitive impairment, determined
by the WAIS-IV’s ability to distinguish between participants with TBI who did versus did not have scores in the clinically significant range on the Trail Making Test (TMT; Reitan & Wolfson, 1993). This test was chosen for validation purposes because it is a widely used instrument that is sensitive to acquired neuropsychological impairment in a wide range conditions, ranging from acute delirium (Pandharipande et al., 2013), to HIV-related cognitive impairment (uñoz-Moreno et al., 2013), to development of Alzheimer’s disease (Weissberger, Salmon, Bondi, & Gollan, 2013), and to sequelae of TBI in particular (Bagiella et al., 2010; Cullen, Krakowski, & Taggart, 2014; Martin, Hoffman, & Donders, 2003). We used a conventional minimum standard for clinical decision making, defined as a likelihood ratio (sensitivity/[1 − sensitivity]) ≥ 2 (Grimes & Schulz, 2005).
Method Research Participants Approval for this investigation was obtained from the Research Institutional Review Board at Mary Free Bed Rehabilitation Hospital, and all data included in this article were obtained and analyzed in compliance with the regulations of this board as well as in accordance with the Helsinki Declaration (http://www.wma.net/en/30publications/10policies/b3/). Two groups of participants were included in this study: a clinical group of patients with TBI and a group of demographically matched controls. The 100 clinical participants were selected from an approximately 4-year consecutive series of referrals for outpatient neuropsychological evaluation, according to the criteria listed below. Data collection continued until there were 100 patients who met all of the inclusion criteria. A control group (n = 100) was subsequently obtained, with permission from the publisher, from the standardization sample of the WAIS-IV. The control participants were matched to the clinical participants on age and education, and when possible also on gender and ethnicity. None of the control participants had any kind of neurological, psychiatric, or special education history. Demographic characteristics of both groups are presented in Table 1. The following criteria were used to select the clinical participants: (a) diagnosis of TBI, defined as an acute, external force to the head with alteration of consciousness; (b) age between 16 and 65 years; (c) absence of any prior neurological, psychiatric, special education, or substance abuse history; (d) evaluation with the WAIS-IV within 1 to 12 months after injury; and (e) acceptable performance on a stand-alone performance validity test, such as the Test of Memory Malingering (Tombaugh, 1996) or the Word Memory Test (Green, 2003). Alteration of consciousness was determined by means of review of medical records for
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
19
Donders and Strong Table 1. Demographic Characteristics and WAIS-IV Performances of Patients With TBI (n = 100) and Standardization Controls (n = 100). Variable Age (years; M, SD) Education (n) 15 years Gender (n) Female Male Ethnicity (n) Caucasian Latina/o Other
TBI
Controla
32.56 (14.78)
32.65 (14.75)
26 31 25 18
17 35 28 20
36 64
39 61
91 5 4
82 13 5
Note. WAIS-IV = Wechsler Adult Intelligence Scale–Fourth Edition; TBI = traumatic brain injury. a Standardization data derived from the WAIS-IV. Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved.
any documentation that the individual was dazed, confused, engaging in repetitive questioning, or not responding to commands at either the accident scene or in the emergency room. During the time period that these data were collected, the WAIS-IV and TMT had been included routinely in outpatient neuropsychological evaluations of patients with TBI, unless there were circumstances that would have compromised test validity (e.g., non-English speaking language background, severe orthopedic injury to the dominant hand). Only first evaluations, no repeat evaluations, were included. The participants with TBI were seen for evaluation with the WAIS-IV at a median of 122 days post injury (M = 143.23, SD = 87.73, range = 30-355). Injury circumstances reflected primarily motor vehicle collisions (64%), falls and recreational activities (28%), and various other causes (8%). Several measures of injury severity were considered. Exact initial Glasgow Coma Scale scores were not always available, and duration of posttraumatic amnesia would have required retrospective estimation in some cases, which was considered unreliable. However, information about the time to follow commands and with regard to the presence or absence of acute abnormalities on neuroimaging was available for all participants with TBI. In the complete group with TBI, median duration of time to follow commands was 0 days (M = 1.58, SD = 4.20, range = 0-28), with this duration being ≥24 hours in 29 participants. Fifty-eight of the participants with TBI had positive neuroimaging findings, including 57 with focal lesions (e.g., hemorrhagic contusion) and 22 with diffuse lesions (e.g., edema), with some participants having multifocal or a combination of focal and diffuse
lesions. A total of 42 participants had uncomplicated mild TBI, defined as duration of time to follow commands .53. There were also no statistically significant correlations between time span since injury and performance on any of the WAIS-IV factor indices (all ps > .06).
Procedure The WAIS-IV and TMT had been administered to clinical patients as part of outpatient neuropsychological evaluations, which were scheduled only when they were medically stable and could recall meaningful information from day to day. All evaluations had originally been completed for clinical reasons with informed consent. Only the 10 subtests that are necessary to compute the four WAIS-IV factor index scores were routinely included in these evaluations. Standard scores (M = 100, SD = 15) for the factor indices and scaled scores (M = 10, SD = 3) for the subtests were used in the statistical analyses. To balance the relative risks of Type I and Type II errors, the stepdown Bonferroni method (Holm, 1979) was used to adjust alpha in any post hoc comparisons. We also specified a priori a minimum acceptable effect size of .05 (η2 for group contrasts; ρ2 for correlations). Because of the skewed distribution of the days to follow commands, Spearman instead of Pearson correlations were used.
Results The average WAIS-IV factor index scores are presented in Table 2 for the three TBI severity groups and the demographically matched control group. For illustrative purposes, the subtest profiles are presented in Figure 1. Because we had predicted a priori that Processing Speed would stand out as the most sensitive index, a univariate instead of a multivariate approach to the data was used. Four separate analyses of variance were performed with in each, groups (n = 4) as the independent variable and one of the four WAIS-IV factor indices as the dependent variable. Inspection of Table 2 reflects that indicated that there was a
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
20
Assessment 22(1)
Table 2. WAIS-IV Performances of Patients With Various Severities of TBI and Standardization Controls. Uncomplicated TBI (n = 42) Variable Verbal Comprehension Perceptual Reasoning Working Memory Processing Speed
Complicated TBI (n = 29)
Moderate-severe TBI (n = 29)
Controla (n = 100)
M
SD
M
SD
M
SD
M
SD
F
p
η2
100.52
13.69
97.28
13.68
97.62
13.69
102.65
13.03
1.86
.14
0.03
102.05
12.66
105.14
12.36
95.41
12.07
101.14
15.37
2.48
.07
0.04
100.64 96.86
10.87 11.52
98.03 99.14
14.59 11.37
96.31 86.21
14.93 10.79
101.74 96.90
13.46 13.26
1.53 7.08
.21 .001
0.02 0.10
Note. WAIS-IV = Wechsler Adult Intelligence Scale–Fourth Edition; TBI = traumatic brain injury. a Standardization data derived from the WAIS-IV. Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved.
Figure 1. WAIS-IV subtest profiles for uncomplicated mild traumatic brain injury (Group 1), complicated mild traumatic brain injury (Group 2), moderate-severe traumatic brain injury (Group 3), and demographically matched standardization controls (Group 4).
Note. WAIS-IV = Wechsler Adult Intelligence Scale–Fourth Edition; BD = Block Design; SI = Similarities; DS = Digit Span; MR = Matrix Reasoning; VO = Vocabulary; AR = Arithmetic; SS = Symbol Search; VP = Visual Puzzles; IN = Information; CO = Coding. Standardization data derived from the WAIS-IV. Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved.
statistically significant main effect only for Processing Speed, which had a moderate overall effect size, whereas the group differences on the other factor indices were not statistically significant. Post hoc contrasts with the stepdown Bonferroni correction for multiple comparisons revealed that the moderate-severe TBI group had worse Processing Speed scores than the uncomplicated mild TBI, t(69) = 4.06, p < .0006, complicated mild TBI, t(56) = 4.64, p < .0006, and control, t(127) = 4.03, p < .0006, groups. All other group comparisons on this variable were not statistically significant. Inspection of Figure 1 suggests that Symbol Search and Coding were the only two WAIS-IV subtests on which the Moderate-Severe TBI group was clearly below that of all the other groups. The effect of Processing Speed can also be illustrated by considering the frequency of unusually large contrasts
between that index and one that was distinctly not sensitive to TBI, such as Verbal Comprehension. In the WAIS-IV standardization sample, discrepancies between Processing Speed and Verbal Comprehension (in favor of the latter) that exceed 20 points occur less than 10% of the time. In the current moderate-severe TBI group, such a discrepancy occurred more than twice as often; that is, 21% of the time (z = 2.10, p < .04). Because group means do not inform directly about classification accuracy, two sets of logistic regressions were used to determine the accuracy with which WAIS-IV Processing Speed could discriminate participants with moderate-severe TBI from, respectively, demographically matched controls and participants with any kind of mild TBI. The uncomplicated and complicated mild TBI were combined for this purpose because they did not differ significantly from each other on this index. For the distinction between the moderate-severe TBI and control groups, sensitivity was 72% and specificity was 65%, with an area under the curve of 0.73 and a likelihood ratio of 1.96. For the distinction between the moderate-severe TBI and combined mild TBI group, sensitivity was also 72% but with slightly better specificity (69%), resulting in an area under the curve of 0.79 and a likelihood ratio of 2.34. As another method of investigating the sensitivity of the WAIS-IV to injury severity, we computed Spearman correlations between each of the 10 subtests and length of time to follow commands. The only subtests that met the a priori specified criterion for minimal acceptable effect size (i.e., r2 ≥ .05) were Block Design (r = −0.29, p < .004), Symbol Search (r = −0.40, p < .0001), Visual Puzzles (r = −0.26, p < .009), and Coding (r = −0.40, p < .0001). We did not want to conflate injury severity and cognitive impairment. For this reason, we performed an additional analysis to determine to what degree the WAIS-IV Processing Speed index could discriminate between participants with TBI who did versus did not have scores in the clinically significant range on the TMT. Impairment on that instrument was defined a priori as scores falling below the
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
21
Donders and Strong 10th percentile (considering the median age and education of the clinical sample) on either Part A or Part B. In the complete sample, median TMT performance was 24 seconds on Part A (M = 27.44, SD = 13.96, range = 9-110) and 57 seconds on Part B (M = 65.91, SD = 29.41, range = 32-211). Both tasks were statistically significantly positively correlated with length to follow commands, suggesting that greater injury severity was associated with longer time to completion, although this relationship met the a priori specified criterion for effect size only for Part A (ρ = 0.26, p < .01) and fell just short of that for Part B (ρ = 0.21, p < .04). In the complete clinical sample there were 19 participants who had TMT Part A scores below the 10th percentile. A logistic regression analysis was used to determine to what extent WAIS-IV Processing speed could differentiate those participants with versus without such scores in the clinically significant range. Sensitivity and specificity were both 74%, with an area under the curve on 0.80 and a likelihood ratio of 2.85. Finally, we also explored how well the Visual Puzzles subtest was able to discriminate between patients with versus without impairment on TMT Part A. Although sensitivity was good (84%), specificity was poor (61%). Furthermore, although this subtest yielded a statistically significant difference between the moderate-severe TBI and control groups, F(1, 127) = 4.34, p < .04, this fell short of the a priori specified required level of effect size (η2 = .03).
Discussion This study was undertaken to ascertain the clinical utility of the WAIS-IV in the evaluation of participants with TBI. Consistent with our hypothesis, only the Processing Speed index met all the a priori specified criteria for clinical utility. Specifically, the Processing Speed scores of participants with moderate-severe TBI were significantly lower than those of demographically matched controls, as well as those of patients with milder forms of TBI, with a moderate effect size. Furthermore, the Processing Speed subtests (along with two of the PRI subtests that involve a timed component) showed sufficiently robust correlations with length to follow commands. Finally, Processing Speed showed adequate sensitivity and specificity to cognitive impairment, accurately classifying 74% of those with versus without cognitive impairment, as measured by the TMT. These findings are compatible with results from previous studies with the WAIS-III in patients with TBI (Axelrod et al., 2001; Donders et al., 2001) as well as with recent studies of the WAIS-IV in persons with MS (Ryan et al., 2012) and schizophrenia (Michel et al., 2013). As part of this study, we were also interested in determining the clinical usefulness of one of the new subtests, Visual Puzzles, particularly as this measure places a greater demand on rapid responding in comparison to some of its
PRI counterparts. Results revealed that Visual Puzzles has adequate sensitivity to cognitive impairment in TBI but poor specificity. In addition, whereas participants with moderate-severe TBI had significantly lower Visual Puzzle scores than controls, the effect size fell short of a priori criteria. In general, the current results suggest that the timed subtests of the WAIS-IV, but especially those forming the Processing Speed index, are most useful in the assessment of the cognitive sequelae of moderate-severe TBI. Our findings address the concerns expressed by Loring and Bauer (2010) and suggest that, at least when it comes to the evaluation of sequelae of moderate-severe TBI, there is sufficient evidence to insure the appropriate application of the WAIS-IV in clinical neuropsychological assessment. At the same time, we do not recommend isolated use of the WAIS-IV Processing Speed index in the evaluation of TBI. After all, in the discrimination of moderate-severe TBI from demographically matched controls, the likelihood ratio fell just below the a priori established minimum standard of 2. Instead, the WAIS-IV needs to be interpreted within the context of a more comprehensive neuropsychological battery, along with consideration of other clinical data. Potential limitations of this study include use of a primarily Caucasian sample, which limits the generalizability of these results to more diverse populations. In addition, we did not have the capability to more specifically analyze lesion location and/or volume and its relationship to WAIS-IV performance. These are both areas that can be addressed through replications in samples with greater proportions of ethnic minorities, and using more advanced neuroimaging techniques such as diffusion tensor and voxel-based morphometry applications. In conclusion, when used in the context of a comprehensive neuropsychological assessment, the WAIS-IV Processing Speed has adequate clinical utility in the evaluation of moderate-severe TBI. Moreover, these results are consistent with results from prior studies with the WAIS-III, so clinicians should feel more confident in drawing conclusions from the WAIS-IV that are similar to those from its predecessor. A specific goal for future research is the determination of the longer-term predictive utility of WAIS-IV Processing Speed, such as with regard to postacute rehabilitation outcomes. Authors’ Note This research was based in part on standardization data derived from the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV). Copyright © 2008 NCS Pearson, Inc. Used with permission. All rights reserved. The authors thank the publisher of the WAIS-IV for allowing access to these data.
Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
22
Assessment 22(1)
Funding The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Campbell Foundation is a private, charitable funding agency that provided financial support for this research through Grant No. MFB-JD-2013.
References Axelrod, B. N., Fichtenberg, N. L., Liethen, P. C., Czarnota, M. A., & Stucky, K. (2001). Performance characteristics of postacute traumatic brain injury patients on the WAIS-III and WMS-III. Clinical Neuropsychologist, 15, 516-520. Bagiella, E., Novack, T. A., Ansel, B., Diaz-Arrastia, R., Dikmen, S., Hart, T., & Temkin, N. (2010). Measuring outcome in traumatic brain injury treatment trials: Recommendations from the Traumatic Brain Injury Clinical Trials Network. Journal of Head Trauma Rehabilitation, 25, 375-382. Benson, N., Hulac, D. M., & Kranzler, J. H. (2010). Independent examination of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-V): What does the WAIS-IV measure? Psychological Assessment, 22, 121-130. Binder, L. M., & Binder, A. L. (2011). Relative subtest scatter in the WAIS-IV standardization sample. Clinical Neuropsychologist, 25, 62-71. Brooks, B. L., Holdnack, J. A., & Iverson, G. L. (2011). Advanced clinical interpretation of the WAIS-IV and WMS-IV: Prevalence of low scores varies by level of intelligence and years of education. Assessment, 18, 156-167. Canivez, G. L., & Watkins, M. W. (2010). Investigation of the factor structure of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV): Exploratory and higher order factor analyses. Psychological Assessment, 22, 827-836. Cullen, N., Krakowski, A., & Taggart, C. (2014). Early neuropsychological tests as correlates of return to driving after traumatic brain injury. Brain Injury, 28, 38-43. Donders, J., & Janke, K. (2008). Criterion validity of the Wechsler Intelligence Scale for Children–Fourth Edition after pediatric traumatic brain injury. Journal of the International Neuropsychological Society, 14, 651-655. Donders, J., & Levitt, T. (2012). Criterion validity of the Neuropsychological Assessment Battery after traumatic brain injury. Archives of Clinical Neuropsychology, 27, 440-445. Donders, J., Tulsky, D. S., & Zhu, J. (2001). Criterion validity of new WAIS-III subtest scores after traumatic brain injury. Journal of the International Neuropsychological Society, 7, 892-898. Grimes, D. A., & Schulz, K. F. (2005). Refining clinical diagnoses with likelihood ratios. Lancet, 365, 1500-1505. Green, P. (2003). Green’s Word Memory Test. Edmonton, AB: Green’s. Heyanka, D. J., Holster, J. L., & Golden, C. J. (2013). Intraindividual neuropsychological test variability in healthy individuals with high average intelligence and educational attainment. International Journal of Neuroscience, 123, 526-531.
Holdnack, J. A., Zhou, X., Larrabee, G. J., Millis, S. R., & Salthouse, T. A. (2011). Confirmatory factor analysis of the WAIS-IV/WMS-IV. Assessment, 18, 178-191. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65-70. Loring, D. W., & Bauer, R. M. (2010). Testing the limits: Cautions and concerns regarding the new Wechsler IQ and memory scales. Neurology, 74, 685-690. Martin, T. A., Donders, J., & Thompson, E. (2000). Potential of and problems with new measures of psychometric intelligence after traumatic brain injury. Rehabilitation Psychology, 45, 402-408. Martin, T. A., Hoffman, N. M., & Donders, J. (2003). Clinical utility of the Trail Making Test ratio score. Applied Neuropsychology, 10, 163-169. Michel, N. M., Goldberg, J. O., Heinrichs, R. W., Miles, A. A., Ammari, N., & McDermid Vaz, S. (2013). WAIS-IV profile of cognition in schizophrenia. Assessment, 20, 462-473. Muñoz-Moreno, J. A., Prats, A., Pérez-Álvarez, N., Fumaz, C. R., Garolera, M., & Doval, E., . . . The NEU Study Group. (2013). A brief and feasible paper-based method to screen for neurocognitive impairment in HIV-infected patients: The NEU screen. Journal of Acquired Immune Deficiency Syndromes, 63, 585-592. Nelson, J. M., Canivez, G. L., & Watkins, M. W. (2013). Structural and incremental validity of the Wechsler Adult Intelligence Scale–Fourth Edition with a clinical sample. Psychological Assessment, 25, 618-630. Pandharipande, P. P., Girard, T. D., Jackson, J. C., Morandi, A., Thompson, J. L., & Pun, B. T., . . . The BRAIN-ICU Study Investigators. (2013). Long-term cognitive impairment after critical illness. New England Journal of Medicine, 369, 13061316. Reitan, R. M., & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery (2nd ed.). Tucson, AZ: Neuropsychology Press. Ryan, J. J., Gontkovsky, S. T., Kreiner, D. S., & Tree, H. A. (2012). Wechsler Adult Intelligence Scale–Fourth Edition performance in relapsing-remitting multiple sclerosis. Journal of Clinical and Experimental Neuropsychology, 34, 571-579. Stucky, K. J., Kirkwood, M. W., & Donders, J. (2014). Traumatic brain injury. In K. J. Stucky, M. W. Kirkwood, & J. Donders (Eds.), Neuropsychology study guide & board review (pp. 432-447). New York, NY: Oxford. Tombaugh, T. N. (1996). Test of Memory Malingering. Toronto, Ontario, Canada: Multi Health Systems. Ward, C. L., Bergman, M. A., & Hebert, K. R. (2012). WAIS-IV subtest covariance structure: Conceptual and statistical considerations. Psychological Assessment, 24, 328-340. Wechsler, D. (1997). Wechsler Adult Intelligence Scale–Third Edition. San Antonio, TX: Psychological Corporation. Wechsler, D. (2008). Wechsler Adult Intelligence Scale–Fourth Edition. San Antonio, TX: Pearson. Weissberger, G. H., Salmon, D. P., Bondi, M. W., & Gollan, T. H. (2013). Which neuropsychological tests predict progression to Alzheimer’s disease in Hispanics? Neuropsychology, 27, 343-355.
Downloaded from asm.sagepub.com at LAKEHEAD UNIV on March 18, 2015
