This article was downloaded by: [Gebze Yuksek Teknoloji Enstitïsu ] On: 21 December 2014, At: 17:55 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
The Clinical Neuropsychologist Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ntcn20
Inadequate Effort on Neuropsychological Evaluation is Associated With Increased Healthcare Utilization ab
ab
cd
Michael David Horner , Kathryn K. VanKirk , Clara E. Dismuke , abe
Travis H. Turner
bf
& Wendy Muzzy
a
Mental Health Service, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA b
Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
Click for updates
c
Center for Disease Prevention and Health Interventions for Diverse Populations, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA d
Center For Health Disparities Research, Department of Internal Medicine, Medical University of South Carolina, Charleston, SC, USA e
Department of Neurosciences, Medical University of South Carolina, Charleston, SC, USA f
Charleston Research Institute, Charleston, SC, USA Published online: 16 Jun 2014.
To cite this article: Michael David Horner, Kathryn K. VanKirk, Clara E. Dismuke, Travis H. Turner & Wendy Muzzy (2014) Inadequate Effort on Neuropsychological Evaluation is Associated With Increased Healthcare Utilization, The Clinical Neuropsychologist, 28:5, 703-713, DOI: 10.1080/13854046.2014.925143 To link to this article: http://dx.doi.org/10.1080/13854046.2014.925143
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
The Clinical Neuropsychologist, 2014 Vol. 28, No. 5, 703–713, http://dx.doi.org/10.1080/13854046.2014.925143
Inadequate Effort on Neuropsychological Evaluation is Associated With Increased Healthcare Utilization Michael David Horner1,2, Kathryn K. VanKirk1,2, Clara E. Dismuke3,4, Travis H. Turner1,2,5, and Wendy Muzzy2,6 Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
1
Mental Health Service, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA 2 Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA 3 Center for Disease Prevention and Health Interventions for Diverse Populations, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA 4 Center For Health Disparities Research, Department of Internal Medicine, Medical University of South Carolina, Charleston, SC, USA 5 Department of Neurosciences, Medical University of South Carolina, Charleston, SC, USA 6 Charleston Research Institute, Charleston, SC, USA Patients who exert inadequate effort on neuropsychological examination might not receive accurate diagnoses and recommendations, and might not cooperate fully with other aspects of healthcare. This study examined whether inadequate effort is associated with increased healthcare utilization. Of 355 patients seen for routine, clinical neuropsychological examination at a VA Medical Center, 283 (79.7%) showed adequate effort and 72 (20.3%) showed inadequate effort, as determined at time of evaluation using the Word Memory Test and/or Test of Memory Malingering. Utilization data included number of Emergency Department (ED) visits and inpatient hospitalizations in the year following evaluation. Patients who had shown inadequate effort on examination had more Emergency Department visits, more inpatient hospitalizations, and more days of inpatient hospitalization in the year after evaluation, compared to patients who had exerted adequate effort. This finding was not attributable to group differences in age or medical/ psychiatric comorbidities. Thus, patients who exerted inadequate effort showed greater healthcare utilization in the year following evaluation. Such patients might use more resources since diagnostic evaluations are inconclusive. Inadequate effort on examination might also serve as a “marker” for more general failure to cooperate fully in one’s healthcare, possibly resulting in greater utilization. Keywords: Health care utilization; Neuropsychological assessment; Clinical neuropsychology; Health outcomes; Effort.
INTRODUCTION It is well established that many patients do not exert adequate effort on neuropsychological evaluation, producing invalid findings that generally overestimate their degree of cognitive impairment. For example, Mittenberg, Patton, Canyock, and Condit (2002) reported estimated prevalence of inadequate effort in 8% to 39% of evaluations across
Address correspondence to: Michael David Horner, Ralph H. Johnson Department of Veterans Affairs Medical Center, Mental Health Service 116, 109 Bee Street, Charleston, SC 29401-5799, USA. E-mail: [email protected] (Received 31 October 2013; accepted 12 May 2014)
© 2014 Taylor & Francis
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
704
MICHAEL DAVID HORNER ET AL.
various settings. Typically, such invalid findings occur when patients are not fully engaged in the evaluation process, for reasons that might include apathy, disinterest in the evaluation or its outcome, fears that the examination will be insufficiently sensitive to deficits, distrust of the examiner or healthcare system, or deliberate embellishment or feigning of cognitive deficit (Carone, Iverson, & Bush, 2010; Iverson, 2006). Invalid findings due to inadequate effort are often of little utility in diagnosing cognitive disorders or in treatment planning. (For the purposes of this paper, we use the term “inadequate effort” to refer to any instance in which a patient, for whatever reason, intentional or unintentional, does not try sufficiently hard on cognitive tests to provide a valid estimate of his or her cognitive ability.) Thus, patients who exert inadequate effort on evaluation might not receive accurate diagnoses or necessary clinical services. In turn, they might continue to receive additional, unnecessary diagnostic services because their diagnosis and cognitive status remain unknown. Similarly, such patients might continue to receive treatment for conditions that are not actually present (e.g., enduring sequelae of traumatic brain injury), and might not receive treatment for conditions that truly are present (e.g., mood or anxiety disorders, or psychosocial problems). Further, expenditure of resources to treat conditions that are not truly present would deplete such resources for patients who truly have those conditions. We thus wondered whether inadequate effort on neuropsychological examination might be associated with increased healthcare utilization and costs. Broadly speaking, such an association would help illuminate the costs and benefits associated with neuropsychological evaluation in the context of larger healthcare systems. To our knowledge, no published studies to date have addressed healthcare utilization or similar outcomes associated with inadequate effort on neuropsychological examinations. We hypothesized that, relative to adequate effort on examination, inadequate effort would be associated with increased healthcare utilization and costs, operationalized as inpatient hospitalizations and visits to the Emergency Department (ED). METHOD Design Data were collected as part of a larger study of healthcare utilization in patients who had undergone neuropsychological evaluation (VanKirk, Horner, Turner, Dismuke, & Muzzy, 2013). The study consisted of a retrospective analysis of hospital service utilization (e.g., ED visits and inpatient hospitalizations) in patients who underwent neuropsychological evaluation in the course of clinical care between 2003 and 2010. Evaluations were conducted in the Neuropsychology Clinic at the Ralph H. Johnson Department of Veterans Affairs Medical Center (VAMC), which serves over 40,000 Veterans in 15 counties along the South Carolina and Georgia coastline. The project was reviewed and approved by the Institutional Review Board of the Medical University of South Carolina and the VA Research and Development Service. Participants The initial sample consisted of all patients who had undergone clinical, outpatient evaluation of cognitive status in the Ralph H. Johnson VAMC’s Neuropsychology
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
705
Clinic between the years 2003 and 2010 (n = 604). Patients who had been referred for differential diagnosis of psychiatric conditions only were not included; neither were patients seen for neuropsychological evaluation in the VAMC’s other clinics (e.g., Memory Disorders Clinic for geriatric patients). Individuals referred for inpatient evaluation were also excluded, as their inclusion would have confounded the utilization outcome measures. While the majority of referrals were generated through Mental Health (41%), Neurology (22%), and Primary Care (21%), the Neuropsychology Clinic’s referral base is facility-wide; other referrals came from other outpatient clinics including TBI Clinic and Infectious Diseases Clinic. The sample included only patients who were referred purely as part of their clinical care. Specifically, the sample did not include patients who were seen for VA Compensation and Pension examinations, a specific type of evaluation within the VA system that is used in determination of military service-connected disability benefits. (Disabling conditions that have been incurred or aggravated during active military service are considered to be “service-connected,” and may be eligible for financial compensation.) No patients in the sample were evaluated as part of a legal claim, or for reasons other than clinical care. Patients for whom there was not complete access to hospital service utilization data (e.g., inpatient admissions, Emergency Department visits, etc. as described below) for the 12 months following evaluation were then excluded. Patients were also excluded if they died or transferred their health care to another VAMC within the 12 months following evaluation, or if data regarding medical and psychiatric comorbidities were not available. For patients who had been evaluated on more than one occasion, only the first evaluation was included in the analyses. Further details of the design and methodology are described by VanKirk et al. (2013) and are not repeated here. Finally, patients were classified according to whether they had exerted adequate effort on neuropsychological evaluation. As a standardized battery is not administered to all patients referred to the Neuropsychology Clinic, not all patients were given the same performance validity tests as part of their neuropsychological evaluation. For the present study, patients were included in the final sample only if they had been administered either the Word Memory Test (WMT; Green, 2003) or the Test of Memory Malingering (TOMM; Tombaugh, 1996). Standard cutoff scores described in each test’s manual were used to determine whether effort was adequate or suboptimal. There were 57 patients who were considered to have passed the TOMM after scoring 47 or higher on Trial 1, without administration of Trial 2, an abbreviated form of the test that has been described previously (Horner, Bedwell, & Duong, 2006). However, as determination of adequate vs. inadequate effort in clinical practice is not typically made solely on the basis of a single test, we used a combination of criteria similar to those proposed by Slick, Sherman, and Iverson (1999), and to those previously reported by our laboratory (e.g., Barker, Horner, & Bachman, 2010; Bortnik, Horner, & Bachman, 2013). In addition to the TOMM and WMT, determination of effort typically took into account embedded effort indices such as Reliable Digit Span (for a review see Schroeder, Twumasi-Ankrah, Baade, & Marshall, 2012), as well as behavioral factors including (1) deficits on testing that were grossly disproportionate to the patient’s observed or reported functional level (e.g., a patient who was amnestic on testing and yet provided rich details about his activities of the preceding few days); (2) implausible errors, patterns of responses, or test scores (e.g., an independently
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
706
MICHAEL DAVID HORNER ET AL.
functional patient who was reportedly unable to remember highly overlearned information that even very severely impaired patients can recall without difficulty, or better performance on harder than on easier tasks throughout the evaluation); and (3) other clear behavioral evidence of inadequate effort (e.g., a patient’s explicit statement that he was not trying his best or was disinterested in the examination, or a clear pattern of giving up easily despite apparent ability to complete tasks). Clinicians’ judgments of these behaviors were made at the time of clinical examination, well before the present study was conceived or executed. These observations were thus made as part of routine, clinical examination, and were not standardized across examiners for research purposes. In order to be assigned to the inadequate effort group, a patient had to have failed either the TOMM or the WMT, in addition to displaying clearly at least one of the other behavioral indicators described above. Similarly, in order to be assigned to the adequate effort group a patient could not have failed either TOMM or WMT, and could not have displayed the behaviors suggestive of inadequate effort described above. Patients who passed effort tests but were judged at the time of examination to have exerted inadequate effort, or who failed effort tests but who did not exhibit other behavioral indicators of inadequate effort, were excluded from the study sample. The final sample consisted of 355 Veterans who completed outpatient, clinically indicated neuropsychological evaluation, for whom complete effort data and healthcare utilization data were available. There were 28 patients who had been administered the WMT and 336 who had been administered the TOMM; 9 patients were administered both tests. Table 1 displays patient demographics. Psychiatric diagnoses that were assigned based on the neuropsychological evaluations are listed in Table 2. Procedures Demographic and healthcare utilization data were collected as part of a larger study (VanKirk et al., 2013). Relevant clinical information for each patient, including demographic data and DSM-IV-TR diagnoses assigned at the time of neuropsychological evaluation, were extracted from the Ralph H. Johnson VAMC Neuropsychology Clinic database and the VA’s electronic medical record system (CPRS). In addition, for each participant, information regarding primary outcome measures (i.e., number of ED visits, number of hospitalizations, and total days of hospitalization) was collected for a period of 12 months following completion of the evaluation. Evaluation was considered complete when the final report was posted to the patient’s medical record, which effectively alerts providers to results of the evaluation. Beginning in 2007, approximately halfway through the period for which neuropsychological and healthcare utilization data were collected, we began systematically Table 1. Mean (SD) values for demographic characteristics of the sample
Age Education (years)* Percent male *p < .05.
Adequate effort (N = 283)
Inadequate effort (N = 72)
52.1(13.9) 13.2 (2.5) 88.0
50.5 (13.1) 12.4 (2.4) 90.3
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
707
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
Table 2. DSM-IV-TR diagnoses assigned at time of neuropsychological evaluation Diagnosis
N
Dementia (includes Dementia of the Alzheimer Type, Vascular Dementia, others) Cognitive Disorder Not Otherwise Specified Learning Disorder Attention-Deficit/Hyperactivity Disorder
25 90 6 16
Major Depressive Disorder Dysthymic Disorder Bipolar Disorder Other Mood Disorder (includes Depressive Disorder NOS, Cyclothymia, others) Generalized Anxiety Disorder Post-Traumatic Stress Disorder Other Anxiety Disorder (includes Panic Disorder, Specific Phobia, others) Schizophrenia / Schizoaffective Disorder Other Psychotic Disorder Somatoform disorder Adjustment disorder Alcohol Abuse / Dependence Cocaine Abuse / Dependence Other Substance Abuse / Dependence
86 12 12 29 19 72 12 5 2 4 16 51 12 6
Personality Change Due to General Medical Condition Obsessive-Compulsive Personality Disorder Borderline personality disorder Other personality disorder Malingering / Factitious Disorder No Diagnosis / Diagnosis Deferred
4 6 4 6 19 33
Most patients met criteria for more than one diagnosis.
recording whether each patient seen for clinical neuropsychological evaluation was receiving service-connected disability benefits for psychiatric illness, brain injury, or other brain illness at the time of the evaluation. These data are readily available in Veterans’ medical records, and were collected as part of routine clinical examination, not for the purposes of the present study. At the same time we began asking all patients, as part of routine clinical examination, whether they were in the process of applying for disability benefits, or for an increase in disability benefits, at the time of the evaluation. These data were collected for 172 of the 355 patients in the sample. Finally, also in 2007, we began asking all patients whether they were involved in any other litigation or legal action through the courts at the time of evaluation. These data were available for 170 patients in the sample. As noted above, however, all participants were seen for clinical evaluation only; none of the evaluations was conducted for purposes of determination of disability benefits, litigation, or any other forensic consideration. As is characteristic of the VA patient population generally, some patients in the study were receiving VA disability benefits for service-connected conditions at the time of evaluation. The majority of these conditions (including, e.g., orthopedic and other medical conditions) were not related to
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
708
MICHAEL DAVID HORNER ET AL.
neurological illness or injury and had no specific relevance to the referral question prompting neuropsychological evaluation. Similarly, some patients happened to be in the process of applying for such benefits, or appealing a previous determination, at the time of examination; in many such cases the condition in question was unrelated to the reason for referral to the Neuropsychology Clinic. Finally, a very small number of patients in each group (see below under Results) were involved in other litigation at the time of evaluation, but this was typically unrelated to the reason for referral to Neuropsychology Clinic. Comorbidities were measured as a count of the following types of diagnoses appearing in each patient’s medical record: cancer, cognitive, cardiovascular, endocrine, gastrointestinal, infectious disease, liver, musculoskeletal, neurological, pain, psychiatric, pulmonary, sensory/perceptual, substance abuse, and other. All ED visits and hospitalizations were included for analysis, regardless of reason for visit or hospitalization. Criteria for an ED visit included being seen in the ED of the Ralph H. Johnson VAMC and evaluation by an ED physician or nurse practitioner. Hospitalizations were included only if length of inpatient hospitalization was greater than 24 hours. Length of hospitalization was determined by counting the total number of days a patient was hospitalized, regardless of the number of hours spent each day. For example, a patient admitted at 4 pm on a Friday and discharged at 9 am on a Monday was determined to have spent 4 days in the hospital. Mean VA cost estimates of ED visits and inpatient days were obtained from patient level costs of a sample of over 800,000 Veterans in 2010 (www.herc.research.va.gov) and were adjusted to 2013 dollar values based on the US Department of Labor’s Inflation Calculator. (www.bls.gov/data/ inflation_calculator.htm). Data analysis Group differences in age and education were analyzed using t-tests. Group difference in gender composition was analyzed by chi-square analysis. Data regarding patients’ disability benefits and litigation status were available only for a subset of the sample, as these data began to be collected systematically during the period of time for which neuropsychological and utilization data were studied. Group differences in these variables were analyzed using chi-square analyses. To address the study hypotheses, three types of statistical analyses were then conducted. First, the outcome variables of interest (number of ED visits, number of hospitalizations, and total days of hospitalization), as well as number of comorbidities, were tested for normality using skewness and kurtosis tests in order to ascertain the appropriate statistical tests to be used for differences in unadjusted and adjusted means. Second, due to the non-normal findings from the normality tests, number of ED visits, number of hospitalizations, and total days of hospitalization were compared between the two groups (adequate versus inadequate effort) using Wilcoxon (Mann-Whitney) rank sum tests for differences in central tendency. No truncation or other adjustment for outliers was performed. The difference in effort for the normally distributed variable, number of comorbidities, was tested using the Student t-test. Third, count models of ED visits, number of hospitalizations, and total days of hospitalization were estimated using Poisson regression to quantify the association of effort with ED visits, number of
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
709
hospitalizations, and total days of hospitalization while adjusting for individual comorbidities. All of these statistical tests were performed using STATA 9.0. The mean VA ED visit cost and the mean inpatient day cost were applied to the estimated difference in number of visits associated with adequate effort.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
RESULTS The groups did not differ in age or gender composition (Table 1). Patients who exerted inadequate effort on examination had less education than those who exerted adequate effort. However, as education was not significantly correlated with any of the outcome variables, it was not used as a covariate in subsequent group comparisons. Of the 172 patients for whom data were available (132 who had shown adequate effort and 40 who had shown inadequate effort), 45.0% of those who exerted adequate effort and 40.9% of those who exerted inadequate effort were service-connected for psychiatric illness, brain injury, or other brain illness at the time of evaluation; this difference was not statistically significant (χ2 = 0.21, ns). However, of this subsample, significantly more patients in the inadequate effort group (62.5%) than in the adequate effort group (23.5%) were in the process of seeking disability benefits, or an increase in disability benefits, at the time of evaluation (χ2 = 21.3, p < .001). Only five patients in each group (3.8% of those with adequate effort, and 12.5% of those with inadequate effort) were involved in litigation at the time of neuropsychological evaluation (χ2 = 4.14, p < .05); such litigation was rarely related directly to the reason for referral to the Neuropsychology Clinic. The unadjusted means of the healthcare utilization variables are shown in Table 3. The number of ED visits in the year following neuropsychological evaluation was significantly higher for patients who had exerted inadequate effort than for those who had provided adequate effort. Patients who had exerted inadequate effort were also hospitalized more times than patients who had exerted adequate effort. These results are especially striking since the patients who had exerted adequate effort actually had a significantly higher number of comorbidities than patients with inadequate effort. In the next set of analyses (Table 4), group differences were adjusted for number of individual comorbidities. The model initially also adjusted for race and gender, but neither of these variables was associated with utilization, and they were removed from the model. After adjusting for number of individual comorbidities, inadequate effort was associated with a rate of 177% more ED visits, 236% more inpatient hospitalizations, and 217% more inpatient days, compared to patients who had exerted adequate Table 3. Unadjusted mean (SD) healthcare utilization as a function of effort on neuropsychological evaluation
Number of ED visits * Number of hospitalizations* Total days of hospitalization Number of comorbidities* *p < .05.
Adequate effort (n = 283)
Inadequate effort (n = 72)
0.61 (1.19) 0.17 (0.54) 0.86 (3.65) 4.47 (1.61)
1.01 (1.71) 0.33 (0.79) 1.44 (4.39) 4.02 (1.40)
710
MICHAEL DAVID HORNER ET AL.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
Table 4. Incidence rate ratios (IRR) and 95% confidence interval (CI) of adjusted association of inadequate effort with Emergency Department (ED), Number of Hospitalizations, and Days of Hospitalization
Inadequate Effort (Reference: Adequate Effort) Number of Comorbidities
ED visits IRR
95% CI
Number of Hospitalizations IRR
95% CI
Days of Hospitalization IRR
95% CI
1.77*
1.34:2.34
2.36*
1.43:3.90
2.17*
1.72:2.74
1.14*
1.05:1.23
1.35*
1.70:1.56
1.55*
1.45:1.65
*p < .05.
effort. The unadjusted mean utilization associated with inadequate effort translated into an estimated 2013 increased VA cost per patient of $466 for ED visits and $649 for inpatient days.
DISCUSSION In this study, patients who were seen clinically for neuropsychological evaluation were followed for 1 year afterward to track their healthcare utilization. After adjusting for number of individual comorbidities, patients who had exerted inadequate effort on examination were found to have more Emergency Department visits, more inpatient hospitalizations, and more days of inpatient hospitalization in the year after evaluation than did patients who had exerted adequate effort. These findings were not attributable to demographic factors or to medical or psychiatric comorbidities. It is well known that, when patients exert inadequate effort on examination, they often provide invalid data that are of limited clinical utility. Thus, from the standpoint of the evaluation alone, such patients utilize significant healthcare resources (e.g., clinicians’ time, use of limited time slots for evaluation) without corresponding clinical benefit. The present data show that, in addition, these patients continue to utilize resources more than patients who exert adequate effort. Comparison of the current findings with previous studies of healthcare utilization is complicated by varying definitions of utilization categories, and by differences between Veteran and non-Veteran populations. Nevertheless, data from the National Ambulatory Medical Care Survey for 2010 indicate that men aged 25–44 had an estimated 0.37 ED visits per year (U.S. National Ambulatory Medical Care Survey, 2013). In our sample, mean number of ED visits was 0.61 for the adequate effort group, and 1.01 for the inadequate effort group; these numbers are substantially higher than those of the national survey but, importantly, our sample was substantially older. Similarly, Cohen et al. (2010) examined healthcare utilization in Veterans of Operation Enduring Freedom/Operation Iraqi Freedom with and without mental health conditions. That study reported 0.54 ED visits per year among Veterans with mental health conditions excluding PTSD, 0.60 for those with PTSD, and 0.31 for Veterans without mental health conditions, somewhat closer to the values reported in the present
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
711
study. Cohen et al. (2010) also found that Veterans with mental health conditions excluding PTSD had a mean of 0.042 inpatient hospitalizations per year, those with PTSD had 0.044 hospitalizations, and Veterans without a mental health condition had 0.014 hospitalizations. Our sample showed a higher number of hospitalizations, but included Veterans from all eras, and was thus older than the sample studied by Cohen et al. It remains unclear why patients who demonstrated inadequate effort later showed increased ED visits and inpatient hospitalizations, relative to other patients. One possibility is that, since neuropsychological evaluation data are inconclusive, accurate diagnosis and treatment planning are substantially hampered. Thus, as these patients’ medical or psychiatric conditions remain unresolved, they may continue to seek and require additional evaluations or visits to aid in diagnostic clarification, or might even receive treatment for inaccurate diagnoses. Conversely, they might continue to receive unnecessary or inappropriate treatments, if their correct diagnoses have not been established. Another possibility is that inadequate effort on examination serves as a “marker” for these patients’ more general lack of full cooperation in their own health care. Thus, patients who (for whatever reason) do not consistently provide valid diagnostic data might also not be fully adherent with other aspects of their treatment. Poor adherence could then necessitate increased medical care and use of resources, including ED visits and hospitalizations. The rather high number of comorbidities in the present sample would render such increased utilization even more likely. A third possible explanation for the observed findings is that inadequate effort in at least some patients could have reflected intentional production or embellishment of symptoms. Such patients could be more likely to seek medical encounters, such as ED visits, and to exaggerate medical as well as cognitive symptoms, leading to increased healthcare utilization. The present study identified a group of patients who had exerted inadequate effort on neuropsychological testing. Importantly, for the majority of these patients, no definitive statement can be made about the underlying motivation or explanation for this lack of full cooperation with the examination. While in some cases it might have reflected deliberate exaggeration or production of cognitive deficit, for most patients this cannot be assumed. Patients who exerted inadequate effort might have done so for various reasons, such as lack of engagement in the evaluation process, more general apathy or decreased motivation as can be present in depressive disorders, distrust in the examiner or the healthcare system, or various other factors (Carone et al., 2010; Iverson, 2006). As few other published data exist regarding the relationship between healthcare utilization and effort on neuropsychological evaluation, the present results will require replication. Several factors might limit the generalizability of the findings. First, the sample was drawn from patients referred to a general VAMC Neuropsychology Clinic. As such, most patients were men, and medical and psychiatric comorbidities were very common. Second, only a small subset of healthcare utilization data were examined. This study focused on ED visits and inpatient hospitalization, costly resources the use of which can affect financial considerations for patients, healthcare organizations, and society as a whole. Nevertheless, other VA healthcare utilization data were not examined in this study, such as outpatient appointments and medication use. Furthermore, this study
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
712
MICHAEL DAVID HORNER ET AL.
captured only those ED visits and hospitalizations at the Ralph H. Johnson VAMC. That is, visits to other Emergency Departments (perhaps closer to some patients’ homes than the VAMC) and hospitalizations elsewhere in the community were not counted in the analyses. Cost estimates were based on mean costs from a large national VA database, not the specific individual patients’ costs. Third, the present study examined healthcare utilization for only 1 year after neuropsychological evaluation. It is unknown whether the patterns observed in the present data would remain after longer periods of follow-up. In the absence of accurate diagnosis provided by neuropsychological evaluation for patients with inadequate effort, higher rates of utilization would be expected to continue well beyond the time frame of this study. Finally, for patients included in this study, effort was determined by the TOMM much more frequently than by the Word Memory Test. Previous literature (Gervais, Rohling, Green, & Ford, 2004), however, has indicated that the TOMM is less sensitive than the Word Memory Test to inadequate effort. It is possible that some patients in the “adequate effort” group had not, in fact, exerted adequate effort. Thus, had a more sensitive performance validity test been used more frequently, even greater group differences in healthcare utilization might have been identified. Regardless, in the present sample inadequate effort on neuropsychological examination was clearly associated with increased use of healthcare resources. These findings argue for the need for interventions or techniques to increase the proportion of patients who exert adequate effort on examination. Patients who exert inadequate effort may be less likely to have all of their clinical needs addressed effectively, and accurate diagnoses and treatment recommendations often cannot be made. The present study also suggests that there is an additional resource burden on individuals and healthcare systems when neuropsychological examination does not yield valid data. In our laboratory, we are currently testing a theoretically derived, low-cost, verbal intervention that might effectively encourage patients to exert adequate effort on neuropsychological examination. Such interventions that effectively motivate patients to exert adequate effort on neuropsychological evaluation would improve the care afforded to these patients, while helping to reduce the unnecessary resource burden and costs to individuals and healthcare systems. ACKNOWLEDGMENT This work was supported by a grant from the American Academy of Clinical Neuropsychology Foundation to Dr. VanKirk. DISCLOSURE STATEMENT There are no conflicts of interest to disclose.
REFERENCES Barker, M. D., Horner, M. D., & Bachman, D. L. (2010). Embedded indices of effort in the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in a geriatric sample. The Clinical Neuropsychologist, 24, 1064–1077.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
713
Bortnik, K. E., Horner, M. D., & Bachman, D. L. (2013). Performance on standard indexes of effort among patients with dementia. Applied Neuropsychology: Adult, 20, 233–242. Carone, D. A., Iverson, G. L., & Bush, S. S. (2010). A model to approaching and providing feedback to patients regarding invalid test performance in clinical neuropsychological evaluations. The Clinical Neuropsychologist, 24, 759–778. doi:10.1080/13854041003712951 Centers for Disease Control/National Center for Health Statistics (2013). National Hospital Ambulatory Medical Care Survey: 2010 Emergency Department Summary Tables. http:// www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2010_ed_web_tables.pdf. Cohen, B. E., Gima, K., Bertenthal, D., Kim, S., Marmar, C. R., & Seal, K. H. (2010). Mental health diagnoses and utilization of VA non-mental health medical services among returning Iraq and Afghanistan Veterans. Journal of General Internal Medicine, 25, 18–24. Gervais, R. O., Rohling, M. L., Green, P., & Ford, W. (2004). A comparison of WMT, CARB, and TOMM failure rates in non-head injury disability claimants. Archives of Clinical Neuropsychology, 19, 475–487. Green, P. (2003). Word Memory Test Manual for Windows: User’s manual. Edmonton, Alberta, Canada: Green’s Publishing. Horner, M. D., Bedwell, J. S., & Duong, A. (2006). Abbreviated form of the Test of Memory Malingering. International Journal of Neuroscience, 116, 1181–1186. Iverson, G. L. (2006). Ethical issues associated with the assessment of exaggeration, poor effort, and malingering. Applied neuropsychology, 13, 77–90. Mittenberg, W., Patton, C., Canyock, E. M., & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 24(8), 1094–1102. Schroeder, R. W., Twumasi-Ankrah, P., Baade, L. E., & Marshall, P. S. (2012). Reliable Digit Span: A systematic review and cross-validation study. Assessment, 19, 21–30. doi:10.1177/ 1073191111428764 Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. Clinical Neuropsychologist, 13, 545–561. doi:10.1076/1385-4046(199911)13:04;1-y;ft545 Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM) manual. North Tonawanda, NY: Multi-Health Systems Inc. VanKirk, K. K., Horner, M. D., Turner, T. H., Dismuke, C. E., & Muzzy, W. (2013). Hospital service utilization is reduced following neuropsychological evaluation in a sample of U.S. veterans. The Clinical Neuropsychologist, 27, 750–761.
The Clinical Neuropsychologist Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ntcn20
Inadequate Effort on Neuropsychological Evaluation is Associated With Increased Healthcare Utilization ab
ab
cd
Michael David Horner , Kathryn K. VanKirk , Clara E. Dismuke , abe
Travis H. Turner
bf
& Wendy Muzzy
a
Mental Health Service, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA b
Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
Click for updates
c
Center for Disease Prevention and Health Interventions for Diverse Populations, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA d
Center For Health Disparities Research, Department of Internal Medicine, Medical University of South Carolina, Charleston, SC, USA e
Department of Neurosciences, Medical University of South Carolina, Charleston, SC, USA f
Charleston Research Institute, Charleston, SC, USA Published online: 16 Jun 2014.
To cite this article: Michael David Horner, Kathryn K. VanKirk, Clara E. Dismuke, Travis H. Turner & Wendy Muzzy (2014) Inadequate Effort on Neuropsychological Evaluation is Associated With Increased Healthcare Utilization, The Clinical Neuropsychologist, 28:5, 703-713, DOI: 10.1080/13854046.2014.925143 To link to this article: http://dx.doi.org/10.1080/13854046.2014.925143
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
The Clinical Neuropsychologist, 2014 Vol. 28, No. 5, 703–713, http://dx.doi.org/10.1080/13854046.2014.925143
Inadequate Effort on Neuropsychological Evaluation is Associated With Increased Healthcare Utilization Michael David Horner1,2, Kathryn K. VanKirk1,2, Clara E. Dismuke3,4, Travis H. Turner1,2,5, and Wendy Muzzy2,6 Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
1
Mental Health Service, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA 2 Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA 3 Center for Disease Prevention and Health Interventions for Diverse Populations, Ralph H. Johnson Department of Veterans Affairs Medical Center, Charleston, SC, USA 4 Center For Health Disparities Research, Department of Internal Medicine, Medical University of South Carolina, Charleston, SC, USA 5 Department of Neurosciences, Medical University of South Carolina, Charleston, SC, USA 6 Charleston Research Institute, Charleston, SC, USA Patients who exert inadequate effort on neuropsychological examination might not receive accurate diagnoses and recommendations, and might not cooperate fully with other aspects of healthcare. This study examined whether inadequate effort is associated with increased healthcare utilization. Of 355 patients seen for routine, clinical neuropsychological examination at a VA Medical Center, 283 (79.7%) showed adequate effort and 72 (20.3%) showed inadequate effort, as determined at time of evaluation using the Word Memory Test and/or Test of Memory Malingering. Utilization data included number of Emergency Department (ED) visits and inpatient hospitalizations in the year following evaluation. Patients who had shown inadequate effort on examination had more Emergency Department visits, more inpatient hospitalizations, and more days of inpatient hospitalization in the year after evaluation, compared to patients who had exerted adequate effort. This finding was not attributable to group differences in age or medical/ psychiatric comorbidities. Thus, patients who exerted inadequate effort showed greater healthcare utilization in the year following evaluation. Such patients might use more resources since diagnostic evaluations are inconclusive. Inadequate effort on examination might also serve as a “marker” for more general failure to cooperate fully in one’s healthcare, possibly resulting in greater utilization. Keywords: Health care utilization; Neuropsychological assessment; Clinical neuropsychology; Health outcomes; Effort.
INTRODUCTION It is well established that many patients do not exert adequate effort on neuropsychological evaluation, producing invalid findings that generally overestimate their degree of cognitive impairment. For example, Mittenberg, Patton, Canyock, and Condit (2002) reported estimated prevalence of inadequate effort in 8% to 39% of evaluations across
Address correspondence to: Michael David Horner, Ralph H. Johnson Department of Veterans Affairs Medical Center, Mental Health Service 116, 109 Bee Street, Charleston, SC 29401-5799, USA. E-mail: [email protected] (Received 31 October 2013; accepted 12 May 2014)
© 2014 Taylor & Francis
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
704
MICHAEL DAVID HORNER ET AL.
various settings. Typically, such invalid findings occur when patients are not fully engaged in the evaluation process, for reasons that might include apathy, disinterest in the evaluation or its outcome, fears that the examination will be insufficiently sensitive to deficits, distrust of the examiner or healthcare system, or deliberate embellishment or feigning of cognitive deficit (Carone, Iverson, & Bush, 2010; Iverson, 2006). Invalid findings due to inadequate effort are often of little utility in diagnosing cognitive disorders or in treatment planning. (For the purposes of this paper, we use the term “inadequate effort” to refer to any instance in which a patient, for whatever reason, intentional or unintentional, does not try sufficiently hard on cognitive tests to provide a valid estimate of his or her cognitive ability.) Thus, patients who exert inadequate effort on evaluation might not receive accurate diagnoses or necessary clinical services. In turn, they might continue to receive additional, unnecessary diagnostic services because their diagnosis and cognitive status remain unknown. Similarly, such patients might continue to receive treatment for conditions that are not actually present (e.g., enduring sequelae of traumatic brain injury), and might not receive treatment for conditions that truly are present (e.g., mood or anxiety disorders, or psychosocial problems). Further, expenditure of resources to treat conditions that are not truly present would deplete such resources for patients who truly have those conditions. We thus wondered whether inadequate effort on neuropsychological examination might be associated with increased healthcare utilization and costs. Broadly speaking, such an association would help illuminate the costs and benefits associated with neuropsychological evaluation in the context of larger healthcare systems. To our knowledge, no published studies to date have addressed healthcare utilization or similar outcomes associated with inadequate effort on neuropsychological examinations. We hypothesized that, relative to adequate effort on examination, inadequate effort would be associated with increased healthcare utilization and costs, operationalized as inpatient hospitalizations and visits to the Emergency Department (ED). METHOD Design Data were collected as part of a larger study of healthcare utilization in patients who had undergone neuropsychological evaluation (VanKirk, Horner, Turner, Dismuke, & Muzzy, 2013). The study consisted of a retrospective analysis of hospital service utilization (e.g., ED visits and inpatient hospitalizations) in patients who underwent neuropsychological evaluation in the course of clinical care between 2003 and 2010. Evaluations were conducted in the Neuropsychology Clinic at the Ralph H. Johnson Department of Veterans Affairs Medical Center (VAMC), which serves over 40,000 Veterans in 15 counties along the South Carolina and Georgia coastline. The project was reviewed and approved by the Institutional Review Board of the Medical University of South Carolina and the VA Research and Development Service. Participants The initial sample consisted of all patients who had undergone clinical, outpatient evaluation of cognitive status in the Ralph H. Johnson VAMC’s Neuropsychology
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
705
Clinic between the years 2003 and 2010 (n = 604). Patients who had been referred for differential diagnosis of psychiatric conditions only were not included; neither were patients seen for neuropsychological evaluation in the VAMC’s other clinics (e.g., Memory Disorders Clinic for geriatric patients). Individuals referred for inpatient evaluation were also excluded, as their inclusion would have confounded the utilization outcome measures. While the majority of referrals were generated through Mental Health (41%), Neurology (22%), and Primary Care (21%), the Neuropsychology Clinic’s referral base is facility-wide; other referrals came from other outpatient clinics including TBI Clinic and Infectious Diseases Clinic. The sample included only patients who were referred purely as part of their clinical care. Specifically, the sample did not include patients who were seen for VA Compensation and Pension examinations, a specific type of evaluation within the VA system that is used in determination of military service-connected disability benefits. (Disabling conditions that have been incurred or aggravated during active military service are considered to be “service-connected,” and may be eligible for financial compensation.) No patients in the sample were evaluated as part of a legal claim, or for reasons other than clinical care. Patients for whom there was not complete access to hospital service utilization data (e.g., inpatient admissions, Emergency Department visits, etc. as described below) for the 12 months following evaluation were then excluded. Patients were also excluded if they died or transferred their health care to another VAMC within the 12 months following evaluation, or if data regarding medical and psychiatric comorbidities were not available. For patients who had been evaluated on more than one occasion, only the first evaluation was included in the analyses. Further details of the design and methodology are described by VanKirk et al. (2013) and are not repeated here. Finally, patients were classified according to whether they had exerted adequate effort on neuropsychological evaluation. As a standardized battery is not administered to all patients referred to the Neuropsychology Clinic, not all patients were given the same performance validity tests as part of their neuropsychological evaluation. For the present study, patients were included in the final sample only if they had been administered either the Word Memory Test (WMT; Green, 2003) or the Test of Memory Malingering (TOMM; Tombaugh, 1996). Standard cutoff scores described in each test’s manual were used to determine whether effort was adequate or suboptimal. There were 57 patients who were considered to have passed the TOMM after scoring 47 or higher on Trial 1, without administration of Trial 2, an abbreviated form of the test that has been described previously (Horner, Bedwell, & Duong, 2006). However, as determination of adequate vs. inadequate effort in clinical practice is not typically made solely on the basis of a single test, we used a combination of criteria similar to those proposed by Slick, Sherman, and Iverson (1999), and to those previously reported by our laboratory (e.g., Barker, Horner, & Bachman, 2010; Bortnik, Horner, & Bachman, 2013). In addition to the TOMM and WMT, determination of effort typically took into account embedded effort indices such as Reliable Digit Span (for a review see Schroeder, Twumasi-Ankrah, Baade, & Marshall, 2012), as well as behavioral factors including (1) deficits on testing that were grossly disproportionate to the patient’s observed or reported functional level (e.g., a patient who was amnestic on testing and yet provided rich details about his activities of the preceding few days); (2) implausible errors, patterns of responses, or test scores (e.g., an independently
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
706
MICHAEL DAVID HORNER ET AL.
functional patient who was reportedly unable to remember highly overlearned information that even very severely impaired patients can recall without difficulty, or better performance on harder than on easier tasks throughout the evaluation); and (3) other clear behavioral evidence of inadequate effort (e.g., a patient’s explicit statement that he was not trying his best or was disinterested in the examination, or a clear pattern of giving up easily despite apparent ability to complete tasks). Clinicians’ judgments of these behaviors were made at the time of clinical examination, well before the present study was conceived or executed. These observations were thus made as part of routine, clinical examination, and were not standardized across examiners for research purposes. In order to be assigned to the inadequate effort group, a patient had to have failed either the TOMM or the WMT, in addition to displaying clearly at least one of the other behavioral indicators described above. Similarly, in order to be assigned to the adequate effort group a patient could not have failed either TOMM or WMT, and could not have displayed the behaviors suggestive of inadequate effort described above. Patients who passed effort tests but were judged at the time of examination to have exerted inadequate effort, or who failed effort tests but who did not exhibit other behavioral indicators of inadequate effort, were excluded from the study sample. The final sample consisted of 355 Veterans who completed outpatient, clinically indicated neuropsychological evaluation, for whom complete effort data and healthcare utilization data were available. There were 28 patients who had been administered the WMT and 336 who had been administered the TOMM; 9 patients were administered both tests. Table 1 displays patient demographics. Psychiatric diagnoses that were assigned based on the neuropsychological evaluations are listed in Table 2. Procedures Demographic and healthcare utilization data were collected as part of a larger study (VanKirk et al., 2013). Relevant clinical information for each patient, including demographic data and DSM-IV-TR diagnoses assigned at the time of neuropsychological evaluation, were extracted from the Ralph H. Johnson VAMC Neuropsychology Clinic database and the VA’s electronic medical record system (CPRS). In addition, for each participant, information regarding primary outcome measures (i.e., number of ED visits, number of hospitalizations, and total days of hospitalization) was collected for a period of 12 months following completion of the evaluation. Evaluation was considered complete when the final report was posted to the patient’s medical record, which effectively alerts providers to results of the evaluation. Beginning in 2007, approximately halfway through the period for which neuropsychological and healthcare utilization data were collected, we began systematically Table 1. Mean (SD) values for demographic characteristics of the sample
Age Education (years)* Percent male *p < .05.
Adequate effort (N = 283)
Inadequate effort (N = 72)
52.1(13.9) 13.2 (2.5) 88.0
50.5 (13.1) 12.4 (2.4) 90.3
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
707
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
Table 2. DSM-IV-TR diagnoses assigned at time of neuropsychological evaluation Diagnosis
N
Dementia (includes Dementia of the Alzheimer Type, Vascular Dementia, others) Cognitive Disorder Not Otherwise Specified Learning Disorder Attention-Deficit/Hyperactivity Disorder
25 90 6 16
Major Depressive Disorder Dysthymic Disorder Bipolar Disorder Other Mood Disorder (includes Depressive Disorder NOS, Cyclothymia, others) Generalized Anxiety Disorder Post-Traumatic Stress Disorder Other Anxiety Disorder (includes Panic Disorder, Specific Phobia, others) Schizophrenia / Schizoaffective Disorder Other Psychotic Disorder Somatoform disorder Adjustment disorder Alcohol Abuse / Dependence Cocaine Abuse / Dependence Other Substance Abuse / Dependence
86 12 12 29 19 72 12 5 2 4 16 51 12 6
Personality Change Due to General Medical Condition Obsessive-Compulsive Personality Disorder Borderline personality disorder Other personality disorder Malingering / Factitious Disorder No Diagnosis / Diagnosis Deferred
4 6 4 6 19 33
Most patients met criteria for more than one diagnosis.
recording whether each patient seen for clinical neuropsychological evaluation was receiving service-connected disability benefits for psychiatric illness, brain injury, or other brain illness at the time of the evaluation. These data are readily available in Veterans’ medical records, and were collected as part of routine clinical examination, not for the purposes of the present study. At the same time we began asking all patients, as part of routine clinical examination, whether they were in the process of applying for disability benefits, or for an increase in disability benefits, at the time of the evaluation. These data were collected for 172 of the 355 patients in the sample. Finally, also in 2007, we began asking all patients whether they were involved in any other litigation or legal action through the courts at the time of evaluation. These data were available for 170 patients in the sample. As noted above, however, all participants were seen for clinical evaluation only; none of the evaluations was conducted for purposes of determination of disability benefits, litigation, or any other forensic consideration. As is characteristic of the VA patient population generally, some patients in the study were receiving VA disability benefits for service-connected conditions at the time of evaluation. The majority of these conditions (including, e.g., orthopedic and other medical conditions) were not related to
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
708
MICHAEL DAVID HORNER ET AL.
neurological illness or injury and had no specific relevance to the referral question prompting neuropsychological evaluation. Similarly, some patients happened to be in the process of applying for such benefits, or appealing a previous determination, at the time of examination; in many such cases the condition in question was unrelated to the reason for referral to the Neuropsychology Clinic. Finally, a very small number of patients in each group (see below under Results) were involved in other litigation at the time of evaluation, but this was typically unrelated to the reason for referral to Neuropsychology Clinic. Comorbidities were measured as a count of the following types of diagnoses appearing in each patient’s medical record: cancer, cognitive, cardiovascular, endocrine, gastrointestinal, infectious disease, liver, musculoskeletal, neurological, pain, psychiatric, pulmonary, sensory/perceptual, substance abuse, and other. All ED visits and hospitalizations were included for analysis, regardless of reason for visit or hospitalization. Criteria for an ED visit included being seen in the ED of the Ralph H. Johnson VAMC and evaluation by an ED physician or nurse practitioner. Hospitalizations were included only if length of inpatient hospitalization was greater than 24 hours. Length of hospitalization was determined by counting the total number of days a patient was hospitalized, regardless of the number of hours spent each day. For example, a patient admitted at 4 pm on a Friday and discharged at 9 am on a Monday was determined to have spent 4 days in the hospital. Mean VA cost estimates of ED visits and inpatient days were obtained from patient level costs of a sample of over 800,000 Veterans in 2010 (www.herc.research.va.gov) and were adjusted to 2013 dollar values based on the US Department of Labor’s Inflation Calculator. (www.bls.gov/data/ inflation_calculator.htm). Data analysis Group differences in age and education were analyzed using t-tests. Group difference in gender composition was analyzed by chi-square analysis. Data regarding patients’ disability benefits and litigation status were available only for a subset of the sample, as these data began to be collected systematically during the period of time for which neuropsychological and utilization data were studied. Group differences in these variables were analyzed using chi-square analyses. To address the study hypotheses, three types of statistical analyses were then conducted. First, the outcome variables of interest (number of ED visits, number of hospitalizations, and total days of hospitalization), as well as number of comorbidities, were tested for normality using skewness and kurtosis tests in order to ascertain the appropriate statistical tests to be used for differences in unadjusted and adjusted means. Second, due to the non-normal findings from the normality tests, number of ED visits, number of hospitalizations, and total days of hospitalization were compared between the two groups (adequate versus inadequate effort) using Wilcoxon (Mann-Whitney) rank sum tests for differences in central tendency. No truncation or other adjustment for outliers was performed. The difference in effort for the normally distributed variable, number of comorbidities, was tested using the Student t-test. Third, count models of ED visits, number of hospitalizations, and total days of hospitalization were estimated using Poisson regression to quantify the association of effort with ED visits, number of
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
709
hospitalizations, and total days of hospitalization while adjusting for individual comorbidities. All of these statistical tests were performed using STATA 9.0. The mean VA ED visit cost and the mean inpatient day cost were applied to the estimated difference in number of visits associated with adequate effort.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
RESULTS The groups did not differ in age or gender composition (Table 1). Patients who exerted inadequate effort on examination had less education than those who exerted adequate effort. However, as education was not significantly correlated with any of the outcome variables, it was not used as a covariate in subsequent group comparisons. Of the 172 patients for whom data were available (132 who had shown adequate effort and 40 who had shown inadequate effort), 45.0% of those who exerted adequate effort and 40.9% of those who exerted inadequate effort were service-connected for psychiatric illness, brain injury, or other brain illness at the time of evaluation; this difference was not statistically significant (χ2 = 0.21, ns). However, of this subsample, significantly more patients in the inadequate effort group (62.5%) than in the adequate effort group (23.5%) were in the process of seeking disability benefits, or an increase in disability benefits, at the time of evaluation (χ2 = 21.3, p < .001). Only five patients in each group (3.8% of those with adequate effort, and 12.5% of those with inadequate effort) were involved in litigation at the time of neuropsychological evaluation (χ2 = 4.14, p < .05); such litigation was rarely related directly to the reason for referral to the Neuropsychology Clinic. The unadjusted means of the healthcare utilization variables are shown in Table 3. The number of ED visits in the year following neuropsychological evaluation was significantly higher for patients who had exerted inadequate effort than for those who had provided adequate effort. Patients who had exerted inadequate effort were also hospitalized more times than patients who had exerted adequate effort. These results are especially striking since the patients who had exerted adequate effort actually had a significantly higher number of comorbidities than patients with inadequate effort. In the next set of analyses (Table 4), group differences were adjusted for number of individual comorbidities. The model initially also adjusted for race and gender, but neither of these variables was associated with utilization, and they were removed from the model. After adjusting for number of individual comorbidities, inadequate effort was associated with a rate of 177% more ED visits, 236% more inpatient hospitalizations, and 217% more inpatient days, compared to patients who had exerted adequate Table 3. Unadjusted mean (SD) healthcare utilization as a function of effort on neuropsychological evaluation
Number of ED visits * Number of hospitalizations* Total days of hospitalization Number of comorbidities* *p < .05.
Adequate effort (n = 283)
Inadequate effort (n = 72)
0.61 (1.19) 0.17 (0.54) 0.86 (3.65) 4.47 (1.61)
1.01 (1.71) 0.33 (0.79) 1.44 (4.39) 4.02 (1.40)
710
MICHAEL DAVID HORNER ET AL.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
Table 4. Incidence rate ratios (IRR) and 95% confidence interval (CI) of adjusted association of inadequate effort with Emergency Department (ED), Number of Hospitalizations, and Days of Hospitalization
Inadequate Effort (Reference: Adequate Effort) Number of Comorbidities
ED visits IRR
95% CI
Number of Hospitalizations IRR
95% CI
Days of Hospitalization IRR
95% CI
1.77*
1.34:2.34
2.36*
1.43:3.90
2.17*
1.72:2.74
1.14*
1.05:1.23
1.35*
1.70:1.56
1.55*
1.45:1.65
*p < .05.
effort. The unadjusted mean utilization associated with inadequate effort translated into an estimated 2013 increased VA cost per patient of $466 for ED visits and $649 for inpatient days.
DISCUSSION In this study, patients who were seen clinically for neuropsychological evaluation were followed for 1 year afterward to track their healthcare utilization. After adjusting for number of individual comorbidities, patients who had exerted inadequate effort on examination were found to have more Emergency Department visits, more inpatient hospitalizations, and more days of inpatient hospitalization in the year after evaluation than did patients who had exerted adequate effort. These findings were not attributable to demographic factors or to medical or psychiatric comorbidities. It is well known that, when patients exert inadequate effort on examination, they often provide invalid data that are of limited clinical utility. Thus, from the standpoint of the evaluation alone, such patients utilize significant healthcare resources (e.g., clinicians’ time, use of limited time slots for evaluation) without corresponding clinical benefit. The present data show that, in addition, these patients continue to utilize resources more than patients who exert adequate effort. Comparison of the current findings with previous studies of healthcare utilization is complicated by varying definitions of utilization categories, and by differences between Veteran and non-Veteran populations. Nevertheless, data from the National Ambulatory Medical Care Survey for 2010 indicate that men aged 25–44 had an estimated 0.37 ED visits per year (U.S. National Ambulatory Medical Care Survey, 2013). In our sample, mean number of ED visits was 0.61 for the adequate effort group, and 1.01 for the inadequate effort group; these numbers are substantially higher than those of the national survey but, importantly, our sample was substantially older. Similarly, Cohen et al. (2010) examined healthcare utilization in Veterans of Operation Enduring Freedom/Operation Iraqi Freedom with and without mental health conditions. That study reported 0.54 ED visits per year among Veterans with mental health conditions excluding PTSD, 0.60 for those with PTSD, and 0.31 for Veterans without mental health conditions, somewhat closer to the values reported in the present
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
711
study. Cohen et al. (2010) also found that Veterans with mental health conditions excluding PTSD had a mean of 0.042 inpatient hospitalizations per year, those with PTSD had 0.044 hospitalizations, and Veterans without a mental health condition had 0.014 hospitalizations. Our sample showed a higher number of hospitalizations, but included Veterans from all eras, and was thus older than the sample studied by Cohen et al. It remains unclear why patients who demonstrated inadequate effort later showed increased ED visits and inpatient hospitalizations, relative to other patients. One possibility is that, since neuropsychological evaluation data are inconclusive, accurate diagnosis and treatment planning are substantially hampered. Thus, as these patients’ medical or psychiatric conditions remain unresolved, they may continue to seek and require additional evaluations or visits to aid in diagnostic clarification, or might even receive treatment for inaccurate diagnoses. Conversely, they might continue to receive unnecessary or inappropriate treatments, if their correct diagnoses have not been established. Another possibility is that inadequate effort on examination serves as a “marker” for these patients’ more general lack of full cooperation in their own health care. Thus, patients who (for whatever reason) do not consistently provide valid diagnostic data might also not be fully adherent with other aspects of their treatment. Poor adherence could then necessitate increased medical care and use of resources, including ED visits and hospitalizations. The rather high number of comorbidities in the present sample would render such increased utilization even more likely. A third possible explanation for the observed findings is that inadequate effort in at least some patients could have reflected intentional production or embellishment of symptoms. Such patients could be more likely to seek medical encounters, such as ED visits, and to exaggerate medical as well as cognitive symptoms, leading to increased healthcare utilization. The present study identified a group of patients who had exerted inadequate effort on neuropsychological testing. Importantly, for the majority of these patients, no definitive statement can be made about the underlying motivation or explanation for this lack of full cooperation with the examination. While in some cases it might have reflected deliberate exaggeration or production of cognitive deficit, for most patients this cannot be assumed. Patients who exerted inadequate effort might have done so for various reasons, such as lack of engagement in the evaluation process, more general apathy or decreased motivation as can be present in depressive disorders, distrust in the examiner or the healthcare system, or various other factors (Carone et al., 2010; Iverson, 2006). As few other published data exist regarding the relationship between healthcare utilization and effort on neuropsychological evaluation, the present results will require replication. Several factors might limit the generalizability of the findings. First, the sample was drawn from patients referred to a general VAMC Neuropsychology Clinic. As such, most patients were men, and medical and psychiatric comorbidities were very common. Second, only a small subset of healthcare utilization data were examined. This study focused on ED visits and inpatient hospitalization, costly resources the use of which can affect financial considerations for patients, healthcare organizations, and society as a whole. Nevertheless, other VA healthcare utilization data were not examined in this study, such as outpatient appointments and medication use. Furthermore, this study
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
712
MICHAEL DAVID HORNER ET AL.
captured only those ED visits and hospitalizations at the Ralph H. Johnson VAMC. That is, visits to other Emergency Departments (perhaps closer to some patients’ homes than the VAMC) and hospitalizations elsewhere in the community were not counted in the analyses. Cost estimates were based on mean costs from a large national VA database, not the specific individual patients’ costs. Third, the present study examined healthcare utilization for only 1 year after neuropsychological evaluation. It is unknown whether the patterns observed in the present data would remain after longer periods of follow-up. In the absence of accurate diagnosis provided by neuropsychological evaluation for patients with inadequate effort, higher rates of utilization would be expected to continue well beyond the time frame of this study. Finally, for patients included in this study, effort was determined by the TOMM much more frequently than by the Word Memory Test. Previous literature (Gervais, Rohling, Green, & Ford, 2004), however, has indicated that the TOMM is less sensitive than the Word Memory Test to inadequate effort. It is possible that some patients in the “adequate effort” group had not, in fact, exerted adequate effort. Thus, had a more sensitive performance validity test been used more frequently, even greater group differences in healthcare utilization might have been identified. Regardless, in the present sample inadequate effort on neuropsychological examination was clearly associated with increased use of healthcare resources. These findings argue for the need for interventions or techniques to increase the proportion of patients who exert adequate effort on examination. Patients who exert inadequate effort may be less likely to have all of their clinical needs addressed effectively, and accurate diagnoses and treatment recommendations often cannot be made. The present study also suggests that there is an additional resource burden on individuals and healthcare systems when neuropsychological examination does not yield valid data. In our laboratory, we are currently testing a theoretically derived, low-cost, verbal intervention that might effectively encourage patients to exert adequate effort on neuropsychological examination. Such interventions that effectively motivate patients to exert adequate effort on neuropsychological evaluation would improve the care afforded to these patients, while helping to reduce the unnecessary resource burden and costs to individuals and healthcare systems. ACKNOWLEDGMENT This work was supported by a grant from the American Academy of Clinical Neuropsychology Foundation to Dr. VanKirk. DISCLOSURE STATEMENT There are no conflicts of interest to disclose.
REFERENCES Barker, M. D., Horner, M. D., & Bachman, D. L. (2010). Embedded indices of effort in the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in a geriatric sample. The Clinical Neuropsychologist, 24, 1064–1077.
Downloaded by [Gebze Yuksek Teknoloji Enstitïsu ] at 17:55 21 December 2014
INADEQUATE EFFORT AND HEALTHCARE UTILIZATION
713
Bortnik, K. E., Horner, M. D., & Bachman, D. L. (2013). Performance on standard indexes of effort among patients with dementia. Applied Neuropsychology: Adult, 20, 233–242. Carone, D. A., Iverson, G. L., & Bush, S. S. (2010). A model to approaching and providing feedback to patients regarding invalid test performance in clinical neuropsychological evaluations. The Clinical Neuropsychologist, 24, 759–778. doi:10.1080/13854041003712951 Centers for Disease Control/National Center for Health Statistics (2013). National Hospital Ambulatory Medical Care Survey: 2010 Emergency Department Summary Tables. http:// www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2010_ed_web_tables.pdf. Cohen, B. E., Gima, K., Bertenthal, D., Kim, S., Marmar, C. R., & Seal, K. H. (2010). Mental health diagnoses and utilization of VA non-mental health medical services among returning Iraq and Afghanistan Veterans. Journal of General Internal Medicine, 25, 18–24. Gervais, R. O., Rohling, M. L., Green, P., & Ford, W. (2004). A comparison of WMT, CARB, and TOMM failure rates in non-head injury disability claimants. Archives of Clinical Neuropsychology, 19, 475–487. Green, P. (2003). Word Memory Test Manual for Windows: User’s manual. Edmonton, Alberta, Canada: Green’s Publishing. Horner, M. D., Bedwell, J. S., & Duong, A. (2006). Abbreviated form of the Test of Memory Malingering. International Journal of Neuroscience, 116, 1181–1186. Iverson, G. L. (2006). Ethical issues associated with the assessment of exaggeration, poor effort, and malingering. Applied neuropsychology, 13, 77–90. Mittenberg, W., Patton, C., Canyock, E. M., & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 24(8), 1094–1102. Schroeder, R. W., Twumasi-Ankrah, P., Baade, L. E., & Marshall, P. S. (2012). Reliable Digit Span: A systematic review and cross-validation study. Assessment, 19, 21–30. doi:10.1177/ 1073191111428764 Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. Clinical Neuropsychologist, 13, 545–561. doi:10.1076/1385-4046(199911)13:04;1-y;ft545 Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM) manual. North Tonawanda, NY: Multi-Health Systems Inc. VanKirk, K. K., Horner, M. D., Turner, T. H., Dismuke, C. E., & Muzzy, W. (2013). Hospital service utilization is reduced following neuropsychological evaluation in a sample of U.S. veterans. The Clinical Neuropsychologist, 27, 750–761.
![Measurement of the [Formula: see text] production cross-section using [Formula: see text] events with [Formula: see text]-tagged jets in [Formula: see text] collisions at [Formula: see text] and 8 TeV with the ATLAS detector.](/assets/img/hold.png)
