APPLIED NEUROPSYCHOLOGY: ADULT, 22: 204–208, 2015 Copyright # Taylor & Francis Group, LLC ISSN: 2327-9095 print=2327-9109 online DOI: 10.1080/23279095.2014.902836
Examination of the Role of Expectancies on Task Performance in College Students Concerned about ADHD Christina Wei and Julie A. Suhr Department of Psychology, Ohio University, Athens, Ohio
Prior research has shown that performance on cognitive tasks can be influenced by expectations (Smith & Sullivan, 2003; Suhr & Gunstad, 2002, 2005). The current study examined whether cuing a belief about the diagnostic saliency of a cognitive task among young adults who expressed concern about having attention-deficit hyperactivity disorder (ADHD) influenced task performance. Participants were randomly assigned to either receive neutral directions or be cued to a belief that the task had diagnostic saliency prior to completing a working-memory task. Supporting our hypothesis, college students with higher prestudy report of ADHD symptoms who were cued with a belief about the diagnostic saliency of the task performed worse compared with students who received neutral instructions. As many researchers and clinicians currently rely exclusively on self-reported symptoms and neuropsychological tests to diagnose ADHD, our findings highlight the importance of comprehensive assessment for provision of appropriate clinical services to adults presenting with ADHD concerns.
Key words:
assessment=diagnosis, cognition, neuropsychology
College students frequently refer themselves for attention-deficit hyperactivity disorder (ADHD) evaluation. Unfortunately, diagnosis of ADHD for the first time in relatively healthy young adults is a difficult task, and often self-report becomes the relied-upon factor for diagnosis (Stefanatos & Baron, 2007). This is of concern because individuals diagnosed with ADHD qualify for academic accommodations and prescriptions for psychostimulant medications. Given evidence that those medications are often misused and even abused and resold, accuracy of diagnostic methods is paramount to provision of appropriate clinical services (Davidson, 2008; Sullivan, May & Galbally, 2007). There are several problems with reliance on self-report in diagnosis of ADHD in adults. First, self-report ADHD measures are not specific to the disorder; individuals with other psychological conditions endorse ADHD symptoms at a high rate (Harrison, 2004; McCann & Address correspondence to Christina Wei, Ph.D., Department of Psychology, Ohio University, 200 Porter Hall, Athens, OH 45701. E-mail: [email protected]
Roy-Byrne, 2004; Schoechlin & Engel, 2005). In addition, self-report ADHD measures have been shown to be vulnerable to malingering (Booksh, Pella, Singh, & Gouvier, 2009; Conti, 2004; Jachimowicz & Geiselman, 2004; Quinn, 2003; Suhr, Hammers, Dobbins-Buckland, Zimak, & Hughes, 2008; Sullivan et al., 2007). Further, healthy individuals may misattribute subclinical behaviors as both pathological and symptomatic of ADHD, leading to overendorsement of symptoms (Suhr & Wei, 2013, in press). Because base rates of subclinical ‘‘ADHD’’ symptoms are high in the general population, including college students (DuPaul et al., 2001; Faraone & Biederman, 2005; Lewandowski, Lovett, Codding, & Gordon, 2008; Mannuzza, Klein, Klein, Bessler, & Shrout, 2002; Murphy & Barkley, 1996; Suhr et al., 2008; Suhr, Zimak, Buelow, & Fox, 2009), individuals exposed to information about symptoms and causes of ADHD may misattribute everyday problems with attention or focus as symptoms of ADHD (Conrad & Potter, 2000; Suhr & Wei, 2013, in press). One way in which this might occur is through the influence of societal beliefs about what illnesses to expect and what ‘‘symptoms’’
EXAMINATION OF THE ROLE OF EXPECTANCIES
are associated with illnesses (Hahn, 1999). In some cases, individuals may strongly identify with having ADHD and hold beliefs about the types of symptoms and impairments that result from ADHD (Suhr & Wei, 2013, in press). Development of this type of ‘‘illness identity’’ may lead to misattribution of common everyday errors as pathological ‘‘symptoms’’ of the disorder, as well as behaving in ways consistent with the disorder, as might be seen on neuropsychological tests. Similar to the pattern seen on ADHD self-report measures, poor performance on neuropsychological measures of constructs such as sustained attention, working memory, and inhibition are not specific to ADHD (Nigg, 2009). In addition, performance on neuropsychological measures can be easily malingered (Booksh et al., 2009; Harrison, Edwards, & Parker, 2007; Suhr, Buelow, & Riddle, 2011; Suhr et al., 2008; Sullivan et al., 2007). Furthermore, behavioral performance on neuropsychological measures is vulnerable to negative expectations that can arise due to belief that one has a disorder. Expectations have been found to have small-to-moderate effects on performance on neuropsychological tests in individuals diagnosed with chronic fatigue syndrome, mild head injuries, and recurrent headaches, as well as in pregnant women and substance abusers (see Smith & Sullivan, 2003 and Suhr & Wei, 2013, for a review). When situationally cued, an individual who identifies with an illness and has negative beliefs about the course and consequences of the illness can behave in ways consistent with expectations. More specifically, an individual may expect to do poorly on an attention task based on their ADHD illness identity. Reminding the individual of their ADHD illness identity, coupled with emphasis on the diagnostic saliency of the task at hand, may trigger expectations of poor performance, which can result in performance deficits. The current study examined the impact of diagnostic saliency on working-memory performance in college students who expressed concern that they may have ADHD. We hypothesized that among college students with a higher level of ADHD illness identity (defined by higher levels of prestudy ADHD symptoms), those cued to a belief that a task had diagnostic saliency would perform worse on a complex working-memory and attention task compared with those who were not cued to the diagnostic saliency of the task. Thus, we expected an interaction between prestudy ADHD identity and exposure to diagnosis cue. METHODS Participants Participants were 72 undergraduates aged 18 to 32 years old attending a Midwestern university. Most participants were freshmen (57%), female (60%), and Caucasian
205
(89%). Eight percent of participants reported having a diagnosis of depression, and 10% reported having a diagnosis of an anxiety disorder. With regard to report of any history of head injury, 25% reported a history of mild head injury. Seventy-eight percent of participants reported alcohol use, 22% reported marijuana use, and 14% reported nicotine use. Measures Adult ADHD Self-Report Scale (Kessler et al., 2005). The Adult ADHD Self-Report Scale (ASRS) measures the self-report of Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition (DSM-IV) symptoms of ADHD in adults. The ASRS has good sensitivity (56.3%), specificity (98.3%), and total classification accuracy (96.2%) in comparison with healthy controls (Kessler et al., 2005). In the present study, the ASRS score from the prescreening session was used for participant selection (as outlined) and was also used as a covariate relevant to the study hypotheses (see Results section). The ASRS demonstrated good internal consistency reliability for the current study (.82). Concern about ADHD. Prior to participating in the study, as part of the prescreening, students were asked to rate themselves on the following item: ‘‘How concerned are you that you may have symptoms of ADHD?’’ This question was rated on a Likert scale ranging from 1 (not concerned at all) to 10 (very concerned). This measure was used for participant selection. Beck Depression Inventory-II (Beck, Steer, & Brown, 1996). The Beck Depression Inventory-II (BDI-II) was used to assess depressive symptoms. The internal consistency for college students has been found to be strong (a ¼ .93; Beck et al., 1996), and in the present study, it was .87. The BDI-II has generally been found to have adequate sensitivity (71%) and specificity (88%) in the diagnosis of depression (Beck et al., 1996). The BDI-II score was used in the present study to remove individuals who endorsed unusually high amounts of depressive symptoms. Dual 2-back (Kirchner, 1958). This computerized version of a classic working-memory and attention task required participants to press a designated computer key when they heard a spoken letter that matched the letter they heard two back, while simultaneously pressing a different computer key when they saw a block light up within a nine-block span that matched the block that lit up two back. This task was selected because it has face validity as a ‘‘computer game’’ similar to popular brain-based games available on the Internet and for
206
WEI & SUHR
portable gaming systems. However, research has shown that dual N-back tasks are correlated with other measures of ‘‘executive’’ working memory, and neuroimaging studies have shown that N-back tasks activate the dorsolateral and ventrolateral prefrontal cortex and frontal poles, areas of the brain associated with working memory and ADHD (Kane & Engle, 2003; Owen, McMillan, Laird, & Bullmore, 2005). Participants received standardized instructions on how to play the computer task and were given up to 100 practice trials to complete. Once they signaled that they were done with practice, they completed 30 trials of the actual task. The dependent variable for the present study was the percentage score on the 30 trials on the dual 2-back task (the percentage score combines the number of correct and incorrect trials across both visual and auditory modalities).
Procedure The study protocol was part of a larger study that was approved by the university’s institutional review board. Undergraduate students in the psychology experiment sign-up system were prescreened at the beginning of the quarter to determine eligibility for the study. The prescreen questionnaire included symptoms of ADHD from the DSM-IV-TR (American Psychiatric Association, 2000) and a Likert scale measuring concern about having ADHD. The students who met prescreen criteria of greater than the 50th percentile for the full sample of students screened on the ADHD screener and who reported high concern about ADHD (rating of 6 or greater) were invited to participate in the current study. Participants were informed that in exchange for their participation, they would receive extra credit in their psychology classes. Participants interested in the study were asked to attend one experimental session that lasted less than an hour. Participants were randomly assigned without replacement to one of two conditions: neutral control or diagnostic saliency. The experimental procedure for the two groups differed only during the experimental manipulation, at which point participants heard two separate sets of instructions. After informed consent procedures, all participants were administered a baseline depression measure. Following the depression measure, all participants underwent the experimental manipulation. Individuals in the neutral condition were instructed that the study examined undergraduate students’ perceptions of computer games: Next, we would like you to play a computer game. This computer game is sort of like those that have become popular on Facebook and on game systems like Nintendo that are more about strategy and thinking than about shooting something. We are interested in seeing
what college students think about this kind of game. We’d like you to give your best effort on the computer game and afterward we will ask you some questions about the game. First we need to fill out a questionnaire. Then we will show you how to play the computer game.
Participants in the diagnostic saliency condition heard instructions highlighting negative expectations about the relationship between ADHD symptoms and test performance and were told that the video game task was like a test used in assessments for ADHD: Next, we would like you to play a computer game. We are interested to see how college students perform on a computer game that measures attention and workingmemory skills, which are commonly assessed as part of evaluation for problems such as attention-deficit hyperactivity disorder. First we need to fill out a questionnaire. Then we will show you how to play the computer game.
Following the experimental manipulation, participants were administered the computerized dual 2-back task. Following the task, participants completed the posttest ADHD symptom measure, answered a demographics questionnaire, and rated their perceived performance on the task. Finally, participants were debriefed, provided information about ADHD, and received credit for their psychology course as compensation for their participation. RESULTS The initial sample consisted of 110 participants who were part of a larger study. For the present study, participants who reported an existing diagnosis of ADHD and=or reported they were on stimulant medication were removed from the sample (N ¼ 35). In addition, 3 outliers on the BDI-II were removed from the sample, which resulted in a final sample size of 72 (neutral condition ¼ 37; diagnostic saliency condition ¼ 35). Group differences on demographic and baseline measures are outlined in Table 1; groups were not different on any of the variables, with the exception of self-reported anxiety diagnoses, which were higher in the neutral condition. To test for the predicted interaction between prestudy ADHD illness belief and test condition, we conducted an analysis of covariance, with prescreen ASRS, experimental condition, and the interaction between ASRS and experimental condition as the independent variables and dual 2-back percentage score as the dependent variable. As predicted, the interaction term was significant, F(3, 69) ¼ 3.45, p ¼ .021. Follow-up tests were conducted by dividing the sample (mean split) into individuals who were high and low in prestudy ADHD symptom report and by comparing the two experimental conditions with t tests. In individuals whose prescreen ADHD self-report symptoms were lower, there was no difference between the two test conditions on dual 2-back performance
EXAMINATION OF THE ROLE OF EXPECTANCIES TABLE 1 Group Comparisons on Relevant Demographic and Study Variables
Age (mean, SD) Education (mean, SD) Beck Depression Inventory-II (mean, SD) Prescreen ASRS (mean, SD) Percent female Percent Caucasian Percent self-reported depression diagnosis Percent self-reported anxiety diagnosisa Percent self-reported mild head injury
Neutral Control (N ¼ 37)
Diagnostic Salience (N ¼ 35)
19.4, 2.7 12.5, 0.7 9.3, 5.1
19.1, 1.04 12.6, 0.9 9.4, 5.4
37.8, 5.7 54 84 5
40.2, 8.6 66 94 11
16
3
22
28
ASRS ¼ Adult ADHD Self-Report Scale. p < .05.
a
FIGURE 1 Illustration of the interaction between attention-deficit hyperactivity disorder symptom report and experimental condition on percent correct on the dual 2-back task.
(neutral X ¼ 53.3, SD ¼ 16.5; diagnostic saliency X ¼ 49.9, SD ¼ 20.1), t(34) ¼ 0.55, p ¼ .59, but in individuals whose prescreen ADHD self-report symptoms were higher, the diagnostic saliency group performed worse on the dual 2-back (X ¼ 41.3, SD ¼ 15.4) compared with the neutral group (X ¼ 54.4, SD ¼ 20.6), t(34) ¼ 2.17, p ¼ .037. See Figure 1.
DISCUSSION Consistent with predictions, college students with higher self-report of prestudy ADHD symptoms who were cued with a belief about the diagnostic saliency of a complex working-memory and attention task performed worse on the task compared with students who received neutral
207
instructions. We believe that calling attention to their concerns about ‘‘ADHD-related symptoms’’ resulted in expectations for performance on salient tasks, which manifested as actual performance deficits. These results are consistent with previous findings that expectations can influence performance on self-report measures and neuropsychological tests (Suhr & Wei, 2013, for a review; Suhr & Wei, in press). For example, previous studies have shown that college students who were cued to their mild head injury history performed worse on cognitive tasks compared with students who received neutral instructions (Suhr & Gunstad, 2002, 2005). Findings from the current study highlight the importance of identifying non-neurological factors that can influence neuropsychological performance and=or self-report of ADHD symptoms to help improve the accuracy of interpreting diagnostic assessments for ADHD. The current study was limited by reliance on a small sample size (N ¼ 72) and a sample of predominantly Caucasian college students. It is unclear how generalizable the results are to minority students, individuals who were denied college admission, or students who left college due to poor academic performance. Further, our sample consisted of college students who expressed concern about having ADHD symptoms but who were not concerned enough to present for an evaluation or treatment. As a future direction, we suggest cueing a belief about diagnosis saliency among a more severe or clinical sample of individuals presenting for evaluation. We believe this will increase external validity and help us to better understand the effect size of this phenomenon in individuals actually presenting to clinical settings for treatment or assessment for ADHD concerns. Finally, although the current study identified a significant impact of expectations on a complex working-memory and attention task, it is unclear how generalizable these findings are to other neuropsychological tests and constructs. Therefore, as a future direction, we suggest examining this interesting phenomenon with alternative neuropsychological tests and constructs. Despite the limitations, results from the current study have implications for researchers and clinicians who continue to rely exclusively on current self-reported symptoms and neuropsychological test performance to diagnose ADHD. Instead of relying solely on self-report measures and performance on neuropsychological tests, we encourage health care providers to collect collateral data, examine academic and other records for evidence of developmental history of symptoms and impairment, carefully consider evidence for current functional impairment, and assess for differential and=or comorbid diagnoses. Recognizing that simply cueing a belief about the purpose of a task in a ‘‘vulnerable’’ group can impact actual performance on a complex working-memory and attention task is the first step in developing ‘‘best practices’’ for providing appropriate clinical services.
208
WEI & SUHR
REFERENCES American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck Depression Inventory (2nd ed.). San Antonio, TX: The Psychological Corporation. Booksh, R. L., Pella, R. D., Singh, A. N., & Gouvier, W. D. (2009). Ability of college students to stimulate ADHD on objective measures of attention. Journal of Attention Disorders, 13, 325–338. doi:10.1177=1087054708329927 Conrad, P., & Potter, D. (2000). From hyperactive children to ADHD adults: Observations on the expansion of medical categories. Social Problems, 47, 559–582. doi:10.1525=sp.2000.47.4.03 0308v Conti, R. P. (2004). Malingered ADHD in adolescents diagnosed with conduct disorder: A brief note. Psychological Reports, 94, 987–988. doi:10.2466=PR0.94.3.987-988 Davidson, M. A. (2008). ADHD in adults: A review of the literature. Journal of Attention Disorders, 11, 628–641. doi:10.1177= 1087054707310878 DuPaul, G. J., Schaughency, E. A., Weyandt, L. L, Tripp, G., Kiesner, J., Ota, K., & Stanish, H. (2001). Self-report of ADHD symptoms in university students: Cross-gender and cross-national prevalence. Journal of Learning Disabilities, 34, 370–379. doi:10.1177=002221940103400412 Faraone, S. V., & Biederman, J. (2005). What is the prevalence of adult ADHD? Results of a population screen of 966 adults. Journal of Attention Disorders, 9, 384–391. doi:10.1177=1087054705281478 Hahn, R. A. (1999). Expectations of sickness: Concept and evidence of the nocebo phenomenon. In I. Kirsch (Ed.), How expectancies shape experience (pp. 333–356). Washington, DC: American Psychological Association. doi:10.1037=10332-014 Harrison, A. G. (2004). An investigation of reported symptoms of ADHD in a university population. ADHD Report, 12, 8–11. doi:10.1521=adhd.12.6.8.55256 Harrison, A. G., Edwards, M. J., & Parker, K. C. H. (2007). Identifying students faking ADHD: Preliminary findings and strategies for detection. Archives of Clinical Neuropsychology, 22, 577–588. doi:10.1016=j.acn.2007.03.008 Jachimowicz, G., & Geiselman, R. E. (2004). Comparison of ease of falsification of attention deficit hyperactivity disorder diagnosis using standard behavioral rating scales. Cognitive Science Online, 2, 6–20. Retrieved from http://cogsci-online.ucsd.edu Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132, 47–70. doi:10.1037=00963445.132.1.47 Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi, E., . . . Walter, E. E. (2005). The World Health Organization adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35, 245–256. doi:10.1017=S0033291704002892 Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55, 352–358. doi:10.1037=h0043688 Lewandowski, L. J., Lovett, B. J., Codding, R. S., & Gordon, M. (2008). Symptoms of ADHD and academic concerns in college students with and without ADHD diagnoses. Journal of Attention Disorders, 12, 156–161. doi:10.1177=1087054707310882 Mannuzza, S., Klein, R. G., Klein, D. F., Bessler, A., & Shrout, P. (2002). Accuracy of adult recall of childhood attention deficit hyperactivity disorder. American Journal of Psychiatry, 159, 1882–1888. doi:10.1176=appi.ajp.159.11.1882
McCann, B. S., & Roy-Byrne, P. (2004). Screening and diagnostic utility of self-report attention deficit hyperactivity disorder scales in adults. Comprehensive Psychiatry, 45, 175–183. doi:10.1016= j.comppsych.2004.02.006 Murphy, K., & Barkley, R. A. (1996). Prevalence of DSM-IV symptoms of ADHD in adult licensed drivers: Implications for clinical diagnosis. Journal of Attention Disorders, 1, 147–161. doi:10.1177=108705479600100303 Nigg, J. T. (2005). Neuropsychologic theory and findings in attention-deficit=hyperactivity disorder: The state of the field and salient challenges for the coming decade. Biological Psychiatry, 57, 1424–1435. doi:10.1016=j.biopsych.2004.11.011 Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25, 46–69. doi:10.1002=hbm.20131 Quinn, C. A. (2003). Detection of malingering in assessment of adult ADHD. Archives of Clinical Neuropsychology, 18, 379–395. doi:10.1016=S0887-6177(02)00150-6 Schoechlin, C., & Engel, R. R. (2005). Neuropsychological performance in adult attention-deficit hyperactivity disorder: Metaanalysis of empirical data. Archives of Clinical Neuropsychology, 20, 727–744. doi:10.1016=j.acn.2005.04.005 Smith, S., & Sullivan, K. (2003). Examining the influence of biological and psychological factors on cognitive performance in chronic fatigue syndrome: A randomized, double-blind, placebo-controlled, crossover study. International Journal of Behavioral Medicine, 10, 162–173. doi:10.1207=S15327558IJBM1002_05 Stefanatos, G. A., & Baron, I. S. (2007). Attention-deficit= hyperactivity disorder: A neuropsychological perspective towards DSM-V. Neuropsychology Review, 17, 5–38. doi:10.1007= s11065-007-9020-3 Suhr, J. A., Buelow, M., & Riddle, T. (2011). Development of an infrequency index for the CAARS. Journal of Psychoeducational Assessment, 29, 160–170. doi:10.1177=0734282910380190 Suhr, J. A., & Gunstad, J. (2002). ‘Diagnosis threat’: The effect of negative expectations on cognitive performance in head injury. Journal of Clinical and Experimental Neuropsychology, 24, 448– 457. doi:10.1076=jcen.24.4.448.1039 Suhr, J. A., & Gunstad, J. (2005). Further exploration of the effect of ‘diagnosis threat’ on cognitive performance in individuals with mild head injury. Journal of the International Neuropsychological Society, 11, 23–29. doi:10.1017=S1355617705050010 Suhr, J., Hammers, D., Dobbins-Buckland, K., Zimak, E., & Hughes, C. (2008). The relationship of malingering test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation. Archives of Clinical Neuropsychology, 23, 521–530. doi:10.1016=j.acn.2008.05.003 Suhr, J. A., & Wei, C. (2013). Response expectancies and their potential influence in neuropsychological evaluation. In P. Arnett (Ed.), Secondary influences on neuropsychological test performance (pp. 182–200). New York, NY: Oxford University Press. Suhr, J. A., & Wei, C. (in press). Attention deficit=hyperactivity disorder as an illness identity: Implications for neuropsychology practice. In K. Boone (Ed.), Neuropsychological evaluation of medically unexplained symptoms: Assessment primer. New York, NY: Oxford University Press. Suhr, J., Zimak, E., Buelow, M., & Fox, L. (2009). Self-reported childhood attention-deficit=hyperactivity disorder symptoms are not specific to the disorder. Comprehensive Psychiatry, 50, 269–275. doi:10.1016=j.comppsych.2008.08.008 Sullivan, B. K., May, K., & Galbally, L. (2007). Symptom exaggeration by college adults in attention-deficit hyperactivity disorder and learning disorder assessments. Applied Neuropsychology, 14, 189–207. doi:10.1080=09084280701509083
Copyright of Applied Neuropsychology: Adult is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Examination of the Role of Expectancies on Task Performance in College Students Concerned about ADHD Christina Wei and Julie A. Suhr Department of Psychology, Ohio University, Athens, Ohio
Prior research has shown that performance on cognitive tasks can be influenced by expectations (Smith & Sullivan, 2003; Suhr & Gunstad, 2002, 2005). The current study examined whether cuing a belief about the diagnostic saliency of a cognitive task among young adults who expressed concern about having attention-deficit hyperactivity disorder (ADHD) influenced task performance. Participants were randomly assigned to either receive neutral directions or be cued to a belief that the task had diagnostic saliency prior to completing a working-memory task. Supporting our hypothesis, college students with higher prestudy report of ADHD symptoms who were cued with a belief about the diagnostic saliency of the task performed worse compared with students who received neutral instructions. As many researchers and clinicians currently rely exclusively on self-reported symptoms and neuropsychological tests to diagnose ADHD, our findings highlight the importance of comprehensive assessment for provision of appropriate clinical services to adults presenting with ADHD concerns.
Key words:
assessment=diagnosis, cognition, neuropsychology
College students frequently refer themselves for attention-deficit hyperactivity disorder (ADHD) evaluation. Unfortunately, diagnosis of ADHD for the first time in relatively healthy young adults is a difficult task, and often self-report becomes the relied-upon factor for diagnosis (Stefanatos & Baron, 2007). This is of concern because individuals diagnosed with ADHD qualify for academic accommodations and prescriptions for psychostimulant medications. Given evidence that those medications are often misused and even abused and resold, accuracy of diagnostic methods is paramount to provision of appropriate clinical services (Davidson, 2008; Sullivan, May & Galbally, 2007). There are several problems with reliance on self-report in diagnosis of ADHD in adults. First, self-report ADHD measures are not specific to the disorder; individuals with other psychological conditions endorse ADHD symptoms at a high rate (Harrison, 2004; McCann & Address correspondence to Christina Wei, Ph.D., Department of Psychology, Ohio University, 200 Porter Hall, Athens, OH 45701. E-mail: [email protected]
Roy-Byrne, 2004; Schoechlin & Engel, 2005). In addition, self-report ADHD measures have been shown to be vulnerable to malingering (Booksh, Pella, Singh, & Gouvier, 2009; Conti, 2004; Jachimowicz & Geiselman, 2004; Quinn, 2003; Suhr, Hammers, Dobbins-Buckland, Zimak, & Hughes, 2008; Sullivan et al., 2007). Further, healthy individuals may misattribute subclinical behaviors as both pathological and symptomatic of ADHD, leading to overendorsement of symptoms (Suhr & Wei, 2013, in press). Because base rates of subclinical ‘‘ADHD’’ symptoms are high in the general population, including college students (DuPaul et al., 2001; Faraone & Biederman, 2005; Lewandowski, Lovett, Codding, & Gordon, 2008; Mannuzza, Klein, Klein, Bessler, & Shrout, 2002; Murphy & Barkley, 1996; Suhr et al., 2008; Suhr, Zimak, Buelow, & Fox, 2009), individuals exposed to information about symptoms and causes of ADHD may misattribute everyday problems with attention or focus as symptoms of ADHD (Conrad & Potter, 2000; Suhr & Wei, 2013, in press). One way in which this might occur is through the influence of societal beliefs about what illnesses to expect and what ‘‘symptoms’’
EXAMINATION OF THE ROLE OF EXPECTANCIES
are associated with illnesses (Hahn, 1999). In some cases, individuals may strongly identify with having ADHD and hold beliefs about the types of symptoms and impairments that result from ADHD (Suhr & Wei, 2013, in press). Development of this type of ‘‘illness identity’’ may lead to misattribution of common everyday errors as pathological ‘‘symptoms’’ of the disorder, as well as behaving in ways consistent with the disorder, as might be seen on neuropsychological tests. Similar to the pattern seen on ADHD self-report measures, poor performance on neuropsychological measures of constructs such as sustained attention, working memory, and inhibition are not specific to ADHD (Nigg, 2009). In addition, performance on neuropsychological measures can be easily malingered (Booksh et al., 2009; Harrison, Edwards, & Parker, 2007; Suhr, Buelow, & Riddle, 2011; Suhr et al., 2008; Sullivan et al., 2007). Furthermore, behavioral performance on neuropsychological measures is vulnerable to negative expectations that can arise due to belief that one has a disorder. Expectations have been found to have small-to-moderate effects on performance on neuropsychological tests in individuals diagnosed with chronic fatigue syndrome, mild head injuries, and recurrent headaches, as well as in pregnant women and substance abusers (see Smith & Sullivan, 2003 and Suhr & Wei, 2013, for a review). When situationally cued, an individual who identifies with an illness and has negative beliefs about the course and consequences of the illness can behave in ways consistent with expectations. More specifically, an individual may expect to do poorly on an attention task based on their ADHD illness identity. Reminding the individual of their ADHD illness identity, coupled with emphasis on the diagnostic saliency of the task at hand, may trigger expectations of poor performance, which can result in performance deficits. The current study examined the impact of diagnostic saliency on working-memory performance in college students who expressed concern that they may have ADHD. We hypothesized that among college students with a higher level of ADHD illness identity (defined by higher levels of prestudy ADHD symptoms), those cued to a belief that a task had diagnostic saliency would perform worse on a complex working-memory and attention task compared with those who were not cued to the diagnostic saliency of the task. Thus, we expected an interaction between prestudy ADHD identity and exposure to diagnosis cue. METHODS Participants Participants were 72 undergraduates aged 18 to 32 years old attending a Midwestern university. Most participants were freshmen (57%), female (60%), and Caucasian
205
(89%). Eight percent of participants reported having a diagnosis of depression, and 10% reported having a diagnosis of an anxiety disorder. With regard to report of any history of head injury, 25% reported a history of mild head injury. Seventy-eight percent of participants reported alcohol use, 22% reported marijuana use, and 14% reported nicotine use. Measures Adult ADHD Self-Report Scale (Kessler et al., 2005). The Adult ADHD Self-Report Scale (ASRS) measures the self-report of Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition (DSM-IV) symptoms of ADHD in adults. The ASRS has good sensitivity (56.3%), specificity (98.3%), and total classification accuracy (96.2%) in comparison with healthy controls (Kessler et al., 2005). In the present study, the ASRS score from the prescreening session was used for participant selection (as outlined) and was also used as a covariate relevant to the study hypotheses (see Results section). The ASRS demonstrated good internal consistency reliability for the current study (.82). Concern about ADHD. Prior to participating in the study, as part of the prescreening, students were asked to rate themselves on the following item: ‘‘How concerned are you that you may have symptoms of ADHD?’’ This question was rated on a Likert scale ranging from 1 (not concerned at all) to 10 (very concerned). This measure was used for participant selection. Beck Depression Inventory-II (Beck, Steer, & Brown, 1996). The Beck Depression Inventory-II (BDI-II) was used to assess depressive symptoms. The internal consistency for college students has been found to be strong (a ¼ .93; Beck et al., 1996), and in the present study, it was .87. The BDI-II has generally been found to have adequate sensitivity (71%) and specificity (88%) in the diagnosis of depression (Beck et al., 1996). The BDI-II score was used in the present study to remove individuals who endorsed unusually high amounts of depressive symptoms. Dual 2-back (Kirchner, 1958). This computerized version of a classic working-memory and attention task required participants to press a designated computer key when they heard a spoken letter that matched the letter they heard two back, while simultaneously pressing a different computer key when they saw a block light up within a nine-block span that matched the block that lit up two back. This task was selected because it has face validity as a ‘‘computer game’’ similar to popular brain-based games available on the Internet and for
206
WEI & SUHR
portable gaming systems. However, research has shown that dual N-back tasks are correlated with other measures of ‘‘executive’’ working memory, and neuroimaging studies have shown that N-back tasks activate the dorsolateral and ventrolateral prefrontal cortex and frontal poles, areas of the brain associated with working memory and ADHD (Kane & Engle, 2003; Owen, McMillan, Laird, & Bullmore, 2005). Participants received standardized instructions on how to play the computer task and were given up to 100 practice trials to complete. Once they signaled that they were done with practice, they completed 30 trials of the actual task. The dependent variable for the present study was the percentage score on the 30 trials on the dual 2-back task (the percentage score combines the number of correct and incorrect trials across both visual and auditory modalities).
Procedure The study protocol was part of a larger study that was approved by the university’s institutional review board. Undergraduate students in the psychology experiment sign-up system were prescreened at the beginning of the quarter to determine eligibility for the study. The prescreen questionnaire included symptoms of ADHD from the DSM-IV-TR (American Psychiatric Association, 2000) and a Likert scale measuring concern about having ADHD. The students who met prescreen criteria of greater than the 50th percentile for the full sample of students screened on the ADHD screener and who reported high concern about ADHD (rating of 6 or greater) were invited to participate in the current study. Participants were informed that in exchange for their participation, they would receive extra credit in their psychology classes. Participants interested in the study were asked to attend one experimental session that lasted less than an hour. Participants were randomly assigned without replacement to one of two conditions: neutral control or diagnostic saliency. The experimental procedure for the two groups differed only during the experimental manipulation, at which point participants heard two separate sets of instructions. After informed consent procedures, all participants were administered a baseline depression measure. Following the depression measure, all participants underwent the experimental manipulation. Individuals in the neutral condition were instructed that the study examined undergraduate students’ perceptions of computer games: Next, we would like you to play a computer game. This computer game is sort of like those that have become popular on Facebook and on game systems like Nintendo that are more about strategy and thinking than about shooting something. We are interested in seeing
what college students think about this kind of game. We’d like you to give your best effort on the computer game and afterward we will ask you some questions about the game. First we need to fill out a questionnaire. Then we will show you how to play the computer game.
Participants in the diagnostic saliency condition heard instructions highlighting negative expectations about the relationship between ADHD symptoms and test performance and were told that the video game task was like a test used in assessments for ADHD: Next, we would like you to play a computer game. We are interested to see how college students perform on a computer game that measures attention and workingmemory skills, which are commonly assessed as part of evaluation for problems such as attention-deficit hyperactivity disorder. First we need to fill out a questionnaire. Then we will show you how to play the computer game.
Following the experimental manipulation, participants were administered the computerized dual 2-back task. Following the task, participants completed the posttest ADHD symptom measure, answered a demographics questionnaire, and rated their perceived performance on the task. Finally, participants were debriefed, provided information about ADHD, and received credit for their psychology course as compensation for their participation. RESULTS The initial sample consisted of 110 participants who were part of a larger study. For the present study, participants who reported an existing diagnosis of ADHD and=or reported they were on stimulant medication were removed from the sample (N ¼ 35). In addition, 3 outliers on the BDI-II were removed from the sample, which resulted in a final sample size of 72 (neutral condition ¼ 37; diagnostic saliency condition ¼ 35). Group differences on demographic and baseline measures are outlined in Table 1; groups were not different on any of the variables, with the exception of self-reported anxiety diagnoses, which were higher in the neutral condition. To test for the predicted interaction between prestudy ADHD illness belief and test condition, we conducted an analysis of covariance, with prescreen ASRS, experimental condition, and the interaction between ASRS and experimental condition as the independent variables and dual 2-back percentage score as the dependent variable. As predicted, the interaction term was significant, F(3, 69) ¼ 3.45, p ¼ .021. Follow-up tests were conducted by dividing the sample (mean split) into individuals who were high and low in prestudy ADHD symptom report and by comparing the two experimental conditions with t tests. In individuals whose prescreen ADHD self-report symptoms were lower, there was no difference between the two test conditions on dual 2-back performance
EXAMINATION OF THE ROLE OF EXPECTANCIES TABLE 1 Group Comparisons on Relevant Demographic and Study Variables
Age (mean, SD) Education (mean, SD) Beck Depression Inventory-II (mean, SD) Prescreen ASRS (mean, SD) Percent female Percent Caucasian Percent self-reported depression diagnosis Percent self-reported anxiety diagnosisa Percent self-reported mild head injury
Neutral Control (N ¼ 37)
Diagnostic Salience (N ¼ 35)
19.4, 2.7 12.5, 0.7 9.3, 5.1
19.1, 1.04 12.6, 0.9 9.4, 5.4
37.8, 5.7 54 84 5
40.2, 8.6 66 94 11
16
3
22
28
ASRS ¼ Adult ADHD Self-Report Scale. p < .05.
a
FIGURE 1 Illustration of the interaction between attention-deficit hyperactivity disorder symptom report and experimental condition on percent correct on the dual 2-back task.
(neutral X ¼ 53.3, SD ¼ 16.5; diagnostic saliency X ¼ 49.9, SD ¼ 20.1), t(34) ¼ 0.55, p ¼ .59, but in individuals whose prescreen ADHD self-report symptoms were higher, the diagnostic saliency group performed worse on the dual 2-back (X ¼ 41.3, SD ¼ 15.4) compared with the neutral group (X ¼ 54.4, SD ¼ 20.6), t(34) ¼ 2.17, p ¼ .037. See Figure 1.
DISCUSSION Consistent with predictions, college students with higher self-report of prestudy ADHD symptoms who were cued with a belief about the diagnostic saliency of a complex working-memory and attention task performed worse on the task compared with students who received neutral
207
instructions. We believe that calling attention to their concerns about ‘‘ADHD-related symptoms’’ resulted in expectations for performance on salient tasks, which manifested as actual performance deficits. These results are consistent with previous findings that expectations can influence performance on self-report measures and neuropsychological tests (Suhr & Wei, 2013, for a review; Suhr & Wei, in press). For example, previous studies have shown that college students who were cued to their mild head injury history performed worse on cognitive tasks compared with students who received neutral instructions (Suhr & Gunstad, 2002, 2005). Findings from the current study highlight the importance of identifying non-neurological factors that can influence neuropsychological performance and=or self-report of ADHD symptoms to help improve the accuracy of interpreting diagnostic assessments for ADHD. The current study was limited by reliance on a small sample size (N ¼ 72) and a sample of predominantly Caucasian college students. It is unclear how generalizable the results are to minority students, individuals who were denied college admission, or students who left college due to poor academic performance. Further, our sample consisted of college students who expressed concern about having ADHD symptoms but who were not concerned enough to present for an evaluation or treatment. As a future direction, we suggest cueing a belief about diagnosis saliency among a more severe or clinical sample of individuals presenting for evaluation. We believe this will increase external validity and help us to better understand the effect size of this phenomenon in individuals actually presenting to clinical settings for treatment or assessment for ADHD concerns. Finally, although the current study identified a significant impact of expectations on a complex working-memory and attention task, it is unclear how generalizable these findings are to other neuropsychological tests and constructs. Therefore, as a future direction, we suggest examining this interesting phenomenon with alternative neuropsychological tests and constructs. Despite the limitations, results from the current study have implications for researchers and clinicians who continue to rely exclusively on current self-reported symptoms and neuropsychological test performance to diagnose ADHD. Instead of relying solely on self-report measures and performance on neuropsychological tests, we encourage health care providers to collect collateral data, examine academic and other records for evidence of developmental history of symptoms and impairment, carefully consider evidence for current functional impairment, and assess for differential and=or comorbid diagnoses. Recognizing that simply cueing a belief about the purpose of a task in a ‘‘vulnerable’’ group can impact actual performance on a complex working-memory and attention task is the first step in developing ‘‘best practices’’ for providing appropriate clinical services.
208
WEI & SUHR
REFERENCES American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck Depression Inventory (2nd ed.). San Antonio, TX: The Psychological Corporation. Booksh, R. L., Pella, R. D., Singh, A. N., & Gouvier, W. D. (2009). Ability of college students to stimulate ADHD on objective measures of attention. Journal of Attention Disorders, 13, 325–338. doi:10.1177=1087054708329927 Conrad, P., & Potter, D. (2000). From hyperactive children to ADHD adults: Observations on the expansion of medical categories. Social Problems, 47, 559–582. doi:10.1525=sp.2000.47.4.03 0308v Conti, R. P. (2004). Malingered ADHD in adolescents diagnosed with conduct disorder: A brief note. Psychological Reports, 94, 987–988. doi:10.2466=PR0.94.3.987-988 Davidson, M. A. (2008). ADHD in adults: A review of the literature. Journal of Attention Disorders, 11, 628–641. doi:10.1177= 1087054707310878 DuPaul, G. J., Schaughency, E. A., Weyandt, L. L, Tripp, G., Kiesner, J., Ota, K., & Stanish, H. (2001). Self-report of ADHD symptoms in university students: Cross-gender and cross-national prevalence. Journal of Learning Disabilities, 34, 370–379. doi:10.1177=002221940103400412 Faraone, S. V., & Biederman, J. (2005). What is the prevalence of adult ADHD? Results of a population screen of 966 adults. Journal of Attention Disorders, 9, 384–391. doi:10.1177=1087054705281478 Hahn, R. A. (1999). Expectations of sickness: Concept and evidence of the nocebo phenomenon. In I. Kirsch (Ed.), How expectancies shape experience (pp. 333–356). Washington, DC: American Psychological Association. doi:10.1037=10332-014 Harrison, A. G. (2004). An investigation of reported symptoms of ADHD in a university population. ADHD Report, 12, 8–11. doi:10.1521=adhd.12.6.8.55256 Harrison, A. G., Edwards, M. J., & Parker, K. C. H. (2007). Identifying students faking ADHD: Preliminary findings and strategies for detection. Archives of Clinical Neuropsychology, 22, 577–588. doi:10.1016=j.acn.2007.03.008 Jachimowicz, G., & Geiselman, R. E. (2004). Comparison of ease of falsification of attention deficit hyperactivity disorder diagnosis using standard behavioral rating scales. Cognitive Science Online, 2, 6–20. Retrieved from http://cogsci-online.ucsd.edu Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: The contributions of goal neglect, response competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132, 47–70. doi:10.1037=00963445.132.1.47 Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi, E., . . . Walter, E. E. (2005). The World Health Organization adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35, 245–256. doi:10.1017=S0033291704002892 Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55, 352–358. doi:10.1037=h0043688 Lewandowski, L. J., Lovett, B. J., Codding, R. S., & Gordon, M. (2008). Symptoms of ADHD and academic concerns in college students with and without ADHD diagnoses. Journal of Attention Disorders, 12, 156–161. doi:10.1177=1087054707310882 Mannuzza, S., Klein, R. G., Klein, D. F., Bessler, A., & Shrout, P. (2002). Accuracy of adult recall of childhood attention deficit hyperactivity disorder. American Journal of Psychiatry, 159, 1882–1888. doi:10.1176=appi.ajp.159.11.1882
McCann, B. S., & Roy-Byrne, P. (2004). Screening and diagnostic utility of self-report attention deficit hyperactivity disorder scales in adults. Comprehensive Psychiatry, 45, 175–183. doi:10.1016= j.comppsych.2004.02.006 Murphy, K., & Barkley, R. A. (1996). Prevalence of DSM-IV symptoms of ADHD in adult licensed drivers: Implications for clinical diagnosis. Journal of Attention Disorders, 1, 147–161. doi:10.1177=108705479600100303 Nigg, J. T. (2005). Neuropsychologic theory and findings in attention-deficit=hyperactivity disorder: The state of the field and salient challenges for the coming decade. Biological Psychiatry, 57, 1424–1435. doi:10.1016=j.biopsych.2004.11.011 Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25, 46–69. doi:10.1002=hbm.20131 Quinn, C. A. (2003). Detection of malingering in assessment of adult ADHD. Archives of Clinical Neuropsychology, 18, 379–395. doi:10.1016=S0887-6177(02)00150-6 Schoechlin, C., & Engel, R. R. (2005). Neuropsychological performance in adult attention-deficit hyperactivity disorder: Metaanalysis of empirical data. Archives of Clinical Neuropsychology, 20, 727–744. doi:10.1016=j.acn.2005.04.005 Smith, S., & Sullivan, K. (2003). Examining the influence of biological and psychological factors on cognitive performance in chronic fatigue syndrome: A randomized, double-blind, placebo-controlled, crossover study. International Journal of Behavioral Medicine, 10, 162–173. doi:10.1207=S15327558IJBM1002_05 Stefanatos, G. A., & Baron, I. S. (2007). Attention-deficit= hyperactivity disorder: A neuropsychological perspective towards DSM-V. Neuropsychology Review, 17, 5–38. doi:10.1007= s11065-007-9020-3 Suhr, J. A., Buelow, M., & Riddle, T. (2011). Development of an infrequency index for the CAARS. Journal of Psychoeducational Assessment, 29, 160–170. doi:10.1177=0734282910380190 Suhr, J. A., & Gunstad, J. (2002). ‘Diagnosis threat’: The effect of negative expectations on cognitive performance in head injury. Journal of Clinical and Experimental Neuropsychology, 24, 448– 457. doi:10.1076=jcen.24.4.448.1039 Suhr, J. A., & Gunstad, J. (2005). Further exploration of the effect of ‘diagnosis threat’ on cognitive performance in individuals with mild head injury. Journal of the International Neuropsychological Society, 11, 23–29. doi:10.1017=S1355617705050010 Suhr, J., Hammers, D., Dobbins-Buckland, K., Zimak, E., & Hughes, C. (2008). The relationship of malingering test failure to self-reported symptoms and neuropsychological findings in adults referred for ADHD evaluation. Archives of Clinical Neuropsychology, 23, 521–530. doi:10.1016=j.acn.2008.05.003 Suhr, J. A., & Wei, C. (2013). Response expectancies and their potential influence in neuropsychological evaluation. In P. Arnett (Ed.), Secondary influences on neuropsychological test performance (pp. 182–200). New York, NY: Oxford University Press. Suhr, J. A., & Wei, C. (in press). Attention deficit=hyperactivity disorder as an illness identity: Implications for neuropsychology practice. In K. Boone (Ed.), Neuropsychological evaluation of medically unexplained symptoms: Assessment primer. New York, NY: Oxford University Press. Suhr, J., Zimak, E., Buelow, M., & Fox, L. (2009). Self-reported childhood attention-deficit=hyperactivity disorder symptoms are not specific to the disorder. Comprehensive Psychiatry, 50, 269–275. doi:10.1016=j.comppsych.2008.08.008 Sullivan, B. K., May, K., & Galbally, L. (2007). Symptom exaggeration by college adults in attention-deficit hyperactivity disorder and learning disorder assessments. Applied Neuropsychology, 14, 189–207. doi:10.1080=09084280701509083
Copyright of Applied Neuropsychology: Adult is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
