Eating Behaviors 15 (2014) 175–181

Contents lists available at ScienceDirect

Eating Behaviors

Validity and reliability of the Behavior Rating Inventory of Executive Function — Adult Version in a clinical sample with eating disorders Stefanie Ciszewski a, Kylie Francis b, Paul Mendella b, Hany Bissada b,c, Giorgio A. Tasca b,c,⁎ a b c

University of New Brunswick, 100 Tucker Park Road, PO Box 5050 Saint John, NB E2L 4L5, Canada The Ottawa Hospital, 501 Smyth Road, Ottawa, ON K1G 0H9, Canada University of Ottawa, 65 Laurier Avenue East, Ottawa, ON K1N 6N5, Canada

a r t i c l e

i n f o

Article history: Received 18 June 2013 Received in revised form 29 November 2013 Accepted 22 January 2014 Available online 31 January 2014 Keywords: BRIEF-A Validity Reliability Eating disorders Executive function

a b s t r a c t This study is a preliminary investigation of the reliability and validity of the Behavior Rating Inventory of Executive Function — Adult Version (BRIEF-A) in a clinical sample of patients with eating disorders (ED). Participants were 252 adult females who were referred to a centre for the treatment of EDs, as well as 31 individuals who completed the informant version of the BRIEF-A. Patients completed the BRIEF-A and other psychological measures on one occasion during their initial clinic visit, and informants nominated by patients completed the informant version at home. Reliability analyses revealed high internal consistency (Cronbach's alpha) of the two indices (Metacognition Index and Behavioral Regulation Index), and for the Global Executive Composite (GEC) of the BRIEF-A (α = .96). Convergent validity was shown by a high positive relationship between the selfreport and informant-report versions of the BRIEF-A, and between the GEC and the Anxiety and Depression scales. Construct validity was assessed by an exploratory and confirmatory factor analysis. The BRIEF-A may be a reliable and valid tool for measuring executive functioning (EF) in an ED population, and may serve as an initial screening tool of EF for clinicians and researchers. © 2014 Published by Elsevier Ltd.

1. Introduction Recent research has begun to investigate neuropsychological functioning in patients with eating disorders. Beyond eating disorder symptoms (i.e., binge-eating, food restriction, and purging), patients with eating disorders may demonstrate cognitive styles characterized by poor set shifting (i.e., difficulty switching between thinking about two different concepts), poor decision making, and a weak central coherence (i.e., a limited ability to understand context) (Danner et al., 2012b). Contemporary models of eating disorders (i.e., Schmidt & Treasure, 2006) suggest the role of cognitive deficits in the maintenance of these disorders (Harrison, Tchanturia, Naumann, & Treasure, 2012). One broad neuropsychological concept characterizing cognitive deficits is executive functioning (EF), which encompasses higher-order cognitive functions that monitor and conduct other cognitive processes (Luria, 1966), and allow for goal-oriented behaviour (Morgan & Lilienfeld, 2000). EF is involved in the “selection, initiation, and monitoring of cognition, emotion and behaviour, as well as other aspects of other motor and sensory functioning” (Roth, Isquith, & Gioia, 2005, p. 1). Deficits in decision-making, set shifting, and central coherence, ⁎ Corresponding author at: Psychology, The Ottawa Hospital, 501 Smyth Road, Ottawa, ON K1H8L6, Canada. Tel.: +1 613 737 8035; fax: +1 613 737 8085. E-mail addresses: [email protected] (S. Ciszewski), [email protected] (K. Francis), [email protected] (P. Mendella), [email protected] (H. Bissada), [email protected] (G.A. Tasca). 1471-0153/$ – see front matter © 2014 Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.eatbeh.2014.01.004

which are noted in patients with eating disorders, can be understood within a framework of EF. EF is often assessed using performance-based tasks, such as the Iowa Gambling Task to measure decision-making, or the Wisconsin Card Sorting Task to measure cognitive flexibility and perseverance (Tchanturia et al., 2012). However, the administration of these performance-based procedures is time consuming and labour intensive. Given the recent interest in EF in eating disorder populations, a short and easy-to-administer self-report questionnaire which measures EF would be useful as an initial screening tool for eating disorder treatment and research. The Behavior Rating Inventory of Executive Function — Adult Version (BRIEF-A; Roth et al., 2005) is a self-report questionnaire that was developed to measure an adult's perception of his or her own EF in real life activities. To our knowledge, the BRIEF-A has not been assessed for psychometric reliability or validity in an eating disorder population. The purpose of the current study is to perform a preliminary examination of the psychometric properties of the BRIEF-A in a clinical population of patients with an eating disorder seeking treatment.

1.1. Executive functioning in eating disorders Decision-making ability may be impaired in patients with Anorexia Nervosa (AN; Cavedini et al., 2004; Tenconi et al., 2010), when measured by the Iowa Gambling Task (Danner et al., 2012b; Tchanturia et al., 2007). Poor decision-making abilities are reflected in the eating

176

S. Ciszewski et al. / Eating Behaviors 15 (2014) 175–181

behaviours exhibited by AN patients in which the avoidance of caloric intake results in the immediate reward of anxiety relief, while ignoring the long term physical consequences (Cavedini et al., 2004). Impaired decision-making has also been demonstrated in patients with Bulimia Nervosa (BN; Liao et al., 2009) and patients with Binge Eating Disorder (BED; Danner, Ouwehand, Haastert, Hornsveld, & de Ridder, 2012a; Svaldi, Brand, & Tuschen-Caffier, 2010). However, the research remains inconclusive as other researchers did not find impaired decisionmaking ability in women with AN and BN (Guillaume et al., 2010). Investigations of set shifting – the ability to shift back and forth between tasks and mental sets (Miyake et al., 2000) – suggest that patients with eating disorders have poorer set shifting abilities than healthy controls (Danner et al., 2012b; Roberts, Tchanturia, & Treasure, 2010; Tchanturia et al., 2004; Tchanturia et al., 2012). However, this relationship is demonstrated less consistently for patients with BN (Tchanturia et al., 2011). In patients with AN, difficulties with set shifting appear to persist post-treatment (Tchanturia et al., 2004), and were identified as a risk and maintenance factor of the disorder (Roberts et al., 2010). Further, poor set shifting ability was associated with heightened anxiety and depression, a longer duration of illness, lower self-esteem, and more severe eating disorder behaviours (Roberts et al., 2010). Danner et al. (2012b) suggest that rigid thinking, characterized by poor set shifting and a weak central coherence, may be a characteristic of some patients with AN. Individuals with AN may have a weak central coherence, which is characterized by attention to detail and an inability to integrate these details into a more global comprehensive picture (Lopez, Tchanturia, Stahl, & Treasure, 2008; Lopez, Tchanturia, Stahl, & Treasure, 2009; Tenconi et al., 2010). A systematic review conducted by Lopez et al. (2008) notes that patients with BN have superior performance on the Matching Familiar Figures Test which suggests an increased attention to detail. Lopez et al. (2008) suggest that certain cognitive traits (i.e., perfectionism, fear of mistakes and resistance to change) found in patients with eating disorders may be related to a weak central coherence. In addition, recovery from an eating disorder may be more difficult when a combination of weak central coherence and difficulties with set shifting is present (Lopez et al., 2009). A systematic review performed by Van den Eynde et al. (2011) found that patients with BN tend to score lower than healthy controls on tasks measuring central coherence, suggesting bias toward detail and local processing. The research to date suggests that patients presenting with an eating disorder may have deficits in EF including: decision-making (Cavedini et al., 2006; Van den Eynde et al., 2011), set shifting (Tchanturia et al., 2004; Tchanturia et al., 2011; Tchanturia et al., 2012), and central coherence (Lopez et al., 2008; Lopez et al., 2009). However, these cognitive styles may not be representative of all individuals with an eating disorder diagnosis (Tchanturia et al., 2011). An easy to administer, reliable, and valid measurement of EF in an eating disorder population could be an important tool in screening for EF deficits in patients with eating disorders (Tchanturia et al., 2012). With such immediate information about EF a clinician may be able to tailor treatments to increase their effectiveness, or pursue a targeted performance-based assessment. Recently, the BRIEF (described below; Gioia, Isquith, Guy, & Kenworthy, 2000) has been used in an adolescent population of eating disorder patients with AN (Dahlgren, Lask, Landrø, & Rø, 2014). Thus, the BRIEF-A may serve as an initial screening tool to aid clinicians in understanding an adults' EF in their everyday environment (Roth et al., 2005). According to Roth et al. (2005), “the BRIEF-A has demonstrated reliability, validity, and clinical utility for ecological assessment of [EF] in individuals with a range of conditions across the adult age spectrum” (Roth et al., 2005, p.1). However, psychometric properties of the BRIEF-A have never been evaluated in a sample of patients with eating disorders. 1.2. The BRIEF-A (Roth et al., 2005) The BRIEF-A was developed as an extension of the original BRIEF questionnaire (designed for assessment of EF in school-aged

children; Gioia et al., 2000), in order to provide a self-reported assessment tool of EF for adults. The BRIEF-A was constructed using items from the BRIEF, modifying the wording of items where the behaviour described was not appropriate for an adult respondent. The final item pool consisted of approximately 160 items including items that had been added to reflect more general statements as well as behaviour-specific statements. Using item-total correlations and principle factor analysis the total item pool was reduced, and standardization of the BRIEF-A was performed using 78 items. The final version of the BRIEF-A is composed of 75 questions yielding nine clinical scales that form two higher-order indices, the Behavioral-Regulation Index (BRI; including the Inhibit, Shift, Emotional Control, and Self-Monitor scales), and the Metacognition Index (MCI; including the Initiate, Working Memory, Plan/Organize, Task Monitor, and Organization of Materials scales). The combination of the nine clinical scales forms one summary score called the Global Executive Composite (GEC) that provides an overall picture of the individual's perception of their EF. The measure includes three validity scales: Inconsistency, Negativity and Infrequency. The self-report scale is accompanied by an informant-report, which can be completed by someone who can comment on the individual's behaviour in his or her everyday environment. The Behavioral-Regulation Index (BRI) measures a respondent's ability to regulate their behaviour and emotional responses. The Inhibit scale measures a respondent's inhibitory control and ability to inhibit their behaviour when appropriate. An example of an item on the Inhibit subscale is “I have problems waiting my turn”. The Shift scale measures the ability of the adult to switch between situations as needed, and includes the item “I get disturbed by unexpected changes in my daily routine”. The Emotional Control scale measures the extent to which the individual is able to mediate emotional responses. “I get upset quickly or easily over little things” is an example of an item on the Emotional Control scale. The Self-Monitor scale measures the extent to which an individual can keep track of his or her behaviour, and the extent to which they are aware of the effect of their behaviour on others. An example of an item on the Self-Monitor scale is “I don't think about consequences before doing something”. The Metacognition Index (MCI) measures an adult's ability to solve problems in a systematic way by using skills involving planning, organization and holding information in working memory. The MCI is composed of Initiate, Working Memory, Plan/Organize, Task Monitor, and Organization of Materials scales. The Initiate scale contains items pertaining to beginning a task or activity and the behaviour associated with this (i.e. generating ideas); for example, “I need to be reminded to begin a task even when I am willing”. The Working Memory scale measures the respondent's ability to hold information in their mind and manipulate this information to achieve task completion; for example, “I have trouble with jobs or tasks that have more than one step”. An adult's ability to manage current and future demands is measured by the Plan/Organize scale. The Task Monitor scale measures the extent to which the individual keeps track of his or her own successes and failures. An item on the Task Monitor scale is “I misjudge how difficult or easy tasks will be”. The Organization of Materials scale measures the individual's organization within their environment and extends to the state of their work, living, and storage spaces. An example of an item on the Organization of Materials scale is “I leave my room or home a mess”. The BRIEF-A is designed to be used with adults between the ages of 18 and 90 years old and has been validated in a variety of populations compared to non-clinical controls. For example, those with attention deficit disorders had greatest difficulty in inhibitory control and working memory, those with multiple sclerosis showed significant problems in shifting and working memory, and those with traumatic brain injury reported prominent difficulties in the Task Monitor scale as well as other domains (Roth et al., 2005).

S. Ciszewski et al. / Eating Behaviors 15 (2014) 175–181

1.3. Objectives of the current study The objectives of this study are to assess the psychometric properties of the BRIEF-A in a clinical sample of patients with eating disorders. We hypothesize that the BRIEF-A will show evidence of internal consistency reliability, with Cronbach's alpha greater than .80 and mean inter-item correlations between .15 and .50 (Clark & Watson, 1995). We also hypothesize that the BRIEF-A will demonstrate construct and convergent validity. We expect that an exploratory factor analysis of the BRIEF-A scales will result in a two-factor solution in an eating disorder sample, similar to that reported in the BRIEF-A professional manual (Roth et al., 2005). As a further test of the hypothesized factors we also conducted a confirmatory factor analysis of the proposed two-factor solution. Further, we expect a moderate to high positive relationship (r N .30) between the BRIEF-A Self-Report and the BRIEF-A InformantReport (BRIEF-A[I]). Anxiety and depression are associated with EF difficulties (Castaneda, Tuulio-Henriksson, Marttunen, Suvisaari, & Lönqvist, 2008); however, as measures of anxiety and depression are not a direct assessment of EF we hypothesize a moderate positive relationship (r N .30) between the BRIEF-A and measures of depression and anxiety. 2. Method 2.1. Participants Between December 2009 and January 2012, 408 participants were referred for assessment and/or treatment at the Regional Center for the Treatment of Eating Disorders at The Ottawa Hospital. Of the 408 referrals, 96 did not answer enough items on the BRIEF-A to compute one or more of the clinical scales, indices, or global score. These 96 individuals were not significantly different on eating disorder diagnosis and demographic variables from those who did complete sufficient BRIEFA items. Hence data from the 96 individuals were removed from the study. In addition, three of the remaining participants did not complete any of the other study measures (see Section 2.3) and so their data were also removed. Of the remaining 309 participants, 33 were excluded because they were under the age of 18 (n = 11) or did not have an eating disorder diagnosis (n = 22). Screening for multivariate outliers was then conducted on the remaining 276 participants. Twelve participants' data were identified as multivariate outliers (Mahalanobis distances N χ2(8) = 15.5073, p b .05) and were excluded from the study. In order to achieve a homogenous sample, men (n = 12) were excluded from all analyses. Two cases had moderately elevated Infrequency scores on the BRIEF-A. However, other validity indices (i.e., Negativity and Inconsistency) were within normal range and so these cases' data were retained. The final eating disorder sample included 252 women between the ages of 18 and 59 (M = 29.70, SD = 10.45). Of the participants, 33.3% were diagnosed with an eating disorder not otherwise specified (EDNOS), 32.6% were diagnosed with BN, 21.0% were diagnosed with AN, and 13.1% were diagnosed with EDNOS-BED. Of the participants, 55.6% of the participants reported their marital status as single, 29.0% were married or in a common-law marriage, 9.6% were separated or divorced, and 0.4% were widowed. Regarding education, 14.3% completed secondary school, 27.4% completed some university or college, 29.8% completed university or college, and 7.9% completed graduate training. A large number of the participants (61.5%) were either employed fulltime or part-time and 32.2% were unemployed. A majority of the participants were European-Canadian (83.3%). Other participants included individuals with Aboriginal, Arab, African, Chinese, Latin American, South or West Asian backgrounds. Thirty-two participants also provided BRIEF-A(I) forms completed by a close friend or family member to accompany their BRIEF-A. Participants who provided BRIEF-A(I)s were not significantly different from those who did not provide complete BRIEF-A(I)s (n = 232) on age,

177

eating disorder diagnosis, living arrangements, employment status, family income and overall BRIEF-A score. 2.2. Procedure Data were collected from patients who were referred to The Ottawa Hospital — Regional Center for the Treatment of Eating Disorders for assessment and/or treatment. At an initial consultation, patients provided informed consent and completed a package of questionnaires. Patients were also given the BRIEF-A(I) and were asked to give the BRIEF-A(I) to someone close to them who could comment on the patient's behaviour. The BRIEF-A(I) was returned by mail by the informant. All patients provided informed consent and the study was approved by the local research ethics board. 2.3. Measures 2.3.1. The Behavior Rating Inventory of Executive Function — Adult Version (Roth et al., 2005) As previously described, the BRIEF-A is composed of 75 items, each with three response items: “never”, “sometimes”, or “often”. Higher scores on clinical scales, indices and the GEC are indicative of more problems or difficulties with the executive function measured by the respective scale. For the self-report normative sample, Roth et al. (2005) reported the moderate to high internal consistency (clinical scales α = .73 to .90, and α = .93 to .96 for the indices and the GEC; Roth et al., 2005). Roth et al. (2005) reported moderate to high correlations between the BRIEF-A and BRIEF-A(I) questionnaires (clinical scales, r = .44 to .68; indices and GEC r = .61 to .63; Roth et al., 2005). Exploratory factor analysis indicated that the nine clinical scales loaded onto two higher order factors corresponding to the BRI and MCI indices. BRIEF-A raw scores can be converted to T-scores derived from population-based data with a mean of 50 and standard deviation of 10 (Roth et al., 2005). Analyses will be conducted on the raw scores, though T score conversions of the mean raw scores also will be presented in the tables for descriptive purposes. 2.3.2. Personality Assessment Inventory (Morey, 2007) The Personality Assessment Inventory (PAI) is a self-report measure designed for use with adults, containing 344 items (Morey, 2007). The 344 items compose 4 validity scales, 11 clinical scales, 2 interpersonal scales, and 5 treatment-oriented scales. Raw scores are converted to population-based T-scores with a mean of 50 and a standard deviation of 10. For the current study, only the PAI Anxiety subscale, and the PAI Depression subscale, will be used. The PAI has been validated for use in an eating disorder population (Tasca, Wood, Demidenko, & Bissada, 2002). The mean inter-item correlations were good for the Anxiety scale (r = .17 to .31; Morey, 2007) and for the Depression scale (r = .24 to .36; Morey, 2007), indicating good internal consistency. 2.4. Statistical analyses Univariate analysis of variance (ANOVA), and multivariate analysis of variance (MANOVA) then Bonferroni correction to correct Type I error, followed by Tukey's post hoc test were used to compare eating disorder diagnostic groups. Cronbach's alpha (α N .80) and the mean inter-item correlations (r = .15 to .50) for each of the nine clinical scales were calculated to assess the internal consistency reliability of the BRIEF-A (Clark & Watson, 1995). Unlike coefficient alpha, the mean inter item correlation is not sensitive to the number of scale items and so may be more appropriate for scales with small or large number of items (Clark & Watson, 1995). Construct validity of the BRIEF-A was examined using an exploratory factor analysis of the nine subscales. Principal axis factoring was used with an oblique rotation (Promax), replicating the procedure used by Roth et al. (2005). Factor loadings greater than .40 were considered to load on a factor (Field, 2009). A

178

S. Ciszewski et al. / Eating Behaviors 15 (2014) 175–181

combination of mean eigenvalues and scree plot point of inflection was used to identify the number of factors to retain (Field, 2009; Jolliffe, 2002). This was done to replicate procedures used in the initial validation of the scale (Roth et al., 2005). Further we conducted a confirmatory factor analysis (CFA) on the same data to evaluate the hypothesized two-factor structure. CFA allows us to evaluate model fit to the data and potential modification to the model if necessary. Pearson correlations were calculated between the factors and the GEC to further assess construct validity. Pearson's correlations were also used to assess the association between the BRIEF-A self-report and the informant-report to evaluate convergent validity. In another evaluation of convergent validity, Pearson's correlations were calculated between the GEC of the BRIEF-A, and between the GEC and the Depression and Anxiety subscales of the PAI.

3. Results 3.1. Preliminary analyses Three of the nine clinical scales on the BRIEF-A were significantly positively skewed: Organization of Materials, Inhibit, and Self Monitor. A square root transformation was used to correct the three positively skewed clinical scales; these transformed scales were used in all analyses. All other scales were normally distributed and there were no univariate outliers. Non-transformed means and standard deviations for BRIEF-A and BRIEF-A(I) scales are reported in Table 1. The means and standard deviations were also calculated for the PAI subscales used in the current study (Anxiety, M = 70.99, SD = 13.92; Depression, M = 75.61, SD = 15.07). We compared those with AN (restricting type; AN-R), AN (binge-purge type; AN-BP), and BN on the BRIEF-A scales. EDNOS was heterogeneous in terms of symptom presentation and so this diagnostic category was not included in these analyses. We found a significant multivariate difference between those with AN-R, AN-BP, and BN on the two BRIEF-A indices, F(2, 132) = 3.011, p = .05. After Bonferroni's correction was applied, univariate ANOVAs did not show a significant effect of eating disorder diagnosis on the BRIEF-A scales. In a second MANOVA, we found a significant multivariate effect when comparing AN-R, AN-BP, and BN on the 9 BRIEF-A clinical scales, F(9, 125) = 3.11, p = .002. After Bonferroni's correction was applied, the univariate ANOVAs showed that there was a significant effect of eating disorder diagnosis on the Organization of Materials

scale, F(1, 138) =11.67, p = .001. Tukey's HSD post hoc test showed that those with BN had significantly higher scores on the organization of materials scale than those with AN-R (p = .005) or AN-BP (p = .041). 3.2. Construct validity of the BRIEF-A For the exploratory factor analysis (EFA) the data met assumptions including no multicolinearity or sphericity. An initial factor analysis was performed to obtain eigenvalues for each of the components in the analysis. An examination of the scree plot showed one large factor and a one to three factor solution based on the point of inflection. We combined the recommendations of Jolliffe (2002) and Field (2009) to retain factors based on the average eigenvalues, and so used eigenvalues greater than .70 as the cut off (Field, 2009). As a result, a two-factor solution was selected. Table 3 reports the factor loadings after rotation based on the pattern matrix. The results showed that the scales loading on Factor 1 are all from the MCI of the BRIEF-A with the exception of the Inhibit scale, and those that load on Factor 2 are from the BRI of the BRIEF-A. The Inhibit scale was a cross loading scale. Factor 1 and Factor 2 accounted for 58.20% and 5.33% of the variance, respectively. Finally, as expected, these two factors were correlated with each other, and highly correlated with the GEC (Table 4). We then conducted a follow-up CFA to assess model fit of the hypothesized two-factor model. Consistent with the EFA, each BRIEF-A scale significantly loaded on its hypothesized factor (Table 3). The two factors, MCI and BRI were correlated at .86, p b .001. However, model fit ranged from adequate, CFI = .93, to poor, RMSEA = .12 (CI: 10, .14), χ2(26) = 130.45 (Browne & Cudeck, 1993; Byrne, 2001). Modification indices indicated that associations between two sets of residuals had to be modelled to improve model fit: (a) residuals for Inhibit and Shift were modelled as correlated, as were (b) residuals for Inhibit and Working Memory. After these modifications, model fit ranged from very good, CFI = .98, to mediocre, RMSEA = .09 (CI: .07, .12), χ2(24) = 56.63, p b .001 (Browne & Cudeck, 1993; Byrne, 2001). 3.3. Convergent validity of the BRIEF-A Significant positive correlations were found between the PAI Depression subscale and the GEC of the BRIEF-A, and between the PAI Anxiety subscale and GEC of the BRIEF-A (Table 4). Convergent validity was also demonstrated by a high positive and significant correlation

Table 1 Descriptive statistics for BRIEF-A and BRIEF-A(I) study measures. Scale

ED sample (N = 252)

AN-R (n = 17)

AN-BP (n = 36)

BN (n = 82)

T

M

SD

T

M

SD

T

M

SD

T

M

SD

BRIEF-A Inhibit Shift Emotional Control Self-Monitor Initiate Working Memory Plan/Organize Task Monitor Organization of Materials MCI BRI GEC

61 65 62 55 64 64 61 60 54 61 63 63

14.57 11.63 20.40 9.97 15.70 15.08 17.69 10.92 14.05 73.44 56.58 130.01

3.38 2.57 5.09 2.83 3.58 3.84 4.26 3.05 4.35 15.74 11.60 25.49

54 65 57 51 61 60 55 51 48 55 58 63

12.94 12.00 18.47 9.00 14.88 13.41 15.88 9.35 11.82 65.35 52.41 117.76

3.63 2.67 6.09 2.57 4.40 3.64 3.82 2.34 3.92 16.22 13.27 28.48

61 65 62 55 64 67 58 60 51 61 61 62

14.76 11.51 19.19 9.69 15.51 16.20 17.26 10.76 13.30 73.04 55.16 128.20

3.74 2.67 5.13 2.90 3.02 3.39 4.09 2.77 4.17 14.61 12.14 24.55

61 65 64 55 64 64 61 60 57 64 62 66

15.23 11.69 21.41 10.49 16.13 15.26 18.38 11.45 15.44 76.65 58.82 135.47

3.12 2.42 4.91 2.88 3.56 3.76 4.05 2.80 4.50 15.10 11.23 24.35

BRIEF-A(I) GEC

60

122.94a

28.46

58

119.00b

–

57

117.40c

21.93

61

125.25d

22.19

Note. M = Mean; SD = Standard Deviation; BRIEF-A = Behavior Rating Inventory of Executive Function — Adult Version, self-report; BRIEF-A(I) = Informant Report; T refers to T-score conversion of the mean raw score value for the BRIEF-A with a normative population M = 50 and SD = 10. a n = 32. b n = 1. c n = 5. d n = 12.

S. Ciszewski et al. / Eating Behaviors 15 (2014) 175–181 Table 2 Reliability analyses of the BRIEF-A clinical scales, indices and summary score. Scale

Cronbach's alpha (α)

Mean inter-item correlations

Inhibit Shift Initiate Emotional Control Self-Monitor Working Memory Plan/Organize Task/Monitor Organization of Materials Metacognition Index Behavioural Regulation Index Global Executive Composite

.74 .69 .77 .91 .81 .83 .83 .77 .88 .94 .93 .96

.26 .27 .30 .49 .41 .37 .33 .36 .48 .29 .29 .26

N = 252. BRIEF-A = Behavior Rating Inventory of Executive Function — Adult Version.

between the GEC of the BRIEF-A and between the GEC of the BRIEF-A informant-report (Table 4). 3.4. Internal consistency of the BRIEF-A Cronbach's alphas and mean inter-item correlations revealed adequate internal consistency for the 9 BRIEF-A clinical scales, and high internal consistency for the two indices (BRI and MCI) and the GEC (Table 2). 4. Discussion To date, no self-report measure of EF has been evaluated in a population of patients with eating disorders. The goal of the present study was to examine the reliability and validity of the BRIEF-A as a selfreport measure of EF in a clinical sample of eating disorder patients. We hypothesized that the BRIEF-A would demonstrate good internal consistency reliability, as well as adequate convergent and construct validity within an eating disorder sample. The first hypothesis, that the BRIEF-A scales would demonstrate high internal consistency reliability, was supported. Moderate to high coefficient alphas for the nine clinical scales (α = .69 to .91), and high alpha coefficients for the two indices (BRI and MCI; α = .93 and .94, respectively) and summary score (GEC; α N .93) indicate good internal consistency reliability of the BRIEF-A scales. Four clinical scales showed internal consistency below the expected value of .80: Shift (α = .69), Inhibit (α = .74), Initiate (α = .77), and Task/Monitor (α = .77). These internal consistency levels mirrored what was found in the original validation of the BRIEF-A performed by Roth et al. (2005). The mean inter-item correlations, which are not influenced by number of scale items, ranged from .26 to .49 for the clinical scales, thus providing Table 3 Factor loadings for exploratory factor analysis (EFA) with Promax rotation and confirmatory factor analysis (CFA) of the BRIEF-A clinical scales. EFA

CFA

Scale

Factor 1

Factor 2

MCI

Plan/Organize Task Monitor Organization of Materialsa Initiate Working Memory Emotional Control Self-Monitora Shift Inhibita

.93 .84 .72 .68 .52 −.11 .07 .07 .41

−.02 .04 −.09 .14 .30 .85 .77 .70 .44

.91 .85 .64 .81 .78

BRI

.71 .82 .74 .81

Note. N = 252. BRIEF-A = Behavior Rating Inventory of Executive Function — Adult Version. EFA factor loadings from pattern matrix N.40 are in boldface. Factor 1 = Metacognition Index (MCI); Factor 2 = Behavioral Regulation Index (BRI). CFA loadings are standardized regression weights, all p b .001.

179

other evidence of adequate internal consistency reliability of all the scales (Clark & Watson, 1995). The second hypothesis concerned the higher order construct validity of the BRIEF-A scales. Specifically, we expected that an exploratory factor analysis of the BRIEF-A would result in a two-factor solution, with high correlations between the two factors and the summary score. The exploratory factor analysis resulted in a two-factor structure in which clinical scales loaded on two higher order factors that are consistent with those reported in the BRIEF-A professional manual (Roth et al., 2005). However, the Inhibit scale cross loaded on both the BRI and MCI factors. In addition, the hypothesized factor structure did not fit the data well when tested with a CFA. Two modifications of correlated residuals involving the Inhibit scale were required to improve model fit to an acceptable level. The correlated residuals in the CFA as well as the cross loadings found on the EFA suggest that an unmodelled latent higher order factor may exist associated with the Inhibit scale. Similarly, Gioia, Isquith, Retzlaff, and Espy (2002) found evidence for a three factor structure for the BRIEF-A when used with children with mixed clinical diagnoses. In Gioia and colleagues' model, the BRI was split up into a Behavior Regulation factor and an Emotion Regulation factor, in which the Behavior Regulation factor was made up of the Inhibit and SelfMonitoring scales. Another possible explanation for the poor fit of the hypothesized model and need for modifications is that the latent BRIEF-A factors may differ for BN versus AN populations. Similarly, researchers found that the structural relationship between maintenance factors and eating disorder psychopathology can differ across diagnostic groups (e.g., Lampard, Tasca, Balfour, & Bissada, 2013). Taken together, these results provide only conditional support for the construct validity of the BRIEF-A in a clinical sample of eating disorder patients. The final hypothesis concerned the convergent validity of the BRIEFA. We expected the GEC of the BRIEF-A to have a high positive association with the GEC of the BRIEF-A(I), and that the GEC of the BRIEF-A would show a positive relationship with the Anxiety and Depression subscales of the PAI. These hypotheses were supported. These latter findings parallel those presented in the original validation of the BRIEF-A (Roth et al., 2005), which reported a significant relationship between the GEC and Beck Depression Inventory II, and between the GEC and State Trait Anxiety Inventory (Roth et al., 2005). Our findings indicate convergent validity of the BRIEF-A in a sample of patients with eating disorders. Examining the T score conversions of the sample raw score means in Table 1 indicates that the average eating disorder patient on average had GEC scores that were more than one standard deviation above the population mean. This suggests overall clinical elevations in problems with EF in this sample. Eating disorder diagnosis was not associated with differences on GEC or the BRIEF-A indices (Table 1). Further, there were few differences between those with AN-R, AN-BP, and those with BN on most BRIEF-A scales. The exception was significantly different scores on the Organization of Materials clinical scale (Table 1). This was not a hypothesized finding but emerged from our initial exploration of the data. This may suggest greater problems among those with BN compared to AN-R and AN-BP on their ability to maintain orderliness in their personal belongings or workspace. Theoretically, differences in EF could contribute to the differentiation between eating disorder diagnostic groups. However, these findings require further research and replication. Future research with a larger sample can identify different domains of EF that may be elevated or impaired, which could inform the clinical treatment of eating disorder patients. For example, based on our findings on the average elevated BRI relative to the norm, eating disorder treatment methods could be tailored to reduce rigidity, increase flexibility and set shifting, and improve emotional and behavioural control. Tchanturia et al. (2004) suggest that broadening a patient's range of flexibility may be advantageous when addressing rigid eating behaviours in a clinical setting. Further knowledge of EF group differences in eating disorders may assist in understanding why some diagnostic

180

S. Ciszewski et al. / Eating Behaviors 15 (2014) 175–181

Table 4 Pearson's correlations between Depression and Anxiety scales and the BRIEF-A factors, BRIEF-A GEC, and the BRIEF-A(I) GEC. Factor 1 (MCI) Factor 1 (MCI) Factor 2 (BRI) BRIEF-A GEC BRIEF-A Ia GEC Anxiety depression

Factor 2 (BRI)

BRIEF-A GEC

BRIEF-A(I)a GEC

Anxiety

Depression

.668⁎

.955⁎ .859⁎

.784⁎ .723⁎ .849⁎

.500⁎ .616⁎ .590⁎ .534⁎

.499⁎ .552⁎ .564⁎ .571⁎ .681⁎

Note. N = 264. BRIEF-A = Behavior Rating Inventory of Executive Function — Adult Version; BRIEF-A(I) = informant version; GEC = General Executive Composite; MCI = Metacognitive Index; BRI = Behavioral Regulation Index; Anxiety = subscale of the Personality Assessment Inventory (PAI); Depression = subscale of the PAI. a n = 32 informants. ⁎ p b .01.

groups may respond and adhere differently to a treatment intervention. Future research with larger samples should be conducted to clarify the latent structure of the BRIEF-A scales. Two or three factor models, similar to those proposed by Gioia et al. (2002) can be tested to identify the best fitting factor structure for an eating disorder population. Further, testing the invariance of the BRIEF-A factor structure across eating disorder diagnoses may clarify important issues regarding the relative importance of EF latent factors in eating disorder diagnostic groups. One limitation of the present study is that the sample consisted primarily of Canadian-European females. Thus, results may not be generalizable to individuals of different race, gender, or ethnicity. Future research should include a more heterogeneous population to increase the generalizability of the results. Regardless, this study is the first to assess the validity of the BRIEF-A as a measure of self reported EF in a clinical sample of eating disorder patients. A second limitation is that the sample size was somewhat small for an exploratory factor analysis, and we did not have an independent sample for the CFA. Future research should test the fit of 2 versus 3 factor models within specific eating disorder diagnostic groups. A third limitation of the present study is that correlations between the GEC of the BRIEF-A and the BRIEF-A(I) were performed on only a small sample of participants (i.e., those who returned complete informant reports). These BRIEF-A(I) data in this study may represent a biassed sub-sampling of cases that may not represent the population. Although we found that those who returned informant questionnaires were not different than those who did on a number of variables, a replication should include a larger sample of BRIEF-A(I) reports to confirm this aspect of convergent validity. The use of the informant report is ideal to capture a more comprehensive picture of an individual's cognitive functioning; however, if informants do not provide reports of the BRIEF-A, then assessment of EF would rely on self-reported behaviours only. It should be noted that the BRIEF-A is a self-report measure that relies on the assumption that the respondent is accurately and consistently reporting their behaviours. Clinical information can certainly be gained by assessing a patient's opinion of his or her behaviour. However, in comparison to performance-based measures of EF, self-report measures may be targeting different aspects of EF (Nęcka, Lech, Sobczyk, & Śmieha, 2012; Toplak, West, & Stanovich, 2013). Specifically, Lounes, Khan, and Tchanturia (2011), did not find a significant relationship between a self-report and experimental test of cognitive flexibility in an AN sample. Clinicians should keep in mind that the BRIEF-A was designed to capture an adult's perspective of his or her EF strengths and weaknesses, and was not designed as a diagnostic tool in isolation. The BRIEF-A should be used in conjunction with other measures and information sources including neuropsychological performance testing. 4.1. Conclusion The results of the present study suggest that the BRIEF-A is potentially a valid and reliable tool for assessing EF in an eating disorder population. With the BRIEF-A, clinicians have an easy administration screening tool that may indicate problems with EF, which requires further assessment and tailored interventions. The BRIEF-A could also aid researchers

in the furthering knowledge concerning EF in eating disorders. Information on EF in an eating disorder population may enhance our understanding of eating disorder diagnoses, and help clinicians better to tailor treatments for eating disorder patients, in order to maximize positive treatment outcomes. Funding sources The University of Ottawa and The Ottawa Hospital had no role in the study design, collection, analysis or interpretation of the data, writing the manuscript, or the decision to submit the paper for publication. Contributors Stephanie Ciszewski and Kylie Francis conducted the literature reviews and the data analysis. Stephanie Ciszewski also wrote the initial draft of the manuscript. Hany Bissada conducted the diagnostic assessments and consulted on the study design. Paul Mendella and Giorgio Tasca conceived of and designed the study and conducted preliminary literature reviews. Giorgio Tasca edited the manuscript and supervised the data analyses. All authors reviewed, commented on, and approved the final manuscript. Conflict of interest The authors report no conflict of interest.

Acknowledgments Giorgio A. Tasca holds the Research Chair in Psychotherapy Research at the University of Ottawa and The Ottawa Hospital.

References Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, & J. S. Long (Eds.), Testing structural equation models (pp. 445–455). Newbury Pk, C.A.: Sage. Byrne, B.M. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming (2nd ed.). New York, NY: Routledge Academy. Castaneda, A. E., Tuulio-Henriksson, A., Marttunen, M., Suvisaari, J., & Lönqvist, J. (2008). A review on cognitive impairments in depressive and anxiety disorders with a focus on young adults. Journal of Affective Disorders, 106(1–2), 1–27. Cavedini, P., Bassi, T., Ubbiali, A., Casolari, A., Giordani, S., Zorzi, C., et al. (2004). Neuropsychological investigation of decision-making in anorexia nervosa. Psychiatry Research, 127(3), 259–266. Cavedini, P., Zorzi, C., Bassi, T., Gorini, A., Baraldi, C., Ubbiali, A., et al. (2006). Decision-making functioning as a predictor of treatment outcome in anorexia nervosa. Psychiatry Research, 145(2–3), 179–187. Clark, L., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319, http://dx.doi.org/10.1037/10403590.7.3.309. Dahlgren, C. L., Lask, B., Landrø, N. I., & Rø, Ø. (2014). Patient and parental self-reports of executive functioning in a sample of young female adolescents with anorexia nervosa before and after cognitive remediation therapy. European Eating Disorders Review, 22, 45–52, http://dx.doi.org/10.1002/erv.2265. Danner, U. N., Ouwehand, C., Haastert, N. L., Hornsveld, H., & de Ridder, D.D. (2012). Decisionmaking impairments in women with binge eating disorder in comparison with obese and normal weight women. European Eating Disorders Review, 20(1), e56–e62. Danner, U. N., Sanders, N., Smeets, P.M., van Meer, F., Adan, R. H., Hoek, H. W., et al. (2012). Neuropsychological weaknesses in anorexia nervosa: Set-shifting, central coherence, and decision making in currently ill and recovered women. International Journal of Eating Disorders, 45(5), 685–694. Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: SAGE. Gioia, G. A., Isquith, P. K., Guy, S.C., & Kenworthy, L. (2000). The Behavior Rating Inventory of Executive Function: Professional manual. Lutz, FL: Psychological Assessment Resources. Gioia, G. A., Isquith, P. K., Retzlaff, P. D., & Espy, K. A. (2002). Confirmatory factor analysis of the Behavior Rating Inventory of Executive Functioning (BRIEF) in a clinical sample. Child Neuropsychology, 8, 249–257.

S. Ciszewski et al. / Eating Behaviors 15 (2014) 175–181 Guillaume, S., Sang, C., Jaussent, I., Raingeard, I., Bringer, J., Jollant, F., et al. (2010). Is decision making really impaired in eating disorders? Neuropsychology, 24(6), 808-812. Harrison, A., Tchanturia, K., Naumann, U., & Treasure, J. (2012). Social emotional functioning and cognitive styles in eating disorders. British Journal of Clinical Psychology, 51(3), 261–279. Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). New York: Springer (Original work published 1986). Lampard, A.M., Tasca, G. A., Balfour, L., & Bissada, H. (2013). An evaluation of the transdiagnostic cognitive-behavioral model of eating disorders. European Eating Disorders Review, 21, 99–107. Lounes, N., Khan, G., & Tchanturia, K. (2011). Assessment of cognitive flexibility in anorexia nervosa — Self-report or experimental measure? A brief report. Journal of the International Neuropsychological Society, 17(5), 925–928. Liao, P. C., Uher, R., Lawrence, N., Treasure, J., Schmidt, U., Campbell, I. C., et al. (2009). An examination of decision making in bulimia nervosa. Journal of Clinical and Experimental Neuropsychology, 31(4), 455–461. Lopez, C., Tchanturia, K., Stahl, D., & Treasure, J. (2008). Central coherence in eating disorders: A systematic review. Psychological Medicine, 38(10), 1393–1404. Lopez, C., Tchanturia, K., Stahl, D., & Treasure, J. (2009). Weak central coherence in eating disorders: A step towards looking for an endophenotype of eating disorders. Journal of Clinical and Experimental Neuropsychology, 31(1), 117–125. Luria, A.R. (1966). Higher cortical functions in man. New York: Basic Books. Miyake, A., Friedman, N.P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex ‘frontal lobe’ tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. Morey, L. C. (2007). Personality Assessment Inventory: Professional manual (2nd ed.). Lutz, FL: Psychological Assessment Resources. Morgan, A.B., & Lilienfeld, S. O. (2000). A meta-analytic review of the relations between antisocial behaviour and neuropsychological measures of executive function. Clinical Psychology Review, 20(1), 113–156. Nęcka, E., Lech, B., Sobczyk, N., & Śmieha, M. (2012). How much do we know about our own cognitive control? Self-report and performance measures of executive functions. European Journal of Psychological Assessment, 28(3), 240–247. Roberts, M. E., Tchanturia, K., & Treasure, J. L. (2010). Exploring the neurocognitive signature of poor set-shifting in anorexia nervosa and bulimia nervosa. Journal of Psychiatric Research, 44(14), 964–970.

181

Roth, R. M., Isquith, P. K., & Gioia, G. A. (2005). Behavior Rating Inventory of Executive Function—Adult Version: Professional manual. Lutz, FL: Psychological Assessment Resources. Schmidt, U., & Treasure, J. (2006). Anorexia nervosa: Valued and visible. A cognitive-interpersonal maintenance model and its implications for research and practice. British Journal of Clinical Psychology, 45(3), 343–366. Svaldi, J., Brand, M., & Tuschen-Caffier, B. (2010). Decision-making impairments in women with binge eating disorder. Appetite, 54(1), 84–92. Tasca, G. A., Wood, J., Demidenko, N., & Bissada, H. (2002). Using the PAI with an eating disordered population: Scale characteristics, factor structure, and differences among diagnostic groups. Journal of Personality Assessment, 79(2), 337–356. Tenconi, E., Santonasto, P., Degortes, D., Bosello, R., Titton, F., Mapelli, D., et al. (2010). Set-shifting abilities, central coherence, and handedness in anorexia nervosa patients, their unaffected siblings and healthy controls: Exploring putative endophenotypes. The World of Biological Psychiatry, 11, 813–823. Tchanturia, K., Davies, H., Roberts, M., Harrison, A., Nakazato, M., Schmidt, U., et al. (2012). Poor cognitive flexibility in eating disorders: examining the evidence using the Wisconsin card sorting task. Plos ONE, 7(1), http://dx.doi.org/10.1371/journal.pone. 0028331. Tchanturia, K., Harrison, A., Davies, H., Roberts, M., Oldershaw, A., Michiko, N., et al. (2011). Cognitive flexibility and clinical severity in eating disorders. Plos ONE, 6(6), http://dx.doi.org/10.1371/journal.pone.0020462. Tchanturia, K., Liao, P., Uher, R., Lawrence, N., Treasure, T., & Campbell, I. C. (2007). An investigation of decision making in anorexia nervosa using the Iowa Gambling Task and skin conductance measurements. Journal of the International Neuropsychological Society, 13(4), 635–641. Tchanturia, K., Morris, R. G., Anderluh, M., Collier, D. A., Nikalaou, V., & Treasure, J. (2004). Set shifting in anorexia nervosa: An examination before and after weight gain, in full recovery and relationship to childhood and adult OCPD traits. Journal of Psychiatric Research, 38(5), 545–552. Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Do performance-based measures and ratings of executive function assess the same construct? Journal of Psychology and Psychiatry, 54(2), 131–143. Van den Eynde, F., Guillaume, S., Broadbent, H., Stahl, D., Campbell, I. C., Schmidt, U., et al. (2011). Neurocognition in bulimic eating disorders: a systematic review. Acta Psychiatrica Scandinavica, 124(2), 120–140.