YNIMG-12378; No. of pages: 10; 4C: 5, 7, 8 NeuroImage xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

NeuroImage journal homepage: www.elsevier.com/locate/ynimg

3Q2

G. Koppe a,b,⁎, A. Heidel a, G. Sammer b, M. Bohus a, B. Gallhofer b, P. Kirsch c, S. Lis a,b

O

F

2

Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli

4 5 6

a

7

a r t i c l e

8 9 10 11

Article history: Received 16 December 2014 Accepted 28 June 2015 Available online xxxx

12 13 14 15 16 17 18

Keywords: Implicit timing Unpredictability Implicit emotion processing Face processing Interstimulus interval fMRI

i n f o

R O

Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Germany Centre for Psychiatry, Justus Liebig University Giessen, Giessen, Germany Department of Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Germany

a b s t r a c t

Most experimental settings in cognitive neuroscience present a temporally structured stimulus sequence, i.e., stimuli may occur at either constant and predictable or variable and less predictable inter-stimulus intervals (ISIs). This experimental feature has been shown to affect behavior and activation of various cerebral structures such as the parietal cortex and the amygdala. Studies employing explicit or implicit cues to manipulate predictability of events have shown that unpredictability particularly accentuates the response to events of negative valence. The present study investigates whether the effects of unpredictability are similarly affected by the emotional content of stimuli when unpredictability is induced simply by the temporal structure of a stimulus sequence, i.e., by variable as compared to constant ISIs. In an fMRI study, we applied three choice–reaction–time tasks with stimuli of different social–emotional content. Subjects (N = 30) were asked to identify the gender in angry and happy faces, or the shape of geometric figures. Tasks were performed with variable and constant ISIs. During the identification of shapes, variable ISIs increased activation in widespread areas comprising the amygdala and fronto-parietal regions. Conversely, variable ISIs during gender identification resulted in a decrease of activation in a small region near the intraparietal sulcus. Our findings reveal that variability in the temporal stimulus structure of an experimental setting affects cerebral activation depending on task demands. They suggest that the processing of emotional stimuli of different valence is not much affected by the decision of employing a constant or a variable temporal stimulus structure, at least in the context of implicit emotion processing tasks. In contrast, temporal structure diversely affects the processing of neutral non-social compared to emotional stimuli, emphasizing the relevance of considering this experimental feature in studies which aim at differentiating social–emotional from cognitive processing in general, and more particularly, aim at identifying circumscribed alterations of social cognition in mental disorders. © 2015 Published by Elsevier Inc.

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

properties of the experimental design affect stimulus categories in an equal manner. Stimulus timing for instance, is one experimental feature which may represent a potential confound. In functional magnetic resonance imaging (fMRI), stimulus timing is partly associated to the improvement of design efficiency (Dale, 1999; Friston et al., 1999; Liu and Frank, 2004; Liu et al., 2001), as variable interstimulus intervals (ISIs) are more efficient, and necessary in rapid event-related designs (Burock et al., 1998; Dale, 1999). However, besides improving efficiency, variable stimulus timing is also less predictable and affects behavior and activation in social and emotionally relevant brain areas (Koppe et al., 2014; Ryan et al., 2010; Wodka et al., 2009). The question that arises is whether unpredictability in the temporal stimulus structure has different consequences on the processing of social–emotional vs. neutral information. It is well established that unpredictability in respect to the temporal onset of an aversive event, promotes anxiety-like behavior in rodents (e.g., Abott, 1985; Fanselow, 1980; Imada and Nageishi, 1982 for

56

D

E

R

R

E

C

c

T

b

P

1Q1

44 42 41 43

N C O

40

Introduction

46

An extensive amount of neuroscience studies has focused on disentangling the contribution of various brain areas to social cognition and emotion processing (e.g., Adolphs and Tranel, 1999; Garvert et al., 2014; Hariri et al., 2002, 2003), and, in the context of mental disorders, has related alterations within associated areas with specific deficits in these domains (e.g., Domes et al., 2009; Evans et al., 2008; Meyer-Lindenberg et al., 2005). To this end, the neuronal response towards social–emotional stimulus material needs to be contrasted to non-social and/or non-emotional (control) stimulation. However, a clear dissociation between these processes is only warranted if all

47 48 49 50 51 52 53 54 55

U

Q3 45

⁎ Corresponding author at: Central Institute of Mental Health, Medical Faculty Mannheim/University of Heidelberg, Department of Psychosomatic Medicine and Psychotherapy, PO Box 12 21 20, 68072 Mannheim, Germany. Fax: +49 62117034005. E-mail address: [email protected] (G. Koppe).

http://dx.doi.org/10.1016/j.neuroimage.2015.06.081 1053-8119/© 2015 Published by Elsevier Inc.

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138

F

O

R O

94 95

139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156

Materials and methods

157

Subjects

158

Thirty healthy subjects (15 male, 15 female; age: 24.2 ± 2.5 years) participated in the study. All subjects were right-handed (Annett, 1967), and had normal or corrected-to-normal vision. All of them were undergraduate students at the Justus-Liebig-University, Giessen, with no history of psychiatric or neurological disorders. They were either awarded credits for research participation or received 15 Euros for participating in the study. All participants gave their written informed consent prior to participating in the study. The study was conducted in accordance with the Declaration of Helsinki, and was approved by the local ethics committee of the University of Giessen, School of Medicine.

159 160

Stimulus material

170

Stimulus material consisted of stimuli conveying emotional information (happy and angry faces), as well as emotionally neutral, nonsocial stimuli. Face stimuli were obtained from the Karolinska Directed Emotional Faces set (Lundqvist et al., 1998). From this set, pictures from 18 male and 18 female identities posing frontally for angry and happy emotional facial expressions were chosen. As non-social, nonemotional control stimuli, we applied geometric shapes, consisting of triangles and squares according to a previous study (Koppe et al., 2014). To reduce differences between shapes and faces in regard to low level visual characteristics, shapes were presented on a background picture of scrambled facial stimuli. Within each task, stimuli were presented in pseudo-random order on a computer screen (stimulus duration 100 ms, see Figure S1). Throughout the tasks, a resting button and two target buttons were displayed and labeled with ‘male’ and ‘female’, or ‘triangle’ and ‘square’, respectively. The subjects were instructed to respond as fast as possible, while avoiding errors. Subjects signaled their choice by releasing a resting button and pressing the appropriate target button with the index finger of the right hand. The currently pressed button was indicated by a cursor on the screen. Responses had to be initiated within 2 s after stimulus onset, slower reactions were processed as errors.

171

Experimental paradigm

192

All subjects were asked to solve choice reaction tasks with three types of stimuli under two conditions of temporal stimulus structure, varying in regard to predictability (see Figure S1). The first experimental factor was thus the temporal structure of the applied stimuli. The temporal structure, i.e., the occurrence of stimuli in

193 194

P

92 93

D

90 91

We focused on brain areas which have previously been linked to the differential effects of constant and variable ISIs in the processing of neutrally valent stimuli, investigated in a design similar to that used in the present study: the amygdala, and the parietal cortex, the supplementary motor area (SMA), and the dorsolateral prefrontal cortex (DLPFC). These areas are more strongly engaged in variable compared to constant ISI conditions (Koppe et al., 2014). The amygdala is of particular interest in the present study since it has been linked to the processing of threat and fear in general (Phelps and LeDoux, 2005), as well as in the context of uncertainty and unpredictability in particular (Bar and Neta, 2008; Herry et al., 2007; Whalen, 2007). Both the inferior and the superior parietal cortices have been associated with temporal orienting and adjustment processes (Cotti et al., 2011; Coull, 2004; Nobre, 2001; Sakai et al., 2000), and are modulated by predictability linked to cueing (Coull, 2004; Sakai et al., 2000), while the DLPFC and the SMA have been implicated in monitoring and updating of the hazard function when the temporal interval between events varies between trials (Coull and Nobre, 2008; Cui et al., 2009; Vallesi et al., 2007a, 2007b, 2009).

T

88 89

C

86 87

E

84 85

R

82 83

R

80 81

O

79

C

77 78

N

75 76

review; Seligman, 1968; Seligman and Meyer, 1970), as well as in humans (e.g., Grillon et al., 2004; Katz and Wykes, 1985; McClure et al., 2003; Price and Geer, 1972), as measured by an increase in avoidance behavior, preference for predictable environments, more disturbed physiological responding, or a potentiated startle reflex. In these studies, temporal cues, sometimes in the form of a conditioned stimulus, are typically employed to signal the occurrence or non-occurrence, the duration, and/or the temporal window of an aversive, unconditioned stimulus (UCS), rendering this event more predictable. It has been proposed that cued, that is, predictable aversive events, are preferred to non-cued, unpredictable events, since they signal reliable periods of safety, allowing the agent to reduce vigilance and relax (Seligman and Binik, 1977). However, in contrast to the employment of temporal cues which signal the onset of an aversive event, temporal unpredictability may also be conceptualized as an inherent property of the stimulus structure, i.e., with the UCS itself occurring at unpredictable, variable as compared to predictable, constant ISIs (Coull and Nobre, 2008; Imada and Nageishi, 1982). When stimuli are presented at variable or constant ISIs, temporal predictions are exogenously triggered and assumed to emerge as a byproduct of the temporal regularity at which they are presented (see Coull and Nobre, 2008 for reviews; Coull et al., 2011). The timing processes underlying these predictions are assumed to differ from implicit timing mechanisms that are triggered by temporal cues (Coull and Nobre, 2008; Coull et al., 2011). It is yet unclear, whether temporal unpredictability induced implicitly by application of variable as compared to constant ISIs may similarly modulate the response to aversive stimulation, and whether this modulation deviates from effects in neutral stimulation. With respect to neutral stimuli, Herry et al. (2007) were the first to show that presenting variable background sound pulses of high frequency (in the range of 200 ms) induces sustained amygdala activation, which is accompanied by increased avoidance behavior in rodents, as well as an increased spatial attention bias towards threatening information in humans. On a time scale of a few seconds, Koppe et al. (2014) observed a similar increase in amygdala activation to neutral visual stimuli presented at variable ISIs, and thus in a time frame relevant to fMRI. Beside activation within the amygdala, variable ISIs additionally engaged prefrontal and parietal areas during a choice reaction task. The increase of activation during this condition could at least partially be attributed to the formation of temporal expectations over time. The BOLD response within these regions co-varied with the cumulative conditional probability that a stimulus would occur, given it had not already occurred, i.e., the cumulative hazard function, an index of temporal expectancy (Cui et al., 2009; see Nobre et al., 2007 for a review). Regarding aversive stimuli, studies which investigate effects of variable and constant ISIs on this time scale are lacking. However, rodent studies employing fixed or variable time schedules of shock application provide first evidence that variably timed shock increases anxiety-like behavior (Bassett et al., 1973; Guile, 1987; Orsini et al., 2002), albeit the interval duration applied in these studies exceed durations customary in fMRI (i.e., N 30 s). In the present study, we investigated how constant and variable ISIs in the seconds range affect processing of emotional and neutral stimuli. To this end, we employed angry facial expressions serving as aversive stimuli since they signal potential threat, and are furthermore well known to induce amygdala activation (Boll et al., 2011). Studies which have employed temporal cues to investigate the relationship of unpredictability and threat, demonstrate that combining unpredictability with aversive stimulation results in a potentiated fear response. This effect has been measured by the fear potentiated startle reflex, with the amygdala representing one underlying neuronal substrate (Grillon, 2008; Vaidyanathan et al., 2009 for reviews). Based on these observations, we hypothesized that effects of temporal unpredictability in the stimulus structure would be potentiated during aversive stimulation, and thus result in a non-additive effect when compared to neutral stimuli or events of positive valence, i.e., happy facial expressions.

U

73 74

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

E

2

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

161 162 163 164 165 166 167 168 169

172 173 174 Q4 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191

195 196 197

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234

240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260

C

217 218

E

215 216

R

F

213 214

R

211 212

N C O

209 210

U

207 208

O

FMRI imaging parameters A 3 Tesla whole-body scanner (Siemens Verio) was used to acquire imaging data. Structural image acquisition consisted of 160 T1weighted sagittal images (MPRage, 1 mm slice thickness). For functional image acquisition, 655 volumes were collected with an axial T2*weighted single shot gradient echo planar sequence (30 slices, slice thickness 5 mm with 1 mm gap, repetition time (TR) 2100 ms, echo time (TE) 59 ms, flip angle 90°, field of view (FoV) 192 × 192 mm, 64 × 64 matrix). The first four volumes were discarded due to an incomplete steady state of magnetization. The phase encoding direction was anterior to posterior. Slices were aligned to AC–PC and rotated 25° in clock-wise direction in order to minimize susceptibility artifacts. Slices were acquired in descending order.

205 206

R O

238 239

204

P

Data acquisition and data analysis

202 203

FMRI data analysis Image processing and statistical analyses were performed using the SPM8 software package (update 3684, Wellcome Trust Centre for Neuroimaging at UCL, London, UK). Data preprocessing was performed according to standard procedures and consisted of realignment and unwarping (second order B-spline interpolation; fieldmap correction), slice-time correction, coregistration of functional data to each participant's anatomical image, normalization to the standard space of the Montreal Neurological Institute (MNI) brain, and spatial smoothing (Gaussian kernel: full width at half maximum of 9 mm).

In order to compare effects of constant and variable ISIs, as well as stimulus type, the six experimental conditions consisting of constant and variable ISI blocks, for blocks involving angry, happy, and neutral stimuli, along with the instruction phase, were modeled by durationdependent functions which were convolved with a canonical hemodynamic response function, and entered as regressors into the general linear model at the individual level of statistical analysis. The six movement parameters of the rigid body transformation obtained by the realignment procedure were introduced as covariates. Linear contrasts were calculated for the 6 experimental conditions of interest. For second level analysis, contrast images of the experimental conditions were entered into a random-effects full factorial analysis of variance (ANOVA). First, main effects of temporal structure, i.e., the contrasts [variable N constant ISIs] and [constant N variable ISIs], and stimulus type, i.e., the contrasts [FAC-angry N neutral], [FAC-happy N neutral], and [FAC-angry N FAC-happy], were assessed via family wise error (FWE) corrected whole brain analyses. Second, to test whether temporal structure, i.e., variable as compared to constant ISIs, differentially influenced the processing of angry emotional faces compared to neutral stimuli, interaction contrasts were calculated and reported; i.e., in more detail, we compared the conditions [variable N constant angry faces] to [variable N constant neutral stimuli] corresponding to the contrasts [1 − 1 − 1 1] and [− 1 1 1 − 1] on the conditions [variable-angry, constant-angry, variable-neutral, constant-neutral]. To account for whether the observed effects were specific to angry faces, identical contrasts were calculated between happy faces and neutral stimuli, as well as happy faces and angry faces. In case significant interaction effects within these contrasts were identified, we further described the exact nature of the interaction by extracting the beta values of the peak-voxels within each statistically significant region of interest for each experimental condition involved in the interaction, and compared them by post-hoc t-tests (Bonferroni-corrected for multiple comparisons). These post-hoc analyses allowed us to render the behavior of the interactive region within each experimental condition separately for the precise location of the interaction effect revealed by the SPM analysis, and was thus applied in substitution of subsequent t-tests within the SPM framework. For the interested reader, we have included subsequent t-tests on a whole brain threshold within the supplementary material (Tables S4 and S5). To control for task difficulty, we further analyzed the above mentioned model with accuracy as a covariate. As this model essentially yielded similar results, we report the interaction analyses in the supplementary material (Table S7). Beside characterizing differences in the processing of temporal structure between emotional and neutral stimuli, we were further interested in identifying possible common mechanisms, since e.g., a potentiation of the fear response to unpredictable angry faces might nevertheless reveal an increase in activation common to all stimulus categories. For this purpose, we performed conjunction analyses according to the minimal statistic/conjunction null method (Nichols et al., 2005). In particular, we compared the contrasts [variable vs. constant ISI]FAC-angry, to [variable vs. constant ISI]FAC-happy, and [variable vs. constant]control, in order to identify either a common increase or decrease in activation towards variable ISIs within the three stimulus categories. Conjunctions and interactions were all assessed as region of interest (ROI) analyses. All ROI analyses were performed using the Wake Forest University (WFU)-Pickatlas software (V2.4) (Maldjian et al., 2003, 2004; Tzourio-Mazoyer et al., 2002). The amygdala, the DLPFC (Brodmann areas 9 and 46), as well as the inferior and superior parietal cortices (IPL and SPL), and the SMA were defined as regions of interest, due to their relevance for the processing of temporal uncertainty and building of temporal expectations (Coull, 2004; Herry et al., 2007; Koppe et al., 2014; Vallesi et al., 2007a, 2007b, 2009). The corresponding masks were extracted from the AAL and Brodmann database implemented in the WFU-Pickatlas. Results are reported at a significance level of

D

237

200 201

T

235 236

time, was either predictable (constant ISI, duration always 4 s), or unpredictable (variable ISI, with durations in seconds drawn from a discrete uniform probability distribution on the values {2.5, 2.875, 3.25, 3.625, 4, 4.375, 4.75, 5.125, 5.5}, corresponding to a mean of 4 s, at a step width of .375 ms). The second experimental factor was stimulus type, manipulated by presenting stimuli which differed regarding social and emotional content: 1) faces with an angry emotional expression (FAC-angry), 2) faces with a happy emotional expression (FAC-happy), and 3) neutral geometric shapes (control). Subjects were asked to perform a choice reaction task that required a decision regarding the gender of the faces (FAC-angry, FAC-happy), or the specific shape (control). Preceding each task block, an instruction screen was displayed informing participants to either respond with ‘triangle’ and ‘square’ to an upcoming triangle or square, or ‘male’ and ‘female’ to an upcoming male or female face stimulus, depending on the stimuli presented within the following block. They were not informed that variations within the temporal structure of the stimulus sequence would occur, neither verbally, nor by any kind of cue. For further description of the tasks see supplementary material. In order to compare effects of constant and variable ISIs, a blocked design was applied, since an event-related analysis of the constant ISI condition would not allow a unique estimation of the BOLD response (Miezin et al., 2000), and would introduce differences in design efficiency depending on ISI condition (Burock et al., 1998; Dale, 1999). The three experimental conditions of stimulus type were thus presented within different blocks separated for the two conditions of temporal structure, resulting in a 2 × 3-blocked design. Each of these 6 experimental conditions was presented in 4 blocks with 9 trials per block. Task blocks were preceded by a resting period (duration: 15 s) followed by an instruction phase (duration: 6 s) which informed subjects on whether the next task required a gender or shape discrimination. Stimulus type order and the temporal structure conditions were pseudorandomized between subjects to control for task sequence effects. The tasks were presented using the Presentation software package (Neurobehavioral Systems, Albany, CA). The stimuli were presented on a computer screen located behind the MR scanner. The subjects saw the screen via a mirror mounted on top of the head coil located approximately 20 cm above their eyes.

E

198 199

3

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326

4

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

p b 0.05 FWE corrected. Additionally, to enable a direct comparison between obtained results with those collected in our previous study, we thresholded the above mentioned ROIs at p b .05 with activation obtained previously (see Koppe et al., 2014). Results are reported in the supplementary material (Table S6). Since temporal expectations rise as time proceeds (Nobre et al., 2007), and a previous study has shown that increased activation to variable ISIs is partly associated to the encoding of the cumulative conditional probability that an event will occur (Koppe et al., 2014), we were interested in examining whether all stimulus categories showed a similar encoding of temporal expectations induced by the passage of time. To this end, we performed an event-related analysis on the data of the variable ISI condition. Onsets of the individual tasks under variable timing were entered into this design, along with a parametric modulator reflecting the cumulative conditional probability of occurrence of each stimulus (for further detail, see supplementary material). The parametric modulations were then contrasted in a second-level mixed-effects model by means of one-sample t-tests, and ROI analyses were performed in analogy to the blocked design analysis.

variable] in the whole-brain or the ROI analyses corrected for FWE 387 (p b .05). 388

356 357

Behavioral data analysis Percentage of correct responses and reaction times (RT) were analyzed with IBM SPSS Statistics 20 (IBM Corp., 2011). For each subject and experimental condition, median RT was calculated and used in further analysis. According to the experimental design, performance and RT were analyzed separately with a 2 × 3-factorial ANOVA with the repeated-measurement factors ‘temporal structure’ (constant and variable ISI) and ‘stimulus type’ (FAC-angry, FAC-happy, control). Degrees of freedom were corrected according to Greenhouse–Geisser. A further description of the results was performed by splitting the ANOVA design into sub-designs and/or paired t-tests (Bonferroni-corrected for multiple testing).

358

Results

359

Behavioral data

360

380 381

All tasks were solved with a high accuracy (N90%, see Table S1, Figure S2). We observed an effect of ‘temporal structure’ on accuracy (F(1,29) = 4.77, p = 0.037, η2 = .141), such that performance was significantly lower during variable ISIs. Performance differed between tasks depending on stimulus type (main effect ‘stimulus type’: F(2,58) = 27.99, p b 0.001, η2 = .491). Post-hoc analysis revealed that the proportion of correct responses was significantly lower for the gender identification tasks than in the control condition presenting geometrical shapes (p b 0.001). Affective valence of the faces did not affect the percentage of correct responses. No interaction effect between ‘temporal structure’ and ‘stimulus type’ was observed (F(1,56) = 0.864, η2 = .005). Reaction time was influenced by stimulus type (main effect of ‘stimulus type’: F(1, 41) = 21.541; p b 0.001, η2 = .426, Table S1, Figure S2). Post-hoc analysis revealed that RT was longest for FAC-angry (FACangry with FAC-happy: p b .001) and generally longer during gender identification compared to control (FAC-happy with control: p = .006), resulting in an order of FAC-angry N FAC-happy N control. ‘Temporal structure’ did not affect RT (main effect of ‘temporal structure’: F(1,29) = 0.463; p = 0.502, η2 = .016, interaction effect between ‘temporal structure’ and ‘stimulus type’ F(1, 49) = 0.538; p = 0.561, η2 = .018).

382

Functional imaging data

383

Main effect of temporal structure No effects of temporal structure were found when averaging across all stimulus types, that is, no brain region showed a significant increase of activation within the contrasts [variable N constant] and [constant N

351 352 353 354 355

361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379

384 385 386

404 405

Effect of temporal structure on happy faces compared to neutral control stimuli. The interaction analysis of effects of temporal structure for the happy face condition (FAC-happy) and the control condition revealed similar results with the exception of a differential engagement of the left amygdala, as well as a disengagement of the superior parietal cortex (see Table 1, Figure S3).

443

O

F

Effect of temporal structure on angry faces compared to neutral control stimuli. The FWE-corrected whole-brain analysis of the interaction contrast did not reveal any significant activation. However, the ROI analysis revealed an interaction between temporal structure and stimulus type, i.e., threatening and emotionally neutral stimuli, within the right amygdala, the DLPFC (bilateral BA9, left BA46), as well as large parts of the parietal cortex and SMA over both hemispheres (see Table 1, Fig. 1). This interaction effect was however not due to a potentiated response to unpredictable angry faces as hypothesized, but alternatively, to rather similar activation intensities for constant and variable ISIs in angry faces, and a distinct difference in activation between variable and constant ISIs in the neutral stimuli. Post-hoc tests of the beta-values of the peak voxels within these regions showed no activation increase to variable ISIs in angry faces, in any of the investigated brain areas (see Fig. 2). Within the FAC-angry task, temporal structure modulated activation within the bilateral parietal lobes, which responded with a decrease of activation to variable ISIs (see Figs. 2a–d). Parts of the DLPFC showed a comparable effect as a trend (right BA9: p = .057, left BA46: p = .072, see Figs. 2g,j). The interaction was further strongly driven by a modulation of activation by temporal structure during the control task: activation increased during the latter in the transition from constant to variable ISIs within most of the investigated regions, namely the right amygdala, the bilateral DLPFC (BA9 and BA46), as well as the right parietal cortex, and the left SMA (see Figs. 2a,c,f–j). Interestingly, these opposing activation patterns resulted therein, that the direct comparison of the FAC-angry task (angry faces) to the control task (neutral shapes) produced different activation patterns, depending on whether the stimuli were contrasted at either constant or variable ISIs (see Fig. 3). In particular, when comparing the two tasks during constant ISIs, the FAC-angry task showed higher activation of the right amygdala and the DLPFC (left BA46, right BA9) than the control, as would be expected when comparing threatening to neutral stimuli. In contrast, during the variable ISI condition, activation within e.g., the right amygdala increased in the control task but not in FACangry, resulting in indistinguishable activation levels between the two tasks within this region. Overall, the control task induced higher activation within the left DLPFC (BA9), the right parietal cortex, and the left SMA, as compared to FAC-angry during variable ISIs.

R O

349 350

403

P

347 348 Q5

Interaction analyses of temporal structure and stimulus type

D

346

389 390

E

344 345

Main effect of stimulus type In comparison to neutral stimuli, both angry and happy faces enhanced activation within the right fusiform gyrus, as well as the bilateral inferior frontal gyri. Angry faces additionally engaged the left middle frontal gyrus comprising parts of the DLPFC, the left fusiform gyrus, the left medial superior frontal gyrus, the right inferior parietal lobe, and parts of the temporal and occipital cortices, when contrasted with neutral stimuli (see in more detail supplementary material, Table S3, Figure S4). In comparison to happy faces, angry faces led to increased activation within large parts of the prefrontal cortex such as the bilateral inferior frontal gyri and left middle frontal gyrus (comprising components of the DLPFC and orbitofrontal cortex), left superior temporal pole, and left insula (see Table S3, Figure S4).

T

342 343

C

340 341

E

338 339

R

336 337

R

334 335

O

333

C

331 332

N

329 330

U

327 328

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

391 392 393 394 395 396 397 398 399 400 401 402

406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 Q6 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442

444 445 446 447 448

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx t1:1 t1:2 t1:3

5

Table 1 Brain activation within the ROIs for the interaction effect of temporal structure × stimulus type, i.e., the contrasts [variable N constant neutral] N [variable N constant faces], for angry faces and shapes (left), or happy faces and shapes (right), p(FWE) b .05.

t1:4

Interaction: FAC-Angry × Control

Interaction: FAC-Happy × Control

ROI

H

x

y

z

k

T

pFWE

η2

x

y

Z

k

T

pFWE

η2

t1:6 t1:7 t1:8 t1:9 t1:10 t1:11 t1:12 t1:13 t1:14 t1:15 t1:16 t1:17

Amygdala

L R L R L R L R L R L R

– 24 −42 45 −39 24 −42 – −54 54 −3 3

– 2 −43 −34 −43 −61 47 – 5 8 2 2

– −11 55 52 55 58 10 – 34 40 52 52

– 2 44 88 25 35 5 – 9 37 43 31

– 2.78 3.86 3.86 3.68 3.93 3.43 – 3.71 4.96 3.91 3.90

– 0.032 0.009 0.006 0.014 0.007 0.015 – 0.015 0.000 0.005 0.007

– .250 .474 .476 .434 .492 .377 – .439 .765 .486 .484

−21 24 −51 36 – – −42 48 −54 51 −15 –

−4 −1 −43 −49 – – 38 20 5 14 −4 –

−11 −11 46 46 – – 13 28 31 31 61 –

3 9 12 29 – – 10 2 3 8 1 –

2.72 3.09 3.55 3.51 – – 4.08 3.13 3.56 3.91 3.18 –

0.035 0.015 0.024 0.016 – – 0.002 0.042 0.024 0.008 0.046 –

.241 .308 .401 .397 – – .521 .318 .408 .489 .326 –

465 466 467 468 469 470 471 472 473 474 475 476 477 478 479

O

R O

P

D

E

463 464

assess common effects of temporal structure among the individual stimulus categories. The analysis revealed no common activation for the gender identification tasks with the control task, that is, no common increase or decrease in activation to variable ISIs was observed between neutral and emotional stimuli. However, the comparison of effects of temporal structure on both gender identification tasks revealed a common decrease of activation in response to variable ISIs within the left inferior parietal lobe (x = −42, y = −28, z = 43, T = 3.29, p = .049). The conjunction analysis thus confirms a common response pattern to temporal structure for happy and angry faces, in agreement with the similar interaction results observed for both stimulus categories.

T

461 462

C

459 460

Analogous to the results observed for the threatening faces, the happy faces did not show increased activation in response to variable as compared to constant ISIs in any of the investigated areas. In contrast, a decrease in activation was observed within the left amygdala, left DLPFC (BA46), and left SMA. The interactions within the other areas were once more primarily driven by an increase of activation from constant to variable ISIs within the control task, as can be seen for the right amygdala, the right DLPFC (BA46 and BA9), and the right inferior parietal lobe, resulting in a similar up-regulation of activation within the control task. Once more, this interaction pattern primarily resulted in significant differences between FAC-happy and control task during the constant ISI condition (Fig. 3). In particular, during constant ISIs, the FAC-happy task enhanced activation in most ROIs compared to control, including the bilateral amygdala, and right DLPFC. Analogous to FAC-angry, this enhancement in activation against control receded during the variable ISI condition. Here, the control task showed increased activation within left DLPFC, the left IPL and SMA, and the right SPL (see Figure S3). Effect of temporal structure on angry compared to happy faces. The ROI analysis of the interaction of the two types of emotional stimuli with temporal structure revealed no significant effects, suggesting a similar modulation of the processing of angry and happy faces by temporal structure, in agreement with the similar interaction profiles for both tasks compared to neutral control.

E

457 458

SMA

R

455 456

dlPFC (BA9)

R

453 454

dlPFC (BA46)

Conjunction analysis for the effect of temporal structure between tasks The interaction analyses primarily aimed at characterizing differences in responsiveness to variable compared to constant ISIs in stimuli of either emotional or neutral content. Aside observing these differences, the distinct stimulus categories may yet have responded in a similar manner to unpredictability, e.g., by a common increase or decrease in activation to variable ISIs. We therefore calculated conjunctions to

N C O

451 452

SPL

480 481 482 483 484 485 486 487 488 489 490 491

The formation of temporal expectations depending on stimulus valence Further, we analyzed whether the formation of temporal expectations could be observed for all the applied stimuli. No variation with the cumulative hazard function was observed for either angry or happy facial stimuli. However, the neutral stimuli within the control task showed a modulation within the bilateral DLPFC (left BA46, bilateral BA9), the IPL as well as left SPL, and the bilateral SMA (see Table S2, Fig. 4), indicating that temporal expectations formed over time only play a role during the control task.

492 493

Discussion

501

The aim of the present study was to investigate whether unpredictability within the temporal stimulus structure affects behavior and cerebral activation during the processing of emotional faces, and whether this differs from effects of unpredictability during ‘pure’ cognitive processing, i.e., during the processing of neutral stimuli. We observed a general reduction in performance to all stimuli presented at an unpredictable rate. In terms of cerebral activation, varying the predictability of the temporal structure resulted in distinguishable effects for

502

U

449 450

IPL

F

t1:5

Fig. 1. Effects of temporal structure on angry (red) and happy (blue) faces compared to the neutral control task, i.e., in the contrasts [variable N constant neutral] N [variable N constant faces], in the separate ROIs, rendered on a whole brain. Both interaction contrasts are presented as transparent overlays, in order to demonstrate the similarities between emotions of positive and negative valence. Images display ROIs significant at p(FWE) b .05, on a more lenient threshold of p(FWE) b .1, for illustrative purposes.

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

494 495 496 497 498 499 500

503 504 505 506 507 508 509

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

E

D

P

R O

O

F

6

518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539

E

R

R

517

O

515 516

C

513 514

emotional facial expressions compared to non-emotional neutral shapes. We did not observe a particularly enhanced brain response to unpredictable threatening stimuli. Our data thus suggest that temporal unpredictability in the stimulus structure will have a different effect on emotional stimuli of either positive or negative valence than on neutral stimuli. When stimuli were presented at a temporally variable rate, tasks were solved with reduced accuracy as compared to when they were presented at a constant predictable rate. Our findings are in line with numerous studies which have shown that the possibility to precisely predict stimuli in time results in an optimization of behavior (see Niemi and Näätänen, 1981; Nobre et al., 2007 for reviews). The observed performance drop was comparable across all stimulus categories, i.e., not modulated by either the occurrence of an emotional face, or the valence of an emotional face. Since gender identification was slower for angry compared to happy facial expressions, in line with current literature (see Pessoa, 2009 for review), the lack in observing valencespecific effects of temporal structure on overt behavior, may not be attributed to a shortcoming in inducing valence-specific processing. Although the variation of emotional stimulus valence in the present study did not contribute to the understanding of the reduction in performance to variably presented stimuli, our results emphasize the relevance of variability in the temporal stimulus structure for performance as an unconscious by-product of cognitive processing based on the temporal properties of a perceptual input: subjects were not informed about this experimental variation and none reported to have noticed differences in timing between ISI conditions. In line with the behavioral results, temporal structure influenced brain activation during the presentation of all stimulus categories. However, we observed a distinction between effects on emotional and

N

511 512

U

510

C

T

Fig. 2. Contrast estimates and post-hoc t-test results of the maximum peak voxel within ROIs that show a differential effect of temporal structure for stimuli with negative emotional faces (black squares, straight lines) compared to the neutral control task (white circles, dotted lines). Error bars represent standard error of the mean after variability associated with between subject differences has been removed (Cousineau, 2005).

neutral stimuli. In contrast to our hypothesis, this distinction was not specific to aversive stimuli, since we found no differences in the response between angry and happy faces. In detail, when effects of temporal structure were contrasted for threatening and neutral stimuli (i.e., temporal structure × stimulus type), we observed differential brain responses within the right amygdala, the DLPFC, the superior and inferior parietal cortices, as well as the SMA. Essentially, an almost identical response pattern was observed for happy and neutral faces. These differences were primarily due to the fact that the application of variable or constant ISIs affected cerebral processing of angry or happy faces by down-regulating activation in the inferior and superior parietal lobes, and/or up-regulating activation within a variety of areas in the neutral control task. This downregulation in response to variable ISIs resulted in reduced activation for facial expressions compared to neutral control stimuli when comparing them in the variable ISI condition, whereas activation within parietal areas was comparable between emotional faces and neutral stimuli in the constant ISI condition. Within the right hemisphere, the interaction effect was not only caused by an activation decrease in response to temporally variable emotional faces, but additionally by an opposing increase towards the presentation of geometric shapes. Although statistically only observable as a trend, parts of the DLPFC displayed a similar antagonistic pattern. Apart from parietal areas, variations in the temporal structure had no effect on brain activation towards emotional faces. Thus, the observed interaction was mainly driven by an activation increase to variable ISIs within the neutral control task. We observed elevated activation levels in the amygdala, the DLPFC, and the SMA, consistent with results obtained in our previous study employing a similar task (Koppe et al., 2014) (see also similar findings with additional ROI

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569

7

C

T

E

D

P

R O

O

F

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596

R

573 574

analyses based on masks derived from this previous study, supplementary material, Table S6). Our data suggest analogous effects of temporal structure on happy and angry faces: both face conditions showed a decrease in parietal areas in response to variable timing, suggesting common mechanisms in the processing of temporal information. Also, no interaction effects were found when contrasting the face conditions to one another. One might ask whether our experimental approach failed to evoke differential cerebral activation for the processing of angry and happy faces in general. However, a comparison of both emotional conditions revealed slower processing times, and a stronger engagement of executive control areas such as the inferior and medial frontal gyri for angry faces (see Table S3, Figure S4), in line with current literature (see Pessoa, 2009 for review). Thus, the lack of observing a differential impact of temporal structure on these stimuli cannot be explained by a failure in engaging brain structures distinct in the processing of emotions of different valence. An explanation in regard to why unpredictability in the temporal stimulus structure increases activation during neutral, but not emotional stimuli, may relate to differences in the encoding of temporal expectations. When neutral stimuli were presented at variable ISIs, we observed expectation-related activation in the investigated brain areas. The BOLD response co-varied with the cumulative conditional probability that a stimulus would occur, given it had not yet occurred, i.e., a cumulative hazard function (Cui et al., 2009; Nobre et al., 2007), replicating recent findings (Koppe et al., 2014). Since ISIs were drawn from a uniform distribution, this nonlinear activation increase can at

N C O

571 572

U

570

R

E

Fig. 3. ROI activation, projected onto a whole-brain, for angry faces N control (top), and happy faces N control (bottom), during constant (red), and variable (yellow) ISI condition. Overlap between constant and variable timing is depicted in orange. Images display ROIs significant at p(FWE) b .05, on a more lenient threshold of p(FWE) b .1, for illustrative purposes. The image illustrates a more distinct difference between emotional and neutral stimuli during constant timing. Note that activation maps may slightly deviate from the post-hoc results obtained from the peak voxel of the interaction contrast.

least partially account for the stronger activation during variable ISIs. In contrast, no expectation-related activation was observed during emotional face processing, potentially accounting for a lack in observing increased activation towards variable ISIs. It may therefore be speculated as to whether emotional stimuli, particularly presented within an implicit emotion processing task, impede the formation of temporal expectations over time. Previous research suggests that the formation of temporal expectancy, at least in the context of temporal cueing tasks, relies on controlled processing (Correa et al., 2004, 2006). Tasks which increment demands on controlled processing result in smaller performance benefits to validly cued stimuli compared to less demanding tasks. The authors explain this finding by proposing that processing resources needed for the building of temporal expectations are withdrawn, and allocated to the more demanding cognitive task (Correa et al., 2006). Thus, tasks which increment demands on processing resources can be expected to interfere when the formation of temporal expectations relies on the monitoring of conditional probabilities over time, as is the case during variable ISIs. In contrast, temporal expectancies based on a predictable rhythm, such as during constant ISIs, are processed more automatically and may thus be independent of available processing resources (Los and van den Heuvel, 2001). By incrementing demands on processing resources, emotional stimuli may have attenuated the formation of temporal expectations during variable ISIs. In this context, it is important to note that on behalf of methodological considerations, we applied an implicit emotion processing task, i.e., subjects were asked to identify the gender, rather than the expressed

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 Q7 618 619 620 621 622 623

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

T

E

D

P

R O

O

F

8

631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652

R

R

O

629 630

C

627 628

N

626

emotion. Several studies suggest that emotion processing and the corresponding recruitment of cerebral structures depend on the specific task context (Critchley et al., 2000; Fusar-Poli et al., 2009; Habel et al., 2007; Lange et al., 2003; Monroe et al., 2013; Scheuerecker et al., 2007). When emotional information is task-irrelevant, facial stimuli constitute distractors to the cognitive task and result in reduced performance (Pessoa, 2009). Thus, the applied emotion processing task requires a high degree of processing resources since the displayed stimuli have a high ecological salience and the emotional content needs to be suppressed to successfully discern between genders (Iordan et al., 2013; Pessoa, 2009). It has further been demonstrated that implicit emotion processing tasks, such as gender labeling, increase activation within the amygdala (Critchley et al., 2000; Hariri et al., 2000; Lange et al., 2003; Lieberman et al., 2007), and decrease activation within prefrontal areas (e.g., Hariri et al., 2000; Lieberman et al., 2007), when compared to explicit emotion processing tasks such as emotion labeling. These observations have been attributed to the fact that the task-irrelevant presentation of emotions withdraws processing resources for other executive functions due to prioritized processing of emotional stimulus content or production of response conflict (Egner et al., 2008; Pessoa, 2009), while explicit emotion labeling, i.e., putting feelings into words, rather helps regulating emotional responses. As the present study further suggests that implicit emotion processing during temporal uncertainty reduces activation in parietal and prefrontal regions, possibly related to the withdrawal of processing resources for temporal information, it may be expected that differences between explicit and implicit emotion processing tasks are particularly enhanced during variable stimulus timing. In this case, implicit emotion processing tasks will more strongly

U

624 625

E

C

Fig. 4. Activation within the shape identification task which covaried with the cumulative conditional probability of stimulus occurrence. No analogous activation was observed in the gender identification tasks. The image displays ROIs significant at p(FWE) b .05, rendered on a whole-brain, and displayed at p(FWE) b .1 for illustrative purposes.

allocate processing resources from the building of temporal expectations during temporal uncertainty and thus reduce activation in dedicated areas, resulting in stronger activation differences between these tasks. Future studies which compare implicit and explicit emotion processing during both variable and constant stimulus timing will be needed to clarify this aspect. The idea that emotional distractors may impede the formation of temporal expectations fits well to a model describing the interaction between cognitive and affective processes, by the interaction between a ‘cold’ dorsal system, comprising DLPFC and lateral parietal cortex, and a ‘hot’ ventral system, associated with the amygdala and ventral parts of the PFC (see Iordan et al., 2013 for review). While the ‘cold’ system is typically associated to executive control, critical to the active maintenance of goal-relevant information in working memory, the ‘hot’ system is more concerned with emotional processing. When a cognitive task is combined with emotional distractors, both systems compete for processing resources. Possibly due to their salience, emotional distractors disrupt the executive control system by drawing on resources otherwise allocated for processing task-relevant information and concurrently decrease activation within the ‘cold’ system (Iordan et al., 2013). As the building of temporal expectations draws on the executive system (Correa et al., 2004, 2006), emotional distractors may disrupt their processing. Alternatively, and not mutually exclusive, emotional faces may constitute particularly highly salient stimuli when temporally unpredictable. If this holds true, temporal unpredictability may reinforce the decrease of the ‘cold’ system during processing of emotional stimuli, i.e., reinforce a decrease of activation of DLPFC and lateral parietal regions and accordingly a drop in performance, in agreement with our

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747

788

Our data confirm that temporal unpredictability differentially affects processing of neutral and social–emotional stimuli, with no difference between stimuli of aversive and positively valent content. The results implicate that heterogeneity between studies on cognition and social cognition in cognitive neuroscience might at least partially be due to a very basic but often overlooked feature of the experimental setting, i.e., the implicit temporal structure of the applied stimulus sequence; and that this design feature is of particular relevance when research aims at differentiating cognitive from social-cognitive functioning in mental disorders.

789

Appendix A. Supplementary data

798 Q8

O

F

Conclusion

R O

701 702

P

699 700

748 749

D

697 698

and neutral stimuli to temporal unpredictability. Thus, it is unlikely that our findings are solely attributable to the increased difficulty of the identification of gender in emotional faces, or to differences in positive or negative valence. One possible explanation as to why temporal unpredictability influences emotional faces to a different extent than neutral stimuli lies in the fact that they are more salient, and/or result in a higher arousal, and thereby withdraw resources needed for the processing of temporal structure. Future studies investigating the effects of unpredictable timing on salience, task difficulty, and specifically emotional arousal of social and non-social stimuli will be necessary to more precisely disentangle the contributing factors of the observed underlying effects. Understanding the interplay between socio-emotional stimulus content and timing may be particularly important in the context of mental disorders. The design of specific therapeutical interventions requires the assertion of whether dysfunctions are in fact related and limited to deficits in social cognition, or whether they affect cognitive processing in a more general way. Differential effects of temporal properties in social cognitive, and cognitive control tasks may cause misleading results which hamper the understanding of the disorder and the development of specific therapeutical interventions. This problem may be aggravated by the fact that timing processes are altered in some disorders such as Borderline Personality Disorder and Attention-Deficit/Hyperactivity Disorder (Berlin and Rolls, 2004; Hart et al., 2012; Noreika et al., 2013). As a result, our findings have implications for the setup of experimental designs and the selection of a temporal setting. We observed that the extent of activation difference between socio-emotional and neutral information depends on the temporal structure of the experimental design, particularly evident in the amygdala and the prefrontal cortex. The presentation of a temporally variable stimulus sequence, such as is typically applied in an event-related fMRI design, may lead to a considerable receding or complete disappearance of activation within these areas. The choice of the temporal structure may thereby contribute to heterogeneity in the neuroimaging literature. Further studies are necessary to deepen our understanding of differential effects of this basic experimental feature in cognitive and affective processing. Our findings emphasize the necessity of taking the temporal structure of the experimental design into consideration when aiming at differentiating cognitive from affective or social information processing in general, as well as in the context of dysfunctions in mental disorders.

E

695 696

T

693 694

C

691 692

E

689 690

R

688

R

686 687

N C O

684 685

findings. This implies that the behavioral effects of temporal structure in implicit emotion processing tasks are the consequence of different cognitive mechanisms than those affecting performance in neutral tasks. To sum up, our data suggest effects of temporal structure are probably modulated by emotional distraction, rather than fear or aversive stimulation. However some limitations need to be addressed. For one, we did not ask participants regarding their anxiety level after performance on the individual experimental conditions, nor did we test implicit effects on processing such as attentional shifts (see Herry et al., 2007). An association of unpredictability and fear could have been established, if for instance, individuals felt more anxious after processing unpredictable as compared to predictable emotional faces. However, the findings of Herry et al. (2007) suggest that subjects who are asked to judge the degree of unpleasantness after auditory stimulation at variable or constant ISIs, do not assess variable stimulation as more unpleasant, i.e., variable ISIs did not modulate the emotional state in this case. However, the performance observed within a dot-probe paradigm during the background presentation of variable tones resembled that of anxious individuals, supporting the idea of an emotional distraction as the process underlying the observed effects of variable ISIs. Also, it has to be held in mind that the present findings rely on emotional and neutral task conditions that differ not only in the applied stimuli, but also in the recruitment of different cognitive processes related to the identification of gender, or the identification of shapes. Any difference between the neutral and emotional conditions may have resulted from differences in visual information or task demands, and not from emotional arousal linked to social stimuli. Indeed, stimulus timing did not modulate brain activation within the two more similar gender identification tasks with angry and happy facial expressions, suggesting that the observed effects are not related to emotional, i.e., positive vs. negative, valence. It can therefore not be ruled out that stimulus predictability selectively interferes with cerebral resources specific to the task at hand, i.e., the identification of gender or the identification of shapes, rather than the actual neutral vs. emotional stimulus valence. It seems worth noting however, that most brain regions which were modulated by both temporal structure and neutral vs. emotional stimulus content, showed no differences between gender and shape identification under predictable timing, arguing against the interpretation that temporal structure selectively interferes with an activation of task specific cerebral resources. An alternative to control for neutral valence would have been to employ neutral facial expressions, as they also engage the amygdala to a lesser extent than emotional expressions (Carvajal et al., 2013; although see Garvert et al., 2014; Santos et al., 2011). However, employing neutral faces as control condition bears the problem that neutral faces are highly rare in everyday life and are thus often misinterpreted as indeed carrying an emotion (Carvajal et al., 2013), and cause physiological responses similar to angry expressions (Vrana and Gross, 2004). In mental disorders such as Borderline Personality Disorder, neutral faces even constitute a particularly strong type of distractor in implicit emotion recognition tasks (Krause-Utz et al., 2013), and are often misattributed in explicit emotion recognition tasks (Daros et al., 2013). Also, they do not allow a clear separation between social cognitive from cognitive processing. Further, as error rates and reaction times suggest that gender identification is more difficult than shape identification, one may speculate whether task difficulty can account for the present findings. However, based on the findings of a recent study (Koppe et al., 2014), it seems less probable that effects of temporal unpredictability were attenuated in the gender identification tasks due to a higher task difficulty. The latter study demonstrated that in more demanding cognitive tasks, as suggested by a drop in accuracy and prolonged reaction times, a more pronounced – rather than attenuated – effect of variable compared to constant ISIs is observed. Consistent with these results, controlling for task difficulty by including task performance as a covariate in our design, did not change the observed differential responding of emotional

U

682 683

9

750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787

790 791 792 793 794 795 796 797

Supplementary data to this article can be found online at http://dx. 799 doi.org/10.1016/j.neuroimage.2015.06.081. 800 References

801

Abott, B., 1985. Rats prefer signaled over unsignaled shock-free periods. J. Exp. Psychol. Anim. Behav. Process. 11, 215–223. Adolphs, R., Tranel, D., 1999. Preferences for visual stimuli following amygdala damage. J. Cogn. Neurosci. 11, 610–616. Annett, M., 1967. The binomial distribution of right, mixed and left handedness. Q. J. Exp. Psychol. 19, 327–333. Bar, M., Neta, M., 2008. The proactive brain: using rudimentary information to make predictive judgments. J. Consum. Behav. 7, 319–330.

802 803 804 805 806 807 808 809

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

D

P

R O

O

F

Krause-Utz, A., Elzinga, B.M., Oei, N.Y., Spinhoven, P., Bohus, M., Schmahl, C., 2013. Susceptibility to distraction by social cues in borderline personality disorder. Psychopathology 47, 148–157. Lange, K., Williams, L.M., Young, A.W., Bullmore, E.T., Brammer, M.J., Williams, S.C., Gray, J.A., Phillips, M.L., 2003. Task instructions modulate neural responses to fearful facial expressions. Biol. Psychiatry 53, 226–232. Lieberman, M.D., Eisenberger, N.I., Crockett, M.J., Tom, S.M., Pfeifer, J.H., Way, B.M., 2007. Putting feelings into words: affect labeling disrupts amygdala activity in response to affective stimuli. Psychol. Sci. 18, 421–428. Liu, T.T., Frank, L.R., 2004. Efficiency, power, and entropy in event-related FMRI with multiple trial types. Part I: theory. NeuroImage 21, 387–400. Liu, T.T., Frank, L.R., Wong, E.C., Buxton, R.B., 2001. Detection power, estimation efficiency, and predictability in event-related fMRI. NeuroImage 13, 759–773. Maldjian, J.A., Laurienti, P.J., Kraft, R.A., Burdette, J.H., 2003. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. NeuroImage 19, 1233–1239. Maldjian, J.A., Laurienti, P.J., Burdette, J.H., 2004. Precentral gyrus discrepancy in electronic versions of the Talairach atlas. NeuroImage 21, 450–455. McClure, S.M., Berns, G.S., Montague, P.R., 2003. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346. Meyer-Lindenberg, A., Hariri, A.R., Munoz, K.E., Mervis, C.B., Mattay, V.S., Morris, C.A., Berman, K.F., 2005. Neural correlates of genetically abnormal social cognition in Williams syndrome. Nat. Neurosci. 8, 991–993. Miezin, F.M., Maccotta, L., Ollinger, J.M., Petersen, S.E., Buckner, R.L., 2000. Characterizing the hemodynamic response: effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing. NeuroImage 11, 735–759. Monroe, J.F., Griffin, M., Pinkham, A., Loughead, J., Gur, R.C., Roberts, T.P., Christopher Edgar, J., 2013. The fusiform response to faces: explicit versus implicit processing of emotion. Hum. Brain Mapp. 34, 1–11. Nichols, T., Brett, M., Andersson, J., Wager, T., Poline, J.-B., 2005. Valid conjunction inference with the minimum statistic. NeuroImage 25, 653–660. Niemi, P., Näätänen, R., 1981. Foreperiod and simple reaction time. Psychol. Bull. 89, 133–162. Nobre, A.C., 2001. Orienting attention to instants in time. Neuropsychologia 39, 1317–1328. Nobre, A., Correa, A., Coull, J., 2007. The hazards of time. Curr. Opin. Neurobiol. 17, 465–470. Noreika, V., Falter, C.M., Rubia, K., 2013. Timing deficits in attention-deficit/hyperactivity disorder (ADHD): evidence from neurocognitive and neuroimaging studies. Neuropsychologia 51, 235–266. Orsini, C., Ventura, R., Lucchese, F., Puglisi-Allegra, S., Cabib, S., 2002. Predictable stress promotes place preference and low mesoaccumbens dopamine response. Physiol. Behav. 75, 135–141. Pessoa, L., 2009. How do emotion and motivation direct executive control? Trends Cogn. Sci. 13, 160–166. Phelps, E.A., LeDoux, J.E., 2005. Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron 48, 175–187. Price, K., Geer, J., 1972. Predictable and unpredictable aversive events: evidence for the safety-signal hypothesis. Psychon. Sci. 26, 215–216. Ryan, M., Martin, R., Denckla, M.B., Mostofsky, S.H., Mahone, E.M., 2010. Interstimulus jitter facilitates response control in children with ADHD. JINS 16, 388–393. Sakai, K., Hikosaka, O., Takino, R., Miyauchi, S., Nielsen, M., Tamada, T., 2000. What and when: parallel and convergent processing in motor control. J. Neurosci. 20, 2691–2700. Santos, A., Mier, D., Kirsch, P., Meyer-Lindenberg, A., 2011. Evidence for a general face salience signal in human amygdala. NeuroImage 54, 3111–3116. Scheuerecker, J., Frodl, T., Koutsouleris, N., Zetzsche, T., Wiesmann, M., Kleemann, A.M., Bruckmann, H., Schmitt, G., Moller, H.J., Meisenzahl, E.M., 2007. Cerebral differences in explicit and implicit emotional processing—an fMRI study. Neuropsychobiology 56, 32–39. Seligman, M.E., 1968. Chronic fear produced by unpredictable electric shock. J. Comp. Physiol. Psychol. 66, 402–411. Seligman, M., Binik, Y., 1977. The safety signal hypothesis. In: Davis, H., Hurwiwtz, H. (Eds.), Operant Pavlovian Interactions. Erlbaum, Hillsdale, NJ, pp. 165–188. Seligman, M.E., Meyer, B., 1970. Chronic fear and ulcers in rats as a function of the unpredictability of safety. J. Comp. Physiol. Psychol. 73, 202–207. Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., Joliot, M., 2002. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage 15, 273–289. Vaidyanathan, U., Patrick, C.J., Cuthbert, B.N., 2009. Linking dimensional models of internalizing psychopathology to neurobiological systems: affect-modulated startle as an indicator of fear and distress disorders and affiliated traits. Psychol. Bull. 135, 909–942. Vallesi, A., Mussoni, A., Mondani, M., Budai, R., Skrap, M., Shallice, T., 2007a. The neural basis of temporal preparation: insights from brain tumor patients. Neuropsychologia 45, 2755–2763. Vallesi, A., Shallice, T., Walsh, V., 2007b. Role of the prefrontal cortex in the foreperiod effect: TMS evidence for dual mechanisms in temporal preparation. Cereb. Cortex 17, 466–474. Vallesi, A., McIntosh, A.R., Shallice, T., Stuss, D.T., 2009. When time shapes behavior: fMRI evidence of brain correlates of temporal monitoring. J. Cogn. Neurosci. 21, 1116–1126. Vrana, S.R., Gross, D., 2004. Reactions to facial expressions: effects of social context and speech anxiety on responses to neutral, anger, and joy expressions. Biol. Psychol. 66, 63–78. Whalen, P.J., 2007. The uncertainty of it all. Trends Cogn. Sci. 11, 499–500. Wodka, E.L., Simmonds, D.J., Mahone, E.M., Mostofsky, S.H., 2009. Moderate variability in stimulus presentation improves motor response control. J. Clin. Exp. Neuropsychol. 31, 483–488.

N

C

O

R

R

E

C

T

Bassett, J.R., Cairncross, K.D., King, M.G., 1973. Parameters of novelty, shock predictability and response contingency in corticosterone release in the rat. Physiol. Behav. 10, 901–907. Berlin, H.A., Rolls, E.T., 2004. Time perception, impulsivity, emotionality, and personality in self-harming borderline personality disorder patients. J. Personal. Disord. 18, 358–378. Boll, S., Gamer, M., Kalisch, R., Buchel, C., 2011. Processing of facial expressions and their significance for the observer in subregions of the human amygdala. NeuroImage 56, 299–306. Burock, M.A., Buckner, R.L., Woldorff, M.G., Rosen, B.R., Dale, A.M., 1998. Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport 9, 3735–3739. Carvajal, F., Rubio, S., Serrano, J.M., Rios-Lago, M., Alvarez-Linera, J., Pacheco, L., Martin, P., 2013. Is a neutral expression also a neutral stimulus? A study with functional magnetic resonance. Exp. Brain Res. 228, 467–479. Correa, A., Lupianez, J., Milliken, B., Tudela, P., 2004. Endogenous temporal orienting of attention in detection and discrimination tasks. Percept. Psychophys. 66, 264–278. Correa, A., Lupianez, J., Tudela, P., 2006. The attentional mechanism of temporal orienting: determinants and attributes. Exp. Brain Res. 169, 58–68. Cotti, J., Rohenkohl, G., Stokes, M., Nobre, A.C., Coull, J.T., 2011. Functionally dissociating temporal and motor components of response preparation in left intraparietal sulcus. NeuroImage 54, 1221–1230. Coull, J.T., 2004. fMRI studies of temporal attention: allocating attention within, or towards, time. Brain Res. Cogn. Brain Res. 21, 216–226. Coull, J., Nobre, A., 2008. Dissociating explicit timing from temporal expectation with fMRI. Curr. Opin. Neurobiol. 18, 137–144. Coull, J.T., Cheng, R.-K., Meck, W.H., 2011. Neuroanatomical and neurochemical substrates of timing. Neuropsychopharmacology 36, 3–25. Cousineau, D., 2005. Confidence intervals in within‐subject designs: a simpler solution to Loftus and Masson's method. Tutor. Quant. Methods Psychol. 1, 42–45. Critchley, H., Daly, E., Phillips, M., Brammer, M., Bullmore, E., Williams, S., Van Amelsvoort, T., Robertson, D., David, A., Murphy, D., 2000. Explicit and implicit neural mechanisms for processing of social information from facial expressions: a functional magnetic resonance imaging study. Hum. Brain Mapp. 9, 93–105. Cui, X., Stetson, C., Montague, P.R., Eagleman, D.M., 2009. Ready … go: amplitude of the FMRI signal encodes expectation of cue arrival time. PLoS Biol. 7, e1000167. Dale, A.M., 1999. Optimal experimental design for event-related fMRI. Hum. Brain Mapp. 8, 109–114. Daros, A.R., Zakzanis, K.K., Ruocco, A.C., 2013. Facial emotion recognition in borderline personality disorder. Psychol. Med. 43, 1953–1963. Domes, G., Schulze, L., Herpertz, S.C., 2009. Emotion recognition in borderline personality disorder—a review of the literature. J. Personal. Disord. 23, 6–19. Egner, T., Etkin, A., Gale, S., Hirsch, J., 2008. Dissociable neural systems resolve conflict from emotional versus nonemotional distracters. Cereb. Cortex 18, 1475–1484. Evans, K.C., Wright, C.I., Wedig, M.M., Gold, A.L., Pollack, M.H., Rauch, S.L., 2008. A functional MRI study of amygdala responses to angry schematic faces in social anxiety disorder. Depress Anxiety 25, 496–505. Fanselow, M.S., 1980. Signaled shock-free periods and preference for signaled shock. J. Exp. Psychol. Anim. Behav. Process. 6, 65–80. Friston, K.J., Zarahn, E., Josephs, O., Henson, R.N., Dale, A.M., 1999. Stochastic designs in event-related fMRI. NeuroImage 10, 607–619. Fusar-Poli, P., Placentino, A., Carletti, F., Allen, P., Landi, P., Abbamonte, M., Barale, F., Perez, J., McGuire, P., Politi, P.L., 2009. Laterality effect on emotional faces processing: ALE meta-analysis of evidence. Neurosci. Lett. 452, 262–267. Garvert, M.M., Friston, K.J., Dolan, R.J., Garrido, M.I., 2014. Subcortical amygdala pathways enable rapid face processing. NeuroImage 102 (Part 2), 309–316. Grillon, C., 2008. Models and mechanisms of anxiety: evidence from startle studies. J. Psychopharmacol. 199, 421–437. Grillon, C., Baas, J.P., Lissek, S., Smith, K., Milstein, J., 2004. Anxious responses to predictable and unpredictable aversive events. Behav. Neurosci. 118, 916–924. Guile, M., 1987. Differential gastric ulceration in rats receiving shocks on either fixed-time or variable-time schedules. Behav. Neurosci. 101, 139–140. Habel, U., Windischberger, C., Derntl, B., Robinson, S., Kryspin-Exner, I., Gur, R.C., Moser, E., 2007. Amygdala activation and facial expressions: explicit emotion discrimination versus implicit emotion processing. Neuropsychologia 45, 2369–2377. Hariri, A.R., Bookheimer, S.Y., Mazziotta, J.C., 2000. Modulating emotional responses: effects of a neocortical network on the limbic system. Neuroreport 11, 43–48. Hariri, A.R., Tessitore, A., Mattay, V.S., Fera, F., Weinberger, D.R., 2002. The amygdala response to emotional stimuli: a comparison of faces and scenes. NeuroImage 17, 317–323. Hariri, A.R., Mattay, V.S., Tessitore, A., Fera, F., Weinberger, D.R., 2003. Neocortical modulation of the amygdala response to fearful stimuli. Biol. Psychiatry 53, 494–501. Hart, H., Radua, J., Mataix-Cols, D., Rubia, K., 2012. Meta-analysis of fMRI studies of timing in attention-deficit hyperactivity disorder (ADHD). Neurosci. Biobehav. Rev. 36, 2248–2256. Herry, C., Bach, D.R., Esposito, F., Di Salle, F., Perrig, W.J., Scheffler, K., Luthi, A., Seifritz, E., 2007. Processing of temporal unpredictability in human and animal amygdala. J. Neurosci. 27, 5958–5966. Imada, H., Nageishi, Y., 1982. The concept of uncertainty in animal experiments using aversive stimulation. Psychol. Bull. 91, 573–588. Iordan, A.D., Dolcos, S., Dolcos, F., 2013. Neural signatures of the response to emotional distraction: a review of evidence from brain imaging investigations. Front. Hum. Neurosci. 7, 200. Katz, R., Wykes, T., 1985. The psychological difference between temporally predictable and unpredictable stressful events: evidence for information control theories. J. Pers. Soc. Psychol. 48, 781–790. Koppe, G., Gruppe, H., Sammer, G., Gallhofer, B., Kirsch, P., Lis, S., 2014. Temporal unpredictability of a stimulus sequence affects brain activation differently depending on cognitive task demands. NeuroImage 101c, 236–244.

U

810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895

G. Koppe et al. / NeuroImage xxx (2015) xxx–xxx

E

10

Please cite this article as: Koppe, G., et al., Temporal unpredictability of a stimulus sequence and the processing of neutral and emotional stimuli, NeuroImage (2015), http://dx.doi.org/10.1016/j.neuroimage.2015.06.081

896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981