Brain and Cognition 91 (2014) 113–122

Contents lists available at ScienceDirect

Brain and Cognition journal homepage: www.elsevier.com/locate/b&c

ERP correlates of script chronology violations Kris Baetens ⇑, Laurens Van der Cruyssen, Marie Vandekerckhove, Frank Van Overwalle ⇑ Department of Psychology and Educational Sciences, Vrije Universiteit Brussel, Brussels, Belgium

a r t i c l e

i n f o

Article history: Accepted 19 September 2014

Keywords: ERP N400 LAN Script

a b s t r a c t Research indicates a distinction between the processing of script content (which events, behaviors, scenes. . . are part of it) and script chronology (what is their usual order of occurrence). Using sequences of two line drawings depicting everyday social script events, we examined the event related potential (ERP) correlates of script chronology violations (i.e., wrong order). An increased left anterior negativity (LAN) following chronology violations suggests similarities between the processing of script chronology in visually observed human behavior and verbal syntax. Consequently, this study extends previous findings suggesting that the LAN is sensitive to structure violation across domains (e.g., verbal syntax, abstract structure), including that of meaningful human actions. Ó 2014 Elsevier Inc. All rights reserved.

1. Introduction Scripts are defined as stereotyped action plans (Read, 1987) which help us to interpret, predict and understand the behavior of others in our complex social world. As the most classical example, the restaurant script entails the sequence of ‘‘taking a seat’’, ‘‘waiting for the menu’’, ‘‘ordering what to eat’’ and so on. Scripts are composed of events and actions with an overarching individual or collective goal (Read, 1987). The comprising events or actions each have a subgoal that contributes to the overall finality of the script. Although scripts and actions share similarities, there are important differences. The goals that define scripts are of a more abstract, conceptual nature than action goals, which are more grounded in concrete motor representations (Vallacher & Wegner, 1987; see also Spunt, Satpute, & Lieberman, 2011). Abelson (1981) proposed that scripts can contain at least two distinct types of information. On the one hand, they consist of knowledge about which behaviors, events, actors and roles occur in a given context (script content). On the other hand, scripts delineate a more or less fixed chronological order in which these events and behaviors normally proceed (script chronology). Behavioral evidence supports the notion that both types of script information enable anticipation of upcoming behaviors in a given context. Regarding the representation of script content in the verbal domain, words denoting script events facilitate the processing of other events belonging to the same script both in ⇑ Corresponding authors at: Department of Psychology, Vrije Universiteit Brussel, Pleinlaan 2, B-1050 Brussel, Belgium. E-mail addresses: [email protected] (K. Baetens), Frank.VanOverwalle@ vub.ac.be (F. Van Overwalle). http://dx.doi.org/10.1016/j.bandc.2014.09.005 0278-2626/Ó 2014 Elsevier Inc. All rights reserved.

relatedness judgment tasks and lexical decision paradigms. This occurs even when excluding normatively associated words (e.g., honey – bee), or words belonging to a mutual category (e.g., honey – sugar). Chwilla and Kolk (2005) demonstrated this using script triplets without associative or obvious semantic relation among individual elements, e.g. director – dismissal – bribe, while an earlier study by Moss, Ostrin, Tyler, and Marslen-Wilson (1995) used normatively unassociated word pairs, e.g., restaurant – wine. The influence of script knowledge can transcend that of simple associative relations between individual events or concepts: a recent study demonstrated that lexical decisions regarding a target event are facilitated when the preceding prime events were presented together, but the same prime events had no facilitating effect when presented on their own (e.g., ‘‘marinate’’ and ‘‘grill’’ as primes for ‘‘chew’’, Khalkhali, Wammes, & McRae, 2012). More fine-grained investigations have disentangled the role of different kinds of script information, demonstrating that event nouns prime actors and objects commonly involved in them; locations prime actors and objects commonly found at those locations and instrument nouns prime objects on which those instruments are commonly used, but not the people who typically use them (Hare, Jones, Thomson, Kelly, & McRae, 2009). Such findings are difficult to reconcile with spreading activation accounts of semantic memory in their simplest form, apparently implying the existence of qualitatively differentiated types of association. Behavioral research has documented the encoding of script chronology in semantic memory as well. As an early indication, even after memorizing an incorrectly ordered list, participants tended to reproduce script events in their canonical order (Bower, Black, & Turner, 1979). More recently, several reaction

114

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

time studies have documented effects of temporal directionality in general event knowledge (Khalkhali et al., 2012; Nuthmann & van der Meer, 2005; Van der Meer, Beyer, Heinze, & Badel, 2002). For example, relatedness judgments regarding script word triplets are faster when the triplets are presented in the correct rather than reversed order (e.g., wake up – shower – brush versus shower – wake up – brush; Khalkhali et al., 2012). In the action domain, evidence for great sensitivity to temporal order comes from a study in which participants judged which phase a photographed athletic action belonged to (approach versus flight of a high jumper; Güldenpenning, Koester, Kunde, Weigelt, & Schack, 2011). The target pictures were subliminally preceded by a picture depicting an earlier or later phase of the same action. Athletes (experts) were significantly faster in classifying the targets when the prime-target pairs constituted a chronologically correct rather than reversed sequence, even within movement phases (i.e., approach/flight). A similar effect occurred in novices only when prime and target were taken from a different phase of the jump (approach versus flight), supposedly as a consequence of their less fine-grained temporal knowledge about these actions. Although an extensive event related potential (ERP) literature documents how script content knowledge shapes our understanding of the world (for a review, see Sitnikova, Perrone, Goff, & Kuperberg, 2010), research on script chronology information is rather scarce. In fact, to the best of our knowledge, there is no study to date on the ERP correlates of script chronology violations. Such an investigation could be of importance in addressing at least two questions. One key issue is whether or not scripts are unitary knowledge structures, storing information about both content and chronology. More central for the present study is the question to which extent the processing of chronological script information concerning actions is similar to the processing of other types of sequential information, language in particular, or rather appeals to a separate, specialized network. Before turning to the hypotheses of the present study, we will briefly review the existing ERP literature on script information processing. 1.1. ERPs and script content: contextual mismatch and the N400 Previous ERP studies have addressed the impact of script content violations. A recurrent finding is that elements or behaviors that do not belong to a previously primed script elicit larger N400 amplitudes than elements or behaviors that do fit the script. The N400 is a negative-going deflection peaking at about 400 ms after stimulus onset. There are at least two functional interpretations of the N400 effect (Lau, Almeida, Hines, & Poeppel, 2009; Lau, Phillips, & Poeppel, 2008). According to the lexical view, the N400 indexes the very process of lexical access (Kutas & Federmeier, 2000), that is, the activation of features of the longterm memory representation associated with a single lexical item. The integration view, on the other hand, proposes that the N400 reflects a combinatorial process of semantic integration of lexical items with the local and global linguistic working context (Hagoort, 2008). Hybrid hypotheses have also been formulated, suggesting that the N400 reflects a conglomeration of several distinct sub-processes (Pylkkänen & Marantz, 2003). Increased N400 amplitudes in response to script content anomalies have been documented in studies using verbal descriptions of behavior (Chwilla & Kolk, 2005) as well as studies using graphic stimulus material (using picture sequences, e.g. showing a man taking a cutting board – a man taking a loaf of bread – a man cutting versus ironing the bread, Sitnikova, Holcomb, Kiyonaga, & Kuperberg, 2008; within pictures, e.g., woman cutting bread with a saw, Proverbio & Riva, 2009; for a review, see Sitnikova et al., 2010). Apparently, the activation of script content information is a broad and rapid process: even information that is anomalous with

the local linguistic context is activated (Metusalem et al., 2012). Consider the following passage: ‘‘A huge blizzard ripped through town last night. My kids ended up getting the day off from school. They spent the whole day outside building a big snowman/jacket/ towel.’’ While both ‘‘jacket’’ and ‘‘towel’’ are semantically anomalous sentence endings, relatively smaller N400 amplitudes were observed in response to ‘‘jacket’’, supposedly reflecting the activation of script content. 1.2. ERPs and structure violations: the LAN Violations of structure in sequences of meaningful stimuli have classically been associated with left anterior negativities (LANs) in the ERP. As such, the LAN has been observed in response to a wide range of violations of syntactic structure (e.g., word order, for a review, see Friederici, 2002). However, the LAN is not only sensitive to the structure of verbal sequences. Cohn et al. (2012) recently demonstrated sensitivity of the LAN to a syntax-like structure in sequences of meaningful non-verbal stimuli (sequential cartoon images). According to these authors, this syntax-like structure entails a narrative architecture, which typically begins by establishing or introducing the characters and context. Next, an event is initiated and then culminates in a peak or climax. Finally, the event is wrapped up at the end of the sequence. Cohn et al. (2012) found increased LANs to image sequences lacking such narrative structure, even when the pictures were thematically unrelated. This is somewhat analogous to increased LANs in response to grammatical errors in nonsense sentences (Hahne & Jescheniak, 2001). While script content provides the basic building blocks of a script, script chronology structures them into a meaningful whole. As such, script chronology violations may elicit increased LANs. However, ERP research involving sequences of meaningful behaviors (as opposed to abstract, e.g., geometric stimuli) is quite scarce (Koester & Prinz, 2007). Nevertheless, some support for this hypothesis can be found in existing neurolinguistic studies on chronological inconsistencies. Baggio (2008) presented sentences containing chronology violations entailing a (mis)match between the tense of the verb and a temporal anchoring expression (e.g., ‘‘Last Sunday, Vincent painted/paints his house’’). He observed increased LANs in response to such anachronisms, setting in between 200 and 300 ms after the onset of the critical word. Similarly, increased LAN amplitudes have been observed when participants were confronted with linguistic bypasses of the ‘‘default’’ order implied by the iconicity assumption (the assumption that the order in which events are presented in language or stories is the order in which they occurred; Hopper, 1979). For example, whereas a sentence of the form ‘‘After she X, she Y’’ is congruent with the iconicity assumption, the reverse implied order ‘‘Before she X, she Y’’ is incongruent with the iconic X–Y order. Münte et al. (1998) found that otherwise identical sentences beginning with the non-iconic ‘‘before’’ elicited an increased LAN compared to sentences beginning with the iconic ‘‘after’’. Importantly, this effect emerged as early as 300 ms after the onset of the first word of the sentence – that is, before all relevant temporal information in the sentence had been processed. This suggests that the retrieval of temporal-conceptual knowledge is part of word comprehension almost immediately. In line with previous work (Kluender & Kutas, 1993), the authors proposed that this LAN effect reflects working memory processes, supported by the finding that its magnitude correlated with individual differences in working memory span. In abstract sequences, structural information symbols, enabling prediction of upcoming events when combined with previous elements, elicit a similar LAN (Hoen & Dominey, 2000). This suggests the LAN may be associated with a general neurocomputational function engaged by both linguistic and non-linguistic sequential structures.

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

In sum, prior research on verbal and non-verbal material demonstrates that LANs are typically elicited by violations of well-defined structures and linguistic chronology. Therefore, we might expect that violations of script chronology will also elicit increased LANs. 1.3. Script chronology representations The prefrontal cortex (PFC) plays a crucial role in the sequential organization of behavior, speech and reasoning (for a review, see Fuster, 2001). The involvement of this region in the processing of script chronology has been demonstrated by many lesion and imaging studies (Cosentino, Chute, Libon, Moore, & Grossman, 2006; Knutson, Wood, & Grafman, 2004; Krueger, Moll, Zahn, Heinecke, & Grafman, 2007; Kuchinke, Van Der Meer, & Krueger, 2009; Ruby, Sirigu, & Decety, 2002; Sirigu, Cohen, & Zalla, 1998; Sirigu et al., 1995; Zalla, Phipps, & Grafman, 2002; Zanini, 2008). Some authors have suggested that the PFC is involved in the representation of a single knowledge structure, encompassing both script content and chronology (for a detailed discussion, see Wood & Grafman, 2003). Support for this proposition comes from singlecell recordings in monkeys (Shima, Isoda, Mushiake, & Tanji, 2007) or the demonstration of left PFC involvement in script content judgments even when no script chronology processing is required (e.g., Wood, Romero, Makale, & Grafman, 2003). In contrast, other studies indicate a dissociation of script content and chronology. For example, patients with lesions to the PFC were able to successfully generate events belonging to a given script, but were impaired in arranging given script elements in their correct temporal order (Zanini, 2008) or prioritize among the generated events (Sirigu et al., 1995). Further, it is unclear whether activation of the PFC in script chronology processing reflects a generalized sequence processing function, or rather a specialized system, restricted to the action domain. Impairments in prefrontal patients have been found not only to affect action sequences, but also sequences of natural events (Zanini, 2008) or lists of historical facts (Shimamura, Janowsky, & Squire, 1990). In addition, the left frontal cortex has been found to be involved in judging structure and content of meaningful sentences as well as abstract sequences (Hoen, Pachot-Clouard, Segebarth, & Dominey, 2006). However, the existence of impairments specific to certain types of sequences is difficult to reconcile with the idea of a general sequence processing function (Zanini, 2008). As a striking example, Sirigu et al. (1998) documented that agrammatic patients with lesions in the left ventrolateral PFC could order word groups to form a logical sequence of actions, but were impaired in ordering similar word groups in a syntactically sound sentence. Patients with lesions to the left dorsolateral PFC showed the inverse pattern. In sum, it is not entirely clear whether the neural substrates of script content and chronology are dissociable. Further, it remains a matter of debate whether the involvement of the left PFC in script chronology processing reflects a domain-specific process or rather a general sequencing capacity. 1.4. The present research As outlined above, chronology violations may be associated with LAN modulations. In the present study, we want to directly test this prediction. If confirmed, it may help to characterize the nature of chronological script information, and strengthen the view that the LAN may index a general neurocomputational function engaged by both linguistic and non-linguistic sequential structures (Hoen & Dominey, 2000). Most relevant studies regarding temporal structure (Baggio, 2008; Münte, Schiltz, & Kutas, 1998) have used stimulus material

115

containing morphosyntactic markers (e.g., painted signaling an event in the past) or structure words (e.g., before) encoding structural information about the sequences. In the present study, we avoid this concurrence of structure and content by using sequences of two line drawings depicting human actions (Fig. 1). These were either presented in a chronological or counter-chronological order (i.e., chronology violation). Participants judged whether the second picture constituted a logical continuation of the sequence of events. Importantly, based on the hypothesis that script chronology processing is part of a domain-general sequencing capacity also used in language, we expect LANs due to the violation of the logical chronological structure. In addition, we also predict increased N400 amplitudes following chronology violations relative to consistent sequences. This prediction is based on reaction time research suggesting that script events serve as asymmetric primes: presentation of a given script event facilitates the processing of events that follow it in the script more strongly than events that typically precede it in the script (van der Meer et al., 2002). 2. Material and methods 2.1. Participants 23 right-handed students at the Vrije Universiteit Brussel participated in exchange for course credit. 10 men and 13 women participated, aged between 18 and 23 (M = 21), without a history of neurological dysfunction and with normal or corrected to normal vision. One female participant was excluded from further analysis due to a large number of movement-related artifacts. 2.2. Stimulus material The black-and-white line drawings used in this experiment were borrowed from Ruby et al. (2002), and additional material was drawn by the same artist (Fig. 1). The original sets consisted of three pictures depicting everyday human behavior, with one clear correct chronological order. Like in the study by Ruby et al. (2002), scripts differed with respect to total time between events and the number of actors and objects involved. A total of 71 sets was employed. We selected 60 scripts from the total of 71 sets. From each of these sets of three pictures, we selected two with a salient correct chronological order at face value. By inverting the order, we obtained chronological violations. 2.3. Procedure Participants sat down in a comfortable chair, received information about the procedure, and signed the informed consent, approved by the medical ethical committee of the Vrije Universiteit Brussel. The stimuli were presented on a TFT-screen at eye level with a refresh rate of 40 Hz at a distance of 50 cm, using E-prime (Psychology Software Tools, Incorporated). Each picture subtended 20.4° visual angle vertically and 12.6° visual angle horizontally. Each trial consisted of the presentation of a blink pause (1.5 s), a fixation cross signaling the start of a new trial (1 s), the first picture of a pair (3 s), the second (3 s), and the question screen (until response). After each block of 40 trials, participant received a 30 s break. The participants’ task was to judge whether the second picture depicted a logical continuation of the behavior expressed in the first (using two keys operated by two fingers of the right hand, Cohn, Paczynski, Jackendoff, Holcomb, & Kuperberg, 2012). To exclude the influence of response preparation from the ERPs, we

116

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

Fig. 1. Example sequences in correct order.

randomly mapped the ‘‘yes’’ and ‘‘no’’ answers to the keys for each trial. This way, participants could only start preparing the right response as soon as the question screen had appeared. To obtain a sufficient number of trials, each of the selected scripts was presented twice during the course of the experiment, randomly assigned to two of totally three experimental conditions: pictures were presented in the correct order (consistent), the

inverse order (chronology violation), or the first picture was followed by an unrelated one (content violation; the results of this condition are not central to the present study and therefore only reported in the Supplementary materials), yielding a total of 40 trials per condition. The allocation of the scripts to the conditions was random across participants, so that any given script was presented equally often in each condition.

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

117

Fig. 1 (continued)

2.4. Electrophysiological registration and analysis The EEG and EOG were continuously recorded during the experiment at a digitizing rate of 256 Hz, using an online low-pass anti-aliasing FIR filter with a cutoff frequency of 69 Hz. The EEG montage included 30 scalp sites following the international 10–20 electrode system. We used sintered AgCl electrodes in a

Waveguard cap from Advanced Neuro Technology (ANT). The montage included six midline sites (Fpz, Fz, Cz, Pz, Poz, Oz) and twelve sites over each hemisphere (Fp1/Fp2, F3/F4, F7/F8, FC1/FC2, FC5/ FC6, C3/C4, T7/T8, CP1/CP2, CP5/CP6, P3/P4, P7/P8, O1/O2), with the average of all EEG-channels serving as recording reference. Data were re-referenced to the algebraic mean of two mastoid electrodes. The ground electrode (AFz) was located between Fz

118

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

and Cz. EOGs were recorded by means of electrodes placed above and below the right eye and 1 cm external to the outer canthus of each eye, respectively. Impedance was kept below 10 kX for each electrode. EEG was recorded with hardware (Cognitrace) and software (Eemagine) developed by ANT. All further ERP data analysis was carried out using the open-software toolbox EEGLAB (Delorme & Makeig, 2004). An offline 0.1- to 30-Hz 2nd order Butterworth band pass was applied. We used the infomax algorithm as implemented in EEGLAB (Delorme & Makeig, 2004) to blindly decompose the data of each subject from all EEG-channels as well as both bipolar EOG channels into maximally independent components, using default parameters (maximum 512 learning steps, stop training if weight change .05). Although we collected delayed responses to avoid motor contamination (see Section 2.3 above), significant differences between the mean reaction times

on correct trials were nevertheless observed. More specifically, participants responded slower in the chronology violation condition (M = 1150 ms) than in the consistent condition (M = 1042 ms), t(21) = 4.84, p < .001. 3.2. ERPs As expected, we found significant differences in the mean amplitude between conditions (Fig. 2), consistent with the prediction of a LAN and N400 effect. Starting from about 250 ms post stimulus, amplitudes following chronology violations started to deflect negatively relative to consistent sequences, peaking at about 285 ms (i.e., LAN), followed by a second, larger negativity, starting from about 350 ms and extending until about 600 ms (N400). 3.2.1. LAN window (250–350 ms) 3.2.1.1. Lateral electrodes. Apart from a main effect of Chronology, F(1, 21) = 15.12, p < .01, the ANOVA for the 250–350 ms window revealed a significant three-way interaction between Chronology, Position and Hemisphere, F(1, 21) = 6.33, p < .05. This interaction reflected the left anterior distribution of this negativity (Figs. 3 and 4). Contrasts revealed significantly larger negativities following chronology violations versus consistent sequences at the left anterior quadrant than at the right anterior quadrant, F(1, 21) = 5.39, p < .05. There were no other significant differences between quadrants across conditions. 3.2.1.2. Midline electrodes. Analysis of the midline electrodes yielded a main effect of Chronology, F(1, 21) = 10.24, p < .01, but the interaction with Position did not reach significance, reflecting the generally more negative amplitudes in the chronology violation condition at the midline sites. 3.2.2. N400 window (350–450 ms) 3.2.2.1. Lateral electrodes. In the 350–450 ms time window, we again found a significant main effect of Chronology, F(1, 21) = 16.21, p < .01, as well as a significant three-way Chronology  Position  Hemisphere interaction, F(1, 21) = 5.64, p < .05. Only the comparison of the right anterior quadrant with the right posterior quadrant approached significance, F(1, 21) = 3.57, p = 0.07, indicating marginally larger negative differences at posterior than at anterior sites, consistent with the right centroparietal distribution typical of the N400 (Fig. 4). 3.2.2.2. Midline electrodes. Again, analysis of the midline electrodes yielded a main effect of Chronology, F(1, 21) = 12.36, p < .01, while the interaction with Position was only marginally significant, F(5, 105) = 2.49, p = 0.09, consistent with the centro-parietal maximum of this effect (Fig. 4). 4. Discussion We presented sequences of line drawings depicting everyday script events in the correct order (consistent) or reversed order (chronology violation). Participants judged whether the second picture was a logical continuation of the first. The results showed that participants performed the task accurately in both conditions, but, despite the delayed nature of their response, were slower in making this judgment for chronology violations. Taken together, this indicates that there was indeed a clear preferential order for the pictures. Crucially, as predicted, we found a left-anterior negativity (LAN) in response to chronology violations, followed by an increased N400.

119

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

Fig. 2. Grand average amplitudes following consistent sequences (black) and chronology violations (red) for nine electrodes. Negativity is plotted upward. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fp1 F7 FC5

Fp2

F3

F4 FC1

FC2

F8 FC6

Fig. 3. Comparison of potential differences between conditions (chronology violation minus consistent) between left hemisphere (blue) and right hemisphere (red).The vertical blue bar denotes the LAN window (250–350 ms). Negativity is plotted downward. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

120

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

LAN (250-350 ms)

N400 (350-450 ms)

Fig. 4. Scalp distribution of amplitude difference chronology violation – consistent for the LAN and N400 time window (lV).

We expected increased LAN amplitudes following chronology violations. Indeed, even in absence of morphosyntactic markers or structure encoding words or symbols featured in previous studies (Baggio, 2008; Hoen & Dominey, 2000; Münte et al., 1998), chronology violations, as specific instances of structure violations, were followed by more negative mean amplitudes than consistent sequences. This deflection was similar in timing and distribution to previously reported ERPs following chronology violations, reporting deflections around 300 ms post stimulus, maximally over left anterior sites (Baggio, 2008; Münte et al., 1998). As such, the present findings support the idea that the LAN may index a general neurocomputational function engaged by both linguistic and non-linguistic sequential structures (Hoen & Dominey, 2000). It can also be taken as tentative support for the proposal that processing script chronology reflects a broad, domain general sequencing function subserved by the left PFC (Hoen et al., 2006). On the other hand, given that ERP effects to sequential, visually presented script content violations have previously been found to elicit bilateral or right-lateralized effects (Sitnikova et al., 2008), this finding speaks against the view that both script content and chronology are represented in one and the same knowledge structure, encoded in the same neural substrates (Wood & Grafman, 2003; but for a left-lateralized effect in response to content violations in verbal scripts, see Chwilla & Kolk, 2005). Language-related LANs have been proposed to index working memory demands (Kluender & Kutas, 1993; Münte et al., 1998; Vos, Gunter, Kolk, & Mulder, 2001). Greater working memory demands may explain increased LAN amplitudes in the present chronology violation condition as well. A possible account may be as follows. Given the relatively faster reaction times for the consistent sequences, participants supposedly generated expectancies regarding the next script event during the presentation of the first (e.g., plausible upcoming events). To the extent that the subsequent second event is pre-activated by this expectancy process (likely larger in the consistent condition), construction of a coherent sequence linking the two events will presumably be faster, involving less potential bridging events to be held in working memory, resulting in a weaker LAN in comparison with script violations. Although we did not analyze beyond the N400 time window, the sustained nature of the negativities are in accordance with a working memory interpretation of the present results, as sustained frontal negativities have been associated with working memory load (Fiebach, Schlesewsky, & Friederici, 2001; Vos et al., 2001). In addition, we predicted increased N400 amplitudes following script violations, since presentation of a given script event facilitates the processing of events that follow it in the script more strongly than events that typically precede it in the script (van

der Meer et al., 2002). It is difficult and beyond the scope of the present study to draw clear theoretical implications from this result with regard to the lexical or integration interpretation of the N400 (see Section 1.1), especially since the graphical stimulus material in the present study does not allow for a straightforward assessment of normative associations of the stimuli. Several spreading of activation accounts (Collins & Loftus, 1975) assume that activation spreads forward from one conceptual node to another. Therefore, assuming that the associations in the semantic network between the script events may have direction-specific strength (Crestani, 1997), spreading of activation could account for the present results. On the other hand, in so far that integration models suppose that semantic associations and integration processes are bidirectional (for a detailed discussion, see Chwilla, Hagoort, & Brown, 1998), they would predict them to be insensitive to presentation order. Some important shortcomings of the present study must be addressed. The multiple use of the same images during the experiment is less than ideal. However, each script was presented in only two of the three conditions for each participant (of which one unreported, see Section 2.3), so that prediction of the upcoming stimulus was never possible. Therefore, this should not undermine the basic conclusions of this study. Further, could lateralization effects perhaps be due to the fact that participants always responded with their right hand? Given that motor responses were on average identical in both conditions, it seems an unlikely explanation for a lateralized difference between the two. This possibility is further diminished by the fact that response preparation was only possible after the offset of the second picture. An important question for further research is to which extent the present results are dependent on the task instructions. In the present study, participants were explicitly asked to judge whether the second picture was a logical continuation of the sequence of events, thereby drawing attention to the sequence in which the behaviors are presented. It remains to be seen to which extent similar effects would be obtained under task conditions that do not require the processing of script chronology. The functional similarities between the LAN observed in the present study and that found in response to verbal structure violations would be more supported in that case, as the LAN in response to violations of verbal syntax seems to occur even when participants are asked to ignore syntax (Hahne & Friederici, 2002). To conclude, the present results contribute to our understanding of the relationship between script chronology processing and sequence processing in other domains such as pictorial narratives and language. The present study extends previous findings suggesting a common neurocognitive capacity underlying the processing of linguistic syntax and abstract sequences to stereotyped

K. Baetens et al. / Brain and Cognition 91 (2014) 113–122

sequences of human actions (Dominey, Hoen, Blanc, & LelekovBoissard, 2003; Hoen and Dominey, 2000; Hoen et al., 2006). Acknowledgments We are deeply indebted to Mathilde-Chevrant Breton for her excellent artwork and Dr. Perrine Ruby for kindly sharing her stimulus material. This research was supported by a PhD research Fellowship to the first author from the Research Foundation – Flanders (FWO). Appendix A. Supplementary material Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.bandc.2014.09. 005. References Abelson, R. P. (1981). Psychological status of the script concept. American Psychologist, 36(7), 715–729. http://dx.doi.org/10.1037//0003-066X.36.7.715. Baggio, G. (2008). Processing temporal constraints: An ERP study. Language Learning, 58, 35–55. http://dx.doi.org/10.1111/j.1467-9922.2008.00460.x. Bower, G., Black, J., & Turner, T. (1979). Scripts in memory for text. Cognitive Psychology, 11, 177–220. Retrieved from . Chwilla, D. J., Hagoort, P., & Brown, C. (1998). The mechanism underlying backward priming in a lexical decision task: Spreading activation versus semantic matching. The Quarterly Journal of Experimental Psychology, 51(3), 531–560. Retrieved from . Chwilla, D. J., & Kolk, H. H. J. (2005). Accessing world knowledge: Evidence from N400 and reaction time priming. Cognitive Brain Research, 25(3), 589–606. http://dx.doi.org/10.1016/j.cogbrainres.2005.08.011. Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P. J., & Kuperberg, G. R. (2012). (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive Psychology, 65(1), 1–38. http://dx.doi.org/ 10.1016/j.cogpsych.2012.01.003. Collins, A., & Loftus, E. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407–428. Retrieved from . Cosentino, S., Chute, D., Libon, D., Moore, P., & Grossman, M. (2006). How does the brain support script comprehension? A study of executive processes and semantic knowledge in dementia. Neuropsychology, 20(3), 307–318. http:// dx.doi.org/10.1037/0894-4105.20.3.307. Crestani, F. (1997). Application of spreading activation techniques in information retrieval. Artificial Intelligence Review, 23, 453–482. Retrieved from . Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. http://dx.doi.org/10.1016/ j.jneumeth.2003.10.009. Dien, J., & Santuzzi, A. M. (2005). Application of repeated measures ANOVA to highdensity ERP datasets: A review and tutorial. In T. C. Handy (Ed.), Event-related potentials: A methods handbook (pp. 1–73). Cambridge, MA: MIT Press. Dominey, P. F., Hoen, M., Blanc, J.-M., & Lelekov-Boissard, T. (2003). Neurological basis of language and sequential cognition: Evidence from simulation, aphasia, and ERP studies. Brain and Language, 86(2), 207–225. http://dx.doi.org/10.1016/ S0093-934X(02)00529-1. Fiebach, C. J., Schlesewsky, M., & Friederici, A. D. (2001). Syntactic working memory and the establishment of filler-gap dependencies: Insights from ERPs and fMRI. Journal of Psycholinguistic Research, 30(3), 321–338. Retrieved from . Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6(2), 78–84. Fuster, J. (2001). The prefrontal cortex—An update: Time is of the essence. Neuron, 30, 319–333. Güldenpenning, I., Koester, D., Kunde, W., Weigelt, M., & Schack, T. (2011). Motor expertise modulates the unconscious processing of human body postures. Experimental Brain Research, 213(4), 383–391. http://dx.doi.org/10.1007/ s00221-011-2788-7. Hagoort, P. (2008). The fractionation of spoken language understanding by measuring electrical and magnetic brain signals. Philosophical Transactions of the Royal Society of London, 363(1493), 1055–1069. http://dx.doi.org/10.1098/ rstb.2007.2159. Hahne, A., & Friederici, A. D. (2002). Differential task effects on semantic and syntactic processes as revealed by ERPs. Cognitive Brain Research, 13(3), 339–356. Hahne, A., & Jescheniak, J. D. (2001). What’s left if the Jabberwock gets the semantics? An ERP investigation into semantic and syntactic processes during

121

auditory sentence comprehension. Cognitive Brain Research, 11(2), 199–212. Retrieved from . Hare, M., Jones, M., Thomson, C., Kelly, S., & McRae, K. (2009). Activating event knowledge. Cognition, 111(2), 151–167. http://dx.doi.org/10.1016/j.cognition. 2009.01.009. Hoen, M., & Dominey, P. F. (2000). ERP analysis of cognitive sequencing: A left anterior negativity related to structural transformation processing. NeuroReport, 11(14), 1–5. Hoen, M., Pachot-Clouard, M., Segebarth, C., & Dominey, P. F. (2006). When Broca experiences the Janus syndrome: An ER-fMRI study comparing sentence comprehension and cognitive sequence processing. Cortex, 42(4), 605–623. Hopper, P. J. (1979). Aspect and foregrounding in discourse. In T. Givon (Ed.), Syntax and semantics: Vol. 12. Discourse and syntax (pp. 213–241). New York: Academic Press. Jung, T. P., Makeig, S., Humphries, C., Lee, T. W., McKeown, M. J., Iragui, V., et al. (2000). Removing electroencephalographic artifacts by blind source separation. Psychophysiology, 37(2), 163–178. Khalkhali, S., Wammes, J., & McRae, K. (2012). Integrating words that refer to typical sequences of events. Canadian Journal of Experimental Psychology = Revue Canadienne de Psychologie Expérimentale, 66(2), 106–114. http://dx.doi.org/ 10.1037/a0027369. Kluender, R., & Kutas, M. (1993). Bridging the gap: Evidence from ERPs on the processing of unbounded dependencies. Journal of Cognitive Neuroscience, 5(2), 196–214. http://dx.doi.org/10.1162/jocn.1993.5.2.196. Knutson, K. M., Wood, J. N., & Grafman, J. (2004). Brain activation in processing temporal sequence: An fMRI study. NeuroImage, 23(4), 1299–1307. http:// dx.doi.org/10.1016/j.neuroimage.2004.08.012. Koester, D., & Prinz, W. (2007). Capturing regularities in event sequences: Evidence for two mechanisms. Brain Research, 1180, 59–77. http://dx.doi.org/10.1016/ j.brainres.2007.08.056. Krueger, F., Moll, J., Zahn, R., Heinecke, A., & Grafman, J. (2007). Event frequency modulates the processing of daily life activities in human medial prefrontal cortex. Cerebral Cortex, 17(10), 2346–2353. http://dx.doi.org/10.1093/cercor/ bhl143. Kuchinke, L., Van Der Meer, E., & Krueger, F. (2009). Differences in processing of taxonomic and sequential relations in semantic memory: An fMRI investigation. Brain and Cognition, 69(2), 245–251. http://dx.doi.org/10.1016/j.bandc.2008. 07.014. Kutas, M., & Federmeier, K. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences, 4(12), 463–470. Retrieved from . Lau, E. F., Almeida, D., Hines, P. C., & Poeppel, D. (2009). A lexical basis for N400 context effects: Evidence from MEG. Brain and Language, 111(3), 161–172. http://dx.doi.org/10.1016/j.bandl.2009.08.007. Lau, E. F., Phillips, C., & Poeppel, D. (2008). A cortical network for semantics: (De)constructing the N400. Nature Reviews Neuroscience, 9(12), 920–933. http:// dx.doi.org/10.1038/nrn2532. Luck, S. J. (2005). An introduction to the event-related potential technique. Journal of health & social policy (vol. 22). Cambridge, MA: MIT Press. Retrieved from