The linguistic context effects on the processing of body-object interaction words: An ERP study on second language learners.

brain research ] (]]]]) ]]]–]]]

Available online at www.sciencedirect.com

www.elsevier.com/locate/brainres

Research Report

The linguistic context effects on the processing of body–object interaction words: An ERP study on second language learners Jin Xuea, Fernando Marmolejo-Ramosb,n, Xuna Peia a

School of English Language, Literature and Culture and Centre for Language and Cognition, Beijing International Studies University, Beijing, China b Gösta Ekman Laboratory, Department of Psychology, Stockholm University, Stockholm, Sweden

art i cle i nfo

ab st rac t

Article history:

Embodied theories of cognition argue that the processing of both concrete and abstract

Accepted 30 March 2015

concepts requires the activation of sensorimotor systems. The present study examined the time course for embedding a sensorimotor context in order to elicit sensitivity to the

Keywords:

sensorimotor consequences of understanding body–object interaction (BOI) words. In the

ERP

study, Event-Related Potentials (ERPs) were recorded while subjects performed a sentence

Linguistic context

acceptability task. Target BOI words were preceded by rich or poor sensorimotor sentential

Body-interaction effect

contexts. The behavioural results replicated previous findings in that high BOI words

Embodiment

received a response faster than low BOI words. In addition to this, however, there was a context effect in the sensorimotor region as well as a BOI effect in the parietal region (involved in object representation). The results indicate that the sentential sensorimotor context contributes to the subsequent BOI processing and that action-and perceptionrelated language leads to the activation of the same brain areas, which is consistent with the embodiment theory. & 2015 Elsevier B.V. All rights reserved.

1.

Introduction

Recent research on the semantic processing of concrete concepts argues that certain concrete concepts are processed faster than others, since they refer to objects with which people can easily interact. For example, while the words “belt” and “sun” represent concrete concepts, it is physically easier to interact with the

former than with the latter. This phenomenon is known as the body–object interaction effect (BOI) (Siakaluk et al., 2008a, 2008b). Words referring to objects with which it is very easy to interact are called high body-interaction words, or high BOI, and words referring to objects with which it is less easy to interact, are known as low body-interaction words, or low BOI. High and low BOI words have different ratings based on the ease with which a

n Correspondence to: Gösta Ekman Laboratory, Frescati Hagväg 9A, Department of Psychology, Stockholm University, Stockholm 114-19, Sweden. E-mail addresses: [email protected] (J. Xue), [email protected] (F. Marmolejo-Ramos). URL: https://sites.google.com/site/fernandomarmolejoramos/ (F. Marmolejo-Ramos).

http://dx.doi.org/10.1016/j.brainres.2015.03.050 0006-8993/& 2015 Elsevier B.V. All rights reserved.

Please cite this article as: Xue, J., et al., The linguistic context effects on the processing of body–object interaction words: An ERP study on second language learners. Brain Research (2015), http://dx.doi.org/10.1016/j.brainres.2015.03.050

2


person can physically interact with each word’s referent (i.e., body-interaction), even when these words match in length, printed frequency, familiarity, numbers of features, numbers of senses, numbers of associates, orthographic and phonological neighbourhood sizes, semantic distance and contextual dispersion (see also Tillotson et al., 2008) as well as in concreteness and imageability (see Siakaluk et al., 2008a, 2008b). Siakaluk et al. (2008a, 2008b) have shown that the semantic processing of words referring to objects with which it is easier to physically interact is faster than the same process for words with which referents do not easily interact. As semantic processing is facilitated when words have, in relative terms, stronger bodily experience, BOI is also referenced as a semantic richness that measures the ease of human body interaction with a word reference. Embodied theories of cognition argue that the processing of both concrete and abstract concepts requires the activation of sensorimotor systems (e.g., Barsalou, 2010; Gallese and Lakoff, 2005; Gallese and Sinigaglia, 2011; Niedenthal et al., 2005). For instance, neurons in the cluster located in the ventral premotor cortex were activated when subjects heard or saw stimuli being moved in their peri-personal space or used tools to perform actions; this is taken as evidence of a mirror neuron system dedicated to match action observation and execution (Gallese and Lakoff, 2005). In a recent review on semantic theories and embodiment evidence, Meteyard et al. (2012) found there was broad agreement regarding the idea that sensory and motor information activate whenever a semantic representation was accessed. However, they concluded that differences remained as to what constituted ‘true’ semantic information that was necessary and sufficient for generating semantic representations. This is somewhat consistent with Weiskopf (2010)’s arguments that the embodied view blurs distinctions between linguistic comprehension and its typical consequences, and between representational content and vehicles. The sensorimotor characteristics of BOI words can be learned from experience as well as from receiving verbal information about the associated actions. Consistent with embodied theories, evidence suggests that the processing of action-related languages leads to activation of the same brain areas as the actions themselves. For instance, premotor and/or motor (sensorimotor) areas are activated during the processing of action or motor verbs. When comparing motor verbs to baseline, Hauk et al. (2004) found both the somatotopically organized activation of motor and premotor cortex and partial overlap of these, along with activations for face, arm and leg actions. Aziz-Zadeh et al. (2006) used fMRI to test whether the mirror neuron areas in human premotor cortex respond both to visually presented actions and actions described by literal phrases. The data indicated direct activation of action representations in the premotor cortex during the reading of action phrases. The results suggest that mirror neuron areas play an important role in sensory–motor representations during conceptual processing of actions evoked by linguistic stimuli. Rüschemeyer et al. (2007) reported that the left precentral gyrus, the central sulcus bilateral postcentral gyrus and the left parietal operculum were activated during the processing of German simple hand-related motor verbs. Hargreaves et al. (2012) used event-related fMRI to examine neural correlates of

BOI in a semantic categorization task. They found that the left inferior parietal lobule, a sensory association area involved in kinaesthetic memory, participated in processing high BOI. In a recent fMRI study, Bracci and Peelen (2013) found that the visual cortex reflects the degree to which objects are controlled by the body to interact with the world. Specifically, there was an overlap between responses to body (e.g., hands) and object (e.g., hammers) effectors in the lateral occipitotemporal cortex and the parietal cortex. The whole-brain representational similarity analysis revealed that the similarity of multivoxel object response patterns in the left lateral occipitotemporal cortex selectively predicted the degree to which objects were rated as controlled by, and extending, the body. These results reveal that the BOI captures the relative availability of sensorimotor information: processing of which is likely to activate the sensorimotor areas of the brain, such as the prefrontal cortex (associated with spatial memory), the parietal lobe (associated with abstract somatosensory knowledge of actions), and the parietal operculum (associated with finger stimulation) (see also Binder et al., 2009; Noppeney et al., 2005). To date, the majority of studies on embodiment have focused on native language (L1). As non-native (L2) speakers are less developed in L2 semantic representations (Finkbeiner et al., 2004), it is argued that the sensorimotor information may not be rich enough to activate sensorimotor brain areas. In addition, BOI effects have for the most part, been examined in single-word tasks, such as in semantic categorization tasks (e.g., Siakaluk et al., 2008b). Major criticisms of singleword studies refer to their oversimplification of language, concluding that multi-word experiments are therefore needed to address this issue. A recent contention holds that psycholinguistic studies should begin to move away from single-word experiments and venture into multi-word experiments (see Sakreida et al., 2013). This view is in line with current embodied theories of language presenting models that add kinesics and paralanguage as pertinent to flesh out the linguistic stream (see Cevasco and Marmolejo-Ramos, 2013). An important characteristic of contextual effects is the rapid on-line integration of diverse sources of information during language comprehension. In the situated context, it has been found that prior experience with, or observation of, similar actions, is influential in predicting subsequent language processing. For example, in Sedivy et al. (1999)’s study, listeners were able to locate a target referent (e.g., “the tall glass”) more quickly when the visual context provided an object within the same category (e.g., a small glass) than when it did not. In the same vein, Weber et al. (2010) showed that lexical decision times were slowed when visuallysupported expectations did not match the target word (e.g., responses to the target word “palm tree” were slow when it was preceded by visually presented nouns that were semantically unrelated to the target word and followed by the sentence fragment “the woman bakes…”). Furthermore, linguistic constraints could even be overridden by greater contextualization of the situation, semantic expectations being supported by the visual scene and the unfolding event in the scene (for details see Weber et al., 2010). The relative importance of the sources of contextual information was investigated by Knoeferle and Crocker (2007) who examined



the temporal interdependency between dynamic visual context and utterance comprehension by using the eye-tracking technique. Event scenes were presented prior to the onset of an utterance and replaced by a blank scene before or during utterance. It was found that the co-present events had relative priority over other contextual sources. As proposed in the Coordinated Interplay Account (Knoeferle and Crocker, 2007), the salience of relevant contextual information synchronizes with the unfolding utterance, resulting in highly sufficient use of the contextual information, and highlighting the importance of the relevant information during on-line integration. Previous studies have considered the role of sentential context in word processing (Coulson et al., 2005; Wlotko and Federmeier, 2012). However, results have been incongruent as to whether the acquisition of semantic information results in sensitivity of the sensorimotor cortex to the predicted BOI words. The recruitment of sensorimotor areas was found to be modulated by the types of sentences used; the less literal the sentence, the less activation of the sensorimotor areas (Desai et al., 2011; Raposo et al., 2009). Aravena et al. (2010)’s ERP study found modulation of motor potentials, which revealed a multimodal semantic facilitation of the motor response. Motor response elicited a motor potential (MP) and a re-afferent potential (RAP), which were both enhanced in the compatible condition. Especially, brain markers of semantic processing exhibited in N400-like effects indicated that incompatibility with motor processes interferes with sentence comprehension in a semantic fashion. Cross et al. (2012)’s study of functional magnetic resonance imaging revealed that observation of knots learned by name was associated with activation in the superior parietal lobule, while observation of knots learned through the experience of tying was associated with activation along the intraparietal sulcus. Moody and Gennari (2010) found that particular pre-motor regions were sensitive to the degree of physical effort implied in short sentences. That is, while a sentence such as “the delivery man is pushing the piano” implies a high physical effort, a sentence such as “the delivery man is pushing the chair” implies a lower level of physical effort. Specifically, the researchers found that the anterior portion of the left inferior frontal gyrus was sensitive to the semantic content implied by the sentences and also showed correlated activity with other sensorimotor and temporal regions. A recent study regarding context effects on BOI words further indicates that semantic processing is dynamic and can be modulated through context (see Tousignant and Pexman, 2012). By comparison, Senot et al. (2011) used trans-cranial magnetic stimulation (TMS) technique and found that there was no difference in motor cortex activation when participants observed actions upon objects labelled as “heavy” vs. “light”. Similar results were found in Quandt and Marshall (2014). This latter study used electroencephalography (EEG) to examine whether different degrees of experience of sensorimotor experience with objects would lead to differential patterns of sensorimotor rhythms during the observation of similar actions on those objects. It was found that for participants who had sensorimotor experience with the objects, the EEG response was differentially sensitive to the anticipated weight of objects. They concluded that the sensitivity was based on the participant’s prior sensorimotor experience with the objects. The participants who

3

received only semantic information about the objects showed no such effects.

1.1.

Present study

Despite a growing body of empirical evidence on the situated contextual effects on sentence processing (see references above), current research has overlooked the real-time integration of sensorimotor information and processing of BOI words. Therefore, the present study examined the time course over which the sensorimotor context is integrated into the understanding of subsequent BOI words for L2 learners. If sensorimotor information related to BOI words is enhanced or attenuated via linguistic contexts, it could be reasoned that the semantic processing of these words could be affected by the phrasing of the linguistic contexts in which BOI words are embedded. The study sought to answer the following two questions: (1) Does richer sensorimotor context lead to increased sensitivity to the anticipated sensorimotor consequences of BOI effect? (2) Do the context and use of BOI word processing activate sensory-and motor-related brain areas? In order to address these questions, ERP (Event-related potentials) data were collected during sentence processing. Participants read high and low BOI words embedded in poor and rich sentential contexts. Data was analyzed for the difference between high and low BOI words and the difference between poor and rich contexts. ERPs provide an on-line measure with millisecond accuracy, of the effects of a specific stimulus on the brain. Thus, ERPs provide a powerful tool to investigate contextual effects on different BOI words. ERPs are labelled by their latency with respect to stimulus onset. The amplitudes of the ERP waveform relative to the baseline can be interpreted as the degree of engagement in a task. The positive and negative deflections (components) are found to be correlated with sensory, motor and cognitive processing (e.g., Kutas and Federmeier, 2000). Equally importantly, ERPs can capture neural activities throughout a time period, and are quite sensitive in revealing individual differences that may not emerge in behavioural studies (e.g., ratings and reaction times) (e.g., Tokowicz and MacWhinney, 2005). The P200 and N400 are two ERP components that have been directly tied to the study of embodied semantics and language comprehension. The P200 (P2) is a positive-going wave around 200 ms after stimulus onset. The P2 is an early component related to memory and attention processing (Mangels et al., 2001). The P2 has also been linked to visual spatial processing, such as spatial memory retrieval (Tlauka et al., 2009), spatial discrimination (O’Donnell et al., 1997), spatial attention (Niu et al., 2008), and stimulus spatialorientation (Song et al., 2007). Heim and Alter (2006) have further shown that the P2 is also involved in early sensory coding during speech and prosody processing. The N400 is the ERP component that has been directly tied to the study of embodied semantics and language comprehension. The N400, a negative deflection, occurs approximately 400 ms after stimulus onset (Friederici, 2002). The N400 is typically largest in central and parietal regions. It has been associated with difficulty in meaning integration in that


4


showed that there was a main effect of BOI word [F(1,16)¼ 16.31, p¼ 0.001]; but there was no effect of context [F(1,16)¼ 2.50, p¼ 0.13] or an interaction between BOI word and context [F(1,16)¼ 0.26, p¼0.62]. Paired-samples t-tests showed that high BOI words were more acceptable than low BOI words in rich contexts [t(16)¼3.99, p¼ 0.001] and, marginally, in poor contexts [t(16)¼ 2.07, p¼0.055]. Mean reaction times across the four conditions are shown in Fig. 1B. Repeated measures ANOVA of within-subject effects showed there was a main effect of BOI words [F(1,16)¼ 13.68, p ¼0.002] but there was neither an effect of context [F (1,16)¼ 0.58, p¼ 0.46] nor an interaction between BOI words and context [F(1,16)¼ 0.11, p ¼ 0.74]. Paired sample t-tests showed significant difference between high and low BOI words in rich contexts [t(16) ¼ 3.21, p¼ 0.005] and between high and low BOI words in poor contexts [t(16)¼ 2.93, p ¼0.01]. As shown in Fig. 1B, there is a trend, though, indicating that rich contexts seem to slow down the processing of BOI words [Mrich contexts ¼ 638.54, SD¼201.20; Mpoor contexts ¼ 589.88, SD ¼153.04; t(33)¼ 1.07, p¼ 0.29]. The behavioural results showed that there was a strong BOI effect in different sensorimotor contexts, whereas, there was no main effect of context. However, it seems that rich vs. poor contexts contributed differentially to behavioural accuracy and RT in the sentence acceptability task. For instance, BOI effects on sentence acceptability are more likely to occur in rich contexts (see Fig. 1A). The behavioural results might not be sensitive enough to implicit context effects. It is hoped that the ERPs are more sensitive to the implicit contextual

1

p = .001

High BOI

Low BOI

0.9

Mean acceptability rates

it is sensitive to target stimuli that are unrelated to probe stimuli (Hagoort, 2008). It has also been found that N400 amplitude is larger when words semantically disassociate from one another (Rugg, 1984). The N400 could also reflect the process of retrieving images from memory. For instance, an N400 component occurred when frontal–central areas, the left temporal gyrus, and anterior–medial temporal lobe were activated during mental rehearsal imagery tasks (Legrain et al., 2002; Van Petten and Luka, 2006). Willems et al. (2008) have shown that larger N400 amplitudes relate to increased semantic load of a word integrated into a preceding context. This pattern has also been found in studies of motor compatibility effects in language comprehension. Specifically, larger N400 amplitudes occur when participants read sentences that present two actions which cannot be performed at once (e.g., “while cleaning the wound, he unrolled the bandage”) (Santana and de Vega, 2013). In keeping with the studies of Siakaluk and colleagues, this research expects to identify a principal effect of BOI words. As the literature suggests, it is likely that the sensorimotor context is embedded in the on-going BOI understanding. That is, from a behavioural standpoint, it should be found that high BOI words receive a faster response than low BOI words. It is also expected to identify contextual effects on the neural processing of high and low BOI words. It is predicted that the sensorimotor linguistic contexts will have a differential effect on high vs. low BOI words. Specifically, it is predicted rich sensorimotor contexts are more influential in triggering high BOI processing, resulting in easier on-line integration. Thus, rich contexts for high BOI words are predicted to have shorter RT. From a neuro-imaging standpoint, generally, it is predicted that BOI activation in the brain may be modulated by sensorimotor semantic-based predictions about body–object interactions. It is predicted that the N400 amplitudes for high BOI words are smaller than those for low BOI words and that N400 amplitudes are smaller in rich context than in poor context. Low BOI words would evoke more brain activities in the sensor motor areas than high BOI words, especially in rich contexts, since it is consuming for the low BOIs to connect with the sensorimotor information from the sentential context. In addition, it is predicted that the sensorimotor areas would be more sensitive to these activations than other regions; hence, the P200 effects are expected to be found especially in the sensorimotor areas.

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Rich context

2.

Poor context

Results

We first analyzed behavioural accuracy and reaction time (RT) on the sentence acceptability task. Then, we analyzed the P200 and N400 effects to examine the context effects on the subsequent BOI word processing. Specifically, we compared the mean activities (μV/per millisecond) in the time window of 600–800 ms (capturing blanks) with the other two time windows, 990–1050 ms (capturing P200 for the critical BOI words) and 1100–1300 ms (capturing N400 for the critical BOI words).

Mean reaction times

750

p = .005

High BOI

Low BOI p = .01

700

650

600

550

500

2.1.

Acceptability results and RTs

Acceptability rates across the four conditions are shown in Fig. 1A. Repeated measures ANOVA of within-subject effects

Rich context

Poor context

Fig. 1 – Mean acceptability rates (A) and mean reaction times (B) of the high and low BOI words when embedded in rich and poor contexts. Error bars represent71 SEM.



effects and therefore reveal whether BOI words can be affected by linguistic contexts.

2.2.

ERP results

Grand-averaged ERPs time-locked to the onset of the critical words high BOI and low BOI in rich and poor context are shown at electrode sites Cz and P4 (Fig. 2) along with the waves representing the grand average of amplitudes evoked by the high BOI words in rich contexts (black line), low BOI words in rich context (red line), high BOI words in poor contexts (blue line), and low BOI words in poor contexts (green line). As shown in Fig. 2, there was a positive-going component (P200) at around

5

200 ms and 1000 ms at the electrode P4. Significant ERP differences were observed between the rich and poor contexts. Cz showed a clear negative-going component (N400) with a peak around 400 ms post word onset and was found in the central– parietal sites, followed by a late negative-going component around 1100–1300 ms. The topography (Figs. 3 and 4) showed that the N400 in the poor context was more negative than that in the rich context, confirming the N400 effect. For high BOI words, the wave contrast (poor context minus rich context) resulted in more obvious N400 effect than that for low BOI words in the central area. These effects were compared through a statistical analysis of the mean activities in the three time windows: 600–800 ms (for blank), 990–1050 ms (capturing P200 of the critical BOI words) and1100–1300 ms (capturing N400 of the critical BOI words), respectively (see Section 5.9).

2.3. Comparison between 600–800 ms and 990–1050 ms time windows

Fig. 2 – Grand average waveforms at electrodes of Cz and P4. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

For the frontal and parietal areas, repeated measures ANOVA analysis showed no effects. For the central area, there was a main effect of time period, [F(1,16)¼7.03, p ¼0.02], and a marginal effect of BOI [F(1,16)¼3.80, p¼ 0.06]. For the occipital area, there was a main effect of time period, [F(1,16)¼4.36, p¼ 0.053], and a three way interaction of time period, context and BOI [F(1,16)¼ 4.76, p¼ 0.04]. Simple effect analysis by repeated measures ANOVA showed that for 600–800 ms, the context effect was not statistically significant for either high BOI [Mrich context ¼ 3.05, SD ¼0.64; Mpoor context ¼2.57, SD ¼0.56; F(1,16)¼ 2.64, p¼ 0.12 ] or low BOI [Mrich context ¼3.13, SD¼0.49; Mpoor context ¼2.73, SD¼0.43; F(1,16)¼ 0.46, p¼ 0.51]. In contrast, for 990–1050 ms, context effect was not significant for high BOI [Mrich context ¼ 3.84, SD ¼0.70; Mpoor context ¼4.37, SD ¼0.95; F(1,16)¼ 0.50, p¼ 0.49], but there was a marginal context effect for low BOI [Mrich context ¼4.91, SD¼ 1.08; Mpoor context ¼3.56, SD¼ 0.65; F(1,16)¼ 3.61, p ¼0.076]. Overall, the above results

Fig. 3 – Scalp distributions of the context effects in the high BOI words (poor context minus rich context). Please cite this article as: Xue, J., et al., The linguistic context effects on the processing of body–object interaction words: An ERP study on second language learners. Brain Research (2015), http://dx.doi.org/10.1016/j.brainres.2015.03.050

6


Fig. 4 – Scalp distributions of the context effects in the low BOI words (poor context minus rich context).

suggest that P200 effects were observed in central and occipital areas (sensor and motor areas).

2.4. Comparison between 600–800 ms and 1100–1300 ms time windows For the frontal areas, repeated measures ANOVA analysis showed a main effect of time period [F(1,16)¼5.74, p¼ 0.029], a marginal context main effect [F(1,16)¼ 4.18, p¼ 0.058] and a marginal time period and context interaction effect [F(1,16)¼ 4.43, p¼ 0.052]. Simple effect analysis by repeated measures ANOVA showed that for 600–800 ms, the context effect was not statistically significant [Mrich context ¼ 2.33, SD¼ 0.28; Mpoor context ¼2.27, SD ¼0.27; F(1,16)¼0.096, p¼ 0.76]. In comparison, N400 effect was significant for 1100–1300 ms, with rich context (M¼ 3.33, SD¼ 0.46) activating higher amplitudes than poor context (M¼2.83, SD¼0.40), F(1,16)¼10.88, p¼ 0.005. For the central area, there was a main effect of time period, [F(1,16)¼9.43, p ¼0.007], and a marginal effect of time and context interaction [F(1,16)¼ 4.15, p¼ 0.059]. For the parietal area, repeated measures ANOVA analysis showed a main effect of time period [F(1,16)¼ 12.42, p ¼0.003]. No other effect was found. For the occipital area, there was a main effect of time period, [F(1,16)¼ 6.18, p ¼0.02], and a main effect of context [F(1,16)¼ 5.002, p¼ 0.04]. The time period effects in frontal, central, parietal and occipital areas showed BOI words segment elicited more brain activations than the blank segment. More importantly, the main effect of context in frontal, central and occipital areas indicated that sensorimotor information in the context was carried over for the subsequent BOI understanding.

3.

Discussion

The purpose of this study was to expand the understanding of sensorimotor information embedded in context by highlighting

the ability of sentential context to influence subsequent neural mirroring of BOI word processing. The behavioural results (Fig. 1) confirmed the a priori predictions and replicated previous findings in that the semantic processing of high BOI words is faster and more accurate than that of low BOI words. The ERP results are in line with the significant effects of context found in the behavioural data.

3.1.

Contextual effects on the subsequent BOI processing

Behavioural accuracy and RT data failed to find context effects on the sentence acceptability. This is partly due to the nature of the task where participants were required to judge whether the sentences were acceptable instead of explicitly judging the effect of the context on the BOI words. However, the accuracy data showed higher acceptability rates for sentences with high BOI in rich contexts than high BOI in poor contexts. Additionally, there was a higher accuracy but slower reaction times for sentences with high BOI, especially in rich contexts (Fig. 1). RT analysis showed a tendency for rich contexts to slow down the BOI activation (Fig. 1B). The correlation analysis did not find a significant correlation between the overall accuracy and reaction time (r¼ 0.32, p ¼0.20), ruling out the possibility of a speed-accuracy tradeoff. Therefore, the finding indicates a differential contextual effect for high vs. low BOI activation. The delayed RTs in rich contexts could not be interpreted as the previously observed semantic priming effect. In traditional priming studies, more informative contexts are integrated faster with target words than less informative contexts (see Meyer and Schvaneveldt, 1971 for behavioural studies; Osterhout and Holcomb, 1995 for ERP studies). High and low BOI words were well matched in a series of linguistic properties (imageability, numbers of associates, etc.), and the ratings on context richness showed that rich vs. poor contexts were discriminatory. Therefore, the delayed RTs in rich context should arise from sensorimotor contexts embedded in BOI words.



ERPs did provide robust evidence that sensorimotor contexts contributed to the following BOI word processing, even for the present L2 speakers. Specifically, context main effects were found at the frontal, central and occipital areas. The N400 effect was revealed at Cz (Fig. 2) and at 1000–1300 ms for processing BOI words (especially for high BOI, Fig. 3) when comparing poor contexts with rich contexts. The N400 effect has been widely observed in previous studies of contextual influence on sentence comprehension (Ferguson et al., 2008; Nieuwland, 2013). The N400 can be modulated by lexical preactivation (Federmeier, 2007) and is an index of the benefits from a semantic context (see Van Petten and Luka, 2012, for a review). Therefore, the present study indicates sensorimotor contexts enter into the interaction with BOI activation. However, it is quite surprising that in the present study the low BOI words results showed relatively less significant N400 effect (see Fig. 4). The reduced N400 effect in the low BOI words could be interpreted as a sign of little contextual benefit. As it is argued that the facilitation effects for high BOI words stem from the relative availability of semantic information (e.g., Siakaluk et al., 2008b), the context contributes relatively less information to the activation of the low BOI words in the present study. Nevertheless, the behavioural results on sentence acceptability revealed a statistical difference between high and low BOI words. It could be argued that the higher or lower N400 amplitudes for BOI words might be attributed to sentence sensibility. According to “Context plus BOI words ratings” (see Section 5.2), there was a BOI effect on the overall BOI ratings, sentence concreteness, and emotion valence. The results suggest that a combination of context and BOI can change the structure and nature of the context, so that contexts bring more or less sensorimotor information to the BOI words. Similar results were revealed in Aravena et al. (2010). Their ERP study reported that N400-like component around the Cz electrode position distinguishes between compatible and incompatible sentences, with a greater negative deflection for incompatible. A latest review on N400 ERPs for actions suggests the N400 and action-N400 reflect a common neurocognitive mechanism involved in the construction of meaning through the expectancies created by previous experiences and contexts (Amoruso et al., 2013). Thus, the results in the present study showed motor–language coupling or bidirectional influence of motor and language components. The present study shows there is an adaptation from contextual sensorimotor information perception to the subsequent BOI processing. It was found that low BOI words induced more brain activities in the occipital area especially in rich context (Fig. 5), indicating that the processing of low BOI words in rich contexts is much more effort-consuming than in poor contexts. Low BOI word representations may need more information rehearsal and updating in the shortterm memory, especially in rich contexts. In other words, low BOI words seemed to be better processed in rich contexts. The decreased activity of low BOI in poor context may indicate less efficient neural processing of less enriched semantic representations. Therefore, it is possible that the widespread brain activation in the time period analyses reflects not only the acquisition of the sensorimotor contextual information, but also increased attention to the subsequent BOI words.

7

Fig. 5 – Mean neural activation (μV/per millisecond) in the occipital area. Error bars represent71 SEM.

The rich sensorimotor context may have primed the participants towards processing the high BOI words. Namely, the differential effects for high vs. low BOI words indicate prior semantic learning of sensorimotor information modulates how the subsequent BOI words are processed.

3.2. Embodiment effects with sensorimotor context and BOI words P2 effect was revealed at electrode P4 (Fig. 2) as well as positive-going waves at approximately 200 ms post word onset (i.e., sensorimotor context segment) and 1000 ms post word onset (i.e., the BOI words segment) at Figs. 3 and 4. This deflection could be interpreted as a novel P300 (or P3) or as a late P200 component. P3 is usually elicited by rare stimuli (Heim and Alter, 2006) but this interpretation does not apply to our experimental paradigm. For this reason, it is more reasonable to take the early positive component as a P200 effect. ANOVA results on the time period of 990–1050 ms (capturing P200) confirmed a main effect of BOI in the central area (sensorimotor region) and occipital region. The P2 effect is related to spatial memory retrieval (Tlauka et al., 2009), spatial attention (Niu et al., 2008), stimulus spatial-orientation and spatial congruity of audio-visual stimulus (Song et al., 2007). The P2 is also involved in early sensory coding in speech and prosody processing (Heim and Alter, 2006). P200 effects in the present study indicate that the learning of sensorimotor information leads to activation of the same brain areas. In other words, sensory and motor neural systems play an important role in the construction of language representations. The results of the time period analyses comparing mean area activities between 600–800 ms and 1100–1300 ms, showed that there was a context effect for both high and low BOI words (Figs. 3 and 4) for all the ROIs (i.e., central area, frontal area and occipital area). The findings further support previous work on contextual effects on subsequent language processing. As mentioned in the literature review, the central area (sensorimotor area) is usually activated in sensory perception and motor action (e.g., Hauk, Johnsrude and Pulvermuller, 2004). Occipitotemporal cortex and parietal cortex are


8


important for visual–spatial representations and kinaesthetic memory (e.g., Bracci and Peelen, 2013), while the premotor region is involved in physical efforts (Moody and Gennari, 2010). Thus, the findings provide additional consistent evidence that there is a mirroring of sensorimotor and BOI linguistic representations in the brain. Further, the distinct topography of the BOI effect and context effect indicates the different neural systems involved in the processing of BOI words and processing of semantic information. However, it is noteworthy that electrophysiological and neurological studies have not found support for a somatotopically-organized distribution of effector-specific regions (e.g., Ibánez et al., 2013). To be specific, the present study indicates that the processing of action words mainly involves cortico-subcortical networks. However, the BOI word processing might not depend on direct cortical activation. On the one hand, the somatotopic coordinates reported in previous research do not overlap with defined maps of the motor- and premotor areas. For instance, Kemmerer and Gonzalez-Castillo (2010) proposed the TwoLevel Theory that root-level motor features of verb meaning are partially subserved by somatotopically mapped mirror neurons in the left primary motor and/or premotor cortices, whereas template-level motor features of verb meaning are partially subserved by representationally more schematic mirror neurons in Brodmann area (BA) 44 of the left inferior frontal gyrus. Moreover, neurological and neurophysiological investigations have failed to support somatotopic findings. Arévalo et al. (2012) examined whether lesions to key sensorimotor regions would preferentially impact the comprehension of stimuli associated with the use of the hand, mouth or foot. They found processing of effector-related stimuli was associated with several regions across the left hemisphere, and not solely with premotor/motor or somatosensory regions. Ibánez et al. (2013) revealed the action-sentence compatibility effect (ACE) deficits in early Parkinson’s disease (EPD) cannot be explained by a general motor or language impairment. Specifically, the ACE is impaired in neurodegenerative motor disease as in EPD. However, preserved motor responses were observed to linguistic variables in EPD. Comparatively, the data on epileptic patients revealed motor preparation affected language processing (N400 at left inferior frontal gyrus and middle/superior temporal gyrus), and language processing affected activity in movement-related areas (motor potential at premotor and M1). These results suggest the presence of a motor–language network, which is not restricted to somatotopically defined brain areas. On the other hand, subcortical structures (basal ganglia) would also play an important role in motor–language integration. The contextual effects on action– language processing are partially dependent on cortical–subcortical integration (Cardona et al., 2013). A recent study by Cardona et al. (2014) examined the non-representational and representational views of embodiment, using the actionsentence compatibility effect (ACE) paradigm to examine early Parkinson’s disease (EPD), neuromyelitis optica (NMO), and acute transverse myelitis (ATM) patients; the results support a brain-based embodied view on action language. The results are consistent with the brain-based embodied view on action language in that action/verb processing depends on distributed brain networks supporting context-sensitive motor–language coupling.

The present study found evidence of language embodiment in advanced L2 learners. Much research on language embodiment has been devoted to native language (L1) processing. Language embodiment study on non-native language (L2) processing, however, is limited and the results are controversial. It has been argued that L2 semantic representations are less developed than L1 (Finkbeiner et al., 2004). This line of research indicates that activation in sensorimotor brain areas for L2 may be absent. The present study of L2 English learners shows that action- and perception-related brain areas for L2 words are activated, indicating that the semantic representations for L2 learners are rich enough for the sensorimotor-related activation. In other words, conceptual features of sensorimotor and body– object interaction are represented in sensorimotor areas, reflecting the specific learning experience during language acquisition. The finding is consistent with De Grauwe et al. (2014). In their study, when German learners of Dutch made lexical decisions about visually presented Dutch motor and non-motor verbs, they showed increased activation for both cognate and noncognate verbs in motor and somatosensory regions (i.e., preand post-central gyrus, central sulcus, and parietal operculum) compared to non-motor verbs. In sum, these findings have implications for the contextual effect on BOI words. On the one hand, it is argued that the objects or scenes depicted in linguistic contexts are likely to blend with the perceptual and motor properties of concrete words embedded in those contexts (see Mishra and MarmolejoRamos, 2010). The BOI word activation is nested in the contextual information. Specifically, the sensorimotor context could contribute to the neural activation for subsequent BOI word processing. The rich contextual sensorimotor information may facilitate a response to anticipated high BOI words, but when the sensorimotor-based expectations are not met by the target BOI words, the processing rate of these words should be attenuated if not inhibited. On the other hand, embodied cognition theorists argue that sensory and motor systems are activated during conceptual processing (e.g., Mahon and Caramazza, 2008), further suggesting that sensorimotor information is likely to be grounded in experience (e.g., Barsalou, 2010). Thus, if enough sensorimotor information is added to a preceding linguistic context, it could be possible that people generate expectations as to the BOI levels (i.e., high or low) of an upcoming concrete word.

4.

Conclusions

The present study demonstrates that the sentential sensorimotor context contributes to the subsequent BOI processing and the sensorimotor cortex is involved in the processing. The contextual sensorimotor information is associated with sensitivity to the predicted sensorimotor consequences on BOI words. These findings are consistent with the embodiment theory. Overall, the present results indicate that contextual sensorimotor information is carried over to the subsequent BOI processing. However, as argued above, the marginal context effect on the subsequent BOI words processing could be influenced by the fact that words and concepts are not deeply grounded into sensorimotor processes in the case of L2. Thus, future studies should investigate how


9


language proficiency level modulates the sensorimotor context effect on the subsequence BOI processing.

5.

Experimental procedures

5.1.

Participants

Seventeen postgraduate students majoring in English (four men, Mage ¼24.33, SDage ¼1.19, rangeage ¼22–27) from the Beijing International Studies University of China participated in the experiments. All reported Chinese as L1 and English as L2. According to a 10-point self-rating scale, the means for their English listening, speaking, reading and writing were 7.50 (SD¼ 1.15), 7.32 (SD¼1.01), 8.00 (SD ¼0.87, and 7.53 (SD¼ 0.78). Their self-ratings on Chinese listening, speaking, reading and writing were 9.29 (SD¼0.69), 8.71 (SD¼ 0.92), 9.29 (SD¼ 0.69), and 8.65 (SD ¼0.86), respectively. They have been studying and learning English for many years (M ¼12.89, SD¼ 1.49) and all passed the highest national test for English majors (Level 8) (similar to TOEFL) when they were undergraduate students. Accordingly, they were a group of near-balance ChineseEnglish bilinguals. All participants gave informed consent prior to the experiment and reported normal or corrected-tonormal vision and no history of psychological, developmental disorders, neurological impairments or prescription drugs.

5.2.

Stimuli

The experiment was run in English, since more evidence is needed to confirm whether the second language speakers would assume that sensory and motor properties are grounded. In addition, it is difficult to match Chinese high BOI with low BOI words in terms of orthographic and phonological neighbourhood sizes, numbers of features, numbers of associates, (for details as to these aspects, see Siakaluk et al., 2008b), due to the lack of such a database. Furthermore, 80% of written Chinese is comprised of one phonetic and one semantic radical, whereby the former indicates the sound and the latter indicates the meaning of the character (Shu et al., 2003; Shu and Anderson, 1999). Thus, the use of Chinese characters would make it difficult to tease the sublexical information of BOI words apart from the context. As a result, the 48 words reported in Siakaluk et al. (2008a, 2008b) were used in this study. Rich vs. poor contexts were generated for the two sets of words. The ‘rich sensorimotor context’ sentences comprised at least one sensorimotor property that could be associated to the word, whereas ‘poor

sensorimotor context’ sentences were plain descriptions of the word. Then, the “youþverb” structures were added to generate a complete sentence. The same set of concrete verbs was used for both types of contexts. The length of sentences ranged from 5 to 9 words (Mlength ¼6.73, SD¼ 1.06), and the length was controlled across conditions, p40.1. Examples for each condition are shown in Table 1. Additionally, 96 semantic anomaly sentences were created as fillers, including semantic violation and non-word violation sentences. Since there is much behavioural and neuro-imaging evidence supporting the link between semantic processing and emotion valence, imageability, concreteness, word cloze probability, etc. (e.g., Balota et al., 2007; Siakaluk et al., 2008b), the high BOI and low BOI words should match on these parameters.

5.3.

BOI words ratings

Participants rated each word according to the ease or difficulty with which a human body can physically interact with each word’s referent, using a scale ranging from 1 (low body– object interaction) to 10 (high body–object interaction). Fourteen Chinese native learners of English, who did not participate in the main ERP experiment, were also asked to rate each word according to BOI, imageability, concreteness and emotional valence on a 10-point scale. The high and low BOI words differed significantly on BOI [F(1,46)¼ 10.81, p¼ 0.002], but they were matched well on imageability [F(1,46)¼0.02, p¼ 0.90], concreteness [F(1,46)¼ 1.70, p ¼0.20], and emotional valence [F(1,46)¼ 0.50, p ¼0.48].

5.4.

Contexts ratings

An additional group of 15 participants were asked to rate on a 10-point scale how rich the context (i.e., the string of words before the BOI word) was. Repeated measures ANOVA with BOI word and context as within-subject factors showed a main effect of context [F(1,23)¼22.29, po0.001], but no BOI effect or context and BOI interaction effect were significance. That is, rich contexts (regardless of the BOI word they embedded) were rated as richer contexts rather than poor contexts [t(47)¼ 5.55, po0.001].

5.5.

Context plus BOI words ratings

Another 14 participants were asked to rate “the context plus target BOI words” in terms of BOI, imageability, concreteness, and emotional valence. Repeated measures ANOVA with BOI word and context as within-subject factors found that the rich

Table 1 – Examples of the high and low BOI words in poor and rich linguistic contexts. Type

Context

Items

Examples

High BOI

Low sensorimotor High sensorimotor Low sensorimotor High sensorimotor

24 24 24 24

You/brush/pieces of/a baked/crumb You/brush/the small/sticky/crumb You/wear/a string/of cotton/lace You/wear/a soft/and smooth/lace

Low BOI

Notes: BOI refers to the degree of body–object interaction. The underlined parts are the target words.


10


or poor context had no effect on the sentence BOI [F(1,23)¼ 2.17, p¼ 0.14], imageability [F(1,23)¼1.02, p¼0.32], concreteness [F(1,23)¼ 0.07, p¼0.79], and emotional valence [F(1,23)¼ 0.044, p¼ 0.83]. However, there was a main effect of BOI on BOI rating [F(1,23)¼ 18.53, po0.001], a BOI effect on concreteness [F(1,23)¼ 4.87, p¼ 0.038] and a BOI and context interaction effect on emotion valence [F(1,23)¼5.36, p¼ 0.03. Simple-effect analysis revealed that the BOI had no effect on emotion valence in the rich context [F(1,23)¼0.01, p¼ 0.93], but a marginal effect in the poor context [F(1,23)¼4.28, p¼ 0.05]. That is, the processing of BOI words embedded in poor contexts might involve some degree of emotion valence. However, the overall results indicate BOI plus context influence the comprehension of BOI words embedded in sentences.

5.6.

Sentence cloze probability ratings

Another 15 participants that did not participate in the ERP experiment completed a questionnaire. They were asked to fill-in a word in the final position of the above contextual sentences. The occurrence rates of the key word produced by the 15 participants for each sentence were calculated. Repeated measures ANOVA with the cloze probability as the dependable variable and BOI word and context as withinsubject factors showed no effect of context [F(1,23)¼ 0.13, p¼ 0.72], or effect of BOI [F(1,23)¼3.02, p ¼0.096], or context and BOI interaction [F(1,23)¼1.00, p¼ 0.33].

5.7.

Sentence final-word expectancy ratings

Fifteen participants who did not participate in the ERP experiment filled in a questionnaire. They were instructed to indicate the degree of their expectancy of the key words in completing the sentences on a scale from 1 to 5 (1 ¼not likely; 2 to 4¼ to some degree; 5 ¼very likely). Repeated measures ANOVA with the expectancy degree as the dependable variable and BOI word and context as within-subject factors showed no effect of context [F(1,23)¼0.22, p ¼0.65], or effect of BOI [F(1,23)¼ 2.99, p¼ 0.097], or context and BOI interaction [F(1,23)¼0.08, p¼ 0.78].

5.8.

Procedure

Each participant read 96 experimental sentences with 24 items for each experimental condition. An additional 96 set of fillers were semantic anomaly sentences. Each sentence was divided into five segments, with the key words located at the end of the sentence (Table 1). The ordering of experimental sentences and filler sentences was pseudo-randomized. The sentences were presented in a random order determined by the computer program E-Prime (Version 2), which also recorded the accuracy and reaction times and sent critical word onset information to the ERP acquisition software. Participants read sentences on a computer screen. During a trial prior to each sentence, a fixation cross appeared at the centre of the computer screen for 300 ms, then sentences were presented segment-by-segment at the centre of the computer screen for 500 ms. In between each segment, there was a blank screen for 300 ms. At the end of the sentence, a prompt “?” appeared on the screen for 2000 ms.

The participants responded by pressing buttons on a computer keyboard. They pressed a button marked “1” with their left hand to indicate if they thought the sentence was acceptable and a button marked “2” with their right hand if they thought the sentence was unacceptable. The sentences were randomly assigned to two versions, with right hand and left hand response types counterbalanced. The two blocks of sentences were counterbalanced. Each participant first completed 10 practice trials, consisting of five target sentences and five semantic anomaly sentences. All practice stimuli were similar to the experimental items.

5.9.

Data acquisition

Continuous EEG was recorded from 64 active electrodes (ActiCap, Brain Products GmbH, Munich) at standard international 10–20 system, referenced to bilateral mastoids and grounded to forehead. To control for vertical eye movements, a vertical electro-oculogram (VEOG) was recorded from Ag/ AgCl electrodes placed closely below the left eye. Horizontal eye movements were measured by a horizontal electrooculogram (HEOG) recorded from Ag/AgCl electrodes placed at the outer canthus of each eye. All impedances were kept below 20 kΩ during the experiment. EEG signals were bandpass filtered between 0.016 and 100 Hz, and amplified and digitized at a rate of 500 Hz using a BrainAmp amplifier (Brain Products GmbH, Munich). All EEG data were collected using Brain Vision Recorder software from Brain Products.

5.10.

Data analyses

The EEG data were processed offline using Brain Vision Analyzer 2. They were re-referenced to the mean of the left and right mastoid, and filtered with a 0.1 Hz high-pass filter to remove drifts and a 30 Hz filter to eliminate line noise. EEG analysis epochs were time-locked to one segment (time 0 ms) before the key words (time 800 ms). Effects of BOI words and context were assessed by measuring the mean neural activities activated for each participant and electrode in intervals over an epoch from 200 ms before to 1800 ms after the onset of the segment, the pre-critical words or one segment before the sentence-final words. The 200 ms pre-stimulus interval was used for baseline correction of the critical words. Epochs were removed from analysis if there was significant artefact in the EEG signal. There was no significant difference in rejection rate across conditions (Mhigh BOI rich context ¼ 96.3%; Mlow BOI rich context ¼96.3%; Mhigh BOI poor context ¼96.76%; and Mlow BOI poor context ¼ 96.53%); F(3,68)¼0.02, p¼ 0.995. Following the EEG study on sensorimotor activation by Quandt and Marshall (2014), a scalp Region of Interest (ROI) for the EEG analyses was defined as the electrodes over sensorimotor cortex. The ROI (central area) included seven electrodes: FC1, FC2, C1, C2, C3, C4, Cz, CP1 and CP2. In order to test the spatial distribution regarding the sensorimotor context effects on BOI processing, data were also analyzed at other 3 ROIs, including frontal (F1, F2, F3, F4, and Fz), parietal (P1, P2, P3, P4, and Pz), and occipital (O1, O2, and Oz) areas, to provide information about the topographic extent of any significant effects. The analyses performed at the frontal,



central, parietal, and occipital electrodes enabled us to assess whether any effects were specific to the sensorimotor area. First, in order to examine how the sensorimotor context brings about the sensitivity in processing the subsequent BOI words, mean activities (μV/per millisecond) in the early time window of the key words (i.e., 990–1050 ms) were analyzed in comparison to the baseline period (i.e., 600–800 ms). The baseline period included the display of the blank screen. Since BOI words and context are the primary interest of the present study, repeated measure ANOVAs were performed on the mean area activities with the within-subject factors ‘time period’ (600–800 ms vs. 990–1050 ms), ‘BOI type’ (high vs. low) and ‘Context type’(rich vs. poor). Second, in order to further examine the context effects on subsequent BOI processing, mean activities (μV/per millisecond) in the late time window of the key words (i.e., 1100– 1300 ms) were analyzed in comparison to the baseline period (i.e., 600–800 ms). Repeated measure ANOVAs were performed on the mean area activities with the within-subject factors ‘time period’ (600–800 ms vs. 1100–1300 ms), ‘BOI type’ (high vs. low) and ‘Context type’ (rich vs. poor). For all analyses, the original degrees of freedom were reported. A Greenhouse–Geisser correction for sphericity was applied to p values when more than two levels of a factor were present (Greenhouse and Geisser, 1959). Reliable main effects and interactions were followed by simple effect analysis when appropriate. Main effects and interactions that involved BOI type and context type were reported here given their theoretical interest. Any main effects or interactions not reported below were all non-significant (all p40.05), unless specified.

Acknowledgments This work was supported by a grant from National Social Science Foundation of China (12CYY027), Social Science Foundation of Beijing, China (14WYC042), a grand from Beijing Higher Education Young Elite Teacher Project (YETP1518), and Special Items Fund of Beijing International Studies University. The authors thank Rosie Gronthos for proofreading this manuscript and the anonymous reviewers for their valuable suggestions.

r e f e r e n c e s

Amoruso, L., Gelormini, C., Aboitiz, F., Gonza´lez, M.A., Manes, F., Cardona, J.F., Ibań˜ez, A., 2013. N400 ERPs for actions: building meaning in context. Front. Hum. Neurosci. 7, 57, http://dx.doi. org/10.3389/fnhum.2013.00057. Aravena, P., Hurtado, E., Riveros, R., Cardona1, J.F., Manes1, F., Ibańez, A., 2010. Applauding with closed hands: neural signature of action-sentence compatibility effects. PLoS ONE 5 (7), e11751. Are´valo, A.L., Baldo, J.V., Dronkers, N.F., 2012. What do brain lesions tell us about theories of embodied semantics and the human mirror neuron system?. Cortex 48, 242–254. Aziz-Zadeh, L., Wilson, S.M., Rizzolatti, G., Iacoboni, M., 2006. Congruent embodied representations for visually presented actions and linguistic phrases describing actions. Curr. Biol. 16, 1818–1823.

11

Balota, D.A., Yap, M.J., Cortese, M.J., Hutchison, K.I., Kessler, B., Loftis, B., Neely, J.H., Nelson, D.L., Simpson, G.B., Treiman, R., 2007. The English Lexicon Project. Behav. Res. Methods 39, 445–459. Barsalou, L.W., 2010. Grounded cognition: past, present, and future. Top. Cogn. Sci. 2 (4), 716–724. Binder, J.R., Desai, R.H., Graves, W.W., Conant, L.L., 2009. Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19 (12), 2767–2796. Bracci, S., Peelen, M.V., 2013. Body and object effectors: the organization of object representations in high-level visual cortex reflects body–object interactions. J. Neurosci. 33 (46), 18247–18258. Cardona, J.F., Gershanik, O., Gelormini-Lezama, C., Houck, A.L., Cardona, S., Kargieman, L., et al., 2013. Action-verb processing in Parkinson’s disease: new pathways for motor–language coupling. Brain Struct. Funct. 218 (6), 1355–1373. Cardona, J.F., Kargieman, L., Sinay, V., Gershanik, O., Gelormini, C., Amoruso, L., et al., 2014. How embodied is action language? Neurological evidence from motor diseases. Cognition 131 (2), 311–322. Cevasco, J., Marmolejo-Ramos, F., 2013. The importance of studying the role of prosody in the comprehension of spontaneous spoken discourse. Rev. Latinoam. Psicol. 45 (1), 21–33. Coulson, S., Federmeier, K.D., Petten, C.V., Kutas, M., 2005. Right hemisphere sensitivity to word- and sentence-level context: evidence from event-related brain potentials. J. Exp. Psychol. Learn. Mem. Cogn. 31 (1), 129–147. Cross, E.S., Cohen, N., Hamilton, A., Ramsey, R., Wolford, G., Grafton, S., 2012. Physical experience leads to enhanced object perception in parietal cortex: insights from knotting. Neuropsychologia 50, 3207–3217. De Grauwe, S., Willems, R.M., Rueschemeyer, S.-A., Lemho¨fer, K., Schriefers, H., 2014. Embodied language in first- and secondlanguage speakers: neural correlates of processing motor verbs. Neuropsychologia 56, 334–349. Desai, R.H., Binder, J.R., Conant, L.L., Mano, Q.R., Seidenberg, M.S., 2011. The neural career of sensory–motor metaphors. J. Cogn. Neurosci. 23 (9), 2376–2386. Federmeier, K.D., 2007. Thinking ahead: the role and roots of prediction in language comprehension. Psychophysiology 44 (4), 491–505. Ferguson, H.J., Sanford, A.J., Leuthold, H., 2008. Eye-movements and ERPs reveal the time course of processing negation and remitting counterfactual worlds. Brain Res. 1236, 113–125. Finkbeiner, M., Forster, K., Nicol, J., Nakamura, K., 2004. The role of polysemy in masked semantic and translation priming. J. Mem. Lang. 51 (1), 1–22. Friederici, A.D., 2002. Towards a neural basis of auditory sentence processing. Trends Cogn. Sci. 6 (2), 78–84. Gallese, V., Lakoff, G., 2005. The Brain’s concepts: the role of the sensory–motor system in conceptual knowledge. Cogn. Neuropsychol. 22 (3-4), 455–479. Gallese, V., Sinigaglia, C., 2011. What is so special about embodied simulation?. Trends Cogn. Sci. 15 (11), 512–518. Greenhouse, S.W., Geisser, S., 1959. On methods in the analysis of profile data. Psychometrika 24 (2), 95–112. Hagoort, P., 2008. The fractionation of spoken language understanding by measuring electrical and magnetic brain signals. Philos. Trans. R. Soc. B 363 (1493), 1055–1069. Hargreaves, I.S., Leonard, G.A., Pexman, P.M., Pittman, D.J., Siakaluk, P.D., Goodyear, B.G., 2012. The neural correlates of the body–object interaction effect in semantic processing. Front. Hum. Neurosci. 6, 22, http://dx.doi.org/10.3389/ fnhum.2012.00022. Hauk, O., Johnsrude, I., Pulvermuller, F., 2004. Somatotopic representation of action words in human motor and premotor cortex. Neuron 41, 301–307.


12


Heim, S., Alter, K., 2006. Prosodic pitch accents in language comprehension and production: ERP data and acoustic analyses. Acta Neurobiol. Exp. 66 (1), 55–68. Ibańez, A., Cardona, J.F., Dos Santos, Y.V., Blenkmann, A., Aravena, P., Roca, M., et al., 2013. Motor–language coupling: direct evidence from early Parkinson’s disease and intracranial cortical recordings. Cortex 49, 968–984. Kemmerer, D., Gonzalez-Castillo, J., 2010. The two-level theory of verb meaning: an approach to integrating the semantics of action with the mirror neuron system. Brain Lang. 112, 54–76. Knoeferle, P., Crocker, M.W., 2007. The influence of recent scene events on spoken comprehension: evidence from eye movements. J. Mem. Lang. 57 (4), 519–543. Kutas, M., Federmeier, K., 2000. Electrophysiology reveals semantic memory use in language comprehension. Trends Cogn. Sci. 4 (12), 463–470. Legrain, V., Gue´rit, J.M, Bruyer, R., Plaghki, L., 2002. Attentional modulation of the nociceptive processing into the human brain: selective spatial attention, probability of stimulus occurrence, and target detection effects on laser evoked potentials. Pain 99 (1–2), 21–39. Mahon, B.Z., Caramazza, A., 2008. A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. J. Physiol. (Paris) 102 (1–3), 59–70. Mangels, J.A., Picton, T.W., Craik, F.I., 2001. Attention and successful episodic encoding: an event-related potential study. Cogn. Brain Res. 11 (1), 77–95. Meteyard, L., Cuadrado, S.R., Bahrami, B., Vigliocco, G., 2012. Coming of age: a review of embodiment and the neuroscience of semantics. Cortex 48, 788–804. Meyer, D.E., Schvaneveldt, R.W., 1971. Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. J. Exp. Psychol. 90, 227–234. Mishra, R., Marmolejo-Ramos, F., 2010. On the mental representations originating during the interaction between language and vision. Cogn. Process. 11 (4), 295–305. Moody, C.L., Gennari, S.P., 2010. Effects of implied physical effort in sensory–motor and pre-frontal cortex during language comprehension. NeuroImage 49 (1), 782–793. Niedenthal, P.M., Barsalou, L.W., Winkielman, P., Krauth-Gruber, S., Ric, F., 2005. Embodiment in attitudes, social perception, and emotion. Pers. Soc. Psychol. Rev. 9, 184–211. Nieuwland, M.S., 2013. “If a lion could speak…”: online sensitivity to propositional truth-value of unrealistic counterfactual sentences. J. Mem. Lang. 68 (1), 54–67. Niu, Y.N., Wei, J.H., Luo, Y.J., 2008. Early ERP effects on the scaling of spatial attention in visual search. Prog. Nat. Sci. 18 (4), 381–386. Noppeney, U., Josephs, O., Kiebel, S., Friston, K.J., Price, C.J., 2005. Action selectivity in parietal and temporal cortex. Cogn. Brain Res. 25 (3), 641–649. O’Donnell, B.F., Swearer, J.M., Smith, L.T., Hokama, H., McCarley, R.W., 1997. A topographic study of ERPs elicited by visual feature discrimination. Brain Topogr. 10 (2), 133–143. Osterhout, L., Holcomb, P., 1995. Event related potentials and language comprehension. In: Rugg, M.D., Coles, M.G.H. (Eds.), Electrophysiology of Mind: Event Related Brain Potentials and Cognition. Oxford University Press, Oxford, England, pp. 171–215. Quandt, L.C., Marshall, P.J., 2014. The effect of action experience on sensorimotor EEG rhythms during action observation. Neuropsychologia 56, 401–408. Raposo, A., Moss, H., Stamatakis, E.A., Tyler, L.K., 2009. Modulation of motor and premotor cortices by actions, action words, and action sentences. Neuropsychologia 47 (2), 388–396. Rugg, M.D., 1984. Event-related potentials in phonological matching tasks. Brain Lang. 23 (2), 225–240.

Ru¨schemeyer, S.-A., Brass, M., Friederici, A.D., 2007. Comprehending prehending: neural correlates of processing verbs with motor stems. J. Cogn. Neurosci. 19, 855–865. Sakreida, K., Scorolli, C., Menz, M.M., Heim, S., Borghi, A.M., Binkofski, F., 2013. Are abstract action words embodied? An fMRI investigation at the interface between language and motor cognition. Front. Hum. Neurosci. 7, 125, http://dx.doi. org/10.3389/fnhum.2013.00125. Santana, E.J., de Vega, M., 2013. An ERP study of motor compatibility effects in action language. Brain Res. 1526, 71–83. Sedivy, J.C., Tanenhaus, M.K., Chambers, C.G., Carlson, G.N., 1999. Achieving incremental semantic interpretation through contextual representation. Cognition 71 (2), 109–147. Senot, P., D’Ausilio, A., Franca, M., Caselli, L., Craighero, L., Fadiga, L., 2011. Effect of weight-related labels on corticospinal excitability during observation of grasping: a TMS study. Exp. Brain Res. 211, 161–167. Shu, H., Anderson, R.C., 1999. Learning to read Chinese, the development of metalinguistic awareness. In: Wang, J., Inhoff, A., Chen, H.-C. (Eds.), Reading Chinese Script: A Cognitive Analysis. Erlbaum, Mahwah, NJ, pp. 1–18. Shu, H., Chen, X., Anderson, R.C., Wu, N., Xuan, Y., 2003. Properties of School Chinese: implications for learning to read. Child Dev. 74 (1), 27–47. Siakaluk, P.D., Pexman, P.M., Aguilera, L., Owen, W.J., Sears, C.R., 2008a. Evidence for the activation of sensorimotor information during visual word recognition: the body–object interaction effect. Cognition 106 (1), 433–443. Siakaluk, P.D., Pexman, P.M., Sears, C.R., Wilson, K., Locheed, K., Owen, W.J., 2008b. The benefits of sensorimotor knowledge: body–object interaction facilitates semantic processing. Cogn. Sci. 32 (3), 591–605. Song, Y., Peng, D.L., Li, X.L., Zhang, Y., Kang, J., Qu, Z., Ding, Y., 2007. Neural correlates of short-term perceptual learning in orientation discrimination indexed by event-related potentials. Chin. Sci. Bull. 52 (3), 352–357. Tillotson, S.M., Siakaluk, P.D., Pexman, P.M., 2008. Body–object interaction ratings for 1,618 monosyllabic nouns. Behav. Res. Methods 40 (4), 1075–7078. Tlauka, M., Clark, C.R., Liu, P., Conway, M., 2009. Encoding modality and spatial memory retrieval. Brain Cogn. 70 (1), 116–122. Tokowicz, N., MacWhinney, B., 2005. Measures of sensitivity to violations in second language grammar. Stud. Second Lang. Acquis. 27 (2), 173–204. Tousignant, C., Pexman, P.M., 2012. Flexible recruitment of semantic richness: context modulates body–object interaction effects in lexical-semantic processing. Front. Hum. Neurosci. 6, 53, http://dx.doi.org/10.3389/fnhum.2012.00053. Van Petten, C., Luka, B.J., 2006. Neural localization of semantic context effects in electromagnetic and hemodynamic studies. Brain Lang. 97 (3), 279–293. Van Petten, C., Luka, B.J., 2012. Prediction during language comprehension: benefits, costs, and ERP components. Int. J. Psychophysiol. 83 (2), 176–190. Weber, A., Crocker, M.W., Knoeferle, P., 2010. Conflicting constraints in resource-adaptive language comprehension. In: Crocker, M.W., Siekmann, J. (Eds.), Resource-Adaptive Cognitive Processes. Springer Verlag, Heidelberg, pp. 119–141. Weiskopf, D.A., 2010. Embodied cognition and linguistic comprehension. Stud. Hist. Philos. Sci. 41, 294–304. ¨ zyu¨rek, A., Hagoort, P., 2008. Seeing and hearing Willems, R.M., O meaning: ERP and fMRI evidence of word versus picture integration into a sentence context. J. Cogn. Neurosci. 20 (7), 1235–1249. Wlotko, E.W., Federmeier, K.D., 2012. So that’s what you meant! Event-related potentials reveal multiple aspects of context use during construction of message-level meaning. NeuroImage 62 (1), 356–366.


Processing morphologically complex words in second-language learners: the effect of proficiency.

An ERP study on initial second language vocabulary learning.

Effects of Semantic Context and Fundamental Frequency Contours on Mandarin Speech Recognition by Second Language Learners.

The influence of context on the processing of emotional and neutral adjectives--an ERP study.

Linguistic coding deficits in foreign language learners.

Language effects in second-language learners: A longitudinal electrophysiological study of spanish classroom learning.

The music of language: an ERP investigation of the effects of musical training on emotional prosody processing.

Dynamic Effects of Self-Relevance and Task on the Neural Processing of Emotional Words in Context.

Influence of Second Language Proficiency and Syntactic Structure Similarities on the Sensitivity and Processing of English Passive Sentence in Late Chinese-English Bilinguists: An ERP Study.

Six principles of language development: implications for second language learners.

The Enhanced Musical Rhythmic Perception in Second Language Learners.

The embodiment of emotional words in a second language: An eye-movement study.

Functionally integrated neural processing of linguistic and talker information: An event-related fMRI and ERP study.

Effects of context and individual predispositions on hypervigilance to pain-cues: an ERP study.

Elicited imitation in second language learners.

Immediate auditory repetition of words and nonwords: an ERP study of lexical and sublexical processing.

An Event Related Field Study of Rapid Grammatical Plasticity in Adult Second-Language Learners.

Brain processing of task-relevant and task-irrelevant emotional words: an ERP study.

Acquisition of Chinese characters: the effects of character properties and individual differences among second language learners.

On the temporal dynamics of sign production: An ERP study in Catalan Sign Language (LSC).

The impact of early bilingualism on controlling a language learned late: an ERP study.

Disentangling Metaphor from Context: An ERP Study.

Effects of Type of Agreement Violation and Utterance Position on the Auditory Processing of Subject-Verb Agreement: An ERP Study.

When Feelings Arise with Meanings: How Emotion and Meaning of a Native Language Affect Second Language Processing in Adult Learners.