Brain and Cognition 94 (2015) 32–43

Contents lists available at ScienceDirect

Brain and Cognition journal homepage: www.elsevier.com/locate/b&c

Evidence of stimulus correlated empathy modes – Group ICA of fMRI data Kavita Vemuri a,⇑, Bapi Raju Surampudi b,a a b

Cognitive Science lab, International Institute of Information Technology, Hyderabad, India Center for Neural & Cognitive Sciences and School of Computer & Information Sciences, University of Hyderabad, India

a r t i c l e

i n f o

Article history: Accepted 24 December 2014 Available online 29 January 2015 Keywords: Empathy networks Movie Group ICA method Cartoon characters fMRI

a b s t r a c t The analysis of the cognitive processes in response to a narrative as presented in a movie provides an insight into momentary reaction to a single depicted action like a facial expression and an aggregate processing of the entire sequence of events. In this study we report results from fMRI data analyzed by group independent component analysis (ICA) method from a free viewing experiment using a diverse set of movie clips – an animation, a Hollywood and an Indian Hindi movie. The fMRI data were collected from 15 college students as they viewed 5–8 min clips from three movies. The movie clips were rated for depiction of emotional expressions, the emotion as per the narrative and the viewer’s own empathy response. The neural correlates attributed to cognitive, motor and emotional empathy were the focus of the study. The methodology of using long duration stimuli in free viewing mode combined with ICA analysis has the potential to tease out spatially distributed but temporally coherent brain activity as demonstrated in this study. The independent components obtained from group ICA method isolated spatial maps with activations which can be safely ascribed to stimulus processing. We found that the activity in the areas attributable to cognitive and motor empathy was comparable for all the three stimuli while certain critical areas for emotional empathy were not noticed for the animation movie. These findings lead to interesting questions on possible differential emotion response in viewer(s) for computer generated actors compared to actors in live-action movies and the role of narrative and exposure to different genre of movies on racial, ethnic and cultural differences in empathy response. Ó 2015 Elsevier Inc. All rights reserved.

1. Introduction Our reactions to events depicted in a movie as in any real-time social interaction are driven by the ability to infer other’s emotional state by assimilating the emoted expressions of the actor(s) and by the narrative. The intense engagement is remarkable considering that viewers are conscious that events on the screen are not ‘real’ but are still able to relate to the events and the state of the actor(s). The engagement with the character could transcend from emotional (‘I feel what you feel’) to cognitive (‘I understand your feelings’) modes of empathy (Decety & Jackson, 2004). The two modes comprise a bottom-up processing covering the emotional and motor areas and a top-down evaluation supported by the cognitive processing areas, thus requiring the engagement of a number of spatially separated brain areas in temporal coherence. The possible temporary affiliation formed between the viewer and ⇑ Corresponding author at: International Institute of Information Technology, Hyderabad, Gachibowli, Hyderabad 500032, Telangana, India. E-mail address: [email protected] (K. Vemuri). http://dx.doi.org/10.1016/j.bandc.2014.12.006 0278-2626/Ó 2015 Elsevier Inc. All rights reserved.

actors in response to the state of the actor has shown to involve both the empathy modes (Blair, 2005; Decety & Jackson, 2004; Preston & de Waal, 2002). Emotion and empathy studies using fMRI have thus far considered perception of static faces and at the most extended to short clips depicting emotional scenes (Adolphs, 2002; Fusar-Poli et al., 2009; Schulte-Rüther, Markowitsch, Fink, & Piefke, 2007). Studies related to pain (Lamm, Batson, & Decety, 2007a; Lamm, Nusbaum, Meltzoff, & Decety, 2007b; Singer et al., 2004) looked at empathy responses to pain inflicted on a stranger, on the participant and on friends. A meta-analysis (Sabatinelli et al., 2011) of research on empathy and emotion revealed the role of set of areas comprising the medial prefrontal cortex (mPFC), bilateral inferior and middle frontal gyrus, superior frontal gyrus (SFG), bilateral amygdala, parahippocampus and fusiform gyrus (FG) for processing emotional faces compared to responses for neutral faces. The study, while comparing responses to emotional scenes versus neutral scenes, indicated additional activation in anterior cingulate, medial dorsal nucleus (MDN) and pulvinar of the thalamus and the right superior temporal gyrus (STG). A similar meta-analysis

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

on empathy related research (Fan, Duncan, de Greck, & Northoff, 2011) found that a majority of the research papers report left and right insula, inferior frontal gyrus, parts of the cingulate cortex (BA 32) and bilateral premotor cortex (BA 6) for empathy. Distinguishing between states termed as ‘affective-perceptual empathy’ and ‘cognitive-evaluative empathy’, meta-analysis of Fan et al. (2011) also reported bilateral responses in insula, inferior frontal gyrus (BA 47) and right supplementary motor area (BA 6) for the former pathway and left insula and left anterior cingulate cortex (BA 32) for the latter. De Greck et al. (2012) looked at ‘intentional empathy’ not restricted to emotional expressions and report a network of areas covering the anterior cingulate cortex, bilateral inferior frontal cortex and bilateral anterior insula. A study on accuracy of empathic interpersonal judgments (Zaki, Weber, Bolger, & Ochsner, 2009) identified the role of medial prefrontal cortex (BA 10) and the dorsomedial prefrontal cortex (BA 9) which was also reported to be active for experienced empathy (Rameson, Morelli, & Lieberman, 2012). Studies on observation of action have shown to trigger empathic action and said to form the ‘mirror neuron’ network. This network covers the premotor, somatosensory, inferior frontal, inferior parietal, STG and insula (Bastiaansen, Thioux, & Keysers, 2009; Carr, Iacoboni, Dubeau, Mazziotta, & Lenzi, 2003; Lawrence et al., 2006). A lesion study (Shamay-Tsoory, AharonPeretz, & Perry, 2009) showed that Brodmann areas 10 and 11 may be necessary for cognitive empathy processes while BA 44 was critical for emotional empathy processes. Frith (2001) reported the significance of the paracingulate cortex (BA 32) for cognitive empathy in an autism study. A study on the role of race on emotional response (Lee et al., 2008) showed activations in the limbic lobe (amygdala and hippocampus) when perceiving own race versus other race faces whereas the contrast of other race emotional face versus own race revealed activations in the frontal, occipital and parietal areas. These studies highlight that a number of areas are engaged in empathic response for either static images or simple short scenes. Studies with dynamic complex stimulus like movie have so far helped in identifying brain networks that process individual features embedded in a scene or of specific events. Hasson, Nir, Levy, Fuhrmann, and Malach (2004) report significant inter-subject correlation (ISC) in brain activity when viewing clips from the movie ‘Good, Bad & Ugly’ and found correlations in the temporal and fusiform area. A comparison of the responses for sad and neutral films (Levesque et al., 2003) showed higher bilateral activations in the midbrain, anterior temporal pole (BA 38, 21) and right ventrolateral prefrontal cortex (BA 47). Han, Yi, Humphreys, Tiangang, and Peng (2005) studied differences in responses to movie clips with cartoon human-like, non-human characters and real actor movie clips and found variations in motion perception for real-actor and human cartoon characters in the medial prefrontal cortex (MPFC) and cerebellum. A study by Mar, Kelley, Heatherton, and Macrae (2007), comparing the ability to perceive intentions from movements by real actors and computer generated cartoons (sharing similar features), found higher responses in the areas associated with mentalizing – the medial prefrontal cortex, the superior temporal sulcus and temporo-parietal junction – for scenes with real actors compared to the depictions by the computer generated animation actors. Studies specific to emotion responses using movies as stimuli include, Goldin et al. (2005) where 2-min clips with sad, amusing and neutral scenes were shown to 13 women to identify the neural correlates associated with emotion processing. For the clips with sad scenes, responses in the medial prefrontal cortex, inferior frontal gyrus, superior temporal gyrus, precuneus, lingual gyrus, amygdala, and thalamus were reported while the amusing clip evoked responses in the medial, inferior frontal gyrus, dorsolateral prefrontal cortex, posterior cingulate, temporal lobes, hippocampus,

33

thalamus, and caudate. Nummenmaa et al. (2012) used 13 movie clips with pleasant, unpleasant and emotional events and found that emotional sensations showed higher inter-subject synchronization. An interesting finding from their study is the higher ISC for negative emotions observed in the thalamus, ventral striatum and insula and the default-mode network areas such as the precuneus, medial prefrontal cortex, posterior superior temporal sulcus and the temporoparietal junction, compared to positive emotions. From the above mentioned studies on empathy using static or dynamic scenes in normal subjects, in patients with lesions or in autistic subjects, a set of areas common and critical for each empathy mode can be identified and summarized. The areas attributed to motor empathy include: superior temporal cortex (BA 22), inferior parietal lobule (BA 39, 40), inferior frontal cortex (BA 44, 45) (Blair, 2005; Carr et al., 2003), pre-motor and primary motor cortex (BA 4, 6) forming the mirror neuron system (Gazzola, Rizzolatti, Wicker, & Keysers, 2007; Rizzolatti, Fogassi, & Gallese, 2001) and the somatosensory cortex (BA 3). Emotional empathy areas include: amygdala, insula, ventrolateral frontal cortex (BA 47), superior temporal cortex and inferior prefrontal gyrus (BA 44) while cognitive empathy mainly covers the temporo-parietal regions (BA 39, 40), temporal pole (BA 38), paracingulate cortex (BA 32) and dorsal prefrontal cortex (BA 9, 10, 46). The challenge in analyzing fMRI data from long duration naturalistic stimulus presentation for activations attributable for complex construct like empathy is the inability to confirm with definiteness, the exclusive activation of a set of areas that are temporally coherent with the stimulus. The second issue that is often discussed in the context of brain activity during passive viewing is the possibility that participants drift into a non-stimulus processing exercise, thereby making it difficult to interpret activations. Hence, most studies using dynamic stimuli to date have adopted simple block designs, wherein the stimulus is on for a few tens of seconds, followed by a blank or no-stimulus stage in the off period and the same process repeated several times. The data are then analyzed using general linear model (GLM) followed by fixed-effects analysis for characterizing within-subject variations and random-effect analysis for characterizing variations between-subjects and to make population inferences. A popular methodology and software that are used to implement these approaches is Statistical Parametric Mapping (SPM (http://www. fil.ion.ucl.ac.uk/spm/software/spm8/)). Inter-subject correlation (ISC) is another method that has been promoted, when the interest is to look at specific brain areas and the correlations between subjects in these specific areas. In empathy literature, researchers have followed two broad approaches — one using GLM and the other using ISC. Methods like GLM allow one to make inferences about the spatial extent of activation in a particular epoch as compared to a control condition. In the ISC analysis, one can investigate the correlation patterns across subjects during events of interest. However, what is desirable, especially for the empathy response studies is to allow subjects to view stimuli such as movies in a free form and be able to analyze the resulting data to study both the spatial distribution and the temporal coherence of brain activity. A method which has found acceptance to isolate spatially distributed brain regions showing temporally coherent responses from fMRI data is the independent component analysis (ICA). This method has been applied to study resting-state and also task-based experiments (Calhoun, 2009; Esposito et al., 2006; Malinen, Hlushchuk, & Hari, 2007; Kim et al., 2009). We suggest that the ICA method is highly suitable for identifying functional networks for an evolving construct like empathy evoked from processing complex multi-modal stimuli like movies. We designed an exploratory study to investigate areas attributed to empathy using ICA methodology to isolate spatio-temporal

34

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

patterns of activity as subjects viewed video clips from three genres, namely, animation, Hollywood and Indian Hindi movie. The depiction and the narrative in the three diverse movie clips had emotional contagion to evoke empathic feelings in the viewers as indicated by the results from the debrief provided by the participants. We applied group-ICA method to identify probable activation across all subjects when viewing these complex stimuli. We start off with the following premises: (a) the movie clips evoke cognitive and emotional processing and (b) the activations recorded are in response to the empathetic relation the subject shares with the actor(s) and the context, both assumptions supported by the ratings provided by the fMRI subjects and also independent set of participants. With this assumption we compare the observed activation to the areas identified in the literature for different empathy modes – cognitive, emotional and motor. Further, the use of group-ICA method allows us to comment not only on the spatial distribution of the activity but also on the temporal coherence that is essential to label them as networks underlying various empathy modes. 2. Material and methods 2.1. Subjects Fifteen healthy multilingual college going adults (age range of 20–25 years) with written informed consent took part in this study. The gender breakup was as follows: 5 female and 10 male participants. The ethics committee of the International Institute of Information Technology, Hyderabad had approved the study. 2.2. Stimuli Participants viewed three 5–8 min clips with no audio from the Hollywood movie ‘The Green mile’, a popular Indian Hindi movie ‘Taare Zameen par’, and the animation movie ‘Up’ (the description of the movies is included in the Supplementary material). The sequences of events in each clip had actors conveying emotional states and a narrative that has the potential to evoke empathy. The actors in the Hollywood movie are mostly Caucasian with an African–American actor as the main character. The actors in the Indian Hindi movie were all of Indian ethnicity and the animation movie had human-like cartoons with Caucasian features. Although the order of presentation of the movie clips was not counterbalanced across subjects, any systematic effects like anxiety and fatigue can be ruled out as confounds in this study as the participants are reasonably experienced with fMRI procedures. The Hindi movie which was shown second in order was familiar to the participants while the other two movies were not. So following a sequence where two unfamiliar movies were shown first and last would ensure sustained interest without boredom. The movies were so diverse in presentation form, cinematography, visuals etc., that any carry-over effects from one clip to the other are expected to be minimal. Resting-state data was recorded for 5 min before the presentation of the stimulus, however analysis of the resting-state data is not included in here. 2.3. Rating the movies An independent set of forty participants were asked to rate the three clips on two main criteria: facial or body emotional expressions as depicted by the actors and second, the emotional context as comprehended from the short clips. The order of the presentation of the clips was randomized during this rating study. The rating was on a scale of 1–5, in ascending order of emotional contagion. The total score, out of a maximum score of 200, rated

for expressions and context for the animation movie was 70 and 68, respectively; 153 and 142 for the Indian Hindi movie and 176 and 178 for the Hollywood movie clip. Thus the Hollywood clip scored higher for both the parameters followed by the Indian Hindi movie with the animation coming in third. 2.4. fMRI imaging and preprocessing A 3T Philips Achieva scanner was set to the following configuration: Gradient echo, echo-planar images TR = 2 s, TE = 35 ms, flip angle = 90, acquisition matrix = 64  64, slice thickness = 5 mm, gap = 1 mm, 30 transverse slices, REC voxel MPS: 1.8  1.8  5 mm, and acquisition voxel MPS: 3.5  3.5  5.0 mm, was used to record functional images. A three-dimensional T1 weighted structural image using a fast field echo (FFE) technique and a Turbo Field Echo sequence was recorded for a duration of 4 min 46 s with a TR = 8.39 ms, TE = 3.7 ms, 150 slices, flip angle of 8, Field of view (FOV) = 250  230 mm and voxel volume: 0.98  0.98  1.0 mm. The number of images collected for each stimulus is as follows: Animation movie clip – 130 images, Hollywood movie clip: 250 images and Indian Hindi movie – 130 images. 2.5. Data analysis The data was preprocessed using SPM8 (Wellcome Department of Cognitive Neurology, UK (1)). The functional images were realigned to the first scan to correct for the head movement between scans. The anatomical image was co-registered with the mean functional image produced during the process of realignment. All images were normalized to a 2  2  2 mm3 Montreal Neurological Institute (MNI) template. Functional images were spatially smoothed using a Gaussian filter with a full-width at half maximum (FWHM) parameter set to 6 mm. 2.5.1. Group ICA method The ICA method follows a purely data-driven approach and hence does not require temporal signals to convolve with the hemodynamic response or the need to specify regions of interest. The ICA method has found particular importance in event-free or task-free mode of experiments to understand functional networks. Calhoun, 2009 gives an overview of the group ICA method for fMRI data highlighting how this method can be applied to make group inferences (Calhoun, Adali, Pearlson, & Pekar, 2001). In our study the smoothed images from SPM8 tool were analyzed using the group ICA tool, GIFT (Mind Research Institute (http://mialab.mrn. org/software/gift/index.html)). The minimum description length (MDL) criterion (Li, Adali, & Calhoun, 2007) was opted to reduce the images to be processed. Thirty ICs were extracted by this method and to check for the consistency of the estimates from the iterative algorithm FastICA (Hyvärinen & Oja, 1997) the ICASSO visualization toolbox provided in the GIFT software was used. This process is applied to evaluate the similarities for multiple runs of the FastICA algorithm (Correa, Adali, & Calhoun, 2007; Himberg & Hyvärinen, 2003; Himberg, Hyvärinen, & Esposito, 2004). A one-sample T test for each IC for the group of subjects was calculated and activations with a statistical threshold of p < 0.001 with voxel extent of 10 were considered. A check on correction for multiple comparisons was also done using the Family Wise Error (FWE) feature in SPM8, with p threshold set at 0.05 and voxel extent of 10. The severe thresholding of FWE filters out areas which could be relevant to the stimuli, however activation of the main areas are comparable to those obtained from the analysis using uncorrected p-value (see Supplementary Fig. S1 for the glass-brain masks with FWE and p-uncorrected for one IC from each stimulus). Talairach Daemon (http://www.talairach.org) (Lancaster, Rainey, Summerlin, Freitas, & Fox, 1997; Lancaster

35

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

et al., 2000) was used to label the Brodmann areas and regions, after conversion of the MNI coordinates from SPM tool into Talaraich coordinates using the approximation proposed icbm2tal(4) based on the work by Lancaster et al. (2007). The tables reported in this work show the MNI coordinates, but Brodmann areas estimated from the converted coordinates.

narrowed down the final number of ICs to 8–12 with temporally coherent responses in the prefrontal cortex, thalamus and certain areas of the parietal and temporal region. The selected independent components for each stimulus are presented individually and the probable empathy networks observed in the data are presented in the discussion section.

3. Results

3.1. Animation movie

Assuming complex processing while viewing natural scenes as in a movie, we expect simultaneous activation in more than one region of the brain. For selecting ICs, three methods were used, one by visual inspection of the time course signals constructed for each component and a second rigorous method of inspecting active regions for all the 30 components as labeled by the Talairach Daemon. The former method filtered out ICs with high frequency signals which indicate physiological artifacts like breathing. In the latter method the independent components with large voxel clusters labeled as activity predominantly in the white matter areas were filtered out. A third criterion applied was to consider ICs with temporally coherent and spatially separated areas rather than ICs with activations in a single region. The filtering steps

Of the 30 ICs, eight components with activations in one or more areas consistent with empathy modes reported in the literature were selected (Table 1). Sixteen representative axial slices overlaid with responses from the 5 ICs (ICs 1–5 in Table 1) are shown in Fig. 1. It can be seen that significant responses are present in the prefrontal, temporo-parietal junction and cingulate cortex along with smaller clusters in the thalamus region. The IC depicted in color red (Table 1, IC1) represents areas in the prefrontal region (BA 9) and inferior parietal lobule (BA 40) areas pertaining to cognitive empathy network, while the other two areas of activation in this IC, the prefrontal areas (BA 45) and superior temporal gyrus (BA 22) correspond to motor empathy network. The activity in the superior frontal gyrus (BA 8) of the frontal eye field region

Table 1 Independent component analysis results for the Animation movie clip: The MNI x, y, z coordinates, T values, brain regions and corresponding Brodmann areas, after conversion of the MNI coordinates into Talaraich coordinates of the selected independent components for the Animation movie clip showing activations in areas that form part of the empathy networks. The underlined Brodman areas or regions are also highlighted in the figure. ICs

Regions

Laterality

Cluster

T value

Coordinates (x,y,z) mm

Brodmann area

1

Inferior frontal gyrus

R

733

10.7

50 24 20

Middle frontal gyrus Superior parietal lobule Inferior parietal lobule Thalamus (VLN) Middle frontal gyrus Fusiform gyrus Middle temporal gyrus Superior parietal lobule Inferior parietal lobule

L R R L R R R L L

232 524

8.91 9.14 8.79 8.84 6.93 6.81 6.57 6.31 6.03

48 14 30 40 60 50 40 42 40 10 12 10 28 40 35 48 54 15 60 36 5 34 66 45 42 62 45

45 9 7 40

Superior frontal gyrus Angular gyrus Cingulate gyrus

R L R

752 106 33

10.32 10.27 5.97

2 32 45 48 68 30 2 22 35

8 39 24 10 7

2

3

4

24 16 21 68 33

Superior frontal gyrus

R

42

6.41

30 54 15

Precuneus

R

11

6

4

44 50

Postcentral gyrus

L

2229

16.88

Inferior parietal lobule Thalamus-pulvinar Lingual gyrus Precentral gyrus Precentral gyrus Middle frontal gyrus

R R R R R L

759 65 45 23

9.21 7.82 6.35 6.76 5.84 5.95

Angular gyrus Precentral gyrus Precentral gyrus

L L L

522 496

11.55 11.19 9.18

Precuneus Posterior cingulate Superior temporal gyrus Superior frontal gyrus Postcentral gyrus

L L R L L

348 27 115 14

9.53 7.82 8.14 7.17 6.16

6 46 35 2 50 25 52 58 20 2 26 50 50 14 45

Parahippocampal gyrus Paracentral lobule

L R

1552 111

12.15 8.29

2

7

Precentral gyrus Precuneus Superior frontal gyrus Precuneus Precentral Gyrus

L L L R R

337 380 27 17 12

8.53 8.06 5.71 7.01 6.98

46 4 35 18 74 45 20 42 35 4 60 35 52 2 30

8

Inferior parietal lobule Precuneus Inferior frontal gyrus

L L L

1040 58 20

10.19 8.91 7.07

5

6

27

VLN: ventral lateral nucleus, T-Value estimated at p < 0.001 with voxel cluster size threshold of 10.

54 36 16 10 62 60 38

28 40 50 28 72 10 16 40

50 5 0 30 40 10

50 66 30 50 0 45 44 24 35

28 32 15 22 45

62 28 25 2 68 25 42 42 5

9 37 22 7 40

2 40 18 4 6 10 39 6 9 31 23 39 8 3 36 31 6 7 9 7 6 40 31 46

36

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

Fig. 1. Activation map when subjects viewed Animation movie clip: five independent components (IC 1, 2, 3, 4 and 5 in Table 1) with activations in the prefrontal, parietal, temporal and premotor indicative of the probable activation of areas related to empathy mode network are shown. Sixteen axial slices are shown. The areas and other details are listed in Table 1. The color code is as follows: IC1: red, IC2: blue, IC3: green, IC4: pinkish purple and IC5: mustard. IFG: inferior frontal gyrus, SFG: superior frontal gyrus, PoG: postcentral gyrus, MFG: middle frontal gyrus, PrG: precentral gyrus. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

possibly from eye movements is shown in blue (IC 2) along with activity in angular gyrus (BA 39). Superior frontal gyrus (BA 10) and Precuneus (BA 7) are observed in IC 3 (green color). The purple cluster (IC 4) indicates a network of the dorsal pathway connecting visual cortex (BA 18) to the prefrontal region (BA 10) through the inferior parietal lobule (BA 40), primary somatosensory cortex (BA 2) and the pulvinar of the thalamus. Similar pathways were reported in a study examining the ‘where/how’ pathway of the visual attention network (Itti & Koch, 2001). Notable activations forming IC 5 (color-mustard) include the angular gyrus (BA 39), precuneus (BA 31), precentral gyrus (BA 6) and superior frontal gyrus (BA 8). The Brodmann area 10 was reported by ShamayTsoory et al. (2009) to be crucial for cognitive empathy, while BA 39 and BA 40 have been reported for both cognitive and motor empathy networks. The precentral gyrus (BA 6) identified in IC 5 has been studied by researchers of the mirror-neuron system and has been included as crucial area for motor empathy. Additionally, we found in IC 6, a large cluster of activity in the parahippocampal gyrus (BA 36), an area that has been implicated in scene categorization and recognition (Epstein & Kanwisher, 1998; Walther, Caddigan, Fei-Fei, & Beck, 2009) and also for episodic encoding (Hasson, Furman, Clark, Dudai, & Davachi, 2008b) of natural scenes.

The areas separated in ICs 7 and 8 include bilateral activity in the precentral gyrus (BA 6), precuneus (BA 7, 31), superior frontal gyrus (BA 9, 46), inferior parietal lobule (BA 40) and the middle temporal gyrus (BA 37). The superior frontal gyrus (9, 10, 46) areas are considered to be part of the cognitive empathy. Some areas like the superior frontal gyrus, inferior frontal gyrus and inferior temporal lobe areas are said to form a possible action perception network (Decety & Grèzes, 1999). 3.2. Hollywood movie Nine independent components indicating activity in gray matter regions are selected (Table 2). Sample axial slices of five components (IC 1–5, Table 2) with activations covering spatially separated temporally coherent areas, probably indicative of empathy response are shown in Fig. 2(a). The major activations separated in IC 1 (color red, Fig. 2a) are right hemispheric middle frontal gyrus (BA 46), the superior parietal lobule (BA 7) along with smaller cluster in the medial frontal gyrus (BA 8) extending to BA 6. Bilateral activity in parietal (BA 40), frontal gyrus (BA 8, 9), visual cortex (BA 18), precentral gyrus (BA 6), dorsal and ventral cingulate cortex (BA 23, 24, 31) are the areas isolated in IC 2 (color blue). The

37

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

Table 2 Independent component analysis results for the Hollywood movie clip: Nine of the total 30 ICs separated for the Hollywood movie clip possibly implicated in different empathy modes. The MNI (x,y,z) coordinates, T values, brain regions and corresponding Brodmann areas from MNI to Talairach coordinates of the selected independent components. The underlined Brodman areas or regions are also highlighted in the figure. ICs

Regions

Laterality

Cluster

T value

Coordinates (x,y,z) mm

Brodmann area

1

Middle frontal gyrus

R

983

13.84

52 22 25

Superior parietal lobule Posterior cingulate Inferior parietal lobule Medial frontal gyrus Medial frontal gyrus

R R L R R

1118 24 15 39

14.7 8.64 7.17 6.74 5.67

26 60 45 6 50 15 36 58 45 6 26 35 4 36 40

46 7 30 7 6 8

Inferior parietal lobule Middle frontal gyrus Middle frontal gyrus

R R R

348 811

15.19 14.99 13.56

52 48 45 44 28 40 42 50 10

40 8

Middle frontal gyrus

L

385

10.67

36 26 35

Inferior parietal lobule Cuneus Cuneus Cingulate gyrus Cingulate gyrus Cingulate gyrus

L R L L R L

158 34

10.27 9.74 5.98 9.3 6.64 5.78

9 40 18 18 23 31

Precentral gyrus Medial frontal gyrus

R L

12 22

7.71 6.35

34 10 60 2 24 65

Anterior cingulate Cingulate gyrus Cingulate gyrus Inferior frontal gyrus

R L R L

1535

15.71 14.86 12.07 11.9

4 20 25 4 12 30 10 18 30 50 20 15

Middle frontal gyrus Inferior parietal lobule Precentral gyrus Inferior frontal gyrus Inferior frontal gyrus Postcentral gyrus Postcentral gyrus

L L R R L R R

22 71 127

8.74 8.21 7.93 7.73 5.61 6.44 6.44

30 4 45 62 28 25 50 0 35 52 2 25 46 6 30 54 26 20 52 28 35

2

3

4

5

6

56

569

40 26

50 48 50 80 25 2 84 15 0 20 30 2 30 40 2 8 35

2

Inferior parietal lobule

L

2299

16.12

54

Middle frontal gyrus

L

587

13.07

48 22 25

Inferior frontal gyrus Thalamus-pulvinar Inferior parietal lobule Superior parietal lobule Inferior temporal gyrus Cingulate gyrus Paracentral lobule Cuneus Precentral gyrus

L R R R L L L R R

12.41 14.6 11.63 10.59 6.37 5.28 6.48 6.46 6.17

54 14 30 24 28 5 46 40 55 32 54 55 50 64 0 2 34 35 4 30 65 4 74 10 56 12 10

Cingulate gyrus Posterior cingulate Superior temporal gyrus Middle frontal gyrus Inferior frontal gyrus

L R L L L

493

Middle temporal gyrus Precentral gyrus Lingual gyrus Precentral gyrus

R L R R

75 63 16 29

83 863 198 17 14 28 11

833 465

17.78 9.3 15.76 12.65 10.69

42 45

2 6

58 25 56 20 48 58 20 38 22 30 48 26 20

11.15 9.86 6.85 6.66

50 60 25 50 4 30 20 76 10 42 18 40

Thalamus-MDN

R

423

8.54

Posterior cingulate Thalamus-pulvinar

R R

57 16

7.42 5.78

6 16 10 2 48 25 18 28 10

7

Postcentral gyrus Lentiform nucleus-putamen Thalamus-pulvinar Precuneus

R L R R

2464 91 11 25

16.05 7.18 7.9 7.42

12 32 65 26 10 0 14 30 10 34 62 40

8

Superior temporal gyrus

L

1174

12.23

Postcentral gyrus Cingulate gyrus Middle frontal gyrus

R L R

584 14 20

10.99 10.24 8.16

Precuneus

L

16

7.48

Superior parietal lobule Cingulate gyrus Middle occipital gyrus Inferior frontal gyrus Insula

R L R R L

383 696 96 44 28

9.15 8.68 9.3 8.76 5.8

Superior parietal lobule

L

18

6.53

9

MDN: medial dorsal nucleus. T-Value estimated at p < 0.001 with voxel cluster size threshold of 10.

48

16 5

56 26 20 2 2 40 44 44 5 14

70 35

26 68 45 16 8 45 44 76 10 60 6 25 42 10 10 30

54 55

10

24 6 6 24 24 32 45 6 40 6 9 9 40 2 40 46 9 40 7 37 31 6 23 44 31 23 39 9 45 39 6 18 4

23 3

19 22 40 24 46 7 7 24 19 9 13 7

38

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

areas covered in green (IC 3) are bilateral activations in the cingulate gyrus (BA 24), left inferior frontal gyrus areas (BA 45, 9) and right precentral gyrus (BA 6) extending to inferior prefrontal gyrus (BA 9). Activations in the Brodmann areas (2, 6, 9, 40, 45) indicate possible motor empathy pathway. The component, IC 4 in the color purple includes bilateral activity in inferior parietal lobule (BA 40), left middle frontal gyrus (BA 46, 9), inferior temporal gyrus (BA 37), small cluster of activation in precentral gyrus (BA 44) and the pulvinar of the thalamus region. Activation in BA 9, 46, and 40 (shown in IC 4) suggest possible cognitive empathy network while BA 44 is reported for emotional empathy (Shamay-Tsoory et al., 2009) and faces and/or bodies perhaps activate the fusiform gyrus (BA 37) area (Bartels & Zeki, 2004). The areas in mustard color (IC 5) show activation areas similar to IC 2 with bilateral activity in the superior and middle temporal gyrus (BA 39), cingulate gyrus (BA 31, 23), left middle and inferior frontal gyri (BA 9, 45) and precentral gyrus (BA 6). Four ICs (6, 7, 8 and 9 shown in Table 2) with activations in the thalamus region and insula along with other areas are shown in Fig. 2(b). The areas covered in color red include the medial dorsal nucleus (MDN) and a smaller cluster over the pulvinar of the thalamus region and the areas in blue (IC 7) show activity in the putamen and pulvinar areas. These areas may have a role in processing emotional scenes (Sabatinelli, 2011). The activations isolated in IC 8 (green) include high activation for the group in the superior temporal gyrus (BA 22), an area found to have strong inter-subject correlation in a study using movie as a stimulus (Hasson et al., 2004). In addition to STG the IC also shows responses in the postcentral gyrus (BA 40) and smaller clusters covering the middle frontal gyrus (BA 46) and cingulate gyrus (BA 24), a network that was also reported to participate in topdown attention control (Hopfinger, Buonocore, & Mangun, 2000). The IC in purple (IC 9) includes activation in the insula reported to be active for pain sensation and perception (Singer, 2006) and the middle prefrontal cortex (BA 9) reported for cognitive empathy.

3.3. Indian Hindi movie Nine independent components indicating activity in gray matter regions are selected (Table 3). Axial slices of the first five components listed in Table 3 (IC 1–5) are shown in Fig. 3(a). The first IC (IC1) shown in color red in the figure indicates gray matter activity in the superior parietal lobule (BA 7) and middle frontal gyrus (BA 8) while other areas in red are white matter. The areas covered in blue (IC 2) represent activity in the right hemisphere frontal gyrus region (BA 6, 9, 8), left superior frontal gyrus (BA 10), bilateral inferior parietal lobule (BA 40) and bilateral cingulate gyrus (BA 31, 23, 32). Significant activity in the right and left cingulate gyrus (BA 24) with smaller clusters over inferior parietal lobule (BA 40) and left inferior frontal gyrus (BA 44, 45) are marked in green. The independent component shown in pinkish-purple (IC 4) represent sizeable cluster in the left inferior parietal lobule (BA 40), right precuneus (BA 7) extending to inferior and superior parietal lobule (BA 40, 7) and smaller clusters in the medial frontal gyrus (BA 9, 6). The areas in mustard (IC 5) indicate responses mostly in the left hemisphere region in precentral gyrus (BA 9), medial frontal gyrus (BA 8) and temporal gyrus (BA 22, 21). The five ICs indicating activations in the prefrontal region suggest cognitive (BA 9, 10) and motor (BA 44, 45, 6) empathy modes as per the existing literature. The next four ICs (6, 7, 8 and 9 shown in Fig. 3(b)) indicate significant activations in the thalamus along with responses in the fusiform gyrus (IC 6: BA 37) and precentral gyrus (IC 7: BA 4, 6). Activation in insula (BA 13) extending from the middle/superior temporal lobe was seen in two ICs (IC 8 and 9) combined, postcentral gyrus (BA 3, 6), cuneus (BA 19) and smaller clusters in the frontal gyrus (BA 8). Insula has been reported to be active for pain sensation and perception (Singer, 2006), and in sensorimotor integration and emotional processing (Cauda et al., 2011). The somatosensory cortex (BA 3) and premotor (BA 6) observed in ICs 7, 8, 9 are areas associated with the mirror neuron system and the motor empathy network.

Fig. 2. Activation map when subjects viewed Hollywood movie clip. (a) five independent components (IC1, 2, 3, 4 and 5 in Table 2) with activations in the prefrontal, parietal, temporal and premotor possibly implicated in empathy mode networks are shown. Sixteen axial slices are shown. The areas and other details are listed in Table 2. The color code is as follows: IC1: red, IC2: blue, IC3: green, IC4: pinkish purple and IC5: mustard. (b) Activation map of areas in the limbic lobe when subjects viewed Hollywood movie clip: isolated in four ICs (IC6,7,8,9 Table 2) with activity in the thalamus region and insula specifically shown on 12 axial slices. Color code IC6: red, IC7: blue, IC8: green and IC9: purple. MFG: middle frontal gyrus, IFG: inferior frontal gyrus, IPL: inferior parietal lobe, STG: superior temporal gyrus, MDN: medial dorsal nucleus. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

39

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

Table 3 Independent component analysis results for the Indian Hindi movie clip: 9 ICs out of the 31 from the analysis of the Indian Hindi movie clip showing possible activation of the empathy networks. The MNI (x,y,z) coordinates in mm, T values, brain regions and corresponding Brodmann areas by converting MNI to Talairach coordinates of the selected independent components. The underlined Brodman areas or regions are also highlighted in the figure. ICs

Regions

Laterality

Cluster

T value

Coordinates (x,y,z) mm

1

Superior parietal lobule Middle frontal gyrus

R R

635 793

8.53 8.66

28 62 50 32 20 45

Precuneus

R

12

5.91

10

Superior frontal gyrus Middle frontal gyrus

R R

733

10.9 8.41

20 16 60 46 32 35

Superior frontal gyrus Cingulate gyrus Cingulate gyrus Superior frontal gyrus

R R L L

274

8.28 10.69 6.46 8.34

22 38 45 4 26 40 0 14 30 24 42 25

Inferior parietal lobule Inferior parietal lobule Cingulate gyrus

R L L

199 68 18

7.56 7.65 6.64

58 34 50 58 46 40 2 30 30

Cingulate gyrus Inferior parietal lobule Inferior frontal gyrus

R,L R L

1259 19 13

15.8 7.2 6.41

4 14 30 58 26 35 42 16 15

Precentral gyrus

L

11

5.81

46 4 35

2

3

4

5

6 7

8

9

86

Inferior parietal lobule

L

2181

13.3

Precuneus Inferior parietal lobule Superior parietal lobule Medial frontal gyrus Middle frontal gyrus

R R R L L

751

12.6 9.49 9.43 7.31 6.43

11 12

54

64 30

Brodmann area 7 8 31 6

44 45

32 62 40 42 48 55 26 62 50 4 40 25 28 0 45

9 8 31 23 10 40 40 32 24 40 44,45 6 40 19 40 7 9 6

Precentral gyrus

L

264

8.33

38 8 35

9

Superior temporal gyrus

L

329

9.28

46

Inferior parietal lobule Middle temporal gyrus Medial frontal gyrus Postcentral gyrus

L L L R

87 157 12

8.7 6.9 7.44 6.95

48 62 40 85 4 34 40 52 10 50

22 39 21 8 3

Thalamus-VLN Fusiform gyrus

R R

317

7.61 6.08

8 85 42 54

15

37

Cuneus precentral gyrus

L L

559 299

7.02 9.22

78 35 12 45

19

Precentral gyrus Middle frontal gyrus

L R

15

7.5 6.32

Middle temporal gyrus

R

359

Insula Precuneus Postcentral gyrus Precentral gyrus Precuneus Precuneus Anterior cingulate

R R R R R L R

Superior temporal gyrus

L

857

10.71

Cuneus Middle frontal gyrus Middle frontal gyrus Inferior parietal lobule

R R R R

117 26

8.67 6.37 6.31 6.15

27 99 20 10

20

12 48

56 15

48 2 35 52 6 45

9.01

54

8.9 8.79 8.01 5.53 6.39 6.02 6.34

40 14 5 8 70 20 62 12 25 58 0 40 4 52 35 0 60 35 4 22 25 52

34 5

42 20

16 88 25 52 14 40 54 6 45 58 32 25

4 6 6 22 13 31 3 6 7 7 24 13 19 8 6 40

VLN: ventral lateral nucleus. T-Value estimated at p < 0.001 with voxel cluster size threshold of 10.

4. Discussion The factors that contribute to empathy response of the viewer to the state of the actor or to the event that is being depicted in a movie are many-fold and can vary widely among viewers. To qualify each of the parameters of divergence, focused experiments have been conducted by researchers and the results integrated to deduce the combined effects by using short (20–30 s) stimuli or static images with emotional contagion (Adolphs, 2002; FusarPoli et al., 2009; Schulte-Rüther et al., 2007). A few studies have reported findings from a single movie shown for long duration where the focus was on studying activation changes in a particular region or area of interest (Hasson et al., 2004; Hasson, Yang, Vallines, Heeger, & Rubin, 2008a). The study reported in this article extends the previous studies using complex stimuli, especially

movies, by combining long duration stimuli and a diverse set of movies, to identify areas attributed to empathy response. Considering the diversity of the movie clips selected in this study, the goal was to acquire a macro-level understanding of responses to narrative empathy, a condition that involves the sharing of feeling and perspective-taking brought on by activities like reading, viewing, hearing, or imagining narratives of another’s situation and condition (Keen, 2006). The activation obtained is categorized as belonging to various empathy modes based on the assumption that the responses from viewer could be a mix of sharing the actor’s emotional state (motor and emotional empathy), taking perspective of the other person(s) and the situations shown while simultaneously being able to differentiate the self versus other experiences (cognitive empathy). The comparative ratings collected from participants on macro features such as expressions and

40

K. Vemuri, B.R. Surampudi / Brain and Cognition 94 (2015) 32–43

Fig. 3. Activation maps when subjects viewed Hindi movie clip. (a) five independent components (IC1, 2, 3, 4 and 5 in Table 3) with activations in the prefrontal, parietal, temporal and premotor indicative of probable empathy mode network areas are shown. Sixteen axial slices are shown. The areas and other details are listed in Table 3. The color code is as follows: IC1-red, IC2: blue, IC3: green, IC4: pinkish purple and IC5: mustard. -3(b): Activation maps of areas specific to the limbic lobe when subjects viewed Hindi movie clip: four ICs (IC 6, 7, 8,and 9 in Table 3) highlighting the activity in the thalamus and insula, premotor and superior temporal gyrus region are presented. Color code IC6: red, IC7: blue, IC8: green, IC9: purple. MFG: middle frontal gyrus, STG: superior temporal gyrus, IFG: inferior frontal gyrus, IPL: inferior parietal lobe. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

context of the clips were also used as reference. Group ICA method was applied as the experiment was a passive viewing non-event based design and also because the primary interest was to identify functional maps that evolve over time. The analysis focuses on the presence of activity evolving over a longer event rather than sudden response to a specific event or to a facial expression. In addition to activity corresponding to a selected set of 8–12 independent components (ICs), the method also isolated spatial maps or ICs with activations in single regions of the brain, like the visual cortex or mid-brain. Though the primary focus was on highlighting areas for empathy modes, we also indicate distinct pathways like the action-perception network or individual areas like the fusiform gyrus. The activity in the insula and thalamus in the ICs separated for the Indian Hindi clip and the Hollywood clip, additional figure is included. To check for false alarms (type I errors), we applied a conservative statistical threshold (p-value) of 0.0001 with a voxel extent of 10. To correct for multiple comparisons, we cross-checked the results using Family Wise Error (FWE) method provided in SPM8 on selected ICs. As shown in the Supplementary material when a conservative threshold enforced by FWE was applied, though the extent of activation reduced in all conditions, the areas of activity did not differ from that observed with p