Changes in pupil-size recruit brain areas involved in social cognition – an fMRI study

(1)

Msc Biomedical Sciences: Neurobiology

Cognitive Neurobiology & Clinical Neurophysiology

Research Project

__________________________________________________

Changes in pupil-size recruit brain areas involved in social

cognition – an fMRI study

__________________________________________________________________

by

Christina Diatchkova

10542485

July, 2014

36 ECTS credits

Jan-July 2014

Supervisor:

Examiner:

Dr. M.E. Kret, MSc

C.A. Bosman Vittini, PhD

(2)

2

Abstract

BACKGROUND: Various species aim to understand each other through different forms of communication varying from explicit verbal messages to implicit cues from another’s eyes. The eyes, and in particular the pupils, can reveal one’s emotional and mental state which is important for the understanding of another’s goals and intentions. A change in pupils is a signal that can be picked up by observers. The current study examined whether the activated brain regions during the observation of decreasing and increasing pupil sizes in an interaction partner correspond to the brain regions commonly seen in theory of mind studies.

METHODS: 34 participants watched in an MRI scanner videos of the eye region of interaction partners in which the pupil size decreased, increased or remained static. Between the videos the participants played a trust game in which they indicated how much money they would invest in the partner of whom the eye region was shown.

RESULTS: Observing changes in pupil size activated among other regions the superior temporal sulcus, the insula, angular gyrus and the postcentral sulcus. Most of these brain regions correspond to brain regions implicated in social cognition, including theory of mind.

CONCLUSION: During social interactions, people focus on another’s facial expression, and on the eyes in particular. When the pupil size in another person changes, people tend to wonder why and how the other one feels, figuring out what their emotional state is. Overall, pupils tend to activate areas involved in social cognition. Whereas increasing pupils activate areas involved in social cognition, observing someone with decreasing pupils recruits the somatosensory areas, suggesting that this intense signal is sensed in one’s own body.

(4)

4

Introduction

Social interactions

For centuries species try to communicate with each other through various forms that varies from explicit verbal messages to implicit cues. Evolutionary, interdependence of group-members is crucial because it provides increasing opportunities for mating, protection from predators, food, and opportunities for learning to cope with the environment (Emery, 2000).

People have a tendency to – unconsciously- mimic these body postures and facial expressions during communication (Kendon, 1970; Dimberg, Thunberg, & Elmehed, 2000). This facilitates the understanding of emotions and intentions between individuals, which can be brought back to the concept of empathy (Hatfield, Cacioppo, & Rapson, 1994).

Trust plays a great role in this relationship and is derived from body postures and facial expressions. Recently, Kret, Fischer, & De Dreu (2013) found that subjects trust individuals with dilating pupils more than individuals with constricting pupils, expressing this by investing more money in these individuals in a trust game.

The eyes reflect our inner state

The eyes facilitate communication and take part in (nonverbal) communication. Decades ago, pupils were thought to only play a role in modulating the light intensity that falls into the eye, i.e. pupils become smaller during great light influx and become larger with less light.

But besides modulating light influx, pupil size may reflect a range of inner states including arousal, attraction, cognitive effort, craving and it is associated with changes in attitude (Hess, 1965, 1975). These signals are important for the understanding of another’s goals and intentions (Emery, 2000). Studies conducted by Hess & Polt (1960) and Hess (1965) showed that pupil responses are influenced by the interest value of the stimuli. People with dilated pupils are generally perceived as more attractive, and elicit positive feelings, whereas people with constricting pupils are related to sadness and elicit negative feelings (Harrison et al., 2006; Hess, 1975). A larger pupillary diameter at the subject implies a greater interest, especially for the sex one is interested in (Hess, 1965). The changes in pupil size can tell much about one and can be picked up by others, stating that the changes in pupil size can serve as a social signal.

When we look someone in the eyes, it is often seen that our own pupils change as well (Harrison et al., 2006). This mirroring whereby subject’s own pupils become the size of the observed pupil is called contagion. Mirroring another’s pupil size may enhance empathic appraisal and understanding of their feelings and actions. This observed change in pupil size is believed to modulate activity in brain regions that are central to social cognition1. Also, it modulates regions that are implicated in neurobiological mechanisms (Harrison et al., 2006).

1

These changes work through different pathways. Pupillary constriction is mediated by the parasympathetic activity, whereas pupillary dilation is mediated by the sympathetic activity (Barbur, 2004, for the whole pathway, see Appendix 5). Both changes are thought to be regulated by specific nuclei in the brain, called the Edinger-Westphal nuclei and the locus coeruleus, which innervate the muscles of the iris (Kozicz, Bittencoirt, May, et al., 2011).

(5)

5

Neural circuitry of social cognition

Several brain regions are implicated in this social cognition and emotional processing. These are medial prefrontal cortex (mPFC), the insula, superior temporal sulcus (STS) and the amygdala (Kawashima, Sugiura, Kato, et al., 1999).

The mPFC is active during observing complex intentional movements (Castelli, Happé, Frith, & Frith, 2000), self-generated thoughts (McGuire, Paulesu, Frackowiak, & Frith, 1996) and when making mental state inferences (Mitchell, Macrae, & Banaji, 2006).

The insula is activated when people see fearful and disgusted facial expressions (Phillips, Young, Scott, et al., 1998). The STS is responsible for the processing of faces and cognitive demands, and deciphering the beliefs and perspectives of others (Hein, & Knight, 2008). Furthermore, this region is sensitive to the eye gaze of another person (Perrett, Hietanen, Oram, & Benson, 1992).

The amygdala is involved in processing emotions and its activity is enhanced for emotionally laden or unfamiliar faces (Breiter et al., 1996; Dubois et al., 1999). Damage to the amygdala impairs the social and empathic behavior and the recognition of facial expressions of sadness (Adolphs, & Tranel, 2004).

Several fMRI studies have also focused on the impact of pupil size on brain regions. For example, Demos, Kelley, Ryan, Davis, & Whalen (2008) found that pupil dilation causes more activity in the amygdala than pupil constriction. Furthermore, pupil dilation activates the superior & middle frontal gyrus, lingual gyrus, inferior parietal lobe and the thalamus whereas pupil constriction activates the precuneus, inferior frontal gyrus and the inferior occipital gyrus.

Amemiya & Ohtomo (2011) also found activation of the amygdala when pupil sizes changed, together with the insula, putamen, perirolandic area, middle frontal gyrus and superior parietal gyrus.

Harrison et al. (2006) found that largest vs. smallest pupils showed activation in the amygdala, STS, frontal operculum, insula and dorsal anterior cingulate, whereas smallest vs. largest pupils showed activation in the Edinger-Westphal nuclei and angular gyrus. Furthermore, they found that faces with pupil constriction are perceived as more negative than faces with pupil dilation, even if they both showed a sad expression.

Previous research has focused on static pupil sizes (large vs. small). However, in daily life, people deal with moving pupils instead of pupils of constantly the same size. Thus, the current study focuses on this concept by using moving pupils what makes the pupils more real and naturally. Theory of Mind

Since changes in pupil size are associated with one’s own mental state, which brain areas are activated in the others when one looks at a person whose pupils change? Several studies have stated that by looking at someone, a certain network becomes active. The network responsible for this is called the Theory of Mind (ToM). ToM creates the capacity to represent mental states, such as thoughts, beliefs, feelings and plans (Premack, & Woodruff, 1978; Dodell-Feder, Koster-Hale, Bedny, & Saxe, 2011). The consideration of others’ mental states helps people in several activities such as flirting, cooperating, playing games, making moral judgments and as mentioned before, it is evolutionary important (Dufour, Redcay, Young, Mavros, Moran, et al., 2013).

The ToM task is developed to individually localize regions that are involved in social cognition and empathy (Dodell-Feder et al., 2011). Several fMRI studies have revealed a group of brain regions that are activated during a ToM task of belief reasoning (for a review, see Carrington, & Bailey, 2009). These regions include the temporo-parietal junction (TPJ), the STS, mPFC and the medial

(6)

6

precuneus (PC; Dufour et al., 2013; Saxe, & Kanwisher, 2003; Vollm, Taylor, Richardson, Corcoran, Stirling, et al., 2006).

The TPJ is believed to be involved in reasoning about the contents of another person’s mind (Saxe, & Kanwisher, 2003) and the precuneus in involved in visuo-spatial imagery and perspective taking (Cavanna, & Trimble, 2006). The ToM localizer has been introduced to emphasize activations in these regions and to link it to the changes in pupil size.

In the current study, we focused on the impact of pupil size on another one’s ToM network. What makes this study unique is that we used moving pupils instead of static. We were curious to know what signal moving pupils give to others. Using moving pupils is more naturally, since this also occurs in daily life.

We hypothesized that seeing one’s pupils changing would activate other’s brain regions which are involved in social cognition, in particular the ToM network. We expected to see these brain regions especially activated during increasing pupils, since they elicit positive feelings and trust.

(7)

7

Methods

Subjects

Forty-one healthy participants (21 females, mean age [± SD] 22.76 [± 2.76] years) were recruited for the study. Data of 7 participants were dropped due to failure of recording fMRI data properly or due to too much movement (more than 1.5mm) of the subjects in the MRI scanner, leaving a total of 34 participants.

Inclusion Criteria

Participants were born and raised in the Netherlands, were preselected based on no current or past psychiatric disorders, did not wear glasses (contacts were aloud), and had no or removable metal in their body (excluding threats behind the teeth). Two weeks after the scanning session participants received the Beck Depression Inventory (BDI, Beck, Steer, & Garbin, 1988). Unfortunately, two participants had symptoms of mild depression. We have excluded one of them due to a divergent activation pattern in the brain. The brain activity of the other participant did not differ from the rest of the group.

Experimental setup

Pupil presentation

The stimulus material consisted of nine female and nine male photos derived from the validated Amsterdam Dynamic Facial Expression Set (ADFES; Van der Schalk et al., 2011). We chose the ADFES for the Caucasian faces as these were taken from the Dutch university students and therefore closer to the participants.

Pictures were standardized in Adobe Photoshop (Adobe systems), had neutral expressions, turned to greyscale and cropped to reveal only the eye region. Eye-white, iris and pupil were erased, average luminance of each photo was calculated and adjusted to the mean. The eyes were then filled with new eye-white (sclera) and irises, based on one iris pair from one picture, and an artificial pupil was added. The sclera was made brighter than the eye white in the outer edges of the eye to emphasize the convex shape of the eye.

Each video consisted of one pair of eyes. After static presentation for 1500ms, the partner’s pupil which was shown in the picture increased (140% of the original size), decreased (60% of the original size) or remained static within the physiological range of 3-7mm for another 1500ms. We changed the pupil size only after 1500ms because it always takes time for a pupil to adjust to new information. In the first 1500ms the pupil automatically decreases and we don’t want to have bias. In the last 1000ms the pupils remained in the same condition as before. To reduce artificiality, a slightly trembling corneal reflection was added. The eye images appeared life-size on the computer screen. The static pupils were added as a control condition.

Trust game

Participants were instructed to play an economic game. They did not know that this game was intended to measure trust. After each stimulus, participants were asked how much money they would like to invest in the partner of whom the eye region was shown. They were told that for each decision they had 6 Euros and they could choose between 0, 2, 4, or 6 Euros to invest in the other player. A time window of 2000ms was given to make their choice (Fig. 1). Three practice questions verified that participants understood the game and the consequences of their decisions. The overall duration of the task was between 60 and 80 minutes divided in three runs, each run

(8)

8

containing 54 trials (18 eye pairs x 3 conditions). Between the runs, two localizers were performed to map empathy and threat-related networks.

The partner’s decision derived from back-transfer-decisions of fifteen students in the role of trustee (13 females, mean age 24 years), collected prior to the experiment. For each trial we randomly drew a decision to calculate participant’s earnings after the experiment was over (no feedback between trials). Participants were informed that we had recordings of their partners and that prior to making decisions they would be shown short video clips of these recordings to enable them to form an impression of the partners. After reading the instructions and practicing with the three questions, the participants pressed a button to start the task. In addition to the investment decision, besides recording participant’s pupil responses, their heart-rate and skin conductance was measured.

Fig. 1: The presentation of the trial outline.

Procedure

The participants were met a couple of days before scanning to instruct them about the procedure and to complete the medical screening. On the testing day, participants provided informed consent in the laboratory.

Prior to the task, participants received the Interpersonal Reactivity Index, to measure one’s empathy (IRI; with Empathic Concern (EC) and Perspective Taking (PT) scales; Davis, 1980); the State-Trait Anxiety Inventory for mental disorders (STAI; Spiegelberger, 2010) and the Liebowitz Social Anxiety Scale, to measure whether participants suffer from social anxiety disorders (LSAS; Heimberg, Horner, Juster, et al., 1999). The BDI was sent two weeks after the scanning, because we assumed some participants suffered from a depression.

We have added these questionnaires, because it is of interest to know if the participants suffered from any depression or anxiety disorder. It is known that people with mental disorders have a dysfunctional brain network in areas such as the amygdala (Price, & Drevets, 2011). Their pupils react differently to light influx, thus they could also react differently when seeing one’s pupil changing (Wang, Fan, Zhao, & Chen, 2014).

Next, two electrodes were attached to participants’ left ring and index finger. After entering the room, a pulse oxidation signal (PO) was recorded from the middle finger. The breathing rate was measured with a band around the participants’ chest from Philips Achieva. First, a sham scan was implemented to ensure the magnetic field was homogeneous. Next, the T1 anatomical scan followed, during which the participants performed a nine-point calibration of the eye-tracking system. Eventually, the functional scans were recorded.

(9)

9

Each trial started with the presentation of a Fourier-scrambled image for 2500ms followed by 3000ms jitter. Next, a fixation cross was presented for 800ms, followed by the pupil presentation video. The videos were viewed in the MRI scanner on a back-projection screen via a mirror system attached to the head coil.

The participants implemented five task units, starting with the trust game. The second task was either the ToM-localizer achieved from the Saxe lab (Dodell-Feder et al., 2011) or a new developed Threat-localizer. After that, a second trust game was played, followed by the remaining localizer and ending with the third run of the trust game.

After the scanning session, the participants implemented four more tasks in a separate test room. Here they rated the pupils shown before on attractiveness and trustworthiness, determined the state of mind a person on a photograph was in (Baron-Cohen, Wheelwright, Hill, Raste & Plumb, 2001), and were instructed to draw pupils in a happy and angry face (Hess, 1975).

Theory of Mind localizer

The ToM localizer is a widely used task to identify brain regions involved in theory of mind and social cognition (for more information, see http://saxelab.mit.edu/). We have translated this task into Dutch since we had only Dutch participants. The task consisted of twenty stories, in which ten of them described a situation in which someone held a false belief. Within these false-belief stories, a subject performs an action that is based on a belief that is false. Participants had to indicate whether the answer was true or false.

Example: Larry chose a debated topic for his class paper due on Friday. The news on Thursday indicated that the debate had been solved but Larry never read it.

Question: When Larry writes his paper he thinks the debate has been solved. True False

The other ten stories were false-photograph photos, which described situations with a false or outdated representation of the world.

Example: A large oak tree stood in front of City Hall from the time the building was built. Last year the tree fell down and was replaced by a stone fountain.

Question: An antique drawing of City Hall shows a fountain in front. True False

These types of stories require the participant to deal with incorrect representations about the world and are therefore matched in their difficulty, logical complexity, and inhibitory demands, but differ in the need to think about someone’s thoughts. Contrasting these stories localizes regions that are recruited for processing mental states (Dodell-Feder et al., 2011).

The localizer was presented in a mini-block design, starting with either the belief or false-photograph story. Prior to the 10s presentation of the story, a fixation cross was presented for 12s. After the story, the participant had 4s to make the decision whether the story was true or false. Threat localizer

The threat localizer consisted of ten threatening and ten non-threatening stories. The stories were rated for threat sensation on a scale from 0 to 10 (0 being non-threatening, 10 being very threatening) and subsequently for probability (0 being improbable, 10 being highly probable) to validate the stories. The ten most threatening stories had a mean score of 8.76 ± 0.73 with a

(10)

10

probability of 6.04 ± 0.83, while the ten non-threatening stories had a mean score of 0.86 ± 0.68 with a probability of 6.94 ± 0.57. Participants read the stories on the projector screen over a mirror, which was mounted on the MRI head coil. They had to indicate via a button box, which was held in the right hand, whether they found the story threatening or not.

Example threat story: You’re at the bank to pick up some money. Just when you receive the money, a man with a gun shows up and yells to give him your money, but you’re so scared that you can’t move.

Question: The situation is threatening. True False.

Example non-threat story: The light on the ceiling doesn’t work anymore. You take a ladder out of the closet and change the bulb easily. You put the ladder back and look at the result.

Question: The situation is threatening. True False.

Data acquisition

Eye-tracking

The pupil size was constantly sampled every 16ms with Eyelink equipment and down-sampled to 100ms timeslots. The outliers (pupil size between two time-samples changed > 2 SD) were removed and gaps smaller than 250ms were interpolated. We smoothed the data with a 10th order low-pass Butterworth filter. A 500ms baseline was subtracted from each sample during stimulus presentation. The baseline was set to 500ms prior to the changes in partner’s pupil-size (i.e. 1000-1500ms).

In addition, skin conductance was collected with Versatile Stimulus Response Registration Program (Vsrrp, Molenkamp, 1998), via a pair of curved Ag/AgCl electrodes (dimensions 20x16mm) placed on the medial phalanges of the ring and middle finger of the non-dominant hand. Skin conductance level is down-sampled to 100ms time points, log transformed to correct for skewness, and analyzed over the same period as participant’s pupil responses and being filtered for skin conductance responses (the number of responses over the last 2s of stimulus presentation (minimum distance through to peak = 200ms, maximum distance = 4000ms, minimum response amplitude = 0.2 micro Siemens)).

Heart-rate was measured with three 3M Red Dot disposable ECG electrodes placed around the heart and down-sampled to 500ms time points.

fMRI scanning and data analysis

Whole-brain fMRI data were acquired on a 3T Achieva magnetic resonance scanner quipped with a standard head coil. Structural images were obtained with a gradient echo-planar T1 sequence (T1 turbo field echo, 240*188mm2field of view (FOV), comprising a full brain volume of 220 slices (1mm slice thickness). Volumes were acquired continuously with a repetition time (TR) of 8.2s and an echo time (TE) of 3.73ms (8° flip angle (FA), sagittal orientation).

Next, five runs (each lasting 10-12.3s) of functional data were collected (2.0s TR, 27.63ms TE, 192*141.24mm2 FOV, 39 slices, 3.3mm slice thickness, 76.1° FA, sagittal orientation) covering the whole brain. fMRI data were analyzed using the FMRI Expert Analysis Tool (FEAT) in Functional MRI of the Brain (FMRIB) Software Library (FSL) version 6.0 (Oxford Centre for Functional MRI of the Brain Software Library (www.fmrib.ox.ac.uk/fsl)).

(11)

11

Preprocessing consisted of motion and slice time correction, spatial realignment, normalization to the Montreal Neurologic Institute template, spatial smoothing with a Gaussian 5mm full-width-at half-maximum kernel and high-pass temporal filtering with a cut-off of 100s. After preprocessing, individual and group-level statistical analyses were performed using the General Linear Model as implemented in FSL. The conditions were separately modeled by convolution with a double-gamma hemodynamic response function in a general linear model. Finally, the functional data were pre-whitened using FSL (Woolrich, Ripley, Brady, & Smith, 2001). For the pupil presentation, we modeled the conditions ‘increase’, ‘decrease’, and ‘static’ pupils. In the false-belief stories, the conditions were ‘ToM’ and ‘neutral’ and in the Threat-localizer the conditions were ‘threat’ and ‘neutral’.

We performed a first level analysis for all runs and the localizers. Prior to the analysis, the data information had to be converted into files that are readable for FSL. This has partly been done in MatLab by running several handmade scripts. Next, the anatomical scans had to be extracted from the non-brain parts, via the Brain Extraction Tool (BET) function implemented in FSL.

Next, the data was imported in FSL. The runs and localizers were selected one by one to make contrasts. The relevant threshold was cluster based corrected with a P threshold of 0.05.

The first level analysis shows how well the registration has been done if and how much the subjects moved. Furthermore, it shows the activation areas in the brain during the changes in pupil size (increase, decrease, static pupils and localizers).

Afterwards, the runs were pooled per subject using the FMRIB's Local Analysis of Mixed Effects (FLAME) stage 1, in which relevant lower-level contrasts were combined. Here, the runs were averaged for every subject to achieve the average activation during the pupil changes (within subject analysis).

Finally, these within-activations per subject were analyzed across all subjects (between subjects analysis) to get a final average activation during the conditions. The highest whole-brain activation was observed during each of the three conditions by contrasting activation in voxels during ‘increase’ being stronger active than during ‘static’ (Incr>Sta), ‘decrease’ stronger than ‘static’ (Decr>Sta) ‘increase’ stronger than ‘decrease’ (Incr>Decr). The effect for each contrast was calculated with FEAT. All cortical regions with a height threshold of Z=2.3 and a cluster probability of p<0.05 were reported. This is corrected for whole-brain multiple comparisons by using Gaussian random field theory (GRFT; Worsely, 2001). Activation maps are overlaid over the MNI-2mm brain and regions are determined using the Harvard-Oxford Cortical Structural Atlas that accompanies FSL.

(12)

12

Results

Participants

Table 1 shows the information from all subjects, such as sex, age, and their scores on the questionnaires.

N Min Max Mean Std. Deviation

Age 41 19 32 22,76 2,764 BDI 36 0 18 4,08 3,988 State 27 36 57 46,30 4,445 Trait 35 43 56 48,66 3,412 EC 40 0 6,57 4,686 1,275 PT 40 0 6,71 4,814 1,203 LSAS_fear 40 0 1,42 0,519 0,334 LSAS_avoid 40 0 1,25 0,486 0,308

Table 1. Characteristics of subjects BDI = Beck Depression Inventory

State & Trait = two subscales of State-Trait Anxiety Inventory

EC = empathic concern, PT = perspective taking, subscales of Interpersonal Reactivity Index LSAS = Liebowitz Social Anxiety Scale

Not every participant filled in the questionnaires, because some of them have forgotten to do this. Several of the questionnaires were given before the test, the other ones were given afterwards. The average score of the BDI questionnaire was 4, what states that the group has minimal depression (Beck, Guth, Steer, & Ball, 1997). The average STAI score was 46-48, while the cutoff score for anxiety in this questionnaire is 54-55 (Kvaal, Ulstein, Nordhus, & Engedal, 2005). Therefore, we can conclude that the group is not anxious. For the IRI, the average score per question is among 3.5 (the half of the seven subscales). This group has an average of 4.6 per question, so it can be stated that the participants are empathetic towards other people. The average score for the LSAS is 0.5, concluding that the group does not have any fear or avoidance.

Localizers

The brain activations during the ToM and Threat localizers did not reach the significant threshold and have not been used for further analyses and conclusions. Therefore, we tested to see brain activation in response to pupils in the regions that are mentioned in previously literature on ToM. We used the MNI coordinates mentioned by Saxe & Kanwisher (2003). These were [-54 -60 21] for the left TPJ, [51 -54 27] for the right TPJ, [-9 -51 33] for the precuneus, [-57 -27 -12] for the left anterior STS and [66 -18 -15] for the right anterior STS.

Changes in pupil size

The significant activations we found was seen in the contrasts ‘increase vs static’ and ‘decrease vs static pupils’. The contrast incr>decr did not reach the threshold. Figure 2 shows the activation pattern for incr>stat and figure 3 decr>stat with the additional cluster tables 2 and 3.

(13)

13

Fig. 2: Average brain activity for the contrast incr vs. stat, cluster based correction with a threshold of p<0.05

Cluster# Voxels P MNI coordinates Z-score

1 r. lateral occipital cortex (BA18/19) ₂₃₂₈ _1.64e-08 _{(50, -62, 2)} _4.45

↓r. lateral occipital cortex (BA18/19) _{(42, -62, 2)} _4.03

↓ r. temporal occipital fusiform cortex (FFA) _{(36, -56, -14)} _3.83

↓ r. lateral occipital cortex (BA18/19) _{(54, -64, 8)} _3.78

↓ r. inferior temporal gyrus (BA20) _{(48, -46, -18)} _3.76

↓ r. lateral occipital cortex (BA18/19) _{(42, -64, 8)} _3.72

2 l. lateral occipital cortex (BA18/19) ₁₆₇₅ _1.37e-0.06_{(-40, -68, 8)} _3.93

↓l. lateral occipital cortex (BA18/19) _{(-44, -74, 0)} _3.93

↓l. lateral occipital cortex (BA18/19) _{(-40, -70, -6)} _3.84

↓l. inferior temporal gyrus (BA20) _{(-46, -52, -16)} _3.54

↓l. middle temporal gyrus (MT/V5/BA21) _{(-42, -58, 8)} _3.36

Table 2. Clusters above the Z score of >2.3 and a p-value <0.05 for the contrast incr vs stat

Cluster 1: consists of lateral occipital cortex – inferior & superior division, middle & inferior temporal gyrus - temporooccipital part, (temporal) occipital fusiform gyrus, angular gyrus.

Cluster 2: consists of lateral occipital cortex - inferior & superior division, middle & inferior temporal gyrus - temporooccipital part, angular gyrus, (temporal) occipital fusiform gyrus, supramarginal gyrus - posterior division.

↓ = Subpeaks of the clusters; r = right; l = left; FFA = fusiform face area; MT = middle temporal; BA= Brodmann area.

(14)

14

1 r. Postcentral sulcus (BA 3,1 &2) ₉₀₉ _{0.00254 (56, -18, 40)} _3.97

↓r. postcentral sulcus (BA 3,1 &2) _{(62, -16, 36)} _3.9

↓r. central opercular cortex (insula) _{(64, -18, 16)} _3.57

↓r. central opercular cortex (insula) _{(62, -14, 18)} _3.51

2 l. postcentral sulcus (BA 3,1 &2) ₅₀₇ _0.0385 _{(-48, -28, 46)} _3.38

↓l. postcentral sulcus (BA 3,1 &2) _{(-60, -20, 34)} _3.24

Table 3. Clusters above the Z score of >2.3 and a p-value <0.05 for the contrast decr vs stat

Cluster 1: consists of postcentral sulcus, supramarginal gyrus – anterior & posterior division, central opercular cortex, planum temporale, parietal operculum cortex, superior temporal sulcus, superior parietal lobule

Cluster 2: consists of postcentral sulcus, supramarginal gyurs – anterior & posterior division, superior parietal lobule ↓ = Subpeaks of the clusters; r = right; l = left; BA = Brodmann area.

We expected to see the strongest brain activity in the contrast ‘incr>decr’, because the thought was that decreased and increased pupils would cause contrary results. Apparently, this was not the case, and we hypothesized whether not the difference in pupils was responsible for the brain activity, but the movement itself in the pupils. We tested our new thought: moving pupils versus static pupils would give the strongest activation pattern (mov>stat; Fig. 4, with the additional cluster table 4).

(15)

15

1 r. lateral occipital cortex (BA18/19) ₁₅₁₄ _1.57e-05_{(50, -62, 2)} _4.62

↓ r. inferior temporal gyrus (BA20) _{(48, -46, -18)} _3.91

↓ r. middle temporal gyrus (MT/V5/BA21) _{(52, -58, 8)} _3.66

↓ r. lateral occipital cortex (BA18/19) _{(42, -62, 2)} _3.56

↓ r. temporal occipital fusiform cortex (FFA) _{(38, -52, -14)} _3.53

↓r. lateral occipital cortex (BA18/19) _{(60, -70, 0)} _3.49

2 l. lateral occipital cortex (BA18/19) ₈₀₉ _{0.00304 (-42, -76, 0)} _3.59

↓l. lateral occipital cortex (BA18/19) _{(-44, -72, -6)} _3.35

3 r. postcentral sulcus (BA 3,1 &2) ₆₅₆ _0.0114 _{(52, -20, 42)} _3.88

↓ r. postcentral sulcus (BA 3,1 &2) _{(62, -16, 36)} _3.61

↓r.central opercular cortex (insula) _{(62, -18, 16)} _3.6

↓ r. supramarginal gyrus _{(68, -28, 26)} _3.09

↓r.planum temporale _{(62, -34, 20)} _3.04

Table 4. Clusters above the Z score of >2.3 and a p-value <0.05 for contrast mov vs stat

Cluster 1: consists of lateral occipital cortex - inferior division, middle & inferior temporal gyrus – temporooccipital part, temporal occipital fusiform cortex, temporal fusiform gyrus – posterior division, angular gyrus.

Cluster 2: consists of lateral occipital cortex – inferior & superior division, middle temporal gyrus – temporooccipital part, occipital fusiform gyrus.

Cluster 3: consists of postcentral gyrus, supramarginal gyrus – anterior & posterior division, central opercular cortex, parietal operculum cortex, planum temporale, superior temporal sulcus – posterior division, angular gyrus.

(16)

16

Discussion

Pupils can serve as a social signal to other people, revealing one’s inner state and enhancing empathic appraisal and understanding of feelings and actions.

In this study we investigated whether one’s pupillary changes are involved in activating other’s ToM network. We expected to see overlap in the ToM network and brain activity when participants observed increasing pupils as compared to static pupils, since they elicit positive feelings, which is more related to ToM than decreasing pupils.

Findings

As the results show, the contrast increase vs. static pupils caused the strongest activity, i.e. the clusters exist of the most voxels. This is in accordance with the study of Demos et al. (2008), as that study showed that big pupils yielded greater activity than small pupils. A possible explanation is that there are multiple reasons for pupils to become increased. One could be attracted to someone, excited or just happy. Increasing pupils are interpreted as an indicators of arousal or interest, and give an alert response related to biologically, and probably socially, relevant cues (Hess, 1965; Steinhauer et al., 2004). This can cause more activation in the brain.

The contrast has nearly the same pattern as the contrast movement vs. static pupils. The first two clusters of incr>stat overlap with the two clusters of mov>stat, whereas the third cluster can be contributed to the activity of decr>stat.

The first cluster exists of the lateral occipital cortex, inferior and middle temporal gyrus, the occipital fusiform gyrus (fusiform face area, FFA), and the angular gyrus.

The FFA is known to be more responding to pictures of faces in comparison with pictures of objects and has an important role in face recognition (Rossion, Caldara, Seghier, Schuller, Lazeyras, & Mayer, 2003; for a review, see Kanwisher, & Yovel, 2006).

Lahnakoski, Glerean, Jaaskelainen, et al. (2014) found that when individuals adopt a similar psychological perspective during natural viewing, one of the brain regions that becomes active is the lateral occipital cortex, stating that synchrony in this region is an important mechanism supporting shared understanding of the environment.

The middle temporal gyrus (also known as MT or V5) is known to take part in the perception of visual motion and the guidance of eye movements (Born, & Bradley, 2005). The inferior temporal gyrus is thought to be the core of the neural system for face perception and processes the significant information picked up from the face (Hoffmann, & Haxby, 2000; Haxby, Hoffman, & Gobbini, 2000). In our study, the significant information comes from the eyes. Both temporal gyri are also involved in several different cognitive functions, such as sensory processing and visual perception (Kuroki, Shenton, Salisbury, et al., 2006).

A meta-analysis of neuroimaging studies concluded that amongst other regions, the angular gyrus plays a role in deception or in delusion stories, such as in the theory of mind task. This brain region showed an increased activity in social interactive deception studies (Lisofsky, Kazzer, Heekeren, & Prehn, 2014). Watabe, Ban, & Yamamoto (2011) showed that this region is active during relevant information about someone´s trustworthiness. After several runs, participants knew that they had to make an investment decision after the stimulus, so during the stimulus itself they were already thinking about their investment choice based on the trustworthiness in the partner’s eyes.

The second cluster includes, in addition to the brain regions in the first cluster, the supramarginal gyrus. This brain region is involved in stimulus driven attention, seeing desired outcomes,

(17)

17

overcoming emotional egocentricity bias in social judgments and helps to distinguish our own emotional state from that of other people (Scalf, Ahn, Beck, & Lleras, 2014; Aue, 2014; Silani, Lamm, Ruff, & Singer, 2013). During the stimuli, participants pay attention to the eyes, try to empathize in the other and think about the trustworthiness of the partner, making social judgments. Thus, it is logical that we see activation in this area.

The third cluster activated during the contrast mov>stat is driven by the decrease of the pupils. It showed a different set of brain regions: postcentral sulcus, central opercular cortex (insula), planum temporale, parietal operculum cortex, and the superior temporal sulcus.

The postcentral sulcus is known to be the location of the somatosensory primary cortex and is responsible for the sense of touch (Cui, Arnstein, Thomas, Maurits, Keysers, & Gazzola, 2014). It has also a sensory function of the face. Patients with excision of this region had a severe sensory deficit of the face (Taylor, & Jones, 1997). Apparently, participants seem to feel a sense of touch by looking at eyes with decreasing pupils. Decreasing pupils are more intense and could even be seen as intimidating, since they elicit more negative feelings (Schrammel et al., 2009).

The insula has many functions, but has often been reported as a region that processes the perception and experience of emotions (Wager, Davidson, Hughes, Lindquist, & Ochsner, 2008). In addition, this region has been also found to be active during reappraisal of other person’s intentions (Grecucci, Giorgetta, Bonin, & Sanfey 2013). In the current study, participants try to imagine the reason for the changes in pupil size in order to make an investment decision, which means that they have to think about other’s intentions.

Although the planum temporale and the parietal operculum are known to take part in auditory and vestibular processing, Cornette et al. (1998), Sadato et al. (2005) and Antal, Baudewig, Paulus, & Dechent (2008) found in PET/fMRI studies that these regions are also activated using various visual motion stimuli.

The superior temporal sulcus has many roles in cognition, including face processing, motion perception, and emotional expression (Hein, & Knight, 2008; Iitaka, 2012). It forms the visual analysis of faces together with the inferior temporal gyrus, but the STS is more involved in the representation of changeable aspects of faces, such as pupils in our study (Hoffmann, & Haxby, 2000). Furthermore, it plays a role in face imitation, together with the FFA (Leslie, Johnson-Frey, & Grafton, 2004). Relating this previous research to the current findings, it becomes clear that changes in pupil size tend to activate the face processing areas more than static pupils.

The last brain region, superior parietal lobule, was only found in decr>stat. This region is known as a somatosensory association area and has a potential role in control of movement and is suggested to take part in visual information processing (Caminiti, Ferraina, & Johnson, 1996).

Comparing with previous studies

The results are partly in accordance with our hypothesis, which stated that the activated brain regions during changes in pupil size are overlapping with the activated brain regions in the ToM network. We see that the STS, the insula, the FFA, angular gyrus and the postcentral sulcus are active when we observe pupillary changes in a partner. It means that during changes in pupil size, people are curious to know what the reason is for this change.

However, there is a difference between decreasing and increasing pupils. Whereas decreasing pupils activate somatosensory brain regions, the increasing pupils mainly activate brain regions involved in face processing. A possible explanation is that decreasing pupils have a more intense, sensitive impact on others, whereas increasing pupils stimulate positive/neutral brain regions

(18)

18

involved with ToM. We are used to seeing positive faces more often than negative ones and thus the decreasing pupils could have a bigger impact on us.

However, we did not find any activations in the amygdala in our contrasts, whereas different studies stated that static pupils have an effect on this region (Demos et al., 2008; Amemiya, & Ohtomo, 2011). It is possible that changing pupils could stimulate this area differently. Furthermore, we used only neutral expressions in our study, whereas others also used pleasant/happy expressions in addition (Demos et al., 2008).

On the other hand, the amygdala is known to be active during the perception of faces in general. In this case a ceiling effect could have occurred: the amygdala is constantly active during all stimuli, so that no significant difference between the contrasts was visible. The changes in the pupils don’t influence the amygdala activity.

Another reason for not having a significant activation of the amygdala could be that the amygdala is sensitive for biologically relevant cues, such as survival signals (Genud-Gabai, Klavir, & Paz, 2013). Seeing a pair of eyes in a safe environment does not elicit such signals thus there is no reason for the amygdala to be active.

We did not exactly find activation of the TPJ, but the supramarginal gyrus is known to be overlapping with this region, so it is possible that both regions are active. However, Mitchell (2008) found that the TPJ is not per se active for inferring the beliefs of other people (i.e. social cognition or ToM), but also during attention in general. Therefore, TPJ cannot be selectively attributed to ToM. The participants paid enough attention to the eyes, but because this happened during every stimulus, we could not find a significant difference between the pupils. This can also be referred to a ‘ceiling effect’, stating that the region is always active and therefore not significantly more active during a specific stimulus.

Although we mentioned that the mPFC is found to be active during visual and social stimuli, the region is often active during rewards, gain probability, and decision-making (Knutson, Taylor, Kaufman, Peterson, & Glover, 2005; Koritzky, He, Xue, et al., 2013). During the pupil stimuli no decisions had to be made, thus it seems likely that this region did not show significant activation. Besides, a study conducted by Raichle, MacLeod, Snyder, et al. (2001) found that the prefrontal regions, together with the medial precuneus, have a high baseline metabolic activity at rest and that this regions decrease in activity during attentional tasks and goal-directed actions (Cavanna, & Trimble, 2006). These regions are, together with the TPJ, also part of the default mode network, stating that social cognition is our default mode of thought and is hard to distinguish during a test (Mars, Neubert, Noonan, et al., 2012). Therefore, it is possible that the activity did not increase during the presentation of pupils.

Limitations & future research

What we did not expect to find is that during these pupillary changes, people also seem to ‘feel’ something, i.e. the somatosensory cortex is active during facial motion (postcentral sulcus and the superior parietal lobule). We did not have a prior hypothesis about this sensory brain region, so it would be interesting to investigate this more in depth in the future.

What is also interesting to mention, is that the right hemisphere is more active than the left one. Dimberg, & Petterson (2000) and Castelli et al. (2000) stated that this region is more sensitive to face expression than the left hemisphere. Unfortunately, we have not tested this statistically, so for the future it would be interesting to add this analysis.

Also, previous studies found a difference in pupil sizes when looking at a man vs. a woman. People fixate longer on male subjects than on female subjects. This has been explained by Hess, Adams, &

(19)

19

Kleck (2005), stating that males draw more attention due to perceived higher social power, status, and dominance. However, the pupils are larger for female than for male characters, irrespective of the subject’s sex. This could be attributed to the fact that women express emotions more strongly, smile more, and are more socially oriented than men (LaFrance, & Hecht, 2000). It would be interesting conducting a study which focuses on the differences in brain activity in men and women when presenting changing pupils of men and women.

Conclusion

During social interactions, people focus on another’s facial expression, and the eyes in particular. When the pupil size changes, people are curious why and how the other one feels, trying to comprehend their emotional state. Overall, pupils tend to activate areas involved in social cognition. Whereas increasing pupils activate areas involved in social cognition, observing someone with decreasing pupils recruits the somatosensory areas, suggesting that this intense signal is intimidating and sensed in one’s own body. This is important, because these findings suggest that looking at one’s eyes helps people to considerate other’s mental state.

(20)

20

References

Aboyoun, D.C., & Dabbs, J.M. (1998). The Hess pupil dilation findings: Sex or novelty? Social Behavior and

Personality, 26, 415-20.

Adolphs, R., & Tranel, D. (2004). Impaired judgments of sadness but not happiness following bilateral amygdala damage. Journal of Cognitive Neuroscience, 16(3), 453-62.

Amemiya, S., & Ohtomo, K. (2011). Effect of the observed pupil size on the amygdala of the beholders.

Social Cognitive and Affective Neuroscience, 7(3), 332-41.

Antal, A., Baudewig, J., Paulus, W., & Dechent, P. (2008). The posterior cingulate cortex and planum temporale/parietal operculum are activated by coherent visual motion. Visual neuroscience, 25(1), 17-26. Aue, T. (2014) I feel good whether my friend win or my foes lose: Brain mechanisms underlying feeling similarity. Neuropsychologia, in press.

Barbur, J.L. (2004). Learning from the pupil – Studies of basic mechanisms and clinical applications. In: L.M. Chalupa, & J.S. Werner, (Eds.), The Visual Neurosciences. Cambridge, MA:MIT Press.

Barbur, J.L., Wolf, J., & Lennie, P. (1998). Visual processing levels revealed by response latencies to changers in different visual attributes. Proceedings of the Royal Society in London B: Biological Sciences, 265(1412), 2321-25.

Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The "Reading the Mind in the Eyes" Test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, and allied disciplines, 42(2), 241-51.

Bailenson, J.N., Blascovich, J., Beall, A.C., & Loomis, J.M. (2003). Interpersonal distance in immersive virtual environments. Personality and Social Psychology Bulletin, 29(7), 1-15.

Beatty, J., & Lucero-Wagoner, B. (2000). The Pupillary System. In: J. Cacioppo, L. Tassinary, & G. Bernston (Eds.), Handbook of Psychophysiology. Cambridge University Press, London.

Beck, A.T., Steer, R.A., & Garbin, G.A. (1988) Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8, 77-100.

Beck, A.T., Guth, D., Steer, R.A., & Ball, R. (1997). Screening for major depression disorders in medical inpatients with the Beck depression inventory for primary care. Behavioral research and therapy, 35(8), 785-91.

Born, R.T., & Bradley, D.C. (2005). Structure and function of visual area MT. Annual review of neuroscience,

28, 157-89.

Bradley, M.M., Miccoli, L., Escrig, M.A., & Lang, P.J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology, 45(4), 602-7.

Breiter, H.C., Etcoff, N.L., Whalen, P.J., et al. (1996). Response and habituation of the human amygdala during visual processing of facial expression. Neuron, 17, 875-87.

(21)

21 Caminiti, R., Ferraina, S., & Johnson, P.B. (1996). The sources of visual information to the primate frontal lobe: A novel role for the superior parietal lobule. Cerebral Cortex, 6(3), 319-28.

Castelli, F., Happé, F., Frith, U., & Frith, C. (2000). Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. NeuroImage, 12(3), 314-25. Cannington, S.J., & Bailey, A.J. (2009). Are there theory of mind regions in the brain? A review of the neuroimaging literature. Human brain mapping, 30(8), 2315-35.

Cavanna, A.E., & Trimble, M.R. (2006). The precuneus: A review of its functional anatomy and behavioural correlates. Brain, 129, 564-83.

Cornette, L., Dupont, P., Rosier, A., Sunaert, S., Van Hecke, P., Michiels, J., et al. (1998). Human brain region involved in direction discrimination. Journal of neurophysiology, 79(5), 2749-65.

Cui, F., Arnstein, D., Thomas, R.M., Maurits, N.M., Keysers, C., & Gazzola, V. (2014). Functional magnetic resonance imaging connectivity analyses reveal efference-copy to primary somatosensory area, BA2. PloS

One, 9(1).

Davis, M. H. (1980). A multidimensional approach to individual differences in empathy. JSAS Catalog of

Selected Documents in Psychology, 10, 85.

Day-Brown, J.D., Wei, H., Chomsung, R.D., Petry, H.M., & Bickford, M.E. (2010). Pulvinar projections to the striatum and the amygdala in the tree shrew. Frontiers in Neuroanatomy, 4(143).

Demos, K.E., Kelley, W.M., Ryan, S.L., Davis, F.C., & Whalen, P.J. (2008). Human amygdala sensitivity to the pupil size of others. Cerebral Cortex, 18, 2729-34.

Dimberg, U., & Petterson, M. (2000). Facial reactions to happy and angry facial expressions: Evidence for right hemisphere dominance. Psychophysiology, 37(5), 693-6.

Dimberg, U., Thunberg, M., & Elmehed, K. (2000). Unconsious facial reactions to emotional facial expressions. Psychological Science, 11(1), 86-9.

Dodell-Feder, D., Koster-Hale, J., Bedny, M., & Saxe, R. (2011). fMRI item analysis in a theory of mind task.

Neuroimage, 55(2), 705-12.

Dubois, S., Rossion, B., Schiltz, C., et al. (1999). Effect of familiarity on the processing of human faces.

NeuroImage, 9, 278-89.

Emery, N. J. (2000). The eyes have it: The neuroethology, function and evolution of social gaze.

Neuroscience & Biobehavioral Reviews, 24(6), 581-604.

Genud-Gabai, R., Klavir, O., & Paz, R. (2013). Safety signals in primate amygdala. Journal of Neuroscience,

33(46), 17986-94.

Gloor, P. (1997). The temporal lobe and limbic system. New York: Oxford University Press.

Grecucci, A., Giorgetta, C., Bonini, N., & Sanfet, A.G. (2013). Reappraising social emotions: the role of the inferior frontal gyrus, temporo-parietal junction and insula in interpersonal emotion regulation. Frontiers in

(22)

22 Harrison, N.A., Singer, T., Rotshtein, P., Dolan, R.J., & Critchley H.D. (2006). Pupillary contagion: central mechanisms engaged in sadness processing. Social Cognitive and Affective Neuroscience, 1(1), 5-17.

Harrison, N.A., Wilson, C.E., & Critchley, H.D. (2007). Processing of observed pupil size modulates perception of sadness and predicts empathy. Emotion, 7(4), 724-9.

Hatfield, E., Cacioppo, J.T., Rapson, R.L. (1994). Emotional Contagion. Cambridge: Cambridge University Press.

Haxby, J.V., Hoffman, E.A., Gobbini, M.I. (2000). The distributed human neural system fore face perception.

Trends in cognitive science, 4(6), 223-33.

Heimberg, R.G., Horner, K.J., Juster, H.R., Safren, S.A, Brown, E.J., Schneier, F.R., & Liebowitz, M.R. (1999).

Psychological Medicine, 29, 199-212.

Hein, G., & Knight, R.T. (2008). Superior temporal sulcus, It´s my area: or is it? Journal of Cognitive

Neuroscience, 20(12), 2125-36.

Hess, E.H. (1965) Pupil responses of hetero- and homosexual males to pictures of men and women: A pilot study. Journal of Abnormal Psychology, 70(3), 165-68.

Hess, E.H. (1975). The role of pupil size in communication. Scientific American, 233(5), 110-19.

Hess, E.H., & Polt, J.M. (1960). Pupil size as related to interest value of visual stimuli. Science, 132(3423), 349-50.

Hess, U., Adams, R.B., & Kleck, R.E. (2005). Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition & Emotion, 19, 515-536.

Hoffman, E.A., & Haxby, J.V. (2000). Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nature neuroscience, 3(1), 80–84.

Iitaka, T. (2012). The role of the superior temporal sulcus in face recognition and perception. Brain and

nerve, 64(7), 737-42.

Kahneman, D., & Beatty, J. (1966). Pupil diameter and load on memory. Science, 154, 1583-5.

Kanwisher, N., & Yovel, G. (2006). The fusiform face area: a cortical region specialized for the perception of faces. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 361(1476), 2109-28.

Kawashima, R., Sugiura, M., Kato, T., Nakamura, A., Hatano, K., Ito, K., Fukuda, H., Kojima, S., Nakamura, K. (1999). The human amygdala plays an important role in gaze monitoring. A PET study. Brain, 122, 779-83. Kendon, A. (1970). Movement coordination in social interaction: some examples described. Acta

Psychologica, 32(2), 100-25.

Knutson, B., Taylor, J., Kaufman, M., Peterson, R., & Glover, G. (2005). Distributed neural representation of expected value. Journal of Neuroscience, 25(19), 4806-12.

(23)

23 Koritzky, G., He, Q., Xue, G., Wong, S., Xiao, L., & Bechara, A. (2013). Processing of time within the prefrontal cortex: recent time engages posterior areas whereas distant time engages anterior areas. Neuroimage, 72, 280-6.

Kozicz, T., Bittencourt, J.C., May, P.J., Reiner, A., Gamlin, P.D.R., Palkovits, M., et al. (2011). The Edinger-Westphal nucleus: A historical, structural and functional perspective on a dichotomous terminology. The

journal of comparative neurology, 519(8), 1413-34.

Kret, M.E., Fischer, A.H., & De Dreu C.K.W. (2013). Humans condition trust in in-group partners on changes in other’s pupil-size via pupil-synchronization, under review.

Kuroki, N., Shenton, M.E., Salisbury, D.F., Hirayasu, Y., Onitsuka, T., Ersner, H., et al. (2006). Middle and inferior temporal gyrus gray matter volume abnormalities in first-episode schizophrenia: An MRI study. The

Americal Journal of Psychiatry, 163(12), 2103-10.

Kvaal, K., Ulstein, I., Nordhus, I.H., & Engedal, K. (2005) The Spielberger State-Trait

Anxiety Inventory (STAI): the state scale in detecting mental disorders in geriatric patients. International

Journal of Geriatric Psychiatry, 20(7), 629 –34.

Lahnakoski, J.M., Glerean, E., Jaaskelainen, I.P., Hyona, J., Hari, R., Sams, M., & Nummenmaa, L. (2014). Synchronous brain activity across individuals underlies shared psychological perspectives. NeuroImage, in

press.

LaFrance, M., & Hecht, M. (2000). Gender and smiling: A meta-analysis. In: A.H. Fischer (Ed.), Gender and

emotion (pp. 118-142). Cambridge, UK: University Press.

Leslie, K.R., Johnson-Frey, S.H., & Grafton, S.T. (2004). Functional imaging of face and hand imitation: Towards a motor theory of empathy. NeuroImage, 21(2), 601-7.

Linke, R., De Lima, A.D., Schwegler, H., & Pape, H.C. (1999). Direct synaptic connections of axons from superior colliculus with identified thalamo-amygdaloid projection neurons in the rat: Possible substrates of a subcortical visual pathway to the amygdala. Journal of Comparative Neurology, 403(2), 158-70.

Mars, R.B., Neubert, F.X., Noonan, M.P., Sallet, J., Toni, I., & Rushworth, F.S. (2012). On the relationship between the ‘default mode network’ and the ‘social brain’. Frontiers in Human Neuroscience, 6(189). McGuire, P.K., Paulesu, E., Frackowiak, R.S., & Frith, C.D. (1996). Brain activity during stimulus independent thought. Neuroreport, 7(13), 2095-99.

Mitchell, J.P. (2008). Activity in right temporo-parietal junction is not selective for theory-of-mind. Cerebral

cortex, 18(2), 262-71.

Mitchell, J.P., Macrae, C.N., & Banaji, M.R. (2006). Dissociable medial prefrontal contributions to judgments of similar and dissimilar others. Neuron, 50(4), 655-63.

Molenkamp E. (1998). Versatile Stimulus Response Registration Program (VSRRP98)

www.test.uva.nl/ozi_psychology/index.php?Page0Software. University of Amsterdam: Amsterdam, The

(24)

24 Perrett, D.I., Hietanen, J.K., Oram M.W., & Benson, P.J. (1992). Organization and functions of cells

responsive to faces in the temporal cortex. Philosophical transactions of the Royal Society of London. Series

B, biological sciences, 335(1273), 23-30.

Phillips, M.L., Young A.W., Scott, S.W., Calder, A.J., Andrew, C., Giampietro, V., et al. (1998). Neural

responses to facial and vocal expressions of fear and disgust. Proceedings. Biological Sciences/Royal Society,

265(1408), 1809-17.

Phillips, M.A., Szabadi E., & Bradshaw, C.M. (2000) Comparison of the effects of clonidine and yohimbine on spontaneous pupillary fluctuations in healthy human volunteers. Psychopharmacology, 150(1), 85-9. Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1(4), 515-26.

Price, J.L., & Drevets, W.C. (2012). Neural circuits underlying the pathophysiology of mood disorders. Trends

in Cognitive Sciences, 16(1), 61-71.

Raichle, M.E., Macleod, A.M., Snyder, A.Z., Powers, W.J., Gusnard, D.A., & Schulman, G.L. (2001). A default mode of brain function. Proceedings of the National Academy of Science of the United States of America,

98(2), 676-82.

Rossion, B., Caldara, R., Seghier, M., Schuller, A.M., Lazeyras, F, & Mayer, E. (2003). A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face

processing. Brain: a journal of neurology, 126(11), 2381-95.

Sadato, N., Okada, T., Honda, M., Matsuki, K., Yoshida, M., Kashikura, K., et al. (2005). Cross-modal integration and plastic changes revealed by lip movement, random-dot motion and sign languages in the hearing and deaf. Cerebral cortex, 15(8), 1113-22.

Saxe, R., & Kanwisher, N. (2003). People thinking about thinking people. The role of the temporo-parietal junction in “theory of mind”. Neuroimage, 19(4), 1835-42.

Scalf, P.E., Ahn, J., Beck, D.M., & Lleras, A. (2014). Trial history effects in the ventral attentional network.

Journal of cognitive neurosciences, in press.

Schrammel, F., Pannasch, S., Graupner, S., Mojzisch, A, & Velichkovsky, B.M. (2009). Virtual friend of threat? The effects of facial expression and gaze interaction on psychophysiological responses and emotional experience. Psychophysiology, 46(5), 922-31.

Silani, G., Lamm, C., Ruff, C.C., & Singer, T. (2013). Right supramarginal gyrus is crucial to overcome emotional egocentricity bias in social judgments. The journal of neuroscience: the official journal of the

Society for Neuroscience, 33(39), 15466-76.

Spiegelberger, C.D. (2010). State-Trait Anxiety Inventory. Corsini Encyclopedia of Psychology. 1.

Steinhauer, S.R., Siegle, G.J., Condray, R., & Pless, M. (2004). Sympathetic and parasympathetic innervation of pupillary dilation during sustained processing. International Journal of Psychophysiology: official journal

of the International Organization of Psychophysiology, 52(1), 77-86.

Taylor, L., & Jones, L. (1997). Effects of lesions invading the postcentral gyrus on somatosensory thresholds on the face. Neuropsychologia, 35(7), 953-61.

(25)

25 Van Der Schalk, J., Hawk, S.T., Fischer, A.H., & Doosje, B. (2011). Moving faces, looking places: Validation of the Amsterdam Dynamic Facial Expression Set (ADFES). Emotion, 11(4), 907-20.

Vollm, B.A., Taylor, A.N., Richardson, P., Corcoran, R., Stirling, J., McKie, S., Deakin, J.F., & Elliot, R. (2006). Neuronal correlates of theory of mind and empathy: A functional magnetic resonance imaging study in a nonverbal task. Neuroimage, 29(1), 90-8.

Wager, T.D., Davidson, M.L., Hughes, B.L, Lindquist, M.A., & Ochsner, K.N. (2008). Neural mechanisms of emotion regulation: Evidence for two independent prefrontal-subcortical pathways. Neuron, 59(6), 1037-50. Watabe, M., Ban, H., & Yamamoto, H. (2011). Jugdments about other’s trustworthiness: An fmri study.

Letters on Evolutionary Behavioral Science, 2(2).

Wang, C.A., Boehnke, S.E., White, B.J., & Munoz, D.P. (2012). Microstimulation of the monkey superior colliculus induces pupil dilation without evoking saccades. The Journal of Neuroscience: The official journal

(26)

26 Appendix 1.

Beck Depression Inventory

Choose the one statement, from among the group of four statements in each question that best describes how you have been feeling during the past few days. Circle the number beside your choice.

1 0 I do not feel bad. 1 I feel sad.

2 I am sad all the time and I can’t snap out of it. 3 I am so sad or unhappy that I cannot stand it. 2 0 I am not particularly discouraged about the future.

1 I feel discouraged about the future. 2 I feel I have nothing to look forward to.

3 I feel that the future is hopeless and that things cannot improve. 3 0 I do not feel like a failure.

1 I feel I have failed more than the average person. 2 As I look back on my life, all I can see is a lot of failure. 3 I feel I am a complete failure as a person.

4 0 I get as much satisfaction out of things as I used to. 1 I don’t enjoy things the way I used to.

2 I don’t get any real satisfaction out of anything anymore. 3 I am dissatisfied or bored with everything.

5 0 I don’t feel particularly guilty. 1 I feel guilty a good part of the time. 2 I feel guilty most of the time. 3 I feel guilty all of the time.

6 0 I don’t feel that I am being punished. 1 I feel I may be punished.

2 I expect to be punished. 3 I feel I am being punished.

7 0 I don’t feel disappointed in myself. 1 I am disappointed in myself. 2 I am disgusted with myself. 3 I hate myself.

8 0 I don’t feel I am worse than anybody else.

1 I am critical of myself for my weaknesses or mistakes. 2 I blame myself all the time for faults.

3 I blame myself for everything bad that happens. 9 0 I don’t have any thoughts of killing myself.

1 I have thoughts of killing myself but I would not carry them out. 2 I would like to kill myself.

3 I would kill myself if I had the chance. 10 0 I don’t cry more than usual.

1 I cry more now than I used to. 2 I cry all the time now.

3 I would kill myself if I had the chance.

11 0 I am not more irritated by things than I ever am. 1 I am slightly more irritated now than usual.

2 I am quite annoyed or irritated a good deal of the time. 3 I feel irritated all the time now.

(27)

27 12 0 I have not lost interest in other people.

1 I am less interested in other people than I used to be. 2 I have lost most of my interest in other people. 3 I have lost all my interest in other people. 13 0 I make decisions about as well as I ever could.

1 I put off making decisions more than I used to.

2 I have a greater difficulty in making decisions than before. 3 I can’t make decisions at all anymore.

14 0 don’t feel I look any worse than I used to. 1 I am worried that I am looking old or unattractive.

2 I feel that there are permanent changes in my appearance that make me look unattractive.

3 I believe that I look ugly.

15 0 I can work about as well as before.

1 It takes an extra effort to get started at doing something. 2 I have to push myself very hard to do anything.

3 I can’t do any work at all. 16 0 I can sleep as well as usual.

1 I don’t sleep as well as I used to.

2 I wake up 1-2 hours earlier than usual and find it hard to get back to sleep. 3 I wake up several hours earlier than I used to and cannot get back to sleep. 17 0 I don’t get more tired than usual.

1 I get tired more easily than I used to. 2 I get tired from doing almost anything. 3 I am too tired to do anything.

18 0 My appetite is no worse than usual. 1 My appetite is not as good as it used to be. 2 My appetite is much worse now.

3 I have no appetite at all anymore. 19 0 I haven’t lost much weight, if any, lately.

1 I have lost more than five pounds. 2 I have lost more than ten pounds.

3 I have lost more than fifteen pounds trying to lose weight.

Score 0 if you have been purposely trying to lose weight.

20 0 I am no more worried about my health than usual.

1 I am worried about my physical problems such as aches and pains or upset stomach. 2 I am very worried about physical problems and it’s hard to think of much else. 3 I am so worried about my physical problems that I cannot think about anything else. 21 0 I have not noticed any recent change in my interest in sex.

1 I am less interested in sex. 2 I am much less interested in sex. 3 I have lost interest in sex completely.

Please indicate if you have felt any of the following, how often and for what period of time: □ Depressed mood

□ Loss of interest or pleasure in usual activities □ Significant change in weight and/or appetite □ Insomnia or hypersomnia

□ Psychomotor agitation or retardation

□ Increased fatigue and loss of energy □ Feelings of self-reproach, worthlessness or inappropriate guilt

□ Slowed thinking or impaired concentration □ Suicide attempt or suicidal ideation