The Musical Motivation Motor: on the role of the mesolimbic reward system in musical emotion and appreciation

(1)

The Musical Motivation Motor

On the role of the mesolimbic reward system in musical emotion and

appreciation

Master’s Thesis

Sophie van Weeren

Music Studies (Arts and Culture)

University of Amsterdam

November 2018

Supervisor: dr. M. Sadakata

Second reader: prof. dr. H. Honing

(2)

(3)

Acknowledgements

This thesis project has been a longer process than I set it out to be. Now that I have finished it, I feel that it is the moment to thank the following people who supported me in various ways throughout the process. First of all, thank you to dr. Makiko Sadakata for being my supervisor in this project; her critical questions and patience in our meetings have helped me form and finish this thesis. Furthermore, I would like to thank prof. dr. Henkjan Honing, who helped me and coached me in this process: his never-ending enthusiasm about science has helped me a great deal.

I also want to thank my close friends and family: Annette Birkhoff and Dorith van Weeren for listening to me simply when I needed someone to do so; Nathan Tax, for being an incredibly patient friend, giving me advice from unexpected vantage points; and Sanne Groothuis, my dear friend who has been my study and sparring partner through many of my academic papers and projects.

Last, I need to thank Samantha Millard for the time she spent reading and correcting my drafts, providing me with both critical notes as useful tips to improve my writing skills.

(4)

This literature review sets out to show that the mesolimbic reward system plays a pivotal part in the processing, emotional response and appreciation of music. This claim arises from an elaborate investigation of the history and the context of musical reward research. A growing set of studies report activity of the mesolimbic reward circuitry during musical processing and emotion and this has led to a separate field of study focusing on specific musical reward. The possibility of the essential role of the reward system within musical processing can have far reaching consequences for our understanding of how and why humans interact with music the way they do. In this paper I support his claim by, firstly, discussing the emergence of musical reward research from musical emotion research and the papers on musical chills and Autonomic Nervous System arousal. It then explains the functioning of the mesolimbic reward circuitry and reviews what is known about it in relation to music. Thirdly, I discuss possible relationships between reward and emotion and substantiate my claim of the potential importance of musical reward. Lastly, I will demonstrate the possible essentiality of the reward system in music processing and the consequences of a lack thereof, by discussing musical anhedonia, a condition that causes a lack of the musical pleasure experience.

(6)

Introduction

Humans use music in various occasions and for a variety of reasons. Music has the power to move us both literally and figuratively. It induces emotions, makes us remember things, and can even give pleasure in such a way that it causes shivers down our spines. When and individual encounters a musical stimulus, the brain activates on various levels. Recently, studies have shown that one of the networks that becomes active during music listening is the mesolimbic reward circuitry. This network is also responsible for the pleasurable feeling we have when we encounter other kinds of rewarding stimuli (e.g. overview by Salimpoor & Zatorre, 2013). With this finding, there is now a rapidly expanding set of literature focusing on musical reward. In this thesis, I will argue that this literature, in addition to the already existing research field that focusses on emotion and appreciation, is pointing towards an essential role of the mesolimbic reward circuitry within musical processing. I will highlight this line of research and, moreover, argue that there is need for new research that explicitly addresses the musical reward system as a potential starting point of music appreciation in general. I will show that this claim is grounded in both prominent theoretical papers of the research field as in the reports of quantitative studies. Recent studies suggest that reward system activation in response to music interferes significantly with other musical processing networks. I will do so by critically reviewing these papers: I will discuss the results, highlight the ambiguities in terminologies, exemplify the implicit stances that currently exist and discuss what is not known yet and how this can be studied.

The term pleasure is crucial within musical reward research. Researchers use it as a tool to study musical preferences. It is a subjective feeling that constitutes the reward experience and it also refers to a specific mechanism of the reward system. This raises questions about the origin (or nature) of pleasure. It is no surprise that the exact definition of pleasure is not agreed upon and that the use of it in papers can thus be described as ambiguous. In this paper I argue that this is not just a matter of confusion of terminologies on a surface level, but that it actually reflects an issue on the nature of pleasure. I thus offer a pragmatic solution, with the main aim to make research aware and to avoid miscommunication in the future. Furthermore, the stances on the relationship between reward and emotion are diverse but not always explicitly stated. One can see that musical reward has positioned itself as topic of research in itself, therefore it is good to investigate the current literature.

The first chapter will discuss the history of musical reward research and explain how it is rooted in musical emotion studies. This discussion will include a section on the ambiguous use of the term pleasure: as pleasure is an essential part of reward, a clear definition is needed for the future of reward research. The second chapter will review the literature on musical chills and Autonomic Nervous System (ANS) arousal, as it can be seen of one of the markers that led research toward the specific study of musical reward. Chapter three will explain how the reward system functions, and discuss the evolutionary importance, the related brain areas and the different phases of reward. It will then discuss the studies that relate musical processing to this reward system and findings that demonstrate that the

(7)

reward system responds to music in similar manners as it does for other (evolutionary relevant) stimuli. Chapter four entails a review about how various papers implicitly treat reward as a part of music. Furthermore, with aid of the paper by Vuust and Kringelbach (2010) on reward and anticipation, I will show that there is a possibility of a more essential role of the reward system within musical appreciation. In the last chapter I will exemplify this claim with a discussion on musical anhedonia. This condition refers to a selective group of people that lack a reward response to musical stimuli. Musical anhedonia provides an excellent case study to further explore the musical reward and emotion responses.

By the end of the thesis, I hope to have shown that musical reward is more important than is currently suggested, and that there is a need for the further study of musical reward and exploration of the relationship between reward and emotion. By working with clear definitions and studying specific case studies, such as the musical anhedonic population, researchers should have the right tools to do so and should be able to extend the knowledge on musical reward.

(8)

Chapter 1: History and context of musical reward research

Musical reward has only recently been studied as an independent research topic. The first studies discussing musical reward focus on musical emotion and processing in the brain. These studies are approximately twenty years old thus the history of reward is quite short. In this chapter I aim to show the historical path that eventually led to the separate study of musical reward. The discussion will include the complexity of musical emotion and reward and different stances that can be taken concerning their intertwined relationship. Additionally, complexities in terminology due to the interdisciplinary of this topic, will be highlighted.

Musical emotion has fascinated humankind for decades. Many researchers have enthusiastically explored and hypothesized why humans can be moved so deeply by music (e.g. Cook & Dibben, 2010 for an overview). Moreover, research has widely discussed the nature of these emotions. This refers to the so-called emotivist-cognitivist debate - a discussion on whether the emotions that music can elicit are actually the same as real emotions, or just cognitive representations or recognitions of these emotions (Peretz, 2010; Koelsch, Siebel & Fritz, 2010).With help from psychophysiological and neuro-imaging techniques researchers have only recently been able to look into the human brain during the experience of musical emotion. This has led to evidence that activations of brain networks during musical emotion are similar to those during other types of emotional experiences, thus supporting the ‘emotivist’ stance of the discussion (Koelsch et al., 2010). This is also the stance that I will stick to in this thesis.

Emotion is ‘‘a quite brief but intense affective reaction that usually involves a number of sub-components – subjective feeling, physiological arousal, expression, action tendency and regulation – that are more or less synchronized.” (Juslin & Sloboda, 2010, p. 10). Subjective feeling refers to the mental experience someone has during an emotion. Physiological arousal relates to the activation of the Autonomic Nervous System (ANS). The expression is what is visible for the outside world (for instance tears during sadness), and the action tendency and regulation refer to the possible actions an individual will do in response to the emotion. A simplified definition that is frequently used in experimental research, states that emotion is a combination of valence and arousal (e.g. Salimpoor, Benovoy, Longo, Cooperstock & Zatorre, 2009). In this context, arousal refers to the arousal of the ANS and Valence represents the positive or negative value an emotion has. The definition is useful for research, because one can measure arousal objectively through psychophysiological measurements (e.g. heartbeat, skin conductance), and valence can be identified by subjective self-reports. Nevertheless, in this second definition there is also still a lot of complexity in terms of the degrees of valence and arousal that can be found. Studies into the type of emotion humans experience during music listening show the diversity of emotions produced by music. Zentner, Granjean and Scherer (2008), for instance, identified no less than 45 emotions in this way that even after categorizing them using confirmatory factor analysis, resulted in no less than 9 different categories. These emotions vary from the four basic emotions (e.g.

(9)

happiness, sadness, fear and anger) to complicated emotions, such as melancholia (Zentner, Granjean & Scherer, 2008).1

An additional challenge in research is that the processing of music happens in a variety of networks in the brain. First of all, an auditory stimulus has to be analyzed so that a listener can understand what it is. Musical stimuli find their way to our brain through our auditory pathway (Camalier & Kaas, 2011). A piece of auditory information is received by the ear and sent to the cochlea in the inner ear. The cochlea transforms the auditory waves into distinct neuronal signals our brain can analyze. From there it passes via the inferior colliculus, medial geniculate complex (part of the thalamus) and to the primary auditory cortex, which is located on the superior temporal gyrus (STG) (on the temporal lobe of the cortex). This input system works roughly the same for all primates (Camalier & Kaas, 2011). From the primary auditory cortex there are projections to various brain areas that further analyze the auditory input, including other parts of the STG, various prefrontal regions, and the insular cortex (Camalier & Kaas, 2011). Activation of such a large brain network demonstrates the complexity produced from seemingly straightforward simple questions such as how the brain analyses sound.

The complexity of all the elements that constitute an emotion, together with the wide distribution of involved brain networks has challenged research to find the right approach of study. In the beginning of this century, it has led to an approach focusing on relatively simple, universal, and contrasting emotions. The main aim of these studies was to map out the important brain areas involved in musical emotion, for instance by contrasting pleasant versus unpleasant music (e.g. Blood, Zatorre, Bermudez & Evans, 1999; Blood & Zatorre, 2001; Menon & Levitin, 2005). Humans universally prefer consonance to dissonance, which inspired a useful research set-up. Blood et al. (1999) defined this pleasantness by the degree of consonance in the musical excerpts they presented participants. The results of this study showed activation of regions that are also known to activate during pleasant emotions in general, such as mesolimbic and prefrontal areas. However, other types of pleasant music listening also activate these networks, for instance when participants listened to self-chosen pleasant pieces compared to the chosen pieces of other participants (Blood & Zatorre, 2001) or when they listened to scrambled versus unscrambled musical excerpts (Menon & Levitin, 2005). These papers were the first examples to find activation of emotional networks during positive emotional listening experiences.

Another contrast that is used often is that between happiness and sadness, which are distinctly associated with respectively the major and the minor mode in Western music.2 There are differences in the neuronal correlates in response to major and minor chords that are also related to musical emotion

1_{The four basic emotions are universally present and recognized by humans. These four emotions have been}

studied in relation to music (Peretz, 2010). Some theories nowadays also add disgust and surprise.

2_{The perceiving of the musical emotion here is not the same as feeling. It is one thing to be able to recognize that}

a major piece, for instance, is meant to convey happiness, but it is another one to experience or feel that emotion yourself. Recognition requires different mechanism than feeling (e.g. Salimpoor et al., 2009). In research on musical emotion and reward, the two are therefore treated as two distinct things.

(10)

Furthermore, activations in response to these chords are dissociated from perceptual areas that primarily process and analyze stimuli (Suzuki et al., 2008). In recent years there is a tendency towards studying specific emotions such as joy, fear or sadness with the aim to relate this to specific patterns of activity in the brain or specific activation in certain areas (e.g. Brattico et al., 2011; Koelsch et al., 2013).

Even though there are a few distinct areas that are linked to specific musical valence, either negative or positive, evidence is pointing towards a more widely distributed pathway of musical emotion, consisting of highly connected brain areas (e.g. Peretz, 2010). This theory rises from the highly distributed activation patterns found in the brain during musical emotions. The pathway can be divided in a cortical and subcortical section (Peretz, 2010). Subcortical structures in the brain are evolutionary older and known to be related to automatic, subconscious processes, whereas the cortical structures are more related to conscious processes (Salimpoor & Zatorre, 2013). The subcortical pathway consists primarily of the limbic system. Broca first identified the limbic system and hypothesized that it acted specifically in response to emotional stimuli (Broca, 1878). The limbic system includes the hypothalamus, amygdala, hippocampus, thalamus, cingulate gyrus and the ventral striatum (Peretz, 2010). It is activated during early stimuli processing and is partly responsible for producing quick responses to potentially harmful threats, even our conscious awareness. Multiple studies have suggested that the limbic system is activated in emotional music listening experiences (e.g. Blood et al. 1999; Blood & Zatorre, 2001; Menon & Levitin, 2005; Koelsch et al. 2006). These studies report general activation of the limbic system during emotional musical excerpts, but also differences in the patterns of activation during positive and negative emotions. Koelsch et al. (2006), for instance, contrasted pleasant (consonant) with unpleasant (dissonant) musical excerpts and reported different forms of activation of the amygdala, hippocampus, parahippocampal gyrus and ventral striatum.

Part of the limbic system is also constituting the mesolimbic reward network. This network is the main motor of our rewarding (pleasurable) sensation, motivating humans and animals to seek for stimuli to help us survive (e.g. sex, food). Early studies into musical emotion found activation of these areas but did not study types or patterns of these activation, or the alikeness of these patterns to other rewarding stimuli (e.g. Blood & Zatorre, 2001). Only later on it became clear that music is not only tapping into the same areas, but it is actually activating the whole reward system in a similar manner as other stimuli would. However, in the early studies the reward areas were more or less approached as just a part of the complete emotional experience.

Specific attention should be given to the activation of the amygdala because it has very specific activation patterns and has many neural connections to the reward system and other limbic areas. The amygdala regulates and modulates emotion networks by initiating, maintaining and terminating emotions, and is also thought to integrate cognitive and emotional information (Koelsch, 2014). The amygdala activates differently for pleasant stimuli versus unpleasant stimuli. Activation decreases during pleasurable music listening (e.g. Blood & Zatorre, 2001) but increases in cases of unpleasant music listening, such as scary music (Gosselin et al, 2005, 2007) and dissonant music. The amygdala

(11)

also responds to more general, evolutionary relevant, sounds: causing the startle reflex one can have during a sudden onset of loud noises. Since similar sounds can also happen in music (a sudden change in musical timbre or dynamics, for instance) the amygdala seems to play an important role in these instinctive responses. In short, evidence suggests that the limbic is involved in bottom-up responses to sounds. It includes specific areas for positive valence and pleasure, and plays a role in negative emotions. Next to the subcortical hubs, some distinct cortical areas are activated in musical emotion (Blood et al, 1999; Blood & Zatorre, 2001; Mitterschiffthaler, Fu, Dalton, Andrew & Williams, 2007; Menon & Levitin, 2005). The orbitofrontal cortex (OFC), insular cortex (or insula) are the most reported areas. These cortical areas are hypothesized to contribute to the cognitive appraisal and (e)valuation of what one is experiencing (Salimpoor & Zatorre, 2013). The OFC has many reciprocal connections with the limbic system and is thus also said to be part of the mesolimbic circuit (Schultz, 2006).3 The OFC has been related to emotional behavior and valence and is specifically thought to contain information about our moral emotions such as guilt, shame and regret (Koelsch et al., 2010). It has been reported in many musical emotion studies (Blood & Zatorre, 2001; Brown, Martinez & Parsons, 2004; Menon & Levitin, 2005; Suzuki et al, 2008; Salimpoor et al., 2013). The insular cortex is one of the areas responsible for arousal of the Autonomic Nervous System (ANS) which is (as discussed above in definitions of emotion) an essential part of emotion (Peretz, 2010; Koelsch et al., 2010).

Next to this, multiple studies report activation of the superior temporal gyrus (STG) (Blood et al., 1999; Brown et al., 2004; Mitterschiffthaler et al., 2007). The STG has a possible interesting role because it is on one hand involved in the primary analysis of auditory stimuli but is now thought to also represent specific valences (Koelsch, Skouras & Lohmann, 2018). Furthermore, the cingulate gyrus is repeatedly being reported in studies of musical reward and emotion (e.g. Blood & Zatorre, 2001; Brown et al., 2004; Menon & Levitin, 2005; Suzuki et al, 2008; Salimpoor et al., 2013). It is therefore hypothesized that the cingulate gyrus aids emotional processing although the exact role is not yet defined. One suggestion is that, due to the great number of bidirectional connections to other areas, the cingulate gyrus aids in integrating and mediating musical emotions (Koelsch et al., 2010). Lastly, involvement of the parahippocampal gyrus has been reported repeatedly (e.g. Mitterschiffthaler, Fu, Dalton, Andrew & Williams, 2007; Blood & Zatorre, 2001; Brown et al., 2004) and is thought to have a role in activation and forming of new memories (Koelsch et al., 2010).

Looking at these studies with the current knowledge I can identify two problems: first of all, there is a great variety of active brain areas of which the exact role is not yet determined. A good example can be taken from activation of the hippocampus, parahippocampal gyrus and temporal poles. These areas together constitute a network that is involved in memory formation and retrieval (Koelsch et al.,

3_{Next to the orbitofrontal cortex there is also various discussion on the role of the ventromedial prefrontal cortex.}

However, there is some confusion on the location of these areas, as both are located on Broadman areas 10 and 11. In this thesis I will thus discuss them as one and the same. This is contrarily to for instance the approach used by Peretz (2010), who discusses the two areas separately.

(12)

2010). Specifically, this network is responsible for the forming of new (emotional) memories during music listening, and additionally may evoke memories during music listening and musical emotion. Even though it is known that memories are important in the experience of music, the role of this network in the specific evoking of emotions during music listening is not entirely clear (Koelsch et al., 2010). One can question if musical emotion can occur without memories and, if so, if this memory network is essential for musical emotions. These types of ambiguities concerning the specific role of areas calls for a detailed study of the areas and networks, for instance by assessing specific activation patterns of areas or the connections between certain areas. In this manner one can study the specific role of this memory network, but also for instance the mesolimbic reward network.

A second problem can be identified in the used terminology of this early research. In the simple contrast studies, as are discussed earlier in this section, the underlying concept of the researchers was mainly to study positive versus negative, or pleasant versus unpleasant. This brings me to the definition of the word pleasure. Within reward research nowadays, pleasure is clearly defined as a specific phase of the reward circuitry (e.g. Schultz, 2006). However, in the early research one finds that researchers contrast pleasant and unpleasant (agreed upon by them and their participants which excerpt is pleasant, and which one is unpleasant) but there is a lack of insight into the cause of this pleasurable experience. Take for example the three studies discussed above, on pleasant and unpleasant music (Blood et al., 1999; Blood & Zatorre, 2001; Menon and Levitin 2005). These three studies al contrast pleasant versus unpleasant music, but they all use different definitions of pleasantness. In the case of Menon & Levitin, pleasantness means the pieces are not scrambled, as they contrast them to scrambled excerpts. In the case of Blood et al. (1999), pleasantness is related to consonance in the pieces while in Blood & Zatorre’s study (2001), pleasantness is elicited by self-chose pieces by the participants. Blood and Zatorre (2001) did not ask their participants for the reasons why certain musical excerpts were chosen as highly pleasurable. It could potentially be that one participant derived the pleasant experience of his piece from the pleasant memory he had with it, whereas another derived it from a distinct happy emotion. However, Blood & Zatorre neither discuss the subjective, conscious reason for the pleasantness, nor the potential underlying brain mechanisms causing the pleasure.4 Even though these studies found correlations between their type of pleasantness and activation of limbic regions, they do exemplify how the underlying mechanisms or cognitive processes defining the pleasantness might be activated through different pathways.

The problem is twofold: firstly, there is no clear definition of pleasure. Secondly, the lack of unified definition makes one realize that we do not know if there is only one type of musical pleasure or if there are perhaps more. In this thesis I will solve this problem as follows: I will use a distinct definition of pleasure, referring to one specific mechanism (see below) but simultaneously I do

4_{Moreover, it is known that general preference and liking for pieces increases when a listener is more familiar}

(13)

acknowledge that there is a lack of knowledge on the specifics of the potential other mechanisms that constitute pleasure in relation to music. The definition I will use has its roots in the general reward research, or science of affect. In this field of study, pleasure relates to one specific mechanism; it is a specific component or phase of the reward system (Schultz, 2006). This system functions, evolutionary, as a motivator for animals to seek rewarding stimuli and is therefore essential for our survival. It consists of a distinct set of subcortical areas, of which some are so-called hedonic hotspots. These hotspots are essential areas of the reward circuitry, as they are hypothesized to release the neurotransmitters (e.g. dopamine) that cause the pleasurable feeling. It is also often referred to as the ‘liking’ phase of reward (Schultz, 2015). This vantage point offers a proper definition for pleasure, as it is related to a very distinct mechanism in the brain. However, it still has a subjective component (i.e. the feeling) that needs to be acknowledged during research. This is often controlled for by objectively measuring arousal of the ANS. Thus, I will refer to this definition of pleasure, derived from the science of affect and well defined as: “[a] construct that refers to a subjective state and implies that the associated behavior is rewarding and likely to be repeated.” (Salimpoor et al., 2009, p. 2). As studies do consistently report specific reward-system brain correlates in relation to different types of pleasure, I do believe that this definition is currently the most viable one and will thus work with it.

(14)

Chapter 2: Chills and Arousal

Chills, thrills or shivers-down-your-spine: these are all names for a the highly pleasurable feeling one can have when listening to music one really enjoys. It is one of the facets that has guided researchers towards the separate study of musical reward. In this chapter I will discuss musical chills and arousal of the Autonomic Nervous System (ANS) in relation to pleasure and music.

A chill is a marker of arousal of the ANS and, during the experience of pleasure, correlates with activation of the reward system. However, the experience of a chill is not a music related event per se. The ANS is, amongst other things, responsible for the fight/flight response that can be found across species and that occurs as a result of any event that causes surprise. To do so, it regulates reflexes and mechanisms such as respiration, heartbeat and vasomotor activity (widening and narrowing of blood vessels). The hypothalamus and insula modulate these mechanisms when they receive input to do so from the limbic system. The majority of the work of the ANS is done subconsciously, although we can be aware of the effects of ANS activation (Gazzaniga, Ivry & Mangun, 2004). Chills can happen in response to a variety of stimuli: auditory, visual, tactile and gustatory (Grewe, Katzur, Kopiez & Altenmüller, 2011). A (unexpected) stimulus causes activation of the ANS that leads to the contraction of the minuscule muscles we have in our skin. In furred animals, the contraction of the muscles will make their hairs erect which makes them look bigger and more frightening. Even though humans do not have fur, this mechanism is still function (Altenmüller, Kopiez & Grewe, 2013). Evolutionary speaking, the experience of a chill is not necessarily a pleasurable one (Grewe et al. 2010; Vuust & Kringelbach, 2010). When a chill happens as a response of pleasurable music listening, however, it can be experienced as pleasurable and this phenomenon has led to various studies.

In research of the pleasure that comes with music making and listening, musical chills have been used in various ways and for various purposes. They can for instance be used as a way of controlling for musical pleasure. Even though a pleasurable chill is to a certain extend a subjective experience, the experience is well-defined, easy to identify by the one experiencing it and therefore easy to self-report. If one wants to study musical pleasure, the chills can thus serve as control for (subjective) self-report of musical pleasure. Chills can also be studied objectively, as they serve as a marker of arousal of the ANS. One can thus research them by assessing other psychophysiological markers. Examples of this are increased heartrate, respiration and sweating (measured through skin conductance) (Zald & Zatorre, 2011). Various papers have shown that the experience of a chill correlates with peak activity of the ANS arousal markers assessed through these psychophysiological measurements (e.g. Blood & Zatorre, 2001; Salimpoor et al., 2009). It is important to note though, that in these studies there is always a subjective report needed to confirm the positive valence of the chills. Studying musical chills in relation to musical pleasure has helped researching the relationship between underlying brain areas and networks that process music. The occurrence of chills together with experiences of pleasure also indicated possible activity of the mesolimbic reward network. This has aided researchers to study reward circuitry activity

(15)

with the help of musical chills (e.g. Blood et al., 1999; Blood & Zatorre, 2001; Salimpoor et al., 2009; Salimpoor et al., 2011).

Even though every human has the evolutionary mechanism to have chills, not everyone has the experience of chills in response to music. Researchers report different numbers on the percentage of humans that experience musical chills. Early studies have estimated the number between 70 and 90% (Goldstein, 1980; Sloboda, 1991). Although sometimes they make a difference between chills and goose-pimples, even though they might actually be a different word for the same phenomenon (Sloboda, 1991). Researchers infer these results from questionnaires that asked participants about their experiences with music in general. In experimental set-ups, the occurrence of musical chills is often significantly lower. Grewe et al. (2005) conducted a study that measured chills during music listening through self-report and found that only 8 out of 37 participants in their study experienced chills which would translate to merely 22%.5 These studies led to the realization that musical pleasure is not exclusively related to musical chills. It seems that pleasure does inevitably go hand in hand with ANS arousal, but the chills seem merely a peak response (e.g. Grewe et al., 2005). These findings were also reported by Salimpoor, Benovoy, Longo, Cooperstock and Zatorre (2009). They found a correlation between ANS arousal and perceived pleasure of the participants, regardless of the presence of chills. From these results it is being made clear a chill experience is a result of arousal that accompanies the reward system activation. However, it does not seem to directly cause the pleasure.6

The question that remains is what causes the arousal and chills, and thus the pleasure. Goldstein (1976) wrote the first the paper that pointed towards the reward system. He found a diminished chill response in some participants of his study after he injected a pharmacological substance (naloxone) to block opiate receptors in the brain (more specifically in our reward system). Neurotransmitters that would normally bind to these receptors could not do so as a result of the injection. We now know that these neurotransmitters play an important part within the activation of the reward system. Blocking them, would thus cause diminished activation. Even though the findings have not been replicated, the paper was very useful as it was the first one to suggest that the involved neurotransmitters were

5_{The individual differences in musical chill experiences can be potentially explained by different elements: some}

individuals might be more susceptible to chills due to genetics, but another reason can be in the music itself. Familiarity with a piece or preference for a certain genre can just as well play part. Within the musical pieces itself, studies have focussed on structural or acoustical elements that might play a role in the occurrence of thrills. Until now, conclusive answers about musical elements that play a role have not been given yet and an elaborate discussion of these potential aspects is beyond the scope of this review. (Panksepp, 1995; Grewe et al., 2005; Grewe, Nagel, Kopiez and Altenmüller, 2007).

6_{On top of this, there is the phenomenon that chills as an experience itself in response to music are being reported}

as ‘extra’ pleasurable/ How is a chill itself being reported as pleasurable on top of the general liking of music? Huron (2006) explains the positive valence that is given to a chill by posing that next to the chill humans have an appraisal mechanism that travels slightly slower than the chill itself. So even though a chill is evolutionary potentially a warning sign, the appraisal mechanism that follows causes the realization that ‘in fact nothing was wrong’. This can explain the delighted feeling that comes with a chill. However, this theory is yet to be proven.

(16)

connected to the chills. With this it found a potential connection between musical chills and reward related neurotransmitters, and therefore the reward system (Goldstein 1976; Goldstein 1980).

Simultaneously with the findings on musical chills, neuro-imaging techniques that allowed research to literally look in the human brain while it is in action, gained popularity in research. The focus of research thus shifted from the role of musical chills to the role of the reward system in the musical pleasure experience (e.g. Blood & Zatorre, 2001; Salimpoor et al., 2009; Salimpoor et al. 2011). These studies confirmed activation of regulators of the ANS (hypothalamus, insula) together with activation of key areas of the reward system during pleasurable music experiences (e.g. Menon & Levitin, 2005, Blood & Zatorre 2001). Blood and Zatorre (2001) provided substantial evidence for the connections between the reward system and musical chills. They conducted a Positron Emission Topography (PET) study and measured significant cerebral flow changes in the ventral striatum and dorsomedial midbrain that correlated with the number of chills participants experienced while listening to music. This correlation was later confirmed by Salimpoor et al. (2011), who found correlations between self-reported pleasure (with and without the actual chills) and neuro-imaging measurements of the reward system.

Thus, the study of musical chills and ANS arousal has guided musical research to investigate the reward system in relation to music. The knowledge about the connection between reward and ANS arousal modulators mediated this process, as well as the studies, like Blood and Zatorre (2001), investigating musical reward in relation to musical chills. Furthermore, the physiological markers of arousal that have been related to musical chills showed to be a very useful tool when studying musical reward. Examples of this will be shown in the next chapter. The next chapters will also investigate the exact role of the reward system and aim to answer questions about why it is that music, as a stimulus with no obvious evolutionary relevance, has the ability to activate this system.

(17)

Chapter 3: Musical Reward

This chapter consists of two parts: the first half discusses the reward system and its function, highlighting the neuronal correlates of the mesolimbic reward circuitry and the different phases of the reward experience. Furthermore, it will review the studies that focus on the reward system in relation to musical stimuli. This includes an overview of brain areas that activate in response to music.

The reward system

The mesolimbic reward circuitry is one of the areas that activates in response to musical stimuli. This system consists of multiple areas or ‘hubs’ that are connected through dopaminergic and glutaminergic white matter tracks (Haber, 2011). It is the common bond for all stimuli that are being perceived as pleasurable in any way. One can think of stimuli that are evolutionary relevant to survive, such as food and sex, but also drugs (cigarettes, cocaine and amphetamines) and esthetical pleasurable stimuli such as music and art (e.g. Schultz, 2006 on natural rewarding stimuli; Barrett, Boileau, Okker, Pihl & Dagher, 2004; Kelley & Berridge, 2002 on drugs; Zald & Zatorre, 2011, on musical stimuli; Kawabata & Zeki, on paintings). The satisfying feeling one has after a good meal is very different from that one has after listening to a favorite pop song and yet both of them cause activation of this reward network. As the reward system plays an important role in survival by stimulating an animal to look for those satisfying products, it is not surprising that the system is present and active from birth. An example of this is the liking response humans and primates have for sweet tastes. This can for instance be objectively measured by the licking of the lips that occurs as a reflex in response to the pleasurable taste: infants and primate respond in this way to a sweet taste. It is thus an indication of rudimentary activation of the pleasure system (Berridge & Kringelbach, 2013).

In the science of affect, the field of study that researchers pleasure and displeasure, various subcortical and cortical areas have repeatedly reported to activate in response to rewarding stimuli. For an overview of the areas, one can look at figure 1. Areas are grouped according to their spatial relation towards each other. The figure also highlights the essential white matter tracks that connect the areas. In the blue area in the figure, one can find essential dopaminergic nuclei that are located in the brainstem, such as the ventral tegmental area (VTA), the dorsal and ventral striatum, the amygdala and the insula. The VTA is the origin of the most essential dopaminergic track of the network, with neurons projecting to the nucleus accumbens (NAcc) in the ventral striatum (e.g. Haber, 2011; Camalier & Kaas, 2011; Berridge & Kringelbach, 2013; Salimpoor & Zatorre, 2013). Both VTA and NAcc are known to be active during reward processing of both natural stimuli (sex, food) as drugs (Kelley & Berridge, 2002; Berridge & Robinson, 2003).

(18)

Apart from the subcortical regions, the cingulate gyrus, orbitofrontal cortex (OFC) and inferior frontal cortex (IFC) are also part of the reward network (Haber, 2011). These areas can be found in the green and yellow areas of figure 1. Together, the discussed areas maintain an intricate balance between pleasure and displeasure. Manipulations of specific areas, for instance with microinjections of drugs that mimic certain neurotransmitters, can change a usual liking response to a response that is highly aversive (Berridge & Kringelbach, 2013). However, in some instances, an individual can consciously manipulate the reward response. This correlates with activation of the prefrontal regions that are known to regulate motivations that influence the reward responses (Zatorre, 2014). The core reward system is identified in both animals (rodents, but also primates and other mammals) and humans and is therefore thought to be

Figure 1: Schematic representation of the crucial areas of the reward network.

Lines indicate important white matter tracks that connect areas, direction not specified. Dotted arrows indicate the dopaminergic projections from the VTA to other areas.

(19)

evolutionary elementary (Zatorre, 2014).7 Yet, what differs humans from other animals is that humans have extensive bidirectional connections between the subcortical areas and the cortical ones. This connectivity causes humans to be able to enjoy abstract stimuli, such as music (Salimpoor & Zatorre, 2013).

The reward system works in a reinforcing cycle consisting of three different phases: wanting (anticipatory phase), liking (peak pleasure, consummatory phase) and learning. Figure 2 shows a schematic representation of these phases and their relationships. We want or anticipate a stimulus that is good for us (‘wanting/anticipation’ in figure 2) and in response to this stimulus we get a rewarding response in the form of a pleasurable feeling (‘liking/pleasure’ in figure 2). In this manner, the reward circuitry enhances adaptive behavior because we learn that exposure to the stimulus will give us this pleasurable feeling again (‘learning’ in figure 2). On the other hand, if we make a wrongful prediction, we won’t experience pleasure and will therefore learn that we need to seek another stimulus next time.

7_{Studies normally specify whether activation of these areas is more left or right orientated. However, in general}

reward and in musical reward, it is hard to say anything conclusive about the potential lateralization of reward processing because studies up until now have not found the same results. Therefore, this text will not go further into the discussion of lateralization and stick to general activation of areas, assuming that it happens both in left as right side of the brain.

Figure 2: Phases of reward

Lines indicate order in which phases occur. This is a simplification and in real-time different phase cycles can occur at the same time or overlap with each other. Red lines show how in aesthetic reward like music, violation can also lead to pleasure.

(20)

This is indicated with the red arrow from ‘wanting/anticipation’ to ‘learning’ in figure 2. However, a stimulus prediction can also be wrong: this will be used by the system to learn and adjust for the next anticipatory phase (Schultz, 2006; Schultz, 2015; Salimpoor & Zatorre, 2013).8 In aesthetic stimuli, a negative outcome can still sometimes lead to the experience of pleasure. This is called a pleasant surprise (discussion on pleasant surprises in musical reward by e.g. Huron, 2006; Gebauer, Kringelbach and Vuust, 2012). Pleasant surprise is thought to be connected to the specific connectivity between subcortical and cortical areas that have a mediating function in aesthetic reward. Cortical areas, such as the prefrontal cortex, play a role in reasoning, evaluation and appraisal. These areas probably contribute to the specific rewarding response for aesthetical stimuli and also to this specific phenomenon of the pleasant surprise (Salimpoor & Zatorre, 2013). Figure 2 shows with a red arrow how a violation can still lead to the liking phase. Within the reward research different theories exists on the different roles of different areas within these phases (for a discussion of this topic in aesthetic rewards, see Salimpoor & Zatorre, 2013).Related to these phases is the activation of the Autonomic Nervous System (ANS). The insula and hypothalamus are important mediators of this connection (they are shown in figure 1, in red). A violation of an expectation can, evolutionary speaking, mean that an animal has to respond extremely fast to survive (e.g. if a zebra suddenly sees a lion approaching, it has to run to not be eaten). Activation of the ANS prepares the body for this fight or flight response. (Gazzaniga et al., 2004). However, for non-threatening stimuli this response can simply lead to arousal that is being perceived as pleasurable, because there is no real threat.

Music

One of the main techniques to study activation of the reward system is with Positron Emission Tomography (PET). With PET, one can measure Cerebral Blood Flow changes, from which one can induce which areas of the brain need more blood compared to other areas. Blood & Zatorre (2001) conducted a PET study while participants were listening to music that induced pleasure and chills. They measured increased activation in the dorsomedial midbrain, including ventral striatum and also reported significant activity in insula and orbitofrontal cortex (OFC). Another paper reported similar results after conducting a PET study that focused on pleasurable music experience in response to unfamiliar musical pieces (Brown, Martinez and Parsons, 2004). The researchers reported increased activity in multiple limbic and paralimbic areas including the nucleus accumbens (ventral striatum), insula and the cingulate gyrus.

PET has both advantages and disadvantages: it has a high temporal resolution and a person can move whilst being tested. However, the spatial resolution is very low, and it requires the injection of radioactive isotopes. Currently, functional Magnetic Resonance Imaging (fMRI) is thus being used more

8_{For a review on the role of specific areas and neurotransmitters in each phase, see Schultz, 2005 or Salimpoor}

(21)

regularly. In an experimental set-up with pleasurable music listening, Menon and Levitin (2005) reported increased activity in the mesolimbic reward circuitry, specifically NAcc and VTA, and in connected areas such as the insula, hypothalamus and orbitofrontal cortex (see figure 1). In another fMRI study, pleasurable music listening correlated with activation of the inferior frontal gyrus, insula and ventral striatum (Koelsch et al., 2006).

With use of PET and fMRI, a small but increasing body of literature has shown activation of key areas of the reward network when humans have a pleasurable music listening experience. The reported areas are largely overlapping with the general reward areas that can be found in figure 1. Both NAcc and VTA are significantly more active during a pleasurable music listening experience compared to a non-pleasurable one (Blood & Zatorre, 2001; Blood, Zatorre, Bermudez & Evans, 1999; Brown, Martinez & Parsons, 2004; Menon & Levitin, 2005; Koelsch, Fritz, Müller, & Friederici, 2006; Suzuki et al., 2008; Salimpoor, Benovoy, Larcher, Dagher & Zatorre, 2011; Salimpoor et al., 2013; Mas-Herrero and 2018). Furthermore, studies report activation of the caudate nucleus (located in the dorsal striatum) in relation to musical pleasure(Salimpoor et al., 2011; Salimpoor et al., 2013; Mas-Herrero, Dagher & Zatorre, 2018). Modulators of the ANS (hypothalamus and insula) also respond to musical stimuli (e.g. Brown et al., 2004; Menon & Levitin, 2005; Koelsch et al., 2006; Salimpoor et al., 2011). Even though these areas are not the essential hubs of the reward system, they play an essential part in mediating between the ANS end the reward system. Activation of them thus shows that the reward system not only activates during musical listening, but also connects to these other hubs in a similar manner as it would do with other stimuli. Lastly, the OFC and IFC are reported in various studies, as well as the cingulate gyrus. It is therefore thought to play an important role in emotional processing (e.g. Blood & Zatorre, 2001; Brown et al., 2004; Menon & Levitin, 2005; Suzuki et al, 2008; Salimpoor et al., 2013).The IFC activation is likely to be related to the valence (positive or negative) representation that comes with pleasure (Bechara et al., 2000). The OFC is hypothesized to be part of the valuation process of musical stimuli: this refers to a process (either subconscious or conscious) where an individual valuates a stimulus as being pleasurable or not (Salimpoor et al., 2013).

In addition to these results that implicate general activation during musical pleasure, one can ask questions concerning (temporal or anatomically) details of this activation and if certain patterns of activation can be related to different phases of reward. One study so far has reported anatomically distinct dopamine release (and subsequent activation of areas) related to certain phases, but these results have not yet been replicated (Salimpoor et al., 2011). In summary, one can conclude that both the key hubs of the mesolimbic reward circuitry as the areas facilitating this network are repeatedly being reported in experiments that study (pleasurable) music listening. The mesolimbic reward system behaves in a similar manner for musical stimuli as it does for evolutionary relevant stimuli. From this research we can conclude that basic musical pleasure is evoked by the behavior of this system. It also is important in the experience of musical emotions.

(22)

Chapter 4: The relationship between reward and emotion

This chapter will discuss the different stances on the relationship between reward and emotion that currently exist. First of all, it will review the (implicit) stances that still treats reward as a (sub)component of emotion. It will also discuss the role of Autonomic Nervous System (ANS) arousal in different studies on music and reward and explain the complex relationship it has with both reward and emotion. In the second section the possibility of musical reward as a force driving musical appreciation will be discussed, with specific attention to the concept of anticipation.

Reward as a part of emotion

Studying the mesolimbic reward system in relation to music has shown that music has the ability to activate this system in a similar manner as other (evolutionary relevant) stimuli do. Even though there is now a distinct line of research focusing on impact of the reward system on our musical experience, it does not mean that that reward and musical emotion are now treated as two completely independent processes. This idea is very unlikely to be true, simply because reward and emotion networks are deeply intertwined in the brain. Yet, there are different stances on the relationship between the two. There are studies that focus on musical reward as a component on emotion, whereas other lean more toward studying reward as an independent mechanism within musical processing. However, not many studies explicitly ask questions about this. Considering the growth in the field, this is remarkable. Interestingly enough, the lack of this explicit question does not mean that there are no implicit answers being given. In the literature that exists, I have discovered implicit stances on this topic, hidden in the choice of terminologies and experimental set-ups. Even though it is not yet possible to give any conclusive answers now, it is worth investigating these different stances to make preliminary hypotheses for future experimentation. Early research on musical emotion has treated the reward areas as a distinct part of the emotional experience. In recent research on emotion, one sees that this stance has weakened: there is acknowledgement of the reward system as a relatively independent network. However, one can see in in various studies on the neural correlates of musical emotion and the mapping of musical emotion in the brain that reward is still often treated more or less as part of emotion (e.g. Trost, Ethofer, Zentner & Vuilleumier, 2011; Koelsch, 2014; Koelsch, 2015; Koelsch, Skouras & Lohmann, 2018).

The experimental set-up and terminology used in several articles demonstrate the role reward is assumed to have in musical emotion. The paper of Blood & Zatorre (2001) provides good example of the entanglement of the systems that can be found in the set-up of the experiments researchers conduct. These authors used the musical chill response, a response we now know to be specifically related to the activation of the reward system, as a model for the objective study of musical emotion and not of musical pleasure. This set-up shows the entanglement of the reward system, the chill response and the experienced musical emotion. Even though this paper is already over fifteen years old, the ambiguous

(23)

terminology is also used in more recent papers. A similar approach and choice of research set-up is used by Grewe, Kopiez and Altenmüller (2009), who studied musical chills ‘as indicators of individual emotional peaks’ (p. 351). The use of terminology is also sometimes confusing and leaves the reader with questions considering the author’s stance on the involved mechanisms. Salimpoor et al., (2011) state that: ‘It has been empirically demonstrated that music can effectively elicit highly pleasurable

emotional responses and previous neuroimaging studies have implicated emotion and reward circuits of

the brain during pleasurable music listening’ (p. 257, italics added). This quote acknowledges the distinct emotional and the reward circuits, but it is interesting that the writers refer to pleasurable emotional responses. It implies that the authors define pleasure as an attribute of emotional experience. Additionally, ambiguity is found in experiments using psychophysiological measurements for arousal of the Autonomic Nervous System (ANS). In experimental research, emotion is defined as the combination of valence and arousal. With this definition, ANS activation (e.g. modulated by e.g. the insula and hypothalamus) is a very useful marker of emotion. However, it is also now known to be an essential marker of activation of the reward system. This again demonstrates complexities in the relationship between reward and emotion. Some studies use the arousal of the ANS to study pleasure and reward, whereas others use it more for musical emotions (e.g. Grewe, Nagel, Kopiez, & Altenmüller 2007; Grewe, Kopiez & Altenmüller, 2009; Salimpoor et al., 2009; Benovoy, Larcher, Dagher & Zatorre, 2011).

A part of the confusion that arises from these set-ups and choices of terminology, can be derived from the fact that studies report correlations between the between arousal, pleasure and emotion (e.g. Salimpoor et al., 2009). Musical pleasure and emotion must thus arise from an interplay of these mechanisms. Up until now, no paper has attempted to study these correlations in detail with the aim to learn more about the relationship of the involved mechanisms towards each other. This is no surprise, as it leads to statements that concern causality, a tricky and highly complicated topic in neuroscience. However, I find the idea that these mechanisms would activate simultaneously in response to music (and are thus all activated by some external input) very unlikely and wonder if it can be the case that one of them is indeed causing the others. For this purpose, I designed figure 3. This figure gives a schematic overview of three mechanisms interacting: the emotional network (possibly leading to an emotional experience, either with a positive or negative valance), the reward network (possibly leading to a pleasurable experience) and the ANS arousal (caused by the ANS modulators). Furthermore, the connections between the mechanisms are simplified, since bidirectional connections have been found between the networks (Peretz, 2010; Koelsch, 2010).

Figure 3 A refers to the findings of a study that correlated reward and emotion with the use of the ANS arousal. (Salimpoor et al., 2009). The researchers studied musical pleasure (caused by the mesolimbic reward circuitry) was by means of the subjective reports of pleasure and the musical chill response (self-reported by means of pushing a button). Emotional ANS arousal was assessed with psychophysiological measurements (heart rate, galvanic skin response, temperature, respiration rate and

(24)

blood volume pulse amplitude). This paper thus correlated subjective pleasure with ANS arousal, finding that a higher self-reported pleasure was positively correlated with measures of ANS arousal, and 80% of the time, chills occurred during the highest arousal. Again, a note can be made here concerning the set-up: Salimpoor and colleagues (2009) defined the ANS arousal as a part of emotion and not as a maker of pleasure, even though ANS arousal is known to also be a marker of activation of the reward system. Nonetheless, this paper is crucial as it is the first one that extensively correlated the three mechanisms with each other.

Figure 3: schematic overview of possible relations between reward network activation, ANS modulators and emotional experience.

If one assumes that there is no other mechanism interacting simultaneously with all three of the mechanisms, and thus activating them at the same time, there can be a couple of suggestions made for

(25)

the possible relationships between the three.9_{First of all, one of the networks could be the cause of the} other. The reward circuitry could cause the activation of the ANS modulators (insula and hypothalamus) which could constitute to the emotional experience. The fact that the reward system is positioned in the limbic system, and the limbic system is can activate quickly and automatically in response to any sensory input supports this hypothesis (see chapter 1 on the limbic system). Figure 3B shows the hypothesis. I have added two other options to the figure that do not attribute a causal role to the reward system. Figure 3C shows the reversed version of 3B, meaning that the emotional experience would cause the reward activation. This is more unlikely, as it is known that the emotional areas are distributed widely over the brain and also consist of cortical areas that activate later than subcortical areas (Koelsch et al., 2010; Peretz, 2010; see chapter 1). A last possibility is that ANS arousal could happen on its own, or by other means than the reward system, and from there on constitutes the emotional experience (see figure 3D). This does not exclude the found connections between the different mechanisms, but it would simply mean that the reward circuitry is not be the cause of the arousal. This is contrary to the ideas by Kringelbach and Vuust (2010) that will be discussed below.

With this section I have tried to show that even though there is a mutual relationship of reward and emotion with physiological measures of arousal, there can be many questions be asked concerning the nature of these relations. There is currently not enough knowledge to determine which plays a larger role. To study this one needs research set-ups that study the details of the neuronal tracks that connect the different active areas. Now that I have highlighted this and the ambiguities in research considering the role of reward and emotion, I will review a theory that does essentialize the role of the reward system.

Reward as driving force for musical emotion

In 2010, Vuust and Kringelbach published a paper about the possible essential role of the reward system in the human ability to appreciate music. With this theory they are the first to make an explicit claim about this, stating that: “[the reward system could be a] fundamental mechanism underlying structuring music as a meaningful percept.” (Vuust & Kringelbach, 2010, p. 166). The researchers argue that the musical reward system is essential in the human capacity to understand and appreciate music (Vuust & Kringelbach, 2010).

The paper starts with the question of why it is possible that music has the ability to activate our reward system the way it does. Their answer lies in the concept of anticipation. This concept is not an unfamiliar one in the history of musical research. Music, as it is a temporally structured phenomenon, derives part of its aesthetical power through playing with prediction, and the confirmation or violation of these predictions (Huron, 2006; Vuust & Kringelbach, 2010). For example, in Western musical theory

9_{In this text I have deliberately chosen to suggest 3 hypotheses that highly contrast each other and that all of them}

assume a clear cause of the activation. These can serve as the ‘extreme’ ends of a spectrum in which more moderate hypotheses can be inserted.

(26)

where the 5th chord of the key in a piece of music is traditionally followed by a 1st chord (an authentic cadence). If the composer instead chooses to go to, for instance, a 6th chord (a deceptive cadence), it will surprise the listener. However, a small violation can still be appreciated. Musical harmony expectancies and violations have been extensively studied and can be found seen in the form of a mismatch negativity in specific areas of the brain if one measures it with electroencephalography (e.g. Koelsch & Friederici, 2003; Koelsch, Friederici & Schröger, 2000). These kind of (implicit) rules that are different in every musical culture ensure an intricate balance of expectation and confirmation that is learned through exposure to music (Soley & Hannon, 2010). They are the result of a learning process that starts as soon as an infant is exposed to music. The learning starts with general rules. An example of this is that infants prefer the rhythms in their own culture over other rhythms (Soley & Hannon, 2010).This and other types of anticipation, as well as how humans are able to learn the implicit rules so quickly, have been widely discussed (e.g. Huron, 2006).

Vuust and Kringelbach (2010) then argue that this anticipation has its roots in the reward circuitry and support this with the musical chill phenomenon. They substantiate their theory with results of different papers. Various studies have found correlations between the occurrence of musical chills and mesolimbic reward circuitry activation (e.g. the paper by Blood & Zatorre, 2001). Whilst listening to music, humans constantly predict what will be heard next. This results in either the confirmation or the violation of these predictions. Wrongful predictions cause Autonomic Nervous System (ANS) arousal to activate evolutionary relevant survival mechanisms. High activation can cause musical chills. Furthermore, the writers hypothesize that the cognitive appraisal following the musical chill, is what actually makes the chill pleasurable. Their theory is highly congruent with the Huron’s theory on anticipation (Huron, 2006). Musical chills are a ‘prove’ of how music taps into an evolutionary relevant system.

If anticipation is indeed key, it implicates that knowledge is needed before it can occur. If an individual knows a piece, one knows what musical input is coming and thus if he or she can expect something to like or something one does not like. However, it seems that this type of explicit familiarity is not needed for reward activation.In reward research, studies have been conducted with different types of familiarity. First of all, it was shown that reward activation happens during explicit familiar pieces. Blood & Zatorre (2001) asked participants to choose pieces of music that elicited strong emotions and used anther participant’s preferred pieces as control stimuli. Salimpoor et al. (2011) used the same approach. Both studies found activation of the mesolimbic areas during intense pleasurable music listening experiences (e.g. when participants experienced chills and reported high pleasure). However, other studies deliberately tested the musical reward response with unfamiliar stimuli and also found activation of these areas (Brown et al., 2004; Menon & Levitin, 2005).

With the use of fMRI and an elegant research design, Salimpoor et al. (2013) observed similar activation: they presented participants with 60 excerpts of unfamiliar music while undergoing fMRI scanning. Next, they offered participants bid on the excerpts with their own money. Buying the excerpt

(27)

would mean they could hear it again. This paradigm is called an auction paradigm and is used as an objective measure to assess the value an item has to someone. If a stimulus is valuable to a participant, he or she will pay for it to hear it again. The value of the stimulus in this case represents the liking response to the stimulus (and hence the activation of the reward system). If a stimulus is less valuable (and thus less pleasurable), a participant will spend less or even no money at all. As a result of this study, Salimpoor and colleagues reported activity in the dorsal and ventral striatum that correlated with the reward value the participants assigned to the musical excerpts. The writers conclude that ‘the explicit familiarity is not necessary for activity in the dopamine target regions, which may also depend on implicitly formed expectations based on previously acquired musical knowledge’ (p. 218, Salimpoor et al., 2013). From these studies we can conclude that, during a pleasurable musical listening experience, activation of the mesolimbic reward circuitry happens regardless of explicit familiarity with the stimuli.10

Research thus supports the theory that explicit familiarity is not needed for the activation of the mesolimbic reward circuitry and that implicit familiarity is sufficient. As soon as we can hear and are exposed to music, we learn about what to expect of our culture’s music.This can be seen in infants that already prefer the meter of their own culture over another but can quickly change this preference if they have more exposure to another meter (Soley & Hannon, 2010). With the theory of Vuust and Kringelbach in mind, one can wonder to what extend there is an innate anticipatory mechanism that functions even before we have heard any music at all. Research suggests that the answer to this positive, because it is known that new born infants already detect the beat (Winkler, Háden, Ladinig, Sziller & Honing, 2009). Beat perception requires the ability to be able to predict what is coming next. If one studies beat perception with electroencephalography, one will find a mismatch negativity signal if a prediction of regularity is being violated (Winkler et al., 2009). A question that remains in the context of musical reward, is whether these abilities also activate the reward system from birth on. Considering the fact that the mesolimbic reward system also works as a motivation system to seek a stimulus again in evolutionary relevant stimuli (Schultz, 2006), it might even be possible that this type of activation would also work if music, mediating a learning process of musical knowledge by motivating humans to seek for music. If this is the case, the basic anticipatory mechanisms would develop during exposure to musical stimuli. Vuust and Kringelbach (2010) suggest that this type of learning could happen through predictive coding. This theory is further developed by Gebauer, Kringelbach and Vuust (2012): they propose a Bayesian predictive coding framework that could mediate changes in the pleasure cycles

10_{One can argue that musical training has an effect on explicit familiarity too. Musical training enhances the}

knowledge of the building blocks that make up musical pieces (e.g. rhythmic structures, tonal patterns). It can make implicit knowledge explicit. As an effect, the individual can pay more attention to these structures and overall listen differently to music than someone without musical training. However, there are studies with participants with and without musical training that show know difference in activation of the reward circuitry (Blood & Zatorre, 2001; Menon & Levitin, 2005; Suzuki et al., 2008; Salimpoor et al., 2011; Mas-Herrero et al., 2018).

(28)

during musical listening. Learning from previous input and by that the change of what to expect can be accounted for by this theory. They emphasize the role of dopamine in learning and anticipation and support this with results of the previously discussed study by Salimpoor et al. (2011), who also found distinct dopamine release during wanting and learning phases of reward.

Lastly, it is needed to discuss the implications of such a theory for the relationship between reward and emotion. Vuust and Kringelbach (2010) do mention this: they connect their theory to the musical emotion evocation mechanisms as described by Juslin and Västfjäll (2008) and hypothesize that their mechanisms would act on top of the general principal of musical anticipation. This shows the fundamental role they attribute to anticipation, and more in general, musical reward. A last important component of Vuust & Kringelbach’s theory is that they state that all musical emotions have the possibility to be appraised as pleasurable. This is a new addition to the existing views on musical emotion and reward. It would mean that, whatever emotion one is feeling, the pleasurable appraisal (mediated by the reward circuitry) is the reason that musical emotion is experienced as pleasurable. It would also give a simple answer to the paradox of sad music (i.e. the phenomenon that we can enjoy listening to music that makes us feel sad) (Zentner, Grandjean & Scherer, 2008; Huron, 2011). I need to note here that this addition does not give any conclusive answers on the actual constitution of the emotional experience itself, or the role of the reward system in this experience. It merely attributes a specific role of the reward circuitry by means of a positive appraisal to the emotions. This could very likely be mediated through cognitive appraisal areas such as the previously discussed OFC and IFC.

The discussed stances and theories on musical reward and emotion show that there is a need for specified research to answer these questions. The next chapter will discuss musical anhedonia as a possibly excellent case study for the further research of musical reward.

The Musical Motivation Motor: on the role of the mesolimbic reward system in musical emotion and appreciation