The effect of Deep Brain Stimulation in the Subthalamic
Nucleus and Nucleus Accumbens on Decision Making
Literature Thesis
M.S. Seinstra
Master Cognitive Neuroscience
University of Amsterdam
Supervisor: Dr. W. van den Wildenberg
Psychology Department
University of Amsterdam
Co-assessor: Prof. Dr. K.R. Ridderinkhof
Psychology Department
University of Amsterdam
2
Contents
Abstract... 3
1. Introduction... 4
1.1 Decision making processes... 4
1.2 Impulsivity and addiction... 5
1.3 Deep brain stimulation therapy and target areas... 6
2. The role of the nucleus accumbens and subthalamic nucleus in decision making... 7
3. Deep brain stimulation and impulsivity... 9
4. Deep brain stimulation and addiction... 10
4.1 Substance addictions... 10
4.2 Non-substance addictions: pathological gambling... 12
5. Discussion... 14
3
ABSTRACT
In the last decade the successful application of deep brain stimulation (DBS) as treatment for several motor- and cognitive disorders has resulted in a large body of clinical and experimental studies advantageous as well as disadvantageous 'side-effects' on cognition. In this literature thesis I discuss the effects of DBS in brain areas involved in processes related to decision making: The Nucleus Accumbens (NAc) is mostly involved in the valuation of different types of rewards and is mainly used as target for DBS in substance addiction. The Subthalamic Nucleus (STN) is the main target for treatment of Parkinson's Disease (PD) and seems to be involved in reward guided action as well as increasing the decision threshold in high-conflict choices. In this thesis the main focus is on impuslivity and addiction, and a distinction is made between behavioral and cognitive impulsivity. While stimulation of both the NAc and STN leads to differential motivation for addictive substances and natural rewards, the precise neural mechanisms for this distinction are unclear. NAc-DBS seems to have a remarkable effect on craving, rendering patients completely abstinent for years after surgery. Though no study has directly assessed the effect of STN-DBS on substance addiction in humans, lesion studies indicate that STN-DBS might increase substance abuse in subjects with a history of substance abuse. STN-DBS seems to alter valuation of losses, resulting in increased cognitive impulsivity. However, the influence of dopaminergic treatment in PD patients might cloud the effects of STN-DBS on decision processes. More research is needed to elucidate the effects of DBS on cognitive impulsivity, in which one should be able to distinguish the effects of behavioral impulsivity from cognitive impulsivity.
4
1. INTRODUCTION
Deep brain stimulation (DBS) is a successful treatment for movement disorders such as Parkinson's disease. Recently, DBS has gained popularity for psychiatric disorders including obsessive compulsive disorder, depression and addiction. In recent years, a large body of literature has been published describing the effects of DBS on cognition. In this literature thesis, I discuss how the use of deep brain stimulation in areas related to the decision network has led to insights about processes related to decision making. To this end I first give a short theoretical background of decision making processes, as well as the relevant brain areas. Then, I will discuss the findings of both clinical and non-human animal studies describing effects of DBS, with a specific focus on impulsivity and addiction.
1.1 Decision making processes
Decision making is an important aspect of our everyday functioning. It comprises facets from choosing which movements enable you to reach your goal to choosing the clothes to wear today, as well as the choice to avoid addictive substances or to inhibit compulsive urges. The decision making process entails several steps that can be distinguished based on theories from psychology and economics and the underlying neuronal processes (see also Rangel et al., 2008; Grabenhorst and Rolls, 2011).
The first step is defined as a sensory step, in which the available stimuli are represented in terms of their identity and subjective intensity, independent of their subjective reward value or pleasantness. In this step information about the available options is gathered and represented in an objective way. In a review, Grabenhorst and Rolls (2011) identify sensory-related regions like the inferior temporal visual cortex, primary taste cortex, olfactory cortex and the primary somatosensory cortex as areas involved in the representation of choice options independent of subjective/reward value. Furthermore, information about the internal and external states are seen as an important aspect of this step, e.g. the satiation level of the organism or the necessary effort or the predation risks in the context of foraging decisions.
During the following step the stimuli representing the different options are valuated, taking into account the internal (motivational) states and the information about the external states gathered in the previous step. In this step it is assumed that the subjective value signals for each option are scaled in such way that they can be compared without a loss of the identity of the distinct options available (Grabenhorst and Rolls, 2011). Several areas are found to represent subjective value signals of which the orbitofrontal cortex (OFC) (Schultz et al., 2000; Grabenhorst and Rolls, 2011), the prefrontal cortex (Watanabe, 1996), the parietal cortex (Dorris and Glimcher, 2004; Sugrue et al., 2004) the amygdala, the anterior cingulate cortex (ACC) (Grabenhorst and Rolls, 2011), and the striatum (Kawagoe et al., 1998) are thought to play a key role in predicting reward value. For example, the activity of neurons in the OFC reduce their firing to zero in response to food stimuli when animals are fed to satiety (Rolls et al., 1989; Critchley and Rolls, 1996), and BOLD signals in the OFC correlate with pleasentness ratings (Grabenhorst et al., 2007; Kringelbach et al., 2003; Rolls et al., 2008) or the amount of money won or lost in single trials (O'Doherty et al., 2001). Also in research on Pavlovian valuation the OFC, amygdala and, through efferent connections from the OFC, the ventral striatum are found to represent rewarding stimuli (Fudge et al., 2002; Haber et al., 2006).
'Value' is mainly an economical term that we use in everyday life to describe the worth of a good, making the good interchangeable with other goods of equal value. The use of the term value to describe specific neural processes or activity might lead to confusion considering that the brain's mechanisms might not remain within the limits set by the definition of 'value' as we use it in everyday life. Since the main function of the brain is the optimization of action to obtain a specific goal, other terms that are more closely associated with action such as 'motivation' or 'incentive salience' signals might describe similar neuronal representations perhaps even better than the term '(subjective) value'.
In decision processes less based on personal preferences (i.e. value-based decision making) and more on external (learned) optimal options, as in the case of reinforcement learning paradigms, two distinctive types of behavior have been described: goal-directed and habitual choice behavior (e.g. Balleine et al., 2009).
Goal-5
directed behavior is different from habitual behavior in that, if the response to a stimulus which was previously rewarded is no longer rewarded, the response to the stimulus diminishes. This also occurs when the reward obtained through responding to the stimulus is devaluated by means of satiation. Therefore, the goal of the action is essential for the behavioral response to occur. In habitual behavior the association between the stimulus and response is what is leading the behavior, regardless of the outcome. Normally choice behavior shifts from goal-directed to habitual with overtraining. For example, rats trained to press a lever (response) when hearing a sound (stimulus) for reward will cease pressing if no food is delivered anymore (goal-directed behavior). Though with overtraining, rats will keep pressing the lever after stimulus presentation, even if no food is delivered anymore (habitual behavior).
These two types of choice behavior seem to have two distinct valuation networks which are connected as a spiral form from the striatum via the thalamus to the cortex and back to the striatum, going from the medial and ventral parts of the prefrontal cortex to the ventral striatum, from the anterior cingulate cortex to medial parts of the caudate and putamen in primates (dorsal striatum in rodents) and from the dorsolateral prefrontal cortex to the more dorsal regions of the caudate and putamen (Knutson et al., 2009; Haber, 2003; Lehéricy et al., 2004). The more associative cortices like the medial prefrontal cortex and its projections to the dorsomedial striatum (DMS, caudate in primates) are more associated with goal-directed behavior, while the sensorimotor cortices (SM) and their connections to the dorsolateral striatum (DLS, putamen in primates) seem to be more involved in habitual behavior (see model of Balleine et al., 2009).
The valuation of the available options lead, in the optimal situation, to the choice of the option with the largest advantage and the lowest disadvantage, i.e. the option with the highest subjective value, after which the appropriate action is initiated. Areas thought to be mainly involved in this decision step are the medial prefrontal cortex (mPFC), cingulate cortex and striatum in cases of value-based decision making, goal-directed and habitual choice behavior, respectively (Grabenhorst & Rolls, 2011). The final step in the process is the evaluation of the outcome of the action, and the accompanying learning processes to optimize future decisions. This step is essential in goal-directed behavior, but seems to be absent in habitual behavior.
1.2 Impulsivity and addiction
The above described steps in the decision process give a global view of the aspects that are important for making optimal decisions and provide a framework for the definition of concepts linked to decision making. One of these concepts is impulsivity. Several different definitions of impulsivity exist, involving different steps in the decision making process. Two well-studied types of impulsivity are cognitive and behavioral impulsivity (White et al., 1994), also called choice and motor impulsivity, respectively. Behavioral impulsivity is defined as the fast and inaccurate responding due to the inability to suppress behavior or a prepotent response, while cognitive impulsivity leads to a bias for choices leading to reward on the short run but less optimal or negative consequences on the long run. Therefore, behavioral impulsivity seems to involve the action-control processes of decision making, while cognitive impulsivity mainly implicates the valuation step in the decision making process, in the sense that more value is attached to immediate gratification.
Whereas differences in impulsivity are common in humans and might not be pathologic, it plays a role in several psychiatric disorders. Addiction is a more encompassing term describing a chronic disease covering several behaviors. Addiction involves reward, motivation, memory and related circuitries in the brain (Smith, 2012). The nucleus accumbens, anterior cingulate cortex, basal forebrain and amygdala are mentioned as a selection of areas involved (Smith, 2012). Typical addictive behaviors are described as altered impulse control, and the dysfunctional pursuit of rewards despite the adverse consequences, which suggests a large role for both behavioral and cognitive impulsivity as well as compulsivity in addiction. Compulsivity is defined by Dalley et al (2011, p. 680) as "actions inappropriate to the situation which persists, have no obvious relationship to the overall goal and which often result in undesirable consequences." The persistence and lack of relationship with the overall goal are aspects shared with habitual behavior, suggesting the involvement of similar
6
and drugs, but also food addictions, pathological gambling, shopping and sex are more and more seen as addictions, of which the latter three are described as non-substance addictions.
All these types of addictions involve impulsive, compulsive and habitual elements. The subtle difference between impulsivity, compulsivity (and possibly habits) are suggested to lie in response control processes (Dalley et al., 2011; Torregrossa et al., 2008), which could be seen as impairments of the valuation circuit involved in goal-directed behavior and a (subsequent) failure to have appropriate influence over the execution of actions: in cases of cognitive impulsivity this would be a non-optimal valuation of the possible consequences of actions leading to a bias for immediate gratification; in cases of behavioral impulsivity a quickened termination of the valuation step, leading to an early decision and quicker response (Frank et al., 2006 ); in cases of compulsivity a decoupling of a goal-directed cognitive valuation process and the subsequent action repertoire so that even though the inappropriateness of actions are known, the actions are still
executed; and in cases of habit the overall absence of value-evaluation of the choice options (Balleine et al., 2009).
1.3 Deep brain stimulation therapy and target areas
Most studies assessing the effect of STN-DBS on cognitive functioning have as treatment group Parkinson patients, of which most receive dopaminergic medication. Parkinson's disease (PD) is characterized by cell loss in the substantia nigra pars compacta, leading to a decrease of dopaminergic transmission to basal ganglia output structures (mainly the striatum). It is hypothesized that this leads to an increased activity in the direct pathway GPi-thalamus) and a decreased activity in the indirect pathway (cortex-putamen-GPe-GPi/STN) of the basal ganglia, resulting in the observed hypokenetic features of PD (DeLong, 2000).
Since the first reports of positive results of deep brain stimulation (DBS) as treatment for the motor symptoms of Parkinson's disease (PD) in 1989 (Benabid et al., 1989), DBS has gained considerable popularity as treatment for not only motor disorders, but more recently also for more cognitive disorders. The advantage of DBS treatment over surgical lesions lies in the adaptability of the stimulation parameters and the reversibility of the treatment. It has been proposed as treatment for Gilles de la Tourette, depression, (substance) addiction, Obsessive Compulsive Disorder (OCD), obesity and even for pathologic gambling. That DBS affects cognitive processes comes forth from studies of DBS in motor disorders, for example the increase or decrease of impulse control disorders (see Broen et al., 2011) or cognitive problems, like apathy or anxiety observed in Parkinson's patients treated with subthalamic nucleus (STN) DBS (e.g. Temel et al., 2006).
The effects of DBS do not only depend on the target area, but also on the surrounding areas and white matter tracts going through or surrounding the target area, as well as the electrode design and the parameters used for stimulation (see Gubellini et al., 2009 for a comprehensive review). These factors make it difficult to make comparisons between studies and might result in conflicting findings. One popular hypothesis regarding the effects of STN-DBS is that DBS inactivates the target structure, mimicking a lesion (Benazzouz et al., 2000; see Gubellini et al., 2009). Another hypothesis is that stimulation of the STN disrupts the abnormal basal ganglia activity in Parkinson's patients (Hammond et al., 2007; Liu et al., 2008), causing tonic activation when using the right parameters.
The STN is part of the so called 'indirect pathway' and the 'hyperdirect pathway' of the basal ganglia. The main glutamatergic output of the STN goes to the internal segment of the globus pallidus (GPi) and the substantia nigra pars reticulata (SNr). Other output projects to the external segment of the globus pallidus (GPe), the ventral pallidum (VP), the pedunculopontine nucleus (PPN), the striatum, the NAc, the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc). The STN receives input from the VP and GPe ('indirect pathway'), several areas of the cortex, including the mPFC ('hyperdirect pathway') and the
parafiscicular nucleus of the thalamus, the PPN, dorsal raphe, and dopaminergic input from the VTA and SNc (Parent and Hazrati, 1995a, 1995b). The position of the STN is optimal for integration of limbic, associative and motor information, which is supported by recent patient studies showing involvement of the STN in, for example, obsessive-compulsive disorder (Mallet et al., 2008) and pathological gambling in PD patients (Ardouin et al., 2006).
7
Next to the STN, the NAc is also often considered as target structure for DBS, though not as treatment for PD. The striatum can anatomically be divided into the dorsal striatum, in humans consisting of the caudate and putamen, and the ventral striatum, of which the NAc only comprises the most ventral part. Within the NAc there is a clear distinction between the shell and core subregions. The NAc receives its main excitatory input from the medial orbitofrontal, anterior cingulate and medial parahippocampal cortical areas, thalamic nuclei, the amygdala and the hippocampus, and is inhibited by the ventral pallidum (see Basar et al., 2010 for a detailed review). The NAc receives dopaminergic input from the VTA and medial part of the SNc. The NAc core projects to the subcommissural ventral pallidum and SNr, while the NAc shell projects to the ventromedial part of the ventral pallidum, the SNc, VTA and lateral hypothalamus (Basar et al., 2010). It is suggested that the NAc integrates emotional, contextual and motivational information which is then led to structures involved in feeding/drinking, cognitive and motor functions (Basar et al., 2010). DBS in the NAc has been applied in patients with OCD (Aouizerate et al., 2005; Franzini et al., 2010; Luigjes et al., 2011; Sturm et al., 2003;), depression (Bewernick et al., 2010; Bewernick et al., 2012; Grubert et al., 2011; Malone et al., 2009; Schlaepfer et al., 2008) and addiction (Heldmann et al., 2012; Kuhn et al., 2009, 2011; Müller et al., 2009; Zhou et al., 2011).
In the following paragraphs I will first go into what is known about the role of the STN and the NAc in decision making, followed by a review of how deep brain stimulation in these areas influences reward learning, impulsivity and addiction, respectively, and what this tells us about the involvement of these areas in decision making processes.
2. THE ROLE OF THE NUCLEUS ACCUMBENS AND SUBTHALAMIC NUCLEUS IN DECISION MAKING
Considered to be an important structure in the decision-making network, the NAc has received far more attention with regard to decision processes than the STN, which was before recently thought to be mainly involved in motor processes. In the last decade, several studies have investigated the role of the STN in valuation of reward and cognitive impulsivity (Baunez et al., 2002, 2005; Bezzina et al., 2009; Uslaner et al., 2005; Uslaner and Robinson, 2006; Winstanley et al., 2005). Given the anatomical connectivity of the NAc and the STN, one could hypothesize that the NAc is more involved in valuation processes, while the STN is more involved in using this value information to guide action.Consistent with this idea, fMRI research indicates that activation of the ventral striatum correlates with expected reward value while manipulating magnitude (Knutson et al., 2001, 2005) or reward probability (Abler et al., 2006; Dreher et al., 2006; Knutson et al., 2005). The NAc is thought to play a key role in the acquisition of Pavlovian stimulus-reward associations, which in turn influences performance on goal-directed tasks (Yin et al., 2008 ). The core region of the NAc seems to be more involved in preparatory appetitive behaviors (Yin et al., 2008), such as approach behavior, while the shell region of the NAc seems implicated in the consummatory appetitive behaviors. Shell neurons respond to rewards and aversive stimuli before learning (i.e. hedonic evaluation and valence), and lesions of the shell affect Pavlovian effects on instrumental paradigms when different rewards are available, but not when only one type of reward is available (Corbit et al., 2001).
One important aspect of value-based decision making is investigated with inter-temporal choice paradigms. In these paradigms subjects make a choice between a smaller, immediately available reward and a larger reward delivered after a delay. Usually either reward magnitude or delay of the larger reward is varied to find the so-called indifference point; the parameters at which the small, immediate option is chosen equally often as the larger, delayed reward, suggesting a similar subjective value for both. It has been found that rewards that will be obtained in the future have a decreased subjective value compared to the same reward received immediately, which is called the discounting of future rewards. Next to its role in Pavlovian
conditioning, the NAc core seems to be involved in inter-temporal choice, as bilateral lesions resulted in more cognitive impulsivity (Cardinal et al., 2001), impaired learning and performance on an task with delayed
8
gratification (Cardinal and Cheung, 2005) and is also involved in effort-based decision making (Hauber and Sommer, 2009; see Salamone et al., 2007 for a review). The NAc thus seems to be mainly involved in motivational (i.e. value) processes.
Regarding the STN, one study by Baunez et al. (2002) found that bilateral lesions of the STN in rats increased the reward-related motor responses, which was attributed to an increase in incentive motivation. The authors found that rats with STN lesions were faster in eating 100 food pellets than the control group, without a significant difference in general food intake. Furthermore, STN lesioned rats had a greater increase in locomotor activity and lever pressing in anticipation of reward compared to controls. Finally, the authors used a progressive ratio paradigm, in which the lever pressing ratio for reward is systematically increased until the rat ceases pressing (i.e. the 'breaking point'), which is assumed to be an indication of the amount of effort the rat is willing to pay to obtain the reward. With this task Baunez et al. found that STN lesioned rats had a higher breaking point than the control group. It is possible that the valuation of the reward has not changed, but that the threshold for responding to rewarding cues has decreased, which would be in line with findings by Frank et al. (2006, 2007) and Cavanagh et al. (2011) discussed in more detail in the following paragraph. A decrease in threshold due to STN lesions could result in increased behavioral impulsivity, since the action is premature, but should have no effect on cognitive impulsivity.
In many inter-temporal choice studies a hyperbolic discount function fits the data better than a linear or exponential function (Ainslie, 1975; Mazur, 1984). Since the subjective value of a reward does not only depend on delay, but also on reward magnitude, Ho et al. (1999) proposed the multiplicative hyperbolic model to distinguish between the effects of magnitude and delay on value and thereby on choice behavior. This model assumes a multiplicative combination of a hyperbolic function for the increase in reward-value with increase in magnitude and the hyperbolic discount function for delayed rewards.
Using this model, it has been found that lesions of the NAc core (Bezzina et al., 2007, 2008a) or disconnecting the NAc core from the orbital prefrontal cortex (OPFC) (Bezzina et al., 2008b) resulted in steeper discounting and thus more cognitive impulsivity in rats, which is in line with findings of Cardinal et al. (2001) and more recently with da Costa Araújo et al. (2009), without changing the sensitivity to reward magnitude. In contrast, lesions of the STN seem to increase the incentive values of the rewards, without a significant change in discount rate (Bezzina et al., 2009). This increased valuation of rewards is in line with findings of increased incentive motivation for food reward (Baunez et al., 2002, 2005), and cocaine ( Uslaner et al., 2005) after lesions of the STN. Since these studies are performed with rats and focus on reward-related behavior (increased speed of food intake, breaking point in progressive ratio schedules ) it could be argued that these findings are the result of a general decrease in threshold for reward-related action, which could be
independent of the actual valuation of the reward (i.e. increased behavioral impulsivity). Of course, this depends on the definition of value/motivation in terms of brain activity and the correlation of value/motivation and action processes that is assumed. A correlation of value/motivation with increased (speed of) action could be evident in healthy subjects. However, this correlation cannot simply be assumed in STN lesioned rats. Although the found effects are clearly not a general motor disinhibition, they could be a specific disinhibition of reward-related action, regardless of the subjective value of the reinforcer.
However, the results of the study by Uslaner et al. (2005) is in conflict with the findings of Baunez et al. (2005), who found decreased motivational action for cocaine after bilateral STN lesions. These contrasting results could be due to a dose specific effect, since the increased action found in Uslaner et al. was most evident with 0.3 mg/kg (roughly 75 µg/injection) while Baunez et al. used a dose of 250 µg/injection. As is also suggested by Uslaner et al., it could be that the higher dose used by Baunez et al. impaired the operant behavior of the lesioned rats.
In contrast to the findings by Bezzina et al. (2009), several studies have found a decrease in discounting of delayed rewards after STN lesions, i.e. a decrease of cognitive impulsivity (Winstanley et al., 2005; Uslaner and Robinson, 2006), though it is suggested by both Uslaner and Robinson (2006) and Bezzina et al. (2009) that this might be a transient effect of the STN lesions. In contrast, NAc lesion studies point to a increase in cognitive impulsivity that persist over months after the lesion (Bezzina et al., 2007, 2008b ; Pothuizen et al., 2005; da Costa Araújo et al., 2009).
9
The functionality of the STN and NAc in decision making seem opposing in the process of using value information and feedback for selecting the appropriate action. Together with the ventral OFC the ventral striatum plays a dominant role in controlling dopamine output (Eblen and Graybiel, 1995), which in turn reflects error magnitude in reward paradigms (Schultz et al., 1997) . Activity in the ventral striatum is also found to reflect the magnitude of (prediction) errors (O'Doherty et al., 2003). Chevrier and Schachter (2010) studied post-error adjustment using fMRI and found that errors leading to slower responses correlate with deactivation in the ventral midbrain, ventral striatum and caudal OFC. Whereas decreased activity the ventral striatum is involved in slower responding after errors, activity in the STN seems to play a role in actively suppressing responses, specifically in conflicting situations (Frank et al., 2006, 2007).
Overall the NAc seems to be mainly involved in the valuation step of the decision process, more specifically Pavlovian associations and the coding of expected reward, which results in appetitive behavior (core) and consummatory behavior (shell). Also the NAc is important for the learning and valuation of delayed rewards. The STN might have an effect on incentive value of reward, but more evidence points at an
involvement mainly in the process of using value information to guide action and regulating the threshold for reward-related action initiation. STN lesions could lead to a decrease in cognitive impulsivity, however whether this is a stable effect should become clear in further studies. The final evaluation step of the decision process again involves the NAc, which is involved in the generation of error signals that lead to updating of value and/or response-slowing.
3. DEEP BRAIN STIMULATION AND IMPULSIVITY
It has been found that impulse control disorders (ICDs), which are defined as behavioral disorders characterized by a failure to resist an impulse, drive or temptation to perform an act (i.e. behavioral impulsivity) that is harmful to that person or others, are more prevalent in patients receiving dopamine agonists (DA) compared to patients not receiving DA treatment (Voon et al., 2006; see Vilas et al., 2012 for a review) . The most common ICDs in PD patients are pathological gambling, hypersexuality, compulsive shopping and compulsive eating (Weintraub et al., 2010). Several studies report a decrease in ICDs after STN DBS (Ardouin et al., 2006; Bandini et al., 2007; Lim et al., 2009; Witjas et al., 2005), while others report an increase or development of ICDs after STN DBS (Hälbig et al., 2009; Lim et al., 2009; Moum et al., 2012). Many factors play a role in these differing results. Decrease in dopaminergic treatment seems to play a major role in the decrease in ICDs, while electrode position and patient history also affect ICD development. The involvement of the STN in setting the decision threshold could play an important role in the increase of ICDs after STN DBS.
The hypothesis that the STN is involved in increasing the decision threshold is supported by Frank et al. (2007) who found that STN-DBS in PD patients increased the speed of responding compared to the 'off' state in the same patients when decision-conflict was high. In this study the patients had to choose between two cues with a pre-learned probability of reinforcement. High-conflict in this context was defined as the choice
between two cues with similar reward probabilities (i.e. 70 and 80%). Dopaminergic treatment had no effect on this increased behavioral impulsivity with STN-DBS. Frank et al. also found that this increased speed of
responding in high-conflict situations depended on whether the cues indicated a 'win/win' or a 'lose/lose' situation. Responses were significantly faster in 'win/win' situations, indicating that not only the level of conflict influences the decision threshold. This increase in behavioral impulsivity also increased the amount of 'error trials' in which the less optimal option was chosen. It is therefore important in other decision context, when cognitive impulsivity is tested, to distinguish between the effects of these two types of impulsivity.
According to the computational model of Frank et al. (2006,2007) the STN is not required for valuation of rewards in intertemporal choice situations. If this is indeed the case, lesions or DBS of the STN should not lead to increased cognitive impulsivity.
Interestingly, while the connection of the mPFC and STN is thought to increase the decision threshold (Cavanagh et al., 2011; Frank et al., 2006,2007), other studies point at the role of the connectivity between the
10
cortex and striatum in reducing the decision threshold (Forstmann et al., 2008, 2010), though it is unknown to what extent the NAc plays a role in the findings of these fMRI studies. As mentioned in the previous paragraph, the NAc core is mostly found to be involved in both behavioral and cognitive impulsivity (see Basar et al., 2010 for a review). Although several clinical studies show increased (behavioral) impulsivity with NAc-DBS (Luigjes et al., 2011) , the effects of NAc-DBS in a purely experimental setting on cognitive impulsivity in the form of changes in delay or risk discounting has not been studied yet. However, research focused on DBS as treatment for addiction might shed some light on the role of the NAc as well as the STN in cognitive impulsivity.
Thus it seems that the STN is involved in reducing or suppressing behavioral impulsivity. Not many DBS studies have focused on impulsivity per se. The following paragraph will discuss the large body of literature on the effects of DBS in the NAc or STN in relation to addiction, where impulsivity also plays a significant role.
4. DEEP BRAIN STIMULATION AND ADDICTION
In line with what is known about the roles of the NAc and STN and the subsequent use of DBS in a clinical setting, studies investigating the effects of NAc-DBS focus more on substance abuse (i.e. alcohol and drug abuse), while studies of STN-DBS have focused more on non-substance addictions and in particular pathological gambling in PD patients.
4.1 Substance addictions
Interestingly, studies of the effects on STN-DBS on intake of the psychostimulant cocaine and
dopaminergic medication abuse seem to point at the beneficial effects of this therapy for substance addictions. Lhommée et al. (2012) found that four PD patients with dopamine dysregulation syndrome were free from their substance addiction after STN-DBS in a 1-year follow-up assessment. Furthermore, Rouaud et al. (2010) found that the motivation of rats for sucrose pellets increased after STN-DBS, but decreased for cocaine after STN-DBS, in line with the findings of Baunez et al. (2005) after STN lesions. In both studies doses of 250 µg/infusion were used, though the observed effects could not have been caused by impairment of operant behavior due to this high dose as STN-DBS on a fixed ratio 1 schedule of reinforcement resulted in
perseverative lever presses in the inactive period after injection.
Cocaine is a dopamine transporter antagonist, thus prolonging the effects of dopamine in areas of the reward circuitry like the NAc and prefrontal cortex. Groups of neurons in the NAc and STN have been found to encode natural reward (i.e. food or water) separately from cocaine (Carelli et al., 2000; Lardeux et al., 2009). It thus seems that STN-DBS influences the dopamine system in such way that it decreases the reinforcement properties of cocaine. If the above described effects of STN-DBS would be caused by a general decrease of dopaminergic output via the SN and VTA, the actual differences in motivation between natural reward and cocaine are caused by distinct processing of cocaine and natural reward involving the NAc, and not necessarily or only by a discriminative function of the STN. However, it is still unclear whether STN-DBS mimics a lesion or causes an increased excitatory output. Furthermore, a direct translation from rat studies to human patients is compromised by the long-term effects of drug use on the reward system of drug addicts, or the long term absence of a normally functioning dopamine system and medication in Parkinson's patients.
It is hypothesized that persistent compulsive drug use is not caused by the adaptations that occur in the brain after repeated drug use, leading to dependence and withdrawal, but are thought to be the result of (long-term) associative memory processes in the reward circuitry (see Hyman et al., 2006). A body of evidence points towards the NAc shell as involved in relapse of cocaine use after detoxification (Anderson et al., 2003, 2006; Schmidt et al., 2006) . A study by Vassoler et al. (2008) investigated the effects of bilateral DBS in the NAc shell on reinstatement of cocaine-seeking behavior in rats and found that stimulation of the shell, but not the dorsal striatum, significantly decreased the amount of lever pressing after a single infusion of 10 mg/kg or 20 mg/kg cocaine compared to a control group, while having no effect on reinstatement of food seeking. The authors suggest that inhibition of NAc shell output and/or the possibly antidromic inhibition of afferent structures, like the mPFC, result in the found attenuation of cocaine seeking behavior. Whether NAc shell
11
stimulation disrupts the subjective pleasantness of cocaine or impairs the cue -induced memory retrieval remains to be further investigated.
Not only the NAc shell is involved in addiction, as chronic unilateral stimulation of the NAc core seems to attenuate morphine reinforcement in rats (Liu et al., 2008). Furthermore, 150 µA stimulation of both the NAc shell and core decreases the ethanol consumption of rats that were trained pre-operatively using a reinforcement schedule to reach a stable ethanol 10% solution intake after 5-7 weeks (Knapp et al., 2009). This is in line with animal studies showing increased activity in both NAc core and shell during alcohol consumption (Porrino et al., 1998a,b; Robinson and Carelli, 2008). Henderson et al., (2010) studied the effects of NAc-DBS in the so-called 'alcohol-preferring' (P) rats; rats that spontaneously consume large amounts of alcohol without the need for reinforcement schedules to induce alcohol consumption. A first group of P rats reduced their relative alcohol intake with NAc-DBS compared to sham stimulation in the same rats. A second group of P rats showed a reduced alcohol intake after a forced abstinence of 4-6 weeks compared to a sham treatment group. Despite the less precise stimulation effects and localization of the stimulation electrodes in this study
(described as placed in the region of or adjacent to the NAc), the results are in line with findings of Knapp et al. (2009) and more generally with Vassoler et al. (2008) and Liu et al. (2008) in confirming the role of the NAc in encoding the salience of addictive substances and NAc-DBS as promising treatment of severe cases of addiction.
In addition, an interesting case report by Kuhn et al. (2007) revealed the possible therapeutic effects of NAc-DBS treatment for alcohol dependence. A 54-year old patient with anxiety disorder and depression was implanted with DBS electrodes in the NAc for these disorders, but stimulation only decreased anxiety slightly with no improvement of the depressive mood. However, the patient also showed a high alcohol dependence prior to surgery, which was drastically decreased after stimulation and persisted one year after surgery, despite the lack of improvement on the patient's anxiety or depression. The patient simply claimed to have lost the desire to drink. Müller et al. (2009) describes three cases of NAc-DBS in male patients suffering from severe alcoholism for whom alternative treatments have not been successful. A one-year follow up assessment revealed that two of the patients remained completely abstinent, while the third had four relapses with a total duration of 10 weeks. All three patients reported a disappearance of craving behavior directly after surgery. Not only cocaine and alcohol addicts, but also heroin addicts could benefit from NAc-DBS in a similar way, as supported by a case study by Zhou et al. (2011), in which the patient, next to no heroin use in the 6-year follow-up, also reduced the amount of sigarettes smoked daily from 40 to 10.
Importantly, the patients described by Müller et al. (2009) reported being able to experience pleasure about common things in life, suggesting that NAc-DBS has not influenced the overall pleasantness of rewarding stimuli, again confirming the discriminative ability of the reward network for different rewarding stimuli.
Figuring out exactly how NAc-DBS restores the dysfunctional brain processes in addicts is the next important step for understanding the altered valuation process caused by addictive drugs. In a more recent report Kuhn et al. (2011) describe the case of a 69-year-old man with severe alcohol dependence, who underwent NAc-DBS in combination with EEG assessment to monitor changes in an event-related potential called the error-related negativity (ERN). This ERN is thought to be generated in the posterodorsal mesial frontal cortex (pMFC), an area implicated in performance monitoring (Ridderinkhof et al., 2004; Ullsperger, 2006), and occurs after erroneous responses in decision reaction time tasks. Kuhn et al. (2011) found that the ERN increased in amplitude 4, 8 and 12 months after surgery compared to the baseline assessment right before surgery, and was significantly higher in the 4, 8 and 12 month measurement during on-stimulation compared to the end of a 24 hour off-stimulation period during each of these measurement times. This finding is in line with attenuated ERN amplitudes found after alcohol consumption (Ridderinkhof et al., 2002). Further studies should elucidate whether this finding by Kuhn et al. is a consequence of reduced alcohol intake or a direct effect of NAc-DBS. It could be that an DBS induced increase of cognitive control reduces alcohol intake when craving is high. However, this would be in contrast with the self-reports of several patients describing a decrease in craving (Heldmann et al., 2012; Kuhn et al., 2007; Müller et al., 2009).
One case study where a gambling task was combined with a PET scan 18 months after surgery revealed an increase tendency for the more risky option (with no difference in overall outcome between the
12
safe and risky option) in the off-stimulation compared to the on-stimulation condition, possibly pointing towards a decreased cognitive impulsivity when the NAc was stimulated (Heldmann et al., 2012). However, this result could not be replicated four months later. The PET scan during the first sessions showed an increased activity in areas involved in aspects of behavioral control and decision making when stimulation was on, like the paracingulate cortex in blocks with mostly wins and the precuneus in blocks with mostly losses (Heldmann et al., 2012). Since these results are from a single case, results must be interpreted with caution.
Since the STN is thought to modulate basal ganglia output (Turner et al., 2001) and is thereby involved in behavioral control, and regarding the evidence of the possible involvement of the STN in motivational processes, especially regarding the distinction between cocaine and natural reward (Baunez et al., 2002, 2005; Uslaner et al., 2005), the STN is likely to also play a role in the motivational processes involved in alcohol addiction. Indeed, using a progressive ratio schedule Lardeux and Baunez (2008) found an increased motivation to work for alcohol in 'High-Drinker' rats after bilateral STN lesions. In contrast, in 'Low-Drinker' rats the motivation for alcohol decreased. STN-DBS studies in this context have not yet been conducted.
In summary, both STN-DBS and NAc-DBS in the context of substance addiction seem to influence the valuation of drugs. Also, STN-DBS and NAc shell-DBS seem to have a differential effect on cocaine and natural reward value, making both targets possible candidates for DBS treatment of addiction. Case studies of NAc-DBS confirm the beneficial effects on addictive behavior by reducing craving for the drug, as well as the ability of the reward network to discriminate between addictive drugs and natural rewards, indicated by the
preservation of experiencing pleasure in common things in life reported by some patients (Müller et al., 2009). How exactly this distinction between addictive drugs and natural rewards is made in the brain needs further assessment. Additionally, some research suggest a possible role of increased cognitive control in the positive effects of NAc-DBS.
4.2 Non-substance addictions
The ICDs pathological gambling and compulsive shopping might also be an indication of increased cognitive impulsivity in PD patients in the sense that spending money for short term rewards is in conflict with saving the money for long term larger rewards. Whether dopamine medication influences cognitive impulsivity remains to be cleared out. Assuming a hyperbolic discounting curve, Milenkova et al. (2011) found a threefold increase in k values (indicator of steepness of discounting) in a group of PD patients without a (history of) ICD(s) compared to healthy controls. There was, however, no difference in k value found within the patient group on dopamine medication and after 12 hours without medication. Whether 12 hours is sufficient to find effects of medication on cognitive impulsivity is an open question.
The tendency of some Parkinson patients on dopamine medication to develop pathological gambling seems to depend, among other things, on genetic factors (Klein et al., 2007; Roussos et al., 2009), personality traits such as novelty-seeking (Bódi et al., 2009) and family history of gambling (Voon et al., 2009). Several PET studies have found an increased dopamine level in the ventral striatum in PD patients with pathological gambling disorder (Steeves et al., 2009) which is suggested to be caused by a lower presynaptic dopamine transporter density (Cilia et al., 2010).
A description by Ardouin et al. (2006) of seven patients with PG who underwent surgery for STN-DBS possibly confirms the role of dopaminergic treatment (i.e. dopamine agonist in combination with levodopa) in the development of PG; PG resolved in all 7 patients after several months of stimulation, correlating with a reduction of dopaminergic treatment. Interestingly, PG started after a mean of 4,6 years after the start of dopaminergic treatment, which has to be kept in mind in reports of PG development after STN-DBS. A transient increase in PG was observed in two patients. The authors suggest that STN-DBS mainly works on the motor system, in contrast with the dopaminergic medication, which acts on both the motor and limbic system, thereby inducing ICDs. Also Bandini et al. (2007) report the improvement in gambling behavior in two patients after STN-DBS combined with a reduction of dopaminergic treatment.
Though evidence points at dopamine agonists as the main cause of PG, several studies also suggest a (transient) direct influence of STN-DBS on gambling behavior. Smeding et al. (2007) report the development of
13
PG after onset of STN-DBS despite a reduction in DA medication, but PG resolved after changing to a different contact point for stimulation in combination with termination of DA medication. Also results of a study by Oyama et al. (2011) point at a possible stimulation effect on gambling behavior, which was studied with the Iowa Gambling Task (IGT) (Bechara et al., 1994); 100 cards need to be picked from four decks, with two decks resulting in overall profit and two decks resulting in overall loss of money. Both the treatment group on- and off-stimulation (tested 2-4 weeks after surgery) and the PD control group, showed no learning effect over the blocks of the task, which is in line with findings by Czernecki et al. (2002). There was only a significant increase of IGT score (i.e. the subtraction of the amount of bad cards picked from the amount of good cards picked per block of 20 trials) during the last block of the task in the off-stimulation condition compared to the on-stimulation condition and compared to the PD control group.
These effects could indicate cognitive impulsive behavior (choosing the decks with larger immediate rewards but losses on the long run) in PD patients on medication or with STN-DBS. In this case Oyama et al. hypothesize a role for the OFC, which is found to be involved in the performance on the IGT (Ernst et al., 2002; Thiel et al., 2003; Northoff et al., 2006) as well as cognitive impulsivity (Mobini et al., 2002; Rudebeck et al., 2006; Winstanley et al., 2004; Zeeb et al., 2010). Oyama et al. suggest that STN stimulation could affect the OFC through the hyperdirect pathway. However, it is unknown whether any of the groups showed an increase in behavioral impulsivity, i.e. an increased pace of picking cards, which could cause a lower score. The absence of learning during the sessions could point to an impaired ability to discriminate the 'good' from the 'bad' decks, which could be caused by impaired feedback-processing (Brand et al., 2004), possibly involving the OFC, a lack of motivation (Czernecki et al., 2002; Oyama et al., 2011), or an increased behavioral impulsivity.
Another specific behavior seen in gambling addicts without PD is described as loss-chasing behavior. A task to assess loss-chasing behavior (continuation of gambling to recover losses) was developed by Campbell-Meiklejohn et al. (2008). In this task participants start with an initial amount of money, and the overall goal is to reduce the amount of losses during the session. During each round, the participants see an initial loss-amount and have to choose between quitting and playing. When they quit the round is over and they lose the amount displayed. If they choose to play they have a chance to win the amount they would otherwise lose and thus end the round with no loss. If they do not win when playing, the loss-value is doubled, and the participants again get the opportunity to quit or play. The maximum amount participants could lose per round was fixed and was reached between 1 and 6 consecutive losses. A positive association between the outcomes of this task with the tendency to chase losses was found in more natural gambling situations (Campbell-Meiklejohn et al., 2008). A study by Rogers et al. (2011) used this task to assess the effects of STN-DBS in PD patients on loss-chasing behavior, in which a group of 22 patients were their own control group in (pseudo-randomized) on- and off-stimulation conditions. They found a significant increase in the value of losses chased only during the second time the task was done in patients in the on- stimulation condition.
Assuming a concave subjective value function for losses (Tversky and Kahneman, 1991), loss-chasing could be explained by a substantially larger negative subjective value for the accumulated losses compared to the subjective value of the possible loss associated with the outcome of a gamble, thereby motivating continuation of gambling behavior. Rogers et al. (2011) suggest that the increased loss-value chased while on STN-DBS could indicate a reduction in negative subjective value attached to the possible additional losses. Also the duration of stimulation could have an effect, as stimulation was stopped 30 minutes before the first session and was started again 30 minutes before the second session in the 'off/on' group where the effect was found (i.e. a possible acute stimulation effect), while it was not stopped and restarted 30 minutes before the first session in the 'on/off' group (i.e. a possible chronic stimulation effect).
All in all it seems that a lot of questions need to be answered before one can reach a conclusion about the effects of STN-DBS on pathological gambling as a non-substance addiction; the decrease of the
dopaminergic treatment of PD patients in combination of STN-DBS clouds the possible effects of the stimulation on cognitive impulsive behaviors leading to decreased gambling seen in patient studies. Some evidence indicates an increase of cognitive impulsivity by both dopaminergic medication or STN-DBS. The role of the OFC in this increased cognitive impulsivity due to STN-DBS requires further evidence. A decrease in
14
negative subjective value for (possible) losses was indicated, also suggesting a deficit in the valuation mechanism in these patients. These conclusions are still rather speculative and need further support.
5. DISCUSSION
In this literature thesis I focused on the effects of deep brain stimulation in two areas of the basal ganglia, the subthalamic nucleus and the nucleus accumbens, on processes related to decision making. Both the subthalamic nucleus and the nucleus accumbens contribute in a distinct way to reward related processes. The view of STN function has shifted from a purely motor-related to more cognitive and more specifically reward related function. While the NAc is related to valuation processes of different types of rewards, as well as the discounting of rewards when a risk or a delay is involved, the STN seems to use this subjective value information to guide actions and increase the decision threshold when necessary. After the outcome of a choice, the subjective value is updated, which is related to dopaminergic projections to the NAc. While decreased activity in the NAc is related to response slowing after errors, increased activity in the STN is related to an increased response threshold.
Stimulation of the STN or the NAc seem to confirm the separate processing of different types of reward, in which the reward circuitry seems to make a distinction between drugs of abuse and natural rewards. Since both STN-DBS and NAc-DBS have a different effect on drugs of abuse and natural reward, none of these areas by themselves seems to be essential for making this distinction. However, the NAc seems to have a specific role in the valuation of addictive substances, since the therapeutic effect of NAc-DBS seems to be mainly the attenuation of craving, as reported by the patients themselves, while the valuation of natural rewarding events seem to return or increase with NAc-DBS. In contrast, animal lesion studies suggest a possibly opposite effect of STN-DBS, but only when the subject has a history of substance abuse. Possibly the increased incentive value of the substance and a low decision threshold would account for this opposite effect.
Studies of STN-DBS in the context of pathological gambling suggest either an increase in cognitive impulsivity, possibly due to altered activity in the OFC, which is connected to the STN through the hyperdirect pathway, or an increased behavioral impulsivity, leading to suboptimal choice due to a shortened deliberation. The latter would be in line with the suggested role of the STN in increasing the decision threshold. These results are, however, clouded by the influence of dopaminergic treatment on behavioral impulsivity, possibly also causing what is interpreted as cognitive impulsivity. Since both cocaine use and pathological gambling are associated with an increase in dopamine in the ventral striatum, NAc-DBS might also reduce the urge for excessive gambling behavior, though up until now only pathological gambling in PD patients treated with STN-DBS has been studied. The use of a possible rat model for studying pathological gambling behavior (described by Adriani et al. (2010) might elucidate whether NAc-DBS has similar effect on gambling behavior as it has on substance abuse.
Several important issues need to be kept in mind when drawing conclusions of what DBS tells us about the functions of the NAc and STN in the reward circuitry. First of all, since deep brain stimulation in humans occurs in a clinical setting, or otherwise in non-human animals, generalizations of findings from patient studies towards the general population need to be done with care and require additional evidence from studies with non-human animals or healthy controls, keeping in mind the dysfunctional circuitry of the patient population. In addition, behavioral measures with nonhuman animals need to be interpreted objectively, avoiding the use of suggestive terms to describe the cognitive processes that might underlie the observed behavior. Secondly, the effects of DBS might differ between studies, since not only the stimulation parameters, but also the type of electrodes used, the exact location and the angle of the electrode within the brain might produce a different pattern of (in)activation in the target areas. Differences in these dimensions are often not mentioned when studies are compared, but could be the cause of different results between studies. Thirdly, most studies differ in the time period of assessment, either in relation to surgery or on/off stimulation periods, which could either
15
lead to false confirmation of results or finding opposing effects, while this could be attributable to transient versus chronic stimulation effects.
Future studies should focus on the effects of STN-DBS or NAc-DBS on cognitive impulsivity, using either gambling or intertemporal choice paradigms. Also whether chronic use of dopaminergic medication influences cognitive impulsivity requires further research. In these studies it needs to be kept in mind that changes in cognitive impulsive choice can also be the result of an increased behavioral impulsivity. Furthermore, how exactly the reward circuitry makes a distinction between different types of rewards in general and addictive substances and natural rewards in particular requires detailed study of all the reward related areas where distinction between rewards are made, including the OFC and ACC, and the effects of DBS on these areas. In addition future studies should investigate whether memory retrieval of the subjective value or a direct alteration of subjective value alters craving in addicts after NAc-DBS.
6. REFERENCES
Abler, B., H. Walter, et al. (2006). "Prediction error as a linear function of reward probability is coded in human nucleus accumbens." Neuroimage 31(2): 790-795.
Adriani, W., F. Boyer, et al. (2010). "Social withdrawal and gambling-like profile after lentiviral manipulation of DAT expression in the rat accumbens." Int J Neuropsychopharmacol 13(10): 1329-1342.
Ainslie, G. (1975). "Specious reward: a behavioral theory of impulsiveness and impulse control." Psychol Bull 82(4): 463-496.
Anderson, S. M., A. A. Bari, et al. (2003). "Administration of the D1-like dopamine receptor antagonist SCH-23390 into the medial nucleus accumbens shell attenuates cocaine priming-induced reinstatement of drug-seeking behavior in rats." Psychopharmacology (Berl) 168(1-2): 132-138.
Anderson, S. M., H. D. Schmidt, et al. (2006). "Administration of the D2 dopamine receptor antagonist sulpiride into the shell, but not the core, of the nucleus accumbens attenuates cocaine priming-induced reinstatement of drug seeking." Neuropsychopharmacology 31(7): 1452-1461.
Aouizerate, B., C. Martin-Guehl, et al. (2005). "Deep brain stimulation for OCD and major depression." Am J Psychiatry 162(11): 2192.
Ardouin, C., V. Voon, et al. (2006). "Pathological gambling in Parkinson's disease improves on chronic subthalamic nucleus stimulation." Mov Disord 21(11): 1941-1946.
Balleine, B. W., M. Liljeholm, et al. (2009). "The integrative function of the basal ganglia in instrumental conditioning." Behav Brain Res 199(1): 43-52.
Bandini, F., A. Primavera, et al. (2007). "Using STN DBS and medication reduction as a strategy to treat pathological gambling in Parkinson's disease." Parkinsonism Relat Disord 13(6): 369-371. Basar, K., T. Sesia, et al. (2010). "Nucleus accumbens and impulsivity." Prog Neurobiol 92(4): 533-557. Baunez, C., M. Amalric, et al. (2002). "Enhanced food-related motivation after bilateral lesions of the
subthalamic nucleus." J Neurosci 22(2): 562-568.
Baunez, C., C. Dias, et al. (2005). "The subthalamic nucleus exerts opposite control on cocaine and 'natural' rewards." Nat Neurosci 8(4): 484-489.
Bechara, A., A. R. Damasio, et al. (1994). "Insensitivity to future consequences following damage to human prefrontal cortex." Cognition 50(1-3): 7-15.
Benabid, A. L., P. Pollak, et al. (1989). "[Treatment of Parkinson tremor by chronic stimulation of the ventral intermediate nucleus of the thalamus]." Rev Neurol (Paris) 145(4): 320-323.
Benazzouz, A., D. M. Gao, et al. (2000). "Effect of high-frequency stimulation of the subthalamic nucleus on the neuronal activities of the substantia nigra pars reticulata and ventrolateral nucleus of the thalamus in the rat." Neuroscience 99(2): 289-295.
Bewernick, B. H., R. Hurlemann, et al. (2010). "Nucleus accumbens deep brain stimulation decreases ratings of depression and anxiety in treatment-resistant depression." Biol Psychiatry 67(2): 110-116.
Bewernick, B. H., S. Kayser, et al. (2012). "Long-term effects of nucleus accumbens deep brain stimulation in treatment-resistant depression: evidence for sustained efficacy." Neuropsychopharmacology 37(9): 1975-1985.
16
Bezzina, G., S. Body, et al. (2008). "Effect of disconnecting the orbital prefrontal cortex from the nucleus accumbens core on inter-temporal choice behaviour: a quantitative analysis." Behav Brain Res 191(2): 272-279.
Bezzina, G., S. Body, et al. (2008). "Effect of quinolinic acid-induced lesions of the nucleus accumbens core on performance on a progressive ratio schedule of reinforcement: implications for inter-temporal choice." Psychopharmacology (Berl) 197(2): 339-350.
Bezzina, G., T. H. Cheung, et al. (2007). "Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis." Psychopharmacology (Berl) 195(1): 71-84. Bezzina, G., T. H. Cheung, et al. (2009). "Quantitative analysis of the effect of lesions of the subthalamic nucleus
on intertemporal choice: further evidence for enhancement of the incentive value of food reinforcers." Behav Pharmacol 20(5-6): 437-446.
Bodi, N., S. Keri, et al. (2009). "Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients." Brain 132(Pt 9): 2385-2395.
Campbell-Meiklejohn, D. K., M. W. Woolrich, et al. (2008). "Knowing when to stop: the brain mechanisms of chasing losses." Biol Psychiatry 63(3): 293-300.
Cardinal, R. N., D. R. Pennicott, et al. (2001). "Impulsive choice induced in rats by lesions of the nucleus accumbens core." Science 292(5526): 2499-2501.
Carelli, R. M., S. G. Ijames, et al. (2000). "Evidence that separate neural circuits in the nucleus accumbens encode cocaine versus "natural" (water and food) reward." J Neurosci 20(11): 4255-4266.
Cavanagh, J. F., T. V. Wiecki, et al. (2011). "Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold." Nat Neurosci 14(11): 1462-1467.
Chevrier, A. and R. J. Schachar (2010). "Error detection in the stop signal task." Neuroimage 53(2): 664-673. Cilia, R., J. H. Ko, et al. (2010). "Reduced dopamine transporter density in the ventral striatum of patients with
Parkinson's disease and pathological gambling." Neurobiol Dis 39(1): 98-104.
Corbit, L. H., J. L. Muir, et al. (2001). "The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell." J Neurosci 21(9): 3251-3260.
Critchley, H. D. and E. T. Rolls (1996). "Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex." J Neurophysiol 75(4): 1673-1686.
Czernecki, V., B. Pillon, et al. (2002). "Motivation, reward, and Parkinson's disease: influence of dopatherapy." Neuropsychologia 40(13): 2257-2267.
da Costa Araujo, S., S. Body, et al. (2009). "Effects of lesions of the nucleus accumbens core on inter-temporal choice: further observations with an adjusting-delay procedure." Behav Brain Res 202(2): 272-277. Dalley, J. W., B. J. Everitt, et al. (2011). "Impulsivity, compulsivity, and top-down cognitive control." Neuron
69(4): 680-694.
DeLong, M. R. (2000). The Basal Ganglia Principles of neural science. E. R. Kandel, J. H. Schwartz and T. M. Jessel, McGraw-Hill Companies, Inc. .
Dorris, M. C. and P. W. Glimcher (2004). "Activity in posterior parietal cortex is correlated with the relative subjective desirability of action." Neuron 44(2): 365-378.
Dreher, J. C., P. Kohn, et al. (2006). "Neural coding of distinct statistical properties of reward information in humans." Cereb Cortex 16(4): 561-573.
Eblen, F. and A. M. Graybiel (1995). "Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey." J Neurosci 15(9): 5999-6013.
Ernst, M., K. Bolla, et al. (2002). "Decision-making in a risk-taking task: a PET study." Neuropsychopharmacology 26(5): 682-691.
Forstmann, B. U., A. Anwander, et al. (2010). "Cortico-striatal connections predict control over speed and accuracy in perceptual decision making." Proc Natl Acad Sci U S A 107(36): 15916-15920. Forstmann, B. U., G. Dutilh, et al. (2008). "Striatum and pre-SMA facilitate decision-making under time
pressure." Proc Natl Acad Sci U S A 105(45): 17538-17542.
Frank, M. J. (2006). "Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making." Neural Netw 19(8): 1120-1136.
Frank, M. J., J. Samanta, et al. (2007). "Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism." Science 318(5854): 1309-1312.
Franzini, A., G. Messina, et al. (2010). "Deep-brain stimulation of the nucleus accumbens in obsessive compulsive disorder: clinical, surgical and electrophysiological considerations in two consecutive patients." Neurol Sci 31(3): 353-359.
17
Fudge, J. L., K. Kunishio, et al. (2002). "Amygdaloid projections to ventromedial striatal subterritories in the primate." Neuroscience 110(2): 257-275.
Grabenhorst, F. and E. T. Rolls (2011). "Value, pleasure and choice in the ventral prefrontal cortex." Trends in Cognitive Sciences 15(2): 56-67.
Grabenhorst, F., E. T. Rolls, et al. (2007). "How pleasant and unpleasant stimuli combine in different brain regions: Odor mixtures." Journal of Neuroscience 27(49): 13532-13540.
Grubert, C., R. Hurlemann, et al. (2011). "Neuropsychological safety of nucleus accumbens deep brain
stimulation for major depression: effects of 12-month stimulation." World J Biol Psychiatry 12(7): 516-527.
Gubellini, P., P. Salin, et al. (2009). "Deep brain stimulation in neurological diseases and experimental models: from molecule to complex behavior." Prog Neurobiol 89(1): 79-123.
Haber, S. N. (2003). "The primate basal ganglia: parallel and integrative networks." J Chem Neuroanat 26(4): 317-330.
Haber, S. N., K. S. Kim, et al. (2006). "Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning." J Neurosci 26(32): 8368-8376.
Halbig, T. D., W. Tse, et al. (2009). "Subthalamic deep brain stimulation and impulse control in Parkinson's disease." Eur J Neurol 16(4): 493-497.
Hammond, C., H. Bergman, et al. (2007). "Pathological synchronization in Parkinson's disease: networks, models and treatments." Trends Neurosci 30(7): 357-364.
Hauber, W. and S. Sommer (2009). "Prefrontostriatal circuitry regulates effort-related decision making." Cereb Cortex 19(10): 2240-2247.
Heldmann, M., G. Berding, et al. (2012). "Deep brain stimulation of nucleus accumbens region in alcoholism affects reward processing." PLoS One 7(5): e36572.
Henderson, M. B., A. I. Green, et al. (2010). "Deep brain stimulation of the nucleus accumbens reduces alcohol intake in alcohol-preferring rats." Neurosurg Focus 29(2): E12.
Ho, M. Y., S. Mobini, et al. (1999). "Theory and method in the quantitative analysis of "impulsive choice" behaviour: implications for psychopharmacology." Psychopharmacology (Berl) 146(4): 362-372. Hyman, S. E., R. C. Malenka, et al. (2006). "Neural mechanisms of addiction: the role of reward-related learning
and memory." Annu Rev Neurosci 29: 565-598.
Kawagoe, R., Y. Takikawa, et al. (1998). "Expectation of reward modulates cognitive signals in the basal ganglia." Nat Neurosci 1(5): 411-416.
Klein, T. A., J. Neumann, et al. (2007). "Genetically determined differences in learning from errors." Science 318(5856): 1642-1645.
Knapp, C. M., L. Tozier, et al. (2009). "Deep brain stimulation of the nucleus accumbens reduces ethanol consumption in rats." Pharmacol Biochem Behav 92(3): 474-479.
Knutson, B., C. M. Adams, et al. (2001). "Anticipation of increasing monetary reward selectively recruits nucleus accumbens." J Neurosci 21(16): RC159.
Knutson, B., M. R. Delgado, et al. (2009). Representation of Subjective Value in the Striatum. Neuroeconomics Decision making and the brain. P. W. Glimcher, C. F. Camerer, E. Fehr and R. A. Poldrack, Elsevier Inc.: 18.
Knutson, B., J. Taylor, et al. (2005). "Distributed neural representation of expected value." J Neurosci 25(19): 4806-4812.
Kringelbach, M. L., J. O'Doherty, et al. (2003). "Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness." Cerebral Cortex 13(10): 1064-1071. Kuhn, J., T. O. Grundler, et al. (2011). "Successful deep brain stimulation of the nucleus accumbens in severe
alcohol dependence is associated with changed performance monitoring." Addict Biol 16(4): 620-623. Kuhn, J., D. Lenartz, et al. (2007). "Remission of alcohol dependency following deep brain stimulation of the
nucleus accumbens: valuable therapeutic implications?" J Neurol Neurosurg Psychiatry 78(10): 1152-1153.
Lardeux, S. and C. Baunez (2008). "Alcohol preference influences the subthalamic nucleus control on motivation for alcohol in rats." Neuropsychopharmacology 33(3): 634-642.
Lardeux, S., R. Pernaud, et al. (2009). "Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus." J Neurophysiol 102(4): 2526-2537.
Lhommee, E., H. Klinger, et al. (2012). "Subthalamic stimulation in Parkinson's disease: restoring the balance of motivated behaviours." Brain 135(Pt 5): 1463-1477.
