Illuminating the role of glutamatergic pyramidal neurons in the prelimbic
cortex
on impulsive decision-making
B.J.G. van den Boom1, M.R. Carr1, Y. van Mourik1, D. Schetters1, M. van der Roest1, T.J. De Vries1,2, T. Pattij1 1
Department of Anatomy and Neurosciences, Neuroscience Campus Amsterdam, Vrije University Medical Center, Amsterdam, The Netherlands
2
Department of Molecular and Cellular Neurobiology, Center for Neurogenomics and Cognitive Research (CNCR), Neuroscience Campus Amsterdam, VU University, Amsterdam, The Netherlands
Summary
Every day we are influenced by our impulsive decisions, either it is by purchasing discount products or taking another cup of coffee. The inability to postpone reward for a greater payoff later is part of the behavioural characteristic impulsivity and has been implicated in several disorders including ADHD, schizophrenia and sub-stance abuse. Recent pharmacological and lesion studies stressed the involvement of the prelimbic cortex (PL) in delay-discounting. However, it remains unclear how timing and patterns of glutamatergic activity here is im-portant in impulsive decision-making. In our current study we address this issue by optically silencing glutama-tergic pyramidal neurons in the PL of Wistar rats using Archaerhodopsin3.0, while they perform on the Delayed Reward Task. We measure their preference for the delayed reward which is typically lower for more impulsive individuals as they exhibit a steeper discounting curve where large reward preference reduces as a function of delay. We show a trend in PL activity involved in online impulsive choice decision-making. Although we find dis-traction with our current laser light setup, we show with laser-ON trial analysis that PL activity effect carryover to other trials with no laser stimulation. Our preliminary data suggests that time-specific inhibition of PLC neurons increases delay-discounting. Future directions will focus specifically different epochs, and possible correlate be-haviour to neuronal correlates.
Introduction
Executive functions including decision-making and cognitive control are important aspects of our daily life and necessary for behaving or choosing appropriately in the situation (Chudasama, 2011). It appears when these processes go wrong we observe impulsive behaviours, i.e.; premature and ill-conceived behaviours. While fast, heuristic decisions may be useful in novel, highly dynam-ic or uncertain environments (Evenden, 1999), in today’s
society impulsivity is a behavioural characteristic with potential negative consequences for the future and it has been implicated in a number of behavioural and psychiatric disorders, including ADHD, schizophrenia and substance abuse (Moschak & Mitchell, 2014; Econo-midou et al., 2009; Diergaarde et al., 2008). Due to its high prevalence in patients with these disorders, standing the neuronal circuitry and mechanisms under-lying the impulsivity trait will hopefully contribute to a
novel intervention strategy for these disorders (Pattij & Vanderschuren, 2008; Dalley et al., 2011).
Impulsivity is a multifactorial phenomenon, largely determined, for example, by the inability to inhibit or withhold an inappropriate motor response (impulsive action), or the inability to postpone reward for a larger payoff (impulsive choice), that is, using delay-aversive strategies (Winstanley et al., 2006). Although impulsive action and impulsive choice might be interrelated or overlapping in a behavioral phenotype they have distinct neuroanatomical, as well as neuropharmacological mechanisms (Broos et al., 2012; Diergaarde et al., 2008). Delay-aversive impulsive choice can be assessed using a behavioural measure of delay-discounting. Delay-discounting paradigms are used in both human and ro-dents and measure the increased preference for smaller, immediate reward over larger, more beneficial but post-poned rewards (Broos et al., 2012). It is thought individ-uals preferring the smaller, immediate reward are less tolerant to delay and therefore classified as more impul-sive (Amitai & Markou, 2011).
The exact cognitive processes underlying this mala-daptation remain unclear, but could be driven by a number of decision aspects, including; the chosen cogni-tive strategy for the task at hand, the evaluation of op-tions by absolute or relative reward values, or even the estimation of the cost, which in this case, is the delay period required to wait until the larger payoff is award-ed (Paine et al., 2013; Gill et al., 2010; Euston et al., 2012).
Several brain areas are thought to play an important role in impulsive behaviour in rodents, namely prefron-tal cortical, striaprefron-tal and limbic brain regions (Cardinal,
2006). However, especially the medial prefrontal cortex (mPFC) seems to be heavily implicated in both delay-discounting and the executive function disorders men-tioned above (Moschak & Mitchell, 2014; Kim & Lee, 2011). Lesions studies using ibotenic acid and pharmaco-logic interventions using muscimol show increased im-pulsive behaviour during different impulsivity tasks, sug-gesting that the mPFC activity is involved in multiple aspect of impulsivity (Paine et al., 2013; Gill et al., 2010; de Visser et al., 2011).
The neuroanatomical circuitry of the mPFC has been well mapped in rodents and consists of, among others, two important sub regions for impulsivity, the Prelimbic (PL) and Infralimbic cortices (IL) (Kalivas et al., 2005; Ongür & Price, 2000; Krettek & Price, 1977). Recent studies show an overlap in involvement of both areas in different aspect of impulsivity, whereby the IL seems to be more involved in impulsive action and the PL in im-pulsive choice (Murphy et al., 2012; Kalivas et al., 2005; Ongür & Price, 2000; Krettek & Price, 1977). The PL has been specifically implicated in delay-discounting using lesion and pharmacological interventions to suppress neuronal activity, resulting in an increase in impulsive choice (Sonntag et al., 2014; Moschak & Mitchell, 2014; Pattij & Vanderschuren, 2008; Pothuizen et al., 2005). The projections to and from the PL have been well mapped (figure 1), with a large reciprocal connectivity with striatal and mid-brain regions, including; the Nucle-us Accumbens (NAc), Ventral Tegmental Area (VTA) and Basolateral Amygdala (BLA) (Bedwell et al., 2014; Sierra-Mercado et al., 2011). These projections seem to facili-tate fundamental roles in evaluating reward-outcomes, behavioural control and subjective evaluation of out-comes (Sul et al., 2010; Euston et al., 2012). Especially
Figure 1 | Schematic representation of the PL connections with optic fiber implantation. Dark square is optic fiber and ferrule secure placement to the skull, aligned right above the Prelimbic cortex (PL). The PL has efferent projections in green to the Nu-cleus Accumbens (NAc), Hippocampus (Hippo), Basolateral Amygdala (BLA), Lateral Hypothalamus (LH), Dorsal Midline Thala-mus (DMT) and Ventral Tegmental Area (VTA). It also receives afferent projections in red from the NAc, Hippo and VTA.
the projection from the PL to the NAc core has been well studied, with projections onto the Medium Spiny Neu-rons of the NAc core, influencing reward value and re-ward expectation, which are important aspects related to, reward-based choice (Russo & Nestler, 2013; Gill et
al., 2010). In addition to sub-cortical glutamatergic
pro-jections, there are also reciprocal projections to its other layers (ipsi- and contralaterally) as well as to adjacent regions, such as the IL and ACC (Euston et al., 2012; Ji & Neugebauer, 2012). Therefore, this highly connected PL area maintains an influential position to integrate and relay contextual information both to downstream tar-gets, as well as influencing other regulatory or executive regions’ during decision-making (Little & Carter, 2012).
Besides allocating specific functions to brain areas, previous studies also suggest that particular neuro-transmitter systems might be involved in impulsive choice behaviour. For example, the dopaminergic sys-tem seems to be important for impulsive choice behav-iour. Injecting animals in the dorsal striatal with the spe-cific dopaminergic toxin, 6-OHDA, shows an increase in delay-discounting (Tedford et al., 2015). Other studies show that this effect might be due to diminished reward value estimation and reward expectation (Saddoris et
al., 2014). More importantly, the PL consists mainly of
glutamatergic excitatory pyramidal neurons and GA-BAergic inhibitory interneurons (Russo & Nestler, 2013), and this influence of the glutamatergic system on impul-sivity remains unclear. Pharmacological glutamatergic inactivation studies using competitive and uncompeti-tive glutamate NMDA receptor antagonists have been performed on animals affecting the mPFC or PL (Pozzi et
al., 2011; Murphy et al., 2010; Sukhotina et al., 2008).
These studies show that glutamate in the mPFC is in-volved in delay-discounting and strengthen the idea that glutamatergic pyramidal output (in the PL is essential for delay-discounting and impulsive choice (Churchwell et
al., 2009; Cottone et al., 2013; Floresco et al., 2008).
While a growing body of evidence suggests that gluta-mate is important in delay-discounting, it remains un-clear if the timing of this activity, prior to choice execu-tion, or at other stages during the decision-making pro-cess is important, or as some groups have demonstrated that the amount of glutamate release on its downstream targets is more important in certain maladaptive behav-iours, than the source of the glutamate. (Britt et al., 2012; Tye, 2012).
While previous studies using lesion and pharmaco-logical interventions have brought much insight into impulsive choice decision-making, we aim to unravel the role of bulk activation of glutamatergic pyramidal neu-rons in the PL on delay-discounting using the delayed reward task (DRT). Where previous studies lack temporal precision to pair inactivation of specific cell-types with initiation or choice cues of the task, in this present study, Archearhodopsin was used to optically inhibit
glutamatergic pyramidal output from the PL, with the aim of revealing at which time points the region’s influ-ence on impulsive behaviour has its greatest influinflu-ence. With novel approaches, come novel challenges; striving to obtain a with-in subject design for online optogenetic manipulation in a complex cognitive task is challenging. For this study we will also present data where the aim has been to test a number of practical challenges, such as light sham conditions and the influence of cable teth-ering, to ensure that the behavioral effects observed in our experimental groups, are due to the optical inhibi-tion, rather than extraneous factors biasing animal be-haviour.
Material & Methods
Subjects
Male Wistar rats (n=32) were purchased from Harlan CPB (Horst, The Netherlands), weighed approximately 200-250 g and were 4-5 weeks old upon arrival. Animals were housed per 2 in standard Macrolon cages on a reversed 12-h day/night cycle (lights on from 7 PM to 7 AM) in a temperature (21 ± 2°C)- and humidity (50 ± 10%)-controlled chamber with water available ad libitum during the entire experiment. Following optic fiber im-plantation, rats were individually housed. During the entire experiment, rats were food restricted, and main-tained at about 85-90% of their free-feeding weight, by providing them with 12-14 g of chow at the end of every day. Behavioural testing was conducted during the dark phase of the day/night cycle. All experiments and proce-dures were approved by the Animal Care Committee of the Free University of Amsterdam (The Netherlands).
Viral Infusion Surgery
CaMKIIa promoter-driven opsin plasmids pAAV-CaMKIIa-ChR2-eYFP, pAAV-CaMKIIa-Arch3.0-eYFP and a control vector that lacked the opsin sequence pAAV-CaMKIIa-eYFP (Tye et al., 2011) from the Karl Deisseroth labora-tory were packaged as AAV serotype 5 virus and ordered at the University of California. Rats were anaesthetized with isoflurane (4% induction, 2% maintenance), re-ceived subcutaneously 5 mg/kg ketofen, 8.33 mg/kg baytril, 10 mg/kg xylocaine and were mounted in a ste-reotactic frame. A custom- made 31 gauge infuser was used to deliver 0.5 µl virus suspension was bilaterally at a rate of 0.1 µl/min into the stereotactic coordinates of the prelimbic cortex (PL) [+2.76 mm AP; +/1.4 mm ML; -3.76 mm DV (10° angle); relative to Bregma] using a syringe pump (SP100i2; World Precision Instruments, Sarasota, FL, USA). Infusion was delivered over 5 min, followed by an additional 15 min of the needle left in place to allow diffusion of the virus. Animals recovered for one week before being transferred to the training facility and adjusting to a 12-hour reversed day-night cycle.
Behavioural Training
Following recovery, rats were handled and habituated in their operant chamber (32 cm x 24 cm x 33 cm; Med Associates, Inc., St. Albans, VT). Each chamber was fitted with grid floors, in a sound-attenuating, ventilated cubi-cle (64 cm x 42 cm x 60 cm).On one wall there is an elongated food magazine to release dustless reward pellets (45 mg, Noyes Precision Pellets, New Brunswick, NJ), above head height a 28v house light and blue or green LED light (to mask light scatter later during stimu-lation protocols), respectively for opsin Arch3.0 or ChR2, for ambient illumination throughout the task. On the opposing wall there are 5 nose poke apertures with stimulus lights and infrared beam detectors. All in-put/output was controlled by a computer using MED-PC (Med Associates Inc.).Training sessions were scheduled from Monday to Friday. Taken together, the length of the shaping and baseline stabilizing phase lasted approx-imately 12-16 weeks. Experimental stimulation protocols spanned 6 weeks with washout sessions in between test days.
Delayed Reward Task
The delayed reward task consists of both forced and choice trials. Shaping and training occurred gradually until the rats reached a maximum delay period of
40seconds, this is described in greater detail in (Van Gaalen et al., 2006). After reaching stable baseline for the delayed reward 40 (DR40) paradigm, animals were tested using the DR20 to increase amount of free choice trials for analysis later on. Difference between the DR40 and DR 20 is that the maximum delay of receiving large reward is 20 instead of 40 seconds.
Briefly, each session consisted of 60 trials, divided in 4 blocks of 15 trials with 4 different delays. Each session started with a block of zero (0) sec delay for the large reward and after each block, the delay period was pro-grammed to increase to 5 sec, 10 sec and eventually 20 sec delay for that block. Every block started with 2 forced trials in which, after initiating the trial by a nose poke in the third nose poke hole, either the left or the right nose poke hole was illuminated in a counterbal-anced fashion. The delayed period for the large reward during the forced trial was fixed for the rest of the block, signaling the proceeding delay for that block. In the sub-sequent 13 trials, the rat was free to choose between the small and large reward (figure 2). No further stimuli were presented during the delay period. The inter-trial interval was programmed such that the duration of each trial lasted 70 sec regardless of delay or choice. During testing, failures of respond within the initiation period were registered as error of omission, however house
Figure 2 | Schematic view of the delayed reward task. Blue squares in front of the animal represent the 5 stimulus apertures, blue square below the animal represents food magazine. Yellow blue square represents illuminated nose poke hole, which an animal is supposed to respond to. Each sessions consists of a Forced Choice for the immediate small reward (1 pellet) and a Forced Choice for the large delayed reward (4 pellets, delay 0, 5, 10 or 20 seconds), following 13 Free Choice trials where the animals can choose between small or large reward. Light inhibition during choice execution, started when middle nose poke illuminates and extinguished when choice has been executed, or after 3 seconds, whichever occurred first.
Figure 3 | Schematic timeline of the behaviour experiments performed for this study. The experiment started with DR40 training on the delayed reward task with only AAV-CaMKIIa viral construct injection into the PL. Next, bilaterally fibers were chronically secured to the skull of the animal. For analysis reasons, we next switched to the DR20 to increase trial number on lower delays. After DR20 baseline, we connected animals to cable to habituate to less movement freedom and distracting cables in the operant chamber. Sham sessions consist of blocking light transmission to the brain with plasticine, followed by actual stimulation session where light did reach the brain. In between stimulation sessions, washout sessions (with blocked light transmission) were conducted.
light remained on but aperture 2 and 4 did not illumi-nate. Nose poking into non-illuminated holes during testing were registered but had no consequence. Per-centage preferred choice for the large reward was calcu-lated for each block of 13 trials [number of large re-wards chosen x 100 / (number of large rere-wards chosen + number of small rewards chosen)]. Impulsive choice was defined as a decreased percentage preferred choice for the large reward and was set out as a delay-discounting curve. On individual animal level, impulsive choice (de-lay-discounting curve) was stable over at least three sessions before starting with the stimulation protocol.
Chronic implantation of optic fiber
After reaching stable baseline performance over three sessions, animals underwent implantation of optic fibers above the prelimbic cortex (PL). Animals were anaesthe-tized using isoflurane (4% induction, 2% maintenance) during surgery and were administered subcutaneously 1% ketofen, 2.5% baytril, 2% xylocaine and were mount-ed on a stereotactic frame. Optic fibers usmount-ed for implan-tation had light transmission of at least 70%. Ferrules
used in this experiment were 3.5 mm in diameter (Preci-sion Fiber Products, Milpitas, CA, USA). Next, optic fibers (200 µm core) were bilateral chronically implanted di-rectly above the stereotactic coordinates of the PL [-3.56 mm DV (10° angle); relative to Bregma]. Optic fibers were fixed to the skull using 6 small jewellers screws into the skull and coated in light hardening dental ce-ment and Simplex Rapid dental cece-ment (Kemdent, Wilt-shire, UK) covering the half the length of the ferrules. A thin layer of dental cement mixed with carbon powder (99.9% pure) was thinly spread over the existing dental cement head cap to prevent light scatter during stimula-tion.
Stimulation Protocol
For inhibition, TTL pulse-generated green laser pulses (3 sec; intensity approximately 7 mW) were bilaterally de-livered to the PL through optical fibers coupled to a 532-nm laser. Behavioural protocol is shown in figure 3. Briefly, animals started with 3 sessions of habituation to cables during the delayed reward task. Next, 3 sessions sham stimulation followed whereby light transmission was blocked at the ferrule by plasticine (figure 4). 8 ses-sions of experimental stimulation are washout session, using plasticine to block laser light. During light stimula-tion sessions, animals received actual light stimulastimula-tion in 50% of the trials. Stimulation occurred during motor execution of the decision.
Immunohistochemistry
At the end of the experiment, animals were used for other bahavioural tests using the same paradigm. How-ever, as this is the second pilot study, we have stainings from previous animals which received viral injected and optic stimulated at the
same location in the PL. For immunohistochemis-try, brains were sliced and rinsed (3 x 10 min) in Tris-buffered saline (TBS) con-taining 0.05 M Tris, 0.15 M NaCL (pH 7.8). Slices were incubated for 1 h in a blocking solution contain-ing TBS with 5% normal goat serum (NGS), 2.5% BAS, and 0.20% Trition X-100. After incubation, sec-tions were washed in TBS and incubated overnight with Rabbit-anti-GFP pri-mary antibody (1:1000
Figure 4 | optogenetic implantation 1) Optic fiber with at-tached metal ferrule of 3.5mm with two slots close to skull for better attachment of dental cement. 2) Ceramic sleeve used to connect the polished side of the ferrule with the polished side of the cable for light transmission. 3) Cable connected to laser for optimal light transmission. Note: between 1 and 3 (in 2) plasticine was put to prevent light reaching the brain but to mimic scattering of light during stimulation trials.
Figure 6 | Fluorescent image of viral infection in PL. Red represents NeuN staining, green shows GFP staining. Image shows a clear spread of virus around the PL. Figure 5 | schematic overview of virus and optic fiber site.
Left part consists of NeuN and GFP staining picture for viral infection. Right side drawn representation of viral infection site and optic fiber implantation site. Optic fibers are chroni-cally bilaterally implanted above the PL. By adjusting the depth of the fiber, light spread and thereby stimulation area is affected, preventing side effects caused by viral spread to other areas.
dilution, AB3080; Chemicon) and mouse anti-NeuN pri-mary antibody (1:1000 dilution, MAB377; Chemicon) in blocking buffer, containing 5% NGS, at 4˚C on a shaker. The following day, slices were again rinsed in TBS (4x 10 min) and incubated for 2 h with secondary Alexa 594-labeled goat anti-mouse antibody (1:400 dilution, A11005; Molecular Probes) and Alexa 488-labeled Goat-anti-rabbit antibody (1:400 dilution, A11070; Molecular Probes) in blocking buffer, containing 5% NGS. Finally, slices were rinsed in TBS (4x 10 min) and mounted on superfrost microscope slices (containing 0.2% gelatin in TBS), air dried, and cover slipped with DABCO anti-gading agent.
Fluorescent images of the mPFC were captured using a Leica DFC 450 color camera attached to a Leica
DM5000B microscope using the Leica Application Suite (LAS) v4.4 (Leica Microsystems, Wetzlar, Germany) and 2.5x/0.07 and 10x/0.4 objectives (protocol from Ray-mond van der Waal, 2015). Figure 5 and 6 show the lo-cation of viral expression and fiber implantation. Alt-hough we did not include these animals in our experi-ment, we expect to have same expression in our animals as protocols were identical.
Statistical Analysis
Two-way repeated measures ANOVO with posthoc tests were performed using Graphpad Prism version 5.00 for Windows (Graphpad Software, San Diego, California,
USA). To compare statistical values with each other, we used a non-linear regression (V = A / (1 + kD)). This re-gression calculated the indifference point (IP), which estimates the delay at which the individual displays a 50% preference for the larger reward (Mazur, 1987; Mazur, 2006). All behavioural data graphs shown in this report are made with Graphpad Prism version 5.00 and presented as means ± standard errors of the mean (SEM).
Results
We challenged rats to choose between a small, but immediate (1 pellet) reward and a larger, delayed reward (4 pellets) (figure 2). For this delayed reward task, the delay between choice and large reward deliv-ery increased across blocks of trials, thus testing the animal’s tolerance for delay. The animals were initially trained to discount across 5 delay blocks, (0, 5, 10, 20 and 40 seconds), widely consistent with a large body of delay-discounting literature ((Broos et al., 2012; Schip-pers et al., 2012; Diergaarde et al., 2008)), which we have termed in this thesis, the DR40 paradigm. In order to increase the number of trials sampled per 70 minute session, we later amended the paradigm (termed DR20), to include only 4 delay bocks, ranging from 0 to 20 se-cond, and then continued with all further testing in this manner. This yielded an additional 3 trials per delay
DR40 session performance eYFP vs Arch3.0
0 20 40 60 80 100 eYFP Arch3.0 5 10 20 40 Delay (s) % L R P re fe re n c e
*
#33 stable baseline over three DR40 sessions
0 20 40 60 80 100 DR40 day 1 DR40 day 2 DR40 day 3 5 10 20 40 Delay (s) % L R P re fe re n c e
DR40 stable preference for large reward
0 20 40 60 80 100 #33 (eYFP) #34 (eYFP) #35 (Arch3.0) #37 (Arch3.0) #38 (Arch3.0) 5 10 20 40 Delay (s) % L R P re fe re n c e
*
Figure 7 | Stable baseline performance calculated as an average over 3 sessions. a) Control animal #33 as an example of three stable DR40 sessions, showing similar delay discounting for multiple sessions. b) Stable baseline performance shows a delay-dependent decrease without difference between individual animals in the DR40. *p<0.0001 c) Stable baseline performance between control and experimental group does not different during DR40 performance. *p<0.0001
All conditions eYFP
0 20 40 60 80 100 DR40 pre surgery DR40 post surgery DR20 baseline Cables Sham 5 10 20 Washout Delay (s) % L R P re fe re n c e
All conditions Arch3.0
0 20 40 60 80 100 DR40 pre surgery DR40 post surgery DR20 baseline Cables Sham 5 10 20 Washout Delay (s) % L R P re fe re n c e
Figure 8 | Baseline performances over multiple conditions included in the experiment a) Stable baseline consistent over dif-ferent conditions for eYFP animals. No difference in delay discounting is observed between conditions. b) Stable baseline over different conditions for Arch3.0 animals show same delay discounting as eYFP animals.
A B C
block, per session.
At the time of writing, five animals had progressed through all stages of testing, and their data are there-fore included for analysis and interpretation in the sub-sequent sections of this thesis.
Animals first initiate a trial by nose poking in a middle aperture, and then nose poke their preference for one of the two illuminated options either side of this aperture (left or right apertures, counterbalanced across rats) (figure 3). Rats were given 2 forced trials at the beginning of every delay block, where only one of the two choice apertures were illuminated; one trial each for the small and the large rewarded option respectively, before continuing with 10 (or 13 in DR20) free-choice trials, where both apertures were illuminated for choice.
Prelimbic cortex expression in pyramidal neurons of AAV-CamKIIa viral construct containing opsin Arch3.0, does not significantly alter discounting behaviour for delayed rewards; The rats develop distinct, but highly stable baseline choice behaviour.
Following a training period of 12-16 weeks, all animals previously infused with viral construct devel-oped a stable discounting behaviour across sessions, (F (2, 8) = 1.019, p = 0.4035). The typical discounting curve is depicted for one rat (figure 7a). All animals decreased their preference for the larger reward as delay in-creased, therefore discounting in a delay-dependent manner (figure 7b). A significant effect of delay was found (F (4, 16) = 64.80, p < 0.0001) but not between animals (F (4, 16) = 1.84, p = 0.1714), therefore baseline performance does not differ between animals
perform-Figure 9 | Baseline performances converted to indifference points using a nonlinear regression analysis. Indifference points over conditions correspond to preference for large reward, showing stable baseline between conditions and no difference between groups.
Figure 10 | Average session performance over 3 stable ses-sions. Shift of indifference point during stimulation sessions between eYFP and Arch3.0 groups. Note the decrease in IP for both groups, suggesting distraction by light.
IP over sessions eYFP vs Arch3.0
0 5 10 15 20 25 Arch3.0 eYFP DR40 pre sur gery DR40 pos t sur gery DR20 bas elin e Cabl es Sham Was hout In d if fe re n c e P o in t (s )
IP over sessions eYFP vs Arch3.0
Stim ulatio n Was hout 0 5 10 15 20 eYFP Arch3.0 In d if fe re n c e P o in t (s )
IP ON trials eYFP vs Arch3.0
Stim ulat ion ON Wa shou t ON 0 5 10 15 20 25 eYFP Arch3.0 In d if fe re n c e P o in t (s )
IP OFF trials eYFP vs Arch3.0
Stimu latio n OF F Wa shou t OF F 0 5 10 15 20 25 Arch3.0 eYFP In d if fe re n c e P o in t (s )
Figure 11 | Graphs illustrating different behavioural responses to laser-ON and laser-OFF stimulation trials, representing car-ryover effects of PL inactivation. a) Shift in indifference point during ON trial analysis between eYFP (orange) and Arch3.0 (blue). Both groups show visually decrease in IP during laser-ON stimulation trials. b) Shift in indifference point during OFF trial analysis between eYFP (orange) and Arch3.0 (blue). Visually analyses seem to show stable IP’s for the control animals and a slight de-crease in IP for the Arch3.0 animals during trials with no laser light inhibition.
ing the delayed reward task. Three of these animals had been infused with a viral construct for expressing Ar-chaerhodopsin (AAV-CamKIIa-Arch3.0-eYFP), in the Pre-limbic cortex (PL). The remaining two animals were in-fused identically, instead with a control viral construct (AAV-CamKIIa-eYFP), lacking the Arch3.0 inhibitory opsin and are henceforth identified as the eYFP group. There was no difference in discounting behaviour between the Arch3.0 and eYFP groups (figure 7c) (delay: F (4, 8) = 132.6, p < .0001), group: (F (1, 2) = 0.22, p = 0.682)
Delay-discounting remains stable following surgical implantation of optic fiber, as well as with subsequent cable tethering and light sham conditions
As shown in figure3, after reaching stable base-line performance in DR40, implantation surgery of opti-cal fibers in the PL was performed, rats performance returned to baseline and was then followed by habitua-tion to optic fiber cables and next light sham sessions. Sham sessions occurred prior to stimulation sessions, with a light block in the sleeve between cable and im-planted ferrule (figure 4), to habituate rats to receiving light to their head cap during choice execution, washout sessions occurred between stimulation sessions, where animals receive light to the head cap, but not transmit-ted to the brain, identical setup as in figure 4, used for light sham sessions. To examine if these conditions in-fluenced baseline discounting, we performed repeated measures ANOVA over data across three stable sessions, (figure 8a & b). After reaching stable baseline per condi-tion, no difference was observed in eYFP performance between conditions (F (5, 6) = 0.65, p = 0.6750) nor be-tween eYFP animals (F (15, 18) = 1.46, p = 0.22). Same results have been obtained for the Arch3.0 animals, respectively (F (5, 11) = 0.40, p = 0.8391 & F (15, 33) =
0.70, p = 0.7606). In addition, no difference was ob-served between eYFP and Arch3.0 groups concerning baseline performance between conditions (F (15, 18) = 1.07, p = 0.4428). We demonstrate here that animals discounting curve is not shifted when the maximum delay is 20sec vs 40 seconds
To compare differences between conditions and experimental groups in their discounting preferences, we use a non-linear regression model (Mazur, 1987) to obtain an indifference point (IP). This estimates the de-lay at which the individual dispde-lays a 50% preference for the larger reward. Similarly to analyzing percentage preference across delay blocks between groups, com-paring indifference point shows stable performance across conditions and between groups (F (4, 16) = 1.41, p = 0.2732) (figure 9).
Prelimbic pyramidal inhibition during choice execution decreases indifference point.
We muted glutamatergic pyramidal activity in neurons expressing the Arch3.0 inhibitory opsin using light at wavelength 532nm. The bilateral inhibition in the PL cortex at temporally discrete moments of the delayed reward task were timed to coincide with the motor exe-cution of the choice, commencing with illumination of the middle initiation aperture and extinguishing once the choice had been executed, or after 3 seconds, whichever occurred first (figure 2). Comparing IP of stimulation and washout sessions, we see a decrease in IP during stimulation sessions (figure 10). Repeated measure ANOVA revealed a significant interaction effect condition x group (F (1, 3) = 38.51, p = 0.0084. This in-teraction effect suggests that the effect of laser light stimulation is inconsistent between groups, possible because of the low sample size. We proceed with
de-PL 3sec OFF trials - inhibition
0 5 10 15 20 25 33 eYFP 34 eYFP 35 Arch 37 Arch 38 Arch Stim ulat ion OFF Was hout OFF In d if fe re n c e P o in t (s )
PL 3sec ON trials - inhibition
0 5 10 15 20 33 eYFP 34 eYFP 35 Arch 37 Arch 38 Arch Stim ulat ion ON Was hout ON In d if fe re n c e P o in t (s )
Figure 12 | IP’s of all individual animals plotted over stimulation and washout trials, divided on laser-ON and laser-OFF trials. a) Individual shift in IP between laser-ON trials during stimulation and washout trials. All Arch3.0 animals show a decrease in IP dur-ing laser-ON stimulation trials, while control animals show unstable behaviour. b) Individual shift in IP between laser-OFF trials during stimulation and washout trials. Again, all Arch3.0 animals show a decrease in IP during laser-OFF stimulation trials, compa-rable with laser-ON stimulation trials. However, both control animals show no decrease.
scriptive statistics for the remainder of this thesis, due to small sample sizes in each group. Visual inspection of session-wise IP changes suggests a trend towards re-duced indifference point for Arch3.0 expressing animals during stimulation sessions, in other words preference to the smaller immediate reward occurs sooner during sessions where they receive light inhibition. We also see a slight reduction in the eYFP group on stimulation ses-sions (figure 10).
Light inhibition occurred only in 50% of the choice trials randomly, to see if biasing the animal’s be-haviour occurs only on trials where the light is received, thus suggesting an online role of the PL cortex during choice execution as opposed to session-wide influence. During trials where the animal received light inhibition (termed, ON trials) we found a similar decrease in IP for both the Arch3.0 and eYFP groups (figure 11a). However, during trials where no light was received, termed OFF trials, we see a decrease in IP only in the Arch3.0 group (figure 11b).
Upon visual inspection, while all Arch3.0 animals showed a decrease in IP during ON trials, the eYFP ani-mals exhibited differential effects of stimulated trials (figure 12a). During OFF trials, all Arch3.0 animals showed this decrease in IP, but the control animals show no difference in responding (figure 12b).
Omission rates increase across delay blocks, but remain similar under all conditions.
To determine whether omission rate altered ei-ther due to distraction by light scatter during washout sessions, or if motivational or attentional factors were involved during stimulation sessions, we compared omission rates, separated for ON and OFF trials, across different conditions. We observed an increase in omis-sion to approximately 20% by the final block for both laser-ON and OFF trials during stimulation and washout sessions (figure 13a & b). This increase was also notable during DR20 baseline sessions, indicating stable
omis-sion rates over all conditions, irrespective of cable teth-ering, light scatter or light inhibition to the PL cortex. Discussion
The initial goal of this preliminary study was to elucidate the role of the glutamatergic pyramidal neu-rons of the Prelimbic (PL) cortex on impulsive choice decision-making and gain a better understanding of its temporal role in this cognitive demanding process. Our results show a trend that PL is indeed involved in impul-sive decision-making and more precisely, the glutama-tergic pyramidal neurons during choice execution of the decision. We also see that the representation we disturb in the PL with light is biasing multiple trials of decision-making, resulting in a shift in preference for the large reward in these animals. Previous pharmacological and lesion studies show that the PL is involved in impulsive choice decision-making (Sonntag et al., 2014; Moschak & Mitchell, 2014; Pattij & Vanderschuren, 2008; Pothuizen
et al., 2005), but lack this temporal and cell type
preci-sion which we achieved using optogenetics. We now can conclude that the PL is an interesting area to study im-pulsive choice in an online fashion and that optogeneti-cal inhibition affects behavioural readout.
Our goal during this experiment was to deter-mine PL involvement during impulsive choice decision-making, using a combination of optogenetic approach with behavioural readout in freely behaving animals. For the first time in our lab, we managed to successfully alter neural activity on a temporal precise scale while animals perform on the delayed reward task. As this lab has repeatedly established successful interventions in decision-making tasks using lesion and pharmacological interventions, we now can manage optogenetic control of brain areas using the same behavioural readouts. However, this was not without any difficulties. We met technical limitations of many sorts, including the fact that optical interfering activity in the PL might diffuse to Omissions during ON trials Arch3.0 and eYFP
0 5 10 15 20 25 DR20 baseline Stimulation on Washout on 5 10 20 Delay (s) O m is s io n s ( % o f tr ia ls )
Omissions during OFF trials Arch3.0 and eYFP
0 5 10 15 20 25 DR20 baseline Stimulation off Washout off 5 10 20 Delay (s) O m is s io n s ( % o f tr ia ls )
Figure 13 | Percentage of omissions over whole session (70 minutes) a) Percentage of omissions of all ON trials during DR20 baseline, laser-ON stimulation and laser-ON washout sessions for both eYFP and Arch3.0. b) Percentage of omissions of all OFF trials during DR20 baseline, laser-OFF stimulation and laser-OFF washout sessions for both eYFP and Arch3.0.
other infected and closely located areas, satiation during the task might conduct, and that distraction by light can occur even when put a lot of effort to prevent this. Fur-ther, although we seem to bias the decision of the ani-mal, we could not specify the time of decision-making, but rather, had to rely on our behavioural readout. De-spite these and other problems, we found a striking in-ternal consistency to the results that we obtained across animals, across behaviour and optogenetic findings. Finally, because experiments are still running with these and other animals not included in this paper, we could not perform immunohistochemistry on animals which behaviour has been used in this paper. Therefore, we are waiting for final validation of opsin expression in the appropriate brain area and actual recordings of infected neurons using in vitro electrophysiology.
Satiation during a session using increased delays in the delayed reward task
Using our delayed reward task, we face the problem of satiation during the task. An inherently property of the delayed reward task using increasing delays, is the decrease in drive of animals to perform the task. While animals are on 90% free food restriction, they are willing to perform, especially during the first blocks. The omission data in figure 13 show that during the session animals are less motivated, get distracted or loose their interest in the task which results in omitting trials. This gradually increase of omissions is likely due to reaching sufficient intake of food pellets, therefore changing the subjective value of a reward over time (McClure et al., 2014). Although omission rates stay below 20%, there has been proposed a possible solution for this effect, namely the adjusted delayed reward task (Papale et al., 2012; McClure et al., 2014; Bett et al., 2015). Instead of increase of delay, the length of delay is determined by the choice the animal makes. When choosing small reward the delay decreases, and when choosing the large reward the delay increases, resulting in an estimation of an individual indifference point. We validated our DR20 model by comparing
delay-discounting and omission rates between DR20 and DR40 baseline performance. We could not find a difference within subjects for DR20 and DR40, nor omissions, show-ing that the DR20 does not alter satiation or delay-discounting. Although preference for the large reward does not seems to be biased using the DR20 paradigm, we found a shift in indifference point, largely deter-mined by a changed MAZUR equation distribution. When using a nonlinear regression analysis, there seems to be a difference in distribution of the regression line when leaving out the outer values of the 40sec delayed block of the DR40, resulting in higher indifference points (Mazur, 1987; Mazur, 2006). Comparing animals within a study does not seem to be biased use this changed dis-tribution of indifference points, although comparing
between studies using both DR40 and DR20 it is im-portant to note the shift in indifference points.
Animals show stable performance over conditions
As this study confirms, animals show highly sta-ble baseline choice behaviour under the delayed reward task. With a new method such as optogenetics and be-haviour combined, it is necessary to provide evidence for comparable baselines with normal baseline (that is, without virus infection or optic fiber implantation). Most recent studies remain vague about these factors, alt-hough all recent optogenetic reviews frequently at-tributed these factors as main challenges for research (Zalocusky & Deisseroth, 2013; Riga et al., 2014; Stuber
et al., 2012). We showed that for behaviour
perfor-mance no differences can be found in stable baseline performance on the delayed reward task if animals are injected with light sensitive opsins, underwent chronical-ly bilateral optic fiber implantation, are connected to laser cables, or when light scatter is involved caused by laser light.
Prelimbic inhibition necessary for stable performance
Due to low sample size and impulsive choice data still being collected with these and other animals, we could not perform reliable statistical analysis on our stimulation data. As seen in figure 10, there seems to be a trend towards increased impulsive choice when inhibi-tion the PL during motor execuinhibi-tion of decision. This is in agreement with previous studies, showing that the PL is involved in impulsive choice decision-making (Sonntag et
al., 2014; Pattij & Vanderschuren, 2008). Compared to
previous findings showing with lesioning the PL (Pot-huizen et al., 2005) and inhibiting all activity using mus-cimol (Moschak & Mitchell, 2014), we show that at least the glutamatergic pyramidal neurons are involved in this process. Those neurons are necessary for successful, stable performance during the task and inhibiting them during a distinct epoch biases the animals behaviour towards more impulsive.
Besides cell type precision, we also aimed at un-raveling temporal aspects of the PL. By inhibiting at a distinct time point during the task, during motor execu-tion of the decision, we show a temporal role for these glutamatergic pyramidal neurons in the PL on impulsive choice. Although inhibiting just lasted for 3 seconds, it seems enough for biasing the animals’ behaviour. As we cannot specify the timing of decision-making, we choose this moment to influence the decision. The task requires the animals to initiate a trial by making a response in the middle stimulus aperture when it illuminates. For the animal, this is a numerously repeated action, always followed by actual execution of his decision by nose poking the stimulus aperture of the small or large re-ward. Therefore, it seems likely that during this time
span, the decision might still be plastic and adjustable to influences (Friedman et al., 2015). Although our stimula-tion protocol was always consistent, it depended on the attention, motivation and willingness of the animal to perform if PL activity was abandoned three seconds during motor execution of the decision or just before motor execution. If the animal was distracted during the first three second of illumination of the middle stimulus aperture, the inhibition took place before initiating the trial, potentially biasing the decision by disturbing repre-sentations in the PL (Riga et al., 2014). Although statis-tics is unfeasiblein this preliminary group, our laser-ON trial analyses show a trend in the control animals, sug-gesting that they might be distracted by the bright laser light. Although we tried to block any scattering of light throughout the room, it is likely that the laser light still shines around the animal’s head, potentially shortly distracting him from the task. Interestingly, we did not find this effect during the laser-OFF trials in the control group, providing additional evidence that they are in-deed distracted by the light in the ON trials. Stimulating the PL before initiating the task by nose poking, thereby preventing any light scattering during performance, would abolish distraction during motor execution of the decision. Hence our next stimulation protocol is 3 se-cond prior to the start of the trial, which could potential-ly conclude if distraction is actualpotential-ly biasing animal’s be-haviour during the laser-ON trials. With stimulation 3 seconds prior to motor execution, we supposedly also prevent variation in inhibition timing, as all animals are still waiting for the next trial and don’t have to perform during those 3 seconds.
Online vs habitual decision-making using different ap-proach
With our current protocol, we aimed at biasing decision-making in an online fashion, shifting choice for the larger delayed reward to the smaller reward more early. By analyzing laser-ON trials to find such an effect, we find an indefinite picture, with lots of noise. It could be that light inhibition of the PL disrupts the internal representation of the task (Paine et al., 2013), or it might be that we only distract animals from performing because of light scattering. Fortunately, we were able to analyze data for laser-OFF trials, providing a clearer pic-ture of the behaviour readout. When looking at figure 11b, we see abolished decrease in IP for the control group during laser-OFF trials, providing evidence for distraction effects during laser-ON trials. Figure 9, points out that Arch3.0 still show increased impulsive choice behaviour on laser-OFF trials. PL pyramidal neurons in-duce carryover effects, lasting for multiple trials. During in between washout session, we see that this bias is abolished because animals return to sham base-line performance (figure 9). The carryover effects there-fore seem to last over trials, but not over sessions. We
know that decisions are among others made by previous experience and therefore carryover effects can influence prospective decisions (Sasaki & Pratt, 2013). It would be very interesting to sort trials on the bases of prior-ON and prior-OFF trials. If carryover effects by light inhibi-tion actually occur, we expect to find increased change for choosing the larger reward when previous trial had light inhibition. One might think that having multiple laser-ON sessions might bias multiple laser-OFF sessions, something that would be highly interesting to investi-gate with more data to analyze.
With comparing laser-ON laser-OFF trials, we can exclude temporal activation of the PL as incentive for the behaviour. A recent study shows that by illumi-nating the brain with laser light, spontaneous neuronal firing can occur (Stujenske et al., 2015). This activity only occurs when using high laser light intensities and for long period of light inhibition. In our protocol, we stimu-late a brief period with low intensity (~7 mW). If neural activity by heating up the brain would drive the behav-iour, we expect to lose this effect on trials with no laser light. Also our control conditions (sham and washout) provide evidence for data not induced by heating up the brain.
Another aspect worthwhile to investigate is if the PL is only involved in this online process of decision making, but also acquisition of delay-discounting. The medial prefrontal cortex is involved in maintenance of task-relevant information during goal-directed behav-iour, and recent studies show an important role in reaching stable baseline performance (Rossi et al., 2012). Using the delayed reward task with increased number of forced trials and less free choice trials, only stimulating the PL during the forced trials would poten-tially show a role for the PL during acquisition of delay-discounting.
Our findings need to be interpreted with some caution. Figure 1 shows all efferents and afferents of the PL with other brain areas. In our stimulation protocol, we inhibit all activity coming from or using the PL as a sort of relay station, but also ipsi and contralateral pro-jections (Ji & Neugebauer, 2012). Therefore, we need to keep in mind that this approach might influence multiple aspects of decision-making (Euston et al., 2012). For example, by inhibiting the PL, we also affect the Pl to Nucleus Accumbens core projection which is know to be involved in reward-estimation (Russo & Nestler, 2013). As it is known that the PL to Basal lateral Amygdala pro-jection codes emotional salient events, we might affect emotional processes underlying decision-making (Sierra-Mercado et al., 2011). Therefore we cannot conclude which behavioural process we are exactly influencing with this protocol. Hebbian learning shows that neurons who fire together, wire together (Johansen et al., 2010). By inhibiting the PL, the strength of the connections to the PL is temporally biased which could lead to persis-tent synaptic changes like long term depression (LTD).
Using a stimulation temporal frame of 3 seconds seems insufficient to bias LTP (Van Huijstee & Mansvelder, 2015), but more research is necessary to exclude LTD effects.
Contingent diametrically opposing effects of the PL during delay-discounting
With the increase in available optogenetic ap-proaches and the decrease in cost for using them, it would be highly interesting to look for diametrically opposing effects of the PL during impulsive choice deci-sion-making. When increased sample size indeed sup-ports our hypothesis, it is interesting to see if activating the PL during the delayed reward task biases their be-haviour towards less impulsive. Using Channelrhodopsin (ChR2) expressing animals in the PL, we could potentially find this effect by activating the PL during motor execu-tion of the decision or maybe even 3 seconds prior to this motor execution. Previous studies show that PL activity coding specific processes regarding reward esti-mation consists of firing frequencies in the beta fre-quency; about 15 Hz (Friedman et al., 2015). If indeed the PL codes for sensitivity to delay, we expect to see a decrease in impulsive choice when activating the PL with this appropriate frequency. Using this approach, results could potentially be biased even more as we also
influ-ence local oscillations and spike firing timing (Nakamura
et al., 2012). By using both light stimulation methods
(Arch3.0 for inhibition and ChR2 for activation), results are likely to be interpreted more reliable, potentially showing diametrically opposing effects.
Underlying neuronal mechanism
As we currently measure behavioural readout, future studies should focus on underlying neuronal cor-relates, representing behaviour of the animals. It would be of particular interest to correlate neuronal activity to behavioural processes, possible influenced by optoge-netic approaches (Friedman et al., 2015). Previous stud-ies show that PL activity correlates, for example, with reward prediction (Sul et al., 2010), reinstatement of cocaine seeking (Stefanik et al., 2012), and other cogni-tive processes (de Visser et al., 2011), but networks are yet to be described. Combining optogenetic with single unit or tetrode recordings would provide insight infor-mation in actual underlying mechanisms of how the brain processes delay during an impulsivity task, which can be specified to projections or neuronal circuits to be even more precise which process we are affecting. This might lead to findings revealing that inhibiting large area not only affects impulsivity, but also cognitive processes like working memory, attention or time estimation.
Reference
Amitai, N., & Markou, A. (2011). Comparative effects of different test day challenges on performance in the 5-choice serial reaction time task. Behavioral Neuroscience, 125(5), 764–74.
Bedwell, S.A., Billett, E.E., Crofts, J.J. & Tinsley, C.J. (2014) The topology of connections between rat prefrontal, mo tor and sensory cortices. Front. Syst. Neurosci., 8, 177-187.
Bett, D., Murdoch, L.H. & Wood, E.R. (2015) Hippocampus, delay discounting, and vicarious trial-and-error. Hippocampus., 25, 643-654.
Britt, J. P., Benaliouad, F., McDevitt, R. A., Stuber, G. D., Wise, R. A., & Bonci, A. (2012). Synaptic and behavioral profile of multiple glutamatergic inputs to the nucleus accumbens. Neuron, 76(4), 790–803.
Broos, N., Diergaarde, L., Schoffelmeer, A. N., Pattij, T., & De Vries, T. J. (2012). Trait impulsive choice predicts resistance to extinction and propensity to relapse to cocaine seeking: a bidirectional investigation.
Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology, 37(6), 1377– 86.
Broos, N., Schmaal, L., Wiskerke, J., Kostelijk, L., Lam, T., Stoop, N., … Goudriaan, A. E. (2012). The relationship between impulsive choice and impulsive action: a cross-species translational study. PloS One, 7(5), e36781. Cardinal, R. N. (2006). Neural systems implicated in delayed and probabilistic reinforcement. Neural Networks : The
Official Journal of the International Neural Network Society, 19(8), 1277–301.
Chudasama, Y. (2011). Animal models of prefrontal-executive function. Behavioral Neuroscience, 125(3), 327–43. Churchwell, J. C., Morris, A. M., Heurtelou, N. M., & Kesner, R. P. (2009). Interactions between the prefrontal cortex
and amygdala during delay discounting and reversal. Behavioral Neuroscience, 123(6), 1185–96. Cottone, P., Iemolo, A., Narayan, A. R., Kwak, J., Momaney, D., & Sabino, V. (2013). The uncompetitive NMDA
receptor antagonists ketamine and memantine preferentially increase the choice for a small, immediate reward in low-impulsive rats. Psychopharmacology, 226(1), 127–38.
Dalley, J.W., Everitt, B.J. & Robbins, T.W. (2011) Impulsivity, Compulsivity, and top-down cognitive control. Neuron, 69, 680-694.
cortex affects both anxiety and decision-making in male wistar rats. Frontiers in Neuroscience, 5, 1-8. Diergaarde, L., Pattij, T., Poortvliet, I., Hogenboom, F., De Vries, W., Schoffelmeer, A.N.M., & De Vries, T.J. (2008)
Impulsive choice and impulsive action predict vulnerability to distinct stages of nicotine seeking in rats. Biological Psychiatry, 63(3), 301–308.
Economidou, D., Pelloux, Y., Robbins, T.W., Dalley, J.W. & Everitt, B.J. (2009) High impulsivity predicts relapse to cocaine-seeking after punishment-induced abstinence. Biol. Psychiatry, 65, 851-856.
Euston, D. R., Gruber, A. J., & McNaughton, B. L. (2012). The role of medial prefrontal cortex in memory and decision making. Neuron, 76(6), 1057–70.
Feja, M., & Koch. M., (2014). Ventral medial prefrontal cortex inactivation impairs impulse control but does not affect delay-discounting in rats. Behavioural Brain Research, 264, 230-239.
Floresco, S. B., Tse, M. T. L., & Ghods-Sharifi, S. (2008). Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology, 33(8), 1966–79.
Friedman, A., Homma, D., Gibb, L.G., Amemori, K., Rubin, S.J., Hood, A.S., Riad, M.H. & Graybiel, A.M. (2015) A corticostriatal path targeting striosomes controls decision-making under conflict. Cell, 161, 1320-1333. Gill, T. M., Castaneda, P. J., & Janak, P. H. (2010). Dissociable roles of the medial prefrontal cortex and nucleus
accumbens core in goal-directed actions for differential reward magnitude. Cerebral Cortex (New York, N.Y. : 1991), 20(12), 2884–99.
Hosking, J. G., Cocker, P. J., & Winstanley, C. A. (2015). Prefrontal Cortical Inactivations Decrease Willingness to Expend Cognitive Effort on a Rodent Cost/Benefit Decision-Making Task. Cerebral Cortex (New York, N.Y. : 1991), bhu321–.
Ji, G., & Neugebauer, V. (2012). Modulation of medial prefrontal cortical activity using in vivo recordings and optogenetics. Molecular Brain, 5(1), 36.
Johansen, J.P., Hamanaka, H., Monfils, M.H., Behnia, R., Deisseroth, K., Blair, H.T. & LeDoux, J.E. (2010) Optical activation of lateral amygdala pyramidal cells instructs associative fear learning. Proc. of the Nat. Acad. of Sci. of the Uni. Sta. of Ame., 107, 12692-7.
Kalivas, P. W., Volkow, N., & Seamans, J. (2005). Unmanageable motivation in addiction: a pathology in prefrontal- accumbens glutamate transmission. Neuron, 45(5), 647–50.
Kim, S., & Lee, D. (2011). Prefrontal cortex and impulsive decision making. Biological Psychiatry, 69(12), 1140–6. Krettek, J. E., & Price, J. L. (1977). The cortical projections of the mediodorsal nucleus and adjacent thalamic nuclei in
the rat. The Journal of Comparative Neurology, 171(2), 157–91.
Little, J.P., & Carter, A.G. (2012). Subcellular synaptic connectivity of layer 2 pyramidal neurons in the medial prefrontal cortex. The Journal of Neuroscience, 32(37), 12808-12819.
Mazur, J.E. (1987) An adjusting procedure for studying delayed reinforcement. Erlbaum; Hilssdale, NJ: 1987. Mazur, J. (2006). Learning and Behavior 6 ed.).New Jersey: Pearson Education, Inc.
McClure, J., Podos, J. & Richardson, H.N. (2014) Isolating the delay component of impulsive choice in adolescent rats. Front. in Neurosci. 8, 1-9.
Moschak, T. M., & Mitchell, S. H. (2014). Partial inactivation of nucleus accumbens core decreases delay discounting in rats without affecting sensitivity to delay or magnitude. Behavioural Brain Research, 268, 159–68.
Murphy, E.R., Fernando, A.B., Urcelay, G.P., Robinson, E.S., Mar, A.C., Theobald, D.E., Dalley, J.W., & Robbins, T.W. (2012). Impulsive behaviour induced by both NMDA receptor antagonism and GABAA receptor activation in rat ventromedial prefrontal cortex. Psychopharmacology, 219(2), 401-410.
Nakamura, S., Baratta, M.V., Pomrenze, M.B., Dolzani, S.D. & Cooper, D.C. (2012) High fidelity optogenetic control of individual prefrontal cortical pyramidal neurons in vivo. F1000Research, 1, 1-7.
Ongür, D., & Price, J. L. (2000). The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cerebral Cortex (New York, N.Y. : 1991), 10(3), 206–19.
Paine, T.A., Asinof, S.K., Diehl, G.W., Frackman, A. & Leffler, J. (2013) Medial prefrontal cortex lesions impair
decision-making on a rodent gambling task: reversal by D1 receptor antagonist administration. Behav. Brain Res., 243, 247-254.
Papale, A.E., Stott, J.J., Powell, N.J., Regier, P.S. & Redish, A.D. (2012) Interactions between deliberation and delay- discounting in rats. Cogn. Affect. Behav. Neurosci.,12, 513-526.
Pattij, T., & Vanderschuren, L. J. M. J. (2008). The neuropharmacology of impulsive behaviour. Trends in Pharmacological Sciences, 29(4), 192–9.
Pothuizen, H.H.J., Jongen-Rêlo, A.L., Feldon, J., & Yee, B.K. (2005). Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats. European Journal of Neuroscience, 22(10), 2605–2616.
induced by blockade of N-methyl D-aspartate receptors in the prefrontal cortex is associated with enhanced glutamate release and cAMP response element binding protein phosphorylation: role of metabotropic glutamate receptors 2/3. Neuroscience, 176, 336-348.
Riga, D., Matos, M.R., Glas, A., Smit, A.B., Spijker, S. & Van den Oever, M.C. (2014) Optogenetic dissection of medial prefrontal cortex circuitry. Front. in Systems Neurosci. 8, 230-249.
Rossi, M.A., Hayrapetyan, V.Y. Maimon, B., Mak, K., Shwan Je, H. & Yin, H.H. (2012) Prefrontal cortical mechanisms underlying delayed alternation in mice. Journal of Neurophysiology, 108, 1211-1222.
Russo, S. J., & Nestler, E. J. (2013). The brain reward circuitry in mood disorders. Nature Reviews. Neuroscience, 14(9), 609–25.
Saddoris, M. P., Sugam, J. A., Stuber, G. D., Witten, I. B., Deisseroth, K., & Carelli, R. M. (2014). Mesolimbic Dopamine Dynamically Tracks, and is Causally Linked to, Discrete Aspects of Value-Based Decision Making. Biological Psychiatry, 15, 903-911.
Schippers, M.C., Binnekade, R., Schoffelmeer, A.N.M., Pattij, T. & De Vries, T.J. (2012) Unidirectional relationship between heroin self-administration and impulsive decision-making in rats. Psychopharmacology, 219, 443- 452.
Sierra-Mercado, D., Padilla-Coreano, N. & Quirk, G.J. (2011) Dissociable Roles of Prelimbic and Infralimbic Cortices, Ventral Hippocampus, and Basolateral Amygdala in the Expression and Extinction of Conditioned Fear. Neuropsychopharmacology, 36, 529-538.
Sonntag, K. C., Brenhouse, H. C., Freund, N., Thompson, B. S., Puhl, M., & Andersen, S. L. (2014). Viral over- expression of D1 dopamine receptors in the prefrontal cortex increase high-risk behaviors in adults: comparison with adolescents. Psychopharmacology, 231(8), 1615–26.
Stefanik, M.T., Kupchik, Y.M., Brown, R.M. & Kalivas, P.W. (2013) Optogenetic evidence that pallidal projections, not nigral projections, from the nucleus accumbens core are necessary for reinstating cocaine seeking. The Journ. of Neurosci., 33, 13654-62.
Stuber, G.D., Britt, J.P. & Bonci, A. (2012) Optogenetic modulation of neural circuits that underlie reward seeking. Biol. Psychiatry, 71, 1061-1067.
Stujenske, J.M., Spellman, T. & Gordon, J.A. (2015) Modeling the spatiotemporal dynamics of light and heat propagation for in vivo optogenetics. Cell, 12, 525-534.
Sukhotina, I.A., Dravolina, O.A., Novitskaya, Y., Zvartau, E.E., Danysz, W., & Bespalov, A.Y. (2008). Effects of mGlu1 receptor blockade on working memory, time extimation, and impulsivity in rats. Psychopharmacology, 196(2), 211-220.
Sul, J. H., Kim, H., Huh, N., Lee, D., & Jung, M. W. (2010). Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron, 66(3), 449–60.
Tye, K. M. (2012). Glutamate inputs to the nucleus accumbens: does source matter? Neuron, 76(4), 671–3. Van Gaalen, M.M., van Koten, R., Schoffelmeer, A.N.M. & Vanderschuren, L.J.M.J. (2006) Critical Involvement of
Dopaminergic Neurotransmission in Impulsive Decision Making. Biological Psychiatry, 60, 66-73.
Van Hyijstee, A.N., & Mansvelder, H.D. (2015) Glutamatergic synaptic plasticity in the mesocorticolimbic system in addiction. Front. Cell Neurosci., 8, 466-501.
Winstanley, C. A., Eagle, D. M., & Robbins, T. W. (2006). Behavioral models of impulsivity in relation to ADHD: translation between clinical and preclinical studies. Clinical Psychology Review, 26(4), 379–95.
Witten, I.B., Steinberg, E.E., Lee, S.Y., Davidson, T.J., Zalocusky, K.A., Brodsky, M., Yizhar, O., Cho, S.L., Gong, S., Ramakrishnan, C., Stuber, G.D., Tye, K.M., Janak, P.H. & Deisseroth, K. (2011) Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron, 72, 721-733. Zalocusky, K. & Deisseroth, K. (2013) Optogenetics in the bahaving rat: integration of diverse new technologies in a
vital animal model. Optogenetics, 1-17.
Teford, S.E., Persons, A.L. & Napier, T.C. (2015) Dopaminergic lesions of the dorsolateral striatum in rats increase delay discounting in an impulsive choice task. PLoS One, 10, 1-17.
