The Predominant Role of the Locus Coeruleus in Facilitating Flexible
Goal-Directed Behaviour
8 - 1 -2017
Tijl van den Bos (5823293)
Thomas Meindertsma
MSc in Brain and Cognitive Sciences, University of Amsterdam
Cognitive Neuroscience
Abstract
In order to adaptively respond in a dynamic environment, animals must be able to flexibly update their understanding of the environment when their expectations are violated. Current models implicate a role for the locus coeruleus (LC) and orbitofrontal cortex (OFC) in flexible goal-directed behaviour. We review neurophysiological and imaging studies with animals and humans to understand the functional connection between the OFC and LC and their effect on goal-directed behaviour. LC neurons exhibit two modes of activity: phasic and tonic. Phasic LC activity is associated with the facilitation of cortical responses to task-relevant stimulus, and is associated with increased task performance. On the other hand, tonic activity is associated with reduced task performance and task disengagement. How the LC determines its mode of firing remains relatively unclear. It has been proposed that the OFC regulates the mode of firing of the LC, by signalling the LC to increase tonic firing when expected rewards are too low. However, no direct evidence has supported this proposition thus far. Conversely, evidence suggests that LC activity modulates membrane potentials in the OFC, possibly pushing the OFC to signal for a change in response strategy, when task contingencies change. Together, the current reviewed studies support an important role of the LC and
OFC in flexible goal-directed behaviour, in which the LC appears to dictate the interaction between the two brain areas.
Introduction
Organisms pursuing goal-directed behaviour in an ever changing environment face two challenges. On the one hand, they must maintain goals in the pursuit of ongoing reward while facing distracting stimuli and inhibit inappropriate competing responses. On the other hand, organisms must monitor the environment for stimuli with potentially higher reward that may require them to interrupt an ongoing action and switch to a different goal. Maintaining focus on one goal and searching for another have complementary costs and benefits. For instance, complete shielding of goals prevents interference of distractors, but may increase the risk of missing significant information. While organisms that are always exploring other options, might risk never gaining any significant reward at all. This dilemma illustrates what is referred to as the exploit/explore trade-of (Aston-Jones & Cohen, 2005): whether to continue pursuing a known source of reward (exploit), or search for new ones (explore). How the brain weighs the trade-off is still relatively unclear. A role for the orbitofrontal cortex (OFC) in choice comparison and reward evaluation has been widely supported (Roesch & Olson, 2004; Rolls et al. , 2004; Tremblay & Schultz, 1999; Wallis & Miller, 2003; Breiter et al., 2001; O’Doherty et al. 2002; Critchley & Rolls 1996, Rolls et al. 1989; Schultz et al. 2000, Hollerman et al. 2000). Likewise, a vast number of studies converge towards a much older brain area, the locus coeruleus (LC), in processing task-relevant stimuli (Aston-Jones et al., 1994; Usher et al., 1999; Aston-Jones et al., 1997; Aston-Jones et al., 2005; Howells et al., 2012; Clayton et al., 2004; Gilzenrat et al., 2010; Murphy et al., 2011; Murphy et al., 2014). A very influential and important model, named the adaptive gain theory (Aston-Jones & Cohen, 2005), provides a theoretical framework in order to better understand the neural mechanism underlying exploiting and exploring behaviour. In short, the adaptive gain theory states that the trade-off between exploring and exploiting behaviour is caused by the mode of activity of the LC: tonic and phasic firing. Since the OFC and LC respond similarly to task-relevant stimuli (Tremblay & Schultz, 1999; Wallis & Miller, 2003; Aston-Jones et al., 1994, Rajkowski et al., 2004), and since the OFC directly projects onto the LC (e.g. Aston-Jones et al. 2002; Rajkowski et al. 2000), Aston-Jones and Cohen (2005) propose that the mode of activity of the LC is (among others) modulated by the OFC. However, up to this point no paper has been published showing that the OFC directly regulates the mode of activity of the LC. Importantly, it is currently unknown how exactly the LC determines its mode of firing in response to its
surroundings. Reversely, the LC is known to modulate activity in the neocortex (e.g. Enschenko et al., 2011; Safaai et al., 2015) and has projections onto the OFC (e.g. Agster et al., 2013; Chandler et al., 2013), which gives reason to believe that the LC modulates activity in the OFC in order to balance the exploit/explore trade-of. In the current review, we explore the interaction between the LC and OFC in order to better understand the neural mechanism underlying goal-directed behaviour. To this end, we review the current state of the adaptive gain theory by introducing new support and discuss its shortcomings. First, we review the mode of firing of the LC in relation to the exploit/explore trade-of. Second, we discuss the function of the OFC in the evaluation of reward and goal-directed behaviour. Continuing, we discuss the possibility of the OFC regulating the mode of firing of the LC. Finally, we review clinical research in support of a recently published model (Sadacca et al., 2016), proposing that the LC modulates activity of the OFC in order to flexibly balance the trade-off between exploiting and exploring behaviour.
Current state of the adaptive gain theory
The importance of the locus coeruleus in optimizing goal-directed behaviour
At the centre of the adaptive gain theory (Aston-Jones & Cohen, 2005) lies the locus coeruleus norepinephrine (LC-NE) system. The locus coeruleus (LC), located in the dorsorostral pons, is the primary source of the catecholamine neurotransmitter norepinephrine (NE) to the cerebral, cerebellar and hippocampal cortices (Amaral & Sinnamon, 1977; Chandler et al., 2012). LC neurons can be distinguished by their characteristic tonic and phasic modes of firing. LC neurons have a diurnal variation of activity; during alert wakefulness the LC fires tonically, while during deep sleep the LC does not fire tonically (Aston-Jones & Bloom, 1981). During alert wakefulness the LC tonically fires in a range of 1-3 hz allowing phasic firing to occur (Howells et al., 2012). Phasic firing of the LC releases NE in the cortex which can have different effects on target neurons, depending on the receptor that is activated (Waterhouse and Woodward, 1980; Foote et al. 1983; Berridge & Foote, 1994; Berridge & Waterhouse 2003; Moxon et al., 2007). Activation of alpha1 NE receptors is generally associated with augmenting excitation, and activation of alpha2 NE receptors is generally associated with increased inhibition (Rogawski & Aghajanian 1982,Williams et al. 1985). Taken together, the release of extracellular NE can have modulatory effects on target cells, increasing the ratio of (synaptically) evoked activity to spontaneous activity in target neurons (Foote et al. 1975, Segal & Bloom 1976; Waterhouse et al. 1980; Waterhouse & Woodward 1980). Because of the widespread projections throughout the cortex and the modulatory effect on activated cells, traditionally investigators thought that the LC mediates arousal (Aston-Jones et al., 2005). Based on the historical model of Yerkes and Dodson (1908), optimal performance on cognitive and behavioural
tasks was and is thought to be held within a narrow range of an individual’s level of arousal (Yerkes & Dodson, 1908; Aston-Jones et al., 2005; Howells et al., 2012). The adaptive gain theory expands on this model and proposes a more sophisticated role for the LC in the control of behaviour. Rather than addressing arousal per se, the authors propose that the LC optimizes behaviour by varying the mode of activity, allowing exploring and exploiting behaviour.
The adaptive gain theory states that the LC is responsible for the trade-off between exploring and exploiting behaviour (Figure 1). This behaviour is acquired by the two different modes of activity in the LC: phasic and tonic. Phasic activity of the LC facilitates behavioural relevant responses, filtering responses to irrelevant events. By selectively facilitating responses to currently task-relevant processes, the phasic LC activity serves to optimize performance of the current engagement. Aston-Jones and Cohen (2005) further propose that the mode of activity of the LC is determined by frontal structures (OFC and anterior cingulate cortex), which are engaged in the assessment of the cost and benefit of a response to a stimulus. When task-outcome is lower than expected, the OFC and anterior cingulate cortex (ACC) signal the LC to increase tonic firing causing a withdrawal from task engagement, facilitating behaviour that serves to explore alternative ways to increase task-outcome.
Figure 1. Inverted-U relationship between LC activity and performance on a task that requires sustained attention. Performance is poor at low levels of LC tonic discharge, associated with an inattentive nonalert state (left side of the spectrum). Performance is optimal with moderate levels of LC tonic discharge, allowing phasic activity in response to
task-relevant stimuli (red surface area). Performance is poor at higher levels of LC activity, as behaviour becomes distractible to encourage exploring alternative resources (right side of the spectrum).
Primary support for the different modes of LC activity comes from studies in which monkeys performed in a signal detection task. In one experiment single and multi-cell activity in the LC was recorded from four monkeys while the monkeys performed in an oddball discrimination task (Aston-Jones et al., 1994). The monkeys were required to release a lever rapidly in response to an infrequent target stimulus that was randomly intermixed with non-target stimuli for which they had to withhold a response. Correct responses were rewarded by the delivery of juice. All LC neurons examined responded phasically and selectively to target stimulus (excluding motor responses, non-target stimuli). With the exception that phasic responses were also made in response to juice rewards. However, activity which preceded a juice reward was most likely caused by the preceding presentation of a target stimulus. During periods of elevated tonic LC activity, the animal’s ability to discriminate targets from distractors and its threshold for responding to stimuli both decreased, as LC tonic activity was consistently accompanied by frequent false-alarm errors (Aston-Jones et al. 1994; Usher et al. 1999). Interestingly, in a reversal experiment in which the distractor becomes the target and vice versa, LC phasic responses are quickly acquired to the new target and extinguished for the new distractor (Aston-Jones et al., 1994; Aston-Jones et al., 1997). The authors reported that LC phasic responses to the former target rapidly diminished, while baseline (tonic) LC firing increased. This was maintained until phasic responses appeared for the new target and disappeared for the old one. That is, the LC transitioned from a phasic to a tonic mode and then reversed as the new target was acquired. Not only does the LC respond to target stimuli, phasic LC activity is also associated with decision processes (Clayton et al., 2004). Monkeys were rewarded for correctly identifying the direction (left or right) of an arrow among flankers by respectively releasing the left or right lever, while LC activity was measured. The timing of phasic LC activity more closely followed behavioural responses than stimulus presentation. LC neurons were phasically activated preceding behavioural responses for both correct and incorrect identifications but were not activated by stimuli that failed to elicit lever responses nor by not-task-relevant lever movements. Clayton et al., 2004 argued that the LC phasically responds to the outcome of task-related decision processes, facilitating their influence on overt behaviour. LC neurons in the rat brain exhibit similar patterns in response to target versus non-target stimuli (Bouret & Sara 2004): LC neurons responded phasically to the food associated stimulus alone, which preceded the behavioural response. Furthermore, the phasic responses were more tightly linked to the behavioural response than to the sensory stimuli. The same study showed that in reversal conditions, the rat LC responses tracked the significance of the stimuli rather than stimulus identity. Together these results show that the LC phasic response is strongly
associated with the identification of task-relevant stimuli, possibly facilitating task-relevant responses in order to allow exploiting behaviour. In contrast, LC tonic activity is associated with distractible behaviour and increased false alarm errors (Aston-Jones et al. 1994; Usher et al. 1999).
Importantly, the same relationship between LC activity and exploiting and exploring behaviour has been established in humans (Gilzenrat et al., 2010; Murphy et al., 2011; Murphy et al., 2014). As pupil diameter correlates remarkably well with tonic LC activity in monkeys (Rajkowski et al. 1993; Joshi et al., 2016), several papers have used pupil diameter as a measure to study the causal influence of the LC in task-related behaviour in humans (Gilzenrat et al., 2010; Murphy et al., 2011; Murphy et al., 2014). Specifically, pupil diameter varies with the LC mode, such that the LC tonic mode is marked by a relatively large baseline pupil diameter and the LC phasic mode is marked by a relatively small baseline pupil diameter (Rajkowski et al. 1993). In addition, tonic LC activity is directly related to baseline pupil diameter: spontaneous and drug induced drowsiness and other low-arousal states, which are characterized by low tonic LC activity, are accompanied by a reduced baseline pupil diameter (e.g., Hou et al., 2005; Morad et al., 2000). Conversely, noradrenergic drugs that increase arousal and tonic LC activity also increase baseline pupil diameter (Phillips et al., 2000). Finally, both task processing (Beatty, 1982; Richer & Beatty, 1987; Einhäuser et al., 2008) and phasic LC activity (Reimer et al., 2016) are accompanied by rapid and dramatic pupil dilation. Importantly, the co-occurrence of rapid pupil dilation and phasic LC activity have been found in response to task-relevant cues and actions, in a single study (Varazzani et al., 2015). Together, these findings support that pupil diameter tracks LC activity, in such that baseline pupil diameter corresponds to tonic LC firing, and task-evoked dilations correspond to phasic LC activity. Gilzenrat et al., (2010) investigated the relationship between pupillary responses and behavioural task engagement and task disengagement, in humans. Participants performed a series of tone discriminations of progressively increasing difficulty. Rewards for correct responses increased in value as difficulty increased. At the beginning of every trial, participants were allowed to press a reset button, which would start a new trial beginning at the lowest difficulty. The authors found that small baseline pupils were associated with better performance (fewer false alarms and shorter less variable reaction times), as well as with larger task-evoked dilations. Conversely, large baseline pupils were associated with poorer performance and attenuated task-evoked dilations. Interestingly, the authors found a reliable increase in baseline pupil diameter - with a peak on the reset trial, indicative of the transition to LC tonic mode - as expected value began to decline and the participants approached a decision to reset. After a reset, baseline pupil diameter decreased and task-related dilations increased - indicative of the LC phasic mode. These findings provide indirect support for a close relationship between LC activity and exploiting and exploring behaviour. Importantly, these results support the results found in monkeys and rats, in that
exploring behaviour is associated with higher tonic LC activity (big baseline pupil diameter) and exploiting behaviour is associated with phasic activity (small baseline with larger task-evoked diameter response; Gilzenrat et al., 2010). A similar pattern of results was found in a study indirectly relating the exploit/explore trade-of to LC activity by measuring pupil diameter and the P3-component (Murphy et al., 2011). The P3-P3-component represents a cortical electrophysiological correlate of the phasic LC response (Nieuwenhuis et al., 2005; Nieuwenhuis et al., 2011). Task-relevant stimuli evoke robust P3-components (Polich, 2007; Aston-Jones et al., 1991) and stimuli accompanied by a large P3-component have higher chance of being detected and responded to (Hillyard, Squires, Bauer, & Lindsay, 1971; Parasuraman & Beatty, 1980). In the study of Murphy et al. (2011), human participants were engaged in an auditory oddball listening task. Participants were instructed to respond to target tones which were presented in 20% of the trials, and ignore presentation of distractor tones. They found that pre-stimulus pupil diameter exhibited an inverted U-shaped relationship to the P3-component and task performance. Largest P3-component amplitudes and optimal performance occurred at the same intermediate level of pupil diameter. By contrast, large phasic pupil dilations, were elicited during periods of poor performance and were often followed by reengagement in the task and increased P3-component amplitudes. These results are consistent with the adaptive gain theory: an intermediate level of pupil dilation (intermediate level of LC tonic activity enabling phasic activity) is associated with optimal performance, whereas large dilated pupils (high LC tonic activity) are associated with distractibility and explorative behaviour. These findings are supported in a study directly relating goal-directed behaviour to LC blood oxygenated level dependent (BOLD) activity and pupil diameter (Murphy et al., 2014). A positive relation was established between pupil diameter and BOLD activity in a dorsal pontine cluster overlapping with the LC. Meaning that an increased pupil diameter related to an increased BOLD response in the LC. This relationship was present both at rest and during performance of a two stimulus oddball task. Furthermore, the spatial extent of this pupil/LC relationship guided a region of interest analysis in which the authors provided support for the fundamental characteristic of animal LC activity in goal-directed behaviour. Namely, target stimuli evoked a larger LC BOLD response than distractor stimuli. In addition, between subjects correlations revealed that larger target-evoked BOLD responses were robustly associated with faster response times across subjects. Murphy 2014 et al., reasoned that the large evoked BOLD responses were associated with phasic LC activity, since they were selective for target stimuli only, which is consistent with selective phasic LC activity as shown in animal research (Aston-Jones et al., 1994; Bouret & Sara 2004).
Overall these results indicate that the LC response is highly plastic and that two modes of LC activity correspond to different patterns of performance as is shown in research with animals and
humans. Intermediate levels of tonic LC activation triggers phasic LC activation in response to the identification of task-relevant stimuli. Phasic activity is consistently associated with high levels of task performance. In increased or decreased tonic activity, the LC is unable to respond phasically to task-relevant stimuli which is associated with poor performance on tasks that require sustained attention. Low levels of tonic activity corresponds to inattentiveness. High levels of tonic activity corresponds to distractibility and task disengagement. What remains unanswered is what determines when the LC should transition between phasic and tonic discharge. To respond adaptively to the environment, the LC has to know when current task utility has fallen below an accepted value. To this end, the LC must have access to information about reward and costs of certain response strategies. Aston-Jones and Cohen (2005) proposed that the OFC and ACC are responsible for driving the mode of activity in the LC. Areas in the frontal, parietal and somatosensory cortex project on the LC (Chandler et al., 2013; Arnsten & Goldman-Rakic, 1984), from which the majority come from the OFC and ACC (Aston-Jones et al. 2002; Rajkowski et al. 2000; Zhu et al. 2004). Of these frontal structures, the OFC is strongly associated with assessing the value of a stimulus and the associated outcome (stimulus-outcome association), as is discussed below. Therefore the OFC seems like an exceptional candidate to signal the LC when task utility exceeds a threshold and alternative response behaviour should be pursued, as proposed by Aston-Jones and Cohen (2005). In the next paragraph we will focus on the function of the OFC in goal-directed behaviour, followed by a discussion of the possibility of the OFC modulating LC activity, as is proposed in the adaptive gain theory.
The OFC in the assessment of stimulus values and associated outcomes
Traditionally, the OFC is regarded as a brain area highly related to the evaluation of reward. Neurons in the monkey OFC have shown to be activated by rewarding stimuli in various modalities, excluding the identification of the stimuli or response preparation (Roesch & Olson, 2004; Rolls, 2004). Furthermore, OFC responses in the monkey brain are sensitive to the proportion of a reward corresponding to a target stimulus (Tremblay & Schultz, 1999; Wallis & Miller, 2003) which has been corroborated in human neuroimaging (Breiter et al., 2001; O’Doherty et al. 2002). Consequently, reward-specific responses weaken as monkeys (Critchley & Rolls 1996, Rolls et al. 1989) and humans (Small et al., 2001) become sated for that particular reward. OFC neurons in monkeys are also sensitive to the anticipation and delivery of reward (Schultz et al. 2000, Hollerman et al. 2000), and evidence suggests that OFC responses are more sensitive to larger delayed rewards, than limbic structures that are more sensitive to immediate rewards (McClure et al., 2004). Collectively these studies indicate that the OFC is highly sensitive to the anticipation, delivery and magnitude of reward. Importantly, recent evidence shows that the sensitivity to reward is a property used in the
assessment of an outcome value when a (learned) response is made towards a specific stimulus, as is discussed below.
Recent evidence converge on a role of the OFC in adaptive responding to unexpected outcomes (Gruber et al., 2010; Hampshire et al., 2012; Boorman et al., 2016; Takahashi et al., 2009). Lesions of orbitofrontal cortex disrupted performance in reversal learning in rats (Chen et al., 2004; Dias et al., 1996; McAlonan and Brown, 2003) and monkeys (Walton et al., 2010), indicating that damage to the OFC impairs the ability to the update learned stimulus-outcome associations when task contingencies change. As part of a frontoparietal network, including the ACC and temporoparietal junction area, OFC activity is related with situations in which a rapid change in goal-directed behaviour is required; like responding to a previously-irrelevant stimulus presented outside of the focus of attention (Gruber et al., 2010). In this study, participants performed in a typical reversal task in which they were instructed to respond to a certain feature of a stimulus (e.g. shape) and ignore other features (e.g. colour), while BOLD activity was measured. During the experiment, the instructions could change, pushing participants to change their representation of the stimulus-outcome association when reward contingencies changed (Gruber et al., 2010). Furthermore, measurements of BOLD activity in another study with a reversal task, shows that specifically activity in the lateral and medial OFC activated in response to reward contingency changes (Hampshire et al., 2012). Indicating that when a specific response to a stimulus did not have the expected outcome, the medial and lateral OFC activated, signalling for a needed change in response strategy. Likewise, feedback responses in lateral orbitofrontal cortex (lOFC) reflected a similar updating term important for revising associations between stimulus identities and expected outcomes (Boorman et al., 2016). In this study the participants were engaged in a two alternative forces choice task in which they had to pair two stimuli in order to get a predefined monetary reward. The results showed that the lOFC signalled for a needed change in stimulus pair updates in order to get the highest reward. Interestingly, lOFC activity was independent of the magnitude of the associated reward, indicating that the lOFC does not respond to reward outcomes, but only responds when a change needs to be made in response strategy. Boorman et al. (2016) proposed that the lOFC does not store learned associations, but merely signals the need for a change in response strategy, which in turn is relayed to the hippocampus where learned associations are possibly stored. Similarly, the signal from the OFC indicative of a needed change in stimulus-outcome associations is related to signalling of prediction errors by dopamine neurons in the ventral tegmental area (VTA; Takahashi et al., 2009). Specifically, as the signal of the OFC rose due to an unexpectedly low outcome, signals in VTA rose and vice versa. Takahashi et al. (2009) proposed that the OFC gives the signal for an unexpected outcome, which could contribute to the generation of a prediction error signal in the VTA. Like Boorman et al. (2016),
Takahashi et al. (2009) proposed that the OFC does not store stimulus-outcome associations, but signals for a need of change in learned associations which is then processed elsewhere.
In contrast, and more in line with earlier work on the OFC in relation to the evaluation of reward (Roesch & Olson, 2004; Rolls, 2004), the human OFC has shown to encode for rewards in the form of identity specific value codes (Howard et al., 2015). Appetizing food odors and pattern-based functional magnetic resonance imaging (fMRI) was used to demonstrate that each reward (e.g. a strawberry or a pizza) has a unique spatial activation pattern in the OFC. Furthermore, the medial part of the OFC was involved in value comparison, by assigning a quantity that reflects how much the value difference was between two choice options (Fitzgerald et al., 2009). The two latter studies indicate that the OFC not only stores specific information regarding the reward related with stimulus-outcome associations, but can also do basic computations regarding the actual value of a stimulus compared to other stimuli. Furthermore, a dissociation had been found between neurons in the monkey OFC representing upcoming reward and neurons representing preceding reward information (Simmons et al., 2008). The proportion of neurons coding for the preceding reward info decreased over the course of a new trial, whereas the opposite was found for the neurons coding for the upcoming reward. The latter increased over the course of a new trial. Interestingly, the specific neurons encoding the identity of a preceding or upcoming reward generally changed when the task context changed, supporting the finding that each reward identity has its own specific neural code. In addition, these results indicate that ‘old’ stimulus-outcome associations are not immediately forgotten and might remain stored in the OFC (Schoenbaum et al., 2008).
In summary, the OFC has been strongly associated with the evaluation of reward (Roesch & Olson, 2004; Rolls, 2004; Tremblay & Schultz, 1999; Wallis & Miller, 2003; Breiter et al., 2001; O’Doherty et al. 2002). Although contradictory evidence has been published, the majority of research (Howard et al., 2015; Simmons et al., 2008; Schoenbaum et al., 2008) shows that the OFC can store a specific representation of a reward and can perform basic computations to assess a difference in value between stimuli (Fitzgerald et al., 2009). Furthermore, the OFC is strongly related to reversal learning (Gruber et al., 2010; Chen et al., 2004; Dias et al., 1996; McAlonan and Brown, 2003; Walton et al., 2010), activating when an expected outcome is too low and a change in stimulus-outcome associations is needed (Boorman et al., 2016; Hampshire et al., 2012). Taken together, these studies indicate that the OFC has a representation of reward values associated with stimuli, which can be compared in order to pursue the largest reward. When an outcome is unexpectedly low, the associated stimulus possibly loses value and a signal for a chance in stimulus-outcome associations is generated. In the next paragraph we will discuss the function of the OFC in relation to the LC, and
discuss the possibility of the OFC regulating the mode of LC discharge, in relation to goal-directed behaviour.
Does the orbitofrontal cortex regulate the mode of activity of the locus coeruleus?
The adaptive gain theory suggests that the OFC (and ACC) regulate the mode of activity in the LC and thereby regulate the balance between exploitation and exploration: “When evaluations in OFC or ACC indicate the current task is providing adequate utility, they drive LC toward the phasic mode … favouring exploitation of that task for associated rewards. However, when utility diminishes sufficiently over prolonged durations, they drive it toward the tonic mode, favouring exploration.” (Aston-Jones & Cohen, 2005, p. 428). The authors base their assumption on findings that the LC receives input from the OFC (Aston-Jones et al. 2002; Rajkowski et al. 2000; Zhu et al. 2004). In turn the OFC receives input from a wide array of sensorimotor areas (Baylis et al., 1995; Carmichael & Price 1995), which makes the OFC a good candidate to modulate cortical responses to incoming stimuli by modulating the LC. However, other cortical areas also project onto the LC (Chandler et al., 2013), including the dorsolateral prefrontal cortex, the dorsomedial prefrontal cortex, the parietal association cortex, the somatosensory cortex, the anterior portion of the inferior temporal gyrus, and the posterior portion of the inferior temporal gyrus (Arnsten & Goldman-Rakic, 1984). The dorsolateral prefrontal cortex is often associated with working memory (e.g. Bogdanov & Schwabe, 2016; Schon et al., 2013) and the medial prefrontal cortex has been associated with attention-shifts (Tait et al., 2007). Both of these cognitive functions have been associated with goal-directed behaviour (e.g. Asplund et al., 2010; Corbetta et al., 2002), theoretically making them candidates to regulate the mode of LC activity. The possibility is that all the brain areas projecting on the LC, modulate the LC to some extent and together regulate the mode of activity. However, no study up to this point has shown evidence that either of these brain areas known to project onto the LC, directly regulates the mode of activity of the LC.
The second argument with which Aston-Jones and Cohen (2005) support their assumption of the OFC as a regulator of the mode of activity in the LC, is that the pattern of task-related activity commonly observed in the OFC and LC are highly related to each other. As was previously discussed, the OFC activates when a reward is unexpectedly low and a change in response behaviour is necessary (Gruber et al., 2010; Hampshire et al., 2012; Boorman et al., 2016; Takahashi et al., 2009 ). In turn the phasic LC response is specific for task-relevant stimuli (not distractor stimuli) with an amplitude modulated by the magnitude of the reward associated with responding to the right stimulus (Aston-Jones et al., 1994, Rajkowski et al., 2004). Both areas appear to respond to the motivational significance of an event, optimizing behaviour in response to the environment.
Theoretically, this suggest that the OFC and LC are part of the same functional network. However, that the OFC and LC share a behavioural function does not necessarily mean that one of the brain areas drives the other. Interestingly, there is a latency difference between activity in the LC and OFC in response to a cue associated with a reward: in two different studies it was shown that r eward related signals in the monkey brain appeared much earlier in the OFC (~60 ms; Bouret et al., 2010) than in the LC (~150 ms; Bouret et al., 2015). This latency difference indicates that the representation of a reward signal might be processed in the OFC first and then continues to the LC. Both studies wanted to assess the factors for controlling the perceived motivational value of task events by making extracellular recordings in the brain areas of interest (LC or OFC). In both studies the designated monkeys were required to respond to a visual task-relevant stimulus by releasing or pressing a lever. Importantly, in both experiments the brain area of interest responded to the task-relevant stimuli and activation increased with the magnitude of the expected reward. Although this is not direct evidence that the OFC regulates the mode of activity in the LC, these studies (Bouret et al., 2010; Bouret et al., 2015) suggest that goal-related responses are first processed in the OFC after which they arrive in the LC. Knowledge of this difference in latency provides support for the assumption that the OFC and LC are in the same functional network since they respond to the same stimuli in very similar task designs. Importantly, these two studies reveal that there might be a potential causal link between the OFC and LC in the context of goal-directed behaviour. Indicating that the two brain areas might be part of the same functional network in which the OFC evaluates the value of a stimulus and continues the signal towards the LC, which then optimizes a response (engaging or disengaging) to a following stimulus. This does not mean that the OFC actively regulates the mode of activity in the LC, only that both brain areas contribute to goal-directed behaviour in a consecutive fashion.
In contrast, Payzan-LeNestour et al. (2013) could not find simultaneous activity in the OFC and LC in response to unexpected uncertainty, while explicitly looking for it. The authors motivated their research on the predictions of the adaptive gain theory which states that descending projections from the prefrontal regions mediate the influence of task-related information on the activity on the LC. The authors defined unexpected uncertainty as the magnitude of surprise given a learned stimulus-outcome association when the learned unreliability is accounted for. In other words, unexpected uncertainty was defined as the unaccounted value - or error - in a learned stimulus-outcome association. Unexpected uncertainty as a signal of contingency change can therefore be used to flexibly check if an update in stimulus-outcome associations needs to be made, as a form of novelty detection (Payzan-LeNestour et al., 2013). Their definition of unexpected uncertainty is consistent with the previously mentioned role of the OFC: signalling for a change in learned associations when a reward is unexpectedly low (Gruber et al., 2010; Hampshire et al., 2012;
Boorman et al., 2016; Takahashi et al., 2009). BOLD activity was measured while participants performed in a restless bandit task (introduced in Payzan-LeNestour & Bossaerts, 2011) in which the participants continuously had to make a choice between three stimuli. Each stimulus contained a reward or a risk of penalty equal to the attainable reward. The outcome probability was different per stimulus, and changed regularly, both of which the participants were not notified. Importantly, the authors could not find any activation related to unexpected uncertainty in any of the brain areas known to project on the LC (regions of interest were the ACC, dorsomedial and dorsolateral prefrontal cortex, and OFC; Arnsten and Gold-man-Rakic, 1984; Aston-Jones et al., 2002; Jodo et al., 1998). This lack of activation in the OFC is quite remarkable evidence against the proposal that he OFC regulates the mode of activity of the LC, since the OFC did not activate in response to a needed change in update, and the LC did. However, it is also quite remarkable that the OFC did not activate at all. A vast amount of literature supports the activation of the OFC in the evaluation of reward (Roesch & Olson, 2004; Rolls, 2004) and the updating of learned associations (Gruber et al., 2010; Hampshire et al., 2012; Boorman, 2016; Takahashi et al., 2009). Therefore, it is unusual that no activity was found in the OFC in response to unexpected uncertainty. Conversely, a reason for a lack of activation in the OFC to unexpected uncertainty, might be caused by cortical modulation by the LC, disabling the OFC to generate a significant response to be measured as BOLD activity. Evidence indicates that the LC modulates cortical activity by altering the membrane potential of neurons (Safaai et al., 2015; Berrige et al., 1991; McGinley et al., 2015). Furthermore, since the LC has direct projections on the OFC (Agster et al., 2013; Chandler et al., 2013, 2014, 2016; Schwarz et al., 2015), it is theoretically possible that activity of the OFC was modulated by the LC. High tonic activity might have depolarized membrane potentials of OFC neurons, thereby increasing spontaneous activity but decreasing the possibility of potentiating a response to unexpected uncertainty (McGinley et al., 2015). As the LC was activated in response to unexpected uncertainty, and the OFC was not (Payzan-LeNestour et al., 2013), the LC might have inhibited activity in the OFC, indicating that the OFC does not directly regulate the mode of activity of the LC, but the other way around. Taken together, there is an indication of a consecutive order in which task-relevant activity is first processed in the OFC, and later in the LC. However, a lack of activation in the OFC - while the LC was activated – during unexpected uncertainty indicates that the OFC does not regulate the mode of activity of the LC in goal-directed behaviour.
Summary
The adaptive gain theory (Aston-Jones & Cohen, 2005) is a very elaborate model that does pretty accurate predictions concerning the mode of LC activity and optimizing behavioural performance. The LC response is highly flexible with two characteristic modes of activity,
corresponding to different patterns of performance. Intermediate levels of tonic LC activation triggers phasic LC activation in response to the identification of task-relevant stimulus. Phasic activity is consistently associated with high levels of task performance. In increased or decreased tonic activity, the LC is unable to respond phasically to task-relevant stimuli which is associated with poor performance on tasks that require sustained attention. Low levels of tonic LC activity corresponds to inattentiveness. High levels of tonic activity corresponds to increases in distractibility and task disengagement. Aston-Jones and Cohen (2005) proposed that the OFC (and ACC) directly regulate the LC mode activity. The OFC is associated with updating stimulus-outcomes associations in response to contingency changes. Furthermore, there is an indication of a consecutive order in which task-relevant activity proceeds through the brain. On a functional and behavioural level, it seems possible that the OFC regulates the mode of activity in the LC, since both the OFC and LC are associated with the same kind of goal-directed behaviour and are part of the same functional network. However, up to this point in time, no study has shown evidence that the OFC directly regulates the mode of activity in the LC. Conversely, projections from the LC to the OFC suggest a modulatory role of the LC on the OFC, which will be discussed in the next paragraph.
A role for norepinephrine in the OFC in facilitating flexible behaviour
The Sadacca model: proposing a modulatory efect of norepinephrine in the orbitofrontal cortex
Indicative of the modulatory influence of NE in the cortex is provided by sleep research with rats (Enschenko et al., 2011). LC neurons increased cortical susceptibility to slow wave oscillations by providing NE modulatory input. Furthermore, a coupling between the LC and the cortex shows that phasic LC bursts can lead to enhanced cortical responses to weaker stimuli and increased temporal precision of cortical stimulus-evoked responses (Safaai et al., 2015). In addition, changes in activity in the rat LC precedes changes in activity in the barrel cortex (Fazlali et al., 2016). Indicative of a causal link from the LC to the barrel- and potentially other parts - of the rat cortex. This causal link was earlier established between the LC and the forebrain in rats (Berrige et al., 1991). In the latter study, LC activation was induced by infusing the cholinergic agonist bethanechol in the LC, causing a shift from low-frequency high amplitude to high-frequency to low amplitude activity in the rat forebrain. Generally, high frequency activity is associated with increased cortical activity (McGinley et al., 2015), indicating that activating the LC potentially caused an increase in cortical activity in the forebrain. Intracellular recordings in awake behaving rodents revealed a U-shaped pattern, representing dependence of cortical membrane potential on arousal (McGinley et al., 2015). Low arousal was associated with an average hyperpolarization of neural membrane potential. Conversely, increases in arousal were associated with an average depolarization of cortical membrane potentials, which were
potentially caused by an increase of NE (McCormick et al., 1992; devilbiss et al., 2011). In their review, McGinley et al. (2015) carefully point towards a link between LC activity, arousal and cortical activity. Taken together, these studies show that activating the LC results in facilitation of cortical activity. The question here is whether the modulatory effect of LC on cortical membrane potentials is also of effect in the OFC, with regard to goal-directed behaviour. Evidence shows that the OFC signals for an update in learned stimulus-outcome associations when task contingencies change. It might be the case that LC activity can modulate membrane potentials in the OFC, inhibiting or pushing the OFC to signal for a change in stimulus-outcome associations.
Recently a model (Sadacca et al., 2016) was published describing how the LC might modulate activity of the OFC through tonic and phasic discharge. In this model (from now on referred to as the Sadacca model) the OFC is referred to as a brain region associated with storing learned relations between stimulus-outcome associations. This is in line with earlier discussed research illustrating that the OFC is involved in updating stimulus-outcome associations in response to contingency changes in reward (Gruber et al., 2010; Hampshire et al., 2012; Boorman, 2016; Takahashi et al., 2009). Importantly, the Sadacca model predicts that the LC allows for exploiting and exploring behaviour by managing the active representation of a learned stimulus-outcome association. Consistent with the adaptive gain theory, Sadacca et al. (2016) propose that the LC optimizes behaviour through the two modes of activity: tonic and phasic. Likewise, they propose that phasic activity is associated with facilitating activity in response to task-relevant stimuli. However, Sadacca et al., (2016) propose that not the OFC regulates activity in the NE, but the other way around. The Sadacca model states that when there are mismatches of expected task contingencies (e.g. between a choice and the outcome of that choice) tonic NE levels rise, signalling for a change of behaviour to cope with a changing environment. Sadacca et al. (2016) propose that this change-signal from the LC would be ideal for updating active representations of learned stimulus-outcome associations in the OFC. As was stated earlier, the OFC receives a wide range of afferent NE projections from the LC (Agster et al., 2013; Chandler et al., 2013, 2014 ,2016; Schwarz et al., 2015). Therefore, Sadacca et al. (2016) propose that rising levels of tonic NE in the OFC serve as an association create signal, driving the OFC to store new stimulus-outcome associations in order to respond to a reward contingency change. This is supported by research showing that increases in global NE augment reversal (Seu et al., 2009) and extinction learning (Merlo & Izquiierdo, 1967; Janak & Corbit, 2011), while decreases in NE in the cortex impair extinction (Mason and Fibiger, 1979; McCormick & Thompson, 1982; Mueller & Cahill, 2010). Sadacca et al., (2016) predict that NE levels directly modulate the stability of learned associations, thereby affecting the flexibility of response behaviour. The authors propose that a detection of change in task contingencies leads to a rise of tonic NE, which leads to a setting aside of previously learned stimulus-outcome associations. Consequently, behaviour approaches chance and OFC activity corresponds less
well with the initial learned stimulus-outcome association. As trial and error behaviour leads to the acquisition of a newly learned stimulus-outcome association levels of NE subsequently drop, as confidence in the task contingency rises, which allows stabilization of the new association in the OFC. Sadacca et al. (2015) further predict that holding NE levels artificially low in the OFC, should lead to the inability to learn new associations, since NE levels are not able to rise to increase flexibility. Vice versa, increasing NE levels in the OFC should lead to more instable associations, forcing the OFC to cycle through different associations, unable to maintain one. Sadacca et al. (2016) argue for a moderate level of NE in the OFC to allow for flexible behaviour in order to adapt to a changing environment. In accordance with the intracellular recordings demonstrating a relationship between cortical activity, arousal and LC activity (McGinley et al., 2015), increased NE in the OFC should increase OFC activity leading to increased reversal learning. Conversely, a decrease in NE in the OFC should decrease cortical activity in the OFC and lead to an absent of the OFC signal to change the stimulus-outcome association (see Figure 2).
Figure 2. Inverted-U relationship between extracellular NE levels in the OFC and performance on tasks requiring adaptive switching between response strategies. Performance is poor at very low levels of NE, as the OFC is inhibited to learn new associations between stimuli values and outcomes (left side of the spectrum). Performance is optimal with moderate levels of NE in the OFC, as the OFC can flexibly maintain learned - or change to new - stimulus-outcome associations (green surface area). With higher NE levels in the OFC, performance is poor, because the OFC is forced to cycle through diferent stimulus-outcome associations, leading to more random behaviour (right side of the spectrum).
Clinical research in support of the modulatory role of NE in the OFC
Research on attention-deficit hyperactivity disorder (ADHD) supports the modulatory role of NE in the OFC in relation to goal-directed behaviour. The main characteristics of ADHD include the inability to sustain attention, impulsivity, and hyperactivity (Barkley 1997; Maedgen & Carlson, 2000). Importantly, ADHD pathology is strongly connected to deficits in noradrenergic neurotransmission and the efficacy of ADHD therapeutics is associated with increasing noradrenergic transmission (Levy, 2009). Specifically, increasing extracellular NE activates alpha2 NE receptors and decreases the concentrations of the intracellular second messenger, cyclic adenosine monophosphate (cAMP), consequently enhancing the strength and duration of firing of pyramidal neurons in the prefrontal cortex (Wang et al., 2007). Enhanced pyramidal neuronal firing is hypothesized to be the mechanism by which ADHD medications improve attention, working memory and impulsivity (Sagvolden, 2006; Wang et al., 2007). Importantly, the mode of LC activity has been implicated in attentional deficits characteristic of psychiatric disorders (Howell et al., 2012): ‘Hypoaroused’ individuals (e.g. diagnosed with an ADHD) have decreased tonic firing of the LC, resulting in decreased cortical arousal and poor attentional performance. The decrease in attentional performance might be caused by the lower levels of extracellular NE in the OFC not enabling the OFC to flexibly switch between stimulus-outcome associations when task contingencies change.
Research with rats shows that attention deficit might be caused by increased activity of the NE transporter (NET) in the OFC (Somkuwar et al., 2015). In vivo voltammetry was used to study NET function in the prefrontal cortex in spontaneously hypertensive rats (SHR) and normal rats. The SHR is a well-established model of ADHD that exhibits several of the behavioural and neurochemical features of ADHD (de Villiers et al., 1995; Oades et al., 2005; Sagvolden et al., 2005). Somkuwar et al. (2015) showed that the NE uptake rate in OFC was greater in SHR than in normal rats. Importantly, chronic treatment with methylphenidate (MPH; a norepinephrine reuptake inhibitor) reduced the deficit of decreased extracellular NE in the OFC in the SHR, compared to normal rats, by inhibiting the clearance of extracellular NE. The authors argued that the attentional deficit in ADHD might be caused by increased NET functioning. Accordingly, NE depletion in the rat OFC (and surrounding areas) caused a drop in performance on a test requiring sustained attention, when the attentional load was increased (Milstein et al., 2007). In this study healthy rats performed in a five-choice serial reaction time (5CSRT) task measuring sustained attention, an analogue of the continuous performance test in humans (Carli et al., 1983; Robbins et al., 2002). Originally introduced by Carli et al. in 1983, the 5CSRT task requires animals to be trained to respond in one of five response holes when a stimulus light located therein is briefly illuminated. A correct response – nose poke into the illuminated hole – is rewarded by delivery of a sugar pellet, whereas a response in any other hole or
failure to respond is scored as incorrect or as an omission. Premature responses made before the stimulus light is illuminated provides an index of impulsivity (Carli et al., 1983). The cortical NE depletion caused a drop in speed and accuracy when stimuli were presented at a high or unpredictable rate compared to controls (Milstein et al., 2007). Demonstrating that a loss of NE in the OFC (and surrounding areas) results in a loss of performance when the task becomes more unpredictable and requires adaptation. According to the Sadacca model, a change in task contingency would normally increase NE levels, signalling the OFC to store new stimulus-outcome associations (e.g. react faster) in order to adapt to the variably stimulus presentations in the 5CSRT task. However, when cortical NE is artificially depleted the OFC receives no signal to change stimulus-outcome associations and adaptability to the task is inhibited, possibly leading to reduced task performance.
In support of the Sadacca model, chronic treatment of healthy rats with atomoxetine (ATX; a norepinephrine reuptake inhibitor) demonstrates that an increase from optimum levels of NE in the OFC reduces efficiency in performance on a task requiring sustained attention (see Figure 2; Sun et al., 2012). After chronic treatment, rats performed in a typical 5CSRT task (as described above). Compared to saline-treated rats, the ATX-treated rats showed no difference in performance on the 5CSRT task. Importantly, when all the rats received an acute administration of ATX just before the task, the ATX-treated rats performed more efficiently than the saline-treated rats. Omission rate and premature responding was lower for the ATX-treated rats than for the saline-treated rats when acute ATX was administered before the task. It might be the case that the ATX-treated rats habituated to the increased levels of extracellular NE in the OFC during chronic treatment. Consequently, the acute administration of ATX just before the test was not out of the ordinary and was in line with their chronic treatment and therefore their performance did not drop. On the other hand, the saline-treated rats were not habituated to higher levels of NE in the OFC. It is possible that the optimal level of NE to perform efficient was thrown off balance slightly hyper arousing the OFC, in the saline-treated rats. Interestingly, the authors found an increase in NET mRNA in the OFC in the ATX-saline-treated rats compared to the saline-treated rats, which might be an indication of habituation in order to clear the high levels of extracellular NE - caused by the chronic ATX treatment - more efficiently (Sun et al., 2012). According to the Sadacca model, the increased levels of NE in the OFC of the saline-treated rats caused an increased signalling to change stimulus-outcome associations, in spite of no changes in task contingencies. Consequently, the OFC might have been signalled to cycle through different stimulus-outcome associations, making response behaviour more random and inefficient for the saline-treated rats compared to rats who were more habituated to higher levels of NE in the OFC.
The detrimental effect of increased NE levels from optimum in the OFC is supported in studies with Yohimbine administration (Sun et al., 2010; Bremner et al., 1997). Yohimbine increases
NE release through blockade of inhibitory auto receptors on noradrenergic neurons (Szemeredi et al., 1991), and administration can lead to a mildly anxious state (Stine et al., 2002). In their study, Sun et al. (2010) administered Yohimbine by intraperitoneal injection after which the rats were required to perform in a typical 5CRST task (described above). The administered Yohimbine increased extracellular NE levels in the OFC which was associated with an increase in premature responding. Theoretically, the high levels of NE in the OFC disabled the OFC to maintain one stimulus-outcome association, making response behaviour more random and inefficient, potentially causing premature responding. Accordingly, positron emission tomography (PET) scans in patients with post-traumatic stress disorder (PTSD) support the modulatory role of NE in the OFC (Bremner et al., 1997, reviewed in Southwick et al., 1999). NE plays a key role in the development and treatment of PTSD (Brennan et al., 2008; Straw et al., 2008). Dysfunction of the OFC during NE stimulation may be relevant to the failure of inhibition of intrusive thoughts that are characteristic for PTSD (Southwick et a., 1999). Intrusive thoughts have been highly related to impulsivity, in that lack of perseverance and a tendency to act rashly predicted difficulties in thought control (Gay et al., 2011). This indicates that PTSD patients experiencing intrusive thoughts, also have difficulties in impulse control (Roberts et al, 2009), in which the latter has been related to high levels of NE in the OFC in healthy rats (Sun et al., 2010). In the PET study of Bremner et al. (1997), increased levels of NE caused by Yohimbine administration caused panic attacks and flashbacks in PTSD patients. PET scans revealed that the increased levels of NE caused a decreased activation of the prefrontal cortex (PFC) in the PTSD patients, and an increase of the PFC in healthy subjects. Interestingly, the greatest magnitude of difference between the PTSD patients and the control subjects was in the OFC. The decrease in OFC activity in the PTSD patients, is most likely caused by increased sensitivity of alpha2 NE receptors to NE, inhibiting cortical activity (Southwick et a., 1999). In their review Southwick et al. (1999) argued that inhibiting the OFC resulted in panic and anxiety because intrusive memories would return. The model of Sadacca et al (2016) states that the modulation of the OFC by NE is responsible for the switching of stimulus-outcome associations. Meaning that if during a task one strategy no longer produces reward, the OFC switches to another strategy by modulation of NE. In the research of Bremner et al. (1997), the PTSD patient might not have been able to switch to a less anxiety inducing state because the OFC was inhibited and therefore could not flexibly switch to and maintain a proper stimulus-outcome association to reduce incoming of intrusive memories. Interestingly, in the healthy control subjects the increased NE levels caused an increase in OFC activity (Bremner et al., 1997, reviewed in Southwick et al., 1999). Reports have been made that acute Yohimbine administration increases impulsivity in humans (Swan et al., 2005) and in rats through increased levels of NE in the OFC (Sun et al., 2010, described above). Although speculative, it is plausible that the healthy subjects
tested in the study of Bremner et al. (1997) might have experienced increased impulsive responding if they were tested on a task requiring sustained attention.
Summary
Extracellular NE in cortical areas has been linked to increased facilitation of cortical responses to stimuli (Safaai et al., 2015), demonstrating that NE has a modulatory role in regulating cortical activity (McGinley et al., 2015). Higher levels of NE have been linked to increased cortical membrane potential, and vice versa (McGinley et al., 2015). In a recently published model Sadacca et al. (2016) propose that the release of NE in the OFC, through tonic LC activity, could modulate the OFC following these principles of elevated cortical membrane potentials. According to this model, a low level of NE should reduce activity of the OFC, disabling it to signal for an update in stimulus-outcome associations, causing an inhibition of flexible behaviour. Reversely, high levels of NE should increase the activity in the OFC, repeatedly forcing a signal for an update, not allowing a stimulus-outcome association to facilitate, causing random and explorative behaviour. Sadacca et al. (2016) argue that a moderate level of NE in the OFC is required to flexibly maintain and switch between stimulus-outcome associations, in order to functionally adapt to the environment. Research on ADHD and PTSD support the modulatory role of extracellular NE in the OFC with regards to goal-oriented behaviour. Decreased levels of extracellular NE (below optimum) in the OFC cause a drop in performance when a task requiring sustained attention becomes unpredictable (Milstein et al., 2007). Indicating that low levels of NE are not sufficient for the OFC to react flexible when task contingencies change, causing an inability to sustain the speed and accuracy when task load is increased. Furthermore, increased levels of NE (above optimum) in the OFC also decrease performance by increasing omission rate and premature responding (Sun et al., 2012; Sun et al., 2010). According to the Sadacca model, the increased levels of extracellular NE cause an over excitation of the OFC generating increased signalling for a change in stimulus-outcome associations, when only a moderate level of excitation is needed. Consequently, leading to more random behavioural patterns as is reflected in more omissions and premature responding (Sun et al., 2012; Sun et al., 2010). Likewise, panic attacks and flashbacks were induced in PTSD patients by increasing extracellular NE levels (Bremner et al., 1997, reviewed in Southwick et al., 1999), potentially in the OFC (Sun et al., 2010). Increased sensitivity of alpha2 NE receptors might have caused the OFC to be inhibited by the increased levels of NE (Southwick et al., 1999) resulting in panic and anxiety because intrusive memories would return. According to the Sadacca model the PTSD patient might not have been able to switch to a less anxiety inducing state because the OFC was inhibited and therefore could not flexibly switch to - and maintain - a proper stimulus-outcome association to reduce intrusive memories. Importantly, these findings illustrate that there is an optimal level of NE in the OFC in
order to optimally balance the trade-off between exploiting and exploring behaviour. Low levels of NE in the OFC are insufficient to signal for a change in response strategy, when task contingencies change. High levels of NE in the OFC over excite the OFC forcing a continues change in response strategy, when only a moderate level of flexibility is sufficient. Or, in the case of an anxiety disorder, high levels of NE can cause the inability to shield oneself from intrusive thoughts. The Sadacca model states that a moderate level of NE in the OFC is needed to adaptively switch between response strategies. This is in line with the adaptive gain theory, stating that too low or too high levels of tonic LC activity respectively cause reduced task engagement or distractible behaviour, necessitating the need for moderate LC activity to flexibly switch between exploiting and exploring behaviour.
Conclusion
In order to remain flexible and maximize gain from a dynamic environment, organisms must be able to adaptively switch between exploiting and exploring behaviour. Here, we reviewed the current literature on the LC and OFC in order to establish a functional relationship between the two brain areas to better understand the neural mechanism of the exploit/explore trade-of. The LC is known to have two modes of activity: tonic and phasic. Intermediate levels of tonic LC activity allows phasic firing to occur in response to the identification of task-relevant stimuli. Increases in tonic LC activity decreases the ability to respond phasically which is associated with decreased performance on tasks requiring sustained attention. The LC is responsible for the trade-off between exploring and exploiting behaviour, as is supported in research with rats, monkeys and humans. By increasing the signal-to-noise ratio for task-relevant activity, responses towards task-relevant stimulus are selectively facilitated to optimize behaviour. Furthermore, when reward contingencies change the LC is triggered to increase tonic activity leading to withdraw from task engagement, and facilitating behaviour that serves to explore alternative ways to increase task outcome. The adaptive gain theory states that the OFC is one of the main areas to trigger the LC in increasing or decreasing tonic activity of the LC. The OFC appears to process the same task-relevant signal ~90 ms earlier than the LC, indicating that the OFC and LC are part of the same functional network in relation to goal-directed behaviour. However, besides evidence that the OFC has direct projections onto the LC, no study so far has found direct support for a regulatory role of the OFC on the LC in goal-directed behaviour. In contrast, an fMRI study could not find simultaneous activity in the LC and OFC in a reversal task, while explicitly looking for it. On the other hand, projections from the LC to the OFC are suggestive of a modulatory role of the LC in the OFC. Since extracellular NE in the mouse sensory cortex have been linked to increased facilitation of activity to task-relevant stimuli, we questioned whether the LC might have a similar modulatory effect in the OFC. Sadacca et al. (2016) proposed that the release of NE in the OFC can modulate activity in the OFC in order to facilitate or inhibited the OFC to signal for a chance in
stimulus-outcome associations. Clinical studies on ADHD and PTSD converge towards a modulatory role of NE in the OFC, supporting the model of Sadacca et al. (2016). Low levels of NE proved insufficient for the OFC to react flexible when task contingencies changed, causing an inability to sustain the speed and accuracy when task load is increased. Furthermore, high levels of NE in the OFC increased omission rate and premature responding, decreasing optimal performance. Sadacca et al. (2016) argued that a moderate level of NE in the OFC is required to flexibly maintain and switch between stimulus-outcome associations, in order to functionally adapt to the environment. Low levels of NE in the OFC are insufficient to signal for a change in response strategy, when task contingencies change. High levels of NE in the OFC possibly over excite the OFC forcing a continues change in response strategy, when only a moderate level of flexibility is sufficient. Taken together, the LC optimizes task-relevant behaviour by adaptively altering between tonic and phasic firing. Phasic firing releases NE in the OFC modulating local excitability. It is possible that low and high levels of NE respectively inhibit or push the OFC to maintain or switch stimulus-outcome associations, influencing goal-directed behaviour. Taken together, the LC and OFC both play an important role in goal-directed behaviour, in which the LC appears to dictate in their interaction.
References
Amaral, D. G., & Sinnamon, H. M. (1977). The locus coeruleus: neurobiology of a central noradrenergic nucleus. Progress in neurobiology, 9(3), 147-196.
Arnsten, A. F. T., & Goldman-Rakic, P. S. (1984). Selective prefrontal cortical projections to the region of the locus coeruleus and raphe nuclei in the rhesus monkey. Brain research, 306(1-2), 9-18.
Asplund, C. L., Todd, J. J., Snyder, A. P., & Marois, R. (2010). A central role for the lateral prefrontal cortex in goal-directed and stimulus-driven attention. Nature neuroscience, 13(4), 507-512.
Aston-Jones, G., & Bloom, F. E. (1981). Activity of norepinephrine-containing locus coeruleus neurons in behaving rats anticipates fluctuations in the sleep-waking cycle. The Journal of Neuroscience, 1(8), 876-886.
Aston-Jones, G., Chiang, C., & Alexinsky, T. (1991). Discharge of noradrenergic locus coeruleus neurons in behaving rats and monkeys suggests a role in vigilance. Progress in brain research, 88, 501-520.
Aston-Jones, G., & Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci., 28, 403-450.
Aston-Jones, G., Rajkowski, J., & Kubiak, P. (1997). Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience, 80(3), 697-715.
Aston-Jones, G., Rajkowski, J., Kubiak, P., & Alexinsky, T. (1994). Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. The Journal of Neuroscience, 14(7), 4467-4480.
Aston-Jones, G., Rajkowski, J., Lu, W., Zhu, Y., Cohen, J. D., & Morecraft, R. J. (2002). Prominent projections from the orbital prefrontal cortex to the locus coeruleus in monkey. In Soc. Neurosci. Abstr (Vol. 28, pp. 86-89).
Barkley, R. A., hill, j., & Schoener, E. (1997). Advancing age, declining ADHD. Authors' reply. The
American journal of psychiatry, 154(9), 1323-1325.
Baylis, L. L., Rolls, E. T., & Baylis, G. C. (1995). Afferent connections of the caudolateral orbitofrontal cortex taste area of the primate. Neuroscience, 64(3), 801-812.
Beatty, J. (1982). Phasic not tonic pupillary responses vary with auditory vigilance performance. Psychophysiology, 19(2), 167-172.
Berridge, C. W., & Foote, S. L. (1991). Effects of locus coeruleus activation on electroencephalographic activity in neocortex and hippocampus. The Journal of neuroscience, 11(10), 3135-3145.
Berridge, C. W., & Foote, S. L. (1994). Locus coeruleus-induced modulation of forebrain electroencephalographic (EEG) state in halothane-anesthetized rat. Brain research bulletin, 35(5-6), 597-605.
Berridge, C. W., & Waterhouse, B. D. (2003). The locus coeruleus–noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Research Reviews, 42(1), 33-84.
Bogdanov, M., & Schwabe, L. (2016). Transcranial Stimulation of the Dorsolateral Prefrontal Cortex Prevents Stress-Induced Working Memory Deficits. The Journal of Neuroscience, 36(4), 1429-1437.
Boorman, E. D., Rajendran, V. G., O’Reilly, J. X., & Behrens, T. E. (2016). Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus. Neuron, 89(6), 1343-1354.
Bouret, S., & Richmond, B. J. (2015). Sensitivity of locus ceruleus neurons to reward value for goal-directed actions. The Journal of Neuroscience, 35(9), 4005-4014.
Bouret, S., & Richmond, B. J. (2010). Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. The Journal of Neuroscience, 30(25), 8591-8601.
Bouret, S., & Sara, S. J. (2004). Reward expectation, orientation of attention and locus coeruleus‐medial frontal cortex interplay during learning. European Journal of Neuroscience, 20(3), 791-802.
Breiter, H. C., Aharon, I., Kahneman, D., Dale, A., & Shizgal, P. (2001). Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron, 30(2), 619-639.
Bremner, J. D., Innis, R. B., Ng, C. K., Staib, L. H., Salomon, R. M., Bronen, R. A., ... & Zubal, G. (1997). Positron emission tomography measurement of cerebral metabolic correlates of yohimbine administration in combat-related posttraumatic stress disorder. Archives of General Psychiatry, 54(3), 246-254.
Brennan, A. R., & Arnsten, A. F. (2008). Neuronal mechanisms underlying attention deficit hyperactivity disorder. Annals of the New York Academy of Sciences, 1129(1), 236-245.
Carmichael, S. T., & Price, J. L. (1995). Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys. Journal of Comparative Neurology, 363(4), 642-664.
Carli, M., Robbins, T. W., Evenden, J. L., & Everitt, B. J. (1983). Effects of lesions to ascending noradrenergic neurones on performance of a 5-choice serial reaction task in rats; implications for theories of dorsal noradrenergic bundle function based on selective attention and arousal. Behavioural brain research, 9(3), 361-380.
Chandler, D. J. (2016). Evidence for a specialized role of the locus coeruleus noradrenergic system in cortical circuitries and behavioral operations. Brain research, 1641, 197-206.
Chandler, D. J., Gao, W. J., & Waterhouse, B. D. (2014). Heterogeneous organization of the locus coeruleus projections to prefrontal and motor cortices. Proceedings of the National Academy of
