The Role of the Hippocampus and the Frontal Cortex in Encoding Predictions About the World

(1)

The Role of the Hippocampus and the Frontal

Cortex in Encoding Predictions About the

World

Nelson Mooren

1

_{MSc in Brain and Cognitive Sciences, Cognitive Science track}

Supervisor: V.L.E. Perdomo2 Co-assessor: H.A. Slagter2

Project weight: 12 ECTS

2_{Brain & Cognition, Faculty of Social and Behavioural Sciences}

Abstract

Predictive processing is a theoretical framework that posits that the brain is a predictive machine, and has gained prominence in cognitive sci-ence, neuroscience and psychology. While it provides a coherent theoreti-cal framework it is not yet entirely clear how it is implemented in the brain. A selection of neuroscientific literature focusing on the hippocampus and related association areas in the cortex is discussed. Together, these find-ings support the idea that predictions errors are propagated first to the hippocampus, to be consolidated into frontally located long-term mem-ory and thus informing the generative models that issue predictions back to the sensory cortices. Thus, providing a neuroanatomical implemen-tation and integration of memory processes in the predictive processing framework.

(2)

1 The Role of the Hippocampus and Frontal

Cor-tex in Encoding Predictions About the World

Predictive processing is a theoretical framework that has gained prominence in cognitive science, neuroscience and psychology (Clark, 2013; Hohwy, 2013). It posits that the brain is a predictive machine, rather than a bottom-up informa-tion processor. The predicinforma-tions of the brain take the form of a set of hierarchical generative models describing the world in varying levels of abstraction to pre-dict incoming sensory data from the top down. These top-down prepre-dictions are met with sensory information, which leads to prediction errors if the models and their predictions are not wholly correct. The bottom-up prediction errors are communicated back up the hierarchy causing generative model updates. This theoretical framework has its roots in, among others, the work of Hermann von Helmholtz, who considered perception to be a process of active, knowledge-driven, inference (Helmholtz, 1860).

While predictive processing has originally dealt with sensory perception and inference, in recent years it has grown beyond this to encompass a range of cognitive processes such as action, attention and memory (Clark, 2013). Pre-dictive processing is sometimes argued to offer a unified theory of the brain (Hohwy, 2013), stating that ‘brains (...) are essentially prediction machines’ (Clark, 2013, p. 181). While hierarchical predictive processing forms a coherent and well-thought-out theoretical framework, it is not yet entirely clear how it is implemented in the brain.

In this paper, I will attempt to give an account of the implementation of predictive processing in the brain. To do this for the brain and its functions in its entirety is rather a herculean task. Sensory processing as performed by the brain is understood rather well and has been interpreted in predictive process-ing terms to explain, for example, extra-classical receptive field effects in the visual cortex (Rao & Ballard, 1999). However, the role of subcortical structures and higher-order cortical areas in the formation and updating of predictions remains unclear. Because of this, I will discuss literature that pertains to the hippocampus and frontal regions of the brain, and how their role in memory pro-cesses is involved in the generation of predictive models for sensory information processing.

In the following section, I will set out to define the concepts and definitions that are central to the predictive processing framework, and flesh out its mech-anisms in comparison to a bottom-up information processing approach. Next, in section 3, I will discuss a selection of neuroscientific literature pertaining to the neuroanatomical structures in relation to the predictive processing frame-work. Here I will specifically focus on how sensory information is transmitted

(4)

to, processed by the hippocampus to be stored in short-term memory. Next, the process of consolidation from short-term into long-term memory is discussed, as well as task-related communication between the hippocampus and the (frontal) neocortex. Finally, the involvement of the frontal cortex in sensory informa-tion processing is discussed. In the discussion, I will highlight two theories that could complement the findings discussed here, but which also give rise to new questions in order for them to be fully integrated.

2 The Predictive Processing Framework

Predictive processing took form with Hermann von Helmholtz’ work and the idea that perception is a process of active inference (Helmholtz, 1860). von Helmholtz’ ideas gave rise to the cognitive psychological tradition known as analysis by synthesis (see Yuille and Kersten (2006) for a review). This tradi-tion, together with connectionism and work on back-propagation learning that started in the 1970s and 1980s (J. McClelland & Rumelhart, 1986; Rumelhart & McClelland, 1986) stands at the base of how hierarchical generative models are now considered part of predictive processing.

2.1 A Hierarchical Model of the World

Predictive processing posits that a series of hierarchical generative models of the world exist in the brain. Whereas a more ‘traditional’ textbook account of brain function takes a bottom-up view of the brain, such as direct realism, where sensory information processing is driving in informing higher order areas (Gibson, 1979), predictive processing looks the other way. In the predictive processing framework, a higher order generative model generates top-down pre-dictions that inform lower order models; this hierarchy consists of models all the way down1_{(Clark, 2013; Hohwy, 2013). Arriving at the lowest level of}

process-ing, the models make direct predictions of the sensory information they expect to encounter from the outside world. The incoming sensory information gives rise to prediction errors, assuming the predictions are not completely accurate, that are relayed back up the hierarchy, all the while becoming more abstract, causing a sweep of generative model updates to occur. There have been claims of peripheral sensory systems, such as the retina, engaging in predictive pro-cessing (Srinivasan, Laughlin, & Dubs, 1982; Hosoya, Baccus, & Meister, 2005), however, for the intents and purposes of this review, I will consider the lowest level to consist of the primary sensory cortices. In other words, the inputs to

1_{A form of this expression has its roots in a metaphor regarding the infinite regress problem}

(5)

the primary sensory cortices are considered to be the sensory information that informs the hierarchical generative models in the brain.

To illustrate this model inversion, consider visual processing with the goal of object perception as an example, looking at the traditional account first. On the lowest cortical level in the brain, we find the primary visual cortex, which receives input from the retina through the lateral geniculate nucleus. This part of the hierarchy is characterized by neurons with small receptive fields that detect simple features such as light-dark contrast and simple lines (Dumoulin & Wandell, 2007). Moving forward in the brain and higher up the conceptual ladder there is an increase in receptive field sizes, with neurons responding to slightly more complex features such as corners. Following this hierarchy from the bottom up, all the while integrating information from the lower levels, we arrive at a place where an object can be represented, and thus perceived; in essence, we follow the ventral visual stream (Carlson, Simmons, Kriegeskorte, & Slevc, 2014). In this bottom-up view, information predominantly travels from the lower levels of the hierarchy, unimodal sensory cortices, upward to association areas in the cortex forming the higher levels.

From a predictive processing point of view, however, prediction is what drives the machinery described above. Tracing the path from the top back down, a predicted object is expected to consist of a collection of abstract visual features giving rise to a certain set of inputs to this hypothetical object detector. These abstract visual features, in turn, might consist of an ensemble of corners and edges that are predicted by the lower levels, respectively. Ultimately, this leads back to a predicted pattern of sensory inputs coming from the retina through the lateral geniculate nucleus. These sensory inputs then reveal to what extent the lowest level predictions are accurate. This accuracy is then relayed to the higher levels by means of prediction error, based on which the model updates itself and its predictions. Ultimately, this leads to an updated representation of the predicted object and the input that is expected from it. Thus, in this model the information passed through the visual system consists of top-down predictions, which are met with bottom-up prediction errors, together causing the systems to create a model that is as accurate as possible given the available information from the outside world.

Both these models involve memory, albeit in different ways. For the bottom-up models, memory serves to check the incoming sensory information against previously encountered events. On the other hand, for predictive processing, it is thought to play an important role, serving as priors for the generative models in the brain. Both perspectives also make testable claims about the flow of information. According to the bottom-up perspective, such as the direct realism account described above, information flows from the primary sensory

(6)

areas to be integrated into higher-order association areas, with a smaller role for feedback connections traveling downward. On the other hand, the predictive processing framework posits that information predominantly flows from these association areas toward the earlier sensory areas, thus placing more emphasis on ‘feedback’ connections, while the ‘feedforward’ connections provide information in the form of prediction errors.

One important observation here is that this system of predictive processing is less computationally demanding compared to bottom-up information process-ing. Instead of processing input as a completely new whole each time, this can be done within the context of existing representations, thus reducing the compu-tational complexity and required energy consumption (Summerfield & Egner, 2009). In addition, Trappenberg and Hollensen (2013) argue that predictive coding requires sparseness of representation, thus providing an energy-efficient neural code.

2.2 Two Examples of Predictive Visual Processing

In this section, I will make the mechanisms described above more tangible by presenting two studies showing the merit of predictive processing in dealing with sensory information. These two studies look at the visual system on two distinct levels, the former dealing with neurons in the early visual cortex, while the latter is more systems level and deals with object perception. This will also show how some less intuitive findings from neuroscience, such as extra-classical receptive field effects, can be explained by predictive processing.

Rao and Ballard (1999) developed a computational model to describe extra-classical receptive field responses, such as end stopping, of neurons in the early visual cortex. End stopping is an effect where a neuron that is normally re-sponsive to a stimulus, such as an oriented bar, shows a decrease in activity for that same stimulus if it extends beyond its receptive field proper. The model describes the earliest stages of visual processing using a hierarchical network consisting of three levels. Neurons in the lowest level, corresponding roughly to neurons in the primary visual cortex, are considered to be error detectors that signal prediction errors. Each level of the model attempts to predict re-sponses at the next lower level via feedback connections. The error between these predictions and the actual response is then sent to the higher level using feedforward connections.

Rao and Ballard (1999) trained this model using patches of natural scene images, motivated by the idea that the statistics of these natural scenes might largely determine the response properties of visual neurons. As in the primary cortex, the first level of the model is characterized by small receptive fields

(7)

responding to oriented light-dark contrasts. The second level has larger receptive fields and exhibits responses to approximate linear combinations of the first level contrasts.

End stopping was demonstrated for the model neurons in the first level of the model. Importantly, Rao and Ballard (1999) showed that this is a feature of feedback connections from the higher levels. When these feedback connections were removed, the first level neurons responded to the stimulus as would be expected based on their described receptive fields, not showing an end stopping effect.

A few years later, Murray, Kersten, Olshausen, Schrater, and Woods (2002) performed a series of experiments to investigate the role of object perception and higher order areas on activation in the early visual cortex. To this end they presented participants with various types of stimuli, ranging from ran-domly arranged visual elements to visual elements being that were grouped into objects. For example, in one experiment these stimuli consisted of either ran-domly arranged disconnected lines, lines forming a two-dimensional object, or lines forming a three-dimensional object.

Murray et al. (2002) showed that activation in the lateral occipital complex (LOC) was greater for the 3D objects than for 2D objects, and was greater for 2D objects than for randomly arranged lines. Conversely, the primary visual cortex (V1) showed greater activation for randomly arranged lines than for 2D objects, and greater activation for 2D objects than for 3D objects. This activa-tion pattern was also present for structure-from-moactiva-tion type stimuli. Addiactiva-tional experiments where Murray et al. (2002) controlled for stimulus differences indi-cated that these activation patterns were not caused by stimulus differences but rather they were due to perceptual interpretations of the stimuli. The LOC is in-volved in object recognition (Grill-Spector, Kourtzi, & Kanwisher, 2001), while the early visual cortex is sensitive to contrast (Rao & Ballard, 1999). There are a lot of feedback connections from higher order to early sensory cortices that play an important role in perceptual information processing. These findings thus suggest that higher-order visual cortex (LOC) modulates responses in V1 and that there is less need of input from V1 when a region like the LOC has formed a stable percept of an object.

Together these two studies indicate that the visual system performs pre-dictive processing on different levels and that prepre-dictive processing is able to explain unintuitive findings. However, these studies so far only deal with infor-mation processing on a rather small spatial and temporal scale, where top-down modulation is rather automatic. This leads to the question of how predictions and prediction errors are signaled between regions with even greater functional differences? On a longer timescale, representations of information need to be

(8)

maintained in memory and communicated back and forth, how is memory in-volved in these processes?

2.3 Bayesian Inference and Prediction Error Minimisation

Before delving into the questions posed at the end of the previous section and the anatomical implementation underlying it, I want to introduce Bayesian in-ference, as this is central to predictive processing. In this section, I will give a short overview of Bayesian inference and the mathematical, statistical basis it provides for predictive processing. However, an in-depth explanation of how the brain, or neurons, in particular, compute(s) Bayesian inference is beyond the scope of this article. Jakob Hohwy elaborates further on this and on how Bayesian inference relates to the phenomenology of experience (Hohwy, 2013).

Bayesian inference provides a normative way to inform and update genera-tive models based on the available evidence. In the Bayesian sense, these models can be seen as hypotheses and the sensory information as evidence that either confirms or discredits these hypotheses. Bayesian inference derives a posterior probability of a hypothesis taking into account relevant evidence and the inde-pendent, prior, probability of the hypothesis. These priors are considered to be set by prior experiences (Hohwy, 2013), and memory can be an important source of these priors. This relation is summarized in Bayes’ rule as follows:

P (hi|e) =

P (e|hi)P (hi

P (e) (1)

where the posterior probability of a hypothesis (hi) given the evidence (e),

P (hi|e), is the product of the likelihood of the evidence given that the

hy-pothesis is correct, P (e|hi), and the prior probability of the hypothesis, P (hi),

normalised for the independent likelihood of the evidence P (e).

Thus, Bayesian inference provides a normative way for the brain to engage in prediction error minimization. According to the free energy principle proposed by Friston, Kilner, and Harrison (2006), any self-organising system, such as the brain, must minimize its free energy while maintaining an equilibrium with the environment. In this formulation free energy is an information-theoretic quan-tity; in the predictive processing framework, this is the prediction error. Fur-thermore, Friston et al. (2006) show that for predictable stimuli the prediction error is suppressed relative to unpredictable stimuli and that this suppression is mediated by backward connections. Minimization of free energy (prediction er-rors) ensures the adaptive functioning of an agent in relation to its environment, thus improving the biological fitness of the agent. A trivial consequence of pre-diction error minimization is the maximization of mutual information between

(9)

a model and the stimuli it predicts (Hohwy, 2013). For an individual agent, this process gives rise to more accurate, context-sensitive, predictions, which improves the biological fitness of this agent by allowing it to better navigate its environment.

Thus, while memory can serve as a source or predictions and priors, it is not the only source, as the generative models have to start off somewhere. The individual brain is shaped by development during a single lifetime, while evolu-tionary mechanisms shape the brain across generations. Thus, it is thought that the ecological niche and cultural practices in which an organism is embedded are a source of innate priors (Clark, 2013; Hohwy, 2013). However, this paper restricts itself to memory as the source of priors and the basis of generative models, with specific roles for short-term and long-term memory.

3 The Anatomical Organisation Underlying the

Role of Memory in Predictive Processing

This section consists of three parts, which, when taken together, present one main idea, namely that the predictive process happens by means of a neural loop. This loop consists of one part where sensory information is transferred from the sensory cortices to the hippocampus, where it is processed and stored in short-term memory (section 3.1). This transfer of information happens by means of prediction error propagation. Next, these short-term memories are consolidated into long-term declarative memory, where the prediction errors get incorporated into the generative model (section 3.2). Thus, information is moved from the hippocampus and the temporal lobe regions of the prefrontal cortex. It is here that, in the final stage that completes the loop, the generative model issues predictions that are sent back to the sensory cortices (section 3.3). It is important to note that in this loop, two types of model updating and prediction (error) propagation are considered. There is the process as described above, which relies on sleep-related memory consolidation, that incorporates an extended network of brain regions and operates on timescales ranging from min-utes to days. However, this loop is not a one-way street, rather each part consists of both bottom-up and top-down connections and information is exchanged in both directions. That is, as prediction errors are transmitted from the sensory cortex to the hippocampus, there is some degree of automatic model updating and feedback, predictions. The same goes for the transmission that happens between the hippocampus and the frontal cortex, as well as between the frontal cortex and the sensory regions. This makes sure that the information that is transmitted is correct and is received correctly, preventing redundant

(10)

process-ing of information already represented. Additionally, this allows the brain to operate on short timescales and allows the agent to aptly respond to changes in its environment.

Additionally, as Hindy, Ng, and Turk-Browne (2016) describes nicely in the introduction of their paper, there is a distinction to be made between two kinds of expectation, or prediction. Namely, on the one hand there are immediate, perceptual, expectations such as motion anticipation. When moving along a street, these predictions deal with the location of other objects that are within range of our sensory experiences. On the other hand, there are mnemonic expec-tations and predictions, which are more abstract in nature and not necessarily directly related to sensory inputs. To continue the analogy, turning a corner will give rise to perceptual predictions of what the street looks like and what buildings there will be, based on memory and familiarity of what streets gen-erally look like. In addition to this sensory information, there are mnemonic expectations, such as the name of the street and that your friend lives at number 34. The former predictions tend to arise automatically and involve mechanisms as described in section 2.2, while the latter predictions reflect mnemonic and declarative memory and involvement of hippocampus and higher order cortical areas.

While this paper primarily deals with the framework of predictive processing, there is an important role for the Complementary Learning Systems theory of memory (J. L. McClelland, McNaughton, & O’Reilly, 1995). According to this theory, there are complementary learning systems located in the hippocampus and the neocortex, respectively. The hippocampus is involved in rapid learning, with memories being stored here via synaptic changes. In turn, the neocortex learns slowly and its learning is reinstated based on the changes that happen in the hippocampal system. These two systems complement each other and this division assures that learning of new information does not disrupt the structure of long-term memories. In light of this theory, Norman and O’Reilly (2003) developed a computational model of recognition memory, mapped to regions of the medial temporal lobe and the hippocampus. With it, they sprovide a comprehensive account of the involvement of the hippocampus in recognition memory that is in line with complementary learning systems theory.

3.1 The Role of the Hippocampus and Subcortical

Struc-tures on Visual Information Processing

In this section, the role of the hippocampus is described. The hippocampus is connected to a wide network of subcortical and cortical regions, both sensory and associative in nature. The studies described here generally also involve

(11)

findings in these regions and these will be described where necessary, however, the main focus of this section is on the hippocampus. Specifically, how feedback connections from the hippocampus influence the visual cortex, and how short-term episodic memory and statistical learning play a role in issuing predictions. Strange, Duggins, Penny, Dolan, and Friston (2005) conducted an fMRI study where they used a sequential reaction time task to assess perceptual in-ference in light of the predictability of a stimulus sequence. In this task, partic-ipants had to identify a target stimulus, one of four possible colored shapes, by pressing one of four buttons. These stimuli were presented in blocks of 40 trials each, which were independently sampled from a probability distribution that re-mained constant within a block, akin to drawing items from a hat and returning them. This probability distribution was different for each block. Thus, within each block, the predictability of stimuli varied, and this probabilistic structure could be implicitly inferred based on previous trials within the same block.

On a behavioural level, reaction times were shown to be significantly modu-lated by surprise and entropy. Here surprise is unique to each stimulus event and measures their individual improbability. Conversely, entropy measures the av-erage surprise over all events, and thus represents a prediction for an upcoming event; lower entropy corresponding to higher predictability.

Furthermore, Strange et al. (2005) found that entropy, but not surprise, mod-ulated activity in the left anterior hippocampus, specifically increased activation was observed for unpredictable stimulus streams compared to predictable ones. However, surprise was shown to modulate responses in an extensive network involving the bilateral fusiform, parietal, premotor and inferior frontal cortices, as well as the thalamus. That is, the response in the anterior hippocampus is related to sequence learning and this response is determined by the predictabil-ity of events before they occur. In addition, Strange et al. (2005) found that activation in a cortico-thalamic network as well as the subsequent motor re-sponse tended to decrease following more frequent events; this conforms to the idea that predictive processing privdes an efficient encoding for visual stimuli.

These findings show that there is a double dissociation between the regions that encode for entropy and surprise. Hippocampal activity, being predictive, is determined by the predictability of events based on the preceding stimulus sequence, while the cortico-thalamic network responds to the probability of each event, being reactive.

Harrison, Duggins, and Friston (2006) used a similar sequential reaction time task as did Strange et al. (2005), however, an important difference is that Harrison et al. (2006) presented stimuli in the form of first-order Markov se-quences by sampling the trials from a transition matrix. The transition ma-trix encoded the dependence between consecutive trials and remained constant

(12)

within a block, allowing the uncertainty to be varied in a structured way. The predictability of each individual stimulus was quantified by surprise, and each sequence was quantified by entropy. Additionally, mutual information between consecutive stimuli was used to quantify the conditional uncertainty, or the av-erage contingency, between consecutive stimuli in a sequence. Thus, a more predictable sequence is characterized by low entropy and high mutual informa-tion, and a predictable stimulus is characterized by low surprise and high mutual information.

On a behavioural level, Harrison et al. (2006) also showed that reaction times were shorter for more predictable stimuli than for less frequent events, that is, they decreased as mutual information increased and entropy decreased. Reaction times were shown to be sensitive to both the probabilistic attributes of specific events and the probabilistic context in which they occurred. These findings are in line with those of Strange et al. (2005).

Furthermore, Harrison et al. (2006) found that activity in the left hippocam-pus, the left retrosplenial cortex, and the bilateral parieto-occipital sulcus was positively correlated with mutual information, but not with entropy. The retro-splenial cortex is implicated in navigation, suggested to be involved in episodic memory, and connects the parieto-occipital sulcus to the medial temporal lobe and the hippocampus.

The lack of a correlation between the left hippocampus and entropy seems to be in contrast with the findings by Strange et al. (2005). To explain this, Harrison et al. (2006) note that they used a more conservative correction for their imaging analysis, thus suggesting that lack of significance does no imply a lack of effect. However, they also point out that the introduction of variations in mutual information had an effect on conditional entropy, which is the entropy of a stimulus given a previous stimulus. More specifically, Harrison et al. (2006) were able to decompose conditional entropy into an instantaneous component (entropy) and one that models the temporal relationship (mutual information) among consecutive stimuli. Harrison et al. (2006) argues that the hippocampus responds selectively to the conditional entropy of the current stimulus given the preceding one given that it is interested in the temporal structure of sequences. Both Strange et al. (2005) and Harrison et al. (2006) use a rather simple task where a statistical model is presented and learned for each block. These studies tell us something about how a model is being updated over time in the early stages of acquisition and how entropy, surprise, and mutual information play a role in this. To further investigate differences between model acquisition and updating, Schiffer, Ahlheim, Wurm, and Schubotz (2012) presented participants with short video clips featuring every-day actions being carried out at a table, shot from a third-person perspective. All movies in a condition started out

(13)

iden-tically and diverged at some point after which no more commonalities between versions existed. Most movie scripts existed in two version, i.e. an identical start with two different endings (named divergents), and some existed in six different versions, an identical start with six endings (named unpredictable). Participants were exposed to movies from these categories before the fMRI session; during the fMRI session they were also shown new original movies to which they had not yet been exposed. The inclusion of new original stimuli allowed a contrast of a predictable model, where there were few outcomes, with an unpredictable model, with many outcomes, as well as stimuli for which no internal generative model existed yet.

Four regions of interest were segmented from the whole-brain T1 scans: the left and right hippocampus, caudate nucleus, habenula, and substantia nigra. The hippocampus is connected to a wide network of both limbic and cortical areas, among which are these regions of interest. The other three regions have also been implicated in learning processes, in (inhibitory) action control and motor planning, as well as with seeking and processing of reward and feed-back through their interactions with the dopamine system (Nicola, Surmeier, & Malenka, 2000; Schiffer & Schubotz, 2011).

The analysis consisted of two stages. First, the conditions of interest were parametrically modeled. Second, the unpredictable scripts were contrasted with divergents, and the divergents were contrasted with the new original stimuli. Both the hippocampus and substantia nigra showed attenuation to repeated exposure of the new originals. Processing divergent stimuli compared to new originals caused significantly less activation in the substantia nigra. The hip-pocampal response was significantly higher for unpredictable violations of a known model compared to completely novel stimuli. This increase for unpre-dictable violations relates to higher surprise, as was measured by Shannon en-tropy, elicited by prediction errors. Here, surprise refers to the information-theoretic quantity, namely the amount of information conveyed by a stimulus event; an unpredictable violation conveys more information than a predictable one because less of the event is known a priori.

The decrease of hippocampal activation can be interpreted as a response to longer stimulus sequences becoming more predictable, or in other words, with the internal model becoming more accurate and thus making better predictions. This fits in with the increased activation found for unpredictable violations in relation to predictable ones since updating of models based on unpredictable events are characterized by a relatively high prediction error. The stronger response to unpredictable events compared to completely novel events is an effect of high entropy for the mismatch between expected and actually occurring events (unpredictable), whereas there is no solid internal model for the novel events, to

(14)

which no solid probability can be ascribed. A potential counter argument could be that the novel events are ‘processed’ according to existing models for the other categories of events, however, this is not reflected in the results because then there should have been no contrasting effect between unpredictable and novel stimuli.

Thus, Schiffer et al. (2012) relates decrease of hippocampal activity to model acquisition and low-surprise input consisting of long, predictable sequences. The increase in hippocampal response to unpredictable violations, high in surprise and prediction error, yet low in mutual information of a known model, suggests prediction errors are being accounted for at this point and the model is updated. The findings of Harrison et al. (2006), showing that activity in the hippocampus is negatively correlated with prediction error, seem to be at odds with those of Schiffer et al. (2012), who found increased activity for unpredicted (or unpre-dictable) events. However, one important difference between these two studies is that Schiffer et al. (2012) found this relationship based on the violation of a prediction based on previous data, while Harrison et al. (2006) established a novel probabilistic relationship between events. Taking this into account, the findings of Strange et al. (2005); Harrison et al. (2006) should be considered alike to the new original stimuli that Schiffer et al. (2012) presented to partici-pants, because during these blocks participants engage in model acquisition; the new original stimuli have not entered in semantic form yet.

To gain a more detailed view of how the hippocampus influences the visual cortex in light of stimulus sequences and outcomes, Hindy et al. (2016) trained participants to associate a certain cue stimulus and a response (left or right button press) with outcomes in the form of a follow-up stimulus. This full sequence was presented to participants, as well as cue and response only trials, where the outcome was replaced by a blank screen, and outcome only trials, thus ommitting the cue stimulus and button press. A classifier was trained on BOLD activity sequence encoding by presenting it with the full sequences, and one was trained on outcome only trials, outcome encoding ; the latter was tested on cue plus action trials to quantify the amount of evidence elicited by a cue and an action. This latter classifier quantifies the predictive information of the cue and response, and thus serves to operationalise predictive coding.

Regions of interest were anatomically defined as the hippocampal subfields CA2 and CA3, the dentate gyrus, and subfield CA1 and the subiculum. In the visual cortex, Hindy et al. (2016) focused on the primary and secondary visual cortex (V1 and V2, respectively). Hindy et al. (2016) showed that outcome decoding in V1-2 was more accurate on cue + action trials where correct vs. incorrect sequence was decoded in hippocampal CA-DG regions of interest. Av-erage outcome decoding in V1-2 correlated with avAv-erage sequence decoding in

(15)

CA-DG; but not the other way around. Looking at the timing, early hippocam-pal decoding (cue+action trials) was predictive for outcome decoding later in that trial. The asymmetry of this cross-correlation suggests a top-down effect (predictions) consistent with ‘the hippocampus reinstating expected outcome in the visual cortex’ (Hindy et al., 2016, p. 667).

These results show that pattern completion in the hippocampus may pro-vide a mechanism for action-based mnemonic expectation and predictive coding in the visual cortex. In addition, pattern completion could provide a potential mechanism for model acquisition and model updating in the hippocampus. So far, the studies described in this section primarily deal with hippocampal learn-ing of statistical regularities between stimuli, statistical learnlearn-ing. However, the hippocampus is also known to specialize in encoding individual episodes. In a recent paper, Schapiro, Turk-Browne, Botvinick, and Norman (2017) presented a model based on the Complementary Learning System (J. L. McClelland et al., 1995; Norman & O’Reilly, 2003) where they investigate the role of the hippocam-pus in rapid learning, making a clear distinction between episodic encoding and statistical learning.

Schapiro et al. (2017) modeled the hippocampus and temporal lobe, incorpo-rating known properties of the hippocampal subfields CA1, CA3 and the dentate gyrus, as well as the entorhinal cortex, with both input and output layers being modeled. The entorhinal cortex serves as an important relay station between the hippocampus and the neocortex and is involved in signaling both ways, and the goal of the model is to adjust connection weights such that patterns from the input layers of the entorhinal cortex can be duplicated in the output layer of the entorhinal cortex. This model follows from Norman and O’Reilly (2003) and has been successful in describing phenomena of episodic memory. While previous versions of the model only relied on Hebbian learning, the implemen-tation by Schapiro et al. (2017) uses Hebbian learning in combination with an error-driven component. The additions to the learning mechanisms as well as simulations of sequential stimulus presentation allowed Schapiro et al. (2017) to disentangle episodic phenomena and statistical learning.

The dentate gyrus and CA3 were found to encode distinct episodes but not regularities. The trisynaptic pathway connects the entorhinal cortex to CA1 through the dentate gyrus and CA3, with heavy cross-connectivity between the latter two regions, and heavy interconnectivity within CA3. The monosynaptic pathway directly connects the entorhinal cortex to CA1 was shown to support statistical learning. CA1 is specifically shown to support overlapping represen-tations, which is related to its structure, and in that sense it is more cortex-like. Additionally, Schapiro et al. (2017) found different learning rates with the trisy-naptic pathway having a higher learning rate than the monosytrisy-naptic pathway;

(16)

furthermore, these different learning rates are placed in a hierarchy that further involves the medial temporal lobe (which includes the entorhinal cortex) and the neocortex having the slowest learning rate.

Thus, pattern completion serves as a way for the hippocampus to engage in predictive processing and to influence the early visual cortex. This could also present a means for the hippocampus to engage in model acquisition and updat-ing fueled by prediction errors comupdat-ing from the sensory cortices. The hippocam-pus is involved in both statistical learning and the encoding of episodic memory, which is subserved by different anatomical pathways within the hippocampus. These pathways having different learning rates places them in a hierarchy that further extends towards the medial temporal lobe and the frontal cortex.

3.2 Communication Between the Hippocampus and the

Frontal Cortex, and the Role of Memory

Consolida-tion

The hippocampus is involved in fast learning and short-term memory retrieval, however, on a longer timescale there is less involvement of the hippocampus in memory processes. While the previous section dealt primarily with the hippocampus, some of the studies already indicate the involvement of fronto-parietal networks in the predictive processing of sensory stimuli. In this section, I will expand on the involvement of this neocortical network by discussing the different roles the hippocampus and the neocortex have in memory processes, and how these two areas exchange information. The main stage here is for task-related communication and for sleep-task-related memory consolidation, in which memories are transferred from the fast learning hippocampus to frontal neocor-tical networks that operate on larger timescales.

In this section, I will discuss two kinds of studies. First, I will discuss studies that investigated cortico-hippocampal signaling during wakefulness and during specific tasks, followed by studies about memory consolidation, from hippocampus based short-term memory to neocortex-based long-term memory, during slow wave sleep. In the previous section, Schapiro et al. (2017) hinted at differential learning rates for the hippocampus, fast learning, the entorhinal cortex (medial temporal lobe), showing a medium learning rate, followed by the neocortex with a slow learning rate. The entorhinal cortex is one of the main relay stations between the hippocampus and the neocortex; it transmits infor-mation from the hippocampus to the neocortex and vice versa, using differential anatomical substrates. Thus, with the importance of the entorhinal cortex be-ing described so far, as well as its involvement in memory consolidation, I feel it is important to show that the entorhinal cortex also engages in predictive

(17)

processing as well as the replay of recent memories.

The first study, by Hindy and Turk-Browne (2016), bears some similarities to that of Hindy et al. (2016), discussed in the previous section, however with some adaptations and expanded take place over multiple days. The main difference is that cue and outcome stimuli consisted of triads with one cue followed by one of two outcomes, with a predictable and an unpredictable condition. Additionally, the first day consisted of an exploratory training session, where participants were free to choose their own actions, followed on a second day by a training session with instructions and a neuroimaging session. Their findings indicated that the perirhinal and entorhinal cortex came to represent these triads of stimuli, as they were linked by predictive actions. Furthermore, these representations exerted a top-down influence on and were linked with, reduced activity in the visual cortex.

Hindy and Turk-Browne (2016) also describe a dissociation between the predictable and unpredictable conditions, with medial temporal lobe to LOC interactions being evident for the former and not the latter. The suggest a few possible explanations for this finding, such as that the medial temporal lobe might recruit distinct mechanisms for object representation and that these mechanisms might differentially affect the visual cortex. Participants may also have formed different representations based on the differences between the ex-ploratory and the instructed training sessions.

To further indicate the importance of the entorhinal cortex in predictive processing, consider Johnson and Redish (2007) and De Almeida, Idiart, Villav-icencio, and Lisman (2012) who showed that hippocampal place cells and en-torhinal grid cells, respectively, fire in different modes: either coding predictively for events in the near-future and/or coding for recent events and recently tra-versed places. Specifically, these modes were shown to occur for decision points and at trajectories, respectively.

De Almeida et al. (2012) studied the firing of entorhinal grid cells, which are a major source of input to the hippocampus, of rats as they explored an open environment and as they traversed a linear track. These grid cells got their name because they were shown to fire for small fields (vertices) which cover the environment in a grid-like way. These grid cells were found to fire in two distinct modes, with about half of them being in each mode at most times. On the one hand, they were shown to have a predictive mode representing a position ahead of the animal, as the animal was inbound to the vertices of the cells. Repre-senting the area the animal just passed through, outbound, was the short-term memory (STM) mode. These inbound and outbound modes seemed to have distinctive firing patterns, however, they have statistically indistinguishable fre-quency and power.

(18)

Johnson and Redish (2007) performed an electrophysiology study of rats at decision points in multiple-T mazes, of which they recorded hippocampal activity in the CA3 subfield specifically. This region of the hippocampus has been implicated in memory consolidation. CA3 receives inputs from the dentate gyrus, which is thought to contribute to the formation of new episodic memories, and both receives inputs and projects to the entorhinal cortex (Schapiro et al., 2017). The main findings are that the pyramidal cells in CA3 fired at specific points in hippocampal oscillatory cycles, encoding a sweep of spatial represen-tation, from back to front. This sweep was linked to the prediction of upcoming areas to traverse and decisions to be made. Similar phenomena occurred at re-covery from error, here specifically local field potentials in these sites contained pronounced theta and gamma frequencies. These results are consistent with the findings that bottom-up prediction errors are communicated in high frequencies (the gamma band) whereas top-down predictions are mediated by slower, beta band, frequencies (Bastos et al., 2015).

Hindy and Turk-Browne (2016) showed how the medial temporal lobe ex-erts a top-down influence on the visual cortex. This influence might be carried by neural oscillations, as they have been shown to play an important role in top-down processing (Engel, Fries, & Singer, 2001; Pennartz, Ito, Verschure, Battaglia, & Robbins, 2011). Additionally, the subfields of the hippocampus described in these studies have been implicated in memory consolidation and communication with the neocortex. Pennartz et al. (2011) also suggests the ventral striatum, which receives inputs from the entorhinal cortex, is part of a larger system that integrates inputs from, among others, the prefrontal cortex and the hippocampus. Now that a number of areas involved in memory con-solidation have been covered, it is time to discuss the process of concon-solidation itself.

Siapas and Wilson (1998) were one of the first to show the occurrence of co-ordinated interactions between hippocampal ripples and cortical spindles during slow-wave sleep. Slow-wave sleep is characterized by low-frequency oscillations (1-4 Hz, delta band) originating in the neocortex, from which it gets its name, with recurring moments of field potentials at 7-14 Hz, called spindles. On the other hand, the hippocampus produces very short bursts of ultra-fast oscilla-tions (100 Hz and above), called ripples. While these phenomena have been observed individually, the finding of temporal occurrence of these features is of interest. Siapas and Wilson (1998) suggest their findings may be functionally relevant in memory consolidation and reorganizing neocortical memory traces. Furthermore, Cox, Hofman, and Talamini (2012) showed that involvement of spindles is specific for slow-wave sleep, and that is directly related to memory performance after a period of sleep containing slow-wave sleep compared to a

(19)

period without. Later studies have built on these findings, by expanding our knowledge of the role of sleep in memory processes. For example, Gais and Born (2004) describe a number of studies showing reactivation of newly formed memories during sleep, as well as studies that provide a basis for spindle activity and slow oscillations in humans.

An EEG study was conducted by Takashima et al. (2009) to test models and theory of sleep-related memory consolidation, where memories are consolidated from the hippocampus to the (frontal) neocortex. Their main findings are that, following consolidation, activation during memory retrieval increased in areas including the inferior frontal gyrus, posterior parietal cortex and a broad net-work involving subcortical areas as well. Interestingly, following consolidation, the hippocampus showed a decrease of activity during memory retrieval. In addition Takashima et al. (2009) showed that functional connectivity of the vi-sual cortex with the posterior hippocampus decreased following consolidation, instead specialized areas in the visual cortex became more connected with the frontal cortex.

Mitra et al. (2016) investigated communication between the hippocampus and the neocortex in resting state and sleeping situations. They found that dur-ing the restdur-ing state functional connectivity, in the form of activation patterns, was directed from sensory cortical areas to the hippocampus. However, during slow-wave sleep, these patterns of functional connectivity were reversed, thus going from the hippocampus to the neocortex, in particular to frontal areas. During wakeful states infra-slow activity (< 0.1 Hz) activity propagates from the hippocampus to the default mode network and sensory cortical regions, while faster delta band activity propagates the other way. This reversal is sug-gested to reflect the functions of declarative memory, as during wakeful states information predominantly flows to the hippocampus, while during slow-wave sleep information is propagated to the neocortex.

Aside from sleep-related consolidation, there is the phenomenon of replay during wakeful states (Carr, Jadhav, & Frank, 2011). During awake replay, recently encoded memories are being replayed in the hippocampus and me-dial temporal lobe through sequential reactivation of, for example, place cells. Specifically, sharp-wave ripples originating in the CA3 subfield are involved in replay and pattern completion in this region.

Taken together, these findings show that the hippocampus and the entorhinal cortex are heavily engaged in predictive processing, both on a local and on a more global scale. The entorhinal cortex specifically shows a predictive and a short-term mode of activation. These findings of task-related activity is then placed into a broader view

(20)

mind, namely to investigate how the (prefrontal) neocortex and the hippocam-pus communicate. First, task-related communication is discussed, showing that the medial temporal lobe, specifically the entorhinal cortex, exhibits two modes of activation, namely a predictive and a short-term memory mode. Addition-ally, the top-down influence of the medial temporal lobe on sensory regions is discussed, as well as its integration within a wider network, including the pre-frontal cortex. Second, the importance of sleep-related memory consolidation was presented, as slow-wave sleep has been shown to improve memory and that after consolidation, recruitment of the hippocampus is significantly decreased while the frontal cortex is recruited more in memory processes. This suggests that prediction errors are transmitted from the hippocampus, from short-term memory to the frontal cortex where they are integrated into the generative model.

3.3 Involvement of the Frontal Cortex in Predictive

Sen-sory Processing

In the previous two sections we have followed prediction errors as they got sent from sensory regions to the short-term storage of the hippocampus, and then got consolidated into frontally located long-term memory based generative models. This section discusses how predictions issued by these generative models modulate functioning in the visual cortex and the resulting effects this has on visual information processing.

Summerfield et al. (2006) presented participants with a decision task to in-vestigate how the medial frontal cortex is involved in the predicted perception of visual stimuli. Participants had to discriminate between images of faces, houses and cars. These stimuli were presented in face blocks, in which participants had to indicate whether a stimulus was a face or not, and house blocks, in which they had to indicate whether a stimulus was a house or not. The task was made to be perceptually challenging by visually degrading the stimuli matching individual thresholds for perception, encouraging participants to form a perceptual ‘set’ during each block.

The medial frontal cortex was found to represent predicted perception of stimuli, specifically in the context of maintaining a perceptual set. Additionally, an increase of top-down connectivity from the frontal cortex to face-sensitive areas, such as the fusiform face area, was found, which is consistent with the matching of predicted and observed evidence for the presence of faces.

For stimuli, face > non-face comparisons elicited activity in the inferior oc-cipital gyrus, the fusiform face area, temporo-parietal junction and the amyg-dala. Comparing face > house sets revealed that the face set elicited greater

(21)

activity in foci in the dorsal and ventral medial frontal cortex (MPFC), re-gardless of the actual stimuli being presented; these areas have previously been implicated in face processing. Because stimuli were controlled per participant for physical characteristics, saliency, and frequency, this response is indicative of maintaining a predictive ‘face’ set; in other words, maintaining information that is relevant to making a distinction between ‘face’ and ‘non-face’. Different clusters of increased activity in, among others, the dorsolateral PFC were found for house sets as well.

Furthermore, Summerfield et al. (2006) used dynamic causal modelling esti-mates (Friston, Harrison, & Penny, 2003) based on known interconnectivity to test the hypothesis that the dorsal and ventral MPFC are driving the response in the fusiform face area and the amygdala. MPFC to fusiform face area feedback was significantly enhanced on face-set trials, with no bottom-up influence.

These findings suggest that the frontal cortex codes for predicted representa-tions of objects in conjunction with maintaining a perceptual set. Since different perceptual sets recruit a predictive network in the frontal cortex, this suggests that these results might generalize to other categories. The findings from mod-elling the functional and effective connectivity suggest that there is indeed a top-down effect.

To investigate the underlying neural substrate in the role prediction plays on attention and perception, Ran, Chen, Cao, and Zhang (2016) performed a study using an orientation identification task. In this task, participants were presented with blocks of trials indicated by a direction prediction cue. Each trial started with a social cue consisting a face in which the eye gaze was directed to the left or the right, either presented subliminally (unconscious processing) or supraliminally (conscious processing). Following this cue, a masked target stimulus was presented on one side of the screen, to which participants had to respond by identifying the stimulus location.

The (left) dorsolateral prefrontal cortex was used as a seed region to deter-mine functional connectivity profiles for the different conditions. Contrasting predictable with unpredictable stimuli revealed activity in the middle frontal gyrus (Broca’s area 9), as the only main effect. Additionally, contrasting pre-diction and attention revealed a cluster of significant activation in the inferior occipital gyrus, where a larger response was found for attended, compared to unattended, stimuli in unpredictable trials. The attention × consciousness in-teraction revealed a cluster in the medial frontal gyrus which showed increased activation for attended compared to unattended stimuli in the conscious trials. This effect was reversed un unconscious trials, thus the attended stimuli elicited reduced activation.

(22)

at-tention and prediction, localised in the left superior frontal gyrus, left superior temporal gyrus, left thalamus and the right medial frontal gyrus. Specifically, activity in the dorsolateral prefrontal cortex was significantly decreased for pre-dictable stimuli in the unconscious-attended trials.

As for functional connectivity profiles. Increases between the left dorsolateral prefrontal cortex and the premotor cortex were found for unconscious attended trials. The synergy between unconscious attention and prediction facilitates stimulus processing, through recruitment of activity in the medial frontal gyrus. The observed hypoactivation of the dorsolateral prefrontal cortex is consistent with theoretical models of predictive coding, being indicative of correct models in the case of predictive stimuli (Friston et al., 2003; Rao & Ballard, 1999).

In an event-related potential (ERP) study, Chen, Ran, Zhang, and Hu (2015) used the same orientation identification task as Ran et al. (2016) to assess mod-ulation of unconscious attention on the silencing effect of top-down predictions. EEG activity for correct responses in each condition were aligned and averaged to yield event-related potentials. In the time-window of 110-150 ms target stim-ulus onset both a positive and a negative component were determined. Because of this, Chen et al. (2015) were able to assess at which stages of processing the recruitment of frontal regions occurs and how this changes over time as a function of attention.

The peak positive amplitude showed a significant effect of prediction, with predictable stimuli eliciting less activity than unpredictable stimuli. There was also a significant hemispheric effect, with the left hemisphere showing increased activity. This positive response also showed an interaction between atten-tion and predicatten-tion: predictable stimuli in the unconscious-attended condiatten-tion elicited smaller responses than did unpredictable stimuli. The peak negative amplitude showed that there was a greater negative amplitude for unconscious-attended stimuli than for ununconscious-attended stimuli. There was also a hemispheric effect with more negative amplitudes in the middle electrodes.

A cluster analysis using independent component analysis (ICA) showed that the positive cluster is characterised by a parieto-occipital distribution, with a dipole in the left visual area. On the other hand, the negative cluster is characterised by a frontal distribution with a dipole that is suggested to be in the orbitofrontal region.

This study fits in with the findings of Ran et al. (2016), as both show an interaction between prediction and unconscious attention. Both studies also show incorporation of frontal and parietal areas. The reduction of activity for predictable stimuli is in line with the ideas that sensory inputs matching top-down predictions are inhabited. This inhibition as a function of attention is suggestive of attention increasing the precision of predictions, which is based on

(23)

the activation in both visual and frontal regions.

While the previous studies do show involvement of the frontal cortex in gen-erating predictions, a true causal relation is difficult to establish truly causal relations using fMRI and EEG. To this end Zanto, Rubens, Thangavel, and Gazzaley (2011) used transcranial magnetic stimulation (TMS), combined with fMRI and MEG, to disturb the functioning of the inferior frontal junctions as participants performed a delayed recognition task under selective attention. It is important to note that Zanto et al. (2011) used a selective-attention, delayed recognition task, meaning participants had to engage working memory rather than rely on long-term memory. The embedded process theory of working mem-ory, developed by Cowan (1999), states that working memory is not a separate process but rather consists of an interplay between long-term memory and a subset of long-term memory that is activated as governed by attention and awareness. Using TMS allowed Zanto et al. (2011) to study the causal relation between the prefrontal cortex and the visual cortex during a visual task. The inferior frontal junction was targeted because this area has been suggested to be a source of top-down modulation that underlies attention to visual features. Higher order visual areas (V4, V5 and the middle temporal gyrus) were used as seed regions for functional connectivity analysis, as these regions have been shown to be selectively responsive to visual features such as motion. Further-more, they have been shown to be modulated by top-down attention processes. Zanto et al. (2011) found that a large fronto-parietal network was engaged in modulating the processing of both color and motion features. The disruption of the inferior frontal junction diminished top-down modulation of activity in the visual cortex. This resulted in a decline of working memory, which improved again as the effects of TMS wore off. These results support the idea that the in-ferior frontal junction is involved in updating task representations as a function of attention.

It is shown that the frontal cortex modulates activity in the visual cortex, through issuing predictions of upcoming stimuli. While the hippocampus-based modulation discussed in previous sections was shown to target the earliest stages of the visual cortex, frontal predictions are shown to target areas further up the visual processing hierarchy; areas that are involved in object recognition as well as the processing of visual features such motion and colour. This modulation is shown to be dependent on attention, and a link to working memory as a subset of long-term memory is made.

(24)

4 Conclusion & Discussion

4.1 Conclusion

In section 3.1 literature involving the hippocampus is reviewed. It is shown that the hippocampus is involved in statistical learning and specifically is sensitive to entropy of the stimulus stream. Given the research described here, there is still some uncertainty to the exact differences between model acquisition and up-dating based on sensory information as is done by the hippocampus. However, pattern completion in combination with feedback projections signaling outcome encoding to the sensory cortices might provide a mechanism for these processes. A differentiation between rote sensory predictions and mnemonic predictions is made as well, with the hippocampus being involved mainly in the latter, while the former is something done by the (higher levels of the) sensory cortices. Finally different anatomical substrates for statistical sequence learning and en-coding of episodic memory are discussed. These different anatomical pathways have different learning rates which places them in a hierarchy relative to the the neocortex.

In section 3.2 two kinds of papers were discussed, but with the same goal in mind, namely to investigate how the (prefrontal) neocortex and the hippocam-pus communicate. First, task-related communication is discussed, showing that the medial temporal lobe, specifically the entorhinal cortex, exhibits two modes of activation, namely a predictive and a short-term memory mode. Addition-ally, the top-down influence of the medial temporal lobe on sensory regions is discussed, as well as its integration within a wider network, including the pre-frontal cortex. Second, the importance of sleep related memory consolidation was presented, as slow-wave sleep has been shown to improve memory and that after consolidation, recruitment of the hippocampus is significantly decreased while the frontal cortex is recruited more in memory processes. This suggests that prediction errors are transmitted from the hippocampus, from short-term memory, to the frontal cortex where they are integrated into the generative model.

In section 3.3 the involvement of the frontal cortex and long-term memory processes is discussed. These studies show that the frontal cortex, known to be involved in cognitive control and attention processes, engages in predictive modulation of higher order regions of the visual cortex. This suggests that the frontal cortex has a more abstract and different function than the hippocampus, which was shown to mainly influence lower order sensory regions. The modu-lation of the visual cortex by the frontal cortex is shown to be dependent on attention.

(25)

Together, these findings support the idea that predictions errors are propa-gated first to the hippocampus, to be consolidated into frontally located long-term memory and thus informing the generative models that issue predictions back to the sensory cortices. Thus, providing a neuroanatomical implementation and integration of memory processes in the predictive processing framework.

4.2 Recommendations for Future Studies

In the studies of Harrison et al. (2006); Strange et al. (2005), individual blocks were considered to be new instances of statistical models that needed to be learned by participants. This could be rephrased where each block is not con-sidered a new model, but rather as presenting aberrant evidence to the existing model to be updated. This method can be used in addition to the existing analyses used and might shed light on how existing generative models get up-dated in the context of both more and less corroborating evidence. Another suggestion for future research consists of varying combinations of the methods presented with those of Schiffer et al. (2012). For example, as done by Schiffer et al. (2012), participants could be trained on specific blocks beforehand, allowing differentiation between updating of existing models and the acquisition of new models. It would be important, however, to make sure that the stimuli used can be differentiated by context, for example, slight variations in stimulus or background color; perhaps this could also be achieved by using more ecologically valid stimuli.

The studies presented in this paper all describe parts of a larger whole, in more or less detail. While I do realize that investigating the entire hierarchical mechanism as proposed in this paper cannot realistically be done by one study, a series of studies designed specifically to take on and integrate the different parts would allow a coherent picture to be formed. This would also allow the researchers involved to tune the methods and the research questions, further improving the integration of each other study.

Another open issue pertains to the interaction between short-term memory and long-term memory, as well as working-memory. As the generative models rooted in long-term memory send predictions from the frontal cortex to the sensory cortices, some amount of updating happens. The same is true for pre-diction errors sent from the sensory cortices to the hippocampus. However, some interplay should be expected between the hippocampus and association areas in the cortex, as was shown by the large fronto-parietal networks activated in accordance to the hippocampus. However, it remains unclear to what cog-nitive processes this recruitment pertains. Awake replay by the hippocampus has shown to provides a way for memories to become consolidated, however, the

(26)

interactions underling this were shown for the resting state. It would be inter-esting to probe the extent to which replay happens during tasks, and whether this allows for ‘online’ updating of long-term memory.

4.3 Cortical Lamination and Predictions

The theory of Chanes and Barrett (2016) deals with cortico-cortico connections and seems to fit in with the findings discussed in this paper. Additionally, it presents a more granular view of cortical regions as it indicates how more fine-grained anatomical differences underly predictive mechanisms, which can pro-vide a deeper understanding of neural functioning. Chanes and Barrett (2016) propose that the flow of information along gradients of laminar differentiation provides an insight into the role of limbic areas in cortical processing. They put forth that areas with a simpler laminar structure, that is, areas with fewer (com-pletely developed) cortical layers such as the cingulate and prefrontal cortices, send predictions to more laminated areas, being the earlier sensory cortices. Several models of cortico-cortical processing discussed by Chanes and Barrett (2016) show that these patterns of ‘feedback’ and ‘feedforward’ projections are predicted by the degree of laminar differentiation.

Many of the findings discussed in this paper are in line with this view, however, in terms of anatomical detail they do not go beyond the cortical re-gions. Chanes and Barrett (2016) argues that feedback projections, relaying predictions, originate in deeper layers of the agranular cortex, i.e. cortical areas with a less developed granular layer four, and terminate in superficial layers of the more granular cortices, while feedforward projections, signalling predic-tion errors, originate primarily in the superficial layers of the granular cortices. Related to the different layers is activity in different frequency bands of EEG and potentially MEG (Bastos et al., 2015; Roopun et al., 2006; da Silva, 2009). Future studies using EEG, potentially in combination with fMRI, can further test these relations and perhaps provide more detail on the mechanisms through which predctions and prediction errors are being transferred between regions of the brain.

4.4 Prefrontal Involvement in Working Memory

As has been mentioned in the discussion of Zanto et al. (2011), the DLPFC has been shown to be involved in working memory (Curtis & D’Esposito, 2003). Considering working memory as a subset of long-term memory, in accordance with the embedded process theory seems to be in line with the findings discussed so far. However, this does give rise to new questions and potential integration of this theory with the anatomical substrates of predictive processing will likely

(27)

result in some issues that would need to be resolves. An open question is to what extent working memory bypasses the hippocampus, and whether this fits in with predictive processing theory. Additionally, if working memory consists of a subset of activations of long-term memory in the frontal cortex, there is the question of whether the same pathways are used that operate in the cortico-hippocampal-cortical loop. If this is the case, how does the brain differentiate between long-term memory processes and working memory processes?

References

Bastos, A. M., Vezoli, J., Bosman, C. A., Schoffelen, J.-M., Oost-enveld, R., Dowdall, J. R., . . . Fries, P. (2015, jan). Vi-sual Areas Exert Feedforward and Feedback Influences through Distinct Frequency Channels. Neuron, 85 (2), 390–401. Re-trieved from http://www.ncbi.nlm.nih.gov/pubmed/25556836http:// linkinghub.elsevier.com/retrieve/pii/S089662731401099X doi: 10 .1016/j.neuron.2014.12.018

Carlson, T. A., Simmons, R. A., Kriegeskorte, N., & Slevc, L. R. (2014). The emergence of semantic meaning in the ventral temporal path-way. Journal of cognitive neuroscience, 26 (1), 120–131. Retrieved from http://dx.doi.org/10.1162/jocn{_}a{_}00409{%}5Cnhttp:// www.mitpressjournals.org/doi/abs/10.1162/jocn{_}a{_}00409 doi: 10.1162/jocn_a_00458

Carr, M. F., Jadhav, S. P., & Frank, L. M. (2011, feb). Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nature Neuroscience, 14 (2), 147–153. Retrieved from http:// www.nature.com/doifinder/10.1038/nn.2732 doi: 10.1038/nn.2732 Chanes, L., & Barrett, L. F. (2016). Redefining the Role of Limbic Areas in

Cortical Processing. Trends in Cognitive Sciences, 20 (2), 96–106. Re-trieved from http://dx.doi.org/10.1016/j.tics.2015.11.005 doi: 10.1016/j.tics.2015.11.005

Chen, X., Ran, G., Zhang, Q., & Hu, T. (2015). Unconscious attention modulates the silencing effect of top-down predictions. Consciousness and Cognition, 34 , 63–72. Retrieved from http://dx.doi.org/10.1016/ j.concog.2015.03.010 doi: 10.1016/j.concog.2015.03.010

Clark, A. (2013, jun). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36 (03), 181–204. Retrieved from http://www.journals.cambridge.org/ abstract{_}S0140525X12000477 doi: 10.1017/S0140525X12000477

The Role of the Hippocampus and the Frontal Cortex in Encoding Predictions About the World