Adaptive filters in neuronal circuits
Birte Zuidinga
1
1Research Master Brain & Cognitive
Sciences, University of Amsterdam, the Netherlands.
&
Circuits - Computation - Models department, Max Planck Institute of Neurobiology, Germany.
Assessor/Supervisor: Dr. Lukas Groschner – Max Planck Institute of Neurobiology, Germany.
Examiner: Dr. Conrado Bosman Vittini – University of Amsterdam, the Netherlands. April 2020 – June 2020
Correspondence Birte Zuidinga
Student number: 10996761 Email: birte.zuidinga@student.uva.nl
To survive in an environment that is constantly changing, animals learn from experience to make predictions about the future and shape behaviour. The computational requirements for such processes have been described in models of adaptive filters, originating from the engi-neering domain. Adaptive filters receive various inputs and modulate the respective gains based on an instructive signal to generate the output. Adaptive filter models can adequately describe key components of the signal processing steps that occur in the cerebellum, and have also been applied to vertebrate cerebellum-like structures. The mushroom body—a higher-order sensory processing structure in invertebrates—has many characteristic features that resemble the cerebellum, so the framework of an adaptive filter may also apply to this structure. Studies of the mushroom body, especially in the fruit fly (Drosophila melanogaster), benefit from the relatively small but functional brain, and from the numerous genetic tools that are available. However, vertebrate and invertebrate neuroscience do not appear use insights from the opposite domains to the fullest potential. In this thesis, I aim to facilitate the translation of knowledge between these fields by taking a comparative approach to investigate the cerebellum, the cerebellum-like structures, and the mushroom body, based on the framework of adaptive filters. K E Y W O R D S
Adaptive filters, cerebellum, cerebellum-like structures, mushroom body
1
| INTRODUCTION
The behaviour of all animals stems from complex inter-actions between internal states and external influences. The degree to which different inputs are relevant will vary depending on the context. For example, recognising sen-sory cues that signal the presence of food is essential for animals to maintain energy reserves, but upon encoun-tering a predator, a hungry animal needs to override its
feeding behaviours to move to safety. Additionally, some components of the signal carried by sensory afferents do not provide useful information to the animal. For in-stance, sensory input that is brought about by voluntary movements of the animal itself, termed reafference (von Holst and Mittelstaedt, 1950), should be removed from the raw input to construct an adequate representation of the outside world. This is a complex task, since the sig-nals caused by external perturbations and those arising 1
from self-motion can be indistinguishable at the sensory level. Extra information about the animal’s movement intentions is therefore projected from motor to sensory brain areas as so-called efference copy (von Holst and Mittelstaedt, 1950) or corollary discharge (Sperry, 1950). The motor plan is then compared to the raw sensory in-put, and signals in this raw input that are predicted by the motor plan are filtered out as reafference.
The characteristic anatomy and physiology of the cerebellum, that was extensively described by Eccles et al. (1967), appears to be well-equipped to perform such functions. Purkinje cells—the principal cerebellar neurons—receive two main input types. The first involves information that arrives via mossy fibres that project to granule cells, a processing step that is associated with a profound signal divergence. This divergence is followed by a strong convergence, as more than a hundred thou-sand granule cell axons, termed parallel fibres, synapse onto the dendritic arborisations of a Purkinje cell. The sec-ond input is governed by a single climbing fibre that exten-sively wraps around the Purkinje cell dendrites and makes many synapses. The influential Marr-Albus model of cere-bellar functioning proposed that the synaptic strengths of different parallel fibre inputs are regulated by an instruc-tional signal provided by the climbing fibre, and that after sufficient updating, Purkinje cell output can drive appro-priate behaviours based on the parallel fibre inputs alone (Albus, 1971; Marr, 1969).
Later interpretations and extensions of the Marr-Albus model highlighted the similarities with computa-tional models of adaptive filters (Dean et al., 2010; Dean and Porrill, 2011; Fujita, 1982; Gilbert, 1974). These models, that originated from the engineering domain, de-scribe that the gain of different inputs is dynamically regu-lated based on an instructional reference signal (Farhang-Boroujeny, 2013). Regarding the cerebellum as a struc-ture with a distinct computational capability—that of an adaptive filter—provides extensive possibilities for mod-elling the various cerebellar functions within a relatively simple framework, to generate hypotheses that can guide experimental research (Dean et al., 2010).
Based on their anatomical and physiological similari-ties, several neural structures in a variety of species have
been described as cerebellum-like, such as the electrosen-sory lobe of weakly electric fish and the mammalian dor-sal cochlear nucleus (Bell et al., 2008). Adaptive filter-ing appears to be a general mechanism that is employed in all of these structures (Bell et al., 2008; Montgomery and Perks, 2019; Requarth and Sawtell, 2011), provid-ing a convenient framework to compare and combine experimental insights from different species to enhance the understanding of their functioning. In the review of Bell et al. (2008), no invertebrate structures were de-scribed as cerebellum-like—which is illustrative of the general scarcity of translational efforts between the ver-tebrate and inverver-tebrate neurosciences. However, espe-cially the insect mushroom body has been compared to the cerebellum rather often, and it may thus be also de-scribed as cerebellum-like (Farris, 2011; Kenyon, 1896; Li and Strausfeld, 1997; Modi et al., 2020; Schürmann, 1974; Stevens, 2015; Yasuyama et al., 2002). Similar to the cerebellum, the mushroom body shows a remarkable divergence–convergence of inputs, and an instructive ref-erence signal that modulates synaptic strength (Farris, 2011; Modi et al., 2020). Classically, the mushroom body has been regarded as a higher-order olfactory processing centre that governs learning and memory (de Belle and Heisenberg, 1994; Dubnau et al., 2001; Heisenberg et al., 1985; McGuire et al., 2001), which is still a major focus of current studies.
Studies on insects, especially the fruit fly (Drosophila
melanogaster), benefit from the extensive availability of
genetic tools to manipulate and measure the relatively small but functional brain (Bellen et al., 2010; Stephenson and Metcalfe, 2013). Considering the mushroom body as a cerebellum-like structure that performs adaptive filter-ing (Farris, 2011) can facilitate the translation of insights between insects and vertebrates. In this literature thesis, I will elaborate on this perspective and explore if mush-room body functioning can indeed be described in terms of an underlying adaptive filter.
First, I describe the main computational features of adaptive filters. Subsequently, I discuss how three essen-tial elements of adaptive filters—basis signal expansion, reference signals, and the covariance learning rule—can be implemented biologically. I take a comparative
ap-proach to examine anatomical and functional correspon-dence between the cerebellum, ‘traditional’ cerebellum-like structures, and the mushroom body. In this text, the term cerebellum-like structures refers to the structures that have traditionally been included by Bell et al. (2008), excluding the mushroom body. Based on the comparisons between these structures, I emphasise worthwhile points of focus for future research.
2
| ADAPTIVE FILTERS
In the most basic form, an adaptive filter is a system that receives various time-varying inputs, imposes adjustable weights on them, and summates the resulting signals to produce an output (Farhang-Boroujeny, 2013) (Figure 1). The weights are modulated based on the correlation be-tween the respective input and a reference signal. The reference signal is often called ‘error signal’ or ‘teaching signal’ (Albus, 1971; Dean and Porrill, 2014; Medina et al., 2002). Both terms come with the connotation that the system knows which outcome is desired, but this is usu-ally not the case. Therefore, I will use the more general term ‘reference signal’.
The learning rule, that is used to update the weights of inputs, serves to minimise the correlations between the inputs and the reference signal. The gains of inputs that correlate with the reference signal are reduced and the gains of uncorrelated inputs are increased. Several termi-nological variations of this learning rule have been used— covariance rule (Sejnowski, 1977b), least-mean-squares rule (Douglas and Pan, 1995), delta rule (Yegnarayana, 2001), and decorrelation rule (Dean et al., 2002)—but they are all based on the same principle. I will use covari-ance learning rule, since this term emphasises that the covariance between the inputs and the reference signal determines the gain change.
To illustrate the computational requirements of a bio-logical adaptive filter in more detail, I will use the example of reafference elimination as one of the problems that an adaptive filter can solve. As pointed out earlier, animals need to distinguish sensory signals brought about by the external world from those arising from their own
move-ments; a task that is complicated by the fact that the raw sensory inputs can be identical. I assume that the com-putational and biological features, that are necessary for adaptive filtering of reafferent inputs, are representative of adaptive filtering in general. However, it is likely that adaptive filters carrying out other functions have been op-timised to these tasks, and therefore differ to some ex-tent.
To accurately filter out the sensory consequences of voluntary motion, it has been suggested that adaptive fil-ters learn to predict the reafferent signal and can generate the opposite signal—a negative image (Bastian, 1996; Bell, 1982; Bell et al., 1993; von Holst and Mittelstaedt, 1950; Sawtell and Williams, 2008; Sperry, 1950). If this nega-tive image is fed into the processing pathway of the raw sensory input, reafferent components will be cancelled out (Porrill et al., 2013). Three computational features that are essential for constructing an accurate negative image of the predicted reafference, are basis signal expan-sion, the reference signal, and the covariance learning rule (Montgomery and Perks, 2019).
3
| BASIS SIGNAL EXPANSION
The basis signal is regarded as the population of inputs that is initially available to the adaptive filter to create a negative image from (Montgomery and Perks, 2019). If the output of the adaptive filter is considered to be a lin-ear combination of all weighted inputs, the inputs deter-mine the time course of the possible outputs, and should thus be extensive enough for the task at hand (Gao et al., 2012; Montgomery and Perks, 2019; Porrill et al., 2013; Roberts and Bell, 2000). This is problematic in the case of reafference filtering; efference copies from motor areas can last much shorter than the sensory reafference they predict. For instance, when an animal jumps upwards, the visual flow that is perceived by the eyes will move down-wards for a longer time than the motor plan that signalled the initiation of the jump. The temporal range of the ba-sis signal, including efference copies, should thus be in-creased in complexity, which is governed by basis signal expansion (Montgomery and Perks, 2019).
Σ
Temporal signalling delay treference signal– texpanded basis signal
0
Covariance learning rule
Δ sy napti c w ei ght
τ
τ
τ
τ
τ
Adaptive gains Reference signal Basis signal Expanded basis signal
Negative image
+
-0
F I G U R E 1 Schematic representation of an adaptive filter.The graphs inside the boxes represent firing rate
(ordinate) over time (abscissa) of individual neurons. Input signals (purple) are semi-randomly combined and subjected to different temporal filters, resulting in an expanded basis signal (green) in intrinsic neurons. These neurons convergently project to the adaptive filter (yellow). The connections correspond to synapses and are subjected to adaptive gains. Whether the synapses are strengthened through long-term potentiation (LTP) or weakened through long-term depression (LTD) is based on the covariance learning rule (dashed box) and the reference signal (blue). Synapses are depressed when the reference signal follows the expanded basis signal input within a specific time window. In other situations, potentiation occurs. The expanded basis signal, filtered with different gains, is then summated. After sufficient gain updating, the output of the adaptive filter will be the inverse of the reference signal, also called a negative image.
3.1
Initial inputs
In the cerebellum and cerebellum-like structures, the ba-sis signal is thought to be carried by mossy fibres. These originate from various areas and pathways and thus pro-vide a diversity of inputs that can be used to predict the reference signal (Bell et al., 2008; Montgomery and Perks, 2019) (Figures 2A and 2C). For instance, cerebellar mossy fibres receive sensory inputs and motor efference copies from the pontine nuclei, where cortical projections arrive via the pyramidal tract. Proprioceptive information from joints and muscles is available from spinal cord projec-tions, and signals of head position and movements are available to the mossy fibres in the vestibular nuclei (Allen and Tsukahara, 1974; Paxinos, 2004). In the
cerebellum-like electrosensory lobe of weakly electric fish, mossy fibres receive efference copy and proprioceptive inputs (Bell and Grant, 1992; Bratton and Bastian, 1990; Mohr et al., 2003; Requarth and Sawtell, 2011; Szabo et al., 1979). Similarly, the outer layer of the optic tectum of ray-finned fishes receives visual inputs and efference copies related to eye movements (Northmore et al., 1983), and the mammalian dorsal cochlear nucleus receives, among others, auditory inputs from the auditory cortex (Weed-man and Ryugo, 1996) and inferior colliculus (Caicedo and Herbert, 1993), vestibular inputs (Burian and Gstoet-tner, 1988), and somatosensory inputs (Wolff and Künzle, 1997). Based on these observations, the basis signal avail-able to adaptive filters in the cerebellum and
cerebellum-like structures appears to be an arrangement of various inputs that can include efference copies, proprioceptive information, and sensory inputs, from multiple modalities and stages of processing.
Mossy fibres of the cerebellum and cerebellum-like structures are analogous to the projection neurons of the mushroom body (Figure 2). In the fruit fly, the majority of these neurons are olfactory projection neurons from the antennal lobe (Aso et al., 2014a; Takemura et al., 2017). The fruit fly mushroom body also receives visual, gusta-tory, mechanosensoy, and temperature inputs (Barth and Heisenberg, 1997; Brembs and Wiener, 2006; Keene and Masek, 2012; Kirkhart and Scott, 2015; Liu et al., 1999; Mamiya et al., 2008; Masek and Scott, 2010; Vogt et al., 2014; Wilson et al., 2004; Yagi et al., 2016). In the honey bee (Apis mellifera), multimodal inputs have also been de-scribed, and the visual projections appear to be extensive (Erber, 1978; Gronenberg, 1986), and in the cockroach (Periplaneta americana), additional efference copy and pro-prioceptive inputs have been reported (Mizunami et al., 1998; Okada et al., 1999). It should thus be recognized that the mushroom body is not a fixed structure; the lay-out may be similar, but the functions that it performs are shaped by the brain in which it is embedded, and different insect species may use the mushroom body for different functions. This fits with the perspective that adaptive fil-ters provide a computational mechanism that can be used for different functions in different structures. Therefore, there are no explicit requirements for the types of inputs to judge whether a structure performs adaptive filtering. However, the input types do determine the kind of signals that are compared to the reference signal, and thus what kind of predictions can be made.
3.2
Transformation of initial inputs
Upon revisiting the example of an animal jumping up-wards, it is recognised that the original inputs to adaptive filters (here efference copy of a motor command) cannot always account for the proper cancellation of predictable components in the reference signal (here the downwards visual flow) (Montgomery and Perks, 2019; Porrill et al., 2013). Therefore, it is necessary that the inputs that make up the basis signal are transformed to signals with
dif-ferential temporal properties, together spanning the to-tal temporal range of sensory signals that they predict. This is most likely implemented in the signal transduc-tion steps from mossy fibres and projectransduc-tion neurons to granule cells and Kenyon cells, respectively in the cere-bellum and cerecere-bellum-like structures, and in the mush-room body. In the remaining text, I will use the term in-put neurons to refer to mossy fibres and projection neu-rons together, and intrinsic neuneu-rons to refer to granule and Kenyon cells.
3.2.1
Pattern separation
In the cerebellum, the cerebellum-like structures, and the mushroom body alike, the signal processing step from in-put neurons to intrinsic neurons involves an enormous di-vergence (Figure 2). The abundant intrinsic cells each re-ceive input from a semi-random combination of the less numerous input neurons. In terms of cell numbers, the expansion has been estimated to be 30-fold in the cere-bellum and 40-fold in the mushroom body (Litwin-Kumar et al., 2017). Approximately 4 mossy fibres (Eccles et al., 1967) and 7 projection neurons (Caron et al., 2013) con-verge onto one cerebellar granule cell and one mushroom body Kenyon cell, respectively. When using a combina-torial code, and given the biological constraints, distinct activity patterns can be optimally decorrelated with this anatomically observed degree of numerical divergence (Litwin-Kumar et al., 2017). Large-scale inhibitory feed-back to intrinsic cells, by Golgi cells in the cerebellum and cerebellum-like structures (Cesana et al., 2013; Kanichay and Silver, 2008; Mugnaini et al., 1980) and by the an-terior paired lateral neuron (APL) in the mushroom body (Lin et al., 2014; Pitman et al., 2011) (Figure 2), addition-ally reduces the number of intrinsic neurons that are ac-tive at a given time, sparsening the activity pattern fur-ther (Bodznick and Montgomery, 1992; Lin et al., 2014; Montgomery and Perks, 2019).
Pattern separation provides the adaptive filters with more distinct inputs, and is therefore a process of basis signal expansion. Important for the efficiency of pattern separation is random connectivity between input neu-rons and intrinsic neuneu-rons, however, experimental obser-vations demonstrate deviations from this requirement.
Purkinje cell Climbing fibre Unipolar brush cell Mossy fibres Parallel fibres Golgi cell Granule cells Vertebrate cerebellum Medium ganglion cell Electroreceptor afferent neurons Unipolar brush cell Mossy fibres Parallel fibres Golgi cell Granule cells Stellate cell Excitatory efferent cell
Mormyrid electrosensory lobe
Projection neurons MBON PPL1 PAM MBON Kenyon cells APL
Fruit fly mushroom body Basket cell
Stellate cell
A
B
C
F I G U R E 2 Cellular circuitry of neural structures functioning as adaptive filters.Simplified schematic overviews
of the microcircuitry of (A) the cerebellum, (B) the cerebellum-like mormyrid electrosensory lobe, and (C) the fruit fly mushroom body. Where possible, the colours correspond to the colour scheme used in Figure 1. Mossy fibres and projection neurons (input neurons) carry the basis signal and synapse onto granule cells and Kenyon cells, respectively. The Golgi and APL neurons provide feedback inhibition that facilitates a sparse code. Unipolar brush cells, that are present in some cerebellar regions and several cerebellum-like structures, possibly play an additional role in temporal filtering of the basis signal by rebound firing upon termination of Golgi cell inhibition (see text). Granule cells and Kenyon cells (intrinsic neurons) synapse onto principal output neurons; Purkinje cells in the cerebellum, excitatory efferent cells in the mormyrid electrosensory lobe, and mushroom body output neurons (MBONs) of the fruit fly mushroom body. Additionally, feedforward inhibitory interneurons project from intrinsic to principal cells; stellate cells synapse onto the dendrites, and basket cells, medium ganglion cells, and certain MBONs synapse onto the axons or somas. In the cerebellum, the reference signal is carried by the climbing fibre that is strongly synaptically connected to a Purkinje cell by wrapping extensively around its dendrites. Electrosensory afferent neurons connect to basal dendrites of medium ganglion cells and excitatory efferent cells. In the mushroom body, the reference signal is carried by the dopaminergic PPL1 and PAM neurons that target Kenyon cells and MBONs.
Mossy fibres are not completely randomly connected to granule cells in the cerebellum (Billings et al., 2014), and a recent electron microscopy reconstruction pro-vided similar evidence for the fruit fly mushroom body (Zheng et al., 2020), substantiating several earlier indica-tions (Gruntman and Turner, 2013; Jefferis et al., 2007; Lin et al., 2007; Tanaka et al., 2004). However, as argued by Litwin-Kumar et al. (2017), in spatially and functionally restricted domains the inputs appear sufficiently diverse (Chabrol et al., 2015; Huang et al., 2013; Ishikawa et al., 2015) for the execution of the specific tasks that the do-mains are involved in.
3.2.2
Temporal filtering
Besides augmenting the number of patterns that can be distinguished, the temporal characteristics of these signals should be sufficiently diverse. Different mech-anisms have been proposed that subject the basis sig-nals to various temporal delays. In the electrosensory lobe of mormyrid weakly electric fish, it was found that unipolar brush cells–excitatory neurons that have long been ignored in cerebellar structures (Mugnaini and Floris, 1994)—can provide delayed efference copy signals to granule cells (Kennedy et al., 2014). The authors sug-gested that rebound firing could play a role, that was ob-served in a subset of unipolar brush cells after the cessa-tion of a hyperpolarising stimulus. The delay between re-moval of hyperpolarisation and rebound firing was modu-lated by the strength of this initial hyperpolarisation, with stronger hyperpolarisations causing longer delays.
The natural source of inhibition to unipolar brush cells could be the Golgi cells (Kennedy et al., 2014)— inhibitory interneurons that are present in the cerebel-lum and cerebelcerebel-lum-like structures (Mugnaini et al., 1997). Golgi cells receive mossy fibre and granule cell inputs, and provide large-scale feedforward and feedback inhibition to granule cells (Cesana et al., 2013; Kanichay and Silver, 2008), but they also target unipolar brush cells (Dugué et al., 2005; Rousseau et al., 2012). Efference copy infor-mation could thus be carried from mossy fibres to Golgi cells that inhibit unipolar brush cells. Rebound firing in unipolar brush cells upon termination of this inhibition
thereby transfers the efference copy to granule cells with a delay.
Unipolar brush cells, or analogous cell types, are not present in all cerebellar compartments and cerebellum-like structures (Figure 2B). In the mushroom body, the only inhibitory neuron targeting Kenyon cells is APL, which provides large-scale feedback inhibition to these cells (Lin et al., 2014; Takemura et al., 2017); instead of providing feedforward inhibition in which the stimulus identity is maintained. Therefore, it is unlikely that the described mechanism of rebound firing in unipolar brush cells represents the only mechanism for generating tem-poral delays in structures functioning as adaptive filters. However, for some structures unipolar brush cells could contribute a robust, additional form of temporal delays on top of a more ubiquitous mechanism.
If unipolar brush cells provide an additional mech-anism of temporal filtering to certain structures, what could be the primary mechanism? An appealing hypoth-esis is that it depends on variation in biophysical proper-ties within the population of intrinsic cells. If different in-trinsic neurons transform inputs into outputs in slightly different ways, this might be sufficient to generate an as-sortment of differently timed signals for the adaptive filter to use. Substantiating this hypothesis, studies of the cere-bellum and cerecere-bellum-like structures have demonstrated diverse requirements for intrinsic cells to fire. For in-stance, a subset of granule cells in the electrosensory lobe of weakly electric fish fired upon receiving two coinciding inputs, but other granule cells already fired reliably from strong activation of just one input (Sawtell, 2010). Like-wise, in the rat cerebellar cortex, it was reported that high-frequency activation of a single mossy fibre could trigger granule cell firing (Rancz et al., 2007). This indicates that input signals to intrinsic cells are temporally summated until the firing threshold is reached, a process that is asso-ciated with ample possibilities for variation in biophysical properties that can provide differences in input–output transformations between cells. Such biophysical proper-ties are for instance input resistance, membrane length and time constants, resting membrane potential, and fir-ing threshold.
The idea of a collection of intrinsic cells, that need to summate different numbers of inputs within different time spans to drive firing, fits well with properties of fruit fly Kenyon cells. These cells summate inputs linearly, showing passive cable properties (Gruntman and Turner, 2013). The number of inputs that a Kenyon cell sum-mates before a spike is initiated varies substantially be-tween and within different Kenyon cell types (Groschner et al., 2018). That this variation is important for the gen-eration of differentially delayed signals that are necessary for adaptive filters, can be an answer to the previously un-resolved question of how the identified biophysical differ-ences between Kenyon cells are useful to the function of the mushroom body (Groschner et al., 2018; Groschner and Miesenböck, 2019; Inada et al., 2017; Tanaka et al., 2008; Turner et al., 2008).
Related to the biophysical properties of the intrin-sic cells themselves, the interaction of these with the different inputs that they receive has also been shown to delay firing to different extents. In a study of the mouse vestibulocerebellum, it was shown that distinctive excitatory post-synaptic events in granule cells could be mapped to five mossy fibre origins (Chabrol et al., 2015). Combinations of these inputs generated granule cell firing responses that differed in first spike latency.
In some structures, the potential for differential de-lays appears to be more restricted. For example, con-trary to the passive temporal summation in the fruit fly, Kenyon cells in the mushroom body of the locust
(Schisto-cerca americana) function as coincidence detectors
(Perez-Orive et al., 2002), governed by supralinear processing of simultaneous inputs and cycles of global feedforward inhi-bition, which strongly curtail integration time (Bazhenov and Stopfer, 2010; Perez-Orive et al., 2004). Similarly, in the cerebellar C3 zone of cats it was estimated that all three to four mossy fibres projecting to a granule cell needed to be coactive to drive firing (Jörntell and Ekerot, 2006). These findings indicate that an assortment of in-puts with different delays might be less important for these (sub)structures than for other structures. However, temporal variability within the population of intrinsic neu-rons was not explicitly addressed in these studies, so
bio-physical differences might still be used to some degree in these structures, to generate slight differences in timing. In summary, it appears that most adaptive filter struc-tures have mechanisms to subject input signals to differ-ent temporal delays. This can be governed by rebound firing in unipolar brush cells, by the biophysical proper-ties of intrinsic cells themselves, by the combination of active inputs, or by combination of these. I consider it likely that the intrinsic cell properties and input combina-tions together can explain temporal delays in most cere-bellar, cerebellum-like, and mushroom body structures, and that additional cell types like unipolar brush cells pro-vide additional tools for some circuits. Pattern separation, that is governed by the distinctive connectivity between input and intrinsic neurons and by large-scale inhibitory feedback, provides the adaptive filters with more distin-guishable input patterns as well, additionally expanding the basis signal. There is a clear need for studies investi-gating the inputs and outputs of multiple intrinsic cells at the same time, to analyse the extent to which input sig-nals are differentially delayed in intrinsic cells, and to as-sociate this to the presence or absence of unipolar brush cells in different structures. A focus on variability instead of the average properties of input processing in intrin-sic neurons is crucial for this. Future studies will benefit from advances in genetically encoded voltage indicators in the fruit fly, with which the sub- and suprathreshold re-sponses of multiple Kenyon cells can be measured at the same time (Chamberland et al., 2017; Yang et al., 2016).
4
| REFERENCE SIGNALS
The expanded basis signal, that is carried by intrinsic cells, is projected onto principal cells. These major output neu-rons are the Purkinje cells, the Purkinje-like cells, and the mushroom body output neurons (MBONs) of the cerebel-lum, the cerebellum-like structures, and the mushroom body, respectively (Figure 2). Each principal cell receives inputs from many intrinsic cells, causing a strong conver-gence of signals (Bell et al., 2008; Eccles et al., 1967; Takemura et al., 2017). The synapses between intrinsic and principal neurons represent the place where the
ex-panded basis signal is subjected to variable gains based on the reference signal (Dean et al., 2010; Montgomery and Perks, 2019). In the cerebellum, the reference sig-nal is provided by climbing fibres that enwrap the den-dritic arborisations of Purkinje cells (Eccles et al., 1967). Climbing fibres are distinctive for of the cerebellum, they are not observed in cerebellum-like (Bell et al., 2008) or mushroom body structures. However, in these structures, the principal neurons do receive reference signals that are used to modulate the strength of the intrinsic cell synapses (Figure 2). In cerebellum-like structures, neu-rons carrying the reference signal often synapse onto the basilar dendrites of principal cells (that are absent in Purkinje cells), whereas the more extensive apical den-drites receive the expanded basis signal (Bell et al., 2008). In the mushroom body, reference signals are carried by dopaminergic neurons (Mao and Davis, 2009; Waddell, 2013) that make synapses with, among others, Kenyon cells and MBONs (Takemura et al., 2017).
The characteristics of the reference signal determine which inputs from the expanded basis signal are modu-lated, and thus ultimately dictate the functional role of the adaptive filter (Dean et al., 2010; Ito, 2006). As re-ferred to earlier, the adaptive filter systems do not directly have access to a genuine sensory ‘error’, since the precise desired output of the system is often unclear. However, using a raw sensory input as a reference signal, instead of a proper error, can still be sufficient for the functioning of an adaptive filter (Porrill et al., 2013, 2004). In the ex-ample of reafference filtering, the efference copy is related with the part of the raw sensory signal that cor-responds to reafference, and not to the parts that corre-spond to external influences. Therefore, the covariance learning rule will cause the gains of only the reafferent inputs to be adapted. After sufficient updating, the refer-ence signal will not approach zero (as would be the case for a genuine error signal), but the predictable elements within the total signal—including reafference—will be fil-tered out. Accordingly, reference signals often have clear sensory origins. Moreover, the type of inputs in the ex-panded basis signal correspond well to the type of ref-erence signal that structures receive. This provides the structures with appropriate predictors that can be used
to perform the specific function that is dictated by the reference signal.
One of the clearest examples can be found in the cerebellar flocculus, a region that controls eye move-ments and has been researched extensively for its role in the vestibulo-ocular reflex (Ito, 1982; Ito et al., 1982a; Maekawa and Simpson, 1972; Noda, 1986). Granule cells receive mossy fibre inputs that carry efference copies of eye movement plans, and other inputs that carry vestibu-lar information about head movements, whereas climb-ing fibre reference signals carry retinal slip information (Maekawa and Simpson, 1972; Noda, 1986; Waespe et al., 1981). Retinal slip refers to unitary motions of the total visual field. Such motions are often the result of volun-tary eye or head movements, and need to be filtered out as reafference in those cases. Clearly, efference copies of intended eye and head movements provide predictive sig-nals for this retinal slip, so the cerebellar flocculus is well suited for this task. Importantly, retinal slip that is caused by external perturbances cannot be predicted in this way. In such situations, the retinal slip signals will not be fil-tered out, allowing an animal to respond to unexpected situations.
Similarly, the mossy fibres of the electrosensory lobe of weakly electric fish carry efference copy and proprio-ceptive inputs (Bell and Grant, 1992; Bratton and Bastian, 1990; Mohr et al., 2003; Requarth and Sawtell, 2011; Sz-abo et al., 1979), whereas primary electrosensory affer-ents represent the reference signal (Bell and Maler, 2005). These fish transmit electric pulses from an electric organ in their tail and use electroreceptors distributed over their body to actively sense the electric fields in the environ-ment; a mechanism that is reminiscent of echolocation in bats. Reafference caused by the direct influence of elec-tric organ discharge on electroreceptors adds noise to the system. Together, the efference copies of electric organ discharge and the proprioceptive information of tail posi-tion provide the necessary predictive informaposi-tion to filter out this reafference in the electrosensory lobe, resulting in a signal that only consists of the electric information originating from the environment (Bell et al., 2008).
The dopaminergic neurons, that provide the refer-ence signals in the mushroom body, can be divided
broadly into two functional classes: PPL1 neurons that signal punishment (Claridge-Chang et al., 2009; Galili et al., 2014; Masek and Scott, 2010) and PAM neurons that signal reward (Burke et al., 2012; Liu et al., 2012). Subsets of these neurons respond to different types of sensory inputs, such as the sweet taste or nutritional value of sugar (Yamagata et al., 2015), temperature (Tom-chik, 2013), or electric shocks (Galili et al., 2014), which is also in line with the notion that reference signals to adaptive filters arise from sensory systems. It is likely that, among others, olfactory, visual, and gustatory inputs, car-ried by the intrinsic cells, are useful predictors for these reinforcing reference signals. This substantiates the hy-pothesis that, also in the mushroom body, the basis sig-nal and reference sigsig-nal are attuned to the same, specific function.
There are indications, however, that dopaminergic signals in the fruit fly mushroom body carry more com-plex information than just reinforcing stimuli of punish-ment and reward. Specifically, dopaminergic neurons were shown to carry additional information about the in-ternal state of the fly, leading to the proposal that dynam-ics of dopaminergic signalling provide contextual informa-tion that influences the processing of incoming olfactory stimuli to drive different behaviours (Cohn et al., 2015). The interpretation that these contextual cues are pro-vided by the dopaminergic signals appears to differ from adaptive filter models of the cerebellum and cerebellum-like structures. In these models, all relevant contextual in-formation is assumed to be provided via the expanded ba-sis signal (Allen and Tsukahara, 1974; Bell, 1982; Blazquez et al., 2004; Dean et al., 2010; Huang et al., 2013) and the reference signals are related to specific sensory cues, that need to be predicted to carry out a specialised func-tion (Andersson and Oscarsson, 1978; Apps and Hawkes, 2009; Dean et al., 2010; Ekerot et al., 1991). However, I believe that both seemingly opposite perspectives are reconcilable within the framework of adaptive filters. For the cerebellum, there are several indications that climb-ing fibres do not merely transmit sensory reference sig-nals. Instead, their activity can be modulated by inter-nal states related to for instance noradreinter-naline siginter-nalling (Carey and Regehr, 2009), cortical processing (Brown and
Bower, 2002), and alcohol levels (Carta et al., 2006). Ad-ditionally, besides processing olfactory input—which al-ready provides a wealth of contextual information—the mushroom body Kenyon cells of different insect species also receive visual, gustatory, mechanosensory, tactile, and putative efference copy inputs (Barth and Heisen-berg, 1997; Brembs and Wiener, 2006; Keene and Masek, 2012; Kirkhart and Scott, 2015; Li and Strausfeld, 1999; Mamiya et al., 2008; Masek and Scott, 2010; Mizunami et al., 1998; Okada et al., 1999; Schildberger, 1984; Vogt et al., 2014; Wilson et al., 2004; Yagi et al., 2016). In my opinion, this suggests that the assumption that one path-way provides most contextual information should be re-vised for the cerebellum and cerebellum-like structures, as well as for the mushroom body. It is more likely that both the intrinsic neurons and the neurons providing ref-erence signals carry a collection of information that is nec-essary to modulate behaviour in specific situations. To some extent, this information might be split into the sen-sory signals that together represent the external world, and reinforcement signals that are modulated by internal states, carried by intrinsic cells and reference signals, re-spectively.
5
| COVARIANCE LEARNING RULE
In computational models of adaptive filters, the inputs providing the expanded basis signal are subject to adap-tive gains based on their correlation with the reference signal. The covariance learning rule describes that the gain of an input is decreased if its activation precedes the reference signal within a certain time frame, and in-creased if it is active at another time (Farhang-Boroujeny, 2013; Sejnowski, 1977b) (Figure 1). The processes that modify input gain at the synaptic level are long-term potentiation (LTP) and long-term depression (LTD), that strengthen and weaken synaptic transmission, respec-tively. LTP and LTD are not unitary mechanisms; they should be regarded as functional consequences that can arise from a multitude of cellular processes (Malenka and Bear, 2004). Plasticity at the excitatory synapses between intrinsic cells and principal cells generally
in-volves associative LTD and nonassociative LTP (Dean et al., 2010; Han et al., 2000; Requarth and Sawtell, 2011). Associative LTD occurs if synaptic activity predicts the ref-erence signal and nonassociative LTP occurs the other sit-uations. These plasticity rules contrast the classical Heb-bian plasticity, that relies on associative synaptic potenti-ation instead of depression (Hebb, 1949), and have there-fore been called anti-Hebbian (Bell et al., 2008, 1993; Dean et al., 2010; Han et al., 2000; Roberts and Bell, 2000).
A crucial aspect for neural circuits in general, but especially for the dynamic nature of adaptive filters, is that synaptic potentiation and depression are reversible. For LTD and LTP to counteract each other, it is neces-sary that they ultimately influence the same processes (Han et al., 2000; Houk et al., 1990; Mauk and Donegan, 1997; Raymond and Lisberger, 1998; Sejnowski, 1977a). In the cerebellum, pre- and postsynaptic mechanisms of LTP and LTD have been described (Coesmans et al., 2004; Hémart et al., 1994; Ito et al., 1982b; Lev-Ram et al., 2002; Linden, 1997; Linden et al., 1991; Qiu and Knöpfel, 2009; Sakurai, 1987; Salin et al., 1996). The presynap-tic mechanisms opposingly influence the vesicular release system (Chavis et al., 1998; Chen and Regehr, 1997; Seino and Shibasaki, 2005), and the postsynaptic mechanisms modulate the insertion and internalisation of α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) re-ceptors in and from the membrane (Hansel et al., 2006; Leitges et al., 2004; Wang and Linden, 2000; Xia et al., 2000). These pre- and postsynaptic processes also influ-ence plasticity at the opposite sites (van Beugen et al., 2006; Jacoby et al., 2001; Qiu and Knöpfel, 2007). There-fore, plasticity is reversible and can be closely controlled in an integrated system (Gao et al., 2012; Hoxha et al., 2016; Mapelli et al., 2015). The mushroom body appears to function in a way that is broadly similar to the anti-Hebbian plasticity in the cerebellum; the mechanisms will be described in more detail in the following sections.
5.1
Timing and temporal order of inputs
In computational models of adaptive filters, the timing and order of the basis and reference signals play an
im-portant role in determining whether a synapse is poten-tiated or depressed (Bell et al., 1997; Marr, 1969; Se-jnowski, 1977a). Studies of the cerebellum, cerebellum-like structures, and the mushroom body, that used dif-ferent stimulation protocols, species, and tissue prepara-tions, have reported a myriad of requirements regarding input order and timing that influenced the induction of ei-ther LTD or LTP (Aso and Rubin, 2016; Bell et al., 1997; Berry et al., 2018; Cassenaer and Laurent, 2007; Ekerot and Kano, 1989; Han et al., 2000; Handler et al., 2019; Harvey-Girard et al., 2010; Hige et al., 2015; Karachot et al., 1994; Owald et al., 2015; Safo and Regehr, 2008; Schutter and Maex, 1996; Tanimoto et al., 2004). This il-lustrates that the exact covariance learning rule is not a fixed characteristic of an adaptive filter, but likely varies between and within structures. In general, however, as-sociative LTD and nonasas-sociative LTP can explain many experimental findings.
An appealing molecular substrate to govern different effects based on the order of two signals, is the inositol-1,4,5-triphosphate (IP3) receptor that regulates Ca2+
re-lease from the endoplasmic reticulum (Finch and Augus-tine, 1998; Sarkisov and Wang, 2008; Wang et al., 2000). This receptor is activated by the second messenger IP3
that can be generated by G-protein coupled receptor sig-nalling cascades. However, increases in cytosolic Ca2+
concentration inactivate the IP3 receptors (Adkins and
Taylor, 1999; Khodakhah and Ogden, 1995; Paknejad and Hite, 2018; Srikanth et al., 2004). Therefore, two signals— the production of IP3and the elevation of cytosolic Ca2+
concentration—temporally compete at the IP3receptors.
Ca2+release from the endoplasmic reticulum only occurs
if IP3can bind the receptors before cytosolic Ca2+
con-centration is increased. This molecular mechanism likely plays a role in different circuits that function as adaptive filters; by gating Ca2+responses based on the temporal
order of the two signals, downstream mechanisms can be predisposed to LTP or LTD.
5.2
Sites of synaptic plasticity
Processes that involve the IP3 pathway have been
cere-bellum, but presynaptically in the Kenyon cells of the mushroom body. I consider it likely that this difference relates to discrepancies in neuronal projections carrying the reference signal; the climbing fibres of the cerebel-lum mainly target the postsynaptic Purkinje cells (Eccles et al., 1966; Maekawa and Simpson, 1972), whereas the dopaminergic neurons of the fruit fly mushroom body al-ready synapse onto the presynaptic Kenyon cells, in addi-tion to the postsynaptic MBONs (Boto et al., 2014; Hige et al., 2015; Takemura et al., 2017). Therefore, the first sites where the expanded basis signal and the reference signal converge are the intrinsic cells of the mushroom body and the principal cells of the cerebellum, which can explain that processes that depend on the relative tim-ing of the two signals are implemented presynaptically in Kenyon cells and postsynaptically in Purkinje cells.
In the cerebellum, glutamatergic transmission from parallel fibres to Purkinje cells activates AMPA receptors and the G-protein coupled mGluR1 receptors. As re-viewed by, among others, Hoxha et al. (2016) and Gao et al. (2012), mGluR1 then activates phospholipase C, re-sulting in the production of IP3 (Conn and Pin, 1997;
Hartell, 1994; Kano et al., 2008; Khodakhah and Arm-strong, 1997; Linden et al., 1991). Additionally, climb-ing fibre activation provokes a strong depolarisation in Purkinje cells, which triggers Ca2+influx through
voltage-gated channels (Gao et al., 2012). If parallel fibre firing and consequent IP3signalling precedes the climbing
fibre-induced Ca2+influx, internal stores will release Ca2+in an
IP3receptor-dependent way (Wang et al., 2000). In the
Purkinje cell, this stimulates protein kinase C that phos-phorylates the GluR2 unit of AMPA receptors (Hansel et al., 2006; Leitges et al., 2004), causing these recep-tors to be recognised for clathrin-dependent internalisa-tion processes (Wang and Linden, 2000; Xia et al., 2000). Thus, supralinear Ca2+signals, caused by the activation
of parallel fibres followed by climbing fibres, induce inter-nalisation of AMPA receptors, resulting in the weakening of the respective parallel fibre synapses.
Low frequency stimulation of parallel fibres in the ab-sence of climbing fibre activation results in calcium re-sponses that are too small to induce LTD. Instead, this acti-vates a Ca2+/calmodulin-dependent pathway involving a
phosphatase cascade that stimulates AMPA receptor inte-gration in the synaptic membrane via N-ethylmaleimide-sensitive factor (Coesmans et al., 2004; Gao et al., 2012; Lev-Ram et al., 2002; Steinberg et al., 2004; Gardner et al., 2005), provoking LTP that can thus reverse LTD.
Postsynaptic plasticity mechanisms have not been described for the synapses between Kenyon cells and MBONs. However, since dopaminergic neurons also synapse onto MBONs directly besides their Kenyon cell projections (Takemura et al., 2017), and since memory functioning is not completely abolished when the presy-naptic plasticity process involving rutabaga adenylyl cy-clase is eliminated (Boto et al., 2020; Livingstone et al., 1984; Tan et al., 2010; Tully and Quinn, 1985)—further discussed in the following section—I consider it likely that postsynaptic mechanisms of plasticity also play a role in the mushroom body. Whether this is the case, and if IP3receptors and receptor trafficking might be involved
as well, are important questions to address in future re-search.
In contrast to postsynaptic plasticity processes, presynaptic plasticity processes have been described in the fruit fly mushroom body. Cholinergic transmission from projection neurons to Kenyon cells delivers the ba-sis signal. This activates nicotinic acetylcholine receptors, causing a depolarisation that increases the intracellular Ca2+concentration via voltage-gated channels
(Campu-sano et al., 2007). Dopaminergic neurons synapsing onto Kenyon cells carry the reference signal. Dopamine acti-vates the G-protein coupled DopR1 and DopR2 receptors (Croset et al., 2018). The two types of input are thus gov-erned by opposite signals in the presynaptic Kenyon cells compared to the postsynaptic Purkinje cells: membrane depolarisation causing Ca2+influx via voltage-gated
chan-nels is related to the basis signal in the mushroom body but to the reference signal in the cerebellum, and activ-ity of G-protein coupled receptors indicates the reference signal in the mushroom body but the basis signal in the cerebellum. Therefore, the order of the signals that bring about LTD and LTP are opposite in these structures, and potential IP3signalling processes should lead to
The two dopamine receptors expressed in Kenyon cells, DopR1 and DopR2, appear to play an important role in regulating synaptic plasticity (Handler et al., 2019; Modi et al., 2020). It is likely that DopR2 receptors are volved in the process that detects the order of the two in-puts, since these receptors are associated with Gαq that regulates Ca2+release from the endoplasmic reticulum,
probably via IP3(Feng et al., 1996; Han et al., 1996;
Han-dler et al., 2019; Himmelreich et al., 2017). On the other hand, DopR1 receptors are involved in the pathway that detects the approximate coincidence of the two inputs in an order-independent way. DopR1 activation is associ-ated with Gαs signalling that, together with Kenyon cell depolarisation through projection neuron input, stimu-lates rutabaga adenylyl cyclase (Boto et al., 2020; Heisen-berg, 2003; Livingstone et al., 1984; Zars, 2000; Zars et al., 2000) to synthesise cyclic adenosine monophos-phate (cAMP) (Gervasi et al., 2010; Sugamori et al., 1995; Tomchik and Davis, 2009). In the absence of endoplas-mic reticulum Ca2+release, cAMP signalling induces LTD;
if Ca2+release does take place alongside cAMP signalling,
LTP occurs (Handler et al., 2019; Modi et al., 2020). This suggests that the detection of the approximate coinci-dence of the two signals might enable a synapse to un-dergo plasticity, and the order of the signals then deter-mines whether this plasticity involves LTD or LTP.
Such distinct processes, for the coincidence and or-der detection of the two inputs, that collaborate to deter-mine downstream effects might also function in cerebel-lar plasticity. However, both in the mushroom body and in the cerebellum, repetitive stimulation of one input alone can also induce LTP. This is caused by parallel fibre stim-ulation in the cerebellum (Chen and Regehr, 1997; Gao et al., 2012; Hoxha et al., 2016; Salin et al., 1996) and dopaminergic stimulation in the mushroom body (Berry et al., 2018; Cohn et al., 2015; Hattori et al., 2017; Modi et al., 2020). Therefore, the coincidence of two inputs is not always necessary to induce plasticity, illustrating the complexity that is associated with the different signalling cascades and their interactions.
Key components, that are involved in the down-stream processes of presynaptic plasticity in the mush-room body and the cerebellum, are cAMP, Ca2+, and
pro-tein kinase A (PKA), that modulate the functioning of the presynaptic vesicle release machinery (Chavis et al., 1998; Chen and Regehr, 1997; Pavot et al., 2015; Seino and Shibasaki, 2005). In the cerebellum, it was shown that processes leading to the activation of PKA induce presy-naptic LTP in the parallel fibres (Gao et al., 2012; Pow-ell et al., 2004; Qiu and Knöpfel, 2007; Salin et al., 1996) and processes inhibiting PKA prevent this LTP (Chu et al., 2014; Qiu and Knöpfel, 2009); however, presynaptic LTD could only be induced if PKA was pharmacologically in-hibited. I consider it likely that the DopR1 and DopR2 sig-nalling pathways in the fruit fly mushroom body also dif-ferentially modulate downstream mechanisms–including PKA activation—to antagonistically affect plasticity. It is worthwhile to focus on these processes, their interac-tions, and the downstream effects in future research.
5.3
Transsynaptic modulation of plasticity
The signalling pathway, that inhibits presynaptic potenti-ation in the cerebellum by suppressing PKA activity, ap-pears to rely on retrograde transmission from the Purk-inje cell to the parallel fibres. Climbing fibre activity stim-ulates the production and release of endocannabinoids from Purkinje cells, activating G-protein coupled cannabi-noid 1 receptors on the parallel fibres; downstream pro-cesses subsequently suppress PKA activity (van Beugen et al., 2006; Brown et al., 2003; Gao et al., 2012; Hoxha et al., 2016; Maejima et al., 2005, 2001). This illus-trates one way in which plasticity mechanisms at differ-ent synaptic loci can influence each other to closely reg-ulate synaptic strength. It is likely that such mechanisms exist in the mushroom body as well, probably involving the dopaminergic neurons, since these synapse onto both Kenyon cells and MBONs. Kenyon cells also project back to dopaminergic neurons (Takemura et al., 2017), there-fore, Kenyon cell activation can theoretically influence dopaminergic transmission to MBONs, and thus modu-late potential postsynaptic plasticity processes.
Additionally, several dopaminergic neurons can syn-thesise nitric oxide (NO), and both Kenyon cells and MBONs express NO receptor subunits (Aso et al., 2019). It was demonstrated that the behavioural effects related
to dopaminergic neuron stimulation were reversed when dopamine synthesis was prevented (Aso et al., 2019). This suggests that NO induces plasticity phenotypes opposite to those induced by dopamine. Since NO is gaseous, it can readily cross membranes and does not need to be re-leased via vesicular transmission. Therefore, suprathresh-old activation of dopaminergic neurons is not critical for NO signalling. This implies that Kenyon cell firing that induces synaptic transmission to dopaminergic neu-rons might be sufficient to activate NO signalling from dopaminergic neurons to MBONs; NO and dopamine modulation of MBONs may thus be provoked in different situations and have different effects.
In the cerebellum, NO signalling plays a role in nonas-sociative presynaptic LTP. NO-dependent LTP is induced in presynaptic terminals that are not active themselves, if NO is synthesised in activated presynaptic terminals and carried to the inactive terminals by diffusion (Gao et al., 2012; Jacoby et al., 2001; Qiu and Knöpfel, 2007). NO signalling by dopaminergic neurons may also reach inactive synapses to induce LTP in the mushroom body. The extent to which retrograde or secondary transmis-sion of (combinations of) signalling molecules plays a role in the different structures is still unclear, as well as the specific molecules that play a role in these systems, which can include endocannabinoids, NO, and other modulators like neuropeptides. I consider it likely that the processes related to transsynaptic plasticity cooperate to carefully tune synaptic strength. Future research is necessary to provide an integrative view of these mechanisms in the different structures.
5.4
Sign inversion
A computationally trivial factor in adaptive filter models is that the signs of the gains can change, which is impor-tant to be able to construct the inverse of a reference sig-nal that contains both positive and negative values. In neuronal circuits this task is more complex, since the sign of a synaptic connection depends on the available neuro-transmitters and receptors, which generally do not switch between excitation and inhibition (Dean et al., 2010). The input signals are therefore duplicated to generate an
ex-citatory and an inhibitory pathway that both converge on principal cells (Albus, 1971). In the cerebellum, the in-hibitory pathway is governed by feedforward inin-hibitory interneurons that receive input from parallel fibres and transmit γ-aminobutyric acid (GABA) to Purkinje cells.
Feedforward inhibitory interneurons are often grouped together as molecular layer interneurons (Dean et al., 2010; Gao et al., 2012), but noteworthy differ-ences exist between the two main types; stellate cells target the dendrites of Purkinje cells (Palay and Chan-Palay, 1974b), whereas basket cells target the soma and the axon initial segment (Palay and Chan-Palay, 1974a; Somogyu and Hámori, 1976). Likely due to these spa-tial differences, basket cells induce stronger inhibitory responses in Purkinje cells than stellate cells (Arlt and Häusser, 2020; Bao et al., 2010). Therefore, single bas-ket cells can potently counteract the excitatory input from many parallel fibres and prevent Purkinje cell firing, whereas single stellate cells influence Purkinje cell output less. Taking the perspective of the adaptive filter, in my opinion the stellate cells are the most likely candidates to provide the inhibitory elements of the expanded basis signal, since they target the Purkinje cell dendrites like the parallel fibres do. Conversely, basket cells might have a strong modulatory function that can facilitate the syn-chronisation of adjacent Purkinje cells (Bao et al., 2010), potentially warranting revisions of existing adaptive filter models to include two types of inhibition.
In the cerebellum, parallel fibre–interneuron and interneuron–Purkinje cell synapses are both subject to plasticity and appear to be modulated, at least to some extent, by climbing fibres (Dean et al., 2010; Gao et al., 2012). For instance, it has been demonstrated that climbing fibres directly contact molecular layer interneu-rons and can evoke responses in these cells (Jörntell and Ekerot, 2003, 2002; Sugihara et al., 1999)—stellate and basket cells were not distinguished. Parallel fibre stimula-tion can induce LTD and LTP at parallel fibre–interneuron synapses; a balance that was strongly shifted to LTP when stimulation was paired with interneuron depolarisa-tion, simulating climbing fibre input (Rancillac and Crépel, 2004; Smith and Otis, 2005). These synapses therefore show hallmarks of Hebbian plasticity involving
associa-tive LTP and nonassociaassocia-tive LTD. This is in line with the theoretical perspective that the gain of inputs that corre-late with the reference signal should be reduced in adap-tive filters; in this case, the inhibitory signal to Purkinje cells will be potentiated which makes the gain more neg-ative (Dean et al., 2010). However, most synapses in the circuit are subject to plasticity, and the plasticity require-ments do not always correspond clearly with the theoreti-cal covariance learning rules of adaptive filters (Gao et al., 2012; Mapelli et al., 2015). I assume that the system, as a whole, still conforms to the adaptive filter theory, but it is important for future studies to focus on plasticity in multiple synapses at the same time to understand their interactions better.
Besides the cerebellum, cells providing feedforward inhibition to principal cell dendrites (like stellate cells) have been described for all vertebrate cerebellum-like structures, whereas cells providing inhibition to axons (like basket cells) have been observed only in some struc-tures (Bell et al., 2008), further supporting the hypothesis that feedforward inhibition to the dendrites of principal cells carries the inhibitory portion of the expanded basis signal to adaptive filters, whereas inhibition to axons may play a different role. For the fruit fly mushroom body, the situation appears to be different. The candidate neurons to provide feedforward inhibition, that have been identi-fied thus far, resemble basket cells more than stellate cells, in the sense that they appear to project to MBON axons instead of dendrites. The putative feedforward inhibitory MBONs, that project to different mushroom body com-partments instead of targets outside of the mushroom body, are the GABAergic MBON–γ1pedc>α/β, and the glutamatergic MBON–γ4>γ1γ2 and MBON–β1>α (Aso et al., 2014a,b). In contrast to the vertebrate brain, glu-tamate is generally considered to be an inhibitory neuro-transmitter in the fly brain (Liu and Wilson, 2013). There-fore, these neurons presumably provide feedforward in-hibition to the MBONs that they project to (Aso et al., 2014b). That these neurons project to MBON axons instead of dendrites, was supported by a recent study that examined one of these neuron types in more detail. Using optogenetics and calcium imaging, it was demon-strated that MBON–γ1pedc>α/β provides feedforward
inhibition to the axons of MBON–γ5β’2a/MBON–β’2mp (Perisse et al., 2016). This circuit, in which signals are car-ried from excitatory Kenyon cells to inhibitory MBONs and then to the axons of other MBONs, clearly resem-bles the way in which basket cells are incorporated in the cerebellar circuit—basket cells receive excitatory parallel fibre input and provide inhibition to the soma and axon of Purkinje cells. As mentioned before, such connectivity has also been described for cells in some cerebellum-like structures, including the inhibitory medium ganglion cells that project to the somas of excitatory efferent cells in the mormyrid (but not gymnotid) electrosensory lobe (Han et al., 1999; Sawtell, 2017) (Figure 2C), and the inhibitory cartwheel cells that project to the somas and proximal api-cal and basilar dendrites of pyramidal cells in the dorsal cochlear nucleus (Berrebi and Mugnaini, 1991; Sawtell, 2017). The feedforward inhibitory MBONs, medium gan-glion cells, cartwheel cells, and to a lesser extent the bas-ket cells, all receive synaptic input from many intrinsic cells and also have access to the reference signal (Aso et al., 2014a; Han et al., 1999; Lemkey-Johnston and Larramendi, 1968; Oertel and Young, 2004); these cells are thus also centres where adaptive filtering can take place. The outputs of these cells, that are available to principal cells, are therefore likely to be only distantly re-lated to specific sensory inputs. This does not conform to the characteristics of the inputs included in the ex-panded basis signal of an adaptive filter. The inhibitory stellate cells—that receive input from fewer intrinsic cells than basket cells and project to principal cell dendrites— together with the excitatory intrinsic cells, are therefore more likely to encompass the expanded basis signal.
More thorough anatomical and functional analyses of the feedforward inhibitory MBONs are necessary to investigate whether they all solely target MBON axons, and to analyse in what way they precisely influence these downstream cells. Currently, it seems likely that most MBONs do not receive an arrangement of dendritic hibitory inputs alongside their excitatory Kenyon cell in-puts. Therefore, it is implausible that MBONs can con-struct negative images like those found in the electrosen-sory lobe, that fully cancel out the reference signal af-ter sufficient learning (Bastian, 1996; Montgomery and
Perks, 2019; Requarth and Sawtell, 2011; Roberts and Bell, 2000; Sawtell, 2017, 2010). This illustrates that, whereas the adaptive filter function in the electrosensory lobe is tailored to reafference filtering—for which pos-itive and negative dendritic inputs with dynamic gains are crucial—this is not the main function of the mush-room body. Instead, the mushmush-room body serves to modu-late approach and avoidance behaviours based on learned associations between sensory inputs and their valences (Aso et al., 2014b; Boto et al., 2020; Owald et al., 2015), for which it is probably not necessary to cancel out the dopaminergic reference signal. Instead, the inhibitory ax-onal projections may play a crucial role to drive only one behaviour at a time, by enabling compartments related to approach and avoidance to mutually inhibit each other (Aso et al., 2014b; Boto et al., 2020; Cohn et al., 2015; Owald et al., 2015). It would be interesting to investigate whether such a mechanism, by which different adaptive filter units modulate each other, also exists in the cere-bellum; the basket cells would be likely to be involved in this.
6
| CONCLUDING REMARKS AND
FUTURE DIRECTIONS
In the preceding text, comparisons between the cerebel-lum, cerebellum-like structures, and the mushroom body have been made at various levels, ranging from anatom-ical circuit characteristics, to neuronal physiology, and to plasticity mechanisms. Overall, the mushroom body shows many characteristics of a structure that is involved in adaptive filtering. It has access to the two important input types: a vast array of sensory information that comprises the basis signal and is carried by projection neurons, and the instructional reference signal provided by dopaminergic neurons. The basis signal is likely in-creased in complexity at the transition to Kenyon cells, that each receive a semi-random sample of projection neuron inputs (Caron et al., 2013; Gruntman and Turner, 2013; Zheng et al., 2020). Different combinations of in-puts, together with variation in biophysical properties be-tween Kenyon cells, might be sufficient to produce an
ex-panded basis signal with a variety of delays in the fruit fly mushroom body. This will need to be addressed in future research, by measuring the input processing prop-erties in multiple Kenyon cells at the same time, for in-stance by using voltage indicators, which can also resolve subthreshold responses (Chamberland et al., 2017; Yang et al., 2016).
Dopaminergic neurons project to the presynaptic Kenyon cells and postsynaptic MBONs (Takemura et al., 2017) and are thus in the appropriate position to influ-ence plasticity of these synapses. Presynaptic plasticity processes have been described, these follow the covari-ance rule of associative LTD and nonassociative LTP (Aso and Rubin, 2016; Cassenaer and Laurent, 2007; Handler et al., 2019; Hige et al., 2015; Tanimoto et al., 2004). Based on the connections from dopaminergic neurons to MBONs, postsynaptic plasticity processes are likely to ex-ist as well, which needs to be focussed on in future search. A relevant first step would be to investigate re-ceptor trafficking mechanisms, which play a major role in cerebellar postsynaptic plasticity (Coesmans et al., 2004; Gardner et al., 2005; Lev-Ram et al., 2002; Steinberg et al., 2004; Wang and Linden, 2000; Xia et al., 2000).
DopR1 and DopR2 receptors are implicated in dif-ferent signalling cascades in the presynaptic Kenyon cell terminals, probably involving detection mechanisms for input coincidence and input order, respectively (Handler et al., 2019). How the downstream signals interact to eventually produce either LTP or LTD is unclear, but IP3
receptors, cAMP, and PKA are likely to play a role (Gervasi et al., 2010; Modi et al., 2020; Tomchik and Davis, 2009). These elements are also implicated in cerebellar plastic-ity, but they are incorporated differently in upstream and downstream pathways, so translating these insights to the mushroom body should be met with caution.
Another aspect that appears to differ between the cerebellum and cerebellum-like structures on the one hand, and the mushroom body on the other hand, is sign inversion. Even though there are indications that certain MBONs provide feedforward inhibition to other MBONs (Aso et al., 2014a,b; Perisse et al., 2016), the current data suggest that these MBONs carry highly processed signals that are only distantly related to sensory input patterns,
and that they synapse onto axons instead of dendrites. If feedforward inhibition of MBON dendrites is indeed absent—which future research will need to determine— I consider it likely that the basis signal available to the mushroom body is governed mainly by the excitatory den-dritic synapses of Kenyon cells. The axonal inhibition probably modulates MBON output to a much larger ex-tent than dendritic inhibition would—like the cerebellar basket cells, electrosensory lobe medium ganglion cells, and dorsal cochlear nucleus cartwheel cells compared to stellate cells (Arlt and Häusser, 2020; Bao et al., 2010; Han et al., 1999). More detailed studies of dendritic and axonal inhibition are necessary to determine whether these indeed perform different functions for adaptive fil-ter structures. If this is the case, current adaptive filfil-ter models might require revision to include two different types of feedforward inhibition.
The adaptive filter mechanisms in different structures appear to be adjusted to specialised functions. For in-stance, the cerebellar flocculus is tuned to reafference fil-tering of visual slip brought about by voluntary eye move-ments (Ito, 1982; Ito et al., 1982a; Maekawa and Simpson, 1972; Noda, 1986), the electrosensory lobe of weakly electric fish is important for the filtering of electric organ reafference (Bastian, 1996; Bell, 1982; Bell et al., 1993; Requarth and Sawtell, 2011), and the fruit fly mushroom body is imperative for learning to predict the value as-sociated with sensory input patterns to drive approach and avoidance behaviours (Aso et al., 2014b; Boto et al., 2020; Owald et al., 2015). However, the mushroom body might be involved in reafference filtering functions as well. In the cockroach, MBONs were shown to receive effer-ence copy and proprioceptive inputs (Farris, 2011; Mizu-nami et al., 1998; Okada et al., 1999), and in the fruit fly, some of the antennal lobe projection neurons respond to mechanosensory stimuli, potentially playing a role in active and passive movements of the antennae (Mamiya et al., 2011, 2008; Wilson et al., 2004), for which reaffer-ence filtering mechanisms would be beneficial.
It should be noted that, even though adaptive filter circuits appear well-equipped to perform reafference fil-tering, other systems can also perform this function. This has for instance been suggested for the fruit fly optic
lobe, in which efference copies appear to play a role in visual reafference filtering (Kim et al., 2015, 2017). More-over, adaptive filter circuits are embedded in larger neu-ral systems, in which other filtering mechanisms can also operate dynamically on sensory input. For instance, re-duced responding to ‘background’ sensory cues by ha-bituation is important to better detect novel stimuli, and can take place at many processing levels independent of the adaptive filter structures. It was shown that the re-sponses of olfactory projection neurons in the fruit fly readily adapt to repetitive stimuli, due to potentiation of inhibitory neuron synapses that provide negative feed-back (Ramaswami, 2014; Shen et al., 2020), which did not rely on a reference signal.
These considerations culminate in the view that a so-called subsumption architecture underlies the way adap-tive filter circuits are embedded in neural systems (Brooks, 1986; Montgomery and Perks, 2019; Yopak et al., 2010). The structures have been proposed to have evolved on top of an existing, functional brain. Therefore, they are not crucial for the basic functions necessary for sur-vival, but the adaptive filters enhance the computational power of the existing brain to make more complex tasks possible (Montgomery and Perks, 2019; Wessnitzer and Webb, 2006). This perspective is substantiated by the no-tion that, for all sensory modalities that the cerebellum, cerebellum-like structures, and the mushroom body re-ceive, other, more direct sensorimotor processing path-ways exist alongside this adaptive filter pathway. In con-clusion, the mushroom body can be regarded as an adap-tive filter, like the cerebellum and cerebellum-like struc-tures. Bearing in mind the differences between these structures, insights in the working mechanisms of the fruit fly brain—which are facilitated by the variety of ge-netic tools available to manipulate and analyse specific neurons—will be valuable to understand the vertebrate brain as well.
7
| ACKNOWLEDGEMENTS
I thank Lukas Groschner for being an amazing supervisor and a very admirable scientist, which was immensely
