Bachelor’s thesis
Performing a sensorimotor perception task with FORCE trained
recurrent spiking neural networks
Hendrik Scheeres
22 January 2021
Student number: 11228962
Program: BSc Psychobiology, University of Amsterdam Supervisor: Fleur Zeldenrust
Department: Neurophysiology
Institute: Donders Institute for Brain, Cognition, and Behaviour,
Radboud University Nijmegen
Abstract
Underlying complex animal behaviours are intricate networks of spiking neurons with chaotic dynamics. The recently developed training method for recurrent neural networks, FORCE learning, has made it possible to create functional models of these spiking networks while preserving the network chaos. Winter (2020) implemented FORCE learning to create a functional model of a sensorimotor task performed by mice. The network did succeed in learning the task, however, its performance was unstable. We set out to improve the performance reliability of the network by taking a deeper look at the input the network receives. While we were unable to improve the performance by training the network on different input signals, we were able to increase the performance variability by altering the noise in the input signal. Thus, demonstrating the complex interplay between the chaotic dynamics and performance of FORCE trained networks.
1
Introduction
Humans and other animals have the ability to actively interact with the world around them. Through the processing of information obtained through their senses, they can make decisions and perform a high variety of complex tasks. A prime example of information processing in animals, is the whisking behaviour in mice. Through active whisking, mice sense their immediate surroundings and are able to act based on this information; either by pursuing an interesting stimulus or avoiding an unpleasant one. Underlying these behaviours are intricate networks of interconnected spiking neurons in the brain, in particular the somatosensory cortex. Researching these networks gives us an important insight into the neural correlates of animal behaviour.
To better understand how neural networks are able to process information and learn specific behaviours, it has become common practice to study Artificial Neural Networks (ANNs). Loosely modelled after networks of neurons in the brain, ANNs are networks of processing units able to receive and send signals to other connected units. Through adjusting the connection strengths or ‘weights’ between the neurons, it is possible to train a network to a certain behaviour. In particular Recurrent Neural Networks (RNNs), a class of ANNs, have proven to be useful tools for modeling and interpreting neurobiological phenomena (Barak, 2017). Not only do RNNs reflect the recurrent connectivity in the brain, but they are also able to generate rich internal dynamics, similar to chaotic cortical activity (Vreeswijk & Sompolinsky, 1996). Throughout the years, different methods to train RNNs have been developed (Atiya & Parlos, 2000; Rumelhart & McClelland, 1987). Early methods based on gradient descent algorithms were able to train RNNs, yet had serious limitations: Besides being computationally expensive, these algorithms were also unable to train RNNs with chaotic activity (Doya, 1992), deeming them biologically unrealistic.
To overcome the inability to train networks with chaotic activity, new structures and learning schemes were developed for RNNs (Jaeger, 2001; Maass et al., 2002), later collectively referred to under the Reservoir Computing (RC) (Lukoševičius & Jaeger, 2009). In its basic form, RC refers to a reservoir of neurons with random recurrent connections. The network output is a linear combination of the activity of the neurons in the reservoir modified by output weights. During supervised training only these output weights are updated to approximate a target signal, while the weights of neurons in the reservoir remain fixed. As an extension of RC, Sussillo & Abbott (2009) created the FORCE learning algorithm to effectively update the output weights of the reservoir during training. As the name First Order Reduced Controlled Error (FORCE) suggests, at the start of training, the output weights are rapidly modified to approximate the target signal as much as possible, thereby significantly minimizing the error. Throughout the rest of training, the output weights are constantly updated to keep the error small. This permits the output to be fed back into the network, without causing a deviation from the desired network activity. In their research, Sussillo & Abbott (2009) were able to demonstrate that
2 FORCE learning as an extension of RC is an effective method for enforcing behaviour on networks with chaotic activity.
Recent research has shown that FORCE learning is also applicable to more biologically realistic networks. For example, Nicola & Clopath (2017) and Thalmeier et al. (2016) were able to FORCE train networks of spiking neurons, as opposed to the rate-based models used in Sussillo & Abbott (2009). Furthermore, Nicola and Clopath (2017) demonstrated, that through FORCE training, networks were also able to learn biologically inspired behaviours. In their research, they were able to create networks capable of reproducing songbird singing or replaying an episodic memory.
To further extend the use of FORCE learning on biologically inspired networks, Winter (2020) created a network of spiking neurons inspired by the mouse barrel cortex to perform a sensorimotor whisking task, using a similar framework to Nicola & Clopath (2017). The task the network was trained on, was derived from experimental research by Peron et al. (2015). In their research, head-fixed mice were trained to distinguish between proximal and distal poles by using a single whisker. During the task, the angle and curvature traces of the whisker were recorded. Winter (2020) trained a network to perform the same task, based on the recorded whisking traces. Before being fed into the network, the whisker traces were converted into thalamic spikes using a model of the thalamus by Huang et al. (2020), adding to the biological realism of the network. The reservoir itself was loosely modelled after layer 2 / 3 of the rodent barrel cortex, in that it had a similar amount of neurons found in a single layer of a barrel. Through accurate scaling of the synaptic weights, the network could be brought in a trainable regime with test accuracies up to 80%, comparable to the deep learning benchmark on the same task. High test accuracies, however, were rare, and more often trained networks had a test accuracy of 50%. Considering that the network was trained to a binary classification task with an equal amount of proximal and distal trials, an accuracy of 50%, in most cases, meant the network had not learned to classify the input, but was simply repeating the same learnt output signal for each trial. Once within the trainable regime, networks trained with identical sets of hyperparameters showed varying test accuracies. This made it hard to relate the network performance to the scaling of the weights with hyperparameters.
As an extension of the work of Winter (2020), we set out to create a functional model of the mouse barrel cortex able to perform a sensorimotor task with more reliable test accuracy. Firstly, this was done by taking a deeper look into the input the network receives. As mentioned earlier, the networks in Winter (2020) were only trained on the thalamic spikes that were made from the recorded whisking traces. It was therefore interesting to see how the network performance would be affected by other forms of input. To do this, we created two new input signals, derived from different stages of the conversion from whisker traces to thalamic spikes; the convolved-signal, and the PSTH-signal. After properly scaling the new input signals, we trained multiple networks on all three input conditions to study how they would affect the network dynamics and performance. Secondly, we looked at the role of background noise in the input signal. The thalamic spikes the network was originally trained on,
3 consisted of two segments. The first segment referred to as the input-based spikes, were made from whisking traces through the use of the thalamus model mentioned earlier. These input-based spikes were prolonged using a segment of random Poisson distributed spikes, referred to as Poisson-based spikes. These Poisson-based spikes served as background noise and had an adjustable average firing rate, the Poisson-rate. While testing the network on different parameters, we encountered a network with an adjusted Poisson-rate that had a remarkably high test accuracy. To explore the influence of the Poisson-rate on the network test accuracy we trained multiple networks with varying Poisson-rates. Finally, we also studied the effect of extending the background noise over the entire input signal by FORCE training multiple networks with masked input.
Methods
Sensory perception task and whisking data
The task and data used to train the network were derived from experimental research by Peron et al. (2015). In their experiment, head-fixed mice were trained to discriminate a pole location through single whisker detection using a scheme by Guo, Hires, et al. (2014) and Guo, Li, et al. (2014). Specifically, the mice were trained to indicate the location of the pole by choosing between two lickports. The pole was either presented in a range of proximal locations, predicting a reward at the right lickport, or a distal location, predicting a reward at the left lickport. Through automated whisker tracking, the whisker position indicated by the azimuthal angle (θ), and the touch induced curvature (κ) were recorded during trials. The times the whisker touched the poles were also measured. Logically, the whisker touched the pole more frequently during proximal trials causing more variation in the curvature traces in contrast to distal trials. The dataset from the animal with ID = 171923 was used, which was made available at the CRNCNS database of the Svoboda lab (S. Peron et al., 2014). Before training, the data was filtered on correct trials, leaving two trial conditions: 867 Proximal trials with pole touch times and 1054 distal trials with no pole touch times.
Figure 1:A visualization sensory perception task setup copied from Peron et al. (2015). a. A schematic setup of the task. The head fixed mouse has to locate the pole location with its whisker. Classification is done by either choosing the right lickport for proximal pole locations (blue), or the left port for distal pole locations (red). b. A snapshot of the whisker location during a trial, with the whisker angle (𝜃) and touch induced curvature (𝜅).
4
Network input
The whisker traces were converted into thalamic spikes using a filter model of the rodent thalamus by Huang et al. (2020). First, the angle and curvature traces were convolved using 200 thalamic kernels. Subsequently, the activity of each thalamic neuron was modelled using filters and activation functions based on work by Petersen et al. (2008). Neurons responded either to the curvature, the angle, or a combination of both. The neuron activity was given as a peristimulus time histogram (PSTH) signal that was used to create the spike trains of each thalamic neuron. The network was separately trained with the three different inputs, forming the convolved signal, the PSTH signal, and the thalamic spikes conditions. The whisker traces and network input conditions all had a resolution of 1 ms.
Figure 2: The curvature (𝜅) traces and the angle (𝜃) traces over time (𝑚𝑠) of a proximal trial are convolved using 200 thalamic kernels. Through filters and activation functions based on the work of Petersen et al, (2008) the convolved signals are translated into PSTH signals which are subsequently used to create the neuron spike trains. The network is separately trained on the convolved signals, the PSTH signals, and the thalamic spikes, forming three conditions. In the figure, the average convolved signal and average PSTH signal of 200 thalamus neurons are given over time (𝑚𝑠). The thalamic spikes are given as a raster plot with 200 neuron spike trains over time in (𝑚𝑠).
Network architecture
The network reservoir was made up of 2000 spiking neurons. This was not only a realistic number of neurons for a single layer of a single barrel, but also suitable to bring the network into a balanced state (Huang et al., 2020). The neurons in the reservoir received input from the thalamus neurons, which were connected through a 200 x 2000 weight matrix, referred to as the input weights. The connection strengths were randomly drawn from a uniform distribution [0 1] and could be further scaled using the hyperparameter Win.The remaining architecture of the network was based on the framework of Nicola & Clopath (2017). As a model for spiking neurons, the leaky integrate and fire (LIF) model was chosen, described by the following equation:
𝜏𝑚
𝑑𝑣𝑖
𝑑𝑡 = −𝑣𝑖+ 𝐼,
5 where the membrane time constant, the voltage of the ith neuron, and the currents are given by 𝜏𝑚
,
𝑣𝑖and𝐼 respectively. The input current I is given by
𝐼 = 𝐼𝐵𝐼𝐴𝑆+ 𝑠𝑖,
(2)
where 𝐼𝐵𝐼𝐴𝑆 is a background current that modifies the firing rate and 𝑠𝑖 the synaptic currents. To ensure
enough activity in the network, the bias was set to the firing threshold of the neurons (−40𝑚𝑉
)
. The synaptic currents, 𝑠𝑖, arriving at the ith neuron are given by𝑠𝑖 = ∑ 𝑤𝑖𝑗𝑟𝑗 𝑁
𝑗=1
,
(3)
where 𝑟𝑗 signifies the filtered spikes of the jth neuron and 𝑤𝑖𝑗 is the overall synaptic weight matrix that
modifies the magnitude of the postsynaptic currents arriving at neuron i from neuron j. The synaptic weight matrix was made up of three components as can be seen in Figure 3 and the following equation:
𝑤𝑖𝑗= 𝐺𝑤𝑖𝑗0 + 𝑄𝜂 ∙ 𝜙𝑗𝑇.
(4)
The reservoir weights 𝑤𝑖𝑗0 (black), or static weights, are added up to the feedback weights 𝜂 (green) multiplied by the (learnt) output weights 𝜙𝑗𝑇 (red). The static and feedback weights are scaled by the
hyperparameters 𝐺 and 𝑄 respectively.
Figure 3: A simplified diagram of the network architecture from Winter (2020). The input, modulated through the input
weights (blue), is given to a sparse recurrent connected reservoir of neurons with static weights 𝑤𝑖𝑗0 (black) scaled by
parameter 𝐺. The network output, modified by the output weights 𝜙𝑗𝑇 (red), is the linear sum of the activity in the
reservoir into a readout unit. The network output is then propagated back to the network through the separate feedback 𝜂 weights (green) scaled by parameter 𝑄. Only the output weights are updated during training.
6 The static weights 𝑤𝑖𝑗0 were initialized in a 2000 x 2000 matrix with a sparsity of 𝑝 = 0.1 and were drawn from a random normal distribution with mean 0 and variance(𝑁𝑝)−1
,
where 𝑁 is the number ofneurons. To induce chaotic spiking, the sample mean of the static weight matrix was explicitly set to 0. The feedback weights 𝜂 were drawn uniformly and randomly from [−1 1]𝑘, with 𝑘 being the
dimensionality of the target signal which in our case 𝑘 = 1, resulting in a 1 x 2000 matrix.
After firing, the neuron spikes are filtered through a double exponential filter:
𝑟̇𝑗= − 𝑟𝑗 𝜏𝑑+ ℎ𝑗,
(5)
ℎ̇𝑗= − ℎ𝑗 𝜏𝑟+ 1 𝜏𝑟𝜏𝑑∑𝑡𝑗𝑘<1𝛿(𝑡 − 𝑡𝑗𝑘),(6)
here the spikes are represented by the Dirac delta function where 𝑡𝑗𝑘 is the kth spike fired by the jth
neuron. The synaptic rise and decay times are given by𝜏𝑟 = 2 and 𝜏𝑑= 50 respectively. The network
output 𝑧̂ is the linear combination of the filtered spikes 𝑟𝑗 multiplied by the 2000 x 1 output weights
matrix: 𝑧̂(𝑡) = ∑ 𝜙𝑗𝑟𝑗 𝑁 𝑗=1 .
(7)
In turn, this is multiplied by the feedback weights and added to the filtered recurrent and input spikes, to make up the summed synaptic input to all neurons. The recurrent spikes in the reservoir and the thalamic input spikes are also filtered using the same double exponential filter given in equations 5 and 6. This results in the following equation for the first exponential filter:
ℎ̇ = − ℎ 𝜏𝑑 + ∑ 𝑤𝑖𝑗 0𝛿(𝑡 − 𝑡 𝑗𝑘) 𝑡𝑗𝑘<1 𝜏𝑟𝜏𝑑 +∑𝑡𝑡𝑘<1 𝐸𝑖𝑡𝛿(𝑡 − 𝑡𝑡𝑘) 𝜏𝑟𝜏𝑑 .
(8)
The first term is part of the filter, the second term represents the spikes in the reservoir modified by the static weights 𝑤𝑖𝑗0
,
and the third term signifies thalamic spikes modified by the input weights 𝐸𝑖𝑡.7 𝑠𝑖 = ∑(𝐺𝑤𝑖𝑗0 + 𝑄𝜂 ∙ 𝜙𝑗𝑇)𝑟𝑗+ 𝑁 𝑗=1 ∑ 𝐸𝑖𝑡𝑟𝑡 𝑇ℎ 𝑡=1 .
(9)
While the aforementioned equations relate to the condition where thalamic spikes are given as network input, only small adjustments needed to be made to implement the convolved signal or PSTH signal as network input. Instead of being filtered with the network spikes, the input modified by the input weights is directly added to the synaptic currents 𝑠𝑖:
𝑠𝑖 = ∑(𝐺𝑤𝑖𝑗0 + 𝑄𝜂 ∙ 𝜙𝑗𝑇)𝑟𝑗+ 𝑁 𝑗=1 ∑ 𝐸𝑖𝑡𝑥𝑡 𝑇ℎ 𝑡=1 ,
(10)
where the signal for thalamus neuron t is given by 𝑥𝑡.
FORCE learning
The objective of FORCE learning is to dynamically update the output weights to minimize the squared error between the network output 𝑧(𝑡) and the target function 𝑧̂(𝑡), 𝑒(𝑡) = 𝑧̂(𝑡) − 𝑧(𝑡). This is done by applying the Recursive Least Squares (RLS) algorithm:
𝜙(𝑡) = 𝜙(𝑡 − Δ𝑡) − 𝑒(𝑡)𝑃(𝑡)𝑟(𝑡),
(11)
𝑃(𝑡) = 𝑃(𝑡 − Δ𝑡) − 𝑃(𝑡 − Δ𝑡)𝑟(𝑡)𝑟(𝑡)
𝑇𝑃(𝑡 − Δ𝑡)
1 + 𝑟(𝑡)𝑇𝑃(𝑡 − Δ𝑡)𝑟(𝑡) ,
(12)
where 𝑟(𝑡) refers back to the filtered spikes, and 𝑃(𝑡)
,
the estimate of the inverse of the correlation matrix, is formed as a product of the N x N identity matrix 𝐼𝑛 and the inverse of the learning rate 𝜆:𝑃(𝑡) = 𝛼𝐼𝑛,
(13)
𝜆 = 1 𝛼.
(14)
The network had an integration step time of 𝑑𝑡 = 0.5 𝑚𝑠 and the output weights were updated every 20 integration steps.
The task at hand was a binary classification task, so a target signal was designed that followed a positive pulse for the proximal trials and a negative pulse for distal trials. The target pulse was presented 500 𝑚𝑠 after the trial onset as a constant value until 100 𝑚𝑠 after the trial input was finished, followed by an exponential decay to zero over 1000 𝑚𝑠. This left the network with room to adjust to the input, just as the mouse would not instantly make its choice after trial onset. FORCE training was only applied
8 while the target pulse was presented, to ensure learning only took place when there was something to learn. An example of a single trial can be seen in Figure 4. To test if the network had successfully classified the input, the mean output was calculated during the presence of the target pulse. A positive mean output corresponded with a proximal trial and a negative output with a distal trial. The test accuracy could then be calculated by comparing the mean network output to the accurate classification over all test trials.
Figure 4: An example from Winter (2020) of the target signal and the angle trace input (blue) of a random trial. The
positive supervisor pulse is indicated by the green dotted lines, during which the FORCE learning is applied.
Both the network scaling parameters and training scheme were based on the work done by Winter (2020).The scaling parameters 𝐺, 𝑄, 𝑊𝑖𝑛 were set to what was found to be within a trainable regime:
1 < 𝐺 < 15, 𝑄 = 1 and 𝑊𝑖𝑛 = 0.5, and were similar for all trained networks. The training scheme
consisted of 600 train trials, during which FORCE learning was applied, and 100 test trials, to calculate the test accuracy of the network. This was repeated for two epochs. Both the training and test trials were randomly chosen from the filtered trials and had an even amount of proximal and distal trials. An overview of the network parameters can be found in Table 1. All simulations were done in MATLAB® of which the code is available on GitHub
(https://github.com/DepartmentofNeurophysiology/Spiking_force_learning_barrel_cortex). We also made use of the Neuroscience Gateway portal for extra computation time (Sivagnanam et al., n.d.).
9
Parameter
Value
𝑵 2000 𝑵𝒕𝒉𝒂𝒍𝒂𝒎𝒖𝒔 200 𝝉𝒎 10 𝑚𝑠 𝝉𝒅 50 𝑚𝑠 𝝉𝒓 2 𝑚𝑠 𝑰𝑩𝑰𝑨𝑺 −40 𝑉 𝒕𝒓𝒆𝒇 (refractory period) 2 𝑚𝑠 𝒗𝒓𝒆𝒔𝒆𝒕 −65 𝑉 𝒗𝒕𝒉𝒓𝒆𝒔𝒉𝒐𝒍𝒅 −40 𝑉 𝒔𝒕𝒆𝒑 20 𝒅𝒕 0.05 𝑚𝑠 𝒑𝒐𝒊𝒔𝒔𝒐𝒏-𝒓𝒂𝒕𝒆 3𝐻𝑧 𝑵𝒕𝒓𝒂𝒊𝒏 600 𝑵𝒕𝒆𝒔𝒕 100 𝒆𝒑𝒐𝒄𝒉𝒔 2Table 1: An overview of the relevant network parameters.
Poisson-based spikes
Considering that the length of a trial was longer than that of the input signal, see Figure 4, it was decided to prolong the thalamic input spikes with Poisson distributed spikes. As can be seen in Figure 5, the input now consisted of a segment of input-based spikes and a segment of Poisson-based spikes functioning as background noise. The average neuron firing rate of the Poisson-based spikes is an adjustable parameter, the Poisson-rate, that was manually set to 3𝐻𝑧.
Figure 5: The raster plot and PSTH of thalamic input spikes. The input-based spikes [0, 1955 ] 𝑚𝑠 are made from the whisking traces. The Poisson-based spikes [1955, 3455 ] 𝑚𝑠 are Poisson distributed spikes, with an average neuron firing rate of 3𝐻𝑧.
10
Data analysis
The activity of the network was evaluated by calculating per neuron the firing rate 𝑟𝑖 and the coefficient
of variation 𝐶𝑉 of the interspike intervals 𝐼𝑆𝐼: 𝑟𝑖 = 𝑁𝑖 𝑇,
(15)
𝐶𝑉 = 𝜇𝐼𝑆𝐼 𝜎𝐼𝑆𝐼 .(16)
The average firing rate 𝑟𝑖 is calculated by dividing the total number of spikes 𝑁𝑖 of neuron by the length
of the trial 𝑇 in seconds, and the 𝐶𝑉 is calculated by dividing the standard deviation of the 𝐼𝑆𝐼 by its mean. The 𝐶𝑉 was used as a measure for the irregularity of firing of a neuron, 0 indicating regular firing and a 𝐶𝑉 of 1 and higher indicating irregular firing.
Results
Training the network with different input conditions
Firstly, we set out to train the network to the three different input conditions: the convolved signal, the PSTH signal, and the thalamic spikes. To accurately compare the effect the three input conditions had on the network dynamics and performance, it was desirable to use them on networks with identical hyperparameters. Therefore, the different input signals were analyzed and scaled accordingly. A visualization of the three input signals can be seen in Figure 6. Here, we looked at the average input a reservoir neuron received, during the average trial, per input condition (Figure 6, left). The average trial was taken over all 1921 input trials (867 proximal, and 1054 distal). The distribution of the input the neurons in the reservoir received during the average trial can also be seen for each condition (Figure
6, right). While the general course and the distribution of the average neuron input per condition are
11 Figure 6: A visualization of the neuron input over the average trial for the three different input conditions. Left: the
average neuron input throughout the average trial in 𝑚𝑠, for the convolved signal, PSTH signal, and the thalamus spikes (from top to bottom). The neuron input was averaged over all 2000 reservoir neurons. Right: the distribution of the average input the neurons receive during the average trial of all three input conditions. It is evident from both plots, that neuron input of the convolved signal (𝜇 = −18 ∗ 10−3, 𝜎 = 7.5 ∗ 10−3), PSTH signal (𝜇 = 1.1 ∗ 102, 𝜎 =
0.80 ∗ 102), and thalamic spikes (𝜇 = 0.10, 𝜎 = 7.5 ∗ 10−3) are on different scales.
To correct this, we decided to scale the convolved signal and PSTH signal to the average neuron input of thalamic spikes condition. In this way, we could apply the same trainable regime that Winter (2020) based on training the network with thalamic spikes, to all three input conditions. The results of the scaled signals can be seen in Figure 7. The course and distribution of the average neuron input remained the same for both the convolved signal and PSTH signal, yet they were now both distributed over the same scale as the thalamic spikes.
Figure 7: The average neuron input over the average trial of the scaled signals and the thalamic spikes (yellow). The convolved signal (blue) and PSTH signal (red) were first standardized by subtracting the corresponding signal mean 𝜇 from each signal and dividing it by its standard deviation 𝜎. Thereafter, both standardized signals were scaled to thalamic spikes, by multiplying them by the thalamic spikes 𝜎 and adding the signal 𝜇 of the thalamic spikes. The left plot again shows the average neuron input throughout an average trial and the right plot the distribution of the neuron input.
12 Now that the input signals were properly scaled, the same trainable regime could be used when training the networks for each condition. Before training the networks, we determined how the different input signals would affect the network activity. This could later be used to relate the performance to the network activity caused by the different input signals. To do this, we simulated networks only receiving input, without any feedback. As a baseline, we simulated a network without input or feedback in order to illustrate the undisturbed activity of the reservoir. The network activity for each condition is shown in Figure 8.
Figure 8: An overview of the network activity of networks with different input conditions. All networks were simulated
with 𝐺 = 10, and without any feedback (𝑄 = 0). The networks with an input signal were set to 𝑊𝑖𝑛= 0.5 a. The raster
plot and PSTH of a random trial for each of the input conditions: the network without input (black) and the networks with the convolved signal (blue), the thalamus spikes (yellow), or the PSTH signal as input. For the raster plots, the spike trains of 100 randomly selected reservoir neurons were chosen. The PSTHs show the entire reservoir (𝑁 = 2000) activity and were calculated with a bin size of 1𝑚𝑠. The transition from input-based spikes to Poisson-based spikes in the thalamic spikes condition is indicated with a vertical black line on the x-axis (1955 𝑚𝑠) b. The neuron CV and firing rate (𝐻𝑧), averaged over 10 random trials, of the simulated networks with different input conditions mentioned in
a.
When comparing the raster plots and PSTHs (Figure 8a) of the three input conditions to the no-input network, it is notable that the convolved signal and PSTH signal did not seem to have a visible effect on the network activity. In contrast, the thalamic spikes seemed to have driven the network activity from chaotic spiking into regular spiking at approximately 1900 𝑚𝑠 into the trial. In the raster plot and PSTH of thalamic spikes, a vertical black line is drawn on the x-axis to indicate the transition from the input-based spikes to the Poisson-based spikes. The onset of regular firing seems to start just before the black line, suggesting the regular firing was triggered by an event in the input-based spikes and was further maintained by the Poisson-based spikes of the input. In Figure 8b, the neurons of each separate condition are scattered based on their coefficient of variation (𝐶𝑉) and firing rate. As mentioned earlier,
13 a 𝐶𝑉 of 1 or higher indicates the irregular firing of a neuron. Interestingly, in this figure, the convolved signal and PSTH signal did seem to have a visible effect on the network activity. Both input signals bring the network into a state with a more focused distribution of neuron 𝐶𝑉 and firing rate, while still showing chaotic activity (𝐶𝑉 ≈ 1). However, the convolved signal and PSTH signal show much overlap in neuron 𝐶𝑉 and firing rate. This, in combination with the fact that there was no noticeable effect of the input seen in the raster plots, suggests that the effect is not specific to the trial or the two input conditions. The thalamic spikes, on the other hand, have shifted the network activity to a lower neuron 𝐶𝑉 and higher firing rates, indicative of fast regular spiking. This matches the regular firing pattern seen in the raster plots.
The final step was to FORCE train networks using the different input signals. For all three input conditions, a network was trained for each value of 𝐺 between 1 and 15, with the other hyperparameters set to 𝑊𝑖𝑛 = 0.5, 𝑄 = 1. Every network was trained on 600 trials and tested on 100 trials, which was
repeated for 2 epochs. This process was repeated three times to increase the sample size, leaving a total of 45 trained networks per input condition. While the network parameters and hyperparameters were identical, each separate network was simulated with newly generated weight matrices. This was true for all separately trained networks throughout this research. The training results can be seen in Figure
9.
Figure 9: The results of FORCE training networks with convolved signals (blue), PSTH signals (red), and the thalamic spikes. All networks were set to 𝐺 = 10, 𝑊𝑖𝑛= 0.5, 𝑄 = 1, with 𝑁𝑡𝑟𝑎𝑖𝑛 = 600, 𝑁𝑡𝑒𝑠𝑡= 100, 𝑒𝑝𝑜𝑐ℎ𝑠 = 2. a. The test
accuracies of the networks trained with different input conditions with multiple values for 𝐺. Per condition, 3 networks were trained for each value of 𝐺, resulting in 45 trained networks per input condition. Nearly all trained networks had a test accuracy of 50%. b. Left: the network output versus the target signal over a sequence of 5 random trials for each input condition (𝐺 = 10). Right: the neuron CV – firing rate relation averaged over 10 random trials. The feedback value, between −1 and 1, each neuron in the reservoir receives is indicated with the colour gradient.
14 The first plot, Figure 9a, shows that for each condition, across almost all values of 𝐺, the networks performed poorly with a test accuracy of 50%. In most cases, this meant that the network was simply repeating the same learnt output signal for each trial, instead of classifying the input. This is confirmed in Figure 9b by the examples of the network output over five random trials. For all three conditions, the output over the trials is characterized by a repetitive sequence of positive or negative pulses. While some trials in the thalamic spikes condition do vaguely resemble the target pulse, the networks of the other two conditions completely failed to reproduce this signal. Next to the network output examples, the network neuron 𝐶𝑉 and firing rate distribution of each corresponding input condition is given. It seems, that the FORCE training has driven the neurons in the network into two separate groups based on their 𝐶𝑉 and firing rate. The first group consists of fast regular spiking neurons, indicated by high firing rates and low 𝐶𝑉𝑠, and the second group consists of slow irregular spiking neurons, indicated by low firing rates and high 𝐶𝑉𝑠. Furthermore, the feedback value of the fast regular spiking neurons, given in colour, was predictive of the network output. Fast regular spiking neurons with positive feedback value resulted in a positive network output, as can be seen in the convolved signal condition. The other way around, fast regular spiking neurons with a negative feedback value resulted in a negative network output, seen in the PSTH signal condition. This finding was ubiquitous over all poor performing networks and was also observed in Winter (2020). Again, there seems to be a similarity in network activity between the convolved signal and PSTH signal conditions. This is evident from the neuron firing rate and 𝐶𝑉 plots of Figure 9b. The additional similarity between the network output of the two conditions further suggests that there is no significant difference between the effect they both had on the network activity.
In conclusion, we were unable to FORCE train the networks using different input signals based on the whisking traces. Moreover, we were also unable to successfully train a network using the original input form (thalamic spikes) and hyperparameter settings from Winter (2020).
Varying Poisson-rate
While testing FORCE training on networks with different parameters, we came across a network with a remarkably high test accuracy of 68%. The network was trained using the thalamic spikes and all hyperparameters (𝐺 = 6, 𝑊𝑖𝑛= 0.5 and 𝑄 = 1) were set within the trainable regime. However, the
Poisson-rate (Figure 5), and the inverse of the learning rate 𝛼 of the network, had been altered. The Poisson-rate was set to 5𝐻𝑧 and the inverse of the learning rate to 𝛼 = 0.1. Previously trained networks, with the standard Poisson-rate and 𝛼 values (Poisson-rate= 3𝐻𝑧, 𝛼 = 0.05), that were trained with identical hyperparameters, generally had lower test accuracies (≈ 50%). This suggested there might be a relation between the Poisson-rate, 𝛼, and the network performance. To find out if this was true, we decided to train networks with varying Poisson-rates and different values for 𝛼. All networks had
15 identical hyperparameters (𝐺 = 6, 𝑊𝑖𝑛= 0.5 and 𝑄 = 1) and were trained and tested on the same
number of trials (𝑁𝑡𝑟𝑎𝑖𝑛 = 600, 𝑁𝑡𝑒𝑠𝑡 = 100, 𝑒𝑝𝑜𝑐ℎ𝑠 = 2). A separate network was trained for each
Poisson-rate between 3𝐻𝑧 and 6𝐻𝑧, and for 𝛼 = 0.05 and 𝛼 = 0.1. For each Poisson-rate and 𝛼 parameter combination, a total of 48 networks were trained. The training results can be seen in Figure
10a.
Figure 10: An overview of the FORCE training results with different Poisson-rates and values for 𝛼. All networks had identical hyperparameters (𝐺 = 6, 𝑊𝑖𝑛= 0.5 and 𝑄 = 1) and were trained on the same amount of training and test trials
(𝑁𝑡𝑟𝑎𝑖𝑛= 600, 𝑁𝑡𝑒𝑠𝑡= 100, 𝑒𝑝𝑜𝑐ℎ𝑠 = 2). For each Poisson-rate – 𝛼 combination, 48 networks were trained. a. The
test accuracy of networks with varying Poisson-rates (3𝐻𝑧 − 6𝐻𝑧) and 𝛼 = 0.05 or 𝛼 = 0.1. b. Right: an example of the output and activity of a network with a 60% test accuracy (Poisson-rate = 4) during a proximal trial. Left: the whisker curvature and angle of the same proximal trial. The vertical black line in the output and activity plots indicates the transition from input-based to Poisson-based spikes (2025 𝑚𝑠). c. The output of the same network with a test accuracy of 60% during 10 random trials.
The majority of trained networks had a test accuracy of 50%, especially the networks with a Poisson-rate of 3𝐻𝑧. For Poisson-Poisson-rates higher than 3𝐻𝑧 there were a handful (±12%) of networks with varying test accuracies, only slightly deviating from a test accuracy of 50%. There was not much difference between networks trained with 𝛼 = 0.05 and 𝛼 = 0.10, and in networks, with a Poisson-rate of 3𝐻𝑧 there was no difference at all. The slightly growing variance in test accuracy thus seems to be mainly caused by higher Poisson-rates. The test accuracies between the networks with Poisson-rate > 3𝐻𝑧, were also very similar. Most scored between the 45% and 55%, with an occasional outlier. The highest test accuracy (60%) was achieved by a network with a Poisson-rate of 4𝐻𝑧 and 𝛼 = 0.05. An example of this network's activity and output during a proximal trial can be seen in Figure 10b. The network output during multiple random trials is shown in Figure 10c. In the left plots of Figure 10b, the whisker
16 angle and curvature of the proximal trial are given, on the right the network output versus the target signal, and below that, the network raster plot and PSTH of the same trial. The vertical black line on the x-axis again indicates the transition from input-based to Poisson-based spikes of the input. In the output versus target plot (top-right), it is seen that the network output suddenly changed at ±650𝑚𝑠, and from there on accurately followed the target signal. This was likely caused by the change in whisker curvature that happened at approximately the same time. This implies the network accurately classified this trial based on the whisking input. The raster plot and PSTH (bottom-right) show that the network activity started high, and then decreased until at 500𝑚𝑠 it looked like only a subset of neurons was firing. Just after the network output suddenly became positive at ±650𝑚𝑠, it seemed that the group of neurons stopped firing, and another group of neurons started firing. This activity pattern is maintained up to about 2400𝑚𝑠, from where it transitioned to a more global regular spiking activity. It could be possible that the first group of neurons was tied to negative output pulses and the second group of neurons to positive output pulses. While the change from input-based to Poisson-based input spikes (vertical black line) did not seem to have a direct influence on the network activity, it is plausible that the Poisson-based spikes were responsible for the global regular spiking activity seen later in the trial. However, it was hard to exactly determine the role the Poisson-based spikes played, while training a network. Lastly, Figure 10c shows that, despite leaning more towards a negative output signal, the network was able to accurately classify some of the random test trials.
Figure 11: A comparison between the network activity and summed synaptic input of a network with an accuracy of
60% and a network with an accuracy of 50%. The first network had a Poisson-rate of 4𝐻𝑧 and the other network a
Poisson-rate of 3𝐻𝑧, while the other hyperparameters were the same for both networks. a. The neuron CV – firing rate relation, averaged over 10 trials, of both networks. The feedback value a neuron receives is indicated with the colour gradient. b. The distribution of the summed synaptic input of the reservoir neurons. The network with an accuracy of 50% (blue) has a much broader distribution, compared to the network with a 60% accuracy (purple).
17 To further investigate the role of the Poisson-rate in network training, we compared the activity of the network with an accuracy of 60% to a network with the original parameter settings (Poisson-rate = 3𝐻𝑧, 𝛼 = 0.05) and test accuracy of 50%. Therefore, we looked at the neuron CV – firing rate relation (Figure 11a) and the summed synaptic input distribution of both networks (Figure 11b). Similar to other poor performing networks, the neurons of the network with an accuracy of 50% seemed to be divided into two groups of neurons based on their firing rate and CV (Figure 11a, right). In contrast, the neurons of the network with a higher test accuracy were more distributed over different CV and firing rate values (Figure 11a, left). Neurons that receive negative feedback did generally have higher firing rates and neurons with positive feedback values lower firing rates. This might explain why the network leaned more towards a negative output, as was seen in Figure 10c. Furthermore, the distribution of the summed synaptic input to neurons in the reservoir was much broader for the network with an accuracy of 50% than for the network with an accuracy of 60% (Figure 11b). These differences were also found in Winter (2020) between good and bad performing networks with identical hyperparameters and Poisson-rates. Thereby implying that these are general attributes of good performing networks, that cannot directly be related to the Poisson-rate of the network.
Because it was difficult to distinguish the effects of input signals with different Poisson-rates on network activity post-training, we also studied the network activity before training. We did this by, again, simulating a network that only received input, without any feedback. Figure 12 shows the network activity of both inputs with a Poisson-rate of 3𝐻𝑧 and 4𝐻𝑧 during the same input trial.
Figure 12: The activity during a single trial of two networks without feedback (𝑄 = 0) receiving input with different
Poisson-rates (3𝐻𝑧 and 4𝐻𝑧). Both networks had identical hyperparameters (𝑊𝑖𝑛= 0.5 and 𝐺 = 6). The vertical black
line in the output and activity plots indicates the transition from input-based to Poisson-based spikes (1955 𝑚𝑠).
Interestingly, there is no visible difference between the network activity caused by different Poisson-rates. Neither did there seem to be a change in network activity after the transition to the Poisson-based spikes in either input signals (vertical black line). Thus, we can conclude that higher Poisson-rates of the Poisson-based spikes seem to cause more variability in network performance. Yet, it is unclear what exactly caused this, by studying the network activity.
18
Poisson mask
As shown in the previous section, the exact effect the Poisson-based input spikes had on the network activity during FORCE training was still unclear. Considering that these spikes were only added during the last 1500𝑚𝑠 as background noise, we wondered what would happen if we added the background noise over the whole input trial. Therefore, we created an input mask of Poisson distributed spikes, the Poisson mask, that can be seen in the raster plots and PSTHs of figure 13.
Figure 13: The neuron spike trains and PSTH of 200 thalamus neurons for the input signal, the Poisson mask, and the
masked input, respectively. The Poisson mask spikes, with a Poisson mask-rate of 3𝐻𝑧, are added to the input spikes to create the masked input.
In this figure, the thalamic input spikes of a trial are overlapped with a Poisson mask, which has an average neuron firing rate of 3𝐻𝑧, the Poisson-mask rate. The result was the masked input. Despite the input being noisier now, trial-specific shifts in activity were still visible in the masked input. For example, the rise in network activity seen just before 2000𝑚𝑠 in the original input signal (Figure 13, left), could still be seen in the masked input (Figure 13, right). However, by spreading the background noise over the entire trial, there also was a noticeable drop in activity in the masked input when the original input signal ended. This might cause an unwanted shift in activity during training. To test the influence of the added background noise on FORCE training, we trained multiple networks with increasing Poisson mask-rates (2𝐻𝑧 − 5𝐻𝑧). All networks had identical hyperparameters and the same amount of training and test trials as the networks in the previous section (𝐺 = 6, 𝑊𝑖𝑛 = 0.5, 𝑄 = 1,
𝑁𝑡𝑟𝑎𝑖𝑛= 600, 𝑁𝑡𝑒𝑠𝑡= 100, 𝑒𝑝𝑜𝑐ℎ𝑠 = 2). For each value between 2𝐻𝑧 and 5𝐻𝑧, 12 separate
networks were trained. Unfortunately, all trained networks had a test accuracy of 50%. To find out what might have caused this, we plotted the network output over multiple trials and the neuron CV – firing rate relation for the different Poisson mask-rates in Figure 14.
19 Figure 14: An example of FORCE trained networks with different Poisson mask-rates between 2𝐻𝑧 and 5𝐻𝑧. All networks were trained with the same hyperparameters and amount of training and test trials (𝐺 = 6, 𝑊𝑖𝑛= 0.5 , 𝑄 = 1, 𝑁𝑡𝑟𝑎𝑖𝑛= 600, 𝑁𝑡𝑒𝑠𝑡= 100, 𝑒𝑝𝑜𝑐ℎ𝑠 = 2) and had a test accuracy of 50%. Left: the network output during 5 random trials for varying Poisson rates. Right: the neuron CV – firing rate relation for the corresponding Poisson
mask-rates, with the neuron feedback value given in colour.
The example network with a Poisson mask-rate of 3𝐻𝑧, was the only trained network that was somewhat able to copy the target signal (Figure 14, left). All networks with higher Poisson mask-rates were not only unable to generate the target signal, but also seem unresponsive to the trial input. The output of these networks was characterized by a noisy signal fluctuating around zero. When looking at the neuron CV and firing rate distribution of the different Poisson mask-rates (Figure 14b), it became clear that from 𝑟𝑎𝑡𝑒 = 3𝐻𝑧, the background noise of the input signal had outweighed the input-based spikes. Almost all neurons in these networks shifted towards high regular firing activity, indicated by their high firing rates and low 𝐶𝑉𝑠.
Overall, we were unable to perform FORCE training with the addition of background noise, in the form of a Poisson-mask, over the input signal. Networks trained with a Poisson mask-rate of 2𝐻𝑧 were somewhat capable of copying the output signals. Yet, networks trained with higher Poisson mask-rates lapsed into high frequent regular activity states unable to learn anything.
20
Discussion
Unfortunately, we were unable to increase the performance reliability of the FORCE training implementation of a sensorimotor task from Winter (2020). While taking a deeper look at the network input, we found that using different signals, based on the whisker traces, did not affect the performance of the networks. FORCE training with any of the input conditions usually resulted in poor performing networks with test accuracies of 50%. Interestingly, we did find that the background noise added to the thalamic spikes input affected the network performance. A slight increase in activity of the background noise at the end of the input signal resulted in more variability in the network test accuracies. However, the increase in variability was small, and it was difficult to determine how it related to the background noise. A Further exploration of the background noise confirmed its subtle role during FORCE training. When extending the background noise over the entire input signal, the noise quickly overshadowed the input signal, leaving the network incapable of learning.
At first it seemed surprising that we were unable to FORCE train networks using the convolved signal or PSTH signal as input, considering both signals were less noisy than the thalamic spikes. Yet, as was shown in Figure 8, the network activity caused by these two input conditions were very similar and seemed indifferent to the particular input trial. Being affected by trial-to-trial variability is rather important while learning to classify an input signal. The similarity between the two conditions was seen again in the network output and activity after training (Figure 9). A possible explanation could be, that both signals were not strong enough and consequently got outweighed by the recurrent and feedback input the neurons received. This could be further investigated, by mapping the different inputs a reservoir neuron receives during the course of a trial. It was also notable, how poorly the networks that were trained on the thalamic spikes performed, in comparison to the training results of the networks with identical hyperparameters in Winter (2020). These FORCE trained networks, also trained with the thalamic spikes, often had more variable test accuracies. A possible explanation for this could be that the Poisson-rate in the supplementary code of Winter (2020) was set to 5𝐻𝑧 instead of the documented 3𝐻𝑧. Since, as we saw earlier, an increase in the Poisson-rate led to more variability in network performance. Although, it is unsure whether this was the setting for all trained networks. So, the performance difference could also be the result of an error made while replicating his work. We might be able to successfully implement FORCE training by further scaling the convolved signal and PSTH signal. Yet, for future research it would be more worthwhile to focus on thalamic spikes as input. Not only did they show more promise in network performance, but they are also the most biologically realistic translation of the whisking input.
As mentioned earlier, it was difficult to find the exact effect different Poisson-rates had on the network activity that caused more variability in network output. In Figure 12 it could be seen that there was hardly any difference in network activity caused by input with a Poisson-rate of 3𝐻𝑧 versus 4𝐻𝑧.
21 Despite this, networks with a Poisson-rate of 4Hz had more variability in test accuracy (Figure 10). While there were visible differences in the activity post-training between these two networks (Figure
11), they were more likely attributes of good and bad performing networks in general. The variability
and test accuracy were also fairly similar for Poisson-rates higher than 3𝐻𝑧, and seemed unaffected by the different values for 𝛼. Furthermore, the extension of the Poisson-based spikes in the form of the Poisson mask demonstrated that too much background noise harmed FORCE training. It could be possible that the networks FORCE trained on input masked with the lowest frequency (2𝐻𝑧) were disrupted by the sudden drop of activity after the input-based spikes had ended, and not the addition of extra background noise. This would explain why these networks were able to somewhat copy the target signal, in contrast to the networks with higher rates. It could be interesting to test this, before completely dismissing the idea of adding background noise over the entire trial. This could be done by designing an input signal that has relative constant activity throughout the trial, even with the added background noise.
In previous literature, a link was found between the chaos and performance of a network. For example, Sussillo & Abbott (2009) found that FORCE training worked better on networks with more chaotic dynamics, caused by stronger synaptic coupling. There was, however, an upper limit to this effect. At a certain point, the feedback in the network was unable to suppress the chaotic dynamics, which disabled FORCE learning. The network performance was the highest at the ‘edge of chaos’, a phenomenon also described in earlier papers (Bertschinger et al., 2005; Bertschinger & Natschläger, 2004; Legenstein & Maass, 2007). It could be that in our research, the added noise to the input signal slightly pushed the network activity to a state similar to the ‘edge of chaos’. While not directly improving the network performance, but only the variability in performance, slightly noisier input did bring the network in a state more susceptible to FORCE training.
In future research, it would be interesting to focus more on how the reservoir of neurons used in FORCE learning is initialized. Recent research has shown that new designs of neuron properties or connectivity can promote robust FORCE learning (Perez-Nieves et al., 2020; Zheng & Shlizerman, 2020). For example, in Perez-Nieves et al. (2020) it was shown that networks of neurons with heterogeneous time scales performed better over different parameter settings than homogeneous networks. This not only made learning more robust but also added to the biological realism of the model. Once the network performance is more stable, it would be interesting to study the effect of adding more biologically realistic attributes. This could be done by introducing separate excitatory and inhibitory neurons in the reservoir, thereby obeying Dale’s Law. Previous research has shown that it is possible to apply FORCE training to such networks of excitatory and inhibitory neurons (Ingrosso & Abbott, 2019; Kim & Chow, 2020; Nicola & Clopath, 2017).
Through its complex relation between chaotic network dynamics and computational capabilities, FORCE learning is a powerful tool for training spiking RNNs. As we saw in recent research, incorporating biological constraints not only added to the realism of the networks but also
22 their computational capabilities. Thus, demonstrating the promising future of using FORCE learning to create functional models of spiking networks underlying behaviour.
References
Atiya, A., & Parlos, A. (2000). New Results on Recurrent Network Training: Unifying the Algorithms and Accelerating Convergence. IEEE Transactions on Neural Networks / a Publication of the IEEE Neural Networks Council, 11, 697–709. https://doi.org/10.1109/72.846741
Barak, O. (2017). Recurrent neural networks as versatile tools of neuroscience research. Current Opinion in Neurobiology, 46, 1–6. https://doi.org/10.1016/j.conb.2017.06.003
Bertschinger, N., & Natschläger, T. (2004). Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks. Neural Computation, 16(7), 1413–1436.
https://doi.org/10.1162/089976604323057443
Bertschinger, N., Natschläger, T., & Legenstein, R. A. (2005). At the Edge of Chaos: Real-time Computations and Self-Organized Criticality in Recurrent Neural Networks. 8.
Doya, K. (1992). Bifurcations In The Learning Of Recurrent Neural Networks. Ieee International Symposium on Circuits and Systems, 2777–2780.
Guo, Z. V., Hires, S. A., Li, N., O’Connor, D. H., Komiyama, T., Ophir, E., Huber, D., Bonardi, C., Morandell, K., Gutnisky, D., Peron, S., Xu, N., Cox, J., & Svoboda, K. (2014). Procedures for Behavioral Experiments in Head-Fixed Mice. PLOS ONE, 9(2), e88678.
https://doi.org/10.1371/journal.pone.0088678
Guo, Z. V., Li, N., Huber, D., Ophir, E., Gutnisky, D., Ting, J. T., Feng, G., & Svoboda, K. (2014). Flow of cortical activity underlying a tactile decision in mice. Neuron, 81(1), 179–194. https://doi.org/10.1016/j.neuron.2013.10.020
Huang, C., Zeldenrust, F., & Celikel, T. (2020). Cortical representation of touch in silico. BioRxiv, 2020.11.06.371252. https://doi.org/10.1101/2020.11.06.371252
Ingrosso, A., & Abbott, L. F. (2019). Training dynamically balanced excitatory-inhibitory networks. PLOS ONE, 14(8), e0220547. https://doi.org/10.1371/journal.pone.0220547
23 Jaeger, H. (2001). The" echo state" approach to analysing and training recurrent neural networks-with
an erratum note’. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, 148.
Kim, C. M., & Chow, C. C. (2020). Training recurrent spiking neural networks in strong coupling regime [Preprint]. Neuroscience. https://doi.org/10.1101/2020.06.26.173575
Legenstein, R., & Maass, W. (2007). Edge of chaos and prediction of computational performance for neural circuit models. Neural Networks, 20(3), 323–334.
https://doi.org/10.1016/j.neunet.2007.04.017
Lukoševičius, M., & Jaeger, H. (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3(3), 127–149.
https://doi.org/10.1016/j.cosrev.2009.03.005
Maass, W., Natschläger, T., & Markram, H. (2002). Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations. Neural Computation, 14(11), 2531–2560. https://doi.org/10.1162/089976602760407955
Nicola, W., & Clopath, C. (2017). Supervised learning in spiking neural networks with FORCE training. Nature Communications, 8(1), 1–15. https://doi.org/10.1038/s41467-017-01827-3 Perez-Nieves, N., Leung, V. C. H., Dragotti, P. L., & Goodman, D. F. M. (2020). Neural
heterogeneity promotes robust learning [Preprint]. Neuroscience. https://doi.org/10.1101/2020.12.18.423468
Peron, S. P., Freeman, J., Iyer, V., Guo, C., & Svoboda, K. (2015). A Cellular Resolution Map of Barrel Cortex Activity during Tactile Behavior. Neuron, 86(3), 783–799.
https://doi.org/10.1016/j.neuron.2015.03.027
Petersen, R. S., Brambilla, M., Bale, M. R., Alenda, A., Panzeri, S., Montemurro, M. A., & Maravall, M. (2008). Diverse and Temporally Precise Kinetic Feature Selectivity in the VPm Thalamic Nucleus. Neuron, 60(5), 890–903. https://doi.org/10.1016/j.neuron.2008.09.041
Rumelhart, D. E., & McClelland, J. L. (1987). Learning Internal Representations by Error Propagation. In Parallel Distributed Processing: Explorations in the Microstructure of
24 Cognition: Foundations (pp. 318–362). MIT Press.
https://ieeexplore.ieee.org/document/6302929
Sivagnanam, S., Majumdar, A., Yoshimoto, K., Astakhov, V., Bandrowski, A., Martone, M., & Carnevale, N. T. (n.d.). Introducing The Neuroscience Gateway. 7.
Sussillo, D., & Abbott, L. F. (2009). Generating Coherent Patterns of Activity from Chaotic Neural Networks. Neuron, 63(4), 544–557. https://doi.org/10.1016/j.neuron.2009.07.018
Thalmeier, D., Uhlmann, M., Kappen, H. J., & Memmesheimer, R.-M. (2016). Learning Universal Computations with Spikes. PLOS Computational Biology, 12(6), e1004895.
https://doi.org/10.1371/journal.pcbi.1004895
Vreeswijk, C. v., & Sompolinsky, H. (1996). Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity. Science, 274(5293), 1724–1726.
https://doi.org/10.1126/science.274.5293.1724
Winter, T. (2020). Understanding FORCE Trained Spiking Neural Networks [Master’s Thesis]. Radboud University.
Zheng, Y., & Shlizerman, E. (2020). R-FORCE: Robust Learning for Random Recurrent Neural Networks. ArXiv:2003.11660 [Cs, q-Bio]. http://arxiv.org/abs/2003.11660
