Oscillations and Competition: Selection and Switching in Simple Oscillating Networks
William Hedley Thompson
Research Project 1, University of Amsterdam.
Supervisor: Hans Phaf.
Oscillations and Competition
Selection and Switching in Simple Oscillating Networks William Hedley Thompson
Introduction
When networks receive conflicting inputs, as they unavoidably do in the real world situations, competition arises. A network that can effectively resolve this competition is able to select an input quickly. This is done by inhibiting the activation that would have been caused by the rejected inputs. This allows the activation caused by the chosen input to “win” the competition. Only the parts of the network responding to this stimulus become activated. This is the selection of one input. This should occur even if the difference in inputs is very small as0 the competition still has to be solved. The network must also be ready to effectively switch to another input if and when the situation changes. The activation corresponding to the newer better input must now “win” the competition. This is the switching from one input to the other. The core quality of an effective competitive network is resolution of both selection and switching. Oscillations have independently been proposed for both, but never studied together.
Regarding switching, computational studies in switching have recently revealed results regarding the possible roles of oscillations to resolve competition. Evolutionary simulations showed that the most evolutionary fitting models oscillated. In these simulations, agents navigating virtual environments containing food patches and predators resulted in organised approach-avoidance behaviour and specific architectures of the quasi-neural networks controlling the agents (Heerebout & Phaf 2010ab). The evolutionary development of oscillations in these networks was associated with large fitness benefits and thus with functional advantages. Here the troughs of the oscillations of the active nodes for the first stimulus, gave a “window of opportunity” for a second stimulus to become active. From these results they proposed a theory that higher frequencies facilitate quicker switching. This is because the time until the next trough in activation will decrease as the frequency increases.
Selection of competition is often demonstrated through selective attention which gates the attended stimulus and suppresses the distracters. (Moran & Desimone 1985). This suppression has been recorded in attention tasks in the visual cortex in macaque monkeys (Luck, Chelazzi, Hillyard, & Desimone 1997; Reynolds, Chelazzi, Desimone 1999). Oscillations have been proposed to be behind this suppressive mechanism. In computational models it has been shown that this suppression of distracters can occur due to oscillations within the network (Börgers, Epstein, & Kopell 2008; Börgers & Kopell 2008). In these models, the oscillations from the inhibitory nodes provide “windows of opportunity” for one of the excitatory neurons/nodes to win the competition.
At first glance, these oscillatory functions proposed for selection and switching appear antagonistic to each other. They both advocate that oscillations provide a window of opportunity to help solve the competition, yet they are different mechanisms. Switching argues that oscillations help enhance the competition, whereas selection postulates that oscillations help suppress the competition. There is however no evidence that denies there compatibility apart from intuition. In this paper we investigate whether the proposed functions of oscillations in selection and switching may compliment each other. If this is the case, then this would be a strong case for oscillating competitive networks.
Compatibility between switching and selection is not enough to advocate that oscillations are effective at resolving competition, they must be better than non-oscillating counterparts. If the oscillations play a role, the frequency of the oscillations should have an affect on the results as this would modulate these windows of opportunity. Taking all this into consideration, we investigated the three following questions. First, we wish to see if stimulus switching and stimulus selection are compatible with each other within the same simple competitive networks. Second, if switching and selection are compatible, we investigate if oscillating networks offer an advantage over non-oscillating networks in both tasks. Third, if oscillating networks perform better, what effect does the frequency have upon switching and selection?
Method
We designed five different simple networks to test, three that were intended to oscillate and two which were intended not to oscillate. In this section we outline the structure and activation rules for the networks. These hold for all three experiments.
Our aim is to see whether selection and switching are possible in simple networks. To test the benefit of oscillations, we needed to construct networks that oscillate and networks that have tonic activation when they receive constant input (non-oscillating). We wanted to construct simple networks with the minimal amount of nodes and connections. The benefit of this is to clearly see how the oscillations affect the networks. A slight disadvantage is that it loses a direct real world application. However the principles found in simpler networks can later be applied to larger networks.
There were two networks which were designed not to oscillate (Figure 1a,b). The first is a simple network with four nodes. This was named SN4. Each excitatory node in this network excites its own inhibitory node which will then suppress the competition. The second network is the same as SN4 with inhibitory to inhibitory connections. This was named SN4I.
For the oscillating networks, we created three networks that used the same mechanism for the oscillations (see Figure 1c,d,e). The computational work for selection and switching have both proposed that the oscillations are created by the interplay or flip-flopping of activation between dually connected excitatory and inhibitory nodes. The first of the simple oscillating networks is the same structure as SN4I, but with inhibitory to excitatory input as well (SON4I). The second of the simple oscillating networks removes the inhibitory-to-inhibitory connections of the former network (SON4), this has a similar structure to SN4 but with inhibitory to excitatory connections to create the oscillations. The last is a simple oscillating network that only has three nodes (SON3), which has a more plausible proportion of the number of inhibitory nodes to the number of excitatory nodes. Here, both excitatory nodes activate the same inhibitory node which suppresses them both. Because there is no mutual inhibition, either directly or indirectly, between the two excitatory nodes, this network would not be able to solve the competition without oscillations. For this reason, it does not have a non-oscillating counter-part. The schemata for all five networks can be found in Figure 1.
a) b) c) d) e)
Figure 1: Schematic representations of the networks used. The blue circles marked E are excitatory
nodes, the pink circles marked I are inhibitory nodes. Arrows indicate excitation. Balled ends indicate inhibition. a. SN4; b. SN4I; c. SON4I, d. SON4, e. SON3
In the networks each node could only connect to other nodes with either excitatory or inhibitory connections. Each network has two excitatory nodes which are named E1 and E2, both of which receive input from Input-1 and Input-2 respectively. The inhibitory nodes are named I1 and I2, or collectively as I. We call the excitatory node we give the most input to, the attending node. We call the excitatory node that will be suppressed for the distracter node. E1 is set as the distracter node, E2 is set as the attending node. There are three types of connections in SON3, SON4 and SN4, the external input weight to the excitatory nodes (InW), excitatory to inhibitory connections (EI) and inhibitory to excitatory connections (IE). There is an additional inhibitory to inhibitory connection (II) in SON4I and SN4I.
We chose the same activation equations as in the evolutionary algorithms that revealed switching (Heerbout & Phaf, 2010ab). A decay to the equations. Time is measured in terms of iterations or time steps, which represent the time taken for an update of all activations according to the below activations. All networks use the following activation rule:
Where N is the number of nodes, wij is the weight between nodes i and j, ai,t is the
activation of node i at time t, decay is the decay parameter, θ is the bias. σ, is a squashing function which specifies the activation of node i at time, t between 0 and 1. σ-function is given by:
The decay is set at 0.1 for both the excitatory nodes and inhibitory nodes (for a similar decay, see Murre, Phaf & Wolters, 1992). This bias is set at 0. For the remaining parameters see the individual experiments.
Experiment 1: Compatibility of switching and selection
In this experiment we investigate whether switching and selection is possible for all five networks. This experiment only sets out to see if, in principle, the networks can switch and select with the same weight configuration. We expect that we selection and switching will be possible in all networks. We do not attempt to compare the oscillating versus the non-oscillating in this experiment (see Experiment 2).
The selection and switching tasks
For this experiment, there were two tasks: the selection task and the switching task. The aim was to see whether switching and selection is possible on all five of the networks. Each network did both tasks using the same parameters for each task.
In the selection task, the networks received inputs simultaneously, starting from t=0. The input to the attending node was 0.7 and to the distracter was 0.5. These inputs were maintained for the entire duration of the simulation. The simulation ran for 100 iterations. Selection is considered successful if the distracter is significantly or entirely suppressed.
In the switching task, the distracter received an input of 0.5 at the start of the simulation; this was maintained throughout the entire 100 iterations. The input to the attending node was 0 up until the 50th iteration where it shot up to 0.9. As with selection, the switching task is successful if the distracter is consistently suppressed.
Parameters
There is no configuration where all non-oscillating and oscillating networks can both select and switch. For this reason we chose to have separate configurations of parameters for the non-oscillating and non-oscillating networks. This experiment only wishes to confirm the possibility of compatibility between selection and switching. The non-oscillating networks had the following configurations: InW: 1; EI: 1; IE: -1; II -0.5. The oscillating networks had the following configurations: InW: 3; EI: 3; IE: -10; II: -0.5.
With the chosen parameters, all networks were able to select the attending input over the distracter input by suppressing the distracter. This can be seen in Figure 2. As intended, the excitatory and inhibitory nodes of oscillating networks oscillate and the non-oscillating have a tonic input. Small remnants of the distracter input can be seen for all oscillating networks.
For switching, all networks were able to switch. This can be seen in Figure 3. It can be seen that all the networks are in a stable state up until the switching point at 50 iterations. At 50 iterations the attending node’s activation shoots up and the distracter is suppressed. The suppression of the distracter does vary for the different networks.
Figure 2: shows all five networks during the selection task. The image shows the inputs (column 1),
I activation (column 2), E1 activation (column 3) and E2 activation (column 4). The red input activates the attending node (also shown in red). The blue input activates the distracter node (also shown in blue). For all networks apart from SON3 the attending node activates the node represented by the yellow inhibitory activation and the distracter node activates the node represented by the black inhibitory activation.
Figure 3: shows all five networks during the switching task. The switching point occurs at 50
iterations. The image shows the inputs (column 1), I activation (column 2), E1 activation (column 3) and E2 activation (column 4). The red input activates the attending node (also shown in red). The blue input activates the distracter node (also shown in blue). For all networks apart from SON3 the attending node activates the node represented by the yellow inhibitory activation and the distracter node activates the node represented by the black inhibitory activation.
Discussion
All networks are able to select and switch. It can be seen that switching and selection is possible for all five networks. In the selection task, all the oscillating networks had very small distracter activation. In the switching task, there is complete suppression of all distracters. Experiment 1 shows us that, contrary to the opposite roles proposed by oscillations, an oscillating network can both switch and select. The same also applies to the non-oscillating networks. At present, these results reveal nothing about the roles of oscillations in selection and switching nor does it reveal which of the networks is resolving the competition quicker.
Experiment 2: Comparing optimal performances
Experiment 1 confirmed that, in principle, all networks could switch and select. We chose to compare all the networks to see if the oscillating networks have an advantage over the non-oscillating networks in switching and/or selection. To prevent biasing a particular network, we found the optimal performing configurations for each of the networks prior to comparing the networks.
Modified selection and switching tasks
For this experiment we modify the selection and switching task in order to give a convergence value to evaluate the performance of each network. For both the selection and switching tasks, the simulations were run three times. The first two simulations ran either the attending input in seclusion or the distracter input in seclusion. By seclusion, it is meant that the other input was not present. In both seclusion trials the input corresponding to either the attending or the distracter was from the first to the last iteration. In these seclusion trials the average activation of the activation peaks were calculated. If the network was not oscillating, this becomes the average activation of all time points which have a tonic activation. We call this average of activation in these trials for the secluded attending activation of peaks and secluded distracter activation of peaks respectively. The seclusion trials were the same for both selection and switching tasks.
After the two seclusion simulations, there is a critical simulation. For selection, there is a single simulation where both inputs began at t=1. The convergence time is considered to be the first time point that meets the convergence condition (see below). For switching, 50 trials were run for each critical simulation. The distracter input was maintained throughout the entire simulation for every trial. The attending input started at 25t and increased by 1 each for each trial. The convergence time for each trial was the first time point that met the convergence condition (see below). The convergence time for the critical simulation of the selection task needed only 1 trial; the critical simulation of the switching task is the average of all 50 trials. This average for switching was needed as the convergence time is dependent upon when in the phase of the oscillation the switch occurs.
The criterion for convergence for both selection and switching is the first time point where the following three conditions were met:
1. aA > CASA
3. aD remains below CDSD for the duration of the simulation.
Where aA is the activation of the attending node, aD is the activation of the distracter node, CA is
the convergence parameter for attending, CD is the convergence parameter for the distracter node,
SA is secluded average attending activation of peaks, SD is the secluded average distracter
activation of peaks. The value of both CA and CD is set to 0.5 for all critical simulations.
Tasks with large convergence times often failed to resolve the competition adequately; there could, for example, be large spike of aA late in the simulation due to a slow build of up
overall inhibition. Such trials should be rejected. If the convergence time of a task was greater than 20 iterations, then it is considered to have failed to resolve the competition and that particular task is considered to be a rejection. The critical simulations continue for 500 iterations so that convergence condition 3 is verified even for later switching points. At the convergence time the competition is considered to be resolved.
Parameters
The parameters of the networks definitely play a role on its performance. When comparing five different networks, it is imperative not to bias one over the other with the parameters. We thus ran simulations on many possible configurations for both simulation tasks and found an optimally performing configuration. To find these optimally performing networks we first constructed candidate networks. These candidate networks performed the selection task, with the input to the attending node at 0.6 and the distracter node at 0.5, and the switching task, with the input to the attending node at 0.9 and distracter node at 0.5. Using these input levels we tested the EI, InW and IE connection for every integer between 1 and 10. For the II connections, we tested at 0.5 intervals between 0.5 and 10. In such small networks II connections could not be too strong. We first obtained a number of candidate networks. A candidate network was a network that could perform both selection and switching. This was done for each of the five networks to find the optimally performing candidate configurations.
We then ran each of the candidate networks for selection and switching tasks with attending inputs varying from 0.4 to 0.9 and distracter inputs from 0.3 to 0.8. For each network, this gives a total of 72 trials per candidate network. Of these trials 30 (15 for each task) the distracter input would be equal or larger than the attending and we expected all these to be rejected. These were expected rejections. From the remaining trials we found the average convergence time for all selection and switching tasks for each candidate network. We then found
how many trials where there was no resolution of competition. These were the unexpected rejection trials for that particular candidate network.
Choosing the optimal performing configuration was not always obvious. For each network there we used the average over convergence times for both tasks and the number of rejected trials to choose a network. If an average convergence of 5.56 with 8 rejections is better than 5.21 with 9 rejection, as is the case SN4, is no easy choice. How much a rejected trial is worth in comparison to a slight variation of convergence time is debatable. We chose in favour of the lowest number of rejections and, if more than one, the lowest convergence times. This gave us five optimal configurations, one for each of the five networks.
We added a condition that the IE weights should be the largest of the three weights. While it is possible for networks to oscillate when InW < IE, the way in which the oscillations are created is slightly differently. This method gives a greater regulatory role to the Input and E nodes whereas the chosen method gives this to the I nodes. Thus, for consistency within the networks, the following condition was imposed while finding the optimal configuration for the oscillating networks: InW < IE and EI < IE. Due to several biological properties of inhibitory internuerons, this condition was seems well founded. Such properties include greater proximity of a GABAergeric to the soma (Papp, Leinekugel, Lee & Buzsáki 2001) and faster rise and decay time of inhibitory postsynpatic potentials than excitatory postsynaptic potentials (Csicsvari, Hirase, Czurko, Buzsáki 1999).
This “IE>EI and IE>InW” condition could produce some oscillating networks that initially oscillated, but slowly converged onto a tonic value. This occurred when the IE weight was only slightly higher than both the EI weight and the InW (For example InW:8, EI:8 , IE:-9). These networks were not excluded from the algorithm used to select the optimally performing networks. However, none of the optimal networks performed in this way.
Inw EI IE II Call Rall Csel Rsel Cswi Rswi SON3 1.00 5.00 -9.00 - 9.73 0.00 13.67 0.00 5.80 0.00 SON4 1.00 6.00 -7.00 - 11.17 9.00 15.89 3.00 6.18 4.00 SON4I 1.00 2.00 -10.00 -0.50 8.73 6.00 11.50 3.00 5.96 3.00 SN4I 9.00 1.00 -10.00 -0.50 5.90 12.00 5.11 3.00 7.08 9.00 Sn4 4.00 1.00 -5.00 - 5.56 8.00 4.75 1.00 6.74 7.00
Table 1: Optimal configurations giving least rejections and lower convergence times for all five
networks. Where Call is the average convergence over selection and switching tasks, Rall is the unexpected rejection number of trials for both tasks; Csel and Rsel are the average convergence and unexpected rejection number of trials for the selection task; Cswi and Rswi are the average convergence and unexpected rejection number of trials for the switching task.
Results
The optimal configurations that were found can be seen in Table 1. Figure 4 shows the convergence rates for the optimal network for varying selection inputs. As it can be seen, the non-oscillating networks performed better than the non-oscillating. The non-oscillating networks performed considerably worse at selection and had less rejections.
The non-oscillating networks performed better on the selection task and worse on the switching task. SN4 had the highest average convergence times and performed the best from all networks in the selection task. It preformed invariantly to changes in input except when there was only a small difference in input (Figure 4a). In the switching task SN4 was unable to resolve the competition when the input to the attending was below 0.6 (Figure 4b). SN4I performed worse on all accounts compared to SN4, it had the highest number of rejections and longer convergence averages for both tasks (Figure 4cd).
The oscillating networks performed better on the selection task, had fewer rejections but longer convergence times. SON4I had the best overall convergence and selection convergence from the oscillating networks (Figure 4ef). SON3 was the most robust with no unexpected rejections. It also performed the best in switching (Figure 4ij) SON4 performed the worst out of the oscillating networks in selection, switching and rejections. (Figure 4gh) For SON4 and SON3 the convergence time for selection was dependent on the attending input. For SON4I selection
occurred quicker when there was more distraction. For all three oscillating networks, convergence was quicker when there was less competition.
a) b)
c) d)
g) h)
i) j)
Figure 4: shows the convergence times for the varying attending and distracter inputs. a. for SN4
during a selection task; b. for SN4 during a switching task; c. for SN4I during a selection task; d. for SN4I during a switching task; e. for SON4I during a selection task; f. for SON4I during a switching task; g. for SON4 during a selection task; h. for SON4 during a switching task; i. for SON3 during a selection task; j. for SON3 during a switching task;
Discussion
This comparative experiment revealed five varying aspects between the networks of the competition: (1) Average convergence for both task; (2) Average convergence for selection; (3) Average convergence for switching; (4) Robustness (number of rejections); (5) How each network deals with varying amounts of competition. SN4 had the best overall convergence and selection. SON3 had the best convergence for switching and was most robust. SON3 and SON4 had reasonable behaviour to input, but SON3, due to its robustness, has a better ability to deal with all
varieties of competition. The network that is resolving the competition better is either SON3 or SN4 depending upon which of the five aspects found is given the greatest importance.
We consider SON3 to be the best. This is for three reasons. First, robustness is an essential parts of competition. SON3 can simply solve competition in more cases than SN4. Second, the near-invariant performance of SN4 in selection seems unlikely in real world situations. When a stimulus is stronger, it seems reasonable that it would get selected quicker than a weaker stimulus. Third, there are greater possibilities of expanding SON3 to improve its capabilities in selection. SON4I performed considerably better at selection than SON3 and SON4. This would suggest that larger SON3 networks with II connections may perform better in selection. Using SON3 as a template module connected by II connections might improve SON3’s selection. Another advantage of the SON3 network is its possibility of expanding the amount of E nodes without an increase in I nodes. This can lead to a biological plausible ratio of approximately 4 excitatory nodes to one I node. Such an increase in excitatory nodes for the selection task will lead to a greater amount of inhibition during the selection task which we predict will suppress all the distracters quicker. These are future directions to take the SON3 network. Trying to improve SN4 by a better ratio between E and I nodes would reduce it to SON3.
This experiment reveals little about whether tonic or oscillating networks would be better in more complex networks. These results are not contradictory to the selection simulations in larger networks (Börgers et al., 2008). It is also worth noting that Börgers et al.’s networks had a 4:1 ratio of E to I nodes and have II connections. This experiment does reveal that the oscillating networks increase the ability of the network to switch when the difference between inputs is small.
Experiment 3: modulation of the internal frequency.
Given that SON3 performs better than its oscillating counterparts and is more robust than the non-oscillating networks, we selected SON3 to test Phaf and Heerebout’s (2010ab) hypothesis that higher internal frequency of the network prior to switching will entail a quicker resolution of the competition.
The Selection and Switching Task
We tested the higher frequency leads to quicker switching hypothesis on both selection and switching tasks. The selection, switching task and convergence times from the previous experiments were kept the same for the SON3 networks outlined in the parameters below.
Parameters
When the weights within a network are modified, the frequency is also modified. As this frequency is dependent upon the internal configuration and not the external input, we call it the internal frequency. All possible SON3 network configurations were tested for the selection and switching task. This means that the IE, InW and EI configuration ranged from 1 to 10. The IE>EI and IE>InW condition was maintained. This gives all possible internal frequencies at that range. Only one set of input values was used for each task otherwise the external input will vary and this affect the frequency. The inputs of 0.7 for the attending and 0.5 for the distracter for the selection task were used. The inputs of 0.9 for the attending and 0.5 for the distracter for the switching task were used.
The frequency was taken by counting the number of peaks. For the distracter in the switching task, the calculation was done during the seclusion trial. For the distracter in selection and the attending nodes, the calculation was done with both inputs present. As can be seen in Figure 2, the distracter can still oscillate in the selection task. The frequency is given per 1000 iterations.
As stated above, all the previous requirements for convergence from experiment 2 were maintained. From all the possible configurations we retained only those where the configuration had a succesful resolution of competition for both selection and switching. We then selected the configurations which could resolve both selection and switching tasks. 119 of the 285 possible configurations were able to meet the convergence requirements for both tasks. This might seem a small number as the optimal SON3 in the previous experiment could convergence in all conditions. However, the configurations here create suboptimal configurations of SON3. For comparative purposes, SN4 only converged in both tasks in 32 out of 285 configurations.
Results
Figure 5 shows the average convergence at the range of frequencies found for the specified parameters. Each dot represents the average convergence time taken from all network configurations at that frequency. It is important to note that there were some configurations in the switching task that, after the switch, resulted in the networks no longer oscillating. As stated in experiment 2, this occurs when the InW is only slightly lower than the IE weight. These are the values found in when the frequency is between 0 and 70. Here for example there would oscillate for around 30 times per 100 iterations for 200 iterations with decreasing amplitudes until reaching
a tonic level of activation. When these are excluded, as in Figure 5, only the switching tasks have significant results. The distracter node has a higher correlation between results. All r and p values have been included in Table 2.
Without this exclusion of values under 70, the correlations for the attending nodes in both tasks and the distracter in selection change quite radically. These values have been included in Table 2. The most radical change is the attending node in the switching task which changes from an insignificant positive correlation to a significant negative correlation.
Figure 5: shows the average convergence time at each frequency, given as peaks per 1000
iterations against the convergence time with internal modifications of the SON3 networks. These internal modifications include the changing of the EI IE and InW weights. (Top left) Shows the distracter node’s frequency for the selection task. (Top Right) Shows the attending node’s frequency for the selection task. (Bottom left) Shows the distracter node’s frequency for the switching task. (Bottom Right) Shows the attending node’s frequency for the switching task. Any convergence with a frequency less than 70 has been excluded (see text)
r Equation of line p
Distracter – Selection Inc. frequency ≤ 70 0.00 y = -0.0001x + 11 0.99
Exc. frequency ≤ 70 0.46 y = 0.019x + 8 0.15
Attending – Selection Inc. frequency ≤ 70 0.03 y = 0.0012x + 11 0.88
Exc. frequency ≤ 70 -0.41 y = -0.019x + 15 0.64
Distracter – Switching Inc. frequency ≤ 70 -0.90 y = 0.022x + 6.6 0.00*
Exc. frequency ≤ 70 -0.90 y = 0.022x + 6.6 0.00*
Attending – Switching Inc. frequency ≤ 70 0.31 y = 0.0043x + 3 0.20
Exc. frequency ≤ 70 -0.54 y = 0.011x + 5.7 0.03*
Table 2: a list of r, the equation of the best fitted line and the p values for all 8 conditions of
frequency correlated with the convergence times of SON3’s various configurations tested in experiment 3. Only switching shows significant results (p<0.05).
Discussion
It is quite clear from these results that the distracter node for the switching task has a significant and very strong negative correlation between convergence time and frequency. This supports the hypothesis that switching time will decrease as the frequency of the network prior to switching increases. This negative correlation for the distracter node after switching indeed corresponds to the locking of attention in the evolutionary simulations by Phaf and Heerebout (2010ab).
For selection, the frequency of the distracter played no significant role. There are very few varying frequencies. The attending nodes for both tasks have quite high negative correlations, but only the attending node for switching had significant results. This is not unexpected since fast frequency results in less iterations to reach the amplitude of the phase. This will entail a quicker convergence.
General Discussion
In this paper, three experiments were conducted. First, it was shown that switching and selection are compatible in both oscillating and non-oscillating networks. Second, it was shown that the SON3 network is the most robust and best at switching, despite SN4 being the better network at selection. Due primarily to its robustness, we concluded that SON3 was the better of the five tested networks. Third, the hypothesis that higher frequency assists switching was tested on many configurations within the SON3 network. We found a strong negative correlation between the frequency and the convergence times, supporting the hypothesis.
Numerous ways to modify SON3 have already been outlined in the discussions of the previous two experiments. These modifications includes the creation of a “module” of 4 excitatory nodes to 1 inhibitory node and linking the modules through II connections. We hypothesize that this will improve the selection times of the network. Another possible direction is to test current larger models for selection to be tested for their switching capabilities. Also, if anything, this investigation has shown that new competitive networks should be tested for both selection and switching abilities.
Lastly, connectionist networks generally do attempt to utilize oscillating activation within their networks. Having the activation oscillate in the ways outlined here has shown to make the network able to resolve competition when the competition is small. Utilizing these mechanisms will provide connectionist networks with an added dimension of processing. This also increases their resemblance to real neural processes. For example, we have found that modifying a learning module such as CALM (Murre et al 1992) to utilize oscillations will allow it to quickly learn new stimuli.
References
Börgers C., Epstein S., & Kopell, N.J. (2008) Gamma oscillations mediate stimulus competition and attentional selection in a cortical network model. Proceedings of the National Academy of Science, 105, 18023-28.
Börgers C., & Kopell, N.J. (2008) Gamma oscillations and stimulus selection. Neural Computation, 20, 383-414.
Csicsvari, J., Hirase, H., Czurko, A., & Buzsáki, G. (1999) Reliability and state dependence of pyramidal cell–interneuron synapses in the hippocampus. Neuron, 21, 179-189.
Heerebout B.T., & Phaf R.H. (2010a) Emergent oscillations in evolutionary simulations: oscillating networks increase switching efficacy. Journal of Cognitive Neuroscience, 22, 807-823. Heerebout B.T., & Phaf R.H.(2010b) Good vibrations switch attention: an affective function for
network oscillations in evolutionary simulations. Cognitive, Affective & Behavioural Neuroscience, 10, 217-229.
Luck S.J., Chelazzi L., Hillyard S.A., & Desimone R. (1997) Neural mechanisms of spatial selective attention V1, V2, and V4 of macaque visual cortex. Journal of Neurophysiology, 77, 24-42. Moran, J., & Desimone, R. (1985) Selective attention gates visual processing in the extrastriate
cortex. Science, 229, 782-784.
Murre, J.M.J., Phaf, R.H., & Wolters, G. (1992) CALM: Categorizing and learning module. Neural Networks, 5, 55-72.
Papp, E., Leinekugel, D.A., Lee, J., & Buzsáki, G. (2001) The apical shaft of CA1 pyramidal cells is under GABAergic interneuronal control. Neuroscience, 102, 715-721
Reynolds, J.H., Chelazzi, L., & Desimone, R. (2001) Competitive mechanisms subserve attention in macaque areas V2 and V4. Journal of Neuroscience, 19, 1736-1753
