Brain-Computer Interfaces: how the firing of single neurons leads to control of devices

Brain-Computer Interfaces: how the firing of

single neurons leads to control of devices

By: Jim Sellmeijer

Student number: 5729920

Literature thesis for Brain and Cognitive Sciences: Neuroscience, University of Amsterdam

Supervisor: Mariska van Steensel, PhD

Co-assessor: Carien Lansink, PhD

Date final draft: 7-8-2012

Abstract

Paralysis can be caused by spinal cord injury, brainstem strokes and a host of neuromuscular disorders. Control over external devices, such as computers or robotic limbs, could partly restore mobility and communication in paralyzed patients. A Brain-Computer Interface (BCI) makes it possible to extract the intent of the user and use neural information for device control.

A wide variety of BCIs exist: e.g. EEG- and ECoG-BCIs, which respectively use electrodes on the scalp and under the dura to measure electrical activity from whole cohorts of neurons. Recordings using intracortical electrodes are known to contain more specific information. The information content from intracortical recordings, could allow for BCI control with many more degrees of freedom in comparison to EEG- and ECoG-BCIs.

This paper is concerned with how intracortical recordings are used in BCI control. The main question of interest is; how many degrees of freedom can be controlled within a BCI using

intracortical recordings? We also have a detailed look at the most commonly used multi-electrode arrays and look into different approaches that optimize the extracted information content. Finally, we look into speculated improvements of the BCI design, such as electrical stimulation for sensory feedback and restoration of skeletal muscle function.

Introduction

Spinal cord injury, brainstem strokes and a host of neuromuscular disorders can cause paralysis. This means that the functional brain is disconnected from activating part or all of the musculature. The worst form of paralysis, locked-in state, completely inhibits voluntary muscle control. Locked-in patients are cognitively intact but no longer mobile or able to communicate. Control over external devices, such as computers or robotic limbs, could partly restore mobility and communication in these patients. If it would be possible to extract the intent of a patient it would allow for new ways of communication and potentially open a way to gain control over prosthetic limbs in people with paralysis (Hochberg and Donoghue, 2006). A system that does exactly this is referred to as a Brain-Computer Interface (BCI). A BCI allows real-time interaction between the brain and external devices. The user’s intent is extracted from the brain signals and translated into device output.

To successfully translate thought into action there are four essential steps. First, the “brain intent” signal must be detected in some real-time fashion. Second, considering noise, the information has to be rapidly extracted. During the third step, translation, the device executes the command. During the final step the user receives feedback, for instance visually, to observe whether the command was executed successfully. In the next part the different ways of signal detection, the first step of a BCI, are explained in more detail.

Signal detection can be achieved through a wide range of techniques. During the signal detection stage a sensor measures the changes in a physiological variable. This has to be done in a timescale relevant to the task performed and is usually accomplished by measuring change in the

electrophysiological brain signal. The measured signal can range from the recording of the fundamental unit of neural activity, the action potential discharge of individual neurons

(microelectrode arrays, MEA), to more widespread activity such as the synchronous firing of huge neuron populations in electrocorticography (ECoG) and electroencephalogram recordings (EEG) (Hochberg and Donoghue, 2006).

In general, the more invasive a recording technique is, the closer the recorded signal comes to the action potential discharge, and the more specific the information content. It seems plausible that the use of single unit activity as recorded by MEAs could allow for many more degrees of freedom compared to those found in ECoG-BCIs. This is because ECoG-BCIs are controlled through the modulation of certain frequency bands. The modulation of one frequency band allows for

one-dimensional control whereas single-units all code for a different aspect of brain activity. These aspects can be decoded from the signal and used to allow for more degrees of freedom when controlling a BCI. MEAs have indeed showed promising results, allowing both monkeys and humans to gain intuitive multi-dimensional control over prosthetic devices (Carmena et al., 2003, Lebedev et al., 2005, Velliste et al., 2008 and Hochberg et al., 2012). However, how many degrees of freedom can be controlled within a BCI using intracortical recordings? What are the control possibilities for these subjects and what are the drawbacks?

In this literature thesis, we will investigate in more detail the possibilities of MEA-BCIs. First, we will describe EEG- and ECoG-BCIs to gain insight in where the use of MEAs stands within the field of BCIs. From there we will go into the methods used by intracortical BCI researchers. We will look into the recording and decoding techniques to gain insight into the degrees of freedom that can be controlled when using MEA-BCIs. Additionally, we will look into the control possibilities and design drawbacks of intracortical BCIs. This paper will be concluded with a view on future prospects on the inclusion of stimulation for sensory feedback and the recovery of hand use.

EEG-BCIs

EEG recordings are commonly used as a diagnostic tool in clinical neurophysiology. The EEG signal is acquired through electrodes mounted on the scalp to record the synchronous changes in

postsynaptic potentials emanating from thousands of, mostly pyramidal, neurons. The frequencies recorded range from 0.3 to 80 Hz artificially divided into delta, theta, alpha, beta and gamma bands (Hochberg and Donoghue, 2006).

EEG-BCIs are generally controlled through learned modulation of a specific frequency band at one or more electrode locations. For example, motor imagery is known to elicit predictable changes in mu and beta frequency bands, which are spatially localized to the sensorimotor cortex (Neuper et al., 2006). Therefore, by performing motor imagery, a person is able to voluntarily control the activity in the sensorimotor cortex. These signals changes can be used to control a BCI. Scherer and

colleagues (2004) used sensorimotor imagery to allow navigation through a virtual keyboard. This allowed the participants to spell words with an average rate of 1.99 letters per minute.

A P300-based EEG-BCI operates in a different way. The P300-based BCI, designed as a communication system for patients with severe paralysis, is unique in the use of the involuntary synchronous cortical response to an external stimulus. When a participant is presented with a

computer screen displaying a grid of letters the rows and columns of which flash at random, the P300 is only generated when the letter flashes that is attended by the participant (Donchin et al., 2000). In research done by Donchin and colleagues (2000) participants were able to communicate at the rate of 7.8 characters a minute with an overall accuracy of 80%.

EEG-BCIs do not require surgery but come with a number of disadvantages. These disadvantages include: the need for extensive training, considerable set up time, harmful skin

breakdown when the electrodes on the scalp remain for extended periods, the requirement for focused attention resulting in fatigue and the limited information transfer. The worst disadvantage is the inaccurate noisy signal as a result of artifacts such as 50Hz noise. As a result EEG-BCIs are severely limited in their possibilities to aid paralyzed patients.

ECoG-BCIs

Invasive electrocorticography (ECoG) recordings are similar to EEG recordings in that they both record voltage fluctuations resulting from the ionic current flows of large groups of neurons. For ECoG, however, electrodes are placed underneath the dura on the surface of the cortex providing better spectral and spatial characteristics than scalp EEG recordings. In addition, the signal to noise ratio (SNR) at all recorded frequencies is greatly improved.

ECoG recordings are commonly used in the clinic to record in patients with medically refractory epilepsy. In these patients electrodes allow localization of the origin of the epilepsy. After localization this area is surgically removed. The localization period provides the unique possibility to record ECoG signals for research goals (Nakasato et al., 1994).

ECoG-BCIs are commonly based on the voluntary modulation of brainpower on certain frequency bands in specific brain areas. Previously Vansteensel and colleagues (2010) have shown that the ECoG signal of the dorsolateral prefrontal cortex (DLPFC) can be used in a reliable way for BCI control. The subdural electrode grids allowed the epilepsy patients to gain cursor control by serial

subtraction. Participants gained control after little practice: during silent subtraction the participants gained a success rate higher than 80% after 6 to 12 trials. In 2004, Leuthardt and colleagues published a paper in which they show that by using motor and speech imagery it is possible to control an ECoG-BCI. After a short period of 3 to 24 minutes of training the patients were able to master closed-loop control and gained 74-100% success rate in a one-dimensional binary task. In addition, it has also been shown that two-dimensional control is possible with an ECoG-BCI. In 2008, Schalk and

colleagues have shown that by using imagined and overt motor activity it is possible to gain a success rate between 53-73% in a two-dimensional four-target center-out task.

Together these findings make ECoG a promising technique for BCI goals. A number of uncertainties remain, however. First, subdural strips and grids are usually implanted for about 1 week, and only seldom for more than three weeks, as they are used for seizure localization in epilepsy patients. BCI electrodes should allow for lifelong recordings. In terms of longevity of the ECoG electrodes, neither preclinical nor human evidence of ECoG’s ability to record safely or effectively for extended periods of time has yet been shown. Recordings can be complicated by hemorrhage,

infection, infarction, cerebral edema and even death (Hamer et al., 2002). Previous research to assess morbidity in intracranial recordings and surgical outcomes reported major complications in 6.6% of ECoG recordings in epilepsy patients. 1.5% of these patients experienced infections, 2.5% represented residual morbidity. Additionally, 3.0% of the patients experienced symptomatic hematomas.

Replacing the current electrode grids with smaller and fewer ECoG electrodes might prove to be a safer way to record from the cortex for an extensive period of time. Second, as mentioned above, ECoG-BCI has been relatively successful in simple one- or two-dimensional control tasks and is therefore promising for relatively simple communication purposes. However, the ability to control robotic limbs with only one- or two- degrees of freedom to the same degree as control of natural limbs seems highly unlikely.

Intracortical Microelectrode Array recordings

Apart from recording on the surface of the cortex it is possible to perform intracortical recordings. Compared to the relatively flat electrodes mounted to the skull or under the dura in EEG and ECoG the electrodes used for intracortical recordings are very different in that thin needle electrodes are implanted in the cortex. Intracortical recordings do not particularly record from huge ensembles of neurons as compared to EEG and ECoG recordings. Instead, local field potentials (LFPs) and single-units can be recorded. As the brain operates individual neurons communicate through minor electrical currents. The sum of these minor currents produces LFPs. The LFP signal reflects activity from approximately 50 to 350µm from the tip of the recording electrode and indicates the degree of coordinated activity among multiple local neurons in the region of intracortical electrode implants

(Hochberg and Donoghue, 2006). LFPs have received only little attention in the development of MEA BCIs. Instead, the firing of single-units is most commonly used.

The fundamental means of communication in the brain is the action potential discharge of single neurons. The ability to record single-unit activity from microelectrodes depends on a number of factors. The electrode impedance, tip size and shape and the size and orientation of the target neuron all influence the single-unit signal (Schwartz et al., 2006 and Patil and Turner, 2008). Additionally the conductance of the extracellular space influences the size of the measured current (Patil and Turner, 2008). In order to record action potentials one needs sufficiently small electrodes. As the axons are too small to record from, the discharge is recorded from the cell body. When a cell reaches the depolarization threshold the current’s return path through the conductive extracellular fluid creates a dipole which in turn creates the extracellular electrical field potential. To record from a maximally large set of single neurons a number of recording electrodes are organized in a multi-electrode spatial array.

The Utah Intracortical Electrode Array

In theory the firing rate of a single neuron could allow for a BCI to control an automated device. However, a large number of recording electrodes, organized in an array, could provide for more complex control. This in turn could allow for cognitive control over a number of devices or one device with multiple degrees of freedom. A number of different microelectrode arrays exist (for review see Schwartz et al., 2006) but as the Utah Intracortical Electrode Array (UIEA) is most commonly used for long-term single-unit BCI studies and is sold as a standardized product this paper will primarily focus on this three-dimensional silicon array.

The UIEA consists of a 10 by 10 grid of electrically isolated needles. Each needle is 1.2 mm long and etched to have a sharp tip. The needles are held together by a 0.2 mm thick block of monocrystalline semiconductor grade silicon with a 4.2 mm2 _{surface (Jones et al., 1992). Electrodes}

consisting out of layers of platinum, titanium-tungsten and platinum provide an electrical interface between the neural interface and the surrounding tissue. In turn, to provide the desired impedance the electrodes are coated with polyimide with the tips exposed.

In comparison to ECoG and EEG electrodes, MEAs in general have been getting a lot of critique about the electrode-tissue interface. As MEA electrodes penetrate the cortex there are indeed some problems that ECoG and EEG electrodes avoid. Even though MEAs provide for high quality signals, most of the published research has not shown long-term performance (as reviewed by Schwartz et al., 2006). After implantation, the number of electrodes that record single-unit activity declines. The recording quality is found to vary across both subjects and electrodes within a subject. The main reason is thought to lie within the immune response triggered by the implanted electrodes.

The mismatch between host tissue and electrode can trigger neuronal loss and glial encapsulation (gliosis). This in turn increases the distance between the neurons and the electrode, which results in signal loss. Biran and colleagues (2005) have examined the effects on implantation of electrode arrays on neuronal density and immune response with immunohistology. They indeed found that when rats were implanted with a silicon electrode array it was possible to find indications of an activated immune response. Persistent ED1 immunoreactivity, a marker of rat microglia and macrophages, was found after 4 weeks of implantation. This was accompanied by a significant reduction in nerve fiber density and cell bodies in the tissue surrounding the microelectrodes. Electrodes were also found to be encapsulated with ED1 immunoreactive cells. This immune response might be due to the mechanical trauma created by the electrodes. As the animal moves around, the brain may move minor distances relative to the electrodes fixed to skull. This in turn could cause tissue rupture and the immune response to get activated.

For the Utah array, the situation may be slightly better, however. First, smart design seems to have ‘solved’ a number of these issues in BCI research. In the UIEA originally designed by Normann and colleagues, the electrodes are continuous with a planar silicon base which provides a platform that rests on surface structures. To eliminate movement of the array in proportion to the brain, due to fluctuations in blood pressure and respiration, the array itself is not fixed to the skull. Instead, the array floats on the cortex and the wires are mounted to the cranium. This ‘self-anchoring’ feature of the array provides for exceptionally stable recordings over long time periods in comparison to other MEAs. The thin silicon substrate and the architecture make grant for on-chip amplifiers to the substrate while still preserving the small size. Additionally, the array is made from biocompatible materials which enhance stability. Second, other authors have suggested that the effects of

immunoreactivity might not be applicable for the UIEA. In 2005, Suner and colleagues published a paper on the reliability of signals from the UIEA in non-human primate primary motor cortex. The recording variability was evaluated up to 569 days after array implantation. They discussed three measures to show that the overall system remained the same quality across this period. First, the signal-to-noise ratio (SNR) was found to remain stable over time. Shortly after implantation they did find fluctuations in signal quality. The authors hypothesized, that is due to the initial damage and tissue reaction caused by implantation as these fluctuations later stabilize. The second measure was impedance. When the electrodes are overgrown by isolating tissue the impedance would increase and signal quality would drop. In some, but not all monkeys impedance increased over time. However, signal quality and SNR did not decrease. The third measure is waveform appearance. The authors reported no correlation between implantation time and overall waveforms. They showed that no failures could be attributed to a biological response around the sensor as recording quality did not decrease over time. More evidence for the longevity of UIEAs came from Simeral and colleagues (2011), who published a paper to estimate how long implanted microelectrodes can record useful neural signals, how well these signals can be decoded and how effectively they can be used to control

various machines such as computers and robotic devices in humans. Simeral and colleagues went back to the now famous patient ‘S3’ and addressed these questions 1000 days after implantation. Patient S3 suffered from a brainstem lesion stroke rendering her tetraplegic. After the implantation of a UIEA she was able to reliably control a computer mouse, which will later be discussed in more detail (Hochberg et al., 2006). 2.7 years after the implantation, results across 5 consecutive days showed that the neural interface system could still provide a repeatable, accurate point-and-click control of a computer interface. In 2012 a paper was published by Hochberg and colleagues showing that 5.3 years after implantation patient S3 was able to control a robotic arm. Interestingly, Hochberg and colleagues (2012) stated that even though they were able to record and use the UIEA for

prosthetic control 5 years after implantation, they did notice decline in spike amplitude. In addition, they noticed fewer channels contributing to the filter than during the first year of implantation. From a functional perspective however, it can be concluded that usable single-unit activity can be recorded long after initial implantation.

Despite the positive results obtained with the UIEA, the drawbacks have inspired researchers to create alternatives to the established electrode array. For example, Nicolelis stated that the UIEA provides only modest neuronal yields and proposes his own technique for long-term intracortical recording (Nicolelis et al., 2003). He implanted rhesus monkeys with MEAs totalling 96-704 microwires per subject. He states that, as he lowers the electrodes slowly over time (100 µm per minute) his technique is in sharp contrast to the UIEA. This and the increased number of

microelectrodes may contribute to the high amount of recorded single neurons (421 neurons during the first month after implantation, compared to ± 12 neurons for the early stages of UIEA recordings as reported in Maynard et al., 1999). However, after 18 months ‘only’ 247 single-units remain. This slow implantation technique seems usable for research purposes but lifelong use of BCIs demands more stable recording techniques.

Although Nicolelis’ (2003) recordings might not yet have shown the same stability as UIEA recordings, the increased population size has one major advantage. In a recently published article, Lebedev and colleagues (2011) state that large neural ensembles offer redundancy in the signal; this can be used to increase the SNR. They state that a realistic SNR is achieved with the simultaneous recording of 1000 neurons. They plan to increase the recorded single-units to about 1500. In order to achieve this they propose a new method in which electrodes are arranged in guiding tubes. These electrodes have different lengths creating a three-dimensional recording cube. They state that this cube improves the size of the recorded sample but also enhances implantation capacity. In addition, they wish to remove the structural elements after implantation. This will free the electrodes, thereby minimizing gliosis and other immune responses. Whereas these plans seem promising they still have to prove their use.

In conclusion the UIEA is a reliable effective MEA to use in long-term intracortical recordings. Other recording techniques seem promising but have yet to achieve the results of the UIEA. In the next section we will discuss the information content of single-unit recordings.

Information content single-unit recordings

There is a lot known about the information content of single-unit activity in a wide spectrum of brain regions. In a number of experiments a relationship can be found between the executed behavior and the recorded neuronal data. For instance in the paper by Lee et al. (2004) some hippocampal pyramidal neurons from rats fire only on specific locations within a circular maze. These cells were discussed to code for place. Georgopoulos and colleagues (1988) described a code in which a

population of motor cortical neurons could determine uniquely the direction of reaching movement in a three-dimensional space. They suggested that even though single cells have a preferred direction, the neuronal ensemble codes for the direction. This is reflected in the way that many cells become active during movement and not just one with the preferred direction corresponding to the movement direction. In order to show this they used 475 directionally tuned cells from the motor cortex of 2 rhesus monkeys. To mathematically define movement direction two assumptions were made. One, a neuron makes a contribution with both magnitude and direction. Two, the sum of these individual contributions create the outcome of the ensemble operation. Following these assumptions, the contribution of single-unit from the ensemble depends on the discharge rate related to the movement in a specific direction. They showed that using the discharge rates of single-units with a preferred direction a population vector could be obtained accurately predicting the direction of the movement in a three-dimensional space. The more cells they used the higher the confidence of the prediction.

In 2002, Kreiman and colleagues recorded single-unit activity with depth electrodes in the medial temporal lobe of human subjects. During the task a stimulus was presented to one eye. After this, the same stimulus was flashed to the same eye at the same time a different stimulus was flashed onto the contralateral eye. During debriefing the participant had to describe what they had seen. They discovered that a proportion of medial temporal lobe neurons to follow the percept of the participant. In addition to single-unit activity in visual areas, the activity of neurons in the auditory cortex has been found to correlate with auditory stimuli. In a task where rhesus monkeys had to localize sounds of different frequencies and bandwidths, neurons in the auditory cortex of the Macaca mulatta responded to these parameters. Around 80% of the neurons in the auditory cortex and the caudomedial field were modulated by the spatial localization of either a tone or a noise stimulus (Recanzone et al., 2000).

In conclusion the above findings show that single-unit activity holds information from a wide range of cognitive processes and executed behaviors. If this information can be controlled voluntarily it may prove useful as a control signal for BCIs.

The beginning of a single-unit BCI

Early experiments have shown that the signal from a single intracortical microelectrode has potential as a signal for BCI control (Humphrey et al., 1970 and Schmidt, 1980). In pioneering work of Humphrey and colleagues (1970) a monkey was trained to pull a vertically oriented handle and move it up or down by alternating between flexion and extension of the wrist. Transducers converted the displacement and force exerted by the monkey on the handle to electrical signals that were fed to a light that indicated when he flexed or extended his wrist. The monkey was trained to move the handle up and down for about 1 second before moving to the next position. As the monkey performed these simple hand movements, several individually selected neurons from the motor cortex were recorded. Spikes were counted in bins of 50 msec for each unit together with the values of the force and displacement traces related to the task. By using cross-correlation the researchers found that spike frequency and the response measures correlated most when the response measures were shifted backwards by 100 msec. This means that neuronal data tells something about the behavior executed 100 msec in the future. From this data empirical transfer functions were derived that allowed the neuronal data to be used for off-line prediction of the simultaneously recorded response

measurements. This was done by simply multiplying the firing frequency of the individual neurons by constant coefficient and then summing the resulting values across the set of units at each point in time.

Accuracy was highest when the discharge patterns of the single-units correlated with a single phase of the movement but not other phases. This means, for instance, that the accuracy to predict force would drop when the neuron also codes for direction. When the spike frequencies of more units are added to the equation the accuracy rose because of the increase of relevant signal in comparison to noise, indicating that the firing frequencies of more single-units allow for more accurate predictions of executed behavior.

Schmidt hypothesized the use of single neuron activity for the control of external devices after finding that monkeys could voluntarily control the firing of single-units. In his animal study electrodes were implanted in the monkey motor cortex. After surgery the majority of the electrodes recorded single-unit activity. The monkey learned to control the firing rate of a single cortical neuron under an operant conditioning paradigm. During the task the monkey was seated in a primate chair facing a wall with 8 lights. These lights, aligned in a row, indicated the target firing rate. The first light indicated the lowest frequency and the eighth light indicated the highest frequency. Each of these

lights was surrounded by a circle of lights indicating the actual firing pattern. The monkey quickly learned to alternate the firing rate in order to get a juice reward. Thus, one of the first BCIs using one degree of freedom, namely firing rate of a single neuron, was established. In his study he assessed the information outflow rates based on the target firing rate and neural controlled firing rate. Information outflow rate was defined as the number of correct responses multiplied by the targeted information content in bits divided by the time on the target. His calculation showed that the best performance obtained contained 2.45 bits/sec. When the monkeys executed the same task but now using the motor system instead of the BCI, the information rate increased to 4.48bits/sec. From these results he concluded that neuronal output provides information only moderately less precise than the intact motor system. The question remains, how can this information harnessed for multi-dimensional BCI control?

Usage of signal features for BCI

Before single-unit activity can be used to control a BCI, it must for be extracted and decoded. A number of different techniques exist to extract and decode the signal features. Preferred direction of populations of neurons was used by Velliste and colleagues (2008) to allow monkeys control a prosthetic arm for self-feeding. For recording two monkeys were both implanted with different MEAs. One monkey was implanted with the UIEA as described earlier. The other monkey was implanted with four 2 by 8 grids, totaling 64 channels. Both grids were implanted on the right primary motor cortex. During this experiment monkeys were presented with a piece of food on a device that slowly moved towards the monkey. When the piece of food was within arm reach the monkey was allowed to grab it and eat it using one-dimensional joystick control over a robotic arm. Joysticks were used because, in contrast to experiments with humans, it is impossible to verbally instruct on how to use the robotic arm when the prosthetic is controlled through brain activity. Using a joystick allows a transitional phase in which the monkey can use the joystick when in fact the robotic arm is controlled by single-unit activity. One-dimensional control was chosen above three-dimensional control as immediate three-dimensional control posed too big of a challenge for the monkeys. For this reason the monkeys were gradually introduced to more dimensional control. During three-dimensional control trials only the gripper was automated and finally during four-dimensional control the monkey gained command over the gripper as well.

The information was extracted by obtaining a population vector of preferred directions. In short, the extraction algorithm was based on three parameters, baseline firing rate, modulation depth and preferred direction. In the self-feeding task the population vector had components for each dimension that was extracted. The first three dimensions were interpreted as endpoint velocity. The final dimension was used to control gripper aperture. The values were sent as commandos to the robot

control software. For calibration, the algorithm allowed tuning parameters to be estimated using data collected during natural arm movement. Eventually, the monkey gained full control of the robotic arm without the aid of preprogrammed movements. It is important to stress that cortical control of the prosthetic arm was essentially gained by creating a vector sum of the preferred directions of the recorded unit population. The monkeys did not have control over the joints of the robotic arm.

In conclusion Velliste and colleagues showed that monkeys were able to modulate single-unit activity to allow for multi-dimensional arm prosthetic control. In addition to this study a number of researchers have studied which signals are optimal for BCI control. In Nicolelis’ group, for instance, it was studied what the effect is of using multiple motor oriented regions.

In research done by Nicolelis’ group monkeys were trained to control robotic arms using BCIs (Carmena et al., 2003 and Lebedev et al., 2005). In these experiments monkeys were implanted with MEAs in the primary motor cortex, dorsal premotor cortex and the supplementary motor cortex. In addition, one monkey had a MEA implanted in the primary somatosensory cortex. By pointing the joystick in a particular direction the monkeys were able to control cursor direction and by squeezing the rod the cursor would enlarge. The monkeys were then trained to control the cursor on the

computer screen by modulating their brain activity. During later stages a robotic arm was introduced. This robotic arm was controlled by the same signals as the onscreen cursor was previously controlled. To train the monkey three tasks were used; a reaching task, a hand-gripping task and a reach-and-grasp task. During all three tasks, population vector were calculated to determine the intended hand direction and force-related modulation to encode gripping force. Neurons in multiple frontal and parietal cortical areas showed task-related modulation of firing rates. For instance, the primary motor cortex neurons were the best to predict motor activity. However, the supplementary motor cortex was also very useful to predict hand position (not used for BCI control) and velocity. Combining the information from multiple brain areas provided far superior predictions than using only one brain area. Thus, these experiments show that as more neurons from a variety of different brain areas contribute to the population vector the prediction becomes more accurate and the ability to control a BCI improves.

Other studies have investigated the effect of the use of velocity or location information, or both, on BCI control (Hochberg et al., 2006, Kim et al., 2008). In 2006, Hochberg and colleagues showed that tetraplegic humans could operate a BCI based on decoded position parameters from motor cortical activity to operate a computer cursor in a two-dimensional plane. Years after spinal cord injury the motor cortex was still able to activate these signals by imagined and attempted movement. However, even though success rate was above chance level cursor control was not very accurate. Therefore Kim and colleagues (2008) started a second study in which they applied the same and a different decoding technique to extract information. During the initial training procedures (similar to the procedures Hochberg and colleagues used) a cursor was displayed on a computer monitor, which was moved in

order to generate cursor trajectories. During this training stage, the participants were instructed to imagine moving their hand as if they were controlling the onscreen cursor. Activity patterns during the imagined movement and the training cursor kinematics were combined to train decoding

algorithms. For the two conditions different aspects of cursor kinematics were used. The first was the same as in Hochberg’s paper, namely cursor position. Neurons whose activity correlated to cursor positions were identified and used to train decoding algorithm. For the other condition, neurons were used whose firing patterns contained velocity information. This was done by exploiting the directional tuning of motor cortical units. Velocity-based decoding was found to be the most accurate. This was in part due to the different amount of single-units involved in velocity and location encoding. 69.9% of the recorded single-units were found to show correlated activity to velocity where only 25.9% showed activity related to cursor location. When a low count of velocity encoding single-units was used, performance was still superior to location-only-based decoding. When both location and velocity information were used performance was inferior to when only velocity information was used. This suggests that it may be important to choose the appropriate kinematic representation to achieve smooth cursor control. In addition it is shown that decoding velocity information is useful for BCI control.

Kim and colleagues suggested that the approach mentioned above used for onscreen cursor control may extend to other types of devices such as robotic arms. Hochberg and co-workers (2012) published a paper in which they investigate the control possibilities for using an arm prosthetic. For this study two tetraplegic patients, implanted with a UIEA on the primary motor cortex, were used. One of these patients, patient S3, also contributed to the study described above (Kim et al., 2008) and was implanted 5 years prior to the experiments using an arm prosthetic (Hochberg et al., 2012). Two different types of arms were used. One, the DLR Light-Weight Robot III, had 7 degrees of freedom for the arm and 15 degrees of freedom (DoF) for hand control. The other robotic arm, DEKA generation 2 prosthetic, had 6 DoFs arm movement and 4 DoFs for hand grasping. For both robotic arms the hand grasping DoFs were combined into a single DoF. This means that the participant was only able to control opening and closing the hand but not the different stages of these actions separately. Similar to the cursor control task described above, hand velocity was determined by calculating a population vector coding velocity. A neuronal controlled state switch determined opening and closing the hand. For grasping target foam balls were used. These foam balls were attached to rods that would lift the ball to a specific location. The participant was required to bring the robotic arm towards the foam ball and grasp it using the robotic hand. To investigate the use of a BCI in everyday life another task was designed. The participant was asked to pick up a bottle of coffee from a table and eventually drink from it through a straw. For this task velocity control was restricted to the two-dimensional table top. This means that the participant was not able to move the arm upwards or downwards as prosthetic control was fixed to a two-dimensional plain. This task was harder than the ball grasping task as the bottle’s diameter was 90% that of the hand aperture.

Velocity information was extracted by recording extracellular action potentials from the neural activity. For calibration, the participant was asked to imagine controlling the robotic arm as it executed preprogrammed movements. This open-loop filter calibration was followed by a closed-loop filter calibration. In this type, the participant gained control over the robotic arm. During the initial trials errors were used to scale down the decoded movement in the direction of the error. This was done until the decoded movement corresponded to the intended movement derived from both target and robotic hand location. Hand grasping was decoded in the same way. The deviation of each single-unit’s baseline firing rate was used to decode endpoint velocity and grasping state. One problem with this method is that firing rates may drift over time. In order to reduce the consequences of this phenomenon the baseline firing rate were re-estimated after ach block using data from the previous block. To select single-units to include in the filter, units were selected on the magnitude of the preferred direction vector. The amount of channels included in the filter ranged from 13 to 50. Together these units formed a population vector that determined hand velocity. The approach was found to be sufficient to provide patients without motoric arm function, with substantial control over the robotic arm with only minimal. Even though control was not as fast and accurate as that of able-bodied humans, these findings demonstrate that complex multidimensional control is possible by recording from a small sample of neurons.

In 1999 Maynard and colleagues demonstrated that more information reside in the firing rate than preferred direction alone. They showed that the interaction among groups of neurons in the primary motor cortex contained more information about motor behavior. In Georgopoulos’ studies response variability was treated as a noise factor (Georgopoulos et al., 1988). Maynard et al. suggested that this assumption is questionable because individual neurons in the primary motor cortex always represent motor variables and weak correlations can be encountered in firing rate variability of pairs of neurons (Maynard et al., 1999). Instead, it was hypothesized that the variability, instead of being noise, can provide additional information about motor execution. In order to test the hypothesis, monkeys were trained to move a two joint joystick in the horizontal plane in 8 directions. A UIEA was planted in the arm area of the primary motor cortex. During the analysis the firing pattern of every single-unit was correlated with all other single-units. The dependence of the resulting correlation coefficients on direction preference was investigated. It was hypothesized that a strong relation between similarity of directional tuning and response correlation of two cells would be expected if the correlation resulted from redundant information about movement. The data, however, revealed only a weak relationship between directional preference and correlation strength. For instance, two cells could have a similar firing rate for one direction but show no similarity in firing rate for another direction. The authors reported this to be the case for 78% of the cell pairs. This in turn indicates that correlation strength does not have to be a fixed property of cell pairs but may be dependent on movement direction.

In order to investigate whether correlations can contribute to directional coding a comparison was made between two models. One model viewed the neurons as being independent of each other, ignoring interactions, where the other model incorporated second-order relations. These models were used to classify single trials into one of eight movement directions as executed by the monkeys. A trial would be classified as successful when the direction chosen by the model was the same as the actual direction during the task. Results showed that when the classification model incorporated second-order relations classification improved with 11%.

In addition to single-unit activity, local field potentials (LFPs) have also been shown to hold relevant information for BCI control. In 2010, Zhuang and colleagues published a paper in which LFPs along 7 different frequency bands (0.3 - 400 Hz) were used to decode three-dimensional reach and grasp kinematics. For this study two monkeys were implanted with UIEAs on the primary motor cortex. Using mutual information analysis they showed that higher frequency bands carried the most information about the executed movements. Using a Kalman filter to reconstruct reach and grasp kinematics they revealed that high frequency LFPs might be useful to control three-dimensional reach and grasp kinematics in a BCI.

Low (<2 Hz) and high frequency (30 – 400 Hz) bands appear to be related to ongoing sensorimotor processing. Mid-bands (2 - 30 Hz) may be coupled to attentive processes and contain state information (as reviewed by Bansal et al., 2012). To understand the relationship between the different frequency bands and to reveal the underlying information content, Bansal and colleagues (2012) compared the information between spikes and different LFP frequency bands. For this study monkeys were implanted with UIEAs in both the primary and ventral premotor cortex and trained to perform a continuous grasping task. The information content was assessed by calculating the decoded accuracy for three-dimensional endpoint and grip aperture. This was done for single-units and

multiband LFPs from both implanted areas. It was found that the information from LFPs was not independent from single-unit activity. Even more so, the information within the LFPs was contained within spiking activity. Both implanted areas did not show a particular bias towards any of the different aspects of the task. Therefore it was concluded that, in the light of BCI research, neuronal ensemble spiking is the preferred signal for decoding. Information within LFPs and the combined signals from the primary and the ventral premotor cortex could only add to the robustness of the extracted information.

Filters & Classifiers for single-unit analysis

The choice of extraction filters has been shown to influence decoding efficiency. In the previously mentioned research by Kim and colleagues (2008) two different types of filters were used. Location

based information was extracted using a linear filter that predicted each dimension (x and y

coordinates) using a combination of the current and past firing rates. The linear filter builds a finite impulse response filter with a 1 s memory of the firing rate history. The velocity based information was extracted using a Kalman filter. This filter creates the cursor kinematics from the history of firing rates. The Kalman filter uses recursive equations, integrating the information from the history of neural signals. The Kalman filter is different from the linear filter in that it also incorporates a prior model of cursor movements. As control, the linear filter was also used to decode velocity information. This was found to be better than using location-based information with a linear filter but worse than using velocity information and the Kalman filter.

Lebedev and colleagues (2011) propose that, instead of using a regular Kalman filter, using an unscented Kalman filter for better BCI accuracy. In contrast to the Kalman filter, the unscented Kalman filter exploits non-linear models of neural tuning and prior knowledge about movement patterns. In addition, they propose to use a multiple-model-switching paradigm in which several submodels are trained to decode behavioral states. A state predictor model would then detect the state and select the appropriate model for execution. In theory, the simplest switching model would consist of three linear decoding models, one for predicting state 1, one for predicting state 2 and a third state to toggle between these models.

In a more complex model holding multiple different states, the kinematics of each joint of a limb would be predicted and converted into the wanted configuration. According to Lebedev and colleagues, this technique resembles the way that the central nervous system executes natural

movement. It is inconvenient for a patient to control each degree of freedom consciously. Therefore it would be salable to have the user be in charge of higher-order control and let an autonomous

controller be in charge of low-level coordination of the movement. The combination of shared control together with increased BCI accuracy seem very promising to create a BCI that mimics natural movement. However, one additional feature should be included; sensory feedback.

Future prospects

A complete solution; including sensory feedback

Artificial sensory feedback has been hypothesized to be of use in BCIs (for review see Bach-y-Rita and Kervel, 2003 and Lebedev et al., 2011). In sensory substitution, information from a device is transmitted to the brain, the exact opposite of a BCI. The brain is able to use this information to create the illusion of a natural sensory experience.

At the Duke University Center for Neuroengineering owl monkeys were instructed to perform arm reaches through intracortical microstimulation (Fitzsimmons et al., 2007). In this study, the

monkeys were implanted with MEAs in several cortical areas. Implants in the primary sensory cortex used to deliver spatiotemporal patterns of stimulation. During the training stages monkeys were trained to choose between two boxes. One box would hold a food reward indicated by vibrotactile cues applied to the upper arm. In a later stage, instead of using vibrotactile stimulation, different spatial patterns of electrical microstimulation on the sensory cortex were used as cues. The stimulated sites corresponded to the receptive fields located on the monkeys’ hands. Monkeys were able to select the box containing the food reward more often than chance level (50%). After 40 trials both monkeys were reported to select the correct box 90% of the time. The animals were able to relearn the

paradigm when the electrical stimulation cues were reversed or when new stimulation trains were applied. These results suggest that microstimulation might be used in BCI applications that require sensory feedback.

Intracortical microstimulation has also been used to cue monkeys to perform certain BCI reaching tasks. O’Doherty and colleagues implanted two macaque monkeys with MEAs in the primary motor and sensory cortex (2009). The monkeys were initially trained using a joystick with two DoFs (left-right and forward-backward) to control an onscreen cursor. The monkeys used the cursor to perform a center-out task and a continuous target pursuit task. Additionally, a target choice task was employed. During this task the animals received tactile stimulation to indicate whether the animal would direct the cursor to the upper or lower onscreen target. For each of these tasks, online closed-loop experiments were performed in which the monkeys directed the cursor by modulating brain activity (as described in the previous chapters). After 12 sessions one of the monkeys was able to correctly choose the intended target upon vibrotactile stimulation in 90% of the trials. When the task was performed using intracortical stimulation of the primary sensory cortex, the monkey’s performance decreased towards chance level. It took 15 sessions for the animal to relearn the paradigm and use the information presented by intracortical stimulation to select the required target. These findings indicated that intracortical stimulation can be incorporated in a BCI.

Lebedev briefly describes a yet unpublished study in which intracortical microstimulation serves as an artificial sense of touch as monkeys control virtual objects (for review see Lebedev et al., 2011). The incorporation of artificial sensory feedback within the BCI creates a system referred to as a Brain-Computer-Brain Interface (BCBI). Two monkeys are able to operate this BCBI without the need to move their own limbs or receive visual feedback.

In conclusion the above mentioned experiments demonstrate the possibility of creating a BCBI in which brain tissue and artificial actuators are connected bi-directionally. However, the use of electrical stimulation has a few drawbacks (as reviewed by Lebedev et al., 2011). For instance, electrical artifacts as a result of the stimulation could saturate neural recording channels and electrical stimulation has only limited spatial resolution. This led Lebedev and colleagues to propose a new device for intracortical stimulation based on optogenetics. Optogenetics uses genetically modified ion channels that respond directly to light. These light-gated ion channels allow for more precise

millisecond stimulation of selected neurons. As stimulation is done through deploying light to the neurons it no longer affects the physiological recordings. The light-gated ion channels will be

delivered by injecting adeno-associated virus vectors to the targeted part of the somatosensory cortex. If optogenetics-based microstimulation can deliver the same intensity as electrical stimulation and remains stable over a long time period (as is the case with MEAs), it might improve the BCBI concept. However, as described, the inclusion of sensory feedback has only recently been used in animals and it will therefore take some time before this is applied to humans.

Restoration of hand use; functional electrical stimulation

In addition to stimulation to create artificial sensory feedback, electrical stimulation might also provide the restoration of hand movement. Pohlmeyer and colleagues (2009) published an article in which they used a monkey’s paralyzed arm as an ‘arm prosthetic’. During these experiments neural signals were recorded using UIEAs implanted in the motor cortex. The initial training stages involved computer cursor control by isometric wrist force. Using both linear and non-linear models multiple-input impulse responses between the neural signal and recorded muscle activity were calculated. This information was used to control simultaneous stimulation-driven contraction of four paralyzed muscles. Using this electrical stimulation set-up, the monkeys were able to control the magnitude and time course of the force with sufficient accuracy. Movement speed was reduced by only one-third to one-half of normal arm movement.

In conclusion these experiments indicate that functional electrical stimulation might potentially restore voluntary control of basic hand movements in spinal cord injured patients. The authors state that there is no obvious reason why their methods would not be applicable for a larger quantity of skeletal muscles. When results from functional stimulation are as promising as those found in intracortical BCI research, these methods might replace the prospects of robotic arm use in a subset of paralyzed patients.

Conclusion

In this literature thesis we looked at the features of intracortical BCIs in order to assess the amount of degrees of freedom that can be controlled when using this recording technique. In addition, we looked into the positive and negative aspects of using MEAs in comparison to other established BCI

techniques.

Multi-dimensional control is established by providing a combination of population vectors and state switches based on the firing patterns of single-units, sometimes with the inclusion of LFPs. Generally,

a population vector is calculated based on velocity information which allows movement within a three-dimensional environment. In robotic arm prosthetics a fourth dimension is added, by incorporating a state switch to allow the opening and closing of a gripper. Within the

three-dimensional environment the population vector can be directed towards all directions. Stating that an intracortical BCI allows for movement in 129600 (360 x 360) directions and therefore has this many degrees of freedom is exaggerating. Basically, the direction of the population vector constitutes of four degrees of freedom, namely; the magnitude of the population vector (velocity) and the three dimensions within the three-dimensional space.

ECoG-BCIs have been relatively successful in one- and two-dimensional control in which each dimension is controlled consciously. In comparison to ECoG-BCIs, intracortical BCIs are a step forward. Both multi-dimensional control and user comfort, as each degree of freedom is no longer controlled separately, are significantly improved.

Intracortical BCIs might allow for multi-dimensional device manipulation; however the level of control is not as accurate as natural movement. Patients and monkeys using robotic arms succeed in performing their respective tasks during the research but movement remains gawky in comparison to natural movement. This might be due to the way the participants are instructed to control the

manipulator and how the information is ultimately decoded and used for manipulation of the device. In experiments involving human participants, a researcher controls the prosthetic while the participant is instructed to imagine as if he or she controls it. The brain activity is then recorded and a population vector is decoded to allow the participant to control the robotic arm or cursor in the next stage of the experiment. Especially in respect to the robotic arm control, the participant imagines as if the robotic arm is his or her own arm, incorporating their own imagined joint movement and arm posture. Population vectors generally only decode hand velocity and location, ignoring other aspects found in natural movement such as posture and joint bending. Some of the cells incorporated within the population vector might code for direction alone, whereas other cells should code for other aspects of natural movement. The firing of these cells corresponds to the direction of hand movement but may not code for direction. In theory, the hypothesized multiple-model-switching paradigm could

potentially extract this information from the intracortical signal. The incorporation of more movement related aspects from the signal could improve the emulation of natural limb movement.

Research has also shown that instead of treating response variability as a noise factor it can also be incorporated within the BCI to allow for the inclusion of second-order relations. Incorporating these second-order relations has been shown to increase the information content of the signal. This added information might improve BCI control.

Another way to improve BCI accuracy is the incorporation of single-units from multiple brain areas instead of only one. The primary motor cortex was shown to be the best predictor of motor activity whereas the inclusion of the supplementary motor cortex was useful to predict hand position

and velocity. Combining the information from multiple brain areas provided far superior predictions than when only one brain area was used. Additionally, increasing the amount of single-units recorded can increase redundancy in the signal hereby increasing the signal to noise ratio. Nicolelis and colleagues (2003) showed that when electrode arrays are implanted slowly and more electrodes are implanted more neurons are yielded.

BCIs can be further improved by the incorporation of sensory feedback. The inclusion of stimulation of sensory brain areas could allow for a sense of touch in the arm prosthetic. This could lead to a more natural control of the arm prosthetic without the need of continuous visual feedback.

Electrical stimulation of skeletal muscles has been shown to partly restore muscle control. This might allow for an even more intuitive and natural BCI. However, more research has to be done in order to explore the viability of this concept.

In conclusion, intracortical BCIs are shown to alleviate a number of the short comings of EEG- and ECoG-BCIs, by improving both the signal to noise ratio and minimizing infection risks. UIEA implantation allows for long-term single-unit recordings at least up to 5 years. The biggest advantage of intracortical recordings, the high information content of the single-unit signal, allows for more intuitive multi-dimensional control without the need to train the manipulation of certain frequency bands. No longer is there the need to control each degree of freedom consciously. The usage of population vectors allows the user to control cursors and robotic arms to perform basic tasks to a satisfactory degree. An intracortical BCI, incorporating a large amount of single-units and more complex decoding techniques could potentially emulate natural movement with greater accuracy and versatility. Therefore, in the near future, intracortical BCIs might be used to significantly improve the quality of life of tetraplagic patients.

References

Bach-y-Rita, P., & W. Kercel, S. (2003). Sensory substitution and the human–machine interface.

Trends in Cognitive Sciences, 7(12), 541–546. doi:10.1016/j.tics.2003.10.013

Bansal, A. K., Truccolo, W., Vargas-Irwin, C. E., & Donoghue, J. P. (2012). Decoding 3D reach and grasp from hybrid signals in motor and premotor cortices: spikes, multiunit activity, and local field potentials. Journal of Neurophysiology, 107(5), 1337–1355.

Biran, R., Martin, D. C., & Tresco, P. A. (2005). Neuronal cell loss accompanies the brain tissue response to chronically implanted silicon microelectrode arrays. Experimental Neurology,

Carmena, J. M., Lebedev, M. A., Crist, R. E., O’Doherty, J. E., Santucci, D. M., Dimitrov, D. F., Patil, P. G., e.a. (2003). Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates. PLoS Biol, 1(2), e42. doi:10.1371/journal.pbio.0000042

Donchin, E., Spencer, K. M., & Wijesinghe, R. (2000). The mental prosthesis: assessing the speed of a P300-based brain-computer interface. Rehabilitation Engineering, IEEE Transactions on,

8(2), 174–179. doi:10.1109/86.847808

Eric C Leuthardt and Gerwin Schalk and Jonathan R Wolpaw and Jeffrey G Ojemann and Daniel W Moran. (2004). A brain–computer interface using electrocorticographic signals in humans.

Journal of Neural Engineering, 1(2), 63.

Fitzsimmons, N. A., Drake, W., Hanson, T. L., Lebedev, M. A., & Nicolelis, M. A. L. (2007). Primate Reaching Cued by Multichannel Spatiotemporal Cortical Microstimulation. The Journal of

Neuroscience, 27(21), 5593–5602.

G Schalk and K J Miller and N R Anderson and J A Wilson and M D Smyth and J G Ojemann and D W Moran and J R Wolpaw and E C Leuthardt. (2008). Two-dimensional movement control using electrocorticographic signals in humans. Journal of Neural Engineering, 5(1), 75. Georgopoulos, A. P., Kettner, R. E., & Schwartz, A. B. (1988). Primate motor cortex and free arm

movements to visual targets in three- dimensional space. II. Coding of the direction of movement by a neuronal population. The Journal of Neuroscience, 8(8), 2928–2937.

Hamer, H. M., Morris, H. H., Mascha, E. J., Karafa, M. T., Bingaman, W. E., Bej, M. D., Burgess, R. C., e.a. (2002). Complications of invasive video-EEG monitoring with subdural grid electrodes.

Neurology, 58(1), 97–103.

Hochberg, L.R., & Donoghue, J. P. (2006). Sensors for brain-computer interfaces. Engineering in

Medicine and Biology Magazine, IEEE, 25(5), 32–38. doi:10.1109/MEMB.2006.1705745

Hochberg, Leigh R., Bacher, D., Jarosiewicz, B., Masse, N. Y., Simeral, J. D., Vogel, J., Haddadin, S., e.a. (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 485(7398), 372–375. doi:10.1038/nature11076

Hochberg, Leigh R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., e.a. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia.

Nature, 442(7099), 164–171. doi:10.1038/nature04970

Humphrey, D. R., Schmidt, E. M., & Thompson, W. D. (1970). Predicting Measures of Motor Performance from Multiple Cortical Spike Trains. Science, 170(3959), 758–762.

J D Simeral and S-P Kim and M J Black and J P Donoghue and L R Hochberg. (2011). Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an

intracortical microelectrode array. Journal of Neural Engineering, 8(2), 025027.

Jones, K., Campbell, P., & Normann, R. (1992). A glass/silicon composite intracortical electrode array. Annals of Biomedical Engineering, 20(4), 423–437. doi:10.1007/BF02368134 Jun Zhuang, Truccolo, W., Vargas-Irwin, C., & Donoghue, J. P. (2010). Decoding 3-D Reach and

Grasp Kinematics From High-Frequency Local Field Potentials in Primate Primary Motor Cortex. Biomedical Engineering, IEEE Transactions on, 57(7), 1774–1784.

doi:10.1109/TBME.2010.2047015

Kreiman, G., Fried, I., & Koch, C. (2002). Single-neuron correlates of subjective vision in the human medial temporal lobe. Proceedings of the National Academy of Sciences, 99(12), 8378–8383. Lebedev, M. A., Carmena, J. M., O’Doherty, J. E., Zacksenhouse, M., Henriquez, C. S., Principe, J.

C., & Nicolelis, M. A. L. (2005). Cortical Ensemble Adaptation to Represent Velocity of an Artificial Actuator Controlled by a Brain-Machine Interface. The Journal of Neuroscience,

25(19), 4681–4693.

Lebedev, M. A., Tate, A. J., Hanson, T. L., Li, Z., O’Doherty, J. E., Winans, J. A., Ifft, P. J., e.a. (2011). Future developments in brain-machine interface research. Clinics, 66, 25 –32.

Lee, I., Yoganarasimha, D., Rao, G., & Knierim, J. J. (2004). Comparison of population coherence of place cells in hippocampal subfields CA1 and CA3. Nature, 430(6998), 456–459.

doi:10.1038/nature02739

Maynard, E. M., Hatsopoulos, N. G., Ojakangas, C. L., Acuna, B. D., Sanes, J. N., Normann, R. A., & Donoghue, J. P. (1999). Neuronal Interactions Improve Cortical Population Coding of

Movement Direction. The Journal of Neuroscience, 19(18), 8083–8093.

Nakasatp, N., Levesque, M. F., Barth, D. S., Baumgartner, C., Rogers, R. L., & Sutherling, W. W. (1994). Comparisons of MEG, EEG, and ECoG source localization in neocortical partial epilepsy in humans. Electroencephalography and Clinical Neurophysiology, 91(3), 171–178. doi:10.1016/0013-4694(94)90067-1

Neuper, C., Wörtz, M., & Pfurtscheller, G. (2006). ERD/ERS patterns reflecting sensorimotor activation and deactivation. Event-Related Dynamics of Brain Oscillations (Vol. Volume 159, pp. 211–222). Elsevier. Verkregen van

Nicolelis, M. A. L., Dimitrov, D., Carmena, J. M., Crist, R., Lehew, G., Kralik, J. D., & Wise, S. P. (2003). Chronic, multisite, multielectrode recordings in macaque monkeys. Proceedings of the

National Academy of Sciences, 100(19), 11041–11046.

Nordhausen, C. T., Rousche, P. J., & Normann, R. A. (1994). Optimizing recording capabilities of the Utah Intracortical Electrode Array. Brain Research, 637(1–2), 27–36. doi:10.1016/0006-8993(94)91213-0

O’Doherty Joseph E, Lebedev Mikhail, Hanson Timothy L, Fitzsimmons Nathan, & Nicolelis Miguel A.L. (2009). A brain-machine interface instructed by direct intracortical microstimulation.

Frontiers in Integrative Neuroscience, 3. doi:10.3389/neuro.07.020.2009

Patil, P. G., & Turner, D. A. (2008). The Development of Brain-Machine Interface Neuroprosthetic Devices. Neurotherapeutics, 5(1), 137–146. doi:10.1016/j.nurt.2007.11.002

Pohlmeyer, E. A., Oby, E. R., Perreault, E. J., Solla, S. A., Kilgore, K. L., Kirsch, R. F., & Miller, L. E. (2009). Toward the Restoration of Hand Use to a Paralyzed Monkey: Brain-Controlled Functional Electrical Stimulation of Forearm Muscles. PLoS ONE, 4(6), e5924.

doi:10.1371/journal.pone.0005924

Recanzone, G. H., Guard, D. C., Phan, M. L., & Su, T.-I. K. (2000). Correlation Between the Activity of Single Auditory Cortical Neurons and Sound-Localization Behavior in the Macaque

Monkey. Journal of Neurophysiology, 83(5), 2723–2739.

Scherer, R., Muller, G. R., Neuper, C., Graimann, B., & Pfurtscheller, G. (2004). An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate. Biomedical

Engineering, IEEE Transactions on, 51(6), 979–984. doi:10.1109/TBME.2004.827062

Schmidt, E. (1980). Single neuron recording from motor cortex as a possible source of signals for control of external devices. Annals of Biomedical Engineering, 8(4), 339–349.

doi:10.1007/BF02363437

Schwartz, A. B., Cui, X. T., Weber, D. J., & Moran, D. W. (2006). Brain-Controlled Interfaces: Movement Restoration with Neural Prosthetics. Neuron, 52(1), 205–220.

doi:10.1016/j.neuron.2006.09.019

Suner, S., Fellows, M. R., Vargas-Irwin, C., Nakata, G. K., & Donoghue, J. P. (2005). Reliability of signals from a chronically implanted, silicon-based electrode array in non-human primate primary motor cortex. Neural Systems and Rehabilitation Engineering, IEEE Transactions on,

Sung-Phil Kim and John D Simeral and Leigh R Hochberg and John P Donoghue and Michael J Black. (2008). Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. Journal of Neural Engineering, 5(4), 455.

Vansteensel, M. J., Hermes, D., Aarnoutse, E. J., Bleichner, M. G., Schalk, G., van Rijen, P. C., Leijten, F. S. S., e.a. (2010). Brain–computer interfacing based on cognitive control. Annals of

Neurology, 67(6), 809–816. doi:10.1002/ana.21985

Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., & Schwartz, A. B. (2008). Cortical control of a prosthetic arm for self-feeding. Nature, 453(7198), 1098–1101. doi:10.1038/nature06996