Software for automated quantification of flexible
behaviour
Erik de Keijzer
Supervisor: Francesco Battaglia fpbattaglia@gmail.com Co-assessor: Umberto Olcese U.Olcese@uva.nl Donders Institute Nijmegen
January 2014 – June 2014
Abstract
The medial prefrontal cortex (mPFC) plays an important role in social decision making and is implicated in multiple poorly understood mental disorders like schizophrenia and autism. Deficits in local synchrony within the mPFC as well as long-range connectivity between mPFC and hippocampus have been implicated in schizophrenia, whereas prefrontal expression of nicotinic receptors is critical in an animal model of autism. A better understanding of prefrontal functioning during social interaction could provide insight into these debilitating diseases characterized by abnormal or impaired social behaviour.
The prelimbic area of the mPFC is thought to be involved in decision making and switching between rules, strategies or attentional sets, which are considered important processes during social interaction. After social deprivation, mice are motivated to interact with a novel conspecific. In a novel environment, this motivation will compete with the motivation to explore the environment. The interplay between competing motivations can evoke flexible behaviour which would likely be represented in the activity in the prelimbic cortex.
During this project, a beginning was made to construct a set-up for the simultaneous recording of camera images and electrophysiology data in freely moving mice. Microdrives for implantation were constructed to record electrophysiology data according to an open-source approach. During the course of the internship, preliminary electrophysiological data was collected in a befriended laboratory using these microdrives. As this data has not yet been analysed, this report will focus on the analysis of behaviour.
To analyse behaviour a top-view camera recorded two mice. These recordings could subsequently be analysed using semi-automated tracking software. Flexible behaviour was evoked using a social interaction task. The recording and automated analysis of behaviour, also known as computational ethology, can provide a powerful tool for deciphering different behavioural states and transitions between them. Especially in combination with the simultaneous recording of electrophysiology data this could provide a wealth of information about activity in the mPFC during behavioural epochs. To investigate prefrontal functioning during social interaction it will be necessary to record and analyse flexible social behaviour automatically. Can we record flexible behaviour and analyse this behaviour automatically?
Introduction
Irregularities in the functioning of the medial prefrontal cortex (mPFC) during social interaction are associated with aberrant social behaviour in mental disorders like schizophrenia 1 and autism 2. The prelimbic- and infralimbic area are considered the rodent homologue of the mPFC in humans. Deficits in local synchrony within the mPFC as well as long-range connectivity between mPFC and hippocampus have been implicated in schizophrenia 1, whereas prefrontal expression of nicotinic receptors is critical in an animal model of autism 2. A better understanding of prefrontal functioning during social interaction could provide insight into these debilitating diseases characterized by abnormal or impaired social behaviour.
The prefrontal cortex and flexible behaviour
Different subregions of the prefrontal cortex (PFC) are thought to support different aspects of flexible behaviour. The prelimbic cortex is part of the ventromedial prefrontal cortex (vmPFC), against the midline of the brain. More ventral parts of the PFC (the orbitofrontal cortex) are implicated in reversal learning, impulsive choices and perseverative tendencies. The anterior cingulate cortex (ACC), adjacent to the prelimbic area along the midline, is associated with emotional control, stimulus-reward contingencies and response conflict 3. The prelimbic area (PL) of the vmPFC is thought to facilitate switching between different strategies and rules, or shifts in attention 3. Lesions of the PL result in perseverative errors in tasks designed to test behavioural flexibility, which is considered the result of an inability to shift to a new rule or strategy.
Removal, inactivation or
disconnection of several subcortical brain areas would contribute to the emergence of errors in tests of behavioural flexibility. Lesions of several subcortical input regions to the mPFC lead to an inability to shift to a new rule or strategy. Disconnection of mPFC-NAc-thalamus circuitry results in increased perseverative and regressive errors 3. Inactivation of the NAc core does not affect initial learning of strategies, but where mPFC lesions generate perseverative errors, NAc core or dorsomedial striatum lesions increase regressive errors which are considered the result of degraded maintenance of a novel rule. NAc core lesions additionally induce ‘never-reinforced errors’ which are indicative of a deterioration in the elimination of inappropriate response options 3. Disconnection of the cortex and medial thalamus likewise induces perseverative errors.
3
Figure 1: The vmPFC connects to a multitude of cortical and subcortical areas. Among these are the amygdala, hippocampus, dense connections to other fronto-cortical areas and an implication in a striato-thalamo-cortical loop. Furthermore, the activity in the vmPFC is influenced by several neuromodulators 11.
Thus, the vmPFC is involved in a network with various subcortical areas including the striatum and medial thalamus, which are engaged in the generation of flexible behaviour (figure 1). The vmPFC is amongst others implicated in a striato-thalamo-cortical loop that involves ventral and dorsomedial parts of the striatum and medial parts of the thalamus. This cortical-basal ganglia connection is likely involved in the motor-execution of actions. The PL projects to the core region of the nucleus accumbens. The adjacent infralimbic cortex projects to the NAc shell, which seems to have an opposite function to the NAc core 4.
Lesions of the NAc shell region before initial discrimination learning seem to improve set shifting performance 3 this could be due to competition over control between different striato-thalamo-cortical loops 5 or NAc shell may mediate ‘learning to ignore’ with a consequent failure to learn the irrelevance of stimuli 3. The PFC itself is hypothesized to follow a dorsal-to-ventral gradient in its inputs and outputs (figure 2). Dorsal areas of the PFC including ACC and dorsal PL are thought to be involved in motor-control, whereas more ventral areas of the mPFC govern emotional and autonomic responses 6. Medial-to-lateral in the OFC a gradient is perceived from more autonomic and visceral motor inputs to more sensory input.
Another important subcortical connection of the vmPFC is with the amygdala. Different subregions of PFC receive projections from different nuclei of the amygdalar complex 7. The amygdala acts as a subcortical hub in circuits driving valence-guided or fear-related behaviour 8. The vmPFC shares dense connections with the basolateral amygdala (BLA). Devaluation training has revealed that the basolateral- and central amygdala play distinct roles in the determination of valence. The BLA contains valence-sensitive neuronal populations that encode the positive or negative value of an outcome regardless of the sensory features of the predictive cue 9,10. The central amygdala (CeA) on the other hand, likely maintains and signals the salience or motivational significance of an outcome via its projections to midbrain dopamine neurons 11.
Medial prefrontal functioning is influenced by several neuromodulators. Dopamine in the mPFC might serve to stabilize representations (rules/ attentional sets) in set-shifting tasks 3. Dopamine receptor blockade (D1 or D2) disrupts ability to shift between strategies. Dopamine receptors are not necessary for rule acquisition 3. When flexible behaviour is evoked through social interaction acetylcholinergic and noradrenergic neurotransmission has been shown to be necessary for flexible social behaviour. Global levels of noradrenaline (NA) influence behavioural flexibility in this task and depletion of NA specifically in the PL leads to an impoverished social repertoire and more rigid social behaviour 12.
4
Figure 2: Connections to the PFC are thought to be organised along a dorsal-to-ventral and medial-to lateral gradient. The dorsal and ventral PL are likely involved in
more motor- and limbic- related executive functions, respectively 11. ACd, dorsal anterior cingulate cortex; PL,
prelimbic area; IL, infralimbic area; DP, dorsal peduncular cortex; VO, ventral orbital-; LO, lateral orbital cortex.
Thus, different subregions of the PFC are involved in different forms of flexible behaviour. The prelimbic area of the vmPFC is known to be necessary for the flexible switching between rules, strategies and attentional sets. Activity in the PL is likely to be an integration of many cortical and subcortical input regions including information about visceral and emotional states 6 and possibly simulations of possible outcomes 13. Recording activity in the PL during flexible behaviour could elucidate the role of this area during flexible decision making. This would require the positioning of electrodes in the PL the gather electrophysiological data.
Electrophysiology
To examine the activity in the PL during flexible behaviour, extracellular recordings of activity in this area would be desired. Local field potentials (LFP’s) could reveal emerging local networks during the execution of flexible behaviour 14, for instance during different behavioural epochs. Large-scale electrophysiology will be required to register the activity of these neuronal ensembles 15. To investigate the correlation between brain activity and behaviour, behaviour can be recorded and a large number of tetrodes in the prelimbic area of freely moving animals can provide electrical measurements. The goal of this project is to record activity in the in the prelimbic area during flexible behaviour and couple this with automated behavioural scoring.
Both tetrodes and silicon probes can be used to implant a high number of recording electrodes in a certain brain area; both have their advantages and disadvantages. Tetrodes as well as silicon probes allow for the sorting of spikes according to the position of the firing neuron relative to the tip of the recording device. Tetrodes only record from the tips of the individual wire electrodes, whereas silicon probes feature electrodes along the entire probe shaft. Due to their rigidity, silicon probes can give a better estimation of the position of electrodes after implantation. Finally, several silicon probes can be arranged in ‘combs’ and three-dimensional arrays, further improving the accuracy of the recording site estimation.
Recent advances in silicon probe design have further brought the integration of functional circuitry in the probe base and shaft for the amplification and multiplexing of neuronal signals 16. Pre-amplification close to the source reduces noise and the multiplexing of signals reduces the number of required output lines, facilitating a small design of connectors and headstages, which reduces discomfort for the animal. Further integration of low energy-consuming CMOS circuitry in silicon probes enables selection of different recording sites, by switching between different electrodes along the probe shaft 16. This allows for repeated measurements at various depths, for instance in different cortical layers.
Recent advances in hardware for tetrode recordings on the other hand have nurtured a movement towards open-source physiology with an associated order of magnitude drop in costs. The introduction of Intan chips in 2007 that could take care of the above described pre-amplification and multiplexing tasks on a small headstage, lead to the development of the Open-Ephys system. The development of an FPGA- based acquisition board, along with the earlier development of a 3D-printed microdrive 17 generated an entirely open-source data acquisition system for tetrode recordings. Tetrodes can be inserted at different depths and are known to travel straight through brain tissue once implanted 17. Thus, with appropriate depth tracking the position of tetrodes can be reliably estimated. At present, electrophysiology set-ups using silicon probes are in general more
Th e fl ex d riv e N eu ra l e n se m b le s c an b e m ea su re d u sin g a 3 D -p rin te d , s p rin g l o ad ed m ic ro d riv e w ith a c ap ac ity to h o ld u p to 1 6 t et ro d es (re f fl ex d riv e) . T h is cu st o m -b u ild fl ex ib le d riv e c o n sis ts o f a 3 D -p rin te d b as e w ith a so ld er ed sp rin g h o ld in g 1 6 sc re w s. A d riv e b u n d le o f fl ex ib le p o ly im id e t u b in g p ro tr u d es fr o m th e b as e a n d is a tt ac h ed to th e s ep ar at e s p rin g a rm s, w h ic h fa cil ita te th e l o w er in g o f in d iv id u al te tr o d es . T h e d riv e b u n d le o f g lu ed p o ly im id e t u b in g i s a d ap ti ve ly sh ap ed to re ac h th e t ar ge te d b ra in re gio n s. A n ele ct ro d e i n te rfa ce b o ar d c o n n ec ts th e m ic ro d riv e t o a d at a a cq u isi ti o n sy st em to c o lle ct u p to 6 4 c h an n els o f n eu ra l d at a s im u lta n eo u sly (r ef fl ex d riv e) . The flexdrive
Neural ensembles can be measured using a 3D-printed, spring loaded microdrive with a capacity to hold up to 16 tetrodes 16. This custom-build flexible drive consists of a 3D-printed base with a soldered spring holding 16 screws. A drive bundle of flexible polyimide tubing protrudes from the base and is attached to the separate spring arms, which facilitate the lowering of individual tetrodes. The drive bundle of glued polyimide tubing is adaptively shaped to reach the targeted brain regions. An electrode interface board connects the microdrive to a data acquisition system to collect up to 64 channels of neural data simultaneously 17.
expensive, especially when compared to an open-source system. This effect is however partly counterbalanced by an increased fabrication time in open-source electrophysiology.
In the current investigation, it was decided to gather the electrophysiological data using an open-source approach. Microdrives for the implantation of tetrodes were 3D-printed and assembled in the lab in Nijmegen. The computer-aided design (CAD) files for the microdrives are readily available 17 and the entire acquisition system can be assembled for a fraction of the costs of conventional data acquisition systems 18. 3D-printed parts can easily be modified to suit the needs of a specific project, which increases flexibility. All components of the acquisition system (microdrive, tethers, casing of the acquisition board) could be constructed in Nijmegen except the headstages for pre-amplification and the acquisition board for connection to a computer.
Electrical signals that have been measured in the above-described way have to be paired with behaviour to investigate flexible behaviour in freely moving animals. The pairing between electrical signals and behavioural data has to be exact, as fluctuations of electrical activity occur on a millisecond timescale. A top-view camera in our set up provides recordings for the scoring of behavioural elements. Together, camera images and electrical data could reveal what happens in the PL during flexible decision making. As the electrophysiology data has not yet been analysed, this report will focus on the analysis of the behavioural data from the video recordings.
Automated analysis of behaviour
Flexible behaviour can be examined using various reversal learning-, inhibitory learning- and set shifting tests or a Social Interaction Task (SIT) 3,19. In a social interaction task, two mice are allowed to interact in a novel environment. During the 8min test, the motivation to interact with the novel 6
conspecific competes with the motivation to explore the arena. One of the mice is deprived of social contact for at least four weeks before testing (figure 3). Thus, the possibility for social interaction is considered a rewarding motivation. This resident-mouse is allowed to explore the testing arena for 30min before the introduction of an intruder. Through manipulation of the size of the arena and habituation of the subject to this environment anxiety levels can be controlled and kept low 3. As the test is not conducted in one mouse’s home-cage and with a conspecific of the same age and the same sex, non-aggressive and non-sexual social encounters are observed, as opposed to when this paradigm is used to evoke a forced confrontation or to study male/female sexual behaviour. As the ‘intruder’ is a novel conspecific and each resident and intruder are only paired once, no memory component is involved.
Neuromodulators influence behaviour during this resident-intruder paradigm or SIT. Behaviour usually evolves during the 8min SIT, in which the last 4 minutes can include i.a. more cooperative or more aggressive behaviour than the first 4 minutes. NA depletion in the PL results in less development of behaviour over time. Acetylcholinergic nicotinic receptors in the PL are involved in this process as mice lacking the β2 subunit of nicotinic acetylcholine receptors (β2-/- KO’s) actually show increased behavioural flexibility after NA depletion. Levels of dopamine in the PL increase after NA depletion, which is sufficient to induce a rigid behavioural phenotype similar to β2-/- autism model mice in healthy controls (C57BL/6) 12.
Based on previous research that has indicated that the prelimbic area of the mPFC is necessary for switching between rules, strategies or attentional sets during flexible behaviour, it is the purpose of this investigation to examine the generation of flexible behaviour and the role of the PL herein. The SIT can be used to evoke flexible behaviour as, on one hand, there is a competition between the motivation to engage in social interaction and the motivation to explore the enclosure, and on the other hand because the subject cannot predict the (re)actions of its unfamiliar conspecific. This creates a highly uncertain environment. The conflict between competing motivations in a highly uncertain environment affects decision-making and requires the emergence of flexible behaviours 3. In this highly uncertain environment in which two motivations compete, sequences of actions, like approaches, escapes and contact can be recorded and (automatically) analysed and categorized. In this way, the SIT can capture the causal relationship between actions and the underlying behavioural state 3.
Manual scoring of behaviour is generally a slow and labour-intensive process. Additionally, there is room for a certain degree of subjectivity, as two experimenters can score behavioural elements or duration of behavioural epochs differently. Automated analysis of behaviour can be more accurate, with the exact same criteria applied to every recording and more detailed- and a higher number of defined behavioural elements20. Furthermore, more accuracy can be achieved regarding the moment and time period of the occurrence of behavioural elements. A computer can score more pre-defined behavioural elements simultaneously, and label sequences of behavioural elements, or might be able to discern elements that would go unnoticed by a human experimenter20. Especially accurate determination of the exact moment of occurrence and duration of behaviour will be important for a good match between behavioural- and electrophysiology data, which raises the research question: “Can we record behaviour automatically?“
7
Introduction isolated host mouse to maze
Figure 3: The social interaction task. Two mice were allowed to freely interact in an arena adapted to represent a novel
environment. One of the mice, the isolated host mouse (IH), was deprived of social contact for at least four weeks before beginning of the experiment and allowed to habituate to the arena for 30min before the test phase. The IH and an intruder raised in a social environment (SV) were allowed to interact for 8min, during which motivations for social interaction and exploration compete. Adapted from 21.
MiceProfiler software has been used previously to analyse behaviour during a social interaction task 20. To determine the reliability of this program, a comparison with manual scoring was made for a subset of behavioural elements that is suitable for both manual and automated analysis.
These behavioural elements include oral-oral contact, a side-by side position in the opposite direction, follow behaviour and a back to back position (figure 4). The emergence of back-to-back positions would be of special interest as it is considered indicative of cooperation between the mice as this is a risk-prone posture in which the mice are not able to see each other 20. Manual scoring by a human experimenter was set as a golden standard and a deviation in the number of scored elements per selected behavioural element of 10% was set as an arbitrary limit that was deemed to be considered acceptable. As the MiceProfiler software is semi-automated and requires manual correction to function properly, the question is raised whether semi-automated scoring with Miceprofiler software is reliable enough?
Ultimately, a verdict about the suitability of the software for registration of flexible behaviour would want to be made. Is the MiceProfiler software suitable for recording flexible behaviour?
Methods
Animals
C57BL/6J mice (Charles River, L’Arbresle Cedex, France) (age: 8 weeks) were social housed in groups of four for at least three weeks before separation into single housed isolated host mice (IH) or social housed visitor mice (SV). Mice, all males between 24-32g, were kept at a reversed day/night cycle. After three weeks, mice in every other group were separated and single housed according to their role as isolated host mouse (IH). Experiments were approved by the Animal Ethics committee of the Radboud University Nijmegen (DEC: 2013-177) and were conducted in agreement with Dutch laws (Wet op de dierproeven, 1996) and European regulations (Guideline 86/609/EEC).
8
Figure 4: Analysed behavioural elements. Behavioural elements were defined for the MiceProfiler software (see
Stereotaxic surgery
Mice (n=3) were anaesthetized with ketamine and xylazine before implantation of an Open Ephys custom built microdrive. A craniotomy (Bregma: anterior-posterior: 1,0; medio-lateral: +/- 0,8 mm) and (Bregma: anterio-posterior: 2,8-0.8; medio-lateral: +/- 0,6 mm) and (Bregma: anterio-posterior: 3,1-0,5; medio-lateral: +/- 0,8 mm) was performed. Implantation of tetrodes began at coordinates dorsoventral 0,5 mm. After surgery, mice received antibiotics and analgesia was provided for a maximum of one week. Animals were allowed to recover for a total of 7 days before start of any in vivo experiments.
One mouse (n=1) was anaesthetized with isoflurane (1,5-3%) and received buprenorphine analgesia (0,05-0,1 mg/kg s.c.) for implantation of an Open Ephys custom built microdrive. A bilateral craniotomy (Bregma: antero-posterior: 3,1-1,1; mediolateral: +/- 0,5mm) allowed for placement of the two guide-tubes under a 12° angle. Implantation of tetrodes began at coordinates: dorsoventral 0,5mm. After surgery, mice received Enrofloxacin antibiotics (5 mg/kg s.c.); Carprofen analgesia (4-5 mg/kg s.c.) was provided for a maximum of one week. Animals were allowed to recover for a minimum of 7 days before start of any in vivo experiments.
Recording apparatus
Mice were recorded during a social interaction task (SIT) as previously described 2,3,12,20,21. The social paradigm consisted of an isolated host mouse (IH) placed in an arena for interaction with a conspecific. The IH mouse was allowed to roam the arena for 30 minutes before introduction of a novel social visitor mouse (SV). After this habituation period, a SV mouse of the same weight, age, and sex was introduced to the testing arena (figure 3).
Arenas had three shapes: round, square and rectangular. The floor area of every arena was approximately 1,3 m2. The round arena consisted of a 20cm high wall adorned with visual cues and 20cm in diameter. The square and rectangular arenas likewise had 20cm high walls of opaque plastic sheeting. Bedding consisted of either matting or sawdust and alternated between trials. Visual and olfactory cues varied between tests. Thus, the interaction arena provided a novel environment for the animals every trial, to promote equal appraisal of competing motivations.
During interaction mice were filmed using a high-resolution camera recording a top view image of social interaction. The social interaction test-session lasted 8 minutes. Thus recorded behavioral data was analyzed manually as well as using MiceProfiler software 20. MiceProfiler was designed to track two mice simultaneously by detection of the boundary between mouse and background and subsequent superposition of a model including head, body and base of the tail over the mouse. The position of the physics model is tracked within a region of interest (image on next page). Position, direction and speed of movement of the model are registered and used to distinguish several modes of interaction (Supplementary figure 1). These interactions can subsequently be combined into second and third order modes of interaction. For instance, a sequence of events like: the isolated host mouse moves towards the social visitor mouse and the social visitor mouse escapes, can automatically be detected. Identified modes of interaction can be plotted in behavioral graphs (Supplementary figure 3). Thus, the transitions between modes of interaction can be visualized and common patterns of switching between different modes of interaction identified.
Additionally, internally developed Rat-tracker software was tested as an alternative for MiceProfiler. Automated analysis of behaviour using this software involves the superposition of a model on the image like the Miceprofiler software, or in an alternative approach a superimposed backbone demarks the position of the mice and a prediction of their position in the subsequent frame is made. Manual and automated scoring of the images captured by the top-view camera was compared to assess the reliability of automated quantification of behaviour.
Behavioural quantification
Video recordings were cut, to isolate the period containing social interaction. The first 8min of these clips were subsequently analysed manually or using the automated behavioural scoring software and scoring of a selection of behavioural elements and duration of social interaction was compared. Behaviour recorded with the top-view camera was scored manually for the ‘oral-oral contact’, ‘side by side (opposite)’, ‘IH follows’ and ‘back to back position’ behavioural elements (figure 4). These four elements are clearly recognizable for a human experimenter and suitable for both manual and automated scoring. Videos were played back at half speed and paused after the occurrence of a behavioural element (for notation) to ensure accuracy. Contact duration was timed in a similar fashion. Timing started when the mice were in less than 1cm proximity of each other and videos were replayed at half speed to improve the accuracy of timing.
With MiceProfiler, movement of the mice was tracked within a specified region of interest (ROI; left picture, in green). Different previously defined behavioural elements could subsequently be calculated from the positions of a superimposed model on the mice. For instance a back-to-back posture (right picture) was detected when mice where further than 3cm apart and, according to calculated fields of view, not able to see each other. The software was allowed to continue as much as possible, with minimal intervention from the experimenter. Only when the program would not be able to recuperate and reposition the models in the correct configuration would the program be corrected.
The four selected behavioural elements could be extracted from the software using the ‘MiceProfiler video label maker’ function (figure 5). The total count of the occurrence of a behavioural element was noted from this function. Likewise, the total time spent within less than 1cm proximity of the conspecific could be acquired in this way.
Verification
For further analysis an option is included in the software to verify the automated analysis, which can be used to identify false positives (figure 5). An image of the frame captured when a behavioural element was recorded is shown.
As an alternative, Rat-tracker software was tested to track the mice. Recorded behavioural data was analyzed using a specially developed Python-programmed analysis tool. The tool calculates a static background from the individual frames and subsequently detects boundaries between the animals and background per frame. A model is superimposed and coordinates calculated and collected. As another option, head and tail of the animal are indicated (a ‘backbone’), whereafter the tool predicts possible ensuing positions. This approach utilizing movement prediction was thought to be able to provide a more accurate tracking of the animal.
11
Figure 5: The MiceProfiler ‘video label maker’ screen. The scale of the model chosen during tracking can be inserted on
the right, after which the occurrence of a number of behavioural elements is calculated as well as the duration of time spent within certain proximity. Counts of the selected behavioural elements as well as contact duration were obtained from this screen. The frame image can be used for verification.
With Rat-tracker, movement of the mice was tracked against a background calculated from the averaged video frames (middle image). Either a model was superimposed (left image) or a line demarked head and tail of the recorded mouse (right image). The latter approach included the prediction of possible positions of the head, body and base of the tail in the subsequent frame, to improve the accuracy of tracking.
Results
Behavioural quantification
A comparison between manual and semi-automated scoring of the four selected behavioural elements revealed much divergence. Manual and semi-automated scoring would match in 3 occasions out of 4 behavioural elements scored in 10 movies. In 7 occasions out of these 40 instances would the number of scored elements by the software lie within the prespecified range. A deviation from the number of behavioural elements obtained through manual scoring of 10% was deemed acceptable.
Figure 6: Manual and automated scoring of four behavioural elements. The count for the occurrence of each element is
shown, as well as curly brackets indicating a deviation of 10% from manual scoring of the specified behaviour.
Figure 7: Manual and automated scoring of four behavioural elements. Curly brackets indicate a
deviation of 10% from manual scoring of the specified behavioural element. Note the difference in the Y axes, as the occurrence of
behavioural elements could vary between videos. Legend in figure 6.
Results of the scoring of oral-oral contact, side by side (opposite) and back to back positions or follow behaviour by the IH are depicted in figure 6 and figure 7. Curly brackets indicate a 10% deviation from the manual scoring of a behavioural element. These results were obtained with minimal intervention in the operation of the MiceProfiler software. However, some correction of the semi-automated analysis was necessary, in instances where the program failed to restore the correct positions of the superimposed models. This would occur primarily when the models would switch to the wrong animal or when the model would rotate into an incorrect position (head-tail orientation) and there would be no prospect of recovering the correct position (e.g. the mouse would be outstretched). The depicted results are not subjected to the secondary verification step.
Figure 9: Interaction time counted by semi-automated (red) or manual scoring (blue). Percentage of difference
between automated and manual scoring on the total time in green, lower means more correspondence.
Figure 8: Differences between analyses of an exemplary behavioural recording. The left group of bars in every graph
depicts the result of manual scoring. The middle graph represents the difference between manual and automated scoring after heavy correction of automated analysis. The right graph shows the difference after correction and verification of automated scoring.
Further analysis
The best match between manual and semi-automated scoring of these four behavioural elements occurred in video D, with an exact match for half the scored behavioural elements. Video D was subjected to further analysis consisting of a thorough manual correction of the positioning of the superimposed model and a second manual verification step of automatically scored behavioural elements. Thorough correction of the semi-automated scoring resulted in a match within the prespecified range for three of the four behavioural elements, but after elimination of incorrectly scored behavioural elements only one match between manual and automated scoring remained, as depicted in figure 8. As the correspondence between behavioural elements scored automatically and the occurrence of these behavioural elements could not be guaranteed even after elimination of incorrectly scored elements, some but not all videos were subjected to this second step in analysis (see discussion).
Interaction time
The duration of contact within the first 8min of a behavioural recording was timed by the MiceProfiler software and manually. A distinction can be noted in which the automated timing of contact duration is consistently higher than manual timing of interaction (figure 9). The difference in manual and automated timing can be calculated as a percentage of the total time measured. This percentage is depicted as the green line in figure 9 and varies between 19% and 35% with an average of 24,5% and more time is counted with semi-automated scoring.
Figure 9: Interaction time counted by semi-automated (red) or manual scoring (blue). Percentage of difference
between automated and manual scoring on the total time in green, lower means more correspondence.
The Rat-tracker tool is still under development and has hitherto not been capable of generating output.
Discussion
The goal of this project was to record activity in the prelimbic cortex during flexible behaviour and couple the electrophysiology data with automated behavioural scoring. To achieve this, software for the automated analysis of flexible behaviour is required. Flexible behaviour was evoked using a social interaction task (SIT). For successful coupling of recorded behaviour and electrophysiology data, it is vital that analysed video footage can be combined with electrical activity data on a sub-second time-scale. The MiceProfiler software was developed for (semi-) automated analysis of behaviour in the SIT. This report discusses the use of MiceProfiler software for the automated analysis of behavioural recordings and its applicability for registering flexible behaviour.
Three microdrives were successfully constructed and implanted in animals for the acquisition of electrophysiological data. Flexible behaviour was evoked using a social interaction task (SIT) which evoked a conflict between two competing motivations (exploration and social interaction). These motivations can be manipulated by the experimenter; the isolated host (IH) mouse was deprived of social contact for at least three weeks before testing and the IH was allowed to explore the testing arena for 30min before a social visitor (SV) was introduced. These manipulations ensured social interaction was perceived as a rewarding motivation by the IH and reduced anxiety levels in the IH during the test, respectively. The uncertainty of the (re)actions of the unfamiliar conspecific in addition help to evoke flexible behaviour.
Is the software reliable enough?
The comparison between manual and semi-automated scoring of the four selected behavioural elements revealed much divergence. The match between manual and automated scoring was expected to approximate 100% as described previously 20. An arbitrary limit of 10% deviation from manual scoring was set and considered an acceptable margin of error for automated scoring of behaviour. In only 7 occasions out of 40 instances of scored behavioural elements did the software 15
remain within this limit. An additional 6 instances can be added in which the count of behavioural elements only differed 1 scored behavioural element, as a deviation of 10% with a total count of that behavioural element below 10 does not amount to a full extra enumeration more or less. Even then, the MiceProfiler software only achieved a hit-rate of 13 out of 40 instances (32,5%). Extensive observation of the software preceded manual scoring of behaviour as to ensure a close match between attributions of behavioural elements.
Several videos were subjected to further analysis to explore the possibilities for improvement of the semi-automated behavioural scoring. The MiceProfiler software includes the option to improve the positioning of the superimposed model manually. Several videos were corrected frame-by-frame using this option. This most thorough analysis ensures the position of the superimposed model is correct in every frame of the 8min behavioural recording. As an example, video D was corrected frame by frame as well subject to a second verification step in analysis. Video D was chosen as an example as it had one of the highest levels of correspondence between scoring methods and initial tracking with the MiceProfiler software went relatively smooth and fast (2h). Although correction of the automated analysis resulted in a match within the prespecified range for three of the four behavioural elements, upon closer inspection after elimination of falsely assigned scored behavioural elements, only one match remained. Follow behaviour, clearly visible in the visual recording was scored on no account. Moreover, the heavy manual correction of the superimposed models could introduce frames with an identical position of the models, which would later hinder the accurate scoring of stop behaviour. Flexible decision making, reflected in the switching between competing motivations, could possibly be represented in decision points. These would likely occur before the mouse engages in one of the competing motivations (like the competing directions in a T-maze). If this is the case, the electrophysiological analysis should detect this decision point, which can then be matched with videoframes/ behavioural recordings. There are indications stop-behaviour could reflect a decision point 20. In this case, frame-by-frame correction of the superimposed model will introduce additional identical frames, which would introduce incorrectly scored stop-behaviour. Because thorough manual correction of the MiceProfiler software could not ensure that all occurrences of a behavioural element were correctly scored and would interfere with subsequent analysis as well as void the advantage over slow and labour-intensive manual scoring, not all videos were analysed this way.
Thus, the MiceProfiler software was not able to reliably detect the occurrence of behavioural elements which could be attributable to four factors, concerning the resolution of the used cameras, the rigidity and scale of the models and the method of interaction detection.
First, a higher resolution of the cameras in our set-up (as compared to those used in 20) could result in a more gradual colour gradient between adjacent pixels and consequent difficulties with edge-detection. Alternatively, the camera fixation height influences the image and the size of the mice in the recorded image. Secondly, the superimposed model could not replicate the agility of the mice and variances in the size of their top-view. The physical model was too rigid and could not replicate the flexibility of the mice. The model constitutes a circle for the body and a circle for the head, opposite to the tail. The representation of the head can bend just a little relative to the body, but remains opposite to the tail. The mouse, whilst exploring, bends its head in every direction, while keeping its hind legs in place. Additionally, the model has a fixed scale whereas the size of a mouse’s top-view can vary significantly. The model generally fits when a mouse is running straight forward, 16
but the mouse’s top-view is in many occasions smaller than the scale of the model. For instance when a mouse is grooming or rearing against a wall, the area detected by the edge detection is too small to fit the fixed-size model, which causes the model to flip back and forth. Fourth, tracking becomes particularly difficult when the two mice are in close proximity of each other (see supplementary material). MiceProfiler counts interaction according to a predefined set of rules that include the distance between the two models. In practise, the position of the physical models often does not match with the image it is superimposed upon when the two mice touch. As soon as the two mice touch, the boundary between mouse and background could not be detected and with enough contact surface the areas associated with each mouse would merge. If the detected area of the two mice together was too small to fit both models, these would flip back, forth and sideways, hindering a realistic representation of the position of the animals.
Several of these issues could possibly be improved by incorporation of the headstage of one of the mice in the model. The rigidity and fixed scale of the models would pose a smaller problem if one of the models was anchored to the headstage of a specific animal and unable to rotate. Similarly, the model would be unable to jump even if the surface areas calculated by boundary detection of the two mice would merge. However, the model is hard-coded in the software and the code not written according to conventions therefore such modifications will be difficult.
Can we analyse behaviour automatically?
As the answer to the previous research question was negative, a reply to this question will unavoidably be a declination in the current context regarding flexible behaviour. The software has hitherto shown several insufficiencies that would first have to be addressed.
The automated analysis of behaviour is also known as computational ethology and could nonetheless provide a wealth of information about animal behaviour. Computerized automated analysis of behaviour can detect more and more complex behavioural elements and quantify behavioural elements less subjectively, more of them simultaneously and multiple sequentially, than a human experimenter. Properly working automated analysis of behaviour could possibly reveal inconspicuous behavioural elements imperceptible for the human eye and make faster and a vaster amount of behavioural analyses possible.
Mapping of the competing motivations in the SIT through composition of an ethogram in pictorial form (supplementary figure 3) could reveal flexible decision making in the switching between behavioural states. Such pictorial ethograms resemble Markov models, which can be used to model the probability that a certain action will be followed by another in randomly changing systems. Analysis of the electrophysiology data could reveal the emergence of different ‘states’ in the prefrontal cortex which might be associated with previously defined behaviours or motivations 14. As these states are only partially observable a hidden Markov model could be used to explore the different states that underlie the directly visible previously defined behaviour. Such a hidden Markov model could be used to reveal a sequence of states that underlies a certain behavioural repertoire. This would be an example of temporal pattern recognition based on Bayesian principles. However, the behavioural classifications based on the different previously defined behavioural elements and the transitions between them and the probabilities of switching from one behaviour to another could not be obtained from the MiceProfiler software. The semi-automated analysis of behaviour with MiceProfiler was not reliable enough and the alternative Rat-tracker tool was still in a too early stage of development to generate output. Moreover, the MiceProfiler software was designed to only score 17
‘social’ behavioural elements. Whereas the interest of this project involved the automated scoring of flexible behaviour which entails the switching between social and explorative activity. Using the MiceProfiler software, ‘exploration’ can only be defined as the time not spent in interaction with the conspecific. Timing of the duration of time spent in contact was shown to differ between manual and automated measurement. As the duration of contact timed by the software always added an average of 24,5% extra to the total time measured, this discrepancy can likely be attributed to a difference in the used definition. It should be noted that manual timing did not measure the time spent in interaction but the duration of contact, as defined for the MiceProfiler software (Supplementary figure 1).
Thus, as the software does not score explorative behavioural elements directly and the duration of contact was always clocked higher than when scored manually (i.e. the timing did not match) the Miceprofiler software is not suitable to score exploration. Therefore, there is to date no software available for the automated quantification of flexible behaviour in freely moving animals during a social interaction task.
In the future, software that is to be used to combine electrophysiology data and the automated analysis of flexible behaviour might profit from the inclusion of the headstage in the tracking, the incorporation of an algorithm that can track one point 22, an additional depth-sensing camera in the set-up 23 and the use of machine-learning algorithms to define behavioural elements 24. Additional depth information can be gathered using a kinect-camera and could be especially useful in situations in which the fitting of a 2D-model on a three-dimensional shape-changing mouse will not suffice. Less experimenter input via the use of algorithms that can learn to identify behavioural elements after a few instances of manual input will support the feasibility of fast automated behavioural scoring and the detection of concealed and unfamiliar elements in behaviour. Ultimately, the issues encountered during testing of the MiceProfiler software pose mere technical challenges and computational ethology will inevitably make a valuable contribution to neuroscience.
Conclusion
Can we record flexible behaviour?
During this project, a beginning was made to construct a set-up for the simultaneous recording of camera images and electrophysiology data in freely moving mice. The fabrication of microdrives yielded usable custom-build flexdrives suited for implantation in mice. These microdrives were implanted and used during behavioural testing in a social interaction task. The SIT evoked flexible behaviour which is known to depend on activity in the PL of the vmPFC. Due to insufficiencies in the software it was hitherto not possible to reliably quantify behaviour for coupling with electrical activity in the PrL. The reliability of the software was outside the prespecified range of 10% deviation from manual scoring. The software can only detect social behavioural elements. The software is not fit to accurately pinpoint transitions between social and explorative behaviour.
Ultimately, the automated analysis of behaviour, or computational ethology 25, in mice, especially in combination with genetic modifications possible in this species provides a very promising avenue for enhancement of our fundamental understanding of flexible behaviour or higher order cognitive functions in general.
Supplementary material
19
Supplementary figure 1: Predefined events including first-, second- and third-order modes of interaction. MiceProfiler
Semi-automated analysis of behaviour with MiceProfiler sometimes failed to detect boundaries between the animals when these were in close proximity of each other. For instance, the superimposed model would consequently pass from one animal to the other or flip, switching head and tail (left picture). In earlier videos, recorded closer to the animals, these results would be aggravated (right picture). These discrepancies could be rectified frame by frame.
Analysis of behaviour with the model-based approach with Rat-tracker likewise resulted in occasional switching of the model between animals. For instance, models (already stacked together) could switch from one animal to the other during a follow sequence (left picture) or the superimposed model from animal one could switch to animal two and vice versa when the animals would pass each other (right picture). With the Rat-tracker tool, these errors could not be corrected manually.
20
Supplementary figure 3: An ethogram with three behavioural classifications. The likelihood of switching from one behaviour to another is given in percentages. In this case, β2-/- KO mice display a tendency to switch to social interaction after exploration 21. The three competing motivations included interaction, exploration and the consumption of sucrose pellets in opposite corners of the arena. Both animals were food-deprived and the resident was deprived of social contact as described above.
References
1. Spellman TJ, Gordon JA. Synchrony in schizophrenia: A window into circuit-level pathophysiology.
Curr Opin Neurobiol. 2015;30:17-23.
2. Granon S, Faure P, Changeux JP. Executive and social behaviors under nicotinic receptor regulation.
Proc Natl Acad Sci U S A. 2003;100(16):9596-9601.
3. Granon S, Floresco S. Functional neuroanatomy of flexible behaviors in mice and rats.
Endophenotypes of psychiatric and neurodegenerative disorders in rodent models.Kerala: Transworld Research Network. 2009:832103.
4. Pennartz C, Ito R, Verschure P, Battaglia F, Robbins T. The hippocampal–striatal axis in learning, prediction and goal-directed behavior. Trends Neurosci. 2011;34(10):548-559.
5. Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nature Reviews Neuroscience. 2006;7(6):464-476.
6. Euston DR, Gruber AJ, McNaughton BL. The role of medial prefrontal cortex in memory and decision making. Neuron. 2012;76(6):1057-1070.
7. Peters J, Kalivas PW, Quirk GJ. Extinction circuits for fear and addiction overlap in prefrontal cortex.
Learn Mem. 2009;16(5):279-288.
8. Tye KM, Prakash R, Kim S, et al. Amygdala circuitry mediating reversible and bidirectional control of anxiety. Nature. 2011;471(7338):358-362.
9. Paton JJ, Belova MA, Morrison SE, Salzman CD. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature. 2006;439(7078):865-870.
10. Schoenbaum G, Chiba AA, Gallagher M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci. 1999;19(5):1876-1884.
11. Han JS, McMahan RW, Holland P, Gallagher M. The role of an amygdalo-nigrostriatal pathway in associative learning. J Neurosci. 1997;17(10):3913-3919.
12. Coura RS, Cressant A, Xia J, et al. Nonaggressive and adapted social cognition is controlled by the interplay between noradrenergic and nicotinic receptor mechanisms in the prefrontal cortex. FASEB J. 2013;27(11):4343-4354.
13. Pezzulo G, van der Meer, Matthijs AA, Lansink CS, Pennartz CM. Internally generated sequences in learning and executing goal-directed behavior. Trends Cogn Sci (Regul Ed ). 2014;18(12):647-657.
14. Benchenane K, Peyrache A, Khamassi M, et al. Coherent theta oscillations and reorganization of spike timing in the hippocampal-prefrontal network upon learning. Neuron. 2010;66(6):921-936.
15. Buzsáki G. Large-scale recording of neuronal ensembles. Nat Neurosci. 2004;7(5):446-451.
16. Ruther P, Paul O. New approaches for CMOS-based devices for large-scale neural recording. Curr
Opin Neurobiol. 2015;32:31-37.
17. Voigts J, Siegle JH, Pritchett DL, Moore CI. The flexDrive: An ultra-light implant for optical control and highly parallel chronic recording of neuronal ensembles in freely moving mice. Frontiers in
systems neuroscience. 2013;7:8.
18. Siegle JH, Hale GJ, Newman JP, Voigts J. Neural ensemble communities: Open-source approaches to hardware for large-scale electrophysiology. Curr Opin Neurobiol. 2015;32:53-59.
19. Hamilton D, Brigman J. Behavioral flexibility in rats and mice: Contributions of distinct frontocortical regions. Genes, Brain and Behavior. 2015;14(1):4-21.
20. de Chaumont F, Coura RD, Serreau P, et al. Computerized video analysis of social interactions in mice. Nature methods. 2012;9(4):410-417.
21. Serreau P, Chabout J, Suarez S, Naudé J, Granon S. Beta2-containing neuronal nicotinic receptors as major actors in the flexible choice between conflicting motivations. Behav Brain Res.
2011;225(1):151-159.
22. Pérez-Escudero A, Vicente-Page J, Hinz RC, Arganda S, De Polavieja GG. idTracker: Tracking individuals in a group by automatic identification of unmarked animals. Nature methods. 2014;11(7):743-748.
23. Ou-Yang T, Tsai M, Yen C, Lin T. An infrared range camera-based approach for three-dimensional locomotion tracking and pose reconstruction in a rodent. J Neurosci Methods. 2011;201(1):116-123.
24. Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K. JAABA: Interactive machine learning for automatic annotation of animal behavior. nature methods. 2013;10(1):64-67.
25. Anderson DJ, Perona P. Toward a science of computational ethology. Neuron. 2014;84(1):18-31.
