1
Are individual differences in brain structure determinative of visual
working memory performance?
Marthe Smedinga Supervisor: Olympia Colizoli Co-assessor: Jaap Murre Brain and Cognition Centre University of Amsterdam2
Introduction
Working memory is the process of actively maintaining a representation for a short period in mind without current input. Working memory storage is often subdivided on the basis of the content of the information that is memorized (Ungerleider, 1998). Mostly, it is divided into a subsystem for verbal information and one for visual information. In this thesis we will focus only on visual working memory. Previous research has identified different brain regions involved in visual working memory, but it remains unclear which information they process or how they interact (Luck, 1997). Visual working memory provides an essential link between perception and higher cognitive functions. It is crucial to observe the world around us to able to adapt to it. Not only do we have to respond quickly to physical stimuli around us, it also can be necessary to respond quickly to social changes. It provides us with the necessary input for higher cognitive functions, such as social interactions. These are sometimes only observable by the visual system. The quicker we can respond to visual changes around us, the better our visual working memory works (Harrisson, 2009). Since general intelligence can be interpreted as the rate in which we adequately react to changes around us, the positive relation between general intelligence and visual working memory is not surprising (Vogel, 2001). On account of the importance of visual working memory in our daily life, it is also important to understand its underlying mechanisms from a social perspective. The specific functions of the brain regions involved in visual working memory have to be studied first, in order to understand its underlying mechanisms. In the end this understanding would allow us to predict visual working memory ability on the basis of brain structure.
In literature, there can be found several theories about the build-up of visual working memory. In this respect two contradictive theories are most dominant: the first focuses on functionally specialized brain areas of the visual cortex and the second one on the importance of the
primary visual cortex.
According to the first theory (theory 1) functionally specialized brain areas of the visual cortex determine visual working memory performance. These areas are activated when a specific function has to be performed, such as recognizing faces or colors, which make them functionally specific. Studies confirming this theory revealed selective activation of these brain regions during recognition of particular visual stimuli, such as faces or color (Kanwisher 2010, Epstein, 1998, Dahaene 2002). Furthermore, the theory states that information of visual input is divided by its content in early visual areas and then send to the corresponding functionally specialized areas of the visual cortex to maintain the representation for a short period of time. This is supported by studies of Kanwisher et al. (1997, 2007, 2010), Epstein (1998) and Dehaene (2002), which showed that the functionally specialized brain regions are involved in visual working memory. There are six brain areas of the visual cortex identified as functionally specific. The first was a region identified by Kanwisher et al. (1997) in the fusiform gyrus, the fusiform face area (FFA). This becomes more active when it is exposed to images of faces compared to images of other objects. This confirms the hypothesis that face and object recognition involve qualitative different processes that may occur in different brain areas. Epstein (1998) showed that a sub-region of the parahippocampal cortex is involved in the analysis of places. Functional magnetic resonance imaging (fMRI) studies indeed indicated that the parahippocampal
3
place area (PPA) is highly active when exposed to images of places, such as landscapes or rooms, compared to images of faces.
Dehaene et al. (2002) found that there is a specialized cortical region in the brain involved in the perceptual function of recognizing words. When words are presented to a person in his own language, brain activity can be observed in this sub region of the left fusiform gyrus: the visual word
form area (VWFA).
Malach et al. (1995) reported a cortical region that responded stronger to images of objects than to images of textures without obvious shape interpretations; the lateral occipital complex (LOC)
(Grill-Spector 2001).
The extrastriate visual cortical area V4 is, similarly to other early visual cortical regions, involved in orientation, recognition of form, spatial frequency, and color. In addition, the amount of attention to the stimulus is from major importance to neuronal activation in V4 (Goddard 2011). If the theory according to which functionally specialized brain regions are crucial to visual working memory is correct, brain structure of V4 should be correlated to visual working memory ability when the subject has to remember the color of images for example.
Hubel and Wiesel (1998) showed that V1 neurons are activated selectively by a number of attributes: orientation of an image, direction of motion, luminance and spatial and temporal frequency.
According to theory 1, we may predict that visual working memory ability is dependent on the content of the visual input. In this case we assume that the information stream of the visual input is directly subdivided on its content and then analyzed by its corresponding function specialized regions of the visual cortex before it is memorized. On the other hand, it is to be questioned if these brain regions are not too upstream in the visual processing system to be involved in visual working memory, since the processing would simply take too long for this function. Visual working memory can maintain representations of visual input while they last less than halve a second. In this perspective, it may seem more logical if a brain area more downstream in the visual system is more important for visual working memory. This approach forms the basis for the second theory (theory 2) about the build-up of visual working memory, which mainly focuses on the primary visual cortex. Supèr et al. (2001) found a relatively greater activity of V1 in monkeys during short-term visual memory use. In this study, monkeys were trained to remember spatial location by means of an eye movement. Chronically implanted microwave electrodes measured activity in the V1 neurons. Although this finding suggests a greater role for V1 in visual working memory, the increase in baseline activity can also reflect the effects of spatial attention, which was already related to V1 in the previous theory (Kastner, 2000; Ress, 2000).
In addition, a more recent study done by Harrisson et al. (2009) provides new evidence that shows that early visual areas (including V1) can retain specific information about visual input, instead of mainly processing visual information forward to areas more upstream of the visual cortex. In this study, participants had to remember a precise orientation of a grating in a delayed orientation discrimination task while cortical activity was measured using fMRI. In absence of direct input V1, as well as other parts of the sensory area, maintained the orientation information to support higher-order cognitive function. The involvement of V1 here is not surprising, given that previous research showed that this region is specialized in preserving orientation input (Kastner, 2000). However, this study contributes to the increasing evidence that V1 is not only important for processing information from direct input, but is also involved in conscious perception, attentional selection and more
4
complex cognitive functions important for general visual working memory ability (Kastner, 2000;
Tong, 2003; Kosslyn, 2001; Roelfsema, 2005).
It is important to investigate which brain regions are responsible for which part of the process, to understand the underlying mechanisms involved in visual working memory,. Therefore, we want to investigate which of the two theories is correct. Clarification about the brain structure that is involved in visual working memory, allows us to study whether visual working memory ability can be predicted from brain structure in one or more regions of interest.
In our approach, we follow a relatively new development in the research field of cognitive neuroscience, which focuses on inter-individual differences in datasets instead of only averaging over groups. Inter-individual variability was often treated as noise in research, but since recent years it is increasingly used as a source of information to link human behavior and cognition to brain anatomy (Kanai & Rees 2011). In this manner, we link visual working memory ability to brain anatomy.
Research Questions and Hypotheses
In order to answer which brain regions are involved in visual working memory and whether visual working memory ability can be predicted from brain structure, we first have to study which of the two previously discussed theories is correct. In other words, we have to answer how the content of visual input determines visual working memory ability and which brain areas are involved in this process. If the content of visual input does influence visual working memory ability, it may be due to the involvement of the functionally specialized brain areas that form the basis of theory 1. If theory 1 is correct, the amount of functionally specialized brain areas of the visual cortex (6) has to be equal to the amount of factors causal for general visual working memory ability. The more factors involved in determining visual working memory ability, the more brain areas are probably involved. As discussed earlier, there are six functionally specialized brain regions in the visual system identified, each specialized in analyzing a different category of images in relation to content (FFA, PPA, VWFA, LOC, V4 and V1). It remains uncertain whether those six regions are also determinative for visual working memory ability. There is a possibility that these regions are indeed involved in analyzing the different categories of images, but there is another determinative region in visual working memory ability possible. This region might be the primary visual cortex, according to the second theory we discussed in the introduction. In this case there will probably be only one factor determinative for visual working memory performance. To find out how many factors are involved in visual working memory, our first research question is:1. Is the general visual working memory ability determined by content of the visual input associated with the six functionally specialized brain regions in the visual cortex (FFA, PPA, VWFA, LOC, V4 and V1)? (Experiment 1)
According to the two contradicting theories about visual working memory, we can hypothesize two different outcomes. According to theory 1, we hypothesize that six factors are causal for visual working memory ability, corresponding to the six functionally specialized regions of interest: orientation and color (V4), shape (LOC), space (PPA), faces (FFA), global-local (V1) and verbal-visual
5 (VWFA). Our second hypothesis is based on theory 2, stating that visual working memory ability can be predicted by V1 performance. According to this theory, we hypothesize that only one factor is causal for visual working memory test performance, regardless the content of the visual input. In this study, all subjects will perform 31 short visual working memory tasks. The only difference between these tasks is the content of the visual input, such as houses, faces and color. All these tasks correspond to one of the seven subjects that are associated with one of the six functionally specialized brain regions of the visual cortex, as is discussed in the introduction.
Using structural equation modeling (SEM) and confirmatory factor analysis (CFA), we can test which of the 31 test outcomes are correlated to each other and therefore if the content of the visual input determines visual working memory ability. If the outcome of all tasks where faces were used as visual input is significantly different from the outcome of tasks where places are used, content apparently influences visual working memory ability. We will use the factor structure calculated by SEM and CFA to study whether there is a correlation between visual working memory performance and brain structure. This brings us to the next research question: 2. Is the factor structure of visual working memory ability correlated with grey matter density or white matter connectivity in the whole brain? (Experiment 2) First of all, the correlation between visual working memory performance and brain structure has to be very strong in order to find a significant outcome in a whole brain analysis due to the multiple comparison problem. The multiple comparison problem arises when the analysis consists of a lot of separate tests, because the limit of significance has to be adjusted for every test that is done. We will perform a permutation test voxel-wise when we analyze the brain scans, and therefore the chances of finding a significant outcome is relatively low.
Since this is an exploratory study, we hypothesize that a correlation between VWM ability and brain structure can result in several outcomes. On the basis of theory 1 and theory 2 discussed in the introduction, we can expect two different outcomes based on literature. One possible outcome is a correlation between the factor structure of experiment 1 and brain structure of the seven functionally specialized regions of interest, based on the studies done by Kanwisher et al. (1997), among others. According to the other hypothesis based on theory 2 (Supèr, 2001; Harrisson, 2009), we may expect to find a correlation between visual working memory performance and brain structure of V1.
Brain structure will be determined via Diffusion Tensor Imaging (DTI) and T1 weighted images. DTI is used to measure white matter connectivity and T1 weighted images to measure grey matter density. A possible correlation between visual working memory ability and brain structure is never tested before in this manner. As a result, it remains unclear which brain structures will correlate to visual working memory ability. The outcome can obviously also result in a correlation between visual working memory ability and a whole other brain region than the functionally specialized ones in the visual cortex or V1. This also depends on the amount of factors we find in the first part of the analysis. Due to the consequences of the multiple comparison problem, we will test also if the factor structure is correlated with brain structure in the function specialized regions of interest (ROIs) or V1:
6
3. Is the factor structure correlated with grey-matter density or white-matter connectivity in the function specialized ROI(s) or V1? (Experiment 2)
Due to the multiple-comparison-problem, the probability of finding a significant correlation between the factor structure of visual working memory ability and brain structure is much higher when we specify our regions of interest in the brain analysis. If the factor analysis of experiment 1 shows that there are several factors involved in memory ability, we expect to find a correlation between the visual working memory performance and brain structure of the functionally specialized regions of
the visual cortex based on theory 1.
If the factor analysis of experiment 1 will result in only one variable that explains the greater part of the variance, we hypothesize in line with theory 2 that brain structure of V1 is will be correlated to visual working memory ability. If this is the case, then visual working memory ability can to a certain extent be predicted by brain structure of V1. In other words, subjects that perform better on the visual working memory test would have a higher grey matter density or more white matter connectivity in V1 than subjects that scored lower on the test. Based on this hypothetical result, we would be able to predict visual working memory performance on the basis of brain structure of V1. In both hypothetical cases, the question remains why visual working memory ability correlates highly with general intelligence, since general intelligence is never before associated with the functionally specific brain regions of the visual cortex or V1 (Vogel, 2001). 4. Is there a correlation between the underlying factors of visual working memory performance and general intelligence? (Experiment 3) We expect to confirm the outcome of a study done by Vogel et al. (2001), which showed a positive correlation between visual working memory and general intelligence (GI). Depending on the outcome of experiment 1 and 2, the functionally specialized regions of the visual cortex or V1 would then indirectly relate to general intelligence. This outcome would implicate that we, humans, possess something like ‘visual intelligence’, just as we possess ‘verbal intelligence’ or ‘mathematical intelligence’ (Ramsden, 2011). If we indeed find a correlation between visual working memory ability and IQ, the outcome of experiment 2 will implicate which brain region is associated with this function. We will elaborate further on this implication in the discussion of this report.
Methods
Experiment 1: Factor Structure of the Online Visual Working Memory Test
First, we analyzed a dataset of an online 2- back test (N = 113). This test is ought to measure visual working memory ability. The University of Amsterdam psychology’s research portal was used to recruit the majority of participants, which resulted in 113 healthy participants (21 males). The participants were on average 20.55 years old with a standard deviation of 3.98 years. Participants completed the battery of 2-back tasks online via the university website (www.memory.uva.nl), which7
The 2-back test was first used by Kirchner in 1958 and is still used to measure visual working memory ability (Kirchner, 1958). During this task a series of images is presented while the participant has to remember which image was presented two images before (see Figure 1). The stream of images was presented on a computer screen and if the current image was identical to the one presented 2 images before, the participant had to click on a ‘ MATCH’ button with the computer mouse. The time window in which the participant had to response consisted of 1000 ms during the presented image and 1000 ms hereafter during the presentation of a small cross. Every presented image was followed by small cross, which makes the total response window 2000 ms.
The task consisted of 31 short tests of 3.5 minutes, all containing a different category of images, such as houses or faces. The participants did not know the faces. All these categories of images were based on one the associations of one of the six functionally specialized brain regions and the content that they were specialized in: orientation and color (V4), shape (LOC), space (PPA), faces (FFA), global-local (V1) and verbal-visual (VWFA). The test could be divided into 8 different sessions of four tests, which took each 15 minutes. Not all these sessions had to be finished in one row, the subject had the possibility to continue with further sessions on the website at any time.
As a result of these measurements, the factor structure of memory ability could be determined using structural equation modeling (SEM). SEM is a statistical technique to test or estimate underlying causal relations in a dataset. We will test two causal models of category structures to find which structure fits best to the behavioral data using SEM in the Lavaan package in R (Rosseel, 2012).
The first model we tested, model 1, corresponded to theory 1 discussed in the introduction. According to this model, visual working memory ability is mainly determined by the performance of six functionally specialized brain areas of the visual cortex. Model 1 consisted of seven latent variables that were hypothetically causal for the outcome of the visual working memory test. The 31 different tests that were used in this test were subdivided on the content of each test: 4 working memory tests where different faces were used as visual stimuli, 5 tests where orientation was tested, 3 different tests using color, 8 tests with shapes, 3 for spatial visual memory, 4 for global-local visual stimuli and there were 4 tests where visual working memory was was tested using written words as visual stimuli. Each latent variable of model 1 corresponded with one of these groups. 'Faces' was for instance one latent variable, and therefore causal for the outcome of the four tests where faces were used as visual stimuli. In Lavaan, we set the factor loading of each first test group per latent variable on one, so that we could compare the covariances between groups. Model 2 corresponded to theory 2 discussed in the introduction. According to this theory V1 is most important in visual working memory performance. This model contained one latent variable causal for the general visual working memory performance. This factor would therefore hypothetically explain greater part of the individual differences in overall outcome of the 2-back
8 tests. This latent variable can be seen as ‘general VWM ability’, because all tests where concluded in this latent variable. If there is one latent variable explaining a relatively large part of the variance caused by individual differences, the SEM outcome of model 1 would practically be quite similar to the SEM outcome of model 2. If this would be the case and we test model 1, the causal factor of one latent variable would be nearly similar to the causal factor of another latent variable, since the individual differences for a large part would be explained by the 'general visual working memory ability' factor. For example, if the overall score of tests that consisted of faces is determined by the ‘general visual working memory ability-factor’ (model 2), the factor score would be quite similar to the factor that is causal for score on the tests that contained places as visual stimuli. All seven latent variables would have different subgroups, but in the end would come down to this factor that determines visual working memory ability in general. In this case the covariances between the latent variables would be relatively close to 1. If the SEM outcome of model 1 is close to the outcome of model 2, then we will test this hypothesis by comparing the covariances.
Since the number of participants (N=113) is relatively small in order to apply SEM on the data in a good manner , we did a confirmatory factor analysis using SPSS to check the factor structure obtained via SEM.
Experiment 2: Brain regions are correlated with visual working memory
ability.
In this neuro-imaging study 42 healthy right-handed female university students participated (age 18-25 years). The test consisted of a visual working memory 2-back test, identical to the one used in experiment 1. Before testing, all subjects were screened by email, had normal or corrected-to-normal visual, were right-handed to the Edinburgh Inventory and reported no neurological or psychiatric history. A written informed consent was obtained of all subjects before scanning. Participants received research credits or money for their participation.MRI data acquisition
The DTI and T1 images were acquired in a separate single session from the online 2 back test, using a Philips 3T scanner (Amsterdam, Netherlands). Functional MRI scans were acquired, but not analyzed for the current experiment. Foam pads were used to immobilize the subjects had and earplugs were used to moderate scanner noise. T1-weighted images T1-weighted scans were used for the voxel-based morphometry based grey matter analysis. The dependent variable is therefore grey matter volume per voxel. Two T1 scans were acquired per participant: one at the very beginning of the experiment and one at the very end. The following scan settings were used to obtain the scan images: three-dimensional T1 turbo field echo, repetition time (TR) = 8132 ms, echo time (TE) = 3.74 ms, flip angle = 8°, 160 sagittal slices of 1.2 mm, field of view (FOV) = 256 x 256, matrix = 256 x 256. Diffusion-weighted scan9 Diffusion-weighted images were used to measure white matter connectivity using diffusion tensor imaging analysis. In this experiment the dependent variable was the fractional anisotropy value per voxel. There were 4 of these types of scans in the middle runs of the experiment. The following scan settings were used to obtain the scan images: diffusion-weighted spin-echo echo-planar imaging (EPI) measurements, TR = 6311 ms, TE = 73.36 ms, flip angle = 90°, FOV = 224 x 120, matrix = 112 x 109, 60 slices per volume, b0 = 1000 s mm-2. MRI data analysis Voxel-based morphometry based on the T1-weighted scans An optimized voxel-based morphometry protocol was used to analyze T1-weighted structural data (Good et al., 2001; Donaud et al., 2007) as part of FSL. FMRIB Software Library (FSL) version 5.0.4, Oxford, UK: http://www.fmrib.ox.ac.uk/fsl (Smith et al., 2004; Woolrich et al., 2009; Jenkinson et al., 2012) was used for the analysis of the MRI images. Structural images were first brain-extracted and then the grey matter was segmented before they were registered to the T1-weighted MNI-152 standard space using non-linear registration (Andersson et al., 2007a; 2007b). The registered images were averaged and flipped (along the x-axis) in order to create a left-right symmetric, study-specific grey matter template. In addition, to correct for local expansion or contraction due to the non-linear component of the spatial transformation, all native grey matter images were non-linearly registered to this study-specific template. Using an isotropic Gaussian kernel with a sigma of 4 mm, the modulated grey matter images were smoothed. To correct for the multiple comparisons across space (family-wise error rate=5%), since we use voxel-baed morphometry, we implemented a voxel-wise t-test, a correlation analyses (GLM) using the Threshold-Free Cluster Enhancement (Smith &Nichols, 2009) option in Randomise (Nichols &Holmes, 2002) and a permutation-based non-parametric test (Anderson &Robinson, 2001) with 25.000 permutations. The permutation-based tests were first done 1) for the whole brain and then 2) for an a priori defined region of interest: V1, based on Harrison (2009). Tract-based spatial statistics (TBSS) based on the diffusion-weighted scan. In FSL (Smith, 2004), we used TBSS (Tract-Based Spatial Statistics; Smith, 2006) to analyze fractional anisotropy (FA) values of white matter voxel wise . First, using the nonlinear registration tool FNIRT (Andersson, 2007a, 2007b) all the FA data from the participants were aligned into a common space. This tool uses a b-spline representation of the registration warp field (Rueckert, 1999). Then, a mean FA skeleton that represents the centers of all tracts common to the group was created with a thinned mean FA image. Lastly, permutation-based non-parametric testing (Anderson &Robinson, 2001) was used to apply cluster-based thresholding (Z-threshold =2.3). We corrected for multiple comparisons across space (family-wise error rate =5%). Region of Interest Primary visual cortex V1 was defined bilaterally based on the Juelich Histological Atlas (part of FSL), we took the voxels corresponding to 50-100% probability. The region of interest for the grey matter analysis (the correlation between working memory visual scores and grey matter) was defined by the intersection of the atlas-based V1 image (in 2 mm T1-weighted MNI-152 space) with the grey matter mask resulting from the VBM analysis.
10
The region of interested for the white matter analysis (the correlation between working memory visual scores and white matter) was defined by the intersection of the atlas-based V1 image (in 1 mm T1-weighted MNI-153 space) with the average fractional anisotropy mask resulting from the TBSS analysis (the white matter skeleton was not used for this analysis).
Experiment 3. Visual working memory and general intelligence
Participants (N=26, 13 male) were on average 24.8 years old with a standard deviation of 3.8 years. All participants were screened for color blindness and had a normal or corrected-to-normal vision. A written formed consent was obtained before the experiment and participants could receive research credits for their participation. A similar 2-back test from experiment 1 was used to measure VWM ability. During the test the subject had to press also the ‘match button’ (LEFT or DOWN on the keyboard) if the current image on the computer screen was identical the image shown two images before (see Figure 1). The test consisted of four rounds of five minutes each. The first of these rounds was meant for the participant to practice and therefore not taken into account during the analysis. The overall test score was calculated by the following formula:! " # $ % =
ℎ& ' ( + & ) " # $ $ % " ' + * & ( ( % (
ℎ
& ' (
Each round consisted of 10 series of images. Per series one of the seven categories of images was showed, such as faces or colors. Therefore, each round consisted of a mix of visual stimuli. During the stream of images, every image was presented for 750 ms. followed by a cross for 250 ms. The time window to respond was in this case 1 sec., instead of 2 sec. in experiment 1. All participants did the 2-back test before they began the IQ test.To measure general intelligence, we used the Raven Working Memory Test, originally developed by John C. Raven in 1936 (Raven, 1936). This test is proved to be correlated with other IQ tests and is independent of culture or language (Carpenter 1990).
The Raven working memory test was done online (http://www.iqtest.dk/main.swf). During the test they had to choose the missing piece of the puzzle (See Figure2). The participants had 40 minutes to answer 39 questions. The level of difficulty of the questions increased gradually. The participants were not rewarded for finishing before the time was up. A Pearson correlation test in SPSS was applied to test correlation between VWM ability and IQ score, corrected for sex and age and both tests were held under supervision.
11 Figure 2: Example of a Raven Working Memory test question.
Results
Experiment 1
Using the Lavaan package in R (Rosseel, 2012), we tested two causal models on the visual working memory dataset applying SEM. Model 1 consisted of seven latent variables, categorized by content of the presented stimuli used in the 2-back test (faces, orientation, color, spatial, shape, global-local and verbal-visual). Model 2 consisted of only one factor that was determinant for VWM ability.Both models gave almost the same goodness of fit, as is shown in Table 1. The following test outcomes give an indication of the goodness of fit for both models:
Akaike information criterion (AIC) is a test of relative model fit; the preferred model is the one with the lowest AIC value. Root mean square of approximation (RMSEA) is another test of model fit, where all values < 0.05 are considered to indicate a good fit. An RMSEA of 0.1 or more is often taken to indicate poor fit. As shown in Table 1, both models do not have a significantly good fit, which can be due to the little amount of participants. Standardized Root mean Residual (SRMR) is another measurement of fit, which is usually the most important indication. A model that fits well should have a SRMR smaller than 0.05 and therefore both models have a significant good fit according to this statistical test. Finally, the comparative fit index (CFI) compares the average correlation between variables. A CFI value of 0.90 or more is desirable.
Measurement/model Model 1 (7 latent variables) Model 2 (1 latent variable)
AIC 27.268 27.233 RMSEA 0.058 0.062 SRMR 0.047 0.048 CFI 0.941 0.929 Table 1. Comparison outcomes of structural equation modeling. Since the SEM test outcome is almost similar in both models, we tested correlations between all latent variables of model 1. As showed in Table 2, the covariances between the seven latent variables of model 1 were extremely high. This indicates that the causal factor of the different latent variables is almost the same, hence there is one determinant factor in visual working memory ability.
Covariance
Faces Orientation Color Spatial ShapeGlobal-local
Faces
12 Color 0.918 0.911 Spatial 0.852 0.906 0.936 Shape 0.971 0.969 0.986 0.899 Global-local 0.881 0.855 0.988 0.855 0.890 Verbal visual 0.946 0.948 1.017 0.941 0.990 0.969 Table 2. Covariences of model 1. The outcome of the factor analysis shows that one factor explained 49,348% of the variance and a possible second factor explained for 4,073% of the variance. We can therefore confirm that there was one factor determinative for visual working memory performance, which can be interpreted as general visual working memory ability.
Experiment 2
Whole-brain analysis We found no significant correlation between the factor structure found in experiment 1, general visual working memory ability, and brain structure in the whole brain analysis. Region of Interest analysis Given the results of experiment 1, which showed one factor causal for visual working memory ability, V1 was our region of interest in the adjusted brain analysis. We found no significant correlation between average memory score and white matter connectivity in V1.We did find a significant correlation between the average memory score and grey matter density in V1 (see Figure 3). The white area is V1 according to the Juelich Histological Atlas (part of FSL) and the blue area is where there was a significant correlation between grey matter volume and the average VWM test scores. The coordinates in MNI space of this image are (16, -70, 6)
Figure 3: V1 and visual memory factor correlation
13 Figure 4 is a graph of the significant correlation between general VWM ability and grey matter density in V1. The x-axis is the average grey matter per person in the significant cluster shown in the previous image. The y-axis is the average visual memory score. This score is demeaned, in order to subtract the total mean from everyone's individual score. Figure 4. Visual Memory and V1 grey matter correlation
Experiment 3
Table 3 shows the descriptive statistics of experiment 3. During this experiment we wanted to answer if visual working memory ability was correlated to general intelligence. From all 26 participants, the lowest IQ-score was 93 and the highest is 139. The average score was 117.04, which is close to the median of 117.5. Next to that, almost 50 percent of the subjects are male. Hence, the data covers a broad range of subjects, which are evenly distributed over the set.
Table 3. Descriptive statistics
Figure 5 and Table 4 show the R² linear regression line for the IQ-score and the overall visual working memory test score. The line is upward-sloping with a R² of 0.146, which means that subjects with higher IQs tend to have higher visual working memory test scores.
14
The Pearson correlation test resulted in a significant outcome of 0.382 (α=0.05), which showed that visual working memory ability and general intelligence are positively related.
To control for sex and age, we did a simple linear regression with age and sex of the participants as control variables (See table 5). Firstly, the table shows that the sex of the subject does not affect its score, but that its age does. Secondly, it shows that subjects with a higher IQ-score significantly had higher average scores on the test. According to these results, increasing IQ over the subjects results in an increase of the average score of 0.006 per IQ-point. Figure 5. Correlation average visual working memory score and IQ score
Average Score IQ score
Average Score Pearson Correlation 1 ,382 Sig. (2-tailed) ,054 N 26 26 IQ score Pearson Correlation ,382 1 Sig. (2-tailed) ,054 N 26 26 Table 4. Pearson correlation coefficient
15 Table 5. Regression for the Average Score
Discussion
During this study we focused on individual differences in visual working memory ability and brain structure. How we analyze visual input using our visual working memory is still relatively unknown. First the brain regions involved in visual working memory and their function has to be identified. In this context, we found two leading theories in previous literature that are contradictive to each other. One theory hypothesizes that functionally specialized areas in the visual cortex are most important for visual working memory ability. This theory states that visual input is analyzed in these areas subdivided by content. The second theory states that V1 is most important for visual working memory ability and that the functionally specialized brain areas are only used in during a longer time
window than that of visual working memory.
In order to study which brain regions are important for visual working memory, we first studied how content of visual input determines visual working memory performance. Using structural equation modeling and confirmatory factor analysis we came to conclusion that there is only one factor determinant for general visual working memory ability and content is therefore not of influence. Using T1 weighted images we found a positive correlation between this factor, which can be interpreted as "general visual working memory ability-factor", and grey matter density in V1. This outcome confirms the literature stating that V1 is most important for visual working memory ability. In addition, we found a correlation between this "general visual working memory
ability-factor" and general intelligence. Experiment 1 In experiment 1 we analyzed a dataset of 113 subjects that had done a 2-back task, which tests visual working memory ability. The test consisted of 31 short 2-back tasks, each containing different visual input that was associated with one of the six functionally specialized brain regions in the visual cortex
16
(FFA, PPA, VWFA, LOC, V4 and V1). If the functionally specialized brain areas would dominantly be involved in analyzing the tests, we would find several factors causal for the outcome of the visual working memory tests. However, using structural equation modeling and confirmatory factor analysis we found only one factor causal for the test outcomes of all 31 tests. This factor can therefore be interpreted as "general visual working memory ability-factor", since outcome is apparently not influence by content of the visual stimuli. Since this outcome confirms the second theory discussed in the introduction, which stated that V1 was most important to determine visual working memory ability in individuals, V1 is now our region of interest for experiment 2.
The disadvantage of applying structural equation modeling to study your dataset is that a minimum of 10 observations per variable (test) is desirable (Rosslee, 2012). Since our dataset consists of 113 participants, and not the desirable 310, we can only observe a trend using structural equation modeling instead of drawing conclusions. It is important to mention that the outcome of structural equation modeling is vulnerable to unsubstantiated speculation. Any claims of causality should always be taken with caution and confirmed by an observational study. Especially when the amount of subjects is not large enough in accordance with the statistical guidelines. On the other hand, the outcome of structural equation modeling was supported by the outcome of the confirmatory factor analysis in SPSS, which fitted to the size of our dataset.
In addition, as a result from experiment 1 we took V1 as the main region of interest for experiment 2 based on literature supporting theory 2. It could be quite possible that the "general visual working memory ability-factor" is correlated in a totally different brain region. On the other hand, if there was a strong correlation of this factor and brain structure, we would have found it during the whole brain analysis.
Experiment 2
During this experiment, we tested a correlation between visual working memory and brain structure. We knew from the results of experiment 1 that only one factor was causal for the outcome of all tests and therefore visual working memory ability could be calculated as one overall score per individual. All participants in experiment 2 (N=42) had also participated in experiment 1, from which the overall score per individual was calculated. We used DTI to measure white matter connectivity and T1 weighted images to measure grey matter density. In the whole brain analysis we did not find any significant correlation in both measurements. V1 became our region of interest for a more specific analysis after experiment 1, based on previous literature. We found no correlation between white matter connectivity and general visual working memory ability per individual. However, we did found a significant correlation between the "visual working memory ability-factor" and grey matter density in V1. We did not use a custom specific ROI in data-analysis of the imaging data in this experiment, which might have changed the outcome a bit, possible in both directions. We probably did not find a correlation between grey matter density in V1 and general visual working memory ability during the whole brain analysis due to the multiple comparison problem. Since we analyzed the correlation voxel-wise, a lot of tests were done, which we all have to control for. As a result, the effect has to be very large in order to get a significant result.
In addition, we also did not find a significant correlation between white matter connectivity in V1 and visual working memory ability. This could be due to the fact that we did not use a custom specific ROI, simply because the subject group was not large enough, or because there is no effect of
17
white matter connectivity of V1 on VWM ability.
Although there is sufficient literature to support the outcome of experiment 1 and 2 (Kastner, 2000; Luck, 2001; Harrisson, 2009), it is on the other hand contradictive to studies that found an association between the functionally specialized brain areas and visual working memory ability (Courtney, 1998; Kanwisher, 2000; Haxby, 2000). One explanation for this difference in outcome could be due to the velocity in which we tested visual working memory. We tested visual working memory in a relatively high rate, which could result in a lack of time to analyze the visual input more upstream. This might be the difference in method that brought the studies to a different outcome. In order to get a real insight in the further meanings of this outcome, more research has to be done to study the differences in studies/methods that support one or the other theory. Secondly, we don't necessarily see a difference in brain structure back in behavior. In our case, we presume that more grey matter density result in better performance associated with this brain area, but this is not necessarily the case. V1 is obviously also involved in other functions than visual working memory. To study this relation on a further level, we first have to confirm that V1 indeed has a higher activation when visual working memory is used via fMRI. Although this is tested before (Súper, 2001; Harrison, 2009), a correlation in humans with a more general visual working memory test, such as the 2-back test, has to confirm this outcome in order to draw conclusions. Experiment 3 A study of Vogel et al. (2001) found a positive correlation between general visual working memory ability and general intelligence. We confirmed this outcome by finding a significant positive correlation between visual working memory ability and IQ score.
We cannot draw firm conclusions from this outcome. Firstly, because we tested only 26 participants. Secondly, because our test population is not representative for the general population. All subjects had an academic background and a relative high IQ score (average of 117, with a standard deviation of 11.3) compared to the general population (average IQ score of 100 per
definition).
Additionally, there are some disadvantages on the methods we used. Males score on average higher than females in the Raven matrices test and most other non verbal IQ tests. Besides, we did not counterbalance the order in which participants did both tasks. All participants did the 2-back test before they did the Raven Matrices test. Many of the participants complained that the 2-back test was too fast to give the right response, but since the speed was the same in all participants, this would in the end have not make a large difference in outcome. Finally, it could have influenced the results that the experimenter knew all participants. Important to address is that we cannot draw any firm conclusions yet about which factor is dominant in determine general VWM ability. What we do show is that VWM is determent by one factor, and not seven factors corresponding to the functionally specialized areas in the visual cortex. In addition, we find that grey matter density of V1 is predictive for this factor. Since this was an exploratory study, more studies have to be done using other visual working memory test to confirm this finding. In the end this might show us new possibilities to measure and train VWM ability, which will have a major societal impact, because of its importance for general intelligence.
We cannot draw any conclusions from our pilot study (experiment 3), because we did have not enough subjects in order have a representative group for the general population. We therefore
18
cannot conclude that visual working memory is determent by general intelligence or that V1 is involved in general intelligence. In order to do so, we first have to study if brain structure in V1 is also correlated with general intelligence and we have to elaborate the pilot experiment with a greater number of participants. If this indeed will confirm our findings, this has major implications for as what we see as general intelligence. This would suggest we have something similar to "visual intelligence". We might test only visual intelligence when we use the Ravens matrices task to measure general intelligence. During this task, the subject is confronted only with visual puzzles and not with auditory questions for example. In order to find the answer to the questions of the test, one needs to have to a good visual imagination, which implicates logically an involvement of the visual system. To confirm our pilot study in a general sense, we would have to combine the 2-back test with a auditory or verbal-visual intelligence test. Previous research showed that verbal IQ changed with grey matter density in a brain region that was activated by speech and non-verbal IQ changed with grey matter density in a region that was activated by finger movement (Ramsden, 2011). This implicate that indeed grey matter density has a positive correlation with the sort of intelligence that the particular brain region is specialized in, which is consistent with the correlation we found in experiment 2. If further research confirms that V1 is a brain area specialized in visual intelligence, our finding would be a great contribution to this field of research. These findings are theoretically not surprising given that more grey matter density means that there are more neuronal cell bodies in this particular area, i.e. the region is generally
more developed.
We did not find a significant correlation between white matter connectivity in V1 and the "general visual working memory ability-factor". This measurement should not be confused with the grey matter density measurement. White matter connectivity is a measurement of the amount of myelinated axons in the particular area, and therefore can be interpreted as the amount of connectedness with other brain areas. Our outcome in experiment 2 can therefore mean that people that score relatively high on visual working memory tests have not particularly more connectivity between V1 and other brain areas. This is not surprising, given that white matter connectivity is no necessary measurement of development of the brain region.
In addition, if our outcomes will be confirmed in further research, it will have major consequences for the functions we relate to V1. Since recent years more studies confirm that V1 is more than information processing from direct input alone. This study does contribute to the increasing evidence that V1 is involved in conscious perception, attentional selection and more complex cognitive functions (Kastner, 2000; Tong, 2003; Kosslyn, 2001; Roelfsema, 2005).
Finally, we did not find any evidence to relate to the theory that visual working memory consists of several subsystems divided by content of the visual input (Kanwisher, 2007). Again, this might be due to the velocity in which we tested visual working memory. To study whether speed of the visual working memory test changes which brain structure is activated, further research has to be done.
References
Anderson M J, Robinson J (2002) Permutation tests for linear models. Australian and New Zealand Journal of statistics. vol. 43.1: 75-88.19 Andersson J L R, Jenkinson M, Smith S (2007a) Non-linear registration aka Spatial normalisation. Oxford university press Andersson J L R, Jenkinson M, Smith S (2007b) Non-linear optimisation. FMRIB Technial Report TR07JA1. Oxford university press. Dehaene S, Le Clec’H G, Poline J-B, Le Bihan D, Cohen L (2002) The visual word form area: a prelexical representation of visual words in the fusiform gyrus. Brain imaging. Vol 13: 3: 321-325 Eng, Chen and Jiang (2005) Visual working memory for simple and complex visual stimuli. Psychonomic Bulletin & Review 12(6), 1127-1133. Epstein R, Kanwisher N (1998) A cortical representation of the local visual environment. Nature 392: 598-601 Fox C J, Iaria G, Barton J J S (2009) Defining the face processing Network: Optimization of the functional Localizer in fMRI. Human Brain mapping 30: 1637-1651 Frangou S, Chitins X, Williams S C R (2004) Mapping IQ and gray matter density in healthy young people. NeuroImage 23: 800-805 Goddard E, Mc Donald J S, Solomon S G, Clifford C W G (2011) Color responsiveness argues against a dorsal component of human V4. Journal of Vision 11: 4: 3 Golarai G, Ghahremani. D G, Whitfield-Gabrieli S, Reiss A, Eberhardt J L, Gabrielie J D E, Grill-Spector K (2007) Differential development of high-level visual cortex correlates with category-specific recognition memory. Nature-Neuroscience vol. 10, no. 4:512-522 Grill-Spector K, Kourtzi, Z, Kanwisher N (2001) The lateral occipital complex and its role in object recognition. Visual Research 41: 1409-1422. Harrison S A, Tong F (2009) Decoding reveals the content of visual working memory in early visual areas. Nature vol. 458: 632-635 Haxby J V, Hoffman E A, Gobbini M I (2000) The distributed human neural system for face perception. Trends in cognitive Sciences vol. 4. 6: 223-233 Hubel D H, Wiesel T N (1998) Early exploration of the visual cortex. Neuron 20:401-412. Jenkinson M, Beckmann C F, Behrens T E J, Woolrich M W, Smith S M (2012) FSL. NeuroImage vol. 62. 2: 782-790. Johnson M H, Grossmann T, Kadosh K C (2009) Functional Brain Development Building a social brain through interactive specialization. Developmental Psychology vol. 45. No. 1, 151-159 Kanai R, Rees G (2011) The structural basis of inter-individual differences in human behaviour and cognition. Nature Neuroscience 12. 231-242. Kanwisher N, McDermott J, Chun M M (1997) The Fusiform Face Area: A module in Human Extrastriate Cortex Specialized for Face Perception. The journal of neuroscience 17:11:4302-4311 Kanwisher N (2010) Functional specificity in the human brain: A window into the functional architecture of the mind. PNAS vol. 107. No. 25: 11163-11170 Kastner S, Ungerleider L G (2000) Mechanisms of Visual attention in the human cortex. Annu. Tec. Neurosci. 23:315-341 Kim M, Ducros M, Carlson T, Ronen I, He S, Ugurbil K, Kim D (2006) Anatomical correlates of the functional organization in the human occipitotemporal cortex. Magnetic Resonance Imaging 24:583-590
20 Kirchner W K (1958) Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology. Vol 55. 4. 352-358 Kosslyn S M, Ganis G, Thompson W L (2001) Neural foundations of Imagery. Nature Neuroscience Vol. 2; 635-642 Luck S J, Vogel E K (1997) The capacity of visual working memory for features and conjunctions. Nature Vol. 390; 279-281 Malach R, Reppas J B, Benson R R, Kwong K K, Jiang H., Kennedy W A, Ledden P J, Brady T J, Rosen B R, Tootell R B H (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc. Natl. Acad. Sci. Vol. 92. 8135-8139. Nichols T E, Holmes A P (2002) Nonparametric permutation tests for functional neuroimaging: A primer with examples. Human brain mapping vol. 15. 1: 1-25. Pierce K, Redcay E (2008) Fusiform function in children with autism spectrum is a matter of "who". Biol psychiatry; 64;552-560 Ress D, Backus B T,Heeger D J (2000) Activity in primary visual cortex predicts performance in a visual detection task. Nature Neuroscience vol. 3: 9; 340-345 Rosseel Y (2012) Lavaan: An R package for Structural Equation Modeling. Journal of statistical software vol. 48: 2: 2-36 Roelfsema P R (2005) Elemental operations in Vision. Trends in Cognitive science vol. 9: 5; 226-233 Rueckert D, Sonoda L I, Hayes C, Hill D L G (1999) Nonrigid registration using free-form deformations: application to breast MR images. Medical Imaging vol. 18. 8:712-721 Scherf S K, Behrmann M, Humphreys K, Luna B (2007) Visual category-selectivity for faces, places and objects emerges along different developmental trajectories. Developmental science 10:4:F15-F30 Smith S M, Jenkinson M, Woolrich M W, Beckmann C F, Behrens E J, Johansen-Berg H, Bannister P R, De Luca M, Drobnjak I, Flitney D E, Niazy R K, Saunders J, Vickers J, Zhang Y, De Stefano N, Brady J M, Matthews P M (2004) Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage vol. 23. 1: 208-219 Smith S M, Nichols T E (2009) Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference. NeuroImage vol. 44. 1: 83-98 Super H, Spekreijse H, Lamme V (2001) A neural correlate of working memory in the monkey primary visual cortex. Science 293, 120-124 Tong F (2003) Primary visual cortex and visual awareness. Nature Neuroscience vol 4; 219-229 Ungerleider L G, Courtney S M, Haxby J V (1998) A neural system for human visual working memory. Proc. Natl. Acad. Sci. USA. vol. 95. 883-890 Vogel E K, Woodman G F, Luck S J (2001) Storage of features, conjunctions and objects in visual working memory. Journal of Experimental psychology: Human perception and Performance 27:1:92-114 Woolrich M W, Jbabdi S, Patenaude B, Chappell M, Makni S, Behrens T, Beckmann C, Jenkinson M, Smith S M (2009) Bayesian analysis of neuroimaging data in FSL. NeuroImage vol. 24. 1: 173-186.
