INVESTIGATING THE EFFECTS OF ATTENTIONAL AND PERCEPTUAL BLINDNESS ON RECURRENT PROCESSING AND METACOGNITION AFTER PERFORMANCE MATCHING
Name student: Margot Steijger Student number: 11680814 Name mentor: Samuel Noorman Name supervisor: Simon van Gaal
19/06/20
2 Abstract
According to many researchers, recurrent processing (RP) is essential to consciousness. It exists as local and global RP, depending on how far information spreads throughout the brain. It is thought that attentional (AB) and perceptual blindness (PB) have a different effect on RP, which would enable researches to isolate the two types of RP. Moreover, it is found that the metacognition of participants is different for these two manipulations. The current experiment will try to better understand how metacognition and RP are affected by AB and PB, by using the attentional blink and the masking paradigm respectively. A crucial addition to previous research is matching the performance on attentional blink and masking trials. Furthermore, this experiment will combine confidence ratings (metacognition) and decoding of EEG signals (associated with the presence of RP) to study the effects of AB and PB. Based on previous research it is expected that behavior is affected in the same way by both
manipulations, while confidence ratings and decoding of EEG signals are affected differently. To support the analysis plan of the current experiment and to replicate findings of previous research, an additional data analysis was conducted on data from an experiment with a similar design. Most results confirmed the findings, but due to some limitations of the dataset not all results were as expected.
19/06/20
3 Disclaimer
This template is meant as an easy-to-use guided document that can be used to get the elements of a project clear before finalizing a registration (e.g., before ‘freezing’ a project on the Open Science Framework [OSF]). This template describes the information that is needed for a registration, and does not claim to be exhaustive. The aim is to standardize pre-registrations, e.g., so researchers can find relevant information in a structured way. Although using this template does not guarantee that a journal, peer-reviewers, or a badge certifying organization will accept the pre-registration, it is set-up to ultimately satisfy this goal. The template is updated regularly to keep up with the rapid developments in our field.
Text descriptions should be as detailed (if not more) as a standard description of procedures in a journal article. For questions or suggestions to improve the template, please contact Anna van ‘t Veer.
Useful sources
• Frequently updated step by step instructions on how to use the OSF for pre-registration: https://osf.io/k5wns/
• Video on how to use registrations on the OSF: http://help.osf.io/m/registrations/l/524205-registrations-101
• Pre-registration badge specifications: https://osf.io/tvyxz/wiki/1.%20View%20the%20Badges/ • A chapter on the subject: https://osf.io/nte3j/
• Pre-Registration paper: van ‘t Veer, A.E., & Giner-Sorolla, R. (2016). Pre-registration in Social Psychology—a discussion and suggested template. Journal of Experimental Social Psychology. doi:10.1016/j.jesp.2016.03.004
Short overview of steps for using the template
- Register on OSF and create a project page there (see https://osf.io/k5wns/files/ for instructions). - Complete this template and agree on the elements with your collaborators (the latest version of this
template can always be found here: https://osf.io/k5wns/files/)
- Optional: Upload any necessary study materials to your project page (for instance: an exported document or .qsf from a Qualtrics survey). This step is optional and can also be done during or after registering (freezing) your project. If you do it after registering, the file will not be registered (you could upload the data to your project page after it is collected, for instance).
- Register your project (see above-mentioned instructions) - Run your study
- Include a link to the timestamped preregistration as part of your paper (see instructions) - Explain any deviations from the pre-registered methods and analyses in the paper
- Make a clear distinction between confirmatory, a priori analyses and exploratory, post-hoc analyses not described in the pre-registration (for instance with two separate headings in your paper).
19/06/20
4
Project working title: Investigating the effects of attentional and perceptual blindness on recurrent processing and metacognition after performance matching
Authors: M.C.E. Steijger, S. Noorman, S. van Gaal
Affiliation: leave this out for blind review of the pre-registration A. Hypotheses
Description of essential elements
1. Describe the (numbered) hypotheses in terms of directional relationships between your (manipulated or measured) variables.
2. For interaction effects, describe the expected shape of the interactions.
3. If you are manipulating a variable, make predictions for successful check variables or explain why no manipulation check is included.
Recommended elements
4. A figure or table may be helpful to describe complex interactions; this facilitates correct specification of the ordering of all group means.
5. For original research, add rationales or theoretical frameworks for why a certain hypothesis is tested.
6. If multiple predictions can be made for the same IV-DV combination, describe what outcome would be predicted by which theory.
Theoretical background
Information processing is key to consciously perceiving stimuli. According to the current knowledge, information processing in the cortex involves roughly two phases. In the first phase information is sent from low hierarchically regions, such as the visual cortex, to higher regions, such as the parietal and frontal lobe. This process is known as the “feedforward sweep” (Lamme and Roelfsema, 2000). Once the information has moved up, recurrent processing ensures that information is sent back to the lower regions. This latter
phenomenon is thought to be (the only) essential for consciousness (Lamme, 2003; Del Cul, Baillet and Dehaene, 2007). Recurrent processing can be divided into local and global recurrent processing. Local recurrent processing involves communication between areas within a small range, whereas with global recurrent processing communication spreads throughout the whole brain. In addition, a distinction is often made between two types of consciousness (Block, 2005). On the one hand there is phenomenal consciousness, often related to local recurrent processing, representing merely the content of an experience. On the other hand, there is access consciousness, generally associated with global recurrent processing and a state in which someone can report about an experience (Lamme, 2006).
Nevertheless, other frameworks have a different interpretation regarding the types of consciousness (e.g. Dehaene and Changeux, 2011). One model implies that there are four different stages, influenced by the factors bottom-up stimulus strength and top-down attention (see Figure 1). This model suggests that the former is responsible for the presence
19/06/20
5
of recurrent processing and the latter for the depth of processing (Dehaene et al., 2006). Both are necessary for access consciousness and by disrupting one of the two factors one could create either perceptual blindness due to lack of bottom-up information or attentional blindness due to lack of access to bottom-up information (Kanai et al., 2010). With their experiment, Fahrenfort et al. (2017) were able to isolate the four stages by manipulating the two factors. They used the attentional blink (AB) paradigm to manipulate top-down
attention and masking for manipulating the bottom-up stimulus strength, creating four different conditions that represent the four stages from Figure 1. As stimuli they used Kanizsa illusions, which are optical illusions that are thought to create a good measure for recurrent processing. The illusion that is reported by participants is not actually present in the image but is filled in by the brain through top-down information (Kok et al. (2016), hence the recurrent processing.
Top-Down attention Absent Present Bottom-up stimulus strength Weak A
Little feedforward processing B
More feedforward processing
Strong C
Local recurrent processing
D
Global recurrent processing
Figure 1. Four stage model of consciousness. Top-down attention and bottom-up stimulus strength determine the depth of processing and the presence of recurrent processing, respectively.
Illustrations are taken from Dehaene et al. (2006).
Fahrenfort et al. (2017) used a classifier to see whether it was possible to distinguish between trials with an illusion and a control image based on EEG signals. When they
decoded the Kanizsa illusions, they found three classification peaks where the accuracy of the classifier was significantly above chance. The first peak was found in all conditions and is thought to represent the feedforward sweep. The second peak was seemingly reduced only by masking and was of similar height for the short (AB) and the long lag, which is why it may represent local recurrent processing. Finally, the third peak was reduced in masking and the
19/06/20
6
AB condition. In combination with the finding of a reduced behavioral accuracy in the AB condition compared to the long lag condition, it is plausible that the third peak represents global recurrent processing, or as one might say access consciousness.
However, one important limitation to the experiment of Fahrenfort et al. (2017) is that the masking condition had a so-called floor effect (behavioral performance was at chance level), while this was not the case for the AB condition, where performance accuracy was much higher. This makes it uncertain whether recurrent processing was interfered by perceptual blindness, for it could also have been the result of low performance accuracy. The current experiment is set up to replicate and adjust the experiment of Fahrenfort et al. (2017) by adding performance matching. The performance on the masking condition will be matched to the performance on AB condition (see Procedure for more information).
Moreover, Fahrenfort et al. (2017) only used on-diagonal decoding for their analysis, leaving out possible off-diagonal decoding patterns. To clarify, on-diagonal decoding is when the training time of the classifier is the same as the testing time, while with off-diagonal
decoding training and testing time are not the same (King and Dehaene, 2014). Weaver et al. (2018) argued that off-diagonal decoding patterns may represent information that is kept active, which could indicate the presence of recurrent processing. Thus, both the addition of performance matching and the use of an additional type of decoding could give more insight in the (true) effects of perceptual and attentional blindness on recurrent processing.
Another addition that will be done to the experiment of Fahrenfort et al. (2017) is metacognition. There are studies that used very similar manipulations as Fahrenfort et al. (2017) but used metacognition instead of decoding EEG signals as measurement to study the effects of perceptual and attentional blindness (Kanai et al., 2010; Meuwese et al., 2014; Kellij et al., 2018). To elaborate a bit more on metacognition, it is a term that covers knowing what you know or being aware of your awareness. Metacognition can be measured, among other things, as confidence that participants have about their performance. To get a good measure of metacognition, the current experiment will use the signal detection theory (SDT; Macmillan and Creelman, 2004) in combination with the subjective discriminability of
invisibility (SDI; introduced by Kanai et al., 2010). In the SDT there is a Type I and a Type II response. The objective performance of participants is a Type I response that contains two factors: the stimulus (absent or present) and the response (choosing between absent or present). It can be categorized into to four types: hits (correct answer, stimulus present), correct rejections (correct answer, stimulus absent), misses (incorrect answer, stimulus present) and false alarms (incorrect answer, stimulus absent). When asking participants how confident they are about their answer, it produces a (subjective) Type II response that can also be divided into four similar types (see Table 1). This type of response is based on whether the participant’s confidence matches the performance (correct or incorrect
answer). The limitation of using the type II response as a measurement is that no distinction is made within the correct and incorrect (type I) responses.
However, this distinction is what Kanai et al. (2010) needed for their experiment in which they intended to distinguish between attentional and perceptual blindness with
19/06/20
7
metacognition. They were interested in how participants adjusted their confidence on missed trials, trials that lack awareness, versus correct rejections, trials without a target. They expected that participants would not be able to adjust their confidence on misses induced by perceptual blindness but would be able to do so on misses caused by attentional blindness. The latter because participants might be aware that they did not have full
attention on the target. Therefore, their predictions were to find a difference in confidence between misses and correct rejections for attentional blindness, but not for perceptual blindness. To compare the confidence ratings on the two trial types, Kanai et al. (2010) created the new index SDI, which is a measure to quantify how well participants can adjust their confidence during impaired awareness. It is a type II response, only specifically focused on misses and correct rejections (see Table 2). The SDI and the decoding accuracy of the illusion (Fahrenfort et al., 2017) are quite similar in the way they are affected by attentional and perceptual blindness. Hence, Kanai et al. (2010) expected the SDI to be above chance level for attentional blindness but drop to chance level for perceptual blindness (as with the second peak in the classifier accuracy). The results of their experiment were in line with their predictions, as they found that the SDI was above chance level only for attentional blindness and not for perceptual blindness.
Table 1. Type II response according to the signal detection theory. This type of response is based on whether the performance matches the confidence of the participant and, unlike the Type I response, it is independent of the presence of the stimulus.
High confidence Low confidence
Correct Type I response Type II hit Type II miss
Incorrect Type I response Type II false alarm Type II correct rejection
Table 2. Subjective discriminability of invisibility (SDI) responses. SDI is a measure to quantify how well participants are able to adjust their confidence during impaired awareness. It is type II response where only misses and correct rejections are considered.
High confidence Low confidence
Correct rejection Type I Type II hit Type II miss
Miss Type I Type II false alarm Type II correct rejection
In this current experiment both metacognition and decoding EEG signals will be combined, to make it possible to compare the two measurements under the same conditions. To sum up, by adding performance matching and confidence ratings to the experiment of Fahrenfort et al. (2017), both (local and global) recurrent processing and metacognition can be further investigated. Sequentially, the research question for this current experiment is: how are recurrent processing and metacognition affected by attentional and perceptual blindness?
19/06/20
8 Hypotheses
In order to find an answer to the research question, the following hypotheses have been formulated:
• H1: Perceptual blindness (masking) interferes with local and global recurrent processing and performance, but leaves the feedforward sweep intact.
o H1.1 Behavioral accuracy will be reduced in the masked condition compared to the unmasked condition, but due to performance matching it will be the same as in the (unmasked) AB condition.
o H1.2 The first decoding accuracy peak will be found for the masked and unmasked condition and will not differ from other conditions.
o H1.3 Despite the performance matching the second peak will be reduced in the masked condition compared to the unmasked and the (unmasked) AB condition.
o H1.4 Despite the performance matching the third peak will be reduced in the masked condition compared to the unmasked condition but will be similar to the (unmasked) AB condition.
o H1.5 Off-diagonal decoding around the second peak will be reduced for the masked condition compared to the unmasked condition and the (unmasked) AB condition. Around the third peak it will be reduced compared to the unmasked condition but will be similar to the (unmasked) AB condition. • H2: Attentional blindness (attentional blink) interferes with global recurrent
processing and performance. Local recurrent processing and feedforward sweep will be intact.
o H2.1 Behavioral accuracy will be reduced in the AB condition compared to the long lag condition, but due to performance matching it will be the same as in the masked condition
o H2.2 The first decoding accuracy peak will be found for both the long lag and the AB condition and will not differ from other conditions.
o H2.3 The second peak will not be affected by AB.
o H2.4 The third peak will be reduced in the AB condition compared to the long lag condition but will be same as for the masked condition.
o H2.5 Off-diagonal decoding around the second peak will be the same for the long lag and AB condition, while it will be reduced for AB condition compared to the long lag condition.
• H3: Metacognition can be used to distinguish between perceptual and attentional blindness.
o H3.1 The SDI will be above chance level for the AB condition, but not for the masking condition.
19/06/20
9 B. Methods Description of essential elements
Design
List, based on your hypotheses from section A: 1. Independent variables with all their levels
a. whether they are within- or between-participants
b. the relationship between them (e.g., orthogonal, nested). 2. Dependent variables, or variables in a correlational design
3. Third variables acting as covariates or moderators. Planned sample
4. If applicable, describe pre-selection rules.
5. Indicate where, from whom and how the data will be collected.
6. Justify planned sample size (if applicable, you can upload a file related to your power analysis here (e.g., a protocol of power analyses from G*Power, a script, a
screenshot, etc.).
7. Describe data collection termination rule. Exclusion criteria
8. Describe anticipated specific data exclusion criteria. For example: a. missing, erroneous, or overly consistent responses;
b. failing check-tests or suspicion probes; c. demographic exclusions;
d. data-based outlier criteria;
e. method-based outlier criteria (e.g. too short or long response times). Procedure
1. (Recommended element, in the online form see next page) Set fail-safe levels of exclusion at which the whole study needs to be stopped, altered, and restarted. If applicable, you can upload any files related to your methods and procedure here (e.g., a paper describing a scale you are using, experimenter instructions, etc.). 2. Describe all manipulations, measures, materials and procedures including the order
of presentation and the method of randomization and blinding (e.g., single or double blind), as in a published Methods section.
Method
Design
The current experiment has a within-participants factorial design with eight factors (also known as the independent variables), each of which has two levels creating 28 = 256 different conditions. The factors can be divided into the factors of the first target (T1) and the second target (T2). To elaborate a bit more on this, T1 and T2 are terms used in an AB paradigm where T2 follows T1 after a specific amount of time. The T1 factors are illusion presence, contrast and collinearity presence, and the T2 factors are lag length, mask presence and the same factors as T1.
19/06/20
10
1. Lag length; the length between two targets will be manipulated in such a way that T2 is presented after T1 with either a long or a short lag.
2. Mask presence; T2 will be either masked or unmasked.
3. Illusion presence; the targets will either contain a Kanizsa illusion or a control image, both are formed with Pacman-like shapes (see Figure 2). The illusion will be used to investigate recurrent processing.
4. Contrast; the targets will have high or low contrast, determined by the orientation of the target (see Figure 2). High contrast means there are two black shapes at the top and low contrast is when only one black shape is at the top. The contrast of the targets will be used to investigate the feedforward sweep.
5. Collinearity presence; the targets will either contain lines that run collinearly, displayed as a non-illusional triangle, or not. Even though this factor is part of the experimental design, it will not be part of the analysis of this experiment.
The dependent variables can be divided in two behavioral variables, performance accuracy and confidence ratings, and the EEG measurements. The performance accuracy is based on how well participants are able to distinguish the illusion from the non-illusion targets, whereas the confidence ratings show how confident participants are about their response (illusion/no illusion).
Figure 2. Appearance of the target. The target has eight different appearances. Besides the illusion versus no illusion, the targets vary in contrast and collinearity.
Planned Sample
For this experiment 30 participants will partake in two sessions. Since the effect size of this experiment was difficult to establish, the sample size is based on previous literature instead (Fahrenfort et al. 2017; Weaver et al., 2018; Kellij et al. 2018). Data collection will be
terminated when 30 participants completed both sessions (see Exclusion criteria).
Participants will be recruited via the lab.uva.nl recruitment website and as compensation for their participation they can choose to receive money (10 euro per hour) or research credits
19/06/20
11
(1 credit per hour). A few pre-selection rules apply for participation in this experiment. Participants must have a normal or corrected to normal vision and must be between 16 and 35 years old. Moreover, given the procedure of this experiment, people with epilepsy will not be able to participate. It is also not possible for a person to participate when he already participated in one of the pilot studies or in a similar experiment with the same tasks and targets. The experiment will be conducted in a behavioral laboratory at the University of Amsterdam.
Exclusion criteria
Above all, participants should be able to complete both sessions of the experiment. Hence, incomplete datasets will be excluded from analyses. In addition, it is required that
participants are able to recognize the illusion when two targets are shown with a long lag (see Procedure for more information). Furthermore, participants are excluded when they do not have an AB or their performance accuracy on T1 is below 80% (see Procedure for more detailed information).
Procedure
The general design of the current experiment is based on Fahrenfort et al. (2017). For this experiment a set of eight different images will be used (see Figure 2). Half of them contains a Kanizsa illusion, the other half are Kanizsa control images. The latter are made up from the same elements as the Kanizsa illusion images, only by rotating the elements the illusion is no longer present. Two types of images will be used, target images (black) and distractor images (red). Distractor images have the same elements as target images, only the six elements have been rotated 180 degrees, see Figure 3A and 3B. Masks are created with random shapes, designed to interfere with every element of the target stimuli. Six different masks are made by rotating the mask five times, leaving the orientation and position of the elements other the same relative to each (see Figure 3C). All stimuli will be presented with Presentation (Neurobehavioral Systems) on a white background with a fixation point in the middle (see Figure 3D). The tasks of both sessions are programmed in Presentation and will be displayed on a 23” monitor with a resolution of 1920x1080 running at 100 Hz.
Participants will be seated approximately 75 cm from the screen in a fully lit room.
Behavioral data will be collected on a desktop computer and EEG data will be collected using a 64-channel ActiveTwo system (BioSemi) at 1024 Hz. In addition, two reference electrodes on the earlobes, one electrode on each temple and one electrode above and one below the eye will be used.
19/06/20
12
Figure 3. Example images that will be used during the experiment. (A) Two examples of a target stimulus. (B) Two examples of a distractor stimulus. Notice that the six elements are rotated 180 degrees relative to the targets. (C) Two examples of a mask. The orientation and position of the six elements remain the same relative to each other, while the mask rotates. The masks are designed to interfere with every element of the target stimuli. (D) The fixation point that will be used on every trial.
At the start of the experiment, participants will be asked to read an information brochure and after they sign an informed consent in which they agree to participate voluntarily they can start the experiment. Participants will partake in two sessions. A brief overview of the tasks that will be done during the sessions can be found in Table 3. The first session consists of a series of training tasks with increasing difficulty during which
participants will get familiarized with the stimuli and the experimental task. In addition, the latency between two targets that will induce an AB and a matching mask strength will be determined for each participant that will be used for the remaining of both sessions. Rapid serial visual presentation (RSVP) will be used for all tasks. Trials will consist of five red distractor images followed by one or two black target images and six distractor images after the last target. The stimulus onset asynchrony (SOA) will always be 100 ms. Participants will be asked to look at a fixation point in the middle of the screen and at the end of each trial, they will have to answer whether they saw an illusion (yes/no) by pressing “A” or “L” on the keyboard. In order to counterbalance for muscle movement in the EEG signal that will be recorded during the experimental task, the button meanings are switched between tasks or blocks. The first two tasks of the first session will only have one target (T1), which will first be presented for 40 ms, but later only for 10 ms. After this, a second target (T2) will be introduced and from now on, T1 will last for 40 ms and T2 for 10 ms. At the end of each trial the participants will have to answer the same question as before, now for T2 as well as T1, in this explicit order. The latency between T1 and T2 at this point is called a long lag since it will not induce an AB and is meant to check whether the participants are able to correctly
19/06/20
13
Table 3. Brief overview of tasks during session one and two.
Task Session 1 Session 2
1 One target 40 ms One target 40 ms
2 One target 10 ms One target 10 ms
3 Two targets, long lag Two targets, long lag
4 Two targets, short and long lag Two targets, confidence ratings
5 Two targets, find AB latency Experimental task, with staircase and EEG 6 Two targets, short and long lag, mask
7 Two targets, mask staircase 8 Two targets, confidence ratings
The next task is designed to determine which latency will induce the largest AB for each participant. During this task the participants will be exposed to two possible latencies, 200 ms (one distractor) or 300 ms (two distractors). The latency with the lowest
performance rate will be the short lag that will be used for the remaining part of the experiment. The long lag is the same for every participant, which will be 900 ms (8 distractors). The actual AB effect of a participant will be calculated by subtracting the accuracy on short lag trials from the long lag trials. When this is below 5%, it is considered that a participant has no AB and will therefore be excluded from further participation in this experiment. In the next part of the session, masks will be introduced to the tasks, creating four conditions (see Table 4). Each time a stimulus is masked, three different masks will be randomly picked and presented for 10 ms each with 10 ms between two masks. In addition, the performance on long lag masked (L1M2) trials will be matched with the performance on the short lag unmasked (L2M1) trials. This is one of the most important parts of the current experiment, for it is meant to avoid a floor effect of masking (Fahrenfort et al., 2017) during the experimental task. Since research showed that for the age group that will participate in this experiment the performance during an AB task will be well above chance level (~ 65%) on most occasions (Georgiou-Karistianis et al., 2007), the performance accuracy on L1M2 trials will be increased to the level of the L2M1 trials. This will be done with a staircase that changes the mask strength until a fitting strength is found (see below for more information about staircases used in this experiment). The mask strength that follows from this staircase, will be the (starting) mask during the second session and for the last task of the first session.
Table 4. Overview of the conditions of the experimental task.
Lag length Mask
presence
Long lag, unmasked (L1M1) Short lag, unmasked (L2M1) Long lag, masked (L1M2) Short lag, masked (L2M2)
After that, the last task of the first session will be a short version of the experimental task (see Figure 4 for a flowchart of the experimental task). The only difference is that the experimental task will have another staircase, with the purpose to check whether the
19/06/20
14
current mask strength is still accurate considering the performance on the L1M2 and L2M1 trials (see Staircases for more information). The last addition to the task will be confidence ratings, which is based on the experiments of Kanai et al. (2010) and Kellij et al. (2018). Participants will be asked about how confident they are about their answer (illusion? yes/no) at the end of each trial. The confidence ratings are asked simultaneously with the question of whether the target contained an illusion (see Figure 4). So together with answering ‘yes’ or ‘no’ (Type I response), the participant will have to rate his confidence on a scale of 1 to 3 about his answer (Type II response), resulting in six possible answers (sure/moderate
sure/unsure x yes/no). The buttons that will be used to answer are “A”, “S” and “D”, and “J”, “K” and “L”. The outer buttons represent the most confidence and the inner buttons the least confidence. Button meanings (left and right for yes and no) will switch after two blocks to counterbalance for muscle movement in the EEG signal.
Since the main interest of this experiment is T2, confidence rating will only be asked for T2. Also, whereas the visibility of T2 is manipulated, the intention is that T1 is fully visible. When confidence is asked for both targets together it is likely that participants will apply binary ratings (high for T1 and low for T2) instead of desired scaled ratings. To further prevent negative influence on the scaled confidence rating, the conditions L1M1 and L2M2 will be separated from L1M2 and L2M1 by putting them in a different block (creating two block types). As with T1 and T2, the conditions L1M1 (fully visible T2) and L2M2 (fully invisible T2) could make confidence ratings binary. Moreover, participants are specifically instructed to spread their confidence ratings along the scale as equally as possible.
After completing session one without meeting one of the exclusion criteria, the participants will come to the lab on a different day for the second session. In order to be certain that participants are at their best before they do the experimental task, they start with four practice tasks (see Table 3). These tasks are the same as the ones from session one (see the descriptions above). At the start of session two the EEG equipment will be setup and the participants will be wired for the whole session, even though EEG will only be recorded during the experimental task. Before the participants start with the experimental task they will be instructed to remain as still as possible and only blink between trials. The experimental task consists of 8 blocks, alternating between the two previously explained blocks types (L1M2 + L2M1 and L1M1 + L2M2). Each block has 128 trials, which corresponds to the 256 conditions divided by the number of block types. It is chosen to let the block types alternate instead of first running all the blocks of one type and then the other to minimize the differences between the blocks. Participant’s performance could decrease due to tiredness or increase due to practice, which would also affect the masks that are used during both blocks. Between two blocks participants may take a break and continue when they feel ready.
This experiment does not contain blinding of any kind and randomization of the trials and stimuli is done automatically with an algorithm from Presentation.
19/06/20
15
Figure 4. Flowchart of the experimental task. As T1 and T2 are separated only by one distractor and T2 is followed by a strong mask, this is an example of a trial from the short lag masked (L2M2) condition. The mask presence and the number of distractors (also known as lag length) determine the condition of the trial.
Staircases
Two different staircases will be used during this experiment. The first staircase determines the mask strength that will match the performance on the L1M2 trials with the performance on the L2M1 trials. The algorithm that is used for this staircase is Sup * p = Sdown * (1 – p),
which is based on the weighted up-down method (Kaernbach, 1991). In this formula, p is the desired performance, which is the performance on the L2M1 condition in this experiment. Sup and Sdown are the step sizes of increasing and decreasing the difficulty of the condition that needs to be matched respectively, which is the L1M2 condition in this experiment. For example, when the performance on L2M1 is 60%, the mask strength will be adjusted by either 3 or 2 steps, decreasing of increasing the mask strength respectively. The mask strength varies within a range from white to black with 256 levels of contrast. One step equals 2.56 levels of contrast (1%), which is rounded up or down after the necessary
multiplication. The staircase will be stopped after 25 reversals and the final mask strength is calculated by taking the average of the last 20 reversals.
The second staircase is a check that runs while the participant is performing the experimental task. It continuously checks the performance on the trials L1M2 and L2M1 and adjusts the mask strength when there is a difference between the two performance rates. For every percentage that the two rates differ, the mask strength will be adjusted by 1.5 level of contrast. For example, when the performance rate of L2M1 is 75% and L1M2 is 72%, the mask strength will be increased by 3 x 1.5 level of contrast. So, the mask strength will be lowered by 5 (4.5 rounded up). The performance rate calculations start when both
19/06/20
16
trials of L1M2 must have passed before the mask strength can be updated again. The staircase only runs during the block with the conditions whose performances are matched (L1M2 + L2M1). Therefore, all the different mask strengths that are used during the block of L1M2 + L2M1, will also be used, in random order, during the block of L1M1 + L2M2. This ensures that the effect of masking is as equal as possible for both masked conditions (L1M2 and L2M2), even though they are not part of the same block.
C. Analysis plan Confirmatory analyses
Describe the analyses that will test each main prediction from the hypotheses section. For
each one, include:
1. the relevant variables and how they are calculated; 2. the statistical technique;
3. each variable’s role in the technique (e.g., IV, DV, moderator, mediator, covariate); 4. rationale for each covariate used, if any;
5. if using techniques other than null hypothesis testing (for example, Bayesian
statistics), describe your criteria and inputs toward making an evidential conclusion, including prior values or distributions.
(the online form asks the above for the first, second, third, fourth and further predictions separately)
Recommended elements
Specify contingencies and assumptions, such as: 6. Method of correction for multiple tests.
7. The method of missing data handling (e.g., pairwise or listwise deletion, imputation, interpolation).
8. Reliability criteria for item inclusion in scale. 9. Anticipated data transformations.
10. Assumptions of analyses, and plans for alternative/corrected analyses if each assumption is violated.
Optionally, upload any files here that are related to your analyses (e.g., syntaxes, scripts, etc.).
Analysis plan
Behavioral data
For each participant the overall performance accuracy of T2 on the four different
experimental conditions (see Table 4) will be calculated. This will be done by dividing the number of correct trials by the total number of trials. Trials on which T1 was incorrect will be excluded from analyses, for it is likely that participants did not experience an AB on those trials. For the purpose of checking the effectiveness of the staircases, performance accuracy on long lag masked and short lag unmasked will be compared. When these conditions show similar results, performance matching was successful. Next, an ROC curve (known from SDT)
19/06/20
17
will be constructed for both type I and type II response. For type I hit rate will be plotted against false alarm rate. For type II, high-confidence correct rejection rate will be plotted against high-confidence miss rate, which is the equivalent of the hit against false alarm rate of a type I ROC curve. Then, performance accuracy will be calculated as area under the curve (AUC) of the type I ROC curve and the SDI will be quantified as AUC of the type II ROC curve. For more detailed information about SDT and corresponding calculations see the user guide of Macmillan and Creelman (2004). The performance accuracy as AUC can be compared between all four conditions with a two-way repeated measures (RM) ANOVA. Poshoc t-tests will be performed find which specific conditions differ significantly from each other. It is expected to find a main effect for both lag length and mask strength, which would account for H1.1 and H2.1. The SDI cannot be compared with an ANOVA, because only the conditions long lag unmasked, and short lag masked will have been performance matched. Hence, the SDI will be compared between those two conditions with a paired t-test, which will account for H3.1. A significant difference is expected between the two conditions.
EEG data
Preprocessing
The EEG preprocessing and data analyses will be performed in MATLAB (MathWorks) using the Fieldtrip toolbox (Oostenveld et al., 2011), the EEGLAB toolbox (Delorme and Makeig, 2004) and the Amsterdam Decoding and Modeling toolbox (ADAM; (Fahrenfort et al. 2018). The data will first be re-referenced to the average of the two reference electrodes on the earlobes, after which a high pass filter of 0.1 Hz will be applied. Next, the data will be epoched from -500 to 1000 ms relative to stimulus onset. Due to the AB element of the experimental task it is difficult to apply a baseline correction on the T2 data. Therefore, the high pass filter will serve as alternative. Trial rejection due to muscle or eye movement will be done using the ft_artifact_zvalue function of the Fieldtrip toolbox. It detects artifacts by applying a frequency filter between 110 and 140 Hz and assigning a z-value to each time point in order to determine the degree to which power values in that frequency range diverge from normality. A trial will be rejected when it contains z-score outliers with more than 3 standard deviations difference from the absolute value of the minimum negative z-value. In addition, independent component analysis (ICA) will be applied to remove any remaining artifacts from the signal (Makeig et al., 1996). Finally, data will be down sampled to 512 Hz.
Multivariate Pattern Analysis
For the analyses of the EEG data MVPA will be applied. The analysis consists of two levels. For the first level of the analysis, a linear discriminant classifier will be trained to
discriminate Kanizsa illusion and control images. Raw EEG activity across electrodes will be used as the features for the backward decoding classification algorithm, either from all or only the occipital electrodes. Fahrenfort et al. (2017) originally trained the classifier on an independent localizer task to remove task and response related processes from the signal.
19/06/20
18
But only when the classifier was trained on the T1 data, the third classifier accuracy peak (as discussed in the Theoretical background) became visible. An additional analysis with similar data was done to confirm the findings of Fahrenfort et al. (2017) and try out a different task for this purpose (see Supplemental information, p.22). The results showed that the classifier trained on the independent localizer task did not show the third peak. Since the third peak is important to this experiment, the T2 classifier will only be trained on the T1 data. For the T1 data an eightfold cross-validation scheme will be used to train the classifier, to avoid the need for an independent localizer task. For the eightfold cross-validation scheme trials will be shuffled, and the classifier will be trained on seven-eighth of the trials and tested on the remaining one-eighth. This will be repeated until all trials have been used for testing once. The training and testing trials will never be the same during one run. The remaining steps will be part of the second level of the analysis.
Since previous research showed that the signal found in this type of paradigm is primarily occipital in nature (Fahrenfort et al., 2017), only occipital electrodes will be used for the second level of the analysis of this experiment to optimize signal-to-noise ratio. Unfortunately, the additional analysis was not able to confirm the benefit of using occipital electrodes (see Supplemental information, p.22). So, to verify that the signal in this
experiment has the same neural sources as the signal of Fahrenfort et al. (2017), a normalized correlation/class separability map will be generated with the data of all electrodes through multiplying classifier weights by the data correlation matrix (following the procedure described by Haufe et al., 2014). During the next step in the analysis, the mean on-diagonal classification accuracy of all subjects will be computed as the area under the curve (AUC) for each condition: long lag masked, long lag unmasked, short lag masked, short lag unmasked, and T1. It will be executed for all time points in a trial with double-sided t-tests across subjects against a standard 50% chance level. In order to correct the t-tests for multiple comparisons over time, cluster-based permutation tests with 1,000 iterations (p < 0.05) will be applied. Then, classifier accuracy over time of each condition will be compared using the function adam_compare_MVPA_stats of the ADAM toolbox. It shows the time points at which two conditions significantly differ from each other. This way it possible to see whether the second and the third peak differ between the attentional and the
perceptual blindness condition. It is expected that short lag differs significantly from long lag (when both are unmasked) only in the third peak, while masked is expected to differ
significantly from unmasked (when both have a long lag) in both peaks, as described in the H1.3, H1.4, H2.3 and H2.4.
Thereafter, the peak accuracy of the classifier will be compared with a 2 x 2 x 2 RM-ANOVA with lag length (long/short), mask presence (absent/present) and peak number (second/third). Additional post hoc t-tests will be performed on each peak to find which specific conditions significantly differ from each other within a peak. The peak range in which the peaks will be selected is based on the peaks of T1 classification and the results of Fahrenfort et al. (2017). Because performance matching will be applied, the main interest is finding a difference between the condition in which only perceptual blindness will be
19/06/20
19
induced and the condition in which only attentional blindness will be induced. It is expected that global recurrent processing is affected by both manipulations, but local recurrent processing only by the attentional blindness. In other words, a significant difference in peak height of the second peak is expected between the long lag masked and the short lag
unmasked condition, but not in the peak height of the third peak. This would result in a main effect for mask strength, as it is expected that both peaks are affected by mask strength. Furthermore, no main effect is expected for lag length, because it is expected that only the third peak is affected by lag length. Therefore, an interaction between lag length and peak number is expected and not for mask presence and peak number. This would also result in an interaction between all three variables. A main effect is also expected for peak number, but this is because the second and third peak differ in height even without manipulation. Results from the three-way RM-ANOVA will account for H1.3, H1.4, H2.3 and H2.4.
During the first level of the analysis, the classifiers will have been trained at one time point and tested on activity at every other time point. This will be repeated until all time points have been used as training time. For the on-diagonal decoding as discussed above, the training and tested time will be the same, whereas for the next analysis all combinations of training and testing times will be used. The next analysis will be done to test the stability of neural representation over time, by making a temporal generalization matrix for the time range -500 to 1000 ms from stimulus onset (King and Dehaene, 2014). This matrix will show the classification accuracies for all combinations of training time points (y-axis) and testing time points (x-axis). Cluster-based permutation tests (1,000 iterations, p < 0.05) will be applied as correction and to find significant clusters within the matrix. With this matrix off-diagonal decoding patterns can be detected. When a horizontal off-off-diagonal decoding pattern is found, the average accuracy within the testing time window (x-axis) of these patterns will be taken over the training time window (y-axis). Off-diagonal patterns will further be analyzed the same way as the on-diagonal patterns. Since it is expected that recurrent processing is still present in the attentional and reduced in the perceptual
blindness condition, off-diagonal patterns are expected to be present in the both conditions but reduced in the latter. These finding will account for H1.5 and H2.5.
Next, the classifier will be trained to distinguish between high and low contrast (see Figure 2) using T1 as training set. Contrast is a stimulus feature and is processed as bottom-up information. Consequently, an early peak representing a feedforward information stream is expected. Considering the H1.2 and H2.2, the early feedforward peak is expected to be present in all four conditions with no significant difference between them. This will be tested by performing a 2 x 2 RM-ANOVA with lag length and mask presence as factors.
Finally, the behavioral and neural results will be compared with a 2 x 2 x 2 RM-
ANOVA with lag length (long/short), mask presence (absent/present) and measurement type (behavioral/neural). This will be done for the second and the third peak separately. For the second peak the same interactions and main effects are expected as for the three-way ANOVA with peak number as third factor. An interaction between lag length and
19/06/20
20
lag length, but behavioral accuracy is not. Also, a main effect is expected for mask strength, as both behavior and the second peak are expected to be reduced. For the third peak on the other hand, a main effect for mask strength and lag length is expected, since both the third peak and the behavioral accuracy are expected to be affected by the two manipulations. For the same reason, no interaction is expected between measurement type and lag length or measurement type and mask strength. If these expectations are found to be true, it would suggest that the neural and behavioral data patterns are very similar at the time of the third but not of the second peak. It could support the idea that the second peak represents local recurrent processing and the third peak global recurrent processing and possibly access consciousness.
Answer the following final questions: Has data collection begun for this project?
• No, data collection has not begun
o Yes, data collection is underway or complete If data collection has begun, have you looked at the data?
o Yes o No
The (estimated) start and end dates for this project are (optional): Any additional comments before I pre-register this project (optional):
References
Block, N. (2005). Two neural correlates of consciousness. Trends in cognitive sciences, 9(2), 46-52.
Dehaene, S., Changeux, J. P., Naccache, L., Sackur, J., & Sergent, C. (2006). Conscious, preconscious, and subliminal processing: a testable taxonomy. Trends in cognitive sciences,
10(5), 204-211.
Dehaene, S., & Changeux, J. P. (2011). Experimental and theoretical approaches to conscious processing. Neuron, 70(2), 200-227.
Del Cul, A., Baillet, S., & Dehaene, S. (2007). Brain dynamics underlying the nonlinear threshold for access to consciousness. PLoS biology, 5(10).
Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of neuroscience methods,
19/06/20
21
Fahrenfort, J. J., Van Leeuwen, J., Olivers, C. N., & Hogendoorn, H. (2017). Perceptual integration without conscious access. Proceedings of the National Academy of Sciences,
114(14), 3744-3749.
Fahrenfort, J. J., Van Driel, J., Van Gaal, S., & Olivers, C. N. (2018). From ERPs to MVPA using the Amsterdam decoding and modeling toolbox (ADAM). Frontiers in neuroscience, 12, 368.
Georgiou-Karistianis, N., Tang, J., Vardy, Y., Sheppard, D., Evans, N., Wilson, M., ... & Bradshaw, J. (2007). Progressive age-related changes in the attentional blink paradigm.
Aging, Neuropsychology, and Cognition, 14(3), 213-226.
Haufe, S., Meinecke, F., Görgen, K., Dähne, S., Haynes, J. D., Blankertz, B., & Bießmann, F. (2014). On the interpretation of weight vectors of linear models in multivariate
neuroimaging. Neuroimage, 87, 96-110.
Kaernbach, C. (1991). Simple adaptive testing with the weighted up-down method.
Perception & psychophysics, 49(3), 227-229.
Kanai, R., Walsh, V., & Tseng, C. H. (2010). Subjective discriminability of invisibility: a framework for distinguishing perceptual and attentional failures of awareness.
Consciousness and cognition, 19(4), 1045-1057.
Kellij, S., Fahrenfort, J., Lau, H., Peters, M. A., & Odegaard, B. (2018). The foundations of introspective access: how the relative precision of target encoding influences metacognitive performance.
King, J. R., & Dehaene, S. (2014). Characterizing the dynamics of mental representations: the temporal generalization method. Trends in cognitive sciences, 18(4), 203-210.
Kitagawa, N., & Sakurai, M. (2016). Memantine‐induced sustained unconsciousness. Neurology and Clinical Neuroscience, 4(6), 236-238.
Kok, P., Bains, L. J., van Mourik, T., Norris, D. G., & de Lange, F. P. (2016). Selective activation of the deep layers of the human primary visual cortex by top-down feedback. Current
Biology, 26(3), 371-376.
Lamme, V. A., & Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences, 23(11), 571-579.
Lamme, V. A. (2003). Why visual attention and awareness are different. Trends in cognitive
19/06/20
22
Lamme, V. A. (2006). Towards a true neural stance on consciousness. Trends in cognitive
sciences, 10(11), 494-501.
Macmillan, N. A., & Creelman, C. D. (2004). Detection theory: A user's guide. Psychology press.
Makeig, S., Bell, A. J., Jung, T. P., & Sejnowski, T. J. (1996). Independent component analysis of electroencephalographic data. In Advances in neural information processing systems (pp. 145-151).
Meuwese, J. D., van Loon, A. M., Lamme, V. A., & Fahrenfort, J. J. (2014). The subjective experience of object recognition: comparing metacognition for object detection and object categorization. Attention, Perception, & Psychophysics, 76(4), 1057-1068.
Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J. M. (2011). FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data.
Computational intelligence and neuroscience, 2011.
Weaver, M. D., Fahrenfort, J. J., Belopolsky, A., & Van Gaal, S. (2019). Independent neural activity patterns for sensory-and confidence-based information maintenance during category-selective visual processing. Eneuro, 6(1).
19/06/20
23
Supplemental information: Data-analysis of MABKIM* study
*Short for all the elements of the experiment: Masking, Attentional blink, Kanizsa Illusion and Memantine
The following analysis has been performed to confirm the findings of Fahrenfort et al. (2017) and therewith support the first two hypotheses of the pre-registration. In addition, it was done to test the analysis that will be performed on the data from the experiment of the pre-registration. The analysis has been performed on data of a different study. This study is done by Samuel Noorman and focuses on the effects of memantine (an NMDA-receptor
antagonist) on consciousness, using a very similar paradigm as the one used by Fahrenfort et al. (2017). This makes it suitable for an exploratory analysis that should confirm their
findings. For Noorman’s study, participants did the same session twice, once with and once without taking memantine, but for the sake of interest of this additional analysis, data of both sessions have been combined. Given the hypothesis that memantine interferes with consciousness and that only six participants participated in the experiment so far, the results may deviate from expectations (see discussion for more details).
Analysis plan
The analysis plan corresponds to a large extent with the analysis plan of the pre-registration, except for the metacognition related analyses, which will not be part of this analysis plan. Only deviating steps will be explained here. For more detail, please read the analysis plan in the pre-registration.
Behavioral data
The analysis of the behavioral data is the same as in the pre-registration. The only two differences are that behavioral accuracy is not calculated as area under the curve (AUC) of an ROC curve and the long lag masked, and short lag unmasked conditions are not checked for the effectiveness of the staircases, as there were no staircases.
EEG data
Preprocessing
Preprocessing has been done according to the description in the pre-registration, with three exceptions. Trials with muscle movements are not rejected, trials are not corrected for eye movements and the data is down sampled to 256Hz instead of 512Hz. This was done to make the data-analysis less time consuming.
Multivariate Pattern Analysis
The same analyses have been conducted as described in the pre-registration, with two exceptions. In addition to using T1 data, data from an independent localizer task was used to train the classifier. This was done to replicate the findings of Fahrenfort et al. (2017), and at the same time try out a different task as training task. Where Fahrenfort et al. (2017) used a 1-back task, for this analysis a template matching task will be used, where participants had
19/06/20
24
to answer whether a presented template matched the target that was shown at the
beginning of the block. For the data of this additional classifier, a two-way repeated measure (RM) ANOVA was needed instead of the three-way RM-ANOVA described the analysis plan of the pre-registration.
Results
Behavioral data
Behavioral accuracies on T2 for all four conditions (lag length x mask strength) of each individual participant were calculated, after which the mean accuracy was calculated and plotted in Figure 1. To test whether the four conditions differed significantly, a two-way repeated-measure (RM) ANOVA was performed on the behavioral data. A main effect was found for both lag length (F(1,5) = 99.93, p = 0.0002) and mask strength (F(1,5) = 72.65, p =
0.0004). Interestingly, post-hoc t-tests only showed a significant effect for lag length in the
unmasked condition (t(10) = 2.59, p = 0.027) and not the masked condition (t(10) = -0.33, p =
0.746). So, only for the unmasked condition it can be concluded that participants had an
attentional blink. The effect of mask strength is significant in both the long and short lag condition (both t(10) > 5.89, p < 0.0002), which means that performance was lowered in both
cases. Moreover, an interaction between the two factors was found (F(1,5) = 37.46, p =
0.002), which is expected when taking the post-hoc t-tests in account.
Figure 1. T2 behavioral accuracy for all four conditions. Participants had an attentional blink and a reduced behavioral accuracy due to masking. Error bars are mean ± SEM. ns, not significant (p > 0.05). *p < 0.05, ***p < 0.001, *******p < 10–7.
EEG data
At first, the classifier was trained on the independent localizer task and tested on T1 and T2. When either all or only the occipital electrodes were used, the classifier was able to decode the illusion for T1 (see Figure 2A). Interestingly, the classifier seemed to be more accurate for only occipital than when all electrodes were used, as the classification accuracy was
19/06/20
25
higher and significant over a longer time period. The same was true for T2 classification, as the classifier was more successful when only the occipital electrodes were used. Whereas with T2 classification on all electrodes, only the unmasked long lag condition showed significant classification accuracies, with only the occipital electrodes, both unmasked long and short lag had significant classification accuracies (see Figure 2B). The only unexpected result was the significant time period in the short lag masked condition. The peak that was found around 232 ms for the unmasked conditions is referred to as the second classification peak in the pre-registration. A two-way RM-ANOVA showed a main effect for mask strength (F(1,5) = 33.93, p = 0.002), but not for lag length (F(1,5) = 5.71, p = 0.062) and no interaction (F(1,5) = 3.98, p = 0.103) was found. Both main effects confirm the expectations, since the second peak was expected to be affected by mask strength, but not by lag length. These results confirmed the findings of Fahrenfort et al. (2017) and showed that by using the classifier trained on the independent localizer task, the so called “third peak” cannot be found. Since the experiment in the pre-registration focusses on both peaks, the remaining analysis was done with the classifier trained on T1 and tested on T1 and T2.
Figure 2. Classification accuracy of the illusion for the occipital electrodes, trained on the independent localizer task. The thickened line shows the time frame in which the classification accuracy was significant after cluster-based correction. (A) T1 classification shows one peak around 200 ms after stimulus onset. (B) T2 classification for the four conditions. Unmasked long and short lag conditions have significant classification accuracies in a peak around 232 ms. Short lag masked also shows a small significant time window around 115 ms.
When the classifier was trained on T1 data, the illusion could be decoded for T1 as well as for T2. In Figure 3A the classification accuracy is visible for T1 classification and in Figure 3B the classification accuracy of all four conditions of T2. Classification accuracy of T1 showed two peaks. The peak that has been referred to as the second peak in the
pre-registration around 177 ms and the peak referred to as the third peak around 357 ms after stimulus onset. The illusion in T2 could be decoded when trials were unmasked and either had long or short lag. The same two significant classification accuracy peaks were found for
19/06/20
26
these two conditions. The second peak was around 230 ms and the third peak around 406 ms after stimulus onset. The mean peak height for all four conditions of the second peak is shown in Figure 4A and the third peak in Figure 4B. After conducting a three-way RM-ANOVA (lag length x mask strength x peak number), a main effect was found for lag length (F(1,5) = 14.66, p = 0.012), mask strength (F(1,5) = 28.82, p = 0.003) and peak number (F(1,5)
= 17.74, p = 0.008). The main effect of peak number was expected but irrelevant, because it
is evident that the two peaks differ from each other. The main effect of mask strength was also expected and confirms that masking affects both peaks. However, the main effect of lag length was not expected, for lag length was expected to affect only the third peak. For this same reason, an interaction was expected between lag length and peak number, but no interaction was found between any of the variables (mask strength x peak number F(1,5) =
5.30, p = 0.070, remaining interactions F(1,5) < 0.86, p > 0.395). The interaction between all
three variables was not expected but given the other results it is unsurprising. Post hoc t-tests resulted in somewhat contradicting findings, as no significant effect was found for either peaks for lag length (second peak for both mask strengths t(10) < 1.33, p > 0.214; third
peak both t(10) < 0.72, p > 0.489). On the other hand, a significant effect was found for both
peaks for mask strength (second peak for both lag lengths t(10) > 4.96, p < 0.0006; third peak
both t(10) > 3.45, p < 0.006), confirming the ANOVA results.
Figure 3. Classification accuracy of the illusion for all electrodes. The thickened line shows the time frame in which the classification accuracy was significant after cluster-based correction. (A) T1 classification when eightfold cross-validation scheme was used. Two peaks are visible, they are referred to as the second and third peak in the pre-registration. The second peak is around 177 ms and the third around 357 ms (B) T2 classification for the four conditions trained on T1. Unmasked long and short lag conditions have significant classification accuracies where the both peaks are also visible; the second peak around 230 ms and the third around 406 ms.
19/06/20
27
Figure 4. Peak classification accuracy of the illusion for all four conditions when all electrodes are used. (A) Peak accuracy of the second peak around 230 ms. (B) Peak accuracy of the third peak around 406 ms. Error bars are mean ± SEM. ns, not significant (p > 0.05). **p < 0.01, ***p < 0.001.
In addition to the ANOVA tests, the classifier accuracies of the four conditions were compared between themselves, within lag length or mask strength, to see where in time the significant differences were located. The results are shown in Figure 5A. As the post hoc t-tests showed, only the factor mask strength differed significantly. In Figure 5A it is visible that the second peak differs significantly only in the short lag condition and the third peak only in the long lag condition. Since it was expected that lag length and mask strength would have the same effect on the third peak and not on the second, the absence of significant difference in the second in the long lag condition (lower left plot) was unexpected. The expected difference between short and long lag in the third peak of the unmasked condition (upper left plot) was not found either.
Figure 5. Classification accuracy differences when the four conditions are compared between themselves either within lag length or mask strength. The thickened line shows the time frame in which the classification accuracy was significant after cluster-based correction. (A) Differences when classifier was trained on all electrodes. (B) Differences when classifier was trained on occipital electrodes.
19/06/20
28
Thereafter, the same analyses were done again but with just the occipital electrodes. T1 classification accuracy still resulted in two significant peaks, the second around 177 ms and the third around 396 ms after stimulus onset. T2 classification still showed significant accuracies for the third peak around 451 ms, but no longer for the second peak around 224 ms after stimulus onset (see Figure 6). The three-way RM-ANOVA that was used for the data of all electrodes was also applied to the data with just the occipital electrodes. This time only a main effect for mask strength was found (F(1,5) = 42.98, p = 0.001) and not for lag length and peak number (both F(1,5) < 3.69, p > 0.112). The first two findings are in line with the expectations and the third is not relevant for this analysis. Unfortunately, no interaction was found between peak number and lag length, peak number and mask strength, or between all three variables (these three interactions F(1,5) < 0.58, p > 0.480). The one interaction that was found was between lag length and mask strength (F(1,5) = 8.08, p = 0.036). Poshoc t-tests showed that lag length does not differ significantly for either the second or third peak (both masked and unmasked t(10) < 1.71, p > 0.119), suggesting that the absence of a main
effect was due to the fact that no peak was affected by lag length, instead of the expected third peak. The effect of mask strength did get confirmed, as it differs significantly for both lag lengths and both peaks (in all four cases t(10) > 3.77, p < 0.004). All mean peak
classification accuracies with their significance are shown in Figure 7.
Figure 6. Classification accuracy of the illusion for the occipital electrodes only. The thickened line shows the time frame in which the classification accuracy was significant after cluster-based
correction. (A) T1 classification when eightfold cross-validation scheme was used. The two peaks are visible. The second peak is around 177 ms and the third around 396 ms (B) T2 classification for the four conditions when classifier was trained on T1. Unmasked short and long lag have significant classification accuracies where only the third peak is visible around 451 ms. The second peak around 224 ms was not significant.
19/06/20
29
Figure 7. Peak classification accuracy of the illusion for all four conditions with the occipital
electrodes only. (A) Peak accuracies of the second peak around 224 ms. (B) Peak accuracy of the third peak around 451 ms. Error bars are mean ± SEM. ns, not significant (p > 0.05). **p < 0.01, ***p < 0.001.
Next, the two factors were compared within themselves to see where in time the conditions differed significantly, as was done with the data of all electrodes. The results of this test are shown in Figure 5B. In accordance with the ANOVA results, only the masked factor was found to be significant and only the second peak is significant when the lag is either long or short. As with the data from all electrodes, the expected difference between short and long lag in the third peak of the unmasked condition (upper left plot) was not found. Moreover, the peak that is significantly different between masked and unmasked for the long lag condition (lower left plot) is not the same as when all electrodes were used. This time only the second peak was significant.
In the next step behavioral data was compared with the neural data of each peak by computing a three-way RM-ANOVA, to see whether these two measurements are affected in a similar way by both manipulations. This was done only on the data where the classifier was trained on the occipital electrodes, because this is what will be done in the pre-registration. For the second peak a main effect was found for lag length and mask strength (F(1,5) >
54.41, p < 0.0007) and for measurement type (F(1,5) = 46.84, p = 0.001). As with the peak
numbers, the main effect for measurement is irrelevant and evident, because the overall accuracy is higher for behavioral data compared to neural data. Furthermore, whereas a main effect of mask strength was expected, the main effect of lag length was not. These findings suggest that mask strength and lag length affect the two measurements in the same way, while this was not expected for the second peak. For this same reason, an interaction between lag length and measurement type was expected, but was not found (F(1,5) = 3.85,
p = 0.107), while the interaction between mask strength and measurement type was not
expected, but found (F(1,5) = 46.34, p = 0.001). Furthermore, an interaction was found between mask strength and lag length, and between all three variables (F(1,5) > 15.51, p <
19/06/20
30
For the third peak a main effect was found again for lag length (F(1,5) = 39.49, p =
0.002), mask strength (F(1,5) = 61.70, p = 0.0005) and measurement type (F(1,5) = 41.71, p = 0.001). The first two were as expected because the third peak and the behavioral data were
expected to be affected in the same way by both lag length and mask strength. On the other hand, an interaction between measurement type and lag length, and between measurement type and mask strength were found (both F(1,5) > 47.88, p < 0.0009). These findings were not expected, as an interaction usually suggests that two factors depend on each other and it was expected that the effect of mask strength and lag length were independent of
measurement type when comparing the third peak. In other words, it was expected that the manipulations would affect behavioral accuracy in a similar way as the third peak. Lastly, an interaction between mask strength and lag length, and between all three variables was found (both F(1,5) > 15.09, p < 0.012).
Next, the neural stability of the illusion was checked with a temporal generalization matrix. This was done for both only the occipital and all electrodes, but only for the
unmasked trials. Since the masked trials showed no significant on-diagonal classification, it would be very unlikely to find significant off-diagonal classification. Most of the significant classification was found on-diagonal, but some off-diagonal classification was found as well (see Figure 8). The area is wide and a bit shewed to the right (not just a thin diagonal line). Moreover, a hint of significant classification was found when training time was around 750 ms after stimulus onset and testing time around 250 ms. Nonetheless, the amount of off-diagonal classification was too minimal for further analyses. Besides, it was not found in the expected locations.
Figure 8. Temporal generalization matrices of unmasked trials divided in long lag and short lag trials, for both only the occipital (lower row) and all electrodes (upper row). The colors represent the classifier’s accuracy as area under curve (AUC) after cluster-based correction. Toward red is above chance level and toward blue is below change level. All four matrices show significant on-diagonal and small amounts of off-diagonal decoding.
