Improving the Signal Quality of fNIRS: An Exploratory Study

Page | 1

Improving the Signal Quality of fNIRS: An Exploratory

Study

Second corrector Hannie van Hooff

Abstract

Functional near infra-red spectroscopy (fNIRS) is among the newest neuro-imaging techniques in neuro-marketing. fNIRS uses the difference in oxygenated- (O2HB) and deoxygenated blood (HHB) in the cortex to map brain activity. This study investigated how the method of obtaining fNIRS data could be improved to gather reliable data for future studies. Interventions (two in total) were implemented between each of the three periods of scanning during which the subjects had to work through a paradigm while their brain activity was measured. This resulted in three groups that were compared with each other. The change in concentration of oxygenated and deoxygenated hemoglobin in a brain area is a decisive factor in establishing the signal quality of fNIRS. The first intervention consisted of a binary signal classifier and the second intervention consisted of the implementation of the signal quality index (SQI) and a real-time graph of the O2HB and HHB blood levels. Correlation differences were calculated based on the changes in concentration of oxygenated blood (O2HB) and deoxygenated blood (HHB). The correlation differences between the group where no intervention was implemented and the group where the SQI and a real-time graph of the O2HB and HHB blood levels were implemented after correlation differences were ranked in tertiles were found to be significantly contrasting. Future research could investigate different fNIRS channel configurations or the influence of specific extrinsic factors on the signal quality. Extrinsic factors such as skin tone, hair color and thickness of the scalp can make the usage of fNIRS difficult.

Page | 2

Introduction

Neuromarketing is an emerging field that connects consumer behavior with neuroscience. Over 400 billion dollars is spent annually in advertising campaigns worldwide (Morin, 2011). Conventional methods to predict the effectiveness of these financial investments have generally failed, due to dependence of these methods on the ability of consumers to describe their feelings when they are exposed to product packaging or advertisements. Neuro-imaging techniques are among the newest techniques to measure marketing stimuli and offer a method to probe minds without requiring conscious and active participation from the subject (Morin, 2011). When these techniques are used to understand or predict consumer behavior concerning marketing, these methods are called neuromarketing techniques (Lee, Broderick & Chamberlain, 2007). Research has shown that consumers find it difficult to express their emotions and preferences when asked (Meyerding & Mehlhose, 2020). The answers to these questions about emotions and preferences can help to develop new marketing theories or complement existing theories in marketing and adjourned disciplines (Lim, 2018). Neuromarketing offers to help marketers in using the right branding and packaging (Meyerding & Mehlhose, 2020). Research subjects’ brains are imaged with non-invasive techniques such as electroencephalography (EEG), functional near infra-red spectroscopy (fNIRS) or functional magnetic resonance imaging (fMRI), while they are presented with marketing and communication material and the responses are used to bypass subjective report.

fMRI is a brain imaging technique that uses the amount of oxygenated blood in a specific brain area or in the whole brain by using the blood-oxygen-level-dependent (BOLD) signal. This signal subsists on the difference in magnetization between oxygenated hemoglobin and

deoxygenated hemoglobin, respectively known as O2Hb and HHb (Thulborn et al. 1982). fMRI still has several shortcomings. fMRI has a poor temporal resolution and latency is about two seconds (de Munck et al., 2007). The signal-to-noise ratio and hemodynamic response are essential for the temporal resolution of fMRI. The signal-to-noise ratio is finite and the hemodynamic response in the human body cannot be accelerated. Both elements are needed to calculate the BOLD signal.

However, due to the limitations of these elements, obtaining a high temporal resolution is not possible (Kim et al. 1997). Moreover, fMRI is expensive, which makes it less attractive to use. Conducting fMRI scans also involves criteria for excluding participants to ensure their safety. For example, people with metal implants, tattoos that cannot be covered, pacemakers or pregnant women cannot participate and are therefore excluded from experiments where fMRI is used as a neuro-imaging technique (Poldrack et al., 2008). Furthermore, participants need to lay as still as possible during the entire experiment to avoid negatively impacting the data quality. This makes it impossible to conduct dynamic experiments with the participants, e.g., walking or moving the head. Because of all these limitations, Neurensics has been developing a multi-instrumental setup that combines measuring brain activity with fNIRS and EEG. In addition to this heart rate (three-lead electrocardiogram), galvanic skin response (GSR), and eye tracking (eye tracker) are also measured in the same set-up. Like MRI, fNIRS uses the difference in concentration of O2Hb and HHb in different brain areas. This difference in concentration can be linked to the amount of cerebral blood flow (CBF) in a specific brain area and can then be linked to activity in that brain area. O2HB and HHB have different absorption coefficients (Sassaroli and Fantini, 2004). fNIRS anticipates the different absorption coefficients that O2HB and HHB have by emitting infrared light photon beams that penetrate human tissue. The amount of absorption is detected by photodetectors. fNIRS is more cost-effective to implement compared to fMRI, has a better temporal resolution, and is less invasive. Temporal resolution is limited compared to EEG. Another - significant - benefit is that fNIRS can be

Page | 3 performed wirelessly, which makes it easier to perform research paradigms that are more dynamic (Wilcox & Biondi, 2015).

This study will investigate the improvement of the operational part and signal quality of fNIRS in the multi-instrumental setup that is being developed by Neurensics. Based on the above, fNIRS is considered a technique with a high potential for neuromarketing, but also comes with weaknesses. One of these weaknesses is that fNIRS is very sensitive to systemic artifacts. These artifacts occur because the infra-red light must travel through superficial (i.e., hair and skin) and cerebral layers (Kato et al., 1993). Intrinsic and extrinsic factors influence the absorption and scattering of the infra-red light that is emitted by the fNIRS optodes. Intrinsic factors are caused by hemodynamics in the cerebral and extracerebral layers (Scholkmann et al., 2014). The functional brain hemodynamics also play a role in these systemic artifacts. A systemic artifact that is present in both the extracerebral and cerebral layers is needed to see if enough infra-red light has reached the brain and that most of the absorption and scattering is caused by intrinsic factors (Huigen et al., 2002). Being able to see a heartbeat in the fNIRS signal is a good example that enough infra-red light has reached the brain (Sappia et al., 2020). Extrinsic factors limit the amount of light reaching the brain and thus decrease the signal quality. Examples of extrinsic factors are skin properties (e.g., color), hair properties (e.g., color and density) and scalp and skull thickness (Orihuela-Espina et al., 2010). All these factors must be considered when investigating solutions to improve the signal quality. This leads to the following research question: How can the method of obtaining fNIRS data be improved to gather reliable data for future studies? In this study, subjects will work through a paradigm while brain-activity is measured with fNIRS and EEG. The scanning of subjects will be done in different stages, to allow for improvements of the scanning method to be implemented between the measuring periods. It is hypothesized that the implementation of a real-time feedback

mechanism, i.e., an objective measure to classify the signal quality and practical improvements to the used protocol, will improve the signal quality and the classification of the strength of the signal quality. fNIRS data will be compared with each other when subjects conduct passive trials (watching tv commercials (TVCs)) and active trials (What’s the price (WTP)). It is assumed that active trials that include decision making will lead to more overall brain activity, especially in frontal areas of the brain (Bechara & van der Linder, 2005). Higher brain activity leads to a lower correlation between oxygenated blood (O2HB) and deoxygenated blood (HHB) (Zimeo Morais et al., 2017). Consequently, it is expected that the difference in correlation between O2HB and HHB during active and passive trials are significantly higher after interventions than they are before those interventions.

Material and Method

12 male and 30 female subjects aged between 20 and 28 years with a mean age of 21.9 years were scanned. Subjects were expected to be fluent in the Dutch language and to have a good or corrected vision. All subjects received a modest financial reward for their participation. Subjects were pre-screened for buying behavior since they had to be recent buyers of products that were implemented in the paradigm (shower gel, chocolate bar, six-pack beer and toothpaste) to be able to participate in the last part of the paradigm.

Page | 4 Preparation and set-up subject

Subjects started with reading an information leaflet, signing a consent form, and providing bank account details for their financial reward. One of the four products implemented in the paradigm was pseudo-randomly chosen by the subject. Three heart rate stickers were placed on the chest area of the subject. Circumference of the head was measured to decide what size fNIRS cap should be used. For all circumferences above 57 cm a size ‘Large’ was used. For all circumferences below 57 cm a size ‘Medium’ was used. Electrodes were attached to these stickers. The subject was positioned on a chair behind a stimulus computer. An eye tracker (Tobii pro Nano) was calibrated with the

compatible software (Tobii Pro Eye Tracker Manager). Distance to the screen and height of the eyes of the subjects could be adapted by moving the chair. The eye tracker was configured to operate in conjunction with the computer screen by providing the eye tracker software with the distance and angle from the eye tracker to the computer screen. An eye tracker calibration sequence was conducted. GSR was measured by 2 electrodes on the left-hand middle finger and left-hand ring finger. The fNIRS cap was positioned on the scalp and the correct positioning on the scalp was determined by making sure the distance from the indicated central part of the cortex (Cz) to the front of the head in between the eyebrows and the back of the head was identical. Identical distance from Cz to each ear is also required. EEG electrodes and fNIRS optodes were placed in the fNIRS cap on the scalp. Wet electrodes were and therefore gel was used on the electrodes. A ground electrode was placed on the left earl of the subject. An impedance of below 25 Ohm was needed to obtain a signal that could be related to brain activity. Underneath the fNIRS optodes hair was removed as much as possible by scrapping it away with a blunt needle. Depending on the used intervention either a “good” signal or a stable heartbeat was required (see “interventions”). Subjects were instructed to move as little as possible and use the built-in pauses to reposition themselves on their chair and/or to move as wished. At this time the stimulus paradigm could be started.

Paradigm

Subjects started with pseudo-randomly picking a card showing a specific product. Based on the product they got a specific budget assigned for the last part of the paradigm (Table. 1).

Products Budget

Chocolate bar €6,50

Six-pack Beer €8,80

Toothpaste €20,00

Shower Gel €20,00

Table 1. Different products that could be pseudo randomly picked with the corresponding budget during part four of the paradigm.

Each product had 4 options that subjects were able to look at and feel in the same way as they can do in a grocery store. The paradigm consisted of four parts. The first part and third part were identical, in these parts an implicit association test was conducted. Subjects were positioned behind a computer screen and looked at logos of brands. The logos of the brands were shown in the middle of the screen for 3500 ms. After 2000 ms, two words that can spark the valuation of a brand were shown together with the brand in the bottom corners of the screen for 1500 ms (Figure 1). Subjects had about 1000 ms to choose which of the two valuation words suited the brand better in their opinion by pressing the corresponding key on the keyboard as instructed. Subjects did not have to press a key if they did not know the brand. Subjects were able to operate the implemented pauses with the keyboard. In total, 100 trials and 3 pauses per part were implemented. In the second part

Page | 5 subjects watched TVCs. 25 TVCs were watched in total per subject. The average time of a TVC was 50 sec. After every TVC a fixation cross screen for the eye tracker recalibration was shown until subjects looked at the fixation cross (Figure 2), pauses were implemented after every five TVCs. The fourth part contained a WTP. Subjects observed one of the options of their product in the middle of the screen with the price under it for 3500 ms. After 2000 ms the options to buy and not to buy were added in the bottom corners of the screen. Subjects had about 1000 ms to press the key

corresponding with their selected option (Figure 3). Pauses were implemented and could be operated by the subjects. In total, 90 trials and 4 pauses were implemented. Six different were versions used for this paradigm, a total of 150 TVCs were used (Appendix 1.).

Figure 1. Overview of one trial of parts one and three of the paradigm. Subjects had to choose which of the two valuation words suited best for the brand. After this sequence a new trial started. Practice trials were implemented at the beginnings of these parts. Pauses were implemented in between multiple trials that could be controlled by the subject.

Figure 2. Overview of part two of the paradigm. 25 commercials were watched by the subjects in a random order. After a commercial a fixation cross was shown where subjects had to fixate their eyes on to start the next commercial. Pauses were implemented in between multiple trials that could be controlled by the subject.

nietszeggend sympathiek *Pause screen* Logo of brand (2000ms) Insertion of valuation words (1500ms)

Pause until subject presses instructed key

TVC +/-50sec

Pause until subject presses instructed key. TVC

+/-50sec Fixation cross until subject

focuses on cross

Page | 6

Figure 3. Overview of one trial of part four of the paradigm. Subjects saw one of the four options of their product with a price followed by the options to buy the product for that price or not. After this sequence a new trial started. Pauses were implemented in between multiple trials that could be controlled by the subject.

Hard- and software

fNIRS data was collected with hardware manufactured by Artinis (Artinis; NIRx Medical Technologies, Elst, The Netherlands). A 24 channels continuous wave portable fNIRS system named the ‘Brite 24’ was used. Eight receiver optodes (R1-r8) and ten transmitter optodes (T1-T10) were configured on the frontal area of the scalp to configure a 24-channel set-up, 12 per brain hemisphere (Figure 4). The paradigm was programmed and executed in EventIDE (Okazolab Ltd. Delft, The Netherlands). Infrared photons had a wavelength of 760 nm and 850 nm. Sampling rate of the data acquisition was set at 125 Hz. fNIRS data is gathered in Oxysoft v3.3.27 (Artinis; NIRx Medical Technologies, Elst, the Netherlands, 2019).

Figure 4. The fNIRS configuration of 10 transmitter- and 8 receiver optodes on the frontal part of the scalp. This

configuration results in 24 channels per subject (12 per brain hemisphere). Frontal areas of the scalp are covered with this channel configuration.

€1,19 €1,19

Kopen Niet kopen

*Pause screen*

Product + Price (2000ms)

Buy or Don’t buy options

(1500ms)

Pause until subject presses instructed key

€1,19 €1,19

Page | 7 Interventions

In total, two interventions were implemented. The first intervention was executed at the beginning, prior to any data acquisition, so that data of the first scanning period could not only be compared with the data of future scanning periods, but also with a data sample that was obtained with the same experimental setup by researchers at Neurensics. However, without any type of feedback on the fNIRS signal quality. Other improvements concerning the other parts of the experimental, i.e., eye-tracking and heart rate, were also implemented. However, this study will solely focus on the changes that are made regarding fNIRS.

The first intervention was the implementation of a real-time feedback system in eventIDE. Prior to this intervention, multiple programs had to be turned on to look at the fNIRS signal. With this intervention we were able to look at the signal quality during scanning, this was not possible before the intervention. The system was able to make a binary classification of the signal. Thus, the signal could be labeled either ‘good’ or ‘bad’. The intervention also reduces workload on the computer since less programs had to be active during scanning.

The second intervention was the implementation of a new algorithm named ‘signal quality index’ (SQI) that classifies the fNIRS signal (Sappia et al., 2020). The real-time feedback of the signal quality in the first intervention had a binary classification. The SQI algorithm comprises two pre-processing steps and three rating stages to be able to rate the signal from one to five with five being the highest score. Together with the SQI a real-time graph feature was also implemented. The feature visualized the concentrations of O2HB and HHB in a graph and enabled a manual checking of the classification of the SQI, e.g., flat lines, too much noise, or stable heartbeat. The aim was to obtain a stable heartbeat for every optode in the fNIRS signal configuration. Both the SQI and the real-time graph feature were implemented in eventIDE.

Data pre-processing

Subjects were excluded for the full paradigms that were not finished. Specific channels for which the correlation between O2HB and HHB showed a value of “-1.000” or “1.000” were excluded, because these values can be interpreted as a lack of acquisition of data on the optodes. Channels that showed a “NA” value were also excluded. These exclusion criteria resulted in three data groups that were used. Outliers in correlation differences were not excluded for all three groups, since optodes were checked and adapted after every part of the paradigm. The first data group consisted of 20 subjects for which 27 channels were excluded. The second group consisted of 12 subjects for which 12 channels were excluded. The third group consisted of 19 subjects for which 25 channels were excluded. For all fNIRS channels the correlation between concentration O2HB and HHB for the active- (WTP) and passive (TVC) trials were subtracted from each other and the resulting values were transformed into absolute values. This resulted in 1160 usable data points. All descriptive and test statistics including plotting were carried out with R (x64 v4.0.3) in R studio (v1.3.1093; Windows NT10.0; Win64; x64).

Page | 8

Results

The goal of this study is to find ways to improve the method of obtaining fNIRS data to gather reliable data. The correlation differences for all groups are compared to determine if the correlation differences are directly proportional to the number of interventions.

Descriptive & Test Statistics I

The correlation differences of all three groups have a mean of respectively 0,2741; 0,3224 and 0,3323 for groups one, two and three. Standard deviations are respectively 0,2894; 0,3476 and 0,3289. Variances are respectively 0,0841; 0,1217 and 0,1083 (Figure 5). The amount of correlations based on the value of the correlation difference can be found in Figure 6. Since ANOVA assumptions were not met due to high variance and abnormal distribution, a Kruskal-Wallis test was conducted between the correlation differences of all three groups (chi-squared = 3,9019; df = 2; p = 0,1421).

Figure 5. Box plot of the distribution of correlation differences for all three groups. all three groups have a mean of respectively 0,2741; 0,3224 and 0,3323 for groups one, two and three. Standard deviations are respectively 0,2894; 0,3476 and 0,3289 Variances are respectively 0,0841; 0,1217 and 0,1083.

Page | 9

Figure 6. Multiple bar chart of the amount of correlations based on the value of the correlation differences for all three groups.

Descriptive & Test Statistics II

All correlation differences are ranked in tertiles where tertile one includes the lowest correlation differences and tertile three includes the highest correlation differences (Figure 7). The correlation differences of all three tertiles have a mean of respectively 0,0387; 0,2125 and 0,6701853. Standard deviations are respectively 0,0319; 0,0692 and 0,2942. Variances are respectively 0,0010, 0,0048 and 0,0866 (Figure 7.) A Kruskal-Wallis test is used as the main statistical test. A parametric statistical test such as an ANOVA test cannot be used since assumptions for a parametric test are not met. All three tertiles consist of correlation differences that are derived from the three original intervention groups. In the Kruskal-Wallis test the distribution of correlation differences of these three

intervention groups is compared between the three tertiles (chi-squared = 6,4262; df = 2; p = 0,0402*) (Figure. 8). A post-hoc Dunn test is used. All p-values of the Dunn test are adjusted for multiple comparisons with the Bonferroni method (Table. 2). A significant difference in distributions of correlation differences was found between tertile one and tertile three (p = 0,0489*)

Page | 10

Figure 7. Box plot distribution of correlation differences for all three tertiles. Differences of all three tertiles have a mean of respectively 0,0387; 0,2125 and 0,6701853. Standard deviations are respectively 0,0319; 0,0692 and 0,2942. Variances are respectively 0,0010, 0,0048 and 0,0866. Figure 8. Multiple bar chart of the amount of correlation difference values per group for every tertile. Significance is found for the source of correlation differences between tertile one and three (p = 0,0489*).

Page | 11

Tertile Comparison Test statistic (Z-value) Adjusted p-value

1-2 -0,4990 1,000

1-3 -2,4019 0,0489*

2-3 -1,9029 0,1712

Table 2. Tertile comparisons, test statistics adjusted p-values of the post-hoc Dunn test between the three tertiles. Significance is found between tertile one and three (p = 0,0489*). P-values are adjusted with the Bonferroni method.

Conclusion & Discussion

The purpose of this study was to find ways to improve the method of obtaining fNIRS data to gather reliable data for future studies. No significance was found when correlation differences between all three groups were compared. When all correlation differences were ranked in tertiles and compared to determine if the source of the correlation differences (original three groups) was different, a significant difference was detected between tertile one and three. Figure 8 showed that in tertile one correlation values predominantly had group one as a source, and in tertile three correlation values predominantly came from group three. Test statistics displayed that the implementation of a binary fNIRS signal classifier did not result in an improvement of the signal quality. In one of the two test statistics, a difference in group one and three was shown. In one of two ways of statistically testing the data, a difference between group one and three was found. Based on the above, it can be concluded that the implementation of the SQI in combination with a real-time graph feature seems an effective method to improve the reliability of fNIRS data.

This study and its results were severely limited due to several factors. Due to the COVID-19 pandemic, a limited number of subjects could be scanned. The low number of subjects had a crucial impact on the statistical power of the data. Group number two was also considerably smaller in size compared to the other two groups. Furthermore, extrinsic factors such as skin tone, hair color and thickness of the scalp can bias the group data even more than in ideal conditions, since subjects were not selected based upon skin tone or hair color and hair length. Also, different size fNIRS caps were used based upon the size of the head. With a smaller cap the angles between receiving and transmitting optodes change, which influences the signal quality as well. Subjects were not excluded or put in a specific group based upon the circumference of the head and the difference in how the circumference of the head influences the angles between transmitting- and receiving optodes. No corrections were made for the fact that, as the study progressed, the optodes could be placed better and more efficiently Evidently, experience in placing the fNIRS optodes on the subjects has a

noticeable impact on the quality of the data collection. This was not taken into consideration when the fNIRS data was analyzed. Brain activity of two parts of a paradigm where one was passive and the other active were compared. Correlation differences could have been bigger and clearer when brain activity of an active paradigm was compared with resting state brain activity. To carry out an active paradigm, all passive skills are manifestly needed to be able to perform. Consequently, a lot of overlay in brain activity will most likely occur if brain activity from passive- and active paradigms are compared (Mousavi & de Sa, 2019.).

Suggestions for future research can take on several different directions. The same kind of set-up can be used to investigate brain activity in resting-state activity and activity during an active paradigm as discussed in the previous paragraph. The extrinsic factors are still a limitation for fNIRS. A set-up can be created where only one extrinsic factor is selected. For instance, creating three groups where one group consists of bald subjects, a second group of subjects with light-colored hair

Page | 12 and a third group of subjects with darker-colored hair. Clear criteria must be decided on how to classify these subjects into the right group. Signal quality for different configurations of the fNIRS channels on the scalp could also be looked further into. Nowadays, functional studies with fNIRS predominantly use a configuration including the frontal regions of the brain, where the infra-red light will penetrate the scalp to the cortex through an area with less hair. Different configurations of fNIRS channels on the scalp would also be interesting for Neurensics, since portability, which should not be at the expense of quality, is one of the main features they are seeking while innovating on mapping brain activity. If fNIRS channel configurations on the back of the head would also result in measuring brain activity matching the quality of EEG and/or fMRI, a completely wireless set-up could be configured, since fNIRS communicates with the computer via Bluetooth. Due to its portability, a set-up with exclusively fNIRS would give companies and research institutions opportunities to scale up abroad. For now, fNIRS remains a brain mapping technique that still requires further research, but undoubtedly has shown ample potential to keep investing in.

Acknowledgements:

I would like to thank Andries van der Leij (Head of Research and Development, Neurensics), Julius Wantenaar (Research and Development, Neurensics), Anne Ebbink (Colleague Intern, Neurensics) and María Sofía Sappia (First stage researcher, Artinis) for investing their time and effort in this research. Without their support a successful completion of my research and thesis would not have been possible.

References

Bechara, A., & Van Der Linden, M. (2005). Decision-making and impulse control after frontal lobe injuries. Current Opinion in Neurology, 18(6), 734–739.

https://doi.org/10.1097/01.wco.0000194141.56429.3c

de Munck, J. C., Gonçalves, S. I., Huijboom, L., Kuijer, J. P. A., Pouwels, P. J. W., Heethaar, R. M., & Lopes da Silva, F. H. (2007). The hemodynamic response of the alpha rhythm: An EEG/fMRI study. NeuroImage, 35(3), 1142–1151. https://doi.org/10.1016/j.neuroimage.2007.01.022

Huigen, E., Peper, A., & Grimbergen, C. A. (2002). Investigation into the origin of the noise of surface electrodes. Medical & Biological Engineering & Computing, 40(3), 332–338.

https://doi.org/10.1007/bf02344216

Kato, T., Kamei, A., Takashima, S., & Ozaki, T. (1993). Human Visual Cortical Function during Photic Stimulation Monitoring by Means of near-Infrared Spectroscopy. Journal of Cerebral Blood

Flow & Metabolism, 13(3), 516–520. https://doi.org/10.1038/jcbfm.1993.66

Kim, K. H. S., Relkin, N. R., Lee, K.-M., & Hirsch, J. (1997). Distinct cortical areas associated with native and second languages. Nature, 388(6638), 171–174. https://doi.org/10.1038/40623

Lee, N., Broderick, A. J., & Chamberlain, L. (2007). What is ‘neuromarketing’? A discussion and agenda for future research. International Journal of Psychophysiology, 63(2), 199–204.

Page | 13 Meyerding, S. G. H., & Mehlhose, C. M. (2020). Can neuromarketing add value to the traditional

marketing research? An exemplary experiment with functional near-infrared spectroscopy (fNIRS). Journal of Business Research, 107, 172–185.

https://doi.org/10.1016/j.jbusres.2018.10.052

Morin, C. (2011). Neuromarketing: The New Science of Consumer Behavior. Society, 48(2), 131–135.

https://doi.org/10.1007/s12115-010-9408-1

Mousavi, M., & de Sa, V. R. (2019). Spatio-temporal analysis of error-related brain activity in active and passive brain–computer interfaces. Brain-Computer Interfaces, 6(4), 118–127.

https://doi.org/10.1080/2326263x.2019.1671040

Orihuela-Espina, F., Leff, D. R., James, D. R. C., Darzi, A. W., & Yang, G. Z. (2010). Quality control and assurance in functional near infrared spectroscopy (fNIRS) experimentation. Physics in

Medicine and Biology, 55(13), 3701–3724. https://doi.org/10.1088/0031-9155/55/13/009

Poldrack, R. A., Fletcher, P. C., Henson, R. N., Worsley, K. J., Brett, M., & Nichols, T. E. (2008). Guidelines for reporting an fMRI study. NeuroImage, 40(2), 409–414.

https://doi.org/10.1016/j.neuroimage.2007.11.048

Sappia, M. S., Hakimi, N., Colier, W. N. J. M., & Horschig, J. M. (2020). Signal quality index: an algorithm for quantitative assessment of functional near infrared spectroscopy signal quality. Biomedical Optics Express, 11(11), 6732. https://doi.org/10.1364/boe.409317

Sassaroli, A., & Fantini, S. (2004). Comment on the modified Beer–Lambert law for scattering media. Physics in Medicine and Biology, 49(14), N255–N257.

https://doi.org/10.1088/0031 9155/49/14/n07

Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Mata Pavia, J., Wolf, U., & Wolf, M. (2014). A review on continuous wave functional near-infrared spectroscopy and imaging

instrumentation and methodology. NeuroImage, 85, 6–27.

https://doi.org/10.1016/j.neuroimage.2013.05.004

Thulborn, K. R., Waterton, J. C., Matthews, P. M., & Radda, G. K. (1982). Oxygenation dependence of the transverse relaxation time of water protons in whole blood at high field. Biochimica et Biophysica Acta (BBA) - General Subjects, 714(2), 265–270.

https://doi.org/10.1016/0304 4165(82)90333-6

Wilcox, T., & Biondi, M. (2015). FNIRS in the developmental sciences. Wiley Interdisciplinary Reviews:

Cognitive Science, 6(3), 263–283. https://doi.org/10.1002/wcs.1343

Zimeo Morais, G. A., Scholkmann, F., Balardin, J. B., Furucho, R. A., de Paula, R. C. V., Biazoli, C. E., & Sato, J. R. (2017). Non-neuronal evoked and spontaneous hemodynamic changes in the anterior temporal region of the human head may lead to misinterpretations of functional near-infrared spectroscopy signals. Neurophotonics, 5(01), 1.

Page | 14

Appendices

Appendix 1. 150 TVC’s were used in total for the second paradigm. Six versions of this part resulted in 25 TVC’s per version. (Format in appendix: TVC- “commercial name and year” – “Duration time in seconds”). Version Commercial 1 TVC-BeslistNL-2015-Winterjas-30 1 TVC-KPN-2015-Beeldbellen-45 1 TVC-Marktplaats-2013-SpontaneVerkopen-25 1 TVC-DeBijenkorf-2017-VogelPauw-30 1 TVC-Gamma-2015-Verfwinkel-30 1 TVC-Eneco-2011-SamenGaanWeVoorDuurzaam-35 1 TVC-HertogJan-2016-Pilsener-24 1 TVC-Hema-2009-Rompertje-25 1 TVC-Robijn-2015-WasBijChantalJanzen-40 1 TVC-Philips-2014-Airfryer-25 1 TVC-CentraalBeheer-2012-Woonverzekering-65 1 TVC-Telfort-2013-Smartpakkers-44 1 TVC-DubbelFris-2013-MeisjeVsJongen-30* 1 TVC-Belastingdienst-2017-DouaneReizenApp-58* 1 TVC-Fietsenwinkel-2016-OnlineKopen-30* 1 TVC-ASN-2016-Gewoontegedrag-35 1 TVC-NOCNSF-2018-ZoDoenWeDat-38* 1 TVC-AlbertHeijn-2017-BuikBillenBonus-78 1 TVC-Zalando-2012-Virus-27 1 TVC-NederlandseEnergieMaatschappij-2011-JohanDerksen-30 1 TVC-Hak-2013-RonaldKoeman-32 1 TVC-Blijdorp-2013-Olli-75 1 TVC-Heineken-2009-WalkInFridge-31 1 TVC-Bol-2011-Takelauto-25 1 TVC-Volkswagen-2011-OudVrouwtje-45 2 TVC-Nuon-2012-EdEnEduardOverMijnNUON-45 2 TVC-DeFriesland-2017-BetereWereld-60 2 TVC-FBTO-2016-SchadeApp-30 2 TVC-ING-2014-WatGaatHetWorden-30 2 TVC-AlbertHeijn-2011-Appie-42 2 TVC-McDonalds-2014-Euroknaller-20* 2 TVC-Skoda-2014-SkodaExperiment-36* 2 TVC-IKEA-2016-Aandacht-60 2 TVC-Eneco-2014-ToonOmruil-40 2 TVC-OldAmsterdam-2015-Karakter-30 2 TVC-Pickwick-2010-Dutchblend-30 2 TVC-MasterCard-2015-IgonedeJong-40* 2 TVC-OHRA-2017-Viervoeters-45 2 TVC-SNS-2015-Motorcross-30 2 TVC-Chocomel-2016-ZoVersZoOP-20* 2 TVC-Wildlands-2016-KindTijger-25* 2 TVC-Essent-2018-Brand-40

Page | 15 2 TVC-Specsavers-2014-ShouldveGoneTo-30 2 TVC-HoyHoy-2014-MakeoverAart-35 2 TVC-Plus-2011-HollandsePrijsweken-40 2 TVC-McDonalds-2018-CadeauKalender-30* 2 TVC-Gamma-2011-Lego-30 2 TVC-Ditzo-2012-JohndeWolf-35 2 TVC-KPN-2015-1eAppje-40 2 TVC-CentraalBeheer-1989-Klok-46* 3 TVC-Interpolis-2018-ThuisWacht-60 3 TVC-Knab-2015-AllesGeven-22 3 TVC-Tele2-2012-Bioscoop-35 3 TVC-BeslistNL-2015-Sportschoenen-30 3 TVC-Defensie-2013-WerkenBijDefensieJeMoetHetMaarKunnen-35 3 TVC-DELA-2012-LeefVandaag-55 3 TVC-Heineken-2016-TheHero-30* 3 TVC-Opel-2013-ADAM-40 3 TVC-CentraalBeheer-2012-HetLaatsteBod-45 3 TVC-TMobile-2013-AliBZonderAnsjovis-40 3 TVC-Eneco-2012-Toon-32 3 TVC-Unox-2012-DeUnoxBoerenScharrelrookworst-35 3 TVC-Ziggo-2013-WifiSpots-30 3 TVC-Shell-2017-AirMiles-34* 3 TVC-ABNAmro-2017-Spaarverslimmers-45 3 TVC-Telfort-2012-LekkerLangBellen-48 3 TVC-MiljoenenSpel-2012-PatriciaPaay-30 3 TVC-Robijn-2017-HuizeGerschanowitz-40 3 TVC-Pricewise-2017-PaulHaenen-30* 3 TVC-Hunkemoller-2013-SylvieMeis-20 3 TVC-Yarden-2017-AliB-40 3 TVC-Bol-2010-Mummiepak-25 3 TVC-Jumbo-2018-Kerst-70 3 TVC-Calve-2010-Pietertje-41 3 TVC-Brand-2009-HetBierWaarLimburgTrotsOpIs-40 4 TVC-ABNAmro-2015-IntroductieTekst-50 4 TVC-Smint-2017-VIP-20 4 TVC-Airbnb-2015-BelongAnywhere-60 4 TVC-KarvanCevitam-2015-Go-30 4 TVC-TMobile-2011-AliBAltijdSamen-45 4 TVC-Andrelon-2014-OilAndCare-40 4 TVC-Essent-2013-ZekerDalen-30 4 TVC-Tele2-2015-OmdatHetKan-60 4 TVC-Coop-2017-SamenMaakJeVerschil-45* 4 TVC-Telfort-2013-Mobiel-54 4 TVC-Rabobank-2016-HypotheekBinnen1Week-30 4 TVC-PostNL-2012-Moeder-25 4 TVC-Nuon-2018-NuonZonnepanelenHuren-40 4 TVC-Clipper-2016-ManyReasons-20 4 TVC-Anderzorg-2016-DeLeven-20 4 TVC-Hak-2015-IlseDeLange-40 4 TVC-Eneco-2013-Toon4JarigContract-40 4 TVC-Mentos-2017-SayHello-30 4 TVC-VanishOxiAction-2013-Vlekkenverwijderaar-45 4 TVC-Autodrop-2010-Vingerneus-30 4 TVC-STRATO-2014-Internet-30

Page | 16 4 TVC-Knorr-2014-WereldgerechtenBurritos-35 4 TVC-KNGF-2014-Buddyhond-30 4 TVC-Rolo-1996-Olifant-36 4 TVC-CentraalBeheer-2016-Rapper-60 5 TVC-TempoTeam-2015-TeamUp-20 5 TVC-Granditalia-2014-pastamaestro-26 5 TVC-Interpolis-2018-ThuismeesterAppREV-62* 5 TVC-BeslistNL-2016-Angry-32 5 TVC-Marktplaats-2015-GaErvoor-30 5 TVC-Jumbo-2014-Moestuin-55 5 TVC-Unox-2012-KnaksKinderfeestje-25 5 TVC-BeterBed-2015-EdithBosch-20* 5 TVC-Independer-2013-Vergelijken-26 5 TVC-Simyo-2014-Vriendendeal-44 5 TVC-Essent-2013-ThermostaatVanEssent-30 5 TVC-McDonalds-2017-Maestro-60* 5 TVC-CarNext-2019-EveryDetailMatters-40* 5 TVC-Plus-2016-AngryBirds-20* 5 TVC-Eneco-2014-HollandseWindorigami-70 5 TVC-Flexa-2016-Kleurtester-20* 5 TVC-Hak-2016-BonenHermandenBlijker-25* 5 TVC-Zalando-2011-Naaktrecreatie-37 5 TVC-Volkswagen-2013-VolkswagenHond-51 5 TVC-BecamFinancieringen-2018-Bouwvakker-20 5 TVC-Bonprix-2013-Cafe-30 5 TVC-AlbertHeijn-2015-Afscheid-80 5 TVC-Bol-2014-Flappie-30 5 TVC-Campina-2013-DieKomtErWel-45 5 TVC-Heineken-2002-DerRudi-40 6 TVC-ABNAmro-2015-Thuis-45 6 TVC-Grolsch-2015-400JaarKarakter-60 6 TVC-Nuon-2017-LuisterenGeeftEnergie-30 6 TVC-Obvion-2014-Andersdenken-30 6 TVC-AmstelRadler-2015-AlcoholVrij-45** 6 TVC-Ford-2017-WelcomeHome-60 6 TVC-Mona-2010-MonaXLDaarWordJeBlijVan-35 6 TVC-HertogJan-2017-Bastaard-25 6 TVC-PostNL-2017-Lippenstiftkus-22 6 TVC-Telfort-2012-AllesInEen-40 6 TVC-Jumbo-2015-BoodschappenGratis-60 6 TVC-CentraalBeheer-2013-MeerVerzekerdDanUDenkt-52 6 TVC-Eneco-2016-Toon-25 6 TVC-Tele2-2016-HappyDance-50** 6 TVC-Robijn-2010-DoetDeWasBij-IlseDeLange-38 6 TVC-Videoland-2016-OnDemand-30 6 TVC-Interpolis-2018-FocusAutomodus-30* 6 TVC-BakkerBart-2010-Krentenbolletjes-40 6 TVC-Yarden-2015-AdelheidRoosen-40 6 TVC-EyeloveBrillen-2018-ReneFroger 6 TVC-Lidl-2014-GerardJoling-45 6 TVC-KleneDrop-2017-Krakers-20** 6 TVC-CentraalBeheer-2010-Muis-45