• No results found

The influence of auditory processing on the P300 and hazard awareness

N/A
N/A
Protected

Academic year: 2021

Share "The influence of auditory processing on the P300 and hazard awareness"

Copied!
24
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

Bachelor Thesis in Artificial Intelligence

Radboud University Nijmegen

The Influence of auditory processing on the P300 and hazard

awareness

Author:

Dylan Opdam - S4318161

Artificial Intelligence

Radboud University Nijmegen

Supervisors:

Jason Farquhar

Donders Institute

Marjolein van der Waal

Donders Institute

(2)
(3)

Contents

1 Introduction 5 2 Method 9 2.1 conditions . . . 9 2.2 participants . . . 10 2.3 data sources . . . 10 2.4 fixation extraction . . . 11 2.5 EEG-slicing . . . 12 2.6 pre-processing . . . 12 2.7 No saccade test . . . 13 2.8 sub slicing . . . 14 3 Results 17

4 Discussion and Conclusion 21

(4)
(5)

Chapter 1

Introduction

Driving a vehicle requires constant surveillance of the environment. The driver needs to be aware of all the hazards and needs to respond appropriately. Failing to do so may result in an accident. Wang et al [24] and Stutts et al[22] investigated the causes of car accidents. Wang et al [24] and Stutts et al[22] showed that distractions are involved in 25% of the accidents as recorded by police reports. A lot is to be gained by understanding different distractions while performing on a hazard detection task. Understanding distractions would be the bases of creating a training paradigm to keep people focused on the task at hand.

Recarte et al[18] and Underwood et al[23] have researched the eye movement of drivers while performing several distracting tasks. While Underwood et al[23] looked at the mental strains of controlling the vehicle in novice drivers versus experienced drivers, Recarte et al[18] looked at the mental strain caused by an auditory task and a imaginary visual task. Both Underwood et al[23] and Recarte et al[18] used the eye movements as indicators for the difficulty that participants had performing the tasks. A disadvantage of using eye movements as an indicator is that the placement of visual stimuli have an influence on it. Though fixation length and task difficulty have a significant correlation, it would be useful to have an indication of the task difficulty without the influence of stimuli placement.

A technique that could do this is Electroencephalography(EEG). EEG measures the electric potentials of neurons by placing electrodes on the scalp of the subject. Because the electrodes are sitting on top of the scalp instead of being implanted in the brain (as happens with ecog or wire-electrodes) EEG is an noninvasive brain measuring technique. This noninvasive property comes with a trade off. The potentials that the neurons emit have to travel through the tissue and bone of the scalp. The already weak signals are even further weakened and distorted by the time it reaches the electrodes. Because of this EEG can only measure the potentials of groups of neurons that all simultaneously fire in the same direction, this gives EEG a poor spacial resolution as a single electrode measures the activity of a space in the order of centimeters of brain surface[15]. Despite this low spacial resolution EEG is still widely used mainly because of its great temporal resolution, low cost and relative portability (relative to measurements like FMRI and MEG).

In EEG two main type of signals are distinguished: Induced related potentials(IRP) and Event related potentials(ERP). IRP’s are the result of internal processes in the brain of the subject. For example the subject is asked to imagine a certain movement, or repeat a conversation in his or her head. In contrast ERP’s are the result of an event in the environment of the subject. For example an image is shown on the screen for few hundred milliseconds or a sound is played for the subject. ERP’s are time locked to presented stimulus making them relatively easy to find.

(6)

6 CHAPTER 1. INTRODUCTION A interesting ERP signal is the P300, this is a positive peak that appears on the top of the head 300 milliseconds after an odd stimulus in a series of stimuli is presented to the subject[17]. The P300 signal is generally used to as an indicator to whether or not a subject has been presented with an oddball in the series of stimuli. Common example of this are the P300 visual spellers[19][4]. As attention get divided between tasks the P300 signal decreases in power[25][20]. Strayer et al[21] used this decrease in power as an indicator for the task difficulty. Strayer et al[21], placed subjects in driving simulators while wearing EEG caps. Subjects where instructed to follow the car ahead of them as it drove through the simulation. The subject’s car would automatically turn and accelerate with the leading car, the only action the subjects had to perform was to press the brakes when the brake lights of the leading car turned on. The leading car always stayed in the middle of the screen, eliminating the need for the subjects to turn there head and allowing them to continuously fixate on the centre of the screen. The experiment used the oddball design were the subject only has to respond when the brake lights turn on while most of the time no action had to be taken.

By first measuring the height of the P300 spike in the simulator with a fully concentrated subject a base line is established. This baseline would be compared with the P300 spikes that resulted of the subject being distracted by a phone call or talking person. From this comparison the reduction in power can be seen, which can be used as indication of how aware subjects are of target stimuli. This way gaining an indicator of task difficulty without the influence of the placements of visual stimuli.

Because the EEG signals of the brain are so weak noise quickly becomes a problem. The brain signals are almost completely overshadowed for example by the 50 hertz interference caused by the electrical wiring inside the walls, or by the muscle activity of a eye blink, eye movement or any other movement made by the subject. For this reason subjects of EEG experiments are instructed to sit still and even to keep there eyes fixed on a given point on the screen. The need for the subject to sit still makes it difficult to test more of the real world scenario’s. However other paradigms open doors to new tests that slowly inch towards more real world scenario’s.

The paradigm on which this experiment is based is called Fixation related potentials (FRP’s)[9][2]. In this setting the subjects still have to sit still. However instead of quickly presenting the stimuli one after another, multiple stimuli are presented simultaneously across the screen. The subject is instructed to use eye movements to look at the stimuli. Using eye tracking the experimenter can determine when and where the subject is looking at. The theory is that like ERP’s the FRP’s are time locked signals but instead of the signals being time locked to the stimuli presentation the signals are time locked to the moment of fixation.

FRP’s have previously been used in research to the processing of reading. By recording the EEG signals while a subject is reading a text researchers can see the brain response after the eyes have fixated on a word. By for example by ending sentences with an unrelated word the brain will produce a N400 signal.[13], giving insight in were and when context related processes occur. It has been shown that the ERP signals N1,P1 and N400 can be found using the FRP setting[3][5][12][16][8].

However the experiments of Brouwers et al.[7] and Kamiekowski at all.[11] have shown that the P300 signal of the FRP’s can be used to distinguish target/non-target fixations in a visual search task. In the experiment of Brouwers et al[7] participants were instructed to make eye movements following a circle of stimuli while trying to identify the target stimuli. The target stimuli are the oddballs in the circles of stimuli and thus enlist a P300 signal, time locked to the fixation onset. The stimuli were placed in a circle to cancel out the influence of the saccade made prior to the fixation as the circle forces the participants to make a saccade in the opposite direction for every saccade made. Figure 1.1 shows the stimuli presentation from the experiment of Brouwers et al[7].

(7)

7

Figure 1.1: Stimuli as presented in the experiment of Brouwers et al[7]

This experiment builds further on the experiment of Brouwers et al[7] to investigate the effect of different sound distractors using the power of the P300 as an indicator for the distractors effectiveness, more specifically to investigate if spoken language is more distracting than random noise.

(8)
(9)

Chapter 2

Method

2.1

conditions

The participants of the experiment where presented with five conditions that will be referred to as: control condition, attended sound condition, unattended sound condition, No saccade condition and complex condition.

• control condition: This condition is a reproduction of the experiment of Brouwer et al.[7] The stimuli consist of a circle of six Landolt Cs. The participants are instructed to look at each C in order, starting from the top C and moving clockwise, while searching for a given C. After the participant is done looking through the circle the participant can continue by pressing the return key on the keyboard. Then the response screen is presented where the participant indicates which of the six Cs where targets by pressing the buttons one through six. To prevent the P300 signals to be contaminated by the next stimuli the subjects are instructed to take there time to look at the stimuli. The participant has to look at the circle for a minimum of three seconds. If participants try to continue before the three seconds are up they will get an warning.

Every C has a diameter of 86 pixels and the opening of 20 pixels. The circle of C’s has a slight elliptic shape with the horizontal diameter 400 pixels and the vertical diameter 350 pixels, center of the circle is the centre of the screen. The smallest distance between the Cs is 200 pixels. The stimuli are placed at 12,2,4,6,8 and 10 O’clock. The number of targets present in a circle can be 0,1,2,3,4 or 5 with a occurrence of respectively 181, 183, 185, 185, 183 and 181. The places of the targets are chosen uniformly to cancel out the influence off the eye position.

• attended sound condition: In this condition the participants essentially perform the control task however during the task a sound fragment is played containing a monologue for the participants to listen to. In this case the fragments were parts of Herman Vinkers’s new Year conference. These fragments where chosen because they are calm monologues designed to entice the listener, and thus distracting the subjects. The participant is instructed to listen to the fragment and at the end of the task report which of the words in a given list he heard. In this condition the reduction in the P300 signal should only be caused by the sound reaching the participant and the processing of the language.

• unattended sound condition: Again the task to perform is the control task. This time there is also a sound fragment playing. The fragments this time are same as the fragments in

(10)

10 CHAPTER 2. METHOD the attended sound condition but played backwards making the sound incomprehensible. By playing the sound fragment backwards the sound still has the same frequencies rhythm and volume as the attended sound fragments. This is to ensure that any difference in the two conditions is due to the ability to comprehend the audio. The participant is instructed to try and ignore the sound and focus on the main task. Any reduction found in the P300 signal in relation to the control condition should only be caused by noise reaching the participant. Any difference in the strength of the P300 between this condition and the attended sound condition should be the result of processing the language.

• No saccade condition: This condition was added to check whether or not the stimuli where able to invoke a P300 signal. This condition uses ERP’s instead of FRP’s. The ERP’s are a more classical approach to EEG research and are easier to implement than the FRP’s. The Landolt Cs from the control condition are presented to the subject one after the other in the middle of the screen. Each stimulus is presented for 1 second and has the same size as the stimuli in the control condition. Participants are instructed to remember which positions in the sequence were target Cs.

• complex condition: In this condition the essence of the task is the same. Finding the given Landolt C’s in a circle of six. This time however the C’s vary in more than just orientation. Other then the orientation the color of the C changes and whether or not a dot is present in the middle of the C. A C is a target when two of the three conditions (orientations, color and dot) are the same as in the given template. This condition is part the experiment of another student. Because the two experiments share a similar setup it was decided to merge the two experiments to share participants and lab time. This condition will no further be discussed in this thisis.

The five conditions are presented in 10 blocks (2 blocks for every condition) of 18 trials each. The first three subjects only have 12 trials per block, however based on the speed with which they finished the experiment, the experiment was extended to have 18 trials per block to be more efficient with the lab time.

The order of the blocks was randomized to prevent possible interactions between the condi-tions. However at the start of the experiment all subjects were allowed two practice trials first a trial of the control condition and second a trial from the complex condition.

2.2

participants

Mainly because of time restrictions the experiment is limited to five participants, four men and one woman. The age range of the participants is 20 to 56, tough when excluding the participant with the age of 56 the age range drops to 20 to 25. All participants were able to perform the task without the need for glasses or lenses. All participants gave their informed consent to participate in the experiment prior to starting the experiment.

2.3

data sources

The experiment records data from three sources:

• Psychopy log: The experiment is written using the python library psychopy. For every circle of six landolt C’s python writes two lines in a log. The first line contains whether or not each of the six C’s is a target or non target. The second line contains which C’s the

(11)

2.4. FIXATION EXTRACTION 11 subject identified as targets. Before each circle is presented an marker is send to both the EEG buffer and the Eye tracking Buffer.

• EEG data: 64 channels of EEG recording the brain signals and four channels of EOG to record the eye muscle activity to filter out eye artifacts. The data contains a marker at the moment a circle of stimuli is presented. The system used to record the EEG data was a Biosemi active two, with a sample rate of 2048Hz. The data is down sampled in the recording buffer to 256Hz.

• Eye data: an eye tracking camera tracks the position of the right eye. This data is also contains markers at the moment when the circle is presented. Using the marker the fixations are timestamped relative to the time of the last received marker. To record the eye tracking data an eyelink 100 was used with a sample rate of 1000Hz

2.4

fixation extraction

Once the data is collected the eye fixations are extracted. The Eye tracking software provides a data stream with event markers at the moment a fixation starts and when the fixation stops. The buffer that read and saves this data stream marks the data when an event is received from the Psychopy experiment. Using these events the data stream can be cut into a table containing all fixations starting times, end times, x coordinate, y coordinate and the value of the last received Psychopy marker. Since the coordinates of the stimuli are known every fixation can be mapped to a stimulus and then labeled target or non target by looking up this stimuli in the Psychopy log. A fixation is mapped to a stimulus when the position of the fixations is within 80 pixels of the center of the stimulus.

Because subjects often look back in to previous stimuli and because they make multiple small fixations when looking at stimuli, every stimulus has more than a single fixation. From these multiple fixation only one will invoke the P300 signal, it is important that during the data processing these fixations are used. In the paper of Brouwers et al [7] and Kamienkowski et al[11] all fixations shorter than 500 milliseconds were discarded. The reason for this was mainly to ensure that the complete P300 signal would fall within the fixation and thus be free of eye artifacts, however this also limited the number of fixations to a degree that there are no longer multiple fixations per stimuli. The down side of this method is that fixations with a length greater then 500 milliseconds are rare and Brouwers et al[7] even had to discard participants for a lack of valid fixations. To filter out the fixations without having to discard to much data the longest fixation per stimulus is selected and participants are instructed to move slowly from C to the next C. This way fixations longer than 500 milliseconds are selected but if no fixation longer than 500 milliseconds is present rather than discarding the stimulus, the next best fixation is selected.

The filtering criterion of the fixations resulted in an average of 284,2 fixations per subject. The maximum amount of fixations would be six fixation per trial. Thus for man1 man2 and vrouw1 who had 12 trials per block the maximum would be 432 and for man3 and man4 who had 18 trial per block the maximum would be 648. Due to subjects skipping stimuli in, looking at stimuli from distances further than 80 pixels and the eye tracker losing track of the eye, the amount of fixations are much lower than this maximum, as can be seen in Table 2.1. The mean of the fixations is higher than 500 milliseconds however the number of fixations that are longer than 500 milliseconds are low, these would be the fixations that would also been selected in the experiment of Brouwers et al[7]. This shows that by selecting the longest fixations instead of only the fixations longer than 500 milliseconds does preserve most of the data.

(12)

12 CHAPTER 2. METHOD

Figure 2.1: figure

All recorded fixations for man1 during the complete experiment

Figure 2.2: figure

fixations selected for attended sound condition for man1

subject mean fixation length fixations longer than 500 ms total number of fixations man1 575.1 ms 82 254

man2 603.0 ms 146 227 man3 726.1 ms 84 209 man4 627.9 ms 192 520 vrouw1 1068.6 ms 92 211

Table 2.1: statistics of filtered fixations

2.5

EEG-slicing

The EEG Buffer reads and saves the EEG data stream and receives the Psycopy event markers. These markers are send every time a new circle of stimuli is presented and contain the block number, trial number and the condition of the circle. Using these markers The EEG data is sliced in chunks of 15 seconds starting from the event. 15 seconds was chosen as size because the maximum duration of a trial is 15 seconds (complete with response and target presentation). These first cuts are ensured to have all the data from the trial.

To ensure that the EEG data and the Fixations data are running synchronous a blink test was used. The eye tracker outputs a error value of -32768 when the eye is lost, this happens for example during a blink. When the eye closes the eye tracker losses track of the eye for a brief interval, this shows up in the data as a narrow spike to -32768 in both the x and y coordinate of the eye. The EOG sensors that are placed around the eye record the activity of the eye muscles in this data stream blinks are also recognizable as a short strong spike. When the eye tracking data and the EOG data are plotted side by side both of the data streams should show blink artifacts at the same time.

2.6

pre-processing

Since the Brain signals in the EEG data are easily overshadowed by noise, the data needs to be cleaned. If the standard deviation of a channel or trial is 3.5 times the average standard

(13)

2.7. NO SACCADE TEST 13

Figure 2.3: eye tracking and EOG plot

deviation the channel or trial is removed. To filter out noise the common average reference[6][14] method is used. This method assumes that noise from outside the skull is far enough away for the noise to distribute uniformly over all the sensors. When an average is calculated over all the sensors on the skull the brain signal is cancelled out but the noise that is present in all the sensor readings remains. This average can be subtracted from the EEG data leaving behind the brain signals.

After the common average reference is computed the EEG channels that were removed are filled back in by interpolating[10] them. When doing this the bad channels are replaced by the average of the surrounding channels. This is possible because of the low spacial resolution of EEG. Sensors that are close to each other have overlap in neurons that are measured. By computing the average of the surrounding sensors only this overlap remains. This can then be used as an approximation for the bad channel.

To filter out eye artifacts a Surface Laplacian Reference[1] is used. Other than that the EOG recorded data is removed to remove eye movement.

Finally a frequency band pass is used to filter out data with specific frequencies. For example the disturbance cause by the power grid has a clear 50 hertz frequency and must be filtered out. The brain also produces signals with a steady frequency like the alpha and delta waves, these waves obscure the P300 signal and thus are filtered out. The band pass filter in this experiment only allows frequencies form 0.5 Hz to 12 Hz.

2.7

No saccade test

After the prepossessing the target and non-target trials of the no saccade condition are averaged and plotted in the same figure resulting in figure 2.4.

(14)

14 CHAPTER 2. METHOD

Figure 2.4: ERP from the No saccade condition

As shown in figure 2.4 the target graph rises to about 10 micro volts around 300 milliseconds, while the non-target graph stays behind on a significantly lower voltage. This would classify as a P300 signal. With this we can conclude that the stimuli are and the current target frequency can enlist a P300 effect. This would mean that if the fixations are chosen correctly the FRP’s should also show a P300.

2.8

sub slicing

After the fixation and the EEG data is cleaned the EEG data is sliced again, this time to create 1 second data packets. For every fixation the corresponding EEG trial is selected. From this EEG trial 1 second of data is selected, this second starts on 200 milliseconds before the fixation starts and end 800 milliseconds after the start of the fixation. This time window is chosen to have a clear picture of the P300 consisting of the base activity to be able to see the activity rise for the P300 at around 300 milliseconds then the activity return to the base activity around 600 milliseconds.

To check if the mapping of the EEG trials and the fixations if performed correctly, an align-ment test was performed. A sample of the fixations is plotted to see if the start of a fixation correlates with an eye artifact. This test results in figures like figure 2.5. The subplot shows the part of the EEG data that is selected while the bottom subplot show the EEG data recorded by the electrode Fz on the forehead. In this figure the start of the selected EEG piece aligns nicely

with a drop in activity. This drop is likely the result of the activity in the eye muscles during an saccade. This would make sense since a fixation always follows a saccade.

(15)

2.8. SUB SLICING 15

Figure 2.5: top plot: indicates the EEG samples selected by the fixation. Bottom plot: EEG activity in channel Fz.

(16)
(17)

Chapter 3

Results

After the participants have seen the stimuli they are asked which of the stimuli are targets. This information is recorded and results in the accuracy’s displayed in table 3.1. The accuracy’s in table 3.1 suggest that the task is fairly easy in all the conditions.

subject control attended sound unattended sound No saccade test man1 0.9861 0.8194 0.5764 0.8333 man2 1.0000 0.9861 0.9722 0.9931 man3 1.0000 1.0000 1.0000 1.0000 man4 0.9955 0.9784 0.9954 0.9954 vrouw1 0.9867 1.0000 1.0000 0.9722 average 0.9937 0.9567 0.9088 0.9588

Table 3.1: Task performance of participants measured in the accuracy of their responses.. A repeated measure anova test shows that the within subject effect has a p-value higher than 0.1, indicating that the there is no significant difference in difficulty between the conditions. Equality in difficulty is a good thing, this ensures that differences in the P300 signal are the cause of language processing and not because one of the conditions required more attention. Ideally the main task should have been more difficult. Since the scale of accuracy ends at a hundred percent it is impossible to see if the score was a hundred percent or more when a score is a hundred percent.

Figure 3.1 shows the the data recorded by the electrodes Fpz, F3, Fz, F4, C3, Cz, C4, P3, Pz and P4 during the control condition of vrouw1. The P300 signal is most prominent on the top of the head[17], these electrodes should record the P300 when it appears.

The plots in figure 3.1 do not show a P300 signal. In Figure 2.4 the we see a spike of activity around 300 milliseconds, this activity lingers for a couple hundred milliseconds before dropping back down to around zero. None of the plots in figure 3.1 shows this kind of behaviour. The plots of the electrodes Fz, F3, C3 and P3 are not even positive at 300 milliseconds. In the plots were the activity is is positive the activity fluctuates rapidly suggesting that the positive spike is only coincidental. The plot of electrode Pz shows the most P300 like shape. At around 300 milliseconds the shape of the Pz plot is characteristic for a P300 signal, however after the activity falls down the activity does not settle but instead it keeps spiking. The clear P300 shape does suggest that the P300 signal is in the data however it is obscured by noise.

Figures 3.1 through 3.7 show the data recorded by electrode Cz and Pz of vrouw1 in all conditions. These figures also show the high frequent spikes. Figure 3.4 shows an interesting

(18)

18 CHAPTER 3. RESULTS negative effect around 300 milliseconds, if the effect had been positive could be labeled as a P300. However non of the steps in the prepossessing of the data would explain why the data would be upside down, therefor this also is no P300. The P300 characteristic shape that is seen in the Pz electrode of the control condition does not show in the other condition making the spike look more like a random occurrence than a solid P300 signal.

The presents of the P300 like spike in electrode Pz is promising for the experiment. However there is no solid P300 effect. The absence of a P300 signal in the control condition is especially surprising. Since Brouwers et al[7] did find a P300 signal. The absence of the P300 in the control condition and the amount of noise that seems to present in the final plots, suggest that something went wrong during the mapping of the fixations and the EEG trials.

(19)

19

Figure 3.2: figure

Control condition of vrouw1 at Cz

Figure 3.3: figure

Control condition of vrouw1 at Pz

Figure 3.4: figure

Attended sound condition of vrouw1 at Cz

Figure 3.5: figure

Attended sound condition of vrouw1 at Pz

Figure 3.6: figure

Unattended sound condition of vrouw1 at Cz

Figure 3.7: figure

(20)
(21)

Chapter 4

Discussion and Conclusion

Since no P300 signal was found no conclusions about the effect of speech distractors on the P300 signal can be made by this experiment. The absence of the P300 signal in the control conditions almost suggests that something went wrong during the data analysis. Therefore all the steps of the analysis were rechecked.

During the experiment the eye tracker lost the eye and as a result the complete experiment has to be restarted. This resulted in the block and trial counter to reset and getting duplicates labels for the EEG trials and fixations. To fix this later everything was relabeled as soon as the data was loaded for analysis and while everything was still listed in chronological order. I do not have complete confidence in the alignment test. While after taking a small sample the mapping of the fixations seem to be in order, the code that performs the relabeling is a rather bulky one. If something went wrong during the analysis the relabeling of the fixations would be the most likely to contain the bug.

For future experiments that are performed using the FRP paradigm I would suggest designing the experiment in such a way that it can be restarted at any point without creating duplicate labels. Because this experiment requires both the EEG, the Eye tracker and the experiment itself to run and if one fails everything needed to be restarted, this not only cost time but also causes segmentation of data that later has to be joined together again. Especially the eye tracker is unstable, if the participant moves his or her head to much or squint his or her eye a little to much the eye tracker loses the eye and cannot find it back.

The unstable behaviour of the eye tracker might be caused by the placement of the camera. The camera was placed in front of the screen, with the screen just above the camera. Even with the camera and the screen all the way at the back of the desk, the camera had to almost lay flat to on its back to be able to see the eye of the participant. This angle deformed the image of the eye making it more difficult to track. If participants would move there head only a little back the eye would be obscured by the cheek bone. A better placement of the camera might be on a pedestal behind the screen with the camera just above the screen. This way the camera is more directly pointed at the eye of the participant.

Another cause could be a difference between our experiment and the original experiment of Brouwers et al[7].

One key difference is the way the fixations are selected. In the experiment of Brouwers et al[7] fixations shorter then 500 milliseconds are discarded and the rest is used. But to prevent too much data to become unusable we selected the longest fixation for each stimulus. These fixations should have a high correlation, however we do have many smaller fixations which might only add noise to the data. Though selecting the longest fixations for every stimulus might preserve more

(22)

22 CHAPTER 4. DISCUSSION AND CONCLUSION data to be used, if the extra data is only noise it better to cut is away.

A second difference is that our stimuli are a lot bigger than in the original experiment. This is mostly because the stimuli are images that get scaled down but when they are scaled down too much they get to pix-elated and the dot in the middle of the complex stimuli fades out. In the experiment of Brouwers et al[7] the size of the stimuli is chosen based on the fact that the subject needs to be forced to make a saccade to the stimulus and cannot determine target/non-target from the corner of the eye, while looking at a different stimulus. In the fixation data we gathered six clusters that are placed on the positions of the six stimuli. This gives confidence that the stimuli are small enough to force the saccade. However the bigger stimuli might mean that the fixations are further away than the labeling function looks, as it is now a fixation is considered to belong to a stimulus when the fixation coordinates are within 80 pixels from the center of the stimulus. This number came to be as the minimum distance between the stimuli is 200 pixels thus a threshold of 100 pixels would be the limed. When classifying the fixation of the first subject this seemed to nicely fit all the fixations in the clusters and ignore the more scattered fixations. In the subject after that the threshold still seemed capture the clusters so it was kept. The fixations of Man2 are not correctly classified, this however is not due to the threshold but rather to the fact that the eye tracker had trouble with tracking the eyes. The cluster for the stimuli are all shifted and the whole shape is slightly bend causing classification to fail.

It might also be possible that the sample size was simply to small. The original experiment of Brouwers et al[7] had thirteen participants that performed 240 trials each. Because we didn’t want participants get to tired we limited the experiment time to one hour. In this hour the par-ticipants performed 180 trials which is only two thirds of the 240 used in the original experiment. To add to that the trials are divided over the five conditions leaving only 36 trials per condition. This smaller number of trials increases the chances off the P300 not showing in the resulting plots and since there are only five participants, chances are that the P300 just would not show in these participants.

For future research I would suggest investigating the fixation selection criteria to find out the best ways to select the fixations that contain the P300 signal. For this particular research question I suggest a more difficult task, making the the stimuli smaller and the display time shorter, to facilitate the mapping of the fixations to the stimuli and get a clearer difference between the condition difficulties.

(23)

Bibliography

[1] F Babiloni, C Babiloni, L Fattorini, F Carducci, P Onorati, and A Urbano. Performances of surface laplacian estimators: a study of simulated and real scalp potential distributions. Brain Topography, 8(1):35–45, 1995.

[2] Thierry Baccino. Eye movements and concurrent event-related potentials’: Eye fixation-related potential investigations in reading. Eye, 2:1, 2011.

[3] Thierry Baccino and Yves Manunta. Eye-fixation-related potentials: Insight into parafoveal processing. Journal of Psychophysiology, 19(3):204–215, 2005.

[4] A Belitski, J Farquhar, and P Desain. P300 audio-visual speller. Journal of Neural Engi-neering, 8(2):025022, 2011.

[5] Artem V Belopolsky, Arthur F Kramer, and Jan Theeuwes. The role of awareness in pro-cessing of oculomotor capture: Evidence from event-related potentials. Journal of Cognitive Neuroscience, 20(12):2285–2297, 2008.

[6] O Bertrand, F Perrin, and J Pernier. A theoretical justification of the average reference in topographic evoked potential studies. Electroencephalography and Clinical Neurophysiol-ogy/Evoked Potentials Section, 62(6):462–464, 1985.

[7] Anne-Marie Brouwer, Boris Reuderink, Joris Vincent, Marcel AJ van Gerven, and Jan BF van Erp. Distinguishing between target and nontarget fixations in a visual search task using fixation-related potentials. Journal of vision, 13(3):17–17, 2013.

[8] Olaf Dimigen, Werner Sommer, Annette Hohlfeld, Arthur M Jacobs, and Reinhold Kliegl. Coregistration of eye movements and eeg in natural reading: analyses and review. Journal of Experimental Psychology: General, 140(4):552, 2011.

[9] Graf V Armington J C Gaarder K, Krauskopf J. Averaged brain activity following saccadic eye movement. science, 146:1, 1965.

[10] Markus Jungh¨ofer, Thomas Elbert, Don M Tucker, and Brigitte Rockstroh. Statistical control of artifacts in dense array eeg/meg studies. Psychophysiology, 37(04):523–532, 2000. [11] Juan E Kamienkowski, Matias J Ison, Rodrigo Quian Quiroga, and Mariano Sigman. Fixation-related potentials in visual search: A combined eeg and eye tracking studyfixation-related potentials in visual search. Journal of vision, 12(7):4–4, 2012.

[12] Koji Kazai and Akihiro Yagi. Comparison between the lambda response of eye-fixation-related potentials and the p100 component of pattern-reversal visual evoked potentials. Cognitive, Affective, & Behavioral Neuroscience, 3(1):46–56, 2003.

(24)

24 BIBLIOGRAPHY [13] Marta Kutas, Steven A Hillyard, et al. Reading senseless sentences: Brain potentials reflect

semantic incongruity. Science, 207(4427):203–205, 1980.

[14] Kip A Ludwig, Rachel M Miriani, Nicholas B Langhals, Michael D Joseph, David J An-derson, and Daryl R Kipke. Using a common average reference to improve cortical neuron recordings from microelectrode arrays. Journal of neurophysiology, 101(3):1679–1689, 2009. [15] Paul L Nunez and Kenneth L Pilgreen. The spline-laplacian in clinical neurophysiology: a method to improve eeg spatial resolution. Journal of Clinical Neurophysiology, 8(4):397–413, 1991.

[16] PIA R¨am¨a and Thierry Baccino. Eye fixation–related potentials (efrps) during object iden-tification. Visual Neuroscience, 27(5-6):187–192, 2010.

[17] Daran Ravden and John Polich. On p300 measurement stability: habituation, intra-trial block variation, and ultradian rhythms. Biological psychology, 51(1):59–76, 1999.

[18] Miguel A Recarte and Luis M Nunes. Effects of verbal and spatial-imagery tasks on eye fixations while driving. Journal of Experimental Psychology: Applied, 6(1):31, 2000. [19] Mathew Salvaris and Francisco Sepulveda. Visual modifications on the p300 speller bci

paradigm. Journal of neural engineering, 6(4):046011, 2009.

[20] Coles M.G. Donchin E Sirevaag E.J., Kramer A.F. Resource reciprocity: an event-related brain potentials analysis. Acta Psychologia, 70(1):77–97, 1989.

[21] D.L. Strayer and F.A. Drews. Cell-phone–induced driver distraction. Current Directions in Psychological Science, 16(3):128–131, 2007.

[22] Jane C Stutts, American Automobile Association, et al. The role of driver distraction in traffic crashes. AAA Foundation for Traffic Safety Washington, DC, 2001.

[23] Geoffrey Underwood, Peter Chapman, Karen Bowden, and David Crundall. Visual search while driving: skill and awareness during inspection of the scene. Transportation Research Part F: Traffic Psychology and Behaviour, 5(2):87–97, 2002.

[24] Jing-Shiarn Wang, Ronald R Knipling, and Michael J Goodman. The role of driver inatten-tion in crashes: New statistics from the 1995 crashworthiness data system. In 40th Annual proceedings of the association for the advancement of automotive medicine, volume 377, page 392, 1996.

[25] Kramer A.F. Vanasse L. Donchin E Wickens, C.D. Per-formance of concurrent tasks: A psycho-physiological assessment of the reciprocity of information-processing resources. Sci-ence, 221, 1983.

Referenties

GERELATEERDE DOCUMENTEN

Although the new world poses serious challenges for individuals, systems, organisations, governments and multinational role-players, the situation is compounded by the fact that

De ‘Instrumentenkaarten’ zijn ontwikkeld door Praktijkonderzoek Plant en Omgeving, CLM en LBI in het project ‘Biodiversiteit in de Praktijk’ in opdracht van LNV, als vervolg op

Moreover, based on our experiments and results, we are able to confirm that the given algorithm can detect clones of type-3, clones that are non-contagious, reordered, intertwined

De rassen (tabel 4.1) zijn door de veredelingsbedijven aangemeld voor het gebruikswaarde-onderzoek in de zomer/herfst van 1988.. Op 6 juli zijn per ras op tw^e veldjes in totaal 16

Door het toepassen van kwalitatief empirisch onderzoek wordt nagegaan in welke mate sprake is van procesoriëntatie binnen Alliander en in welke mate de kritieke succesfactoren deze

The argument is informed by field research during 2006 on the management of knowledge in the Great Lakes region of Africa, including research on how knowledge on the

Spirituality in terms of ‘seeking God’s help’ was associated with lower levels of sexual QoL in the unadjusted model, but when adjusted for other factors (e.g., gender, age,

business model. Sustainable innovations lead to a high standardized quality of service and inevitable the associated premium prices. Paradoxical equilibrium of sustainable