Increasing the sensitivity of concealed information testing via
probe-irrelevant proportion manipulation: A pilot study
University of Amsterdam
Department: Clinical psychology
Supervisor: Bruno Verschuere Co-assessor: Jos Bosch
MSc in Brain and Cognitive Sciences, University of Amsterdam, Cognitive Neuroscience
Internship Report
Benjamin Germain-10396411
10
thJanuary 2014
Introduction:
Deception research: From arousal to cognition
For years lie detection testing has been implemented in order to help determine a person’s true knowledge (e.g., Ford, 1995; DePaulo et al., 2003). In the past, polygraph and other comparable tests have relied on recording a person’s physical response to a question. More recently, alternative methods have been developed with aspirations of designing a precise test that can be easily and accurately executed. One modern method that has been extensively investigated is known as the Concealed Information Test (CIT). CIT is a paradigm designed in order to determine if a participant is concealing knowledge via their reaction to known information (Ben-Shakhar, 2012).
Psychophysiological
Psychophysiological methods (Polygraph techniques) have been investigated ever since Marston (1917) first proposed that physiological recordings can be used to determine if a person is giving a truthful response or not ( e.g., Larson, 1932; Iacono and Patrick, 1988). Polygraph testing uses simple
physiological changes (heart rate, breathing, electrodermal response) which are assessed in reaction to a carefully structured set of questions. In order to obtain physiological recordings, the polygraph test induces stress and doubt into the participant. The examiner stresses that the participant must be able to fully answer the questions with a simple "yes" or "no" answer and that the polygraph will record any confusion, misgivings, or doubt (Saxe, Dougherty, & Cross, 1985). Patterns of autonomic arousal to these questions are used to infer a subject's truthfulness or deception. However, there is no unique physiological response to deception (Lykken, 1981; Orne, 1975), making interoperation of the data a complex clinical task. Additionally, the validity and reliability of polygraph testing has been controversial (e.g.,
Ben-Shakhar, Bar-Hillel, & Lieblich,1986; Furedy & Heslegrave, 1988). A problem concerning the polygraph is that the data must be interoperated by an examiner and these interoperations can vary between
examiners. Also, a validity study conducted by Saxe et al., (1985) stated that the accuracy of a polygraph is highly dependent on the development of the questions and the situation established by the examiner. Due to these failings, cognitive techniques have been investigated with aspirations of designing a more standardized test.
Cognitive Approach
Recent research has been elucidating cognitively based detection methods as an alternative to the physiological stress detection approach. A supported theory for deception is that lying requires more cognitive resources than telling the truth (Van Bockstaele et al., 2012, Verschuere, Spruyt, Meijer, 2011). Christ, Van Essen, Watson, Brubaker and McDermott (2009) implemented functional magnetic resonance imaging (fMRI) in order to observe which brain areas show additional activation during deception. Using their collected fMRI data, the team compiled a region of interest (ROI) map consisting of the brain regions
with significant activation during deception. Analysis showed that 10 out of the 13 deception related ROI’s were associated with working memory, inhibitory control and/or task switching. Thus supporting the theory that: Humans experience an additional cognitive load during deception via prefrontal cortex activation. Resulting in the meta-analysis suggesting that “truth constitutes the default of the human brain, and that lying involves intentional suppression of the predominate truth response.” (Verschuere et al., 2011, p. 908). This results in an aberrant lie response that is more cognitively demanding and subsequently slower. Testing methods and paradigms have been designed with intentions of exploiting the additional cognitive demand brought about during deception.
A cognitive test: the Concealed Information Test
One cognitive testing method is known as the Concealed Information Test (CIT). CIT is a paradigm designed in order to determine if a participant is concealing knowledge via their reaction to known information (Ben-Shakhar, 2012). In order to achieve this, the CIT implements information that only the ‘guilty’ person would know. The participant is given a statement and then repeats possible answers for this statement. One of these answers corresponds to an event that only a guilty person could know (such as the murder item). To an innocent participant, all answers will evoke a standard response. However, a guilty participant will recognize the correct item and is then forced to lie and hide the truth. If a participant replies a standard innocent response to crime related item, it implies innocents. On the other hand, if a participant has a significantly different response to a crime related item as compared with their own standard innocent responses, then this implies concealed knowledge.
The CIT contains a series of multiple-choice questions. Each question contains a relevant Probe item (e.g., a feature from the crime) and several neutral (irrelevant) alternatives, chosen so that the ‘innocent’ participant cannot distinguish the relevant item from a neutral item. Such as, asking the multiple choice question, “If you are the person who broke into the house, you know the colour of the front door. Repeat after me these colours of the door”. Participants are then asked to repeat a list of stimuli items read by an instructor who also does not know which items are critical or irrelevant (Krapohl, McCloughan, Senter, 2006). An innocent participant will show no significant differences between responses to the stimuli (color of the door) because they have no idea what color the door actually was. However, when the guilty subject acknowledges that the door actually was RED, their lie response will be significantly different than a standard irrelevant response. This variation in response is used to determine if the participant is concealing knowledge.
The CIT has a large average effect size (Cohen’s d = 1.55; Ben-Shakhar et al., 2003) advocating its ability for individual classification. Using standard restrictions, the CIT has low rates of false-positive errors (5% wrongful accusations) in both laboratory and field settings. However, it has much larger rates of false-negatives, particularly in field settings (up to 58% guilty individuals escaping detection; Elaad, 1990). Studies implementing a RT-CIT for detection have shown the sensitivity to be between 81-90%, with the specificity being between 85-98% (Seymour et al., 2000; Verschuere et al., 2010). While the CIT can accurately determine specificity, the sensitivity of the test can be improved. Thus far, attempts to
increase the sensitivity by means of combining response measures (Meijer et al., 2007; Gamer et al., 2008) has not led to a substantial improvement.
CIT: from SCR to RTs
The well documented CIT has shown great promise by easily and effectively detecting concealed information in research, as well as practical applications (Osugi, 2011). Japan currently implements CIT regularly during police investigations (Nakayama, 2002). However, the classic CIT implements skin conducting electrodes (SCR) to acquire information. Skin conductance responses are characterized by habituation; resulting in a decrease in responses with repeated presentation. Habituation restricts the length of the test and may hinder repeated testing. Skin conductance responses also demonstrate large individual differences, with some participants failing to show any response. (Dawson et al., 2007). The use of physiological measures also makes the entire testing system more complicated, expensive, and time consuming. In addition to the cost of special monitoring equipment and electrodes, testing must also involve a properly trained technician. Furthermore, the CIT has shown to be susceptible to counter measures, particularly when implementing SCR (Ben-Shakhar 2011). A counter measure is any technique applied in order to change the outcome of a test (physical or mental). Limitations in using physiological measures have led to the exploration of different methodical approaches.
In addition to SCRs, event related potentials (ERP) have also been investigated in order to determine concealed information. ERP’s are waveforms of neurological activation measured with an electroencephalogram (EEG). Whereas SCR measures can be influenced by factors irrelevant to item recognition (eg. Motor movement), ERP’s P300 component measures the cerebral process underlining cognition, attention and language (Hillyard & Picton, 1987). The P300 amplitude is inversely related to the probability of occurrence of a stimulus (Pritchard, 1981). A meta-analysis observing different detection methods for the CIT concluded that the P300 out preformed other measurement methods, with an average d of 2.55 (Ben-Shakhar, 2012). However, P300 testing has still been shown to be affected by countermeasures. Some studies implementing P300 also include reaction time (RT) analysis in an attempt to identify the use of countermeasures (Xu et al., 2012, Winograd and Rosenfeld, 2011). Consequently, Rosenfeld et al. (2008) developed the Complex Trial Protocol (CTP), which has shown to be resistant to both physical and mental countermeasures (Rosenfeld et al., 2008; Winograd and Rosenfeld, 2011). CTP testing also implements RT analysis to determine the use of countermeasures (Winograd and Rosenfeld,
2011)However, P300 procedures and analysis are more complex, expensive and time consuming, requiring an EEG and trained technician for testing as well as analysis.
As a more practical alternative, RT based testing has been investigated. RT based tests show great benefits due to its speed and ease of application. A test can be conducted in under 15 minutes and require no physiological measures, making them cheaper (only requires a single computer) as well as eliminates the need for a technician trained with skin conductance electrodes. Due to this potential, a variety of studies have been investigated on the validity of RT’s as a deterministic factor in CIT. Several EEG studies using ERP found that RTs successfully identified concealed information (Allen, Iacono, &
Danielson, 1992; Rosenfeld, Biroschak, & Furedy, 2006; Rosenfeld et al., 2004; Verschuere et al., 2008). Subsequently, there have been many studies investigating the use of RT measurements as a
deterministic factor in CIT. Some studies have failed to produce a RT variation between probe and irrelevant stimuli. As well as stating that RT’s can be manipulated by the participant and therefore invalid. (Farwel & Donchin, 1991) However, many of the previous studies have used other RT paradigms that do not require a response to a significant piece of concealed information. The behavioural tasks in these studies required a response based on a trivial aspect (such as the colour of stimuli, whether if the stimuli was a word or non-word) or to a physically isolated stimuli (pair of dots or auditory tones). Yet, significant probe-irrelevant differences have been seen in RT paradigms which implement a meaningful probe evoking a strong response. This is known as the failure of arousal theory, stating that the information must be critical enough to evoke a strong response in the participant. If they stimuli does not evoke a strong enough response, the probe-irrelevant difference is no longer seen.
Seymour, Shafto, & Mosmann, (2000) established a Reaction Time based Concealed Information Test (RT-CIT) paradigm implementing a mock-crime. In order to obtain a meaningful probe, participants executed a mock crime prior to the experiment. Using the meaningful information from the mock crime, a significant probe-irrelevant difference was seen, resulting in an overall discrimination accuracy of 95%. Seymour et al., (2000) concluded that the RT-based test for concealed information detection is a viable alternative to the polygraph. Seymour and Kerlin (2008) recently replicated the previous finding stating that RTs alone are highly valid in concealed information detection. Verschuere et al., (2010) conducted a CIT and polygraph comparison study, which corroborated that a RT-based test can be effectively
implemented without concurrent physiological recordings (Seymour & Kerlin, 2008; Seymour et.al, 2000). Application of RT based data analysis without the need for coinciding physiological measurements results in the CIT becoming quicker and easily executed.
The mechanisms of the CIT
The most well established theory behind the CIT implements orientating responses (OR) and subsequent habituation (Lykken, 1974). The OR is a person’s reaction to a novel stimulus or change in stimulus. Therefore, repeated presentation of a stimulus results in a gradual decline in response magnitude, known as habituation (Ben–Shakhar & Gati, 2003). The dominate theoretical approach proposed to account for orientation and habituation is known as the comparator theory (Siddle, 1991), stating that the OR is the result of a comparison between stimulus input and expectations. Lykken (1974) deduced that the properties of OR enables them with the potential ability to determine guilty knowledge. He argued that “for the guilty subject only, the ‘correct’ alternative will have a special significance, an added ‘signal value’ which will tend to produce a stronger orienting reflex than that subject will show to other alternatives” (p. 728). Meaning that knowledge of an item endows it with significance (signal value), which results in those particular items evoking stronger ORs relative to an irrelevant item. Participants’ with no knowledge of the stimuli will show a standard OR response and demonstrate no increased signal value.
Additionally, the Response Inhibition (RI) theory may contribute in CIT lie detection, especially when observing reaction times. RI theory states that since truth is the predominate response; suppression of this response in order to lie causes an increased cognitive load. The CIT capitalizes on this
phenomenon by observing the aberrant lie response of a participant compared with their own predominate truthful responses. The response for a irrelevant stimulus will be quick and easy as it is the brains
standard truth response. However, during a lie trial, suppression of this predominate truth response results in a significantly slower and more demanding response. We aim to capitalize on this phenomenon through the use of proportion manipulation. By increasing the proportion of truthful responses (irrelevant stimuli) and reducing the frequency of lies (probe stimuli), we aim to make the aberrant lie response even more difficult for the participant, as determined by a relative increase of RT to the lie response. By increasing the difficulty of a lie response, we hope to obtain a larger variation in RT between an irrelevant and a probe item. This will result in a more precise CIT with less false negatives, aking it more applicable for real world applications.
Proportion Effect Manipulation
Prior research with similar cognitive tasks have shown that the strength of the dominate response can be manipulated though proportion effects (Blais, 2012). A well investigated cognitive test that
efficiently incorporates proportion effect is the Stroop test. The Stroop effect (deriving from the test) is the interference on reaction times due to trial congruency manipulation. The magnitude of the Stroop effect can be manipulated by the number of overall congruent trials, known as list-wide proportion congruent manipulation (LWPC). LWPC demonstrates that Stroop effects are larger for mostly congruent compared with mostly incongruent trials. (e.g., Logan, Zbrodoff and Williamson, 1984; Lindsay and Jacoby, 1994; Kane and Engle, 2003) An increase in the Stroop effect can be seen when the proportion of congruent trials is high (Lindsay & Jacoby, 1994). By increasing the number of congruent trials during a Stroop test, a participant’s response becomes more automatic, requiring greater recourses when suppressing the automatic response on incongruent trials.
Van Bockstaele et al., 2012 has demonstrated that proportion manipulation can make it more or less challenging for a participant to lie. The team attempted to increase cognitive load in order to make the aberrant lie response more challenging. To do this, the group used a Sheffield Lie test in which they manipulated the proportion of truth to lie responses. During the test phase, deception was easier for participants in the frequent lie group (75% lies) and harder for participants in the frequent truth group (75% truth). However, these results were limited to the information used during the training session. Verschuere et al., 2011 also demonstrated that lying becomes more difficult as the proportion of truth increases, as well as, lying becomes easier when the proportion of lies increase; corroborating that a participant’s automatic response is malleable through proportion manipulation. (eg. Ben-Shakhar, 1977)
The Present Study
Manipulation of the RT-CIT proportion effect could result in a more sensitive test. This idea is supported in the RI theory which attests that by increasing the amount of truthful responses (congruent trials), the participant will be more accustomed to truthful responses. Since truth is the natural dominate response, the deviant lie response will become even more cognitively demanding, requiring increased activation in working memory, inhibitory control and/or task switching areas; thus resulting in a slower response time. By increasing the amount of congruent trials (truth), a rare incongruent trail (lie) should elicit a greater cognitive demand which slows down the participants RT. Additionally, the OR theory states that an increase in response is seen when a stimuli opposes expectations. Therefore, an increase in the irrelevant proportions will cause and increase in expectation of an irrelevant (truthful) response. When a probe item appears requiring a lie response, the adverse of expectation will result in a slower response. By increasing the amount of irrelevant stimuli (congruent trials) to probe stimuli (incongruent trials) the participant should be more compelled to expect an irrelevant trial, resulting in a more drastic OR to probe stimuli. Therefore we would like to elucidate if an increase in the proportion of irrelevant stimuli vs. probe stimuli during an autobiographical CIT will result in a significantly larger contrast in mean RT values when comparing irrelevant vs. probe stimuli.
An increase in differential responding will be achieved by altering the proportion of irrelevant to probe stimuli. The standard RT-CIT method implements a 1 probe and 1 target for every 4 irrelevant stimuli proportion. We will use this as the control, Group 1:4. The first experimental group will be Group 1:8, and will consist of 1 probe and 1 target for every 8 irrelevant stimuli. All eight of these irrelevant stimuli will be different. The second experimental group, Group 1:4x2 will contain 1 probe and 1 target for every 8(4x2) irrelevant stimuli. The 4x2 will consist of 4 irrelevant stimuli that are repeated twice per trial. This repetition is being used because it can sometimes be difficult to find 8 irrelevant items that correctly match a probe item. Therefore we would also like to observe if the 1:4x2 group has similar results to the 1:8 group. RT and ER results of all blocks will be analyzed between subjects.
Methods
Participants:
Thirty five subjects participated in this study at the University of Amsterdam. The participants age ranged from 18-68, with the average age being 29.46 (SD= 13.89), containing nearly half (51%) male. Group 1:4 contained n=10 participants, group 1:8 contained n=11 and group 1:4x2 contained n=14. The study was approved by the local IRB, and informed consent was obtained from each participant prior to the experiment. Subjects received a reward of 0.5 Research Credits for participation.
Table 1
Amount of participants, mean age, and percent male per condition.
Condition # of Participants Mean Age Amount Male (%)
1:4 10 32.60 30
1:8 11 24.18 55
1:4x2 14 31.35 64
Apparatus:
Stimuli were presented and RT was recorded by a 60 Hz PC, using Inquisit 4 software (2013). The software allows recording of error rates as well as RT’s with millisecond accuracy. The data was analysed using SPSS version 20.0 (IBM Software, Armonk, NY, U.S.).
Procedures:
After providing a written informed consent, participants were asked to fill out a short
autobiographical survey containing first name, last name, father’s name, mother’s name, birthday as well as any other personally significant names and dates. The personal information from their survey was then implemented into their personalized RT-CIT as probe stimuli and crossed checked to insure no irrelevant stimuli overlapped with significant names and dates. If an overlap occurred, then the problematic
irrelevant stimuli were changed. Participants were informed that they will be tested in a lie detection based paradigm. Participants were randomly assigned evenly into one of the three conditions.
Target Learning Phase: CIT
In order to complete the RT-CIT test, participants were first asked to memorize a set of target items. Participants were informed that they are to memorize these items for use in the next test. The participant was placed in front of a standard computer in an isolated room. The learning phase consisted of a set of five stimuli (target stimuli) being presented simultaneously on the screen for 30 seconds (see
Table 2). Learning phase was the same for all three conditions but varied between genders. Men and
first name. Males received sets of male names and females received sets of female names (where appropriate)
Table 2
Set of target items that the participants memorized during the learning phase. Male subjects received Female subjects received
First Name DAN ANNA
Surname CRUISE VISSER
Father’s name ADAM IAN
Mother’s name GRACE LUCY
Date of Birth 15 MARCH 8 JULY
After this learning phase, memory of the five target items was assessed. After each presentation, participants were instructed to type the words they had previously memorized individually as instructed (eg. “First name?”). The whole learning and recall procedure was repeated two more times in order to ensure memorization. After completion of the training phase, the RT-CIT testing began.
RT- CIT
During the entire task a heading ‘DO YOU RECOGNIZE THIS STIMULUS?’ and the response labels ‘YES’ and ‘NO’ are presented on the screen; having “Q” representing ‘YES’ and “M” representing ‘NO’ on a standard QWERTY keyboard. Stimuli are presented as text in white capital letters on a black screen for 800 milliseconds. If the 800 milliseconds are exceeded before a response, ‘TOO SLOW’ in red appears on the screen for 1 second. The time between stimuli presentation was a random selection of 500, 800 or 1000 milliseconds. Participants were informed that pressing ‘YES’ meant recognition and ‘NO’ meant non-recognition of the stimuli. Participants were instructed to only respond ‘YES’ to the five items that they have just memorized (target items), and the ‘NO’ for all other items (irrelevant and probe).
All conditions started with a ‘Practice’ block that is not included in analysis and is intended to acclimate the participant to the program. Participants in group 1:4 were tested in two blocks of 90 trials per block. Each trial contains 1 probe, 1 target or 1 irrelevant stimulus. Participants in group 1:8 and 1:4x2 were tested in two blocks of 150 trials per block. In group 1:8, each trial also contained 1 probe, 1 target or 1 irrelevant stimulus. Group 1:4 contains an overall ratio of four irrelevant for every one probe. Group 1:8 and 1:4x2 contains an overall ratio of eight irrelevant for every one probe (see Table 3-5).
Table 3
Condition 1:4: Control- One probe appears for every four irrelevant stimuli.
Stimulus Stimulus type Required response
ERICK Irrelevant ‘No’
BENJAMIN Probe ‘No’
RYAN Irrelevant ‘No’
DAN Target ‘Yes’
ANDREW Irrelevant ‘No’
JAN Irrelevant ‘No’
Table 4
Condition 1:8- One probe appears for every eight irrelevant stimuli. All eight irrelevant stimuli are different.
Stimulus Stimulus type Required response
ERICK Irrelevant ‘No’
ANDREW Irrelevant ‘No’
RYAN Irrelevant ‘No’
DAN Target ‘Yes’
PAUL Irrelevant ‘No’
JAN HARRY BENJAMIN ADAM VICTOR Irrelevant Irrelevant Probe Irrelevant Irrelevant ‘No’ ‘No’ ‘No’ ‘No’ ‘No’
Table 5
Condition 1:4x2 - One probe appears for every eight irrelevant stimuli. Four irrelevant items are repeated twice to obtain eight irrelevant per probe.
Stimulus Stimulus type Required response
ERICK Irrelevant ‘No’
ANDREW Irrelevant ‘No’
RYAN Irrelevant ‘No’
DAN Target ‘Yes’
ANDREW Irrelevant ‘No’
JAN RYAN BENJAMIN ERICK JAN Irrelevant Irrelevant Probe Irrelevant Irrelevant ‘No’ ‘No’ ‘No’ ‘No’ ‘No’
For condition 1:4 (control) each of the five target items was repeated six times, totaling 30 target items or 1/6 of the trials. Each of the five probes was repeated six times, totaling 30 probes or 1/6 of the trials. Each of the 20 irrelevant items was repeated six times, totaling 120 irrelevant items or 4/6 of the trials. Resulting in a test consisting of 180 trials, with a 1:4 proportion of probe to irrelevant items.
In condition 1:8 each of the five target items was repeated six times, totaling 30 target items or 1/10 of the trials. Each of the five probes was repeated six times, totaling 30 probes or 1/10 of the trials. Each of the 40 irrelevant items was repeated six times, totaling 240 irrelevant items or 8/10 of the trials. Resulting in a test consisting of 300 trials, with a 1:8 proportion of probe to irrelevant items.
For condition 1:4x2 each of the five target items was repeated six times, totaling 30 target items or 1/10 of the trials. Each of the five probes was repeated six times, totaling 30 probes or 1/10 of the trials. Each of the 20 irrelevant items was repeated twelve times, totaling 240 irrelevant items or 8/10 of the trials. Resulting in a test consisting of 300 trials, with a 1:8 proportion of probe to irrelevant items.
As you can see, group 1:4 has less overall trials than 1:8 and 1:4x2. It is designed this way in order to obtain the same amount of probe and target stimuli across conditions, resulting in all three conditions containing 30 probes and 30 targets. Thereby, making the amount of irrelevant stimuli the only discrepancy between conditions, this should isolate the amount of irrelevant stimuli as the deterministic factor during analysis.
After the participants completed the testing, they were asked to take a post test survey. The survey asked them to rate on a scale of 0-100 (easy-hard) the challenges of the task. Such as “How hard was it to respond NO to your personal information”. It also asks the participant to recall the Target names again, in order to confirm their knowledge throughout the testing. As well as recall any personal items they may have detected during the test.
Results
RTIn order to reduce the impact of outlying values, RTs faster than 150ms and slower than 800 ms were excluded from analysis (6%). Additionally, incorrect responses (responding ‘no’ to target or ‘yes’ to probe or irrelevant) were excluded from RT analysis. A 2 (Block: 1 x 2) x 2 (stimulus: Probe x Irrelevant) x 3 (Conditions: 1:4 x 1:8 x 1:4x2) repeated measure ANCOVA utilizing age as a covariate was conducted. Analysis revealed an effect for item F (1, 31) =45.05, p<.001 demonstrating significantly slower mean RTs to probe( Block 1 M=556.02 SD=9.51, Block 2 M= 514.85 SD=7.92) than to irrelevant stimuli ( Block 1: M=493.74 SD=7.42; Block 2: M= 482.52 SD=8.18) (see Figure 1). This effect was seen though all conditions, demonstrating the CIT effect. However, there was no main effect of conditions, F (2, 31) = .74,
p = .487, indicating no significant difference in mean RTs between conditions. The covariate, age, was
significantly related to the participants RT, F (1,31) = 9.99 p = .004. With older participants having generally slower responses. No block effect of was seen between blocks F (1, 31) =3.970, p=.055 resulting in no significant increase in RT from block one to block two.
Figure 1. Mean RTs (ms) to the probe and irrelevant stimuli across participants per condition. The standard CIT effect is seen by the increased RT for probes compared with irrelevant. No differences were seen between conditions. Error bars represent standard errors.
420 440 460 480 500 520 540 560 1;4 1;8 1;4X2 M e an R e ac tion Ti m e ( m s) Conditions Probe Irrelevant
ER
ER analysis included incorrect responses (responding ‘no’ to target or ‘yes’ to probe or irrelevant) and participants with more than 35% overall ER were excluded from the trial (n=3). A 2 (Block: 1 x 2) x 2 (stimulus: Probe x Irrelevant) x 3 (Conditions: 1:4 x 1:8 x 1:4x2) repeated measure ANCOVA utilizing age as a covariate was conducted. The analysis revealed no effect of stimulus F (1, 31) =1.36, p=.252, indicating no significant difference between probe are irrelevant ERs, see Figure 2. The condition effect was also not significant, F(2,31) = .45, p = .642, indicating that there was no significant variation in ER between conditions see. The covariate, age, was significantly related to the amount of errors, F (1,31) = 4.66, p =.039. Older participants had increased amounts of ER. No block effect was seen F (1, 31) = 3.99,
p = .055 indicating no difference in ER between blocks.
Figure 2. Mean ER (%) for probe and irrelevant trials, across participants, per condition. The standard CIT effect is not seen indicating no significant increase in errors for probes compared with irrelevant stimuli. No differences were seen between conditions. Error bars represent standard errors.
Supplemental ER - Age
Due to an age effect during analysis, a supplement analysis was ran excluding participants more than 1 standard deviation (13.89) years of age away from the mean age (29.76). This excluded 5
participants, all which were over the age of 55. A 2 (Block: 1 x 2) by 2 (stimulus: Probe x Irrelevant) by 3 (Conditions: 1:4 x 1:8 x 1:4x2), repeated measure ANOVA revealed no effect of stimulus F (1, 27) =3.53,
p=.071, indicating no significant difference between probe are irrelevant ER responses. The main
condition effect was also not significant, F(2,27) = 0.62, p = .544, indicating that there was no significant 0% 2% 4% 6% 8% 10% 12% 14% 1;4 1;8 1;4x2 M e an E rr o r r ate (% ) Conditions Probe Irrelevant
variation in ER between conditions. Due to no change in results with or without the 55+ participants data, supplemental analysis confirms that age did not significantly impact the results of this experiment.
First 60 trials RT
RT-CIT testing is known to contain habituation effects, which can alter the results of testing. Therefore, a first 60 trials supplementary analysis was conducted.. Additionally, this analysis eliminates the discrepancy in the amount of trials between 1:4 and 1:8/1:4x2 as mentioned above. Analysis of the first 60 trials should eliminate the habitation effect seen in later testing. The first 60 trials contains the first 60 irrelevant items, first 15 probe and the first 15 target. The trials are acquired prior to any processing (removal of time errors, incorrect response errors, ect.) A 2 (stimulus: Probe x Irrelevant) x 3 (Conditions: 1:4 x 1:8 x 1:4x2) repeated measure ANCOVA utilizing age as a covariate was conducted, revealing an effect of stimulus F (1, 31) =14.68, p=.001, demonstrating significantly slower mean RTs to probe ( M=547.76, SD=9.12) than to irrelevant stimuli ( M= 492.70 SD=9.18) see Figure 3. This effect was seen though all conditions. This analysis also found no main effect of conditions, F(2, 31) = .32, p = .732, indicating no significant difference in mean RTs between conditions. No significance was seen when comparing age with stimulus response F (1, 31) =2.00, p=.168
Figure 3 First 60 trial analysis, mean RT for probe and irrelevant trials per condition. The standard CIT effect is seen by the increased RT for probes compared with irrelevant. No differences were seen between
conditions. Error bars represent standard errors. 420 440 460 480 500 520 540 560 580 1;4 1;8 1;4x2 M e an R e ac tion Ti m e ( m s) Conditions Probe Irrelevant
First 60 trials ER
A 2 (stimulus: Probe x Irrelevant) x 3 (Conditions: 1:4 x 1:8 x 1:4x2) repeated measure ANCOVA utilizing age as a covariate was conducted, revealing an effect of stimulus F (1, 31) =5.23, p=.029 demonstrating significantly more errors to probe (M=.067 SD=.020) than to irrelevant (M=.008 SD=.002) stimuli (see Figure 4). This effect was seen though all conditions. However, there was no main interaction effect of conditions, F(2, 31) = .42, p = .662, indicating no significant difference in mean ERs between conditions . A significant age effect was seen F(1, 31) = 13.74, p =.001 indicating that age had an effect on ERs (increased ER).
Figure 4. First 60 trial analysis, mean ER for probe and irrelevant trials per condition. The standard CIT effect is shown by the increased amount of errors for probes compared with irrelevant. Error bars represent standard errors.
Post-test survey
A post-test survey was conducted in order to help elucidate any unforeseen complications or interactions; as well as to confirm proper memorization of the target stimuli. The survey asked participants to answer questions by rating on a scale of 0-100 (0 being easiest and 100 being most challenging). The questions consisted of, how difficult it was to say ‘NO’ to personal information, ’NO’ to irrelevant
information, ‘YES’ to memorized information, stress level of the test, over all difficulty, and self-reported accuracy (in percentage). As well as three free response questions asking participants to name the memorized items (targets), personal items (probes) and other items (irrelevant) that they noticed during testing.
The post-test survey was completed by 27 of the 35 participants. The eight missing participants were part of the pilot study which predated the post-test survey. Individual one way ANOVAS were ran on
0% 2% 4% 6% 8% 10% 12% 14% 16% 1;4 1;8 1;4x2 M e an E rr o r R ate (% ) Conditions Probe Irrelevant
each question. No group differences were seen across any of the six questions, indicating that all three test were equally balanced (see Table 6).
Table 6
Average responses to the post-test survey on a scale of 0-100, with 0 being easiest and 100 being most challenging.
Difficulty of
Probe
Difficulty of
Irrelevant
Difficulty of
Target
Stress
Accuracy
(%)
Overall Difficulty
Group Mean(SD) Mean(SD) Mean(SD) Mean(SD) Mean(SD) Mean(SD)
1:4 63.13 (18.89) 26.25 (13.30) 51.88 (18.70) 42.50 (26.05) 81.25 (9.16) 46.88 (22.83) 1:8 52.78 (18.63) 26.11 (11.12) 43.89 (15.37) 32.78 (16.98) 80.00 (7.91) 42.22 (16.22)
1:4x2 56.50 (25.06) 23.80 (17.48) 56.00 (22.09) 41.20 (26.03) 79.80 (7.48) 45.10 (26.71)
Discussion
The present study attempts to investigate if the addition of irrelevant stimuli results in an increased standard deviation between average RTs for probe vs. irrelevant stimuli, during an autobiographical CIT. Doing so would result in increased sensitivity during RT-CIT and less false-negatives. As expected, the CIT effect was present, indicated by the significant differences in RT between probe vs. irrelevant stimuli, indicating concealed knowledge. However, the goal of this experiment was not reached, as we did not find a significant increase in probe response, for neither RT nor ER between the conditions. For the initial hypothesis to be accepted, we should have seen a slowing in mean RT for probes when comparing the control condition (1:4) with either experimental conditions (1:8 & 1:4x2). While there was no variation seen between conditions, this test did show proper CIT effect for all three conditions.
The CIT effect
A strong CIT effect was seen for all RT analysis with a significance of at least p=.001. This is a favourable result as is it is accordance with the CIT effect in other findings. This effect is supported by other recent research which also reveal at least p=.001 significance when comparing probes to irrelevant RTs (e.g., Seymour et al., 2000, Verschuere, Crombez, & Koster, 2004). The CIT effect for RT in this study appears to be strong, valid and in accordance with other similar studies. This study supports the idea that RT can be used to determine discrepancies between probe and irrelevant items in a CIT. As well as supporting the failure of arousal theory, by obtain a proper probe-irrelevant variance using critical items that the participant must deny knowledge of.
We did not see CIT effect for ER during full test analysis. This is not unexpected as RT’s have been shown to be more valid than ER (Noordraven, & Verschuere, 2013). The CIT effect was seen during the first 60 trials analysis of ER F (1, 31) =5.23, p=.029. While the full analysis came back insignificant, the CIT effect seen during the first 60 trials ER is on par with other literature. ER reported in similar studies resulted in a significance of p<.05 ( Verschuere et al., 2010, Rosenfeld, et. al., 2008). While the CIT effect for ER during the full analysis fell short, the supplementary analysis revealed a strong ER significance that is as good or better then reported in other findings. Significance seen during the first 60 trials analysis might be due to lack of habituation. Additionally, participants in conditions 1:8 and 1:4x2 had more time to acclimate to the test. By analyzing the first 60 trials, all participants are placed on an equal template. Since ER significance is seen in first 60 and not in the full analysis, the difference in amount of trials per block appears to contribute to analysis.
Failure to Respond
The failure to respond theory also appears to play a role here. Verschuere et al., (2010) stated that to achieve proper RT-CIT effect results, an RT task needs to impose a response conflict. Without a strong response conflict, probe-irrelevant differences have not been seen as demonstrated in Meijer et al., (2007). The Verschuere et al., (2010) study altered the instructions given in an attempt to annul the response conflict. Half of the participants were instructed that ‘yes’ meant recognition and that ‘no’ means non-recognition (typical instructions as used in this study). Thereby, when a participant responds ‘no’ to a probe, they are engaged in deception. Whereas the other half of the participants in the study received instructions that ‘yes’ means target recognition, and that ‘no’ simply means that it was a non-target. Resulting in these participants not actually being engaged in deception. Analysis showed that the Probe-irrelevant difference was stronger under the typical instructions (d=1.23) compared with the revised instructions (d=.80). The current study did not have any problems exciting enough of a reaction in order to achieve the CIT effect. However, failure to respond theory shows that if the information is not critical enough, then response is not shown. Therefore, future testing should attempt to involve a strong
deception response conflict in order to achieve optimum results. By increasing the response conflict, one can expect the CIT effect to also increase.
Proportion manipulation
Conditions 1:8 and 1:4x2, which had an additional increase in the irrelevant to probe stimuli proportion, showed no significant increase in RT nor ER when comparing the irrelevant and probe stimuli with the control 1:4. This does not support the hypothesis that, additional irrelevant stimuli can increase the mean difference between a probe and irrelevant stimuli RT and/or ER response. There are a few possible explanations for why the proportion manipulation failed to achieve an increase in RT. One possible problem could be due to the small sample size used in this study. Ideally we would have liked to have at least 60 participants over all to make it a more complete sample. This study only managed to collect useable data from 35 participants. In addition to the small sample, the sample age also varied
drastically, with the youngest participant being 18 and the oldest 68. Since ER’s showed to increase with age, sample size should be confined within a predetermined age range.
Additionally it appears that the different number of trials played a role. Condition 1:4 had 90 trials per block, where conditions 1:8 and 1:4x2 had 150 trials per block. This was done in order to achieve the same amount of probe and target items, while increasing the amount of irrelevant stimuli. However, the difference in overall amount of trials might have contributed during analysis. The entire test for participants in the control group was 60% the length of the experimental conditions. Participants in the experimental conditions would have had more questions to answer over all, must focus longer and keep the targets memorized longer. All which could possibly lead to variations in testing as well as habituation and subsequent weakening of the CIT effect. Eliminating the number of trials difference with the First 60 supplementary analyses increased the significance of ER, meaning that the varying lengths of the three test affected ER analysis. Further proportion manipulation investigations should be careful about a difference in overall trials between conditions.
Limitations
This study did not achieve the desired variation between conditions via proportion manipulation. Therefore, the uses of this paradigm are limited. One possible limitation with in this study appears to be the difference in amount of trials between the control and experimental condition. The trial amount difference was a factor as shown by the first 60 trials analysis. Having varying amount of trials per conditions changes participants’ habituation between conditions, as well as their overall experience; such as how long they much: focus their attention, memorize targets and accurately execute the test.
Also, results could have been due to a failure to increase signal value for the probe. By
decreasing the appearance of the probe stimuli, we expected to cause an increase in OR when a probe does appear. However, it appears to have not been successful due to the coequal results across conditions. This could have been due to insufficient signal value for the probe. Signal value has been shown to contribute to accuracy and efficacy of a CIT. The use of autobiographical information may not contain enough signal value in this context to evoke variations between conditions. Autobiographical information it´s self may not be able to have an increase in OR attributed to it, due to the common application of this information
Future research
With no conditional difference being seen in this paradigm, future research should implement a alterations. Studies have shown that the stimuli must create a sufficient conflict in order to achieve a reliable probe-irrelevant difference. Participants engage in a response conflict when they must reclassify a recognized object and respond non-recognition. While we did sufficiently obtain a probe-irrelevant
difference, use of more critical information could increase the RT/ER enough to observe a difference between conditions. By increasing the difficulty of reclassifying a critical item as irrelevant, the rare probe item should become even more challenging to reply to amidst the increased amount of non-recognized
irrelevant items. Since it appears that autobiographical information did not achieve an increased OR for probe stimuli, future research should implement more salient probes. The use of more salient probes that are not common (such as your own name), should have stronger signal value. Probes with increased signal values (such as in a mock crime) should achieve a strong OR response when compared with irrelevant stimuli. Applying a proportion manipulation paradigm with salient probes could increase the OR enough in order to view a difference between conditions. Additional use of P300 EEG testing would help to elucidate if an increase in arousal and increase in irrelevant stimuli proportion has an effect on the overall CIT effect between conditions. Since a P300 based test has shown a direct relationship with frequency of stimuli (possibly same mechanism as OR) the addition of more irrelevant stimuli should make the P300 even stronger when confronted with a relatively rare probe stimuli (Pritchard, 1981). The
application of more critical information such as in a mock-crime could delay habitation and increase probe OR, resulting in a more complete test.
Also the amount of trials per condition appeared to contribute as seen in the first 60 analysis. Future testing should look into designing a proportion manipulation paradigm in which all of the conditions have equal amount of trials. Having an equal amount of trials across conditions would help to control any habituation and testing bias that one condition may have over another due to unequal lengths.
Conclusions
The RT-CIT has shown great promise in its ability to quickly and effectively determine concealed information. In order to make the RT-CIT more applicable for real world use, we investigated increasing the sensitivity through proportion manipulation. The test accurately showed that a proper CIT effect is achieved when you increase irrelevant stimuli. Additionally that the experimental conditions 1:8 and 1:4x2 resulting in similar data. However, we failed to observe a significant difference between the RT-CIT sensitivity between conditions, through increasing the amount of irrelevant stimuli. Failure to achieve the desired results could have been due to a few factors such as varying block length, use of autobiographical information and sample size. However, previous evidence supports the idea that proportion manipulation can increase the CIT effect. By applying the mentioned modifications, future investigation of proportion manipulation could still result in the CIT becoming more precise and applicable. Resulting in the overall goal of a precise, sensitive, easy and fast way of determining deception.
References
Abootalebi, V., Moradi, M. H., & Khalilzadeh, M. A. (2009). A new approach for EEG feature extraction in P300-based lie detection. Computer Methods and Programs in Biomedicine, 94(1), 48-57. Allen, J. J., Iacono, W. G., & Danielson, K. D. (1992). The Identification of concealed memories
using the event-related potential and implicit behavioral measures—A methodology for prediction in the face of individual-differences. Psychophysiology, 29, 504–522.
Ben-Shakhar, G. (2012). Current research and potential applications of the concealed information test: an overview. Frontiers in psychology, 3.
Ben–Shakhar, G., & Gati, I. (2003). The effects of serial position and frequency of presentation of common stimulus features on orienting response reinstatement. Psychophysiology, 40(1), 139-145.
Ben-Shakhar, G., & Elaad, E. (2002). The Guilty Knowledge Test (GKT) as an application of
psychophysiology: Future prospects and obstacles. Handbook of polygraph testing, 87-102. Ben‐Shakhar, G., Bar‐Hillel, M., & Lieblich, I. (1986). Trial by polygraph: Scientific and juridical issues in lie
detection. Behavioral Sciences & the Law,4(4), 459-479.
Ben‐Shakhar, G. (1977). A further study of the dichotomization theory in detection of information. Psychophysiology, 14(4), 408-413.
Blais, C., Harris, M. B., Guerrero, J. V., & Bunge, S. A. (2012). Rethinking the role of automaticity in cognitive control. The Quarterly Journal of Experimental Psychology, 65(2), 268-276. Bouton, M. E. (2007). Learning and behavior: A contemporary synthesis. Sinauer Associates..
Bugg, J. M. (2012). Dissociating Levels of Cognitive Control The Case of Stroop Interference. Current
Directions in Psychological Science, 21(5), 302-309.
Christ, S. E., Van Essen, D. C., Watson, J. M., Brubaker, L. E., & McDermott, K. B. (2009). The contributions of prefrontal cortex and executive control to deception: evidence from activation likelihood estimate meta-analyses. Cerebral Cortex, 19(7), 1557-1566.
Dawson, M. E., Schell, A. M., & Filion, D. L. (2007). 7 The Electrodermal System. Handbook of
psychophysiology, 166-167.
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Psychological bulletin, 129(1), 74.
Ford, C. V., & Price, J. S. (1996). Lies!, lies!!, lies!!!: The psychology of deceit(p. 118). Washington, DC: American Psychiatric Press. Washington, DC.
Furedy, J. J., & Heslegrave, R. J. (1988). Validity of the Lie Detector A Psychophysiological Perspective. Criminal Justice and Behavior, 15(2), 219-246.
Gamer, M., Verschuere, B., Crombez, G., & Vossel, G. (2008). Combining physiological measures in the detection of concealed information. Physiology & Behaviour, 95(3), 333-340.
Gronau, N., Ben-Shakhar, G., & Cohen, A. (2005). Behavioural and physiological measures in the detection of concealed information. Journal of Applied Psychology, 90, 147–158.
Hillyard, S. A., & Picton, T. W. (1987). Electrophysiology of cognition. Comprehensive Physiology. Hu, X., Hegeman, D., Landry, E., & Rosenfeld, J. P. (2012). Increasing the number of irrelevant stimuli
increases ability to detect countermeasures to the P300‐based Complex Trial Protocol for concealed information detection.Psychophysiology, 49(1), 85-95.
Iacono, W. G., & Patrick, C. J. (1988). Assessing deception: Polygraph techniques.
IBM Corp. Released 2011. IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY: IBM Corp Inquisit 4.0.3.0 [Computer software]. (2013). Seattle, WA: Millisecond Software.
Kane, M. J., & Engle, R. W. (2003). Working-memory capacity and the control of attention: the
contributions of goal neglect, response competition, and task set to Stroop interference. Journal of
experimental psychology: General,132(1), 47.
Krapohl, D. J., McCloughan, J. B., & Senter, S. M. (2006). How To Use the Concealed Information Test. Polygraph, 35(3), 123-138.
Larson, J. A. (1932). Lying and its detection. Chicago: University of Chicago Press. Lindsay, D. S., & Jacoby, L. L. (1994). Stroop process dissociations: The relationship
between facilitation and interference. Journal of Experimental Psychology: Human Perception and
Performance, 20, 219–234.
Logan G. D., Zbrodoff N. J., Williamson J. (1984). Strategies in the color-word Stroop task. Bull. Psychon. Soc. 22, 135–138.
Meijer, E. H., Smulders, F. T., Johnston, J. E., & Merckelbach, H. L. (2007). Combining skin conductance and forced choice in the detection of concealed information. Psychophysiology, 44(5), 814-822. Meijer, E. H., Smulders, F. T., Merckelbach, H. L., & Wolf, A. G. (2007b). The P300 is sensitive to
concealed face recognition. International Journal of Psychophysiology, 66(3), 231-237. Nakayama, M. (2002). Practical use of the Concealed Information Test for criminal investigations in
Japan. In M. Kleiner (2011), Handbook of Polygraph Testing. San Diego: Academic Press. Noordraven, E., & Verschuere, B. (2013). Predicting the Sensitivity of the Reaction Time‐based
Concealed Information Test. Applied Cognitive Psychology.
Osugi, A. (2011). 14 Daily application of the Concealed Information Test: Japan. Memory detection:
Theory and application of the Concealed Information Test, 253.
Pritchard, W. S. (1981). Psychophysiology of P300. Psychological bulletin,89(3), 506.
Rosenfeld, J. P., Labkovsky, E., Winograd, M., Lui, M. A., Vandenboom, C., & Chedid, E. (2008). The Complex Trial Protocol (CTP): A new, countermeasure‐resistant, accurate, P300‐based method for detection of concealed information.Psychophysiology, 45(6), 906-919.
Rosenfeld, J. P., Biroschak, J. R., & Furedy, J. J. (2006). P300-based detection of concealed
autobiographical versus incidentally acquired information in target and non-target paradigms.
International Journal of Psychophysiology, 60, 251–259.
Rosenfeld, J. P., Soskins, M., Bosh, G., & Ryan, A. (2004). Simple, effective countermeasures to P300‐based tests of detection of concealed information. Psychophysiology, 41(2), 205-219.
Saxe, L., Dougherty, D., & Cross, T. (1985). The validity of polygraph testing: Scientific analysis and public controversy. American Psychologist, 40(3), 355.
Seymour, T. L., & Kerlin, J. R. (2008). Successful detection of verbal and visual concealed knowledge using an RT-based paradigm. Applied Cognitive Psychology, 22, 475–490. Seymour, T. L., Seifert, C. M., Shafto, M. G., & Mosmann, A. L. (2000). Using response time
measures to assess ‘‘guilty knowledge’’. Journal of Applied Psychology, 85, 30–37. Van Bockstaele, B., Verschuere, B., Moens, T., Suchotzki, K., Debey, E., & Spruyt, A. (2012).
Learning to lie: Effects of practice on the cognitive cost of lying. Frontiers in psychology, 3.
Verschuere, B., & De Houwer, J. (2011). Detecting concealed information in less than a second: response latency-based measures. In B. Verschuere, G. Ben-Shakhar & E. Meijer (Eds.),Memory Detection
Theory and Application of the concealed information test (pp. 46-50). Cambridge, NY: Cambridge
University Press.
Verschuere, B., Spruyt, A., Meijer, E. H., & Otgaar, H. (2011). The ease of lying. Consciousness and
cognition, 20(3), 908-911
Verschuere, G. Ben-Shakhar & E. Meijer (2011), Memory Detection Theory and Application of the
concealed information test. Cambridge, NY: Cambridge University Press.
Verschuere, B., Crombez, G., Degrootte, T., & Rosseel, Y. (2010). Detecting concealed information with reaction times: Validity and comparison with the polygraph. Applied Cognitive
Psychology, 24(7), 991-1002.
Verschuere, B., Rosenfeld, J. P., Winograd, M. R., Labkovsky, E., & Wiersema, R. (2009). The role of deception in P300 memory detection. Legal and Criminological Psychology, 14(2), 253-262. Verschuere, B., Crombez, G., Smolders, L., & De Clercq, A. (2009b). Differentiating orienting and
defensive responses to concealed information: the role of verbalization. Applied psychophysiology
and biofeedback, 34(3), 237-244.
Verschuere, B., Crombez, G., De Clercq, A., & Koster, E. H. (2004). Autonomic and behavioral responding to concealed information: Differentiating orienting and defensive
responses. Psychophysiology, 41(3), 461-466.
Winograd, M. R., & Rosenfeld, J. P. (2011). Mock crime application of the Complex Trial Protocol (CTP) P300‐based concealed information test.Psychophysiology, 48(2), 155-161.
