The Paradoxical Effects of Checking on Checking Cognitions and
Overestimation of Threat
Sophie A. Rameckers University of Amsterdam
Date: 27-05-2016
Table of Contents Abstract p. 3 Introduction p. 3 Methods p. 9 Participants p. 9 Materials p. 10 Procedure p. 12 Results p. 14 Discussion p. 20 References p. 25
The Paradoxical Effects of Checking on Checking Cognitions and Overestimation of Threat
Abstract
The purpose of this study was to investigate the effects of checking on checking cognitions. Checking is thought to contribute to the development of obsessive-compulsive disorder symptoms. Forty-five psychology students were randomly assigned to a control group (n = 23) or an experimental group (n = 22). The experimental group spent repeating checks on four objects during one week. The control group performed their normal checking behaviour. After this week the experimental group returned to their normal checking behaviour. An extra week was added to investigate follow-up effects. Checking cognitions were measured with the Checking Cognitions Scale. In contrast to the hypothesis, the results showed no effects of checking on checking cognitions. Both groups showed equivalent increases in checking cognitions. Future research is necessary, as there were some important methodological shortcomings. Implications for future research are discussed.
Obsessive-compulsive disorder (OCD) is a heterogeneous disorder characterised by the occurrence and performance of obsessions and compulsions (Abramowitz, Taylor & McKay, 2009). In most cases obsessions and compulsions occur simultaneously. Obsessions are recurrent, intrusive and persistent thoughts that cause distress and anxiety. For example, thinking obsessively that something might happen to another person. Compulsions are physical or mental acts with the purpose of lowering or neutralising distress caused by
obsessions, or to prevent a feared threat (Abramowitz, Taylor & McKay, 2009). Compulsions are often accompanied by distress and tension and tend to develop gradually. Finally,
compulsions are associated with doubt and indecisiveness (Rachman, 2001). The most prevalent compulsions are checking compulsions, followed by cleaning compulsions,
although much more varieties exist (Rachman, 2001).
The onset of OCD is during early adulthood or late adolescence, although the disorder can occur at any age and it is seen in children as well (Dell’Osso, Altamura, Mundo,
Marazziti & Hollander, 2007). The prevalence of OCD is estimated between 2 and 3% of the total population (Abramowitz, Taylor & McKay, 2009). OCD has very severe consequences, because obsessions and compulsions can take over a patient’s life entirely, often resulting in exhaustion and an impaired daily functioning (Piacentini, Bergman, Keller, McCracken, 2003). Furthermore, the disorder is often chronic if it remains untreated. OCD is also associated with an average lower income, a smaller chance of marriage and employment (Torres et al., 2006), social impairment and impairment at work (Abramowitz, Taylor & McKay, 2009). In addition, OCD has high comorbidity with anxiety disorders, especially generalised anxiety disorder and panic disorder, and depressive disorders (Torres et al., 2006), which leads to even more impairment (Huppert, Simpson, Nissenson, Liebowitz & Foa, 2009).
Exposure and response prevention (ERP) appears to be effective in reducing OCD symptoms (Fisher & Wells, 2005; McLean et al., 2001). ERP is a behavioural treatment where patients are prevented to perform any compulsions when when they feel anxious. The theory behind this intervention is that individuals will experience that compulsions are not necessary acts to perform and that this also promotes habituation to their obsessive thoughts and fears (Fisher & Wells, 2005; McLean et al., 2001). However, research suggests that a significant proportion of patients do not fully recover after treatment because only 30% of the patients are completely symptom free after treatment (Fisher & Wells, 2005). Although ERP seems to be the most effective treatment, there remains much room for improvement. More knowledge about the etiology of OCD could therefore contribute to higher treatment effectiveness.
Both OCD and anxiety disorders are associated with safety behaviours such as avoidance of public places in agoraphobia (Rachman, Radomsky, & Shafran, 2008). Safety behaviours are acts performed with the purpose to secure for safety or to prevent a feared threat (Rachman, Radomsky & Shafran, 2008). Hence, compulsive checking is a form of safety behaviour. Paradoxically, safety behaviours seem to maintain fear instead of reducing it, because they prevent disconfirmation of a feared threat (Salkosvkis, 1991). For example, when the stove is repeatedly checked, one is not able to disconfirm that the kitchen will not burn down. In addition, research suggests that anxious individuals are more likely to use their safety seeking behaviour as cues for threat (Gangemi, Mancini & van den Hout, 2012). Patients with an anxiety disorder rated a script describing an objective safe situation as more dangerous if safety behaviour was performed in this script, compared to no safety behaviour. This effect was replicated in a non-student sample (van den Hout, Gangemi, Mancini,
Engelhard, Rijkeboer, van Dams & Klugkist, 2014). Thus, safety behaviours play an
important role in maintaining fear because of two possible mechanisms: First, anxious people are more likely to use safety behaviour as cues for possible threats and second these
behaviours prevent disconfirmation of inaccurate threats.
A functional relationship between obsessions and compulsion is suggested in the definition of compulsions, because it is stated that compulsions are performed in reaction to obsessive thoughts (Abramowitz, Taylor & McKay, 2009; American Psychiatric Association, 2013). However, compulsions also seem to play a pivotal role in intensifying the symptoms of OCD. Rachman (2002) developed a cognitive theory about compulsive checking in OCD. He explains the occurring of compulsive checking as well as a self-perpetuating mechanism that explains why checking is repeated. First, motivation to perform checks is influenced by an elevated sense of responsibility and an exacerbated rating of the likelihood and severity of harm a threat will cause. The stronger these factors are present, the higher the motivation to
perform checks will be. Second, compulsive checking leads to a lower confidence in memory, an increase in responsibility, the perceived severity and the likelihood of threat. Thus,
although checking is performed to ensure safety, paradoxically this behaviour seems to induce the opposite: the feeling of being less secure. These factors in turn increase the motivation to check and this process is what Rachman (2002) describes as the self-perpetuating mechanism. Research has at least partly found evidence to confirm some of the mechanisms of the theory of Rachman (2002). First, multiple studies have shown that checking lowers memory
confidence (Boschen, Wilson & Farrell, 2011; Radomsky & Alcolado, 2010; Radomsky, Gilchrist, Dussault, 2006; Van den Hout & Kindt, 2003). Second, studies show that
manipulating responsibility beliefs lead to more checking by participants with OCD (Arntz, Voncken & Goosen, 2007).
Research shows that safety behaviour can indeed directly contribute to the development and increase of anxiety. Deacon and Maack (2008) performed a study on psychology students who were allocated into a low contamination fear group and a high contamination fear group. The experiment consisted of three phases: one baseline phase and an experimental phase followed by another baseline phase. In both baseline phases
participants performed their normal safety behaviours. In the experimental phase participants were instructed to engage in as much safety behaviours as possible. After the experimental phase, both the low and high contamination fear groups showed an equivalent increase in threat overestimation and contamination fear symptoms. After the second baseline phase the concerns about contamination declined in both groups, but did not return to full baseline level. A limitation of this study is that no control condition was used. To address this problem, Olatunji, Etzel, Tomarken, Ciesielski and Deacon (2011) studied the effects of safety behaviours on health anxiety. They assigned undergraduate students either to a control or safety behaviour group. The experimental design was the same as in the Deacon and Maack
(2008) study, consisting of two baseline phases and one experimental phase. In the
experimental phase participants in the safety behaviour group were asked to perform as many health-related safety behaviours as possible. The results showed that health anxiety,
hypochondriacal beliefs and avoidance of possible health threats increased in the safety behaviour group, compared to the control group. After the second baseline phase the safety behaviour group still showed higher levels of avoidance, hypochondriacal beliefs and health anxiety, compared to the control group.
The two studies described above form evidence for the effects of safety behaviour on fear. Rachman (2002) states that both compulsive cleaning and checking behaviour in hypochondriasis show strong similarities with compulsive checking. Therefore, it is likely to assume that compulsive checking also directly increases perceptions of threat. Empirical support for this theory about OCD is partly found by Van Uijen and Toffolo (2015). They conducted an experiment to study the effects of checking on obsession-related cognitions. Students were assigned to a control group, a monitor group and a checking group. The checking group was asked to record their daily checking behaviour and to perform every check once more for one week. For instance, if a participant normally checked the stove once a day, they were asked to do this twice. The monitor group was asked to record their daily checking behaviour and the control group received no further instructions regarding checking behaviour. After one week all groups completed a series of questionnaires. The results
showed that the experimental group showed increases in the perceived severity of threat, compared to the monitor group and the control group. However, in all groups, no changes were found in the perceived likelihood of threat or general obsessive beliefs. There appeared to be no clear differences between the monitor and the control group.
Since compulsions are characterised by repetitiveness and are usually domain-specific, meaning that patients with OCD tend to check only a few specific objects (Abramowitz,
Taylor & McKay, 2009), the manipulation of checking behaviour needs to mirror these characteristics as good as possible. For example, checking only a few specific objects multiple times in a row. There are two important methodological limitations of the study by Van Uijen and Toffolo (2015). Firstly, they instructed their participants to check 59 different items, which is not domain-specific. Secondly, they asked their participants to perform every check once more than they usually did, which is not repetitive. Therefore, this
operationalization of checking is not sufficient enough to mirror compulsive checking in OCD.
The most important goal of the present study was to look at the effects of repeated checking on threat estimation and checking cognitions. In an attempt to mirror compulsive checking as valid as possible, the study of Van Uijen and Toffolo (2015) was replicated with some important adjustments. The checking group was asked to repeat every check repetitively while the control group received no further instructions. The focus of the checking group was on four specific objects, instead of a whole list. A monitor group as Van Uijen and Toffolo (2015) used was not included. This was decided because this monitor group did not differ from the control group in their study. In addition, the present study included a follow-up measurement, to see whether the influence of checking would still be present after one week.
This study revolves around five hypotheses: First it was hypothesised that the experimental group, compared to the control group, would show an increase in checking related cognitions, consisting of the likelihood of a threat, the importance of checking and the severity of threat after one week of performing checking behaviour. Second, it was expected that there would be an increase in general obsessive beliefs in the experimental group
compared to the control group. Third, it was hypothesized that there would not be an increase in general anxiety, but only in OCD-related cognitions, because the manipulation is supposed to influence only OCD symptoms. Two hypotheses were made for the follow-up
measurement. It was hypothesised that the experimental group would show a slight, but significant decrease in checking related cognitions and general obsessive beliefs, but not with a full return to baseline level. These last two hypotheses are based on the findings of Deacon and Maack (2008) and Olatunji, Etzel, Tomarken, Ciesielski and Deacon (2011), where the effects of safety behaviour declined after a week, but did not return to full baseline level.
Methods Participants
Forty-seven undergraduate psychology students participated in exchange for research credit. The participants were recruited online at the University of Amsterdam and assigned either to the control group (n = 25) or the experimental group (n = 22) in order of arrival in the lab. Participants with current or past psychological problems were excluded from
participation. The age ranged from 18 to 27 years, (M = 20.17, SD = 2.03) and the majority of participants were women (n = 41). Participants that scored higher than 16 on obsessive-compulsive tendencies and participants that checked too much repetitively in their normal daily lives were also excluded from participation. From all participants informed consent was obtained.
Materials
Obsessive-Compulsive Inventory-Revised (OCI-R). The OCI-R (Foa et al., 2002)
measures obsessive-compulsive tendencies, and a Dutch version of this test (Cordova-Middelbrink, Dek & Engelbarts, 2007) was used in this study. This test consists of 18 statements that are rated on a 5-point Likert scale from 0 = not at all to 4 = extremely. An example of some statements: “I find it difficult to control my own thoughts”, and “I feel I have to repeat certain numbers”. The minimum score on this test is 0 and the maximum score
is 72. A score of 21 or higher indicates the possible presence of OCD. Test-retest reliability and validity were good (Foa, Huppert, Leiberg, Langner, Kichic, Hajcek & Salkovskis, 2002; Hajack, Huppert, Simons & Foa, 2004), as well as the internal consistency (Huppert, Walther, Hajcak, Yadin, Foa, Simpson, & Liebowitz, 2007).
Checking Cognitions Scale. The Dutch version of this scale was adjusted and based
on the scale used in the study of Van Uijen and Toffolo (2015). The CCS measures checking cognitions through three subscales: likelihood and severity of threat when a check is not performed and the importance of checking. The original CCS consisted of 21 items in 7 categories. However, the current scale was reduced to 12 items in 4 categories. The four categories of items were: front door, stove, fridge and a specific lamp. Every category consists of three questions regarding the three subscales that are rated on a visual analogue scale from 0-100. An example: “To me, checking if the front door is locked properly before I leave the house” is rated from not at all important to very important; “How likely do you think it is that something dangerous will happen if you do not check if the front door is locked when you leave the house?” is rated from very unlikely to very likely; “How severe are the possible consequences if you do not check if the front door is locked when you leave the house” is rated from not at all severe to very severe. The score on the CCS was calculated by averaging the scores for each participant across all items (Deacon & Maack, 2008; Van Uijen & Toffolo, 2015) for the CCS overall and the three subscales. The internal consistency was high at the three time points, 𝑎𝑎 = .83 and 𝑎𝑎 = .87 and 𝑎𝑎 = .86. The test-retest reliability was good, 𝑟𝑟 = .78 and 𝑟𝑟 = .84.
Obsessive Beliefs Questionnaire-44 (OBQ-44) Responsibility and Threat Estimation (RT) Scale. A Dutch translation of this questionnaire consists of 16 items and
measures obsessive beliefs related to responsibility and threat. It is a subscale of the OBQ-44 (Obsessive Compulsive Cognitions Working Group, 2005). Every item is a statement that was
rated on agreement on a 7-point Likertscale, varying from 1 = disagree very much to 7 = agree very much. For example: “ I often think that things around me are unsafe”, and “Even when I am very careful, I think that harmful things will happen”. The minimal score on this test is 16, and the maximum score on this questionnaire is 112, which equals very severe obsessive beliefs related to harm, danger and responsibility. The internal consistency is very high, 𝑎𝑎 = .93 (Obsessive Compulsive Working Group, 2005). In our study the internal consistency was also good at all three time points, 𝑎𝑎 = .82, 𝑎𝑎 = .88 and 𝑎𝑎 = .86. The test-retest reliability was good, with 𝑟𝑟 = .82 and 𝑟𝑟 = .79.
Beck Anxiety Inventory (BAI). The BAI (Beck, Epstein, Brown & Steer, 1988) is a
measure of general anxiety symptoms. A Dutch translation of this test was used. The BAI consists of 21 symptoms that are rated to the extent these symptoms have been present in the last week on a 4-point Likert scale from 0 = not at all to 4 = very much. These symptoms include: “transpire” and “nervous”. The minimum score is 0 and the maximum score is 84. A score between 0 - 21 reflects low anxiety, a score between 22 – 35 reflects moderate anxiety and a score of 36 and above reflects high anxiety. The internal consistency was high, 𝑎𝑎 = .92 and the BAI also has high test-retest reliability and discriminant validity (Beck, Epstein, Brown & Steer, 1988).
The Checklist. The Checklist was developed by Van Uijen and Toffolo (2015) but
was adjusted for this study. Originally, the checklist consisted of 59 specific checking behaviours in six categories. The six different categories were: “Before leaving the house,” “Before going to bed,” “Before having been away,” “After being in the kitchen,” “After using the bathroom or cosmetic appliances,” “Other situations.” For all checking behaviours,
respondents noted whether they had performed each check that day by either indicating “yes,” “no” or “not applicable”. In this study, the checklist was adjusted to make it possible for participants to also indicate how often they repeated each check in a row. The score was
computed by looking at the percentage of “yes” responses. The test-retest reliability was good between pre-test and follow-up test, 𝑟𝑟 = .79 and 𝑟𝑟 = .71, using the scores of the control group.
Social Desirability. A question was included that measured the rate participants had
given social desirable answers. It consisted of one question that participants answered on a visual analogue scale from 0 to 100. The question was: “I answered all questions honestly.”
The Short Checklist. The Short Checklist was developed for the purpose of this
study. The checklist consists of the four checking behaviours that participants in the
experimental condition were asked to perform. With every item, participants noted how often they had the opportunity that day to check that item. For example: “How often in the last 24 hours have you had the opportunity to check is the stove was off?” Following this question, if for example, a participant had checked if the stove was off four times that day, they had to note down separately for every time how often they repeated the check. For example: “For every time you checked if the stove was off, can you indicate how many times you checked this in a row?” The Short Checklist was scored by dividing the total amount of checks by the amount of possible checks the participants could have performed.
Qualtrics. Qualtrics (http://www.qualtrics.com, version 2016) was used to digitalize
all questionnaires and to distribute them to the participants online.
Procedure
All participants received written and verbal instructions that they were participating in a study to validate questionnaires about OCD. Participants first completed the OCI-R online outside the lab. The OCI-R was used as a screening instrument to exclude participants that scored 16 or higher. After this, participants were invited to the laboratory at the University of Amsterdam individually, where they received the package of other questionnaires (CCS,
OBQ-44 RT, BAI, Checklist, Social desirability) on a computer through Qualtrics in a soundproof laboratory. The CCS was used to measure changes in checking cognitions, the OBQ-44 RT was used to measure changes in general obsessive beliefs and the BAI was used to measure changes in general anxiety symptoms. Also, after completing the questionnaires, participants were asked if they had answered all questions honestly.
The manipulation followed after the participants completed the questionnaires. The experimental group was asked to fill in the Short Checklist, which was used as a manipulation check. The experimenter instructed the experimental group by reading the written
instructions. The experimental group was instructed to perform each of the four checking behaviours (front door, fridge, stove and lights) five times more then they usually did, every time they used that object. For instance, if the participant locked the door, the participant was asked to perform this check five times in a row. Participants also received instructions how to fill in the daily short checklist and were also given reminders for post-test and follow-up test. The Short Checklists were available between four o’clock in the afternoon until four o’clock the next morning. The control group received no special instructions after completing the questionnaires. They were only told the exact timespan and date to fill out the post-test and follow-up test questionnaires. The control group was told nothing else about the purpose of this study.
After 1 week of visiting the lab, all participants received a reminder email with a link to the same package of online questionnaires (CCS, OBQ-44 RT, BAI, The Checklist, Social desirability) as the first session. Everyone had 12 hours to complete this session. Participants in the experimental group were told explicitly to return to their normal checking behaviour. After another week, the third session was constructed to be exactly the same as the second session. All questionnaires were sent to the participants online. After completing the last
session, all participants were thanked for participation and received a thorough debriefing by email, including contact information of the researchers.
Results Randomisation and manipulation checks
Two participants in the experimental condition were excluded, because they filled in the Short Checklist only two times. Therefore, it was not sure if they performed all checks. Furthermore, one of these participants also failed to complete the follow-up test. The final sample consisted of 45 participants. The data on general anxiety (BAI) were log transformed, because the scores were positively skewed.
Independent t-tests were performed to check whether age was equally distributed across the two groups, and to check for differences in pre-test scores on the questionnaires. Age was equally distributed between the experimental (M = 19.95, SD = 2.15) and the
control group (M = 20.48, SD = 1.97), 𝑡𝑡(43) = 0.852, 𝑝𝑝 = .40. The two groups did not differ in general anxiety (BAI), 𝑡𝑡(43) = -0.09, 𝑝𝑝 = .93, obsessive beliefs (OBQ-44 RT), 𝑡𝑡(43) = 0.77, 𝑝𝑝 = .445, and obsessive-compulsive tendencies (OCI-R), 𝑡𝑡(43) = -0.30, 𝑝𝑝 = .77. Furthermore, no pre-test differences were found for checking cognitions (CCS) 𝑡𝑡(43) = 1.53, 𝑝𝑝 = .134 and for the three dimensions of checking cognitions: importance of checking, 𝑡𝑡(43) = 1.38, 𝑝𝑝 = .176, likelihood of checking, 𝑡𝑡(43) = 1.05, 𝑝𝑝 = .30 and severity of checking, 𝑡𝑡(43) = 1.33, 𝑝𝑝 = .192. There were also no differences between the two groups in the amount of items they checked at the pre-test (Checklist), 𝑡𝑡(43) = 0.67, 𝑝𝑝 = .51; see Table 1.
As a manipulation check, a mixed ANOVA with Time (pre-test vs. post-test) as the within variable and Condition (control vs. experimental) as the between variable, to assess whether participants in the experimental group checked more items on the long checklist, compared to the control group between pre- and post-test. There was a significant main effect
of Time, 𝐹𝐹(1.43) = 5.09, 𝑝𝑝 = .029, = .11, but no significant main effect of Condition, 𝐹𝐹(1,
43) = 0.54, 𝑝𝑝 = .465, = .01. The crucial interaction effect was significant, 𝐹𝐹(1, 43) = 11.75, 𝑝𝑝 = .001, = .22. This means that the experimental group showed a stronger increase in
items checked, compared to the control group.
During the first week, participants in the experimental group filled in their daily short checklists. The mean amount of checks was M = 4.40, (SD = 0.74), the percentage of items that was checked five times in a row, was 67.27% (SD = 25.87). The mean scores for socially desirable answers were M = 97.09 (SD = 0.92) in the control group and M = 98.79 (SD = 2.00) in the experimental group.
Table 1
Mean (SD) Dependent Variables by Condition and Participant Characteristics
Experimental group (𝑛𝑛 = 23) Control group (𝑛𝑛 = 22) Total (𝑛𝑛 = 45) OCI-R Pre 6.36(4.50) 6.00(3.61) 6.18(4.02) CCS Pre 22.58(13.06) 28.28(11.91) 25.49(12.70) Post 27.55(15.82) 30.10(15.24) 28.85(15.40) Follow-up 25.35(15.66) 30.13(16.49) 27.79(16.08) CCSimp Pre 34.72(16.81) 42.03(18.78) 38.46(18.02) Post 37.50(20.35) 38.46(21.22) 37.99(20.57) Follow-up 33.56(19.50) 35.58(21.12) 34.59(20.14) CCSlik Pre 11.18(9.66) 14.46(11.19) 12.86(10.48) Post 18.31(15.10) 18.82(13.27) 18.57(14.03) Follow-up 16.34(13.56) 21.78(17.04) 19.12(15.50) CCSsev Pre 21.83(17.60) 28.34(15.28) 25.16(16.60) Post 26.83(17.51) 33.02(17.05) 30.00(17.36) Follow-up 26.15(19.64) 33.04(17.62) 29.67(18.75) BAI Pre 8.68(4.38) 8.52(7.40) 8.60(6.05) Post 8.64(5.55) 5.52(5.48) 7.04(5.67) Follow-up 8.86(8.70) 4.91(5.69) 6.84(7.50) OBQ-44 Pre 31.46(10.01) 33.78(10.24) 32.64(10.08) Post 30.10(12.59) 30.40(11.73) 30.24(12.02) Follow-up 25.68(10.13) 27.65(10.31) 26.69(10.16) Checklist Pre 22.46(12.66) 25.07(13.49) 23.79(13.00) Post 31.15(11.40) 23.28(14.19) 27.13(13.37) Follow-up 20.31(12.31) 17.67(13.88) 18.96(13.06)
Note. OCI-R = Obsessive-Compulsive Inventory-Revised; CCS = Checking Cognitions Scale; CCSimp = CCS
importance subscale; CCSlik = CCS likelihood subscale; CCSsev = CCS severity subscale; BAI = Beck Anxiety Inventory; OBQ-44 = Obsessive Beliefs Questionnaire-44 Responsibility/Threat Estimation Scale; Pre = pretest; Post = posttest; Follow-up = Follow-up test
Main Analyses
To test the hypotheses, a series of 3 (Time: pre-test vs. post-test vs. follow-up test) x 2 (Condition: control vs. experimental) mixed ANOVAs were conducted to study the
hypotheses about the effects of checking on cognitions related to checking (CCS), obsessive beliefs related to threat and responsibility (OBQ-44 RT) and symptoms of general anxiety (BAI). An alpha level of .05 was used for all statistical tests. The OCI-R was exploratory added as a covariate to assess whether obsessive-compulsive tendencies were related to checking cognitions (CCS).
Checking cognitions (CCS). There was a significant main effect of Time, 𝐹𝐹(2, 86) =
3.42, 𝑝𝑝 = .037, = .07. Repeated contrasts on Time only showed a significant difference in
change in CCS score between pre-test and post-test, 𝐹𝐹(1, 43) = 5.47, 𝑝𝑝 = .024, = .11. This means that there was only an increase in CCS score between pre-test and post-test. There was no main effect of Condition, 𝐹𝐹(1, 43) = 1.1, 𝑝𝑝 = .3, = .03 and the crucial Time x Condition
interaction was also not significant, 𝐹𝐹(2, 86) = 0.75, 𝑝𝑝 = .48, = .02; see Figure 1. Exploratory, the pre-test score on the OCI-R was added as a covariate. However, pre-test OCI-R score was not significant related to CCS score, 𝐹𝐹(1, 42) = 3.89, 𝑝𝑝 = .055.
The CCS importance of checking subscale showed no main effect of Time, 𝐹𝐹(2, 86) = 1.79, 𝑝𝑝 = .17, = .04. There was no main effect of Condition, 𝐹𝐹(1, 43) = 0.42, 𝑝𝑝 = .52, = .01. In contrast to our expectations, there was no interaction effect, 𝐹𝐹(2, 86) = 1.89, 𝑝𝑝 = .31,
= .03; see Figure 2. OCI-R score was significantly related to CCS importance, 𝐹𝐹(1, 42) = 5.95, 𝑝𝑝 = .019. However, there was no significant effect of condition on CCS importance after controlling for obsessive-compulsive tendencies, 𝐹𝐹(1, 42) = 0.63, 𝑝𝑝 = .43.
The likelihood subscale showed a significant main effect of Time, 𝐹𝐹(2, 86) = 8.31, 𝑝𝑝 = .001, = .16. Repeated contrasts only revealed a significant effect of Time between pre-test
and post-test, 𝐹𝐹(1, 43) = 9.76, 𝑝𝑝 = .003, = .19, which means that there was an increase in CCS importance scores between pre-test and post-test. There was no main effect of
Condition, 𝐹𝐹(1, 43) = 0.76, 𝑝𝑝 = .39, = .02, and the crucial interaction effect was also not
significant, 𝐹𝐹(2, 86) = 1.06 𝑝𝑝 = .35, = .02; see Figure 2. Also, OCI-R score was not significantly related to the likelihood of checking, 𝐹𝐹(1, 42) = 4.02, 𝑝𝑝 = .052.
On the CCS severity subscale, there was a main effect of Time, 𝐹𝐹(2, 86) = 5.53, 𝑝𝑝 = .005, = .11. Repeated contrasts revealed an increase in CCS severity score between pre-test
and follow-up test, 𝐹𝐹(1, 32) = 8.93, 𝑝𝑝 = .005, = .17. There no significant main effect on
Condition, 𝐹𝐹(1, 43) = 1.81, 𝑝𝑝 = .19, = .04 and against expectations also no significant
interaction effect, 𝐹𝐹(2, 86) = 0.02, 𝑝𝑝 = .98, = .001; see Figure 2. OCI-R score was not significantly related to severity of checking, 𝐹𝐹(1, 42) = 0.9, 𝑝𝑝 = .35. In sum, in contrast to our hypothesis, the experimental group did not show a stronger increase in CCS overall score between pre-test and post-test and there were no decreases in CCS score between post-test and follow-up test, while it was expected that the experimental group would show a stronger decrease in CCS score between post-test and follow-up test.
Figure 1: Mean scores on the Checking Cognitions Scale (CCS) for the
experimental group and the control group at pre-test, post-test and follow-up test.
Figure 2: Mean scores on the Checking Cognitions Scale (CCS) subscales likelihood of checking
(right panel), importance of checking (middle panel), and severity of threat (left panel) at the pre-test, post-test and follow-up test for the experimental group and the control group.
Obsessive beliefs (OBQ-44 RT). Because the assumption of sphericity was violated,
𝑋𝑋2(2) = .75, 𝑝𝑝 = .002, the degrees of freedom were corrected using Huyn-Feldt estimates of sphericity (𝜀𝜀 = .84). There was a significant main effect of Time, 𝐹𝐹(1.68, 72.63) = 21.57, 𝑝𝑝 < .001, = .33. Repeated contrasts showed a significant effect of Time between pre-test and
post-test, 𝐹𝐹(1, 43) = 5.23, 𝑝𝑝 = .027, = .31, and between post-test and follow-up test, 𝐹𝐹(1,
43) = 30.59, 𝑝𝑝 < .001, = .42. This means that, in contrast to the expectations, the experimental group did not show an increase in OBQ score compared to the control group between pre-test and follow-up test, because both groups showed a decrease between pre-test and follow-up test. There was not a significant main effect of Condition, 𝐹𝐹(1, 43) = 0.25, 𝑝𝑝 = .62, = .01, and the crucial Time x Condition interaction was also not significant, 𝐹𝐹(1.68,
72.63) = 0.7, 𝑝𝑝 = .48, = .02. Thus, the results are not in accordance to the expectations.
General anxiety symptoms (BAI). Because the assumption of sphericity was
violated, 𝑋𝑋2(2) = 7.22, 𝑝𝑝 < .027, the degrees of freedom were corrected using Huyn-Feldt estimates of sphericity (𝜀𝜀 = .92). There was a significant main effect of Time, 𝐹𝐹(1.83, 78.89) = 12.60, 𝑝𝑝 < .001, = .23, a significant main effect of Condition, 𝐹𝐹(1, 43) = 4.54, 𝑝𝑝 = .039,
= .10. Against expectations, the Time x Condition interaction was significant, 𝐹𝐹(1.84,
78.89) = 3.37, 𝑝𝑝 = .043, = .07. To break down this interaction, simple contrasts were performed. These contrasts revealed a significant interaction when comparing the BAI scores at the pre-test to the post-test of the experimental group and the control group, 𝐹𝐹(1, 43) = 4.59, 𝑝𝑝 = .038, = .01. In other words, the decrease in BAI score was larger in the control group compared to the experimental group. When comparing the BAI score on the post-test to the follow-up test, the interaction was not significant, 𝐹𝐹(1, 43) = 0.36, 𝑝𝑝 = .55, = .01; see
not stronger in the control group, compared to the experimental group. Based on the results, it can be stated that these are in contrast with the expectations, because the control group
showed a stronger decrease in BAI score compared to the experimental group between pre-test and post-pre-test.
Discussion
The present study investigated the effects of repeated checking on estimation of threat and checking related cognitions. It was theorized that increasing checking behaviours would lead to stronger checking related cognitions (Rachman, 2002; Van Uijen & Toffolo, 2015),
measured with the CCS. However, the results of the current study are not in accordance to this theory and the results found in other studies (Deacon & Maack, 2008; Olatunji, Etzel,
Figure 3: Mean BAI score for each condition at pre, post
hypothesized that there would be an increase in checking cognitions in response to increased checking and that these checking cognitions would show a slight decrease between post-test and follow-up. In contrast, there was an increase in checking cognitions for both groups on the subscales likelihood of threat and severity of threat and the overall checking cognitions score between pre-test and post-test. No effects were found for the importance of checking , nor were there other effects of repeated checking on checking cognitions. Second, it was hypothesized that obsessive beliefs would increase in response to increased daily checking. The results showed that general obsessive beliefs decreased in both groups between pre-test and follow-up test. Third, it was expected that compulsive checking would not have an effect on general anxiety. However, there was a stronger decrease in general anxiety in the control group compared to the experimental group between pre-test and follow-up test.
There are two unexpected findings that cannot be explained. First, increasing daily checking behaviours did not increase general obsessive beliefs related to responsibility and threat estimation measured with the OBQ-44 RT scale. Instead, both groups showed a decrease in obsessive beliefs. This effect is in contrast to the findings by Van Uijen and Toffolo (2015), who found no effects on general obsessive beliefs. The reason for this
decrease in both groups is unknown. The same applies for the finding concerning the decrease in general anxiety symptoms measured with the BAI. This decrease was accounted for by a stronger decrease in the control group, compared to the experimental group between pre-test and post-test. The reason for this stronger decrease in the control group is not clear. Other studies found either no effects on general anxiety (Olatunji, Etzel, Tomarken, Ciesielski & Deacon, 2011; Van Uijen & Toffolo, 2015) or an overall decrease in both groups (Deacon & Maack, 2008). However, because the BAI might measures panic symptoms rather than
general anxiety (Leyfer, Ruberg & Woodruff-Borden, 2006), a different measure can be used, for example the Depression, Anxiety, and Stress Scale (Henry & Crawford, 2005) to further
investigate the effects of increased checking on general anxiety. In short, no strong conclusions can be made related to the effects of increased checking on general obsessive beliefs and general anxiety.
It was hypothesized that more daily checking increases checking cognitions based on the cognitive theory of OCD (Rachman, 2002) and findings of previous studies. Based on the cognitive theory of OCD it was argued that compulsive checking is repeated because it leads to a higher estimation and likelihood of threat through a self-perpetuating mechanism
(Rachman, 2002). Studies to the more general effects of safety behaviours on anxiety showed that safety behaviours increased overestimation of threat and contamination fear (Deacon & Maack, 2008) and health anxiety (Olatunji, Etzel, Tomarken, Ciesielski & Deacon, 2011). Van Uijen and Toffolo (2015) examined the effects of safety behaviour more specifically on OCD by increasing checking behaviour. Their results showed an increase in checking related cognitions in the experimental group, although this effect was mainly accounted for by an increase in threat estimation. In contrast, the current study found an increase in overall checking cognitions and likelihood and severity of threat for both groups.
There is one important difference between previous studies, the study of Van Uijen and Toffolo (2015) and the present one: compulsive checking was operationalized in the current study by instructing participants to check four specific objects five times in a row, while Van Uijen and Toffolo (2015) instructed their participants to check 59 different objects once more than they normally did. Because compulsive checking is characterized by checking very specific objects repetitively (Abramowitz, Taylor & McKay, 2009), it was expected that the operationalization of compulsive checking in the current study would be more similar to compulsive checking in OCD, compared to Van Uijen and Toffolo (2015). Therefore, it was expected to find stronger effects in the current study. However, against all expectations no interpretable effects were found. The evidence gathered in the current study is not conclusive
enough to reject the theory of Rachman (2002), even so because similar studies did find significant effects. In addition, the reason for the increase in checking cognitions in both groups is unknown. However, it is possible to point out some explanations for not finding the expected effects.
First, a cut-off score of 16 on obsessive-compulsive tendencies measured by the OCI-R was used in the current study, while the clinical cut-off score is 25. This means that the participants in the present study scored very low on obsessive-compulsive tendencies. It is possible that the influence of checking is only applicable to people with stronger obsessive-compulsive tendencies. The scores on the OCI-R at pre-test were exploratory included as a covariate, but these did not seem to have an influence on checking cognitions. However, since obsessive-compulsive tendencies were low, controlling for these tendencies might only have an effect on checking cognitions when these are stronger present. The mean scores on the OCI-R in the Van Uijen and Toffolo (2015) study were higher compared to the present one. In addition, the dispersion in OCI-R scores in their study was higher, compared to the current study. This could explain why the present study failed to find any effects of increased
checking on checking cognitions. For this reason, future research should also include people with higher tendencies related to OCD.
Second, the testing environment in the present study was less controlled compared to similar studies, including Van Uijen and Toffolo (2015), who required their participants to return to the lab at all testing times and used paper versions of the questionnaires. In the present study participants visited the laboratory at pre-test, but opposed to Van Uijen and Toffolo (2015) the participants were not required to return to the lab on the second and third time points. As a result, it is not known in which circumstances and environment participants completed the questionnaires. Consequently, the circumstances were less controlled for. This could have influenced the results to be less reliable, because some participants were more
distracted and less concentrated than participants in similar studies, because participants were watching television while completing the questionnaires for example.Future research could focus on requiring all tested individuals to return to the lab at all time points.
Third, in the current study, the CCS used by Van Uijen and Toffolo (2015) was adjusted to be more consistent with the items that were checked. However, both questionnaires were not validated. However, the internal consistency of the CCS in the present study on the three time-points was good, although looking at the internal consistency of the three subscales, some appeared to be much lower than the CCS overall. Although it is unsure whether theCCS is a valid measure of checking cognitions, similar questionnaires were constructed by Van Uijen and Toffolo (2015), Deacon and Maack (2008) and Olatunji, Etzel, Tomarken, Ciesielski and Deacon (2011). These studies also found effects and self-constructed their safety behaviour manipulation. Therefore it can also be expected that CCS surely measured reliably, although future research should focus on validating the CCS and the subscales, by either performing replication studies or by validating the CCS with similar questionnaires.
Overcoming the limitations of the current study discussed above will create an opportunity to collect stronger evidence for the effects of compulsive checking on checking related cognitions. If this evidence is found, this could have important implications for the clinical field. More knowledge about the etiology of OCD and compulsive checking could contribute to higher treatment effectiveness and more knowledge about the mechanisms of change. As explained, ERP, a treatment for OCD, is theorised to work through giving patients insight in the fact that no threats will occur when compulsions are not performed. However, the mechanism that makes this therapy so effective could be different. By preventing the performance of compulsions, the related checking cognitions decrease, which is followed by a decreased motivation to check. Furthermore, knowledge about the effects of checking on
OCD symptoms could be used to compare with other theories about the development of OCD. Together, this knowledge contributes to a stronger base of knowledge about this disorder.
In conclusion, the results derived from the present study provide too little knowledge to either accept or reject the cognitive theory of compulsive checking (Rachman, 2002). Because the current studyies results were inconsistent with comparable studies, important limitations were given regarding some important methodological issues. It is thought that if these issues are resolved in future research, this would create an opportunity to gain more insight in the effects of checking on checking cognitions and threat overestimation.
References
Abramowitz, J. S., Taylor, S., & McKay, D. (2009). Obsessive-compulsive disorder. The Lancet, 374, 491-499.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, (5th ed.). Washington, DC: Author.
Arntz, A., Voncken, M., & Goosen, A. C. (2007). Responsibility and obsessive–compulsive disorder: An experimental test. Behaviour Research and Therapy, 45, 425-435. Barbeite, F. G., & Weiss, E. M. (2004). Computer self-efficacy and anxiety scales for an
Internet sample: testing measurement equivalence of existing measures and development of new scales. Computers in Human Behavior, 20, 1-15.
Boschen, M. J., & Vuksanovic, D. (2007). Deteriorating memory confidence, responsibility perceptions and repeated checking: Comparisons in OCD and control
samples. Behaviour Research and Therapy, 45, 2098-2109.
Cougle, J. R., & Lee, H. J. (2014). Pathological and non-pathological features of obsessive-compulsive disorder: Revisiting basic assumptions of cognitive models. Journal of Obsessive-Compulsive and Related Disorders, 3, 12-20.
Deacon, B., & Maack, D. J. (2008). The effects of safety behaviours on the fear of
contamination: An experimental investigation. Behaviour Research and Therapy, 46, 537-547.
Dell'Osso, B., Altamura, A. C., Mundo, E., Marazziti, D., & Hollander, E. (2007). Diagnosis and treatment of obsessive‐ compulsive disorder and related
Fisher, P. L., & Wells, A. (2005). How effective are cognitive and behavioural treatments for obsessive–compulsive disorder? A clinical significance analysis. Behaviour Research and Therapy, 43, 1543-1558.
Foa, E. B., Huppert, J. D., Leiberg, S., Langner, R., Kichic, R., Hajcak, G., & Salkovskis, P. M. (2002). The Obsessive-Compulsive Inventory: development and validation of a short version. Psychological Assessment, 14, 485.
Gangemi, A., Mancini, F., & van den Hout, M. (2012). Behavior as information: “If I avoid, then there must be a danger”. Journal of Behavior Therapy and Experimental
Psychiatry, 43, 1032-1038.
Huppert, J. D., Simpson, H. B., Nissenson, K. J., Liebowitz, M. R., & Foa, E. B. (2009). Quality of life and functional impairment in obsessive–compulsive disorder: a comparison of patients with and without comorbidity, patients in remission, and healthy controls. Depression and Anxiety, 26, 39-45.
Huppert, J. D., Walther, M. R., Hajcak, G., Yadin, E., Foa, E. B., Simpson, H. B., & Liebowitz, M. R. (2007). The OCI-R: validation of the subscales in a clinical sample. Journal of Anxiety Disorders, 21, 394-406.
Jenike, M. A. (2004). Obsessive–compulsive disorder. New England Journal of Medicine, 350, 259-265.
McLean, P. D., Whittal, M. L., Thordarson, D. S., Taylor, S., Söchting, I., Koch, W. J., Paterson, R. & Anderson, K. W. (2001). Cognitive versus behavior therapy in the group treatment of Obsessive-Compulsive disorder. Journal of Consulting and Clinical Psychology, 69, 205.
Obsessive Compulsive Cognitions Working Group. (2005). Psychometric validation of the obsessive belief questionnaire and interpretation of intrusions inventory—Part 2: Factor analyses and testing of a brief version. Behaviour Research and Therapy, 43, 1527-1542.
Olatunji, B. O., Etzel, E. N., Tomarken, A. J., Ciesielski, B. G., & Deacon, B. (2011). The effects of safety behaviors on health anxiety: An experimental
investigation. Behaviour Research and Therapy, 49, 719-728.
Pevalin, D. J. (2000). Multiple applications of the GHQ-12 in a general population sample: an investigation of long-term retest effects. Social psychiatry and psychiatric
epidemiology, 35, 508-512.
Piacentini, J., Bergman, R. L., Keller, M., & McCracken, J. (2003). Functional impairment in children and adolescents with obsessive-compulsive disorder. Journal of Child and Adolescent Psychopharmacology, 13, 61-69.
Rachman, S. (2002). A cognitive theory of compulsive checking. Behaviour Research and Therapy, 40, 625-639.
Rachman, S., Radomsky, A. S., & Shafran, R. (2008). Safety behaviour: A reconsideration. Behaviour Research and Therapy, 46, 163-173.
Radomsky, A. S., & Alcolado, G. M. (2010). Don’t even think about checking: Mental checking causes memory distrust. Journal of Behavior Therapy and Experimental Psychiatry, 41, 345-351.
Radomsky, A. S., Gilchrist, P. T., & Dussault, D. (2006). Repeated checking really does cause memory distrust. Behaviour Research and Therapy, 44, 305-316.
Salkovskis, P. M. (1991). The importance of behaviour in the maintenance of anxiety and panic: A cognitive account. Behavioural Psychotherapy, 19, 6–19.
Torres, A. R., Prince, M. J., Psych, M. R. C., Bebbington, P. E., Bhugra, D., Brugha, T. S., ... & Meltzer, H. (2006). Obsessive-compulsive disorder: prevalence, comorbidity, impact, and help-seeking in the British National Psychiatric Morbidity Survey of 2000. American Journal of Psychiatry, 163, 1978-1985.
Van den Hout, M., & Kindt, M. (2003). Repeated checking causes memory distrust. Behaviour Research and Therapy, 41, 301-316.
Van den Hout, M., Gangemi, A., Mancini, F., Engelhard, I. M., Rijkeboer, M. M., van Dams, M., & Klugkist, I. (2014). Behavior as information about threat in anxiety
disorders: A comparison of patients with anxiety disorders and non-anxious controls. Journal of Behavior Therapy and Experimental Psychiatry, 45, 489-495. Van Uijen, S. L., & Toffolo, M. B. (2015). Safety behavior increases obsession-related
