Time for a change:
Reactivity to an Experimental Perturbation as a
Predictor of Response to Cognitive Behavioral Therapy for Social
Anxiety Disorder
Rachel van Loenen (10004500) Research Master Thesis
Department: Developmental Psychology University of Amsterdam
Supervisor: Dr. Elske Salemink Second Assessor: Prof. Reinout Wiers
Abstract
Social anxiety disorder (SAD) is one of the most common psychiatric disorders and is associated with high personal and societal costs. However, treatment is only effective for roughly 50% of the patients and no reliable predictor of treatment success has been identified. Translating dynamical system theory (DST) to treatment prediction, successful psychological interventions could be conceptualized as a critical transition from a pathological to a non-pathological state and these transitions could possibly be predicted by generic “early-warning signals”. This is the first study to investigate whether the use of these early warning signals (i.e. reactivity and recovery), resulting from an experimental perturbation, could predict treatment response to evidence-based CBT in patients with SAD. This was investigated in 31 participants with SAD, who completed and experimental perturbation (a single-session of interpretation bias modification training) before the start of a 6-weekly group CBT program. The results showed that none of the early-warning signals predicted improved treatment outcome in SAD. Thus, this study did not find any support for the use of reactivity and recovery of interpretations as early warning signals of treatment change in SAD. Several shortcomings of the current study design are discussed and suggestions for future research are given.
Keywords: Cognitive behavioral therapy; critical transitions; dynamical system theory; early
TABLE OF CONTENT
1. Introduction 3
2. Materials and Methods 8
3. Results 17
4. Discussion 29
References 33
Appendix A: Assumptions of conducted analyses 40
Appendix B: Drop-out analyses 41
1. Introduction
Social anxiety disorder (SAD) is one of the most persistent and common psychiatric disorders, with lifetime prevalence rates estimations as high as 12.1% (Kessler, Berglund, Demler, Jin, & Walters, 2005). Briefly, this disorder is characterized by an intense fear of negative evaluation from others in social situations, leading to extreme distress and/or avoidance of social situations (5th ed.; DSM-5, American Psychiatric Association, 2013). SAD is associated with losses in quality of life (Stein, & Kean, 2000), significant functional impairments in the educational, social and occupational domains (Fehm, Pelissolo, Furmakr, & Wittchen, 2005; Schneier et al., 1994) and high socioeconomic costs due to healthcare consumption and decreased work productivity (Acarturk et al., 2009). Clearly, effective and efficient psychological treatments are of great importance to reduce these substantial personal and societal costs.
Currently, the first-line treatment for SAD is cognitive behavioral therapy (CBT; Pilling et al., 2013), with exposure and cognitive restructuring as the most important treatment components. Meta-analyses have shown that CBT is effective in reducing social anxiety symptoms, with moderate to large effect sizes (Norton & Price, 2007; Rodebaugh, Holaway, & Heimberg, 2004). However, as response rates are roughly 50%, half of the patients with SAD show little or no improvement after CBT (Davidson et al., 2004; Heimberg et al., 1998; Stein & Stein, 2008). To make matters worse, at present, there is no way to predict a priori which half will be a responder or a non-responder. Despite substantial research efforts, few predictors of treatment response have been identified, and none has been consistently supported in the literature (Eskildsen, Hougaard, & Rosenberg, 2010; Taylor, Abramowitz, & McKay, 2012) .
A shortcoming of this research, however, is that it has almost exclusively focused on static predictors of change. The shared assumption of these studies is that a single baseline assessment of a variable (e.g. gender, age of onset, severity of depressive symptoms) can reliably predict change in SAD symptoms from pre- to post-treatment. However, psychopathology is increasingly recognized as highly dynamic and changeable in nature and its symptoms can be influenced from moment-to-moment by various factors internal (e.g. cognitive, biological) and external (e.g. events) factors. In addition, there are substantial individual differences in the response to these internal and external influences (Hayes, Laurenceau, Feldman, Strauss, & Cardaciotto, 2007; Nelson, McGorry, Wichers, Wigman, & Hartmann, 2017). Consequently, static observations might not provide enough information to predict response to treatment in SAD, which is reflected by the poor predictive validity of these static predictors. Rather, theoretical models that are built on the dynamical and complex nature of psychopathology might be more powerful to predict which individuals will respond to treatment.
1.1 Dynamical System Theory
Dynamical systems theory (DST) may provide a novel route to the prediction of treatment response, since it highlights the crucial role of dynamic instead of static predictors of change (Nelson et al., 2017; Scheffer et al., 2009). DST aims to explain the behavior of complex dynamic systems (e.g. oceans, climate, ecosystems, financial markets) and the manner in which these systems changes over time (Scheffer et al., 2009; Thelen, 1995). One of the most important features of dynamical systems is that they can have multiple stable states and may shift from one state to another in response to changes in underlying conditions (Scheffer et al., 2012). Although this change can be gradual and linear, there is increasing evidence across disciplines that these
shifts are often abrupt and non-linear (Scheffer et al., 2009). Such abrupt shifts can simply be the results of a large external shock. However, it is also possible that only a minor disturbance can suffice to invoke a ‘critical transition’ to another stable state. In the latter case, the dynamical system surpasses a critical threshold, a so-called tipping point, at which a system shifts abruptly from one state to another (Scheffer et al., 2009). As reaching these tipping points can have large consequences, there is much interest in finding ways to identify whether such an abrupt transition is near.
Theory predicts that, before tipping points, the system tend to become increasingly slower in recovering from small perturbations, a phenomenon called ‘critical slowing down’ (Scheffer et al., 2009; Strogatz, 1994). This phenomenon can be exemplified using a ball-in-the-valley diagram (see Figure 1), in which the ball represents the state of the system and the slope of the valley represents the rate of change. Far away from the tipping point (Figure 1A), the valley is deep, and the ball is relatively difficult to perturb (i.e. low reactivity) and will quickly return to its original state (i.e. high recovery rate). However, when the ball is close to the tipping point (Figure 1B), the valley becomes shallower, and the ball is easy to perturb (i.e. high reactivity) and will take more time to return to its original state (i.e. low recovery rate). Interestingly, recent research have revealed that critical slowing down can be detected across disciplines as diverse as ecosystems and human physiology, through three generic statistical indicators; 1) slower recovery rates from perturbations, 2) increased autocorrelation; the state of the system becomes more like its past state closer to a tipping point, and 3) increased variance; the accumulating impact of perturbations prior to a tipping point increases the variance of the systems elements (Dakos, Carpenter, van Nes, & Scheffer, 2015; Scheffer et al., 2009). The first indicator is the most direct and straightforward indicator of critical slowing down, that can be obtained by measuring the recovery rate of a system after a small experimental perturbation (van Nes & Scheffer, 2007). The last two indicators are indirect indicators of critical slowing down, that can be obtained by
measuring small fluctuations in the state of a system that are induced by natural perturbations. To capture these indirect indicators of critical slowing down, data is gathered frequently and over a prolonged period of time, using intensive time-series design (Scheffer, Carpenter, Dakos, & van Nes, 2015). Because these three indicators can be detected before the occurrence of a critical transition, these could work as so-called “early warning signals”. Therefore, measuring these signals could make it possible to predict upcoming drastic changes.
1.2 A dynamical system approach to treatment prediction
Psychological disorders can also be conceptualized as a dynamic systems in the sense that they can go to critical transitions between two alternative states; a “pathological” state and a “healthy” state (Cramer, Waldrop, van der Maas, & Borsboom, 2010). From a DST perspective, the aim of psychological interventions is to destabilize the pathological state of the patient and to help patients move to a healthy state (Cramer, Waldrop, van der Maas & Borsboom, 2010; Hofmann, Curtiss, & McNally, 2016). Accordingly, successful psychological interventions could be conceptualized as a critical transition from a pathological state to a non-pathological state (Schiepek, 2003). Indeed, change in psychological treatment has also been conceptualized as nonlinear and discontinuous (see Hayes, Laurenceau, Feldman, Strauss, & Cardaciotto, 2007, for
A
B
Figure 1. Heuristic illustration of critical slowing down. Adapted from “Early warning of climate tipping points” by T. M. Lenton, 2011, Nature climate change, 1, p. 203. Copyright 2011 by Nature Publishing Group.
a review), suggesting that principles from DST might apply well to psychological interventions. Therefore, psychological interventions might be especially effective when provided to patients who are close to a tipping point, since these patients are expected to have a higher likelihood of transitioning to a healthy state compared to patients who are further removed from a tipping point.
Up to now, the limited work into early-warning signals in psychopathology has been focused on ways to infer slowing down from indirect indicators (autocorrelation and variance) to predict future transitions into the opposite direction (from a normal state into a pathological state) using intensive time-series analyses. These studies found early-warning signals in emotion dynamics predicting the onset and termination of depression (Van de Leemput et al., 2014; Wichers, Groot, Psychosystems, ESM Group, & EWS Group, 2016), providing some validity for the application of DST to psychopathology. However, to predict treatment success, the most straightforward and time-saving way to measure the vicinity of a potential tipping point would be to directly measure the reactivity and the recovery rate of the system back to its initial state after an experimental perturbation (Scheffer et al., 2015). Therefore, the aim of the current study is to test this novel application of DST to change in psychological treatment and to test whether early warning signals have predictive validity in the context of treatment response.
1.3 IBM training as an experimental perturbation for SAD
Insight into what kind of perturbation might be effective to detect early-warning signals of treatment response in SAD can be obtained from successful perturbation experiments used in other scientific fields (e.g. Dai, Vorselen, Korolev, & Gore, 2012; van Nes & Scheffer, 2007; Veraart et al., 2012). Although these studies used perturbations that were based on the characteristics of that specific system, three generic criteria for effective perturbations can be extracted from these studies: 1) The experimental perturbation is known to be able to create a positive-feedback loop with the dynamic system, causing a system to have alternative stable states, 2) The experimental perturbation needs to be small, posing no risk of driving the system over the tipping point, 3) The size of the experimental perturbation can be standardized, allowing for a direct comparison of reactivity and recovery rates across individuals.
Based on these three criteria, a promising candidate to induce a perturbation in SAD might be a single session of positive interpretation bias modification training (IBM; Mathews & Mackintosh, 2000). IBM is a computerized procedure in which participants are trained to interpret ambiguous social situations in a positive way. This cognitive perturbation fits the three criteria above. That is; 1) Negative interpretations of social situations play a key role in the development and maintenance of social anxiety (Amir, Beard, & Bower, 2005; Clark & Wells,
1995; Moscovitch & Hofmann, 2007) and multiple-sessions of IBM were able to produce a critical transition in patients with SAD from a ‘social anxious’ state towards a healthy state (e.g. Amir & Taylor, 2012; Beard & Amir, 2008), 2) A single-session of IBM is able to temporarily induce reactivity in negative and positive interpretations in patients with SAD but is unable to move patients towards a healthy state (Nowakowski, Antony, & Koerner, 2015; Steinman & Teachman, 2015), 3) IBM can be easily standardized (e.g. same amount of stimuli), allowing for direct comparisons of reactivity scores and recovery rates across individuals. Moreover, a recent study has shown that the reactivity to a different type of cognitive bias modification training (negative attention bias modification training; ABM) could predict response to CBT in individuals with SAD (Clark, Chen, & Guastalla, 2012). Although this study was not framed as an application of DST and it trained the patients into the negative instead of the positive direction, the observation that this study used a similar design as our study provides some confidence in cognitive bias modification as a novel perturbation tool.
1.4 Present study
Building on the principles from DST (Scheffer et al., 2009), that suggests the existence of generic early-warning signals that can predict the approach of critical transitions in complex dynamical systems, the aim of the present study was to examine whether measures of reactivity to, and recovery from an experimental perturbation (i.e. a single-session of IBM training) can predict treatment response to evidence-based CBT in individuals with SAD. Reactivity and recovery predictors were examined separately for negative and positive interpretational styles, since previous studies have differentiated these interpretational styles as separate constructs (e.g. Huppert, Foa, Furr, Filip, & Mathews, 2003). It was hypothesized that greater reductions in SAD symptoms from pre- to post-treatment is predicted by 1) a stronger reactivity (as indexed by the change in negative and positive interpretations) in response to the experimental IBM perturbation and 2) slower recovery rates of both negative and positive interpretations following the experimental IBM perturbation. Exploratory, it was examined whether these dynamic predictors outperform the most often studied static predictors of treatment response to CBT in SAD; gender, age, age of onset, duration of the SAD, and severity of depressive symptoms (Eskildsen et al., 2012). If it is indeed possible to predict which patients are closer to a transition point, timing of interventions could be adjusted to the state of the patient, resulting in more effective treatments.
2. Materials and Methods
2.1 Participants and Recruitment
Participants were recruited through posted announcements on various Facebook groups and through cooperation with study advisors of higher education institutions in Amsterdam. A study webpage was created that included psycho-educational information about public speaking anxiety, information about CBT in general, an outline of the study and the possibility to sign up for the study. To be included in the study, participants had to meet the following inclusion criteria: (a) at least 18 years of age; (b) a total score of 16 or higher on the Personal Report of Confidence as Speaker (PRCS; Phillips, Jones, Rieger, & Snell, 1997), which is indicative of public-speaking anxiety (Phillips, et al., 1997); (c) a total score of 30 or higher on the Liebowitz Social Anxiety Scale (LSAS-SR; Liebowitz, 1987) or a total score of 20 on the Brief Fear of Negative Evaluation Scale-II (BFNE-II; Carleton, Collimore, & Asmundson, 2007; Carleton, McCreary, Norton, & Asmundson, 2006), which have both been suggested as cut-offs for SAD (Carleton, Collimore, McCabe, & Antony, 2011; Mennin et al., 2002), and (d) a primary diagnosis of SAD according to DSM-IV-TR criteria (APA, 2013). Exclusion criteria were the following; (a) no access to internet; (b) insufficient Dutch literacy; (c) change in psychotropic medication during the 8 weeks prior to study entry; (d) current evidence-based psychological treatment; (e) current psychosis; (f) current substance dependence (other than nicotine); and (g) active suicidal plans.
The selection of participants was conducted in two steps. First, applicants filled out three online screening questionnaires; the PRCS, BFNE-II and the LSAS-SR. Second, applicants who passed this screening phase were telephoned for a diagnostic interview using the social anxiety section of the Structured Clinical Interview for DSM-IV – Axis I disorders (SCID-I; First, Spitzer, Gibbon, & Williams, 1994) to determine whether they met diagnostic criteria for a SAD. The interview was administered by a graduate clinical psychology student trained in using the SCID-I interview. The study was approved by the Ethics Review Board of the Developmental Psychology department of the University of Amsterdam (2016-DP-6501) and informed consent was obtained from all participants. Participants received a small financial reimbursement (€30, -) for completing the experimental phase of the study (IBM-training, questionnaires and Recognition Tasks).
2.2 Treatment and Therapists
Treatment was based on a previously validated group CBT protocol (Hofmann & Otto, 2008; Smits et al., 2014; Hofmann et al, 2006), consisting of 6 weekly 2-hour group sessions, with two therapists and 5-6 participants per group. In the first treatment session, participants received psychoeducation about the nature of SAD and the treatment model, emphasizing the importance of repeated exposure practice. In sessions 2 through 6, participants completed increasingly difficult public speaking exposure tasks, which were followed by video feedback and cognitive restructuring techniques. At the end of each session, participants received homework assignments to practice the techniques they learned during the session and expand their social interaction and public speaking activities. Although the in-session exposure practices of this treatment were limited to public speaking situations, previous studies have shown that treatment effects generalize to other social situations (Hofmann, 2004; Newman, Hofmann, Trabert, Roth, & Taylor, 1994). Group treatment provides many opportunities for in-session exposure exercises, while it is as effective as individual treatment (Powers et al., 2008). The treatment was given by a licensed CBT and healthcare psychologist (‘GZ-psycholoog’) and an advanced graduate clinical psychology student experienced in treating anxiety disorders. Therapists were trained in the treatment protocol and attend supervision once every two weeks, led by a health care psychologist experienced working with the Hofmann protocol.
2.3 Experimental Perturbation: Positive Interpretation Bias Modification
The experimental perturbation consisted of a single session of a positive IBM training (Mathews & Mackintosh, 2000), to temporarily reduce negative and enhance positive interpretations of social situations. In this training, participants were presented with three-line social scenarios, with a missing word in the last sentence, presented as a word fragment. Participants read each scenario line by line on a computer screen, at a rate that was self-paced by a spacebar press. After appearance of the last sentence, pressing the spacebar revealed the word fragment. Participants were instructed to type in the first missing letter of the word fragment, after pressing the spacebar as soon as they recognized the word. Completing the word fragment always resolved the ambiguity of the scenarios in a positive, non-threatening way. After completion of the word fragment, the correct word was displayed on the computer screen, followed by a yes/no comprehension question about the social scenario. Feedback was presented to reinforce the positive interpretation given to the social scenario. An example of a scenario is presented below;
You are at a course that your company have sent you on.
Your tutor ask each member of the group to stand up and introduce themselves. After your brief presentation, you guess that the others thought you sounded — con-id-nt (confident)
An example of a comprehension question with relevant feedback for this scenario might be;
Did the members of the group think you sounded uncertain? (No) Correct, You delivered your presentation with confidence.
Participants received a total of 64 ambiguous social scenarios, which were presented in eight blocks with optional rest between each block. Scenarios were based on those employed successfully in previous studies (Mathews & Mackintosh, 2000; Salemink, van den Hout, & Kindt, 2007a,b). Some of the original scenarios were replaced or modified, so that all training scenarios reflected social evaluative (28 scenarios) and public speaking (36 scenarios) situations. The same order of scenarios was applied to all participants. At the start of the training, participants were instructed to imagine themselves being in the described situation, as the use of mental imagery has been found to enhance the effects of the IBM training (Holmes, Lang, & Shah, 2009; Holmes, Mathews, Dalgleish, & Mackintosh, 2006;).
2.4 Interpretation assessment: Recognition Task
The Recognition Task (REC-T; Mathews & Mackintosh, 2000; Salemink & van den Hout, 2010) was used to assess the effects of the IBM training on negative and positive interpretations at five time points (see 2.6 Procedure and Study Design). Each REC-T consisted of six ambiguous social situations. As in the IBM training, participants were presented with short ambiguous social scenarios and were asked to complete the word fragment and to answer a comprehension question. However, here the scenario remained ambiguous after completion of the word fragment. An example scenario is as follows;
The evening class.
You’ve just started going to an evening class. The instructor asks a question and no one in the group volunteers an answer, so he looks directly at you. You answer the question, aware of how your voice must sound to the — oth--s (others).
After the presentation of six ambiguous scenarios, participants were presented with the title of each scenario and two sentences representing; a possible negative interpretation (e.g. “You answer the question, aware of how unsteady your voice sounds”) and a possible positive interpretation (e.g. “You answer the question, aware of the others listening attentively”). Participants were asked to rate how similar each sentence was to the meaning of the original scenario, using a 4-point scale (1 = very different in meaning, 4 = very similar in meaning).
To obtain an interpretation score, mean similarity ratings were calculated separately for the negative and positive interpretations, with higher scores indicating respectively a stronger negative and positive interpretational style. For each recognition task, a different sets of scenarios was used consisting of three social evaluative scenarios and three public speaking scenarios. Most scenarios were adapted from existing materials that were used successfully in previous studies (Mathews & Mackintosh, 2000; Salemink, van den Hout & Kindt, 2007a,b). New scenarios for public speaking anxiety were also created since not enough scenarios were available that focused on this specific fear. The recognition task proved to be a valid measure of interpretational styles and scores are not affected by temporarily-induced mood states (Salemink & van den Hout, 2010). To date, the test-retest reliability of the recognition task is unknown.
2.5 Measures
2.5.1 Screening instruments
The PRCS (Paul, 1966) is a 30-item true/false self-report questionnaire used for assessing public speaking anxiety. Total scores range from 0 (i.e. no fear of public speaking) to 30 (i.e. highest level of fear) with a recommended clinical cut-off score of 16 or greater (Paul, 1966; Philips, Jones, Rieger, & Snell, 1997). The PRCS has shown to have high internal consistency ( =0.91) and good convergent validity (Daly, 1978; Phillips, Jones, Rieger, & Snell, 1997).
The LSAS-SR (Liebowitz, 1987) is a 24-item self-report questionnaire that assesses anxiety and avoidance of a wide range of social situations over the past week. Items are rated on a 4-point scale (0 = none and never to 3 = severe and usually), with higher scores indicating higher levels of social anxiety symptoms. A score of 30 or above has been identified as an optimal cut-off for identifying individuals who met DSM-IV criteria for a non-generalized SAD (Mennin et al., 2002; Rytwinski et. al., 2009). The LSAS-SR has a high internal consistency ( = .95) and test-retest reliability (r= .83; Fresco et. al., 2001; Rytwinski et al., 2009).
The SAD section of the SCID-I (First, Spitzer, Gibbon, & Williams, 1994; Dutch version: Groenestijn, Akkerhuis, Kupka, Schneider, & Nolen, 1999) was administered to screen
and assess for the presence of a SAD. The inter-rater reliability for the Dutch version of the SCID-I proved to be excellent (kappa = .83; Lobbestael, Leurgans, & Arntz, 2011).
2.5.2 Treatment outcome measures
The PRPSA (McCroskey, 1970) is a 34-item self-report questionnaire that assesses participants’ affective and behavioral reactions to public speaking situations. Items are rated on a five-point scale (1 = strongly disagree to 5 = strongly agree). Scores of 131 or higher indicate high public speaking anxiety, scores of 98–131 indicate moderate public speaking anxiety and scores below 98 indicating low public speaking anxiety. The PRPSA has excellent internal consistency ( = .90) and good test-retest reliability (ρ = .84; McCroskey, 1970).
The SATI (Cho, Smits, & Telch, 2004) is a 23-item self-report questionnaire used to assess participants’ maladaptive cognitions related to public speaking anxiety, rated on a 5-point scale ( 1= I do not believe this statement at all to 5 = I completely believe this statement). Total scores range from 23 to 115 and higher scores indicate higher levels of maladaptive cognitions. The SATI has good test–retest reliability, good internal consistency ( = .95), good convergent and discriminant validity and proved to be sensitive to the effects of exposure based treatment (Balon, 2007; Cho et al., 2004).
The BFNE-II (Carleton et al., 2007, Dutch version: Cieraad & de Jong, 2007) is a 12-item self-report questionnaire assessing participants’ fear of negative evaluation, often considered as a core feature of SAD. Items are rated on a 5-point scale (0 = not at all characteristic of me, 4 =
extremely characteristic of me). Higher scores reflect greater fear of negative evaluation. The BFNE-II
has excellent internal consistency (a = .96), good convergent and discriminant validity and proved to be sensitive to the effects of CBT (Carleton et al., 2007; Carleton et al., 2006).
2.5.3 Exploratory predictors
The Beck Depression Inventory – Primary Care (BDI-PC; Beck, Guth, Steer, & Ball, 1997) was used to assess severity of depressive symptoms. The BDI-PC consists of seven-items, each scored on a four-point scale ranging from 0 to 3. Higher scores indicate more severe depressive symptoms. The BDI-PC has good internal consistency (a = .78 to 86; Beck et al., 1997; Scheinthal, Steer, Giffin, & Beck, 2001; Steer, Cavalieri, Leonar, & Beck, 1999).
Demographic characteristics (age, gender) and clinical characteristics (age of onset and duration of SAD symptoms) were collected via a short, author constructed questionnaire. The age, age of onset and duration of the SAD were coded in the number of years.
2.6 Study procedure
This study consisted of two phases (see Figure 2); the experimental perturbation phase (Week 1) and the treatment phase (Week 2-8).
2.6.1. Experimental perturbation phase
Practice session (day 1): One day before coming to the lab, participants were asked to
complete a single recognition task at home over the internet. This recognition task operated as a practice trial to ensure adequate understanding of the task and to encounter any technical difficulties or computer problems before the start of the experiment.
Laboratory session (day 2). The next day, the experimental perturbation session was
conducted at the lab. Upon entrance of the lab, participants were informed about the total procedure and asked to sign the informed consent. Following signed informed consent, participants completed a number of self-report questionnaires (BFNE, SATI, PRPSA, BDI, demographics questionnaire). Next, each participant was individually seated in front of a computer screen in a sound attenuated cubicle and completed the online pre-training recognition task (REC-T1). Afterwards, participants completed the IBM training, which was preceded by three neutral practice trials of the scenarios. The IBM training lasted for approximately 30 minutes and was followed by the online post-training recognition task (REC-T2). The laboratory session took around 60 minutes to complete.
Home-based sessions (day 3-5): After the experimental session, participants completed
another three online recognition tasks for three consecutive days at home over the internet (REC-T3 – REC-T5). To ensure that sessions were spaced as evenly as possible (24 hours), a fixed time-point to complete these tasks was agreed upon in the laboratory session. To prevent missing data, completion of the recognition task was encouraged in three ways: 1) by sending automatic emails and text messages directly when a new task became available and to remind them of completing the recognition 4 hours after the appointed time, 2) by contacting participants by phone when they had not yet completed the recognition task 6 hours after the appointed time, and 3) by the possibility of email and telephone contact with the experimenter in case participants had questions or encountered technical difficulties.
2.6.2. Treatment Phase
The next week, participants started with the six-session group CBT program. Participants completed the self-report questionnaires (SATI, PRPSA and the BFNE) at the start of the first session (pre-treatment assessment) and one week after the last treatment session (post-treatment
assessment). Moreover, participants filled out the SATI every week in the beginning of each treatment session1. After post-assessment questionnaires were obtained, participants were debriefed and compensated for their participation. At no time during the completion of the CBT program were therapists made aware of participant results derived from the pre-treatment
experimental session.
Figure 2. Overview of the study design.
Note. REC-T = Recognition Task, SATI = Speech Anxiety Thoughts Inventory, PRPSA = Personal Report of Public Speaking Anxiety, BFNE-II = Brief Fear of Negative Evaluation Scale-II.
2.7 Data reduction and statistical analyses
2.7.1 Operationalization of Reactivity and Recovery
Both reactivity and recovery scores were computed separately for the negative and positive interpretations (e.g. Huppert, Foa, Furr, Filip, & Mathews, 2003). For a graphical representation of the operationalization and computation of these predictor variables for the negative interpretations, see Figure 3.
Reactivity to IBM
Reactivity to IBM was operationalized as the change in interpretation score from pre- to post-IBM training. It was calculated by subtracting the mean pre-post-IBM interpretation score (REC-T1) from the mean post-IBM interpretation score (REC-T2), separately for negative interpretations and positive interpretations (Figure 3A). For both the negative and the positive interpretation
1 One of the analysis that was taken into consideration was also growth curve modeling, to examine whether early-warning signal could predict the shape or the rate of symptom change during CBT. However, the sample size of this
reactivity index (Reactivity_NegInt; Reactivity_PosInt respectively), negative values indicate a decrease of mean interpretation scores in response to the IBM training and positive values indicate an increase of interpretation scores response to the IBM training.
Recovery from IBM
Recovery from IBM was operationalized as the rate of return to baseline of interpretation scores after the initial reaction to IBM. Given the explorative nature of this study and the current lack of knowledge on the time span and pattern of the recovery period of a changed interpretational style, recovery rate was operationalized in two different (but overlapping) ways. Again, it was calculated separately for the negative interpretations and the positive interpretations.
The first operationalization of the recovery time was the one-day recovery rate, which was calculated by the slope between the interpretation score directly after the IBM training (REC-T2) and the interpretation score one-day after the IBM training(REC-T3) (Figure 3B). Since the time interval between REC-T2 and REC-T3 was variable between participants (range 22-32 hours), exact time points between the two task were used for the computation of the slope, using the following formulas:
1-day Recovery_PosInt = [Mean similarity rating for positive interpretations on REC-T2] – [Mean
similarity rating for positive interpretations on REC-T1] / time between REC-T1 and REC-T2 in days]
1-day Recovery_NegInt = [Mean similarity rating for negative interpretations on REC-T2] – [Mean
similarity rating for negative interpretations on REC-T1] / time between REC-T1 and REC-T2 in days]
The second operationalization of the recovery time was the three-day recovery rate (3-day Recovery_NegInt; 3-day Recovery_PosInt) which was obtained by extracting individual slope values from a simple linear regression model based on the scores for REC-T2 –REC-T5 (Figure 3C). More specifically, participants’ interpretation scores (REC-T2 - REC-T5) were regressed on time (time of completing each recognition task in days) and REC-T2 was appointed as the intercept through which the linear regression line was fitted (Lorch & Myers, 1990; Pfister, Schwarz, Carson, & Jancyzk, 2013).
For both the 1-day and the 3-day recovery rate indices; positive values for the negative interpretations (1-day Recovery_NegInt, 3-day Recovery_NegInt respectively) represents an increase in negative interpretations, indicating a recovery of negative interpretations after IBM. Negative values for the positive interpretations (1-day Recovery_PosInt, 3-day Recovery_PosInt
respectively) represent a decrease in positive interpretations, indicating a recovery of the positive interpretations after IBM.
Figure 3. Illustration of reactivity and recovery parameters for negative interpretations derived from the IBM Training: A) Reactivity: Mean interpretation score post-IBM (REC-T2) – Mean interpretation score pre-IBM (REC-T3), B) One-day recovery rate: Slope between REC-T2 and REC-T3, C) Three-day recovery rate: Slope of the linear trendline for REC-T2 to REC-T5.
Note. IBM Training = Interpretation Bias Modification Training.
2.7.2 Public speaking anxiety composite score (PSA-Composite)
In view of the large number of (predictor and) outcome variables, a composite was created combining the two public speaking anxiety questionnaires; SATI and PRPSA2. This composite score reduces multiple testing and hence the Type 1 error rate. Moreover, it increases the reliability of the outcome measure (Horowitz, Inouye, & Siegelman, 1979). Following the procedure recommended by Rosnow and Rosenthal (1991), each questionnaire was standardized across pre- and post-treatment assessments to z-scores and then the composite score was computed by calculating the mean of these two z-scores. Positive values of the PSA-composite (i.e. above the sample mean) represent higher levels of public speaking anxiety, and negative values of the PSA-composite (i.e. below the sample mean) represent lower levels of public speaking anxiety.
2 Correlational analyses revealed that the PRPSA and the SATI were significantly correlated with one another, both pre-treatment, r(31) = .61 , p = <.001, and post-treatment, r(31) =.49, p = < .01, suggesting that they indeed measure a comparable construct.
2.7.3 Data analyses
Data were analyzed with the Statistical Package for Social Sciences (SPSS), version 22.0. To examine the main question whether reactivity and recovery predicted treatment response, separate series of multiple hierarchical regression analyses were conducted for each of the predictors (Reactivity_NegInt, Reactivity_PosInt, 1-day Recovery_NegInt, 1-day Recovery_PosInt, 3-day Recovery_NegInt, 3-day Recovery_PosInt,) on each of the two types of treatment outcomes (PSA-Composite and BFNE respectively). Each multiple regression analysis included the baseline assessment of the outcome variable in step 1, to allow for a prediction of change in treatment outcome.3 After controlling for the baseline severity, the proposed predictor variable was included in step 2. For the regression analysis that focused on the predictive value of the recovery rate (1-day Recovery_NegInt, 1-day Recovery_PosInt, 3-day Recovery_NegInt, 3-day Recovery_PosInt), reactivity of the corresponding interpretational style (negative or positive) was entered in step 1 as an additional covariate to control for the initial level of reactivity.
Unless otherwise stated, test assumptions can be considered as met. Although some influential cases were identified, similar results were obtained when these cases were excluded. Therefore, no adjustments were made (for a detailed discussion of test assumptions and potential outliers, see Appendix A). In light of the modest sample size and the preliminary character of the study, an level was set at .05 for all analyses (Bender & Lange, 2001). However, it should be noted that p-values need to be interpreted with caution given the fact that multiple testing leads to an increased chance of making a Type I error.
3. Results
3.1 Participants flow
The flow of participants throughout the study is presented in Figure 4. A total of 121 interested participants signed up for the study during the one month recruitment period (April 2016 to May 2016), of whom 84 returned the screening questionnaires. Of these, 39 participants met all inclusion criteria and were included in the treatment. Participants were defined as drop-outs if they missed two or more CBT sessions (Heimberg et al., 1998; Hofmann & Suvak, 2006;).
3 This method was chosen instead of using simple or residual change scores, because it controls for the phenomenon of regression to the mean and the possibility that change between pre- and post-treatment is related to the initial level of symptom severity (i.e. participants with higher baseline severity might generally improve more than those with lower baseline severity; Vickers & Altman, 2001).
Seven participants (18%) dropped out of treatment4, all in the early stages of the intervention (before the start of the 4th session). This rate of drop-out is comparable to dropout rates reported in previous CBT studies for SAD (Hofmann & Suvak, 2006; Taylor, 1996). Additionally, one participant did not return the post-treatment questionnaire, resulting in a final sample of 31 participants. A comparison of treatment completers and non-completers revealed that there were no significant between-group differences on any of the demographic or clinical characteristics, nor on the dependent measures at baseline, all p-values > .10 (Appendix B: Table S1)5.
3.2 Participants characteristics
Demographic and clinical characteristics of the final sample at baseline are summarized in Table 1. Participants were aged between 20 and 47 years, with an average age of 24.97 years (SD = 5.38). The majority was female (84%). SAD was generally chronic, with a mean duration of 11.35 years (SD = 6.09), and an average age of onset of 13.61 years (SD = 3.51). The mean score on the PRCS was 23.90 (SD = 2.75), suggesting high public speaking anxiety (85th percentile; Phillips et al., 1997).
3.3 Missing data
A total of 26 participants (83.9%) completed all three online recognition tasks (REC-T3-REC-T5), while 5 participants (16.1%) had missing data on at least one of these tasks. Missing data percentages for each recognition task were; 9.68% for REC-T3, 12.90% for REC-T4 and 6.45% for REC-T5. Participants with missing data on one of these recognition tasks were excluded from the relevant regression analyses. As a consequence, sample size varies slightly across the predictors and various hierarchical analyses. There was no indication of selective attrition. That is, at baseline, there were no differences between participants who completed all recognition task and those who had missed at least on recognition task on any of the outcome measures or reactivity predictors, all p-values > .60.
4 Treatment completers (n=32) attended on average 5.81(SD = .40) of the 6 available sessions and treatment non-completers (n=7) attended an average of 2.00 (SD = .82) sessions.
5Analyses were also performed for the intent-to-treat sample. Results were comparable and can be requested from the author.
Table 1. Demographic and Clinical Characteristics at Baseline (n=31)
Mean (SD) Min Max
Demographic variables
Age 24.97 (5.38) 20 47
Gender (Male/Female) 5/26
Clinical Characteristics
Age of onset of SAD 13.61 (3.51) 7 22
Duration of SAD 11.35 (6.09) 3 32 Outcome Variables PSA-Composite .79 (.48) -.43 1.73 BFNE-II 33.58 (7.76) 11 48 Screening Variables LSAS 54.23 (17.35) 24 92 PRCS 23.90 (2.75) 19 29 BDI-PC 4.03 (2.74) 0 9
Note. SAD = Social Anxiety Disorder, PSA-composite = Public Speaking Anxiety composite score, BFNE-II = Brief Fear of Negative Evaluation Scale-II, LSAS = Liebowitz Social Anxiety Scale, PRCS = Personal Report of Confidence as a Speaker, BDI-PC = Becks Depression Inventory – Primary Care.
3.4. Examination of the IBM training as an experimental perturbation
Three sets of manipulation checks were conducted to examine the validity for the use of the IBM training as an experimental perturbation. First, to determine whether the IBM training was successful in modifying the positive and negative interpretations, paired-samples t-tests were performed that revealed a significant decrease of negative interpretations (from Mpre-IBM = 2.28,
SD = .49 to Mpost-IBM = 1.85, SD =.56), t(30) = -4.10, p = <.001, d = 0.74, and a significant
increase in positive interpretations (from Mpre-IBM = 2.15, SD = .05 to Mpost-IBM = 2.68, SD = .07) , t(30) = 4.42, p = <.001 , d = 0.79. Thus the IBM training was, on average, effective in inducing the respective changes in interpretations.
Second, changes in interpretations scores from the pre-IBM recognition task (REC-T1) to the post-IBM recognition task (REC-T1) were compared on an individual-level to select participants for whom the perturbation was successful. This revealed that six participants (19.4%) showed a paradoxical increase in negative interpretations and six participants (19.4%) showed a paradoxical decrease in positive interpretations. Consequently, these participants were excluded from the relevant reactivity and recovery analyses. In addition, three participants (9.7%)
experienced no change in negative interpretations and two participants (6.45%) experienced no change in positive interpretations. Given that recovery cannot be assessed when there is no initial reactivity, these participants were excluded from the recovery rate analysis of that specific interpretation. The final sample sizes for each predictor variable are presented in Table 2.
Finally, it was examined whether the IBM training did not trigger itself a transition in public speaking or social anxiety symptoms. Paired sample t-tests revealed that, on a group level, there were no significant reductions in any of the public speaking or social anxiety measures from the lab session to the start of treatment, all p-values >.30. Although severalindividuals did report a reduction in public speaking or social anxiety severity from the lab session to the start of treatment, partial correlations showed that symptom reductions were not significantly associated with a change in negative or positive interpretations after the IBM training, all p-values >.15. Taken together, despite the fact the IBM training failed to induce reactivity of interpretations in several participants, the IBM training proved to be a valid experimental perturbation for the majority of the participants.
3.5 Differential Reactivity and Recovery responses to the Cognitive Perturbation
The characteristics of the reactivity and recovery predictors for the positive and negative interpretations are presented in Table 2. Figure 5 shows the development of interpretation scores throughout the multiple assessments with the recognition tasks.
3.5.1 Reactivity
Consistent with the position that individuals will vary in their degree of reactivity to the IBM training, the degree to which participants responded to the IBM training was highly variable for both the negative interpretations and positive interpretations (see Table 2). Although negative interpretation scores obtained in the pre-IBM recognition task (REC-T1) were correlated with those in the post-IBM recognition task (REC-T2), r(25) = .44, p =.03, only 19.6% of the variance was shared, further indicating that responses to the IBM training were highly variable. Similarly, positive interpretation scores obtained in the pre-IBM training (REC-T1) were correlated with those in the post-IBM recognition task (REC-T2), r(25) = .47, p = .02, with only 22% of the variance shared.
3.5.2. Recovery rates
There was a significant increase of the negative interpretation scores from REC-T2 to REC-T5, t(21) = 2.39, p = .03, and a significant decrease of positive interpretation scores from REC-T2 to REC-T5, t(20) = -2.04, p = .05, indicating respectively a recovery of the negative and
positive interpretations (Figure 5). However, participants did not recover fully back to their baseline levels, since there were significant differences between the baseline interpretation score (REC-T1) and the interpretation score at REC-T5 for both the negative interpretations (Mchange = -.27, SD = .52), t(20) = 2.39, p = .03, and the positive interpretations (Mchange = .48, SD = .64),
t(20) = -3.45, p < .01, indicating that on average the IBM training effects remained even three
days later. As can be seen in Table 2, there was considerable variance in the recovery rates of both the positive and negative interpretations. (For the individual patterns of similarity ratings for each interpretation bias throughout the series of recognition tasks, see Appendix C).
3.6 Treatment effect
Regarding the effectiveness of the CBT program, there were significant reductions in all measures of public speaking and social anxiety from pre- to post-treatment (see Table 3), suggesting that the treatment was successful in reducing anxiety. Important for the present study, there was considerable variance in the magnitude of the change on all outcome measures. As can be seen in Table 3, participants showed substantial individual differences in the size of symptom reduction after the group CBT program.
Table 2. Mean (and SD) and range for the predictor variables.
Predictor variable n Mean (SD) Range
Reactivity_NegInt 25 -.59 (.52) -2.17 - 0 Reactivity_PosInt 25 .72 (.60) 0 – 2.00 1-day Recovery_NegInt 22 .14 (.48) -.68 - 1.05 1-day Recovery_PosInt 21 -0.37 (.54) -1.88 - .46 3-day Recovery_NegInt 20 .10 (.18) -.19 - .48 3-day Recovery_PosInt 19 -.11 (.23) -.52 - .41
Note. Reactivity_NegInt = Reactivity of negative interpretations, Reactivity_PosInt = Reactivity of positive interpretations, day Recovery_NegInt = One-day recovery rate of negative interpretations, 1-day Recovery_NegInt = One-1-day recovery rate of positive interpretations, 3-1-day Recovery_NegInt = Three-day recovery rate of negative interpretations, 3-day Recovery_PosInt = Three-day recovery rate of positive interpretations.
Figure 5. Mean interpretation scores, separately for negative interpretations (n=22) and positive interpretations (n=23). Error bars represents standard errors of the means. Mean interpretation scores ranged from 1 (very different) to 4 (very similar). Note. IBM Training = Interpretation Bias Modification Training.
Table 3.
Mean (and SD) questionnaire scores at pre- and post-treatment, and the mean, maximum, minimum, significance and effect size of decrease from pre- to post-treatment (n=31).
Symptom reduction pre-post
Questionnaire Pre Post Mean Max Min t d
PSA-Composite .79(.48) -.79(.57) -1.59 -2.94 -.14 13.09* 2.35 SATI 84.55 (11.51) 50.87 (13.06) -33.68 -64.00 -3.00 11.97* 2.19 PRPSA 141.94 (11.19) 108.71 (14.78) -33.23 -71.00 -3.00 11.63* 2.09 BFNE-II 33.58 (7.76) 25.68 (10.33) -7.90 -21.00 5.00 5.69* 1.02 Note. PSA-composite = Public Speaking Anxiety composite score, SATI = Speech Anxiety Thoughts Inventory, PRPSA = Personal Report of Public Speaking Anxiety, BFNE-II = Brief Fear of Negative Evaluation Scale-II.
3.7 Zero-order Correlations
The zero-order correlations among the predictor variables and outcome variables are presented in Table 4. The one-day and three-day recovery rates were highly positively correlated with each other for both the negative (1-day and 3-day Recovery_NegInt) and positive interpretations (1-day and 3-day Recovery_PosInt), confirming that these constructs are conceptually related. Regarding correlations with pre-treatment scores, a stronger reactivity of negative interpretations (Reac_NegInt) was associated with a higher pre-treatment PSA-composite score. Moreover, a faster recovery of negative interpretations in the three days after the IBM training (3-day Recovery_NegInt) was related to higher pre-treatment PSA-composite and BFNE scores. Concerning post-treatment scores, post-treatment PSA-composite scores correlated neither with the pre-treatment PSA-composite scores nor with any of the predictor variables. In contrast, post-treatment BFNE scores were positively correlated with pre-treatment BFNE scores, indicating that higher pre-treatment social anxiety symptoms were related to higher post-treatment social anxiety symptoms. Furthermore, the only predictor that was significantly correlated to the post-treatment BFNE scores was the 3-day recovery rate of negative interpretations (3-day Recovery_NegInt), suggesting that a faster recovery of negative interpretations following the IBM training was related to a poorer treatment outcome in terms of social anxiety symptoms.
3.8 Main analyses
3.8.1. Does reactivity to the IBM training predict change in social anxiety symptoms following CBT?
The first hypothesis was that greater reactivity of both negative and positive interpretations in response to the experimental perturbation (IBM training) would predict greater reductions in SAD symptoms from pre to post-treatment. Results revealed that neither reactivity of negative nor of positive interpretations predicted change in PSA-composite scores (negative: = -.201, p =.373; positive: = .113 , p =.595) or BFNE scores (negative: = -.082, p =.593; positive: = .09, p =.579) at post-treatment (see Table 5). Thus, inconsistent with the first hypothesis, reactivity to the IBM-training was not predictive of symptom improvement from pre- to post-treatment in participants with SAD.
Table 4. Zero-Order Correlations for predictors variables and treatment outcomes 1 2 3 4 5 6 7 8 9 1. Reac_NegInt (n=25) - 2. Reac_PosInt (n=25) -.170 - 3. 1-day Recovery_NegInt (n=22) -.164 -.345 - 4. 3-day Recovery_NegInt (n= 20) .172 -.332 .711*** - 5. 1-day Recovery_PosInt (n=21) -.178 -.432 -.139 -.031 - 6. 3-day Recovery_PosInt (n=19) -.541* -.521* .110 .007 .728*** - 7. Pre-PSA Composite (n=31) .427* -.083 .239 .470* -.113 -.213 - 8. Post-PSA Composite (n=31) -.039 .096 -.045 -.221 -.037 -.196 .011 - 9. Pre-BFNE (n=31) .123 -.078 .302 .590* -.010 -.150 .577*** .094 - 10. Post-BFNE (n=31) .006 .043 .323 .502* -.178 -.244 .280 .257 .668*
Note. Reactivity_NegInt = Reactivity of negative interpretations, Reactivity_PosInt = Reactivity of positive interpretations, 1-day Recovery_NegInt = One-day recovery rate of negative interpretations, 1-day Recovery_NegInt = One-day recovery rate of positive interpretations, 3-day Recovery_NegInt = Three-day recovery rate of negative interpretations, 3-day Recovery_PosInt = Three-day recovery rate of positive interpretations, Pre-PSA = Pre-CBT Public Speaking Anxiety composite score; PSA = CBT Public Speaking Anxiety composite score; Pre-BFNE = Pre-CBT Brief Fear of Negative Evaluation score,
Post-BFNE = Post-CBT Brief Fear of Negative Evaluation score.
3.8.2 Does recovery from the IBM training predict change in social anxiety symptoms following CBT?
The second hypothesis was that a slower recovery rate of both positive and negative interpretations following the experimental perturbation (IBM training) would predict greater reductions in SAD symptoms from pre to post-treatment. Results revealed that the one-day recovery rate of negative interpretations did not significantly predict change in PSA-Composite ( = -.266, p =.248) or BFNE scores ( = .089, p =.620) at post-treatment (see Table 6). Similarly, the three-day recovery was not a significant predictor of change in BFNE scores ( = .129, p =.547). However, the three-day recovery rate did significantly predict change in PSA-Composite scores ( = -.543, p =.024). Contrary to the direction hypothesized, faster recovery rates of negative interpretations were related to a greater decline in presentation anxiety symptoms at post-treatment. This predictor accounted for an additional 22.6% of the variance in public speaking anxiety outcome, after controlling for baseline PSA-Composite and initial reactivity, and this increase was significant, F(1,16) = 6.255, p = .024. Given the small sample size of this analysis and the possibility that this finding is highly unstable, this regression was repeated using the simple change score of the PSA-Composite (Post PSA-composite – Baseline Table 5.
Hierarchical multiple regression analyses: Reactivity measures predicting treatment outcome (n=25)
PSA-Composite BFNE-II Predictor B SE B B SE B Negative interpretations Step 1 ( R² = .087) ( R² = .500***) Baseline severity .320 .216 .295 .873 .182 .707*** Step 2 ( R² = .120, R² = .033) ( R² = .507***, R² = .007) Baseline severity .413 .240 .381 .885 .186 .717*** Reactivity_NegInt -.213 .235 -.201 -1.537 2.835 -.082 Positive interpretations Step 1 ( R² = .035) ( R² = .400***) Baseline severity .192 .211 .187 .846 .216 .633*** Step 2 ( R² = .048, R² = .013) ( R² = .409***, R² = .009) Baseline severity .202 .215 .196 .856 .220 .640*** Reactivity_PosInt .102 .189 .113 1.564 2.775 0.09
Note. Reactivity_NegInt = Reactivity of negative interpretations, Reactivity_PosInt, PSA-Composite = Public Speaking Anxiety composite score; BFNE-II = Brief Fear of Negative Evaluation-II;
PSA-Composite) as the dependent variable. Again, the three day recovery rate of negative interpretations was a significant predictor of presentation anxiety symptoms ( = .614, p = .004),
validating the stability of this finding for at least this sample. For the positive interpretations, neither the one-day recovery rate nor the three-day recovery rate predicted changes in the PSA-composite day: = .013, p =.962; three-day: = -.145, p = .637) or BFNE scores (one-day: = -.205, p =.299; three-(one-day: = -.135, p = .552) at post-treatment (see Table 7). Hence, the only recovery rate predictor that was associated with increased symptom improvement post-treatment was the 3-day recovery rate of the negative interpretations. However, in contrast with the direction hypothesized, a faster three-day recovery rate of negative interpretations in response to the IBM training predicted more improvement in public speaking anxiety symptoms.
3.9 Exploratory analysis
As an additional exploratory analysis, it was examined whether significant dynamic predictors from the previous regression analysis (i.e. the 3-day Recovery_NegInt) could outperform the most often studied static predictors of treatment response in SAD (i.e. gender, age, age of onset, duration of the SAD, and severity of depressive symptoms). Given the small sample size (20 participants) available for the three-day recovery rate of negative interpretations, this exploratory analysis was conducted in two steps (Deblinger, Mannarino, Cohen, & Steer, 2006; van Minnen, Arntz, & Keijser, 2002). First, separate simple regression analyses were conducted to determine which static predictors were, by itself, associated with change in PSA-Composite scores. Each regression analysis included the pre-treatment PSA-PSA-Composite in step 1 and the static predictor in step 2, to allow for a prediction of change in treatment outcome. Second, a hierarchical regression analyses was conducted with the baseline score of the PSA-Composite entered in step 1, the preselected static predictors entered in step 2, and the dynamic predictor (3-day Recovery_NegInt) entered in step 3. Because 18 of the 20 participants were female, the relationship between gender and treatment response was not evaluated (Cohen, Cohen, West, & Aiken, 2013). To prevent type II errors in the first step, static predictors with a
p-value of <0.1 were included in the hierarchical linear regression model (Stevens, 1996).
However, the separate regression analyses failed to demonstrate significant associations between treatment response and any of the static predictor variables in the present study, including age ( = .133, p = .483), age of onset ( = .048, p = .799), duration of SAD ( = .089, p = .638), or BDI-PC ( = -.065, p = .734). Thus, no static predictors were identified.6
6 These results are based on the total sample of 31 participants. Results were comparable when conducted for the 20 participants included in the 3-day recovery rate analyses for negative interpretations, all p-values <.50.
Table 6.
Hierarchical multiple regression analyses: 1-day and 3-day recovery rates of negative interpretations predicting treatment outcome variables.
PSA-Composite BFNE-II Predictor B SE B B SE B 1-day Recovery (n=22) Step 1 ( R² = .200) ( R² = .513**) Baseline severity .539 .249 .517* .891 .199 .730*** Reactivity_NegInt -.339 .254 -.319 -3.208 3.211 -.163 Step 2 ( R² = .259, R² = .059) ( R² = .520**, R² = .007) Baseline severity .660 .266 .633* .853 .216 .699*** Reactivity_NegInt -.449 .267 -.422 -2.803 3.372 -.142 1-day Recovery_NegInt -.297 .249 -.266 1.844 3.652 .089 3-day Recovery (n=20) Step 1 ( R² = .196) ( R² = .534**) Baseline severity .529 .260 .523+ .863 .196 .748** Reactivity_NegInt -.299 .266 -.288 -2.392 3.171 -.133 Step 2 ( R² = .422*, R² = ..226*) ( R² = .545**, R² = .011) Baseline severity .818 .225 .808** .777 .244 .574 Reactivity_NegInt -.359 .234 -.347 -2.599 3.236 -.139 3-day Recovery_NegtInt -1.605 .642 -.541* 6.875 11.183 .129
Note. Reactivity_NegInt = Reactivity of negative interpretations, 1-day Recovery_NegInt = One-day recovery rate of negative interpretations, 3-day Recovery_NegInt = Three-day recovery rate of negative interpretations, PSA-Composite = Public Speaking Anxiety composite score; BFNE-II = Brief Fear of
Negative Evaluation-II.
4. Discussion
According to dynamical system theory, generic dynamic early warning signals, signaling critical slowing down of the system, can be used to predict changes across a wide range of complex systems (Scheffer et al., 2009). This is the first study to investigate the performance of these early-warning signals in predicting treatment response to evidence-based CBT in individuals with SAD. For this purpose, participants completed an experimental perturbation (a single-session of IBM training) before the start of treatment and their reactivity and recovery responses to this experimental perturbation were measured as predictors of treatment change. It Tabel 7.
Hierarchical multiple regression analyses: 1-day and 3-day recovery rates of positive interpretations predicting treatment outcome variables.
PSA-Composite BFNE-II Predictor B SE B B SE B 1-day Recovery (n=21) Step 1 ( R² = .076) ( R² = .459**) Baseline severity .267 .220 .277 .919 .236 .679*** Reactivity_PosInt .034 .209 .038 .212 3.171 .012 Step 2 ( R² = .076, R² = .000) ( R² = .493**, R² = .034) Baseline severity .269 .231 .279 .903 .236 .667*** Reactivity_PosInt .040 .240 .043 -1.424 3.508 -0.078 1-day Recovery_PosInt .013 .261 .013 -4.056 3.784 -.205 3-day Recovery (n=19) Step 1 ( R² = .088) ( R² = .489**) Baseline severity .284 .230 .300 .922 .236 .706*** Reactivity_PosInt .074 .230 .078 1.679 3.350 .090 Step 2 ( R² = .102, R² = .014) ( R² = .501**, R² = .012) Baseline severity .240 .253 .253 .883 .249 .676** Reactivity_PosInt -.007 .289 -.007 .300 4.103 .016 3-day Recovery_PostInt -.359 .745 -.148 -6.384 10.505 -.135 Note. Reactivity_NegInt = Reactivity of negative interpretations, 1-day Recovery_NegInt = One-day recovery rate of negative interpretations, 3-day Recovery_NegInt = Three-day recovery rate of negative interpretations, PSA-Composite = Public Speaking Anxiety composite score; BFNE-II = Brief Fear of Negative Evaluation-II.
was hypothesized that a greater degree of symptom improvement was predicted by a stronger reactivity (hypothesis 1) and a slower recovery rate (hypothesis 2) of both negative and positive interpretations in response to this perturbation.
The results did not support our hypotheses.Despite substantial individual differences in the size of symptom reduction by the end of treatment, neither reactivity of negative nor of positive interpretations predicted improved treatment effects, thus failing to provide support for the first hypothesis. With respect to the second hypothesis, recovery rates of both negative and positive interpretations were not predictive of a greater degree of symptom improvement following CBT. Sole exception was the finding that a slower three-day recovery of negative interpretations was predictive of a smaller decrease in presentation anxiety symptoms, contradicting our initial hypothesis. However, it should be empathized that this finding was not consistently found across outcome measures and recovery periods and, in light of the small sample size and many regressions performed, needs to be interpreted cautiously and requires replication.
The current findings are in contrast with previous perturbation experiments that revealed that direct early warning signals were predictive of critical transitions in diverse dynamic systems in other fields, including yeast (e.g. Dai et al., 2012; 2013), cyanobacterial populations (Veraart et al., 2012) and lakes (van Nes & Scheffer, 2007). However, although indirect early-warning signals (i.e. autocorrelation and variance) have proven to be predictive in follow-up onset of depression (Van de Leemput et al., 2014; Wichers et al., 2016), this was the first study that designed and conducted a perturbation experiment to directly measure whether early warning signals might signal change towards a non-pathological state in response to CBT. Given this preliminary nature, it is important to re-examine the study design and consider the validity of the current findings. There are several potential shortcomings of the current study design that might explain the current null results. All point to interesting potential directions for future research.
A first factor that may explain the differences between previous perturbation experiments and the work presented here concerns the operationalization of the perturbation in our study design. Previous perturbation experiments examined whether early-warning signals preceded critical transitions by gradually increasing the strength of the perturbation until the point of a critical transition, or by applying the same perturbation to systems with different distances to a tipping point (e.g. Dai et al., 2013; Drake, & Griffen, 2010). In contrast, the current study measured early-warning signals following a perturbation (IBM) that differed from the intervention (CBT) that was eventually used to induce a critical transition from a pathological state to a non-pathological state. In other words, we examined the responsiveness (i.e. reactivity
