Parental Characteristics as Predictors of Treatment
Effectiveness in a Clinical Setting for Youth with
Mental Health Problems
Ayla Vreeken, Francisca J. A. Van Steensel and Susan M. Bögels Graduate School of Childhood Development and Education, University of Amsterdam
Research Master Child Development and Education Thesis 2
Student: Ayla Vreeken, BSc Student number: 5882508
Supervisors: Drs. F. J. A. Van Steensel and Prof. dr. S. M. Bögels Date: December, 2013
Abstract
In previous studies, results regarding the effect of parental characteristics on child
psychotherapy effectiveness have been inconsistent. In addition, many studies were conducted under highly controlled circumstances that differ from “real life” clinical settings. Therefore, this study investigated the role of parental involvement, parental psychopathology, and parenting efficacy in child mental health treatment effectiveness in a clinical setting. The data contained information about 196 clinically referred children (55.1% boys; M age = 11.30 years), as reported by their parents (190 mothers and 145 fathers). Child problem behavior, parental psychopathology, and parenting efficacy were assessed at pre- and post-treatment with the Child Behavior Checklist 6-18, Adult Self-Report Form for Ages 18-59, and the competence subscale of the Parenting Stress Index, respectively. Parental involvement was operationalized as the percentage of the total number of therapy sessions that one or both parents were actively participating. It was found that parents reported a less steep decline in child problem behavior from pre- to post-treatment when parents were more involved in the treatment process and/or when parents had less psychopathological problems. The results could indicate that children benefit more from treatment if their parents are not involved, but, as parental involvement was not randomly allocated, results may also indicate that if children respond less well to treatment, parents are involved more in treatment. However, more research on the long-term effects of parental involvement, as well as research in which parental involvement is randomly assigned, is necessary in order to provide a more complete picture.
Keywords: psychotherapy, effectiveness research, clinical setting, parental involvement, parental psychopathology, parenting efficacy
Parental Predictors of Child Psychotherapy Effectiveness
Mental health problems such as psychosocial, emotional, and behavioral problems are commonly observed in children and adolescents (Costello, Egger, & Angold, 2005). The origins of adult mental health problems frequently lie in this early life stage (Hofstra, Van der Ende, & Verhulst, 2002). When these problems remain untreated, they may have major implications like school failure, criminality, violence, and lower income which may eventually lead to high societal costs (Fazel & Grann, 2006; Kessler et al., 2008; Vander Stoep, Weiss, Kuo, Cheney, & Cohen, 2003). Therefore, treatment of youth with mental health problems is of utmost importance.
Within clinical practice there is a demand for evidence based practices. However, most of these practices are based on efficacy research (i.e., whether treatment produces the
expected results under ideal circumstances). In these studies, the sample is often non-clinical, has no comorbidity, and/or treatment is given by a well-trained and carefully monitored researcher (Weisz, Donenberg, Han, & Weiss, 1995). Consequently, treatments that have proven to be effective under these highly controlled circumstances may result in less positive treatment outcomes when implemented in “real life” clinical settings (Westen, Novotny, & Thompson-Brenner, 2004). Although treatment effectiveness (i.e., whether treatment produces beneficial effects in real life settings) – as opposed to treatment efficacy - is more often being examined within clinical samples, in- and exclusion criteria like the absence of comorbidity are still applied (e.g., Bodden, Bögels, et al., 2008). Thus, real life effectiveness studies are scarce, but heavily needed.
The positive outcomes of efficacy studies provide hope for clinical practice: If the right conditions are met, therapy may be very effective. Nevertheless, even in efficacy studies, treatment is not or less effective for some individuals. For example, it was estimated that about one third of the clinically anxious children who receive cognitive behavioral therapy
(CBT) are not free of their pre-treatment primary anxiety diagnosis at posttreatment (In-Albon & Schneider, 2007). It is therefore important to examine factors that may explain differences in treatment response. Additionally, it is important to examine the characteristics of so called non-responders (i.e., those who do not respond favorably to treatment) as opposed to children who respond well to treatment. This will provide information about the conditions that promote clinically meaningful changes. One factor that may play an important role in the effectiveness of mental health treatment for children are parental characteristics. For example, parental involvement in their child’s treatment, parental psychopathology, and parenting efficacy have shown to be predictive of treatment outcome for youth with mental health problems (Beauchaine, Webster-Stratton, & Reid, 2005; Berman, Weems, Silverman, & Kurtines, 2000; Diamond & Josephson, 2005; Dowell & Ogles, 2010; Hinshaw et al., 2000; Liber et al., 2008). Each of these predictors will be discussed next.
Parents are often involved in the treatment process of their child. The underlying idea is that by educating parents about the therapeutic process and how to contribute to their child’s problem behavior reduction, parents are encouraged to take a proactive role in treatment (Walker, 2012). Several studies show that this may result in more beneficial
treatment outcomes than when parents are uninvolved (Diamond & Josephson, 2005; Dowell & Ogles, 2010). However, results have been inconsistent. For example, in the study of Bodden, Bögels, et al. (2008) it was found that child CBT was more efficacious at posttest than family CBT in treating clinically anxious children, whereas at 3-month follow-up there was no difference in efficacy between child and family CBT. The authors suggest that parents are less skilled in transferring treatment information to their child compared to therapists. Alternatively, since the effect of family CBT eventually leveled up with the effect of child CBT, it could be expected that the transfer of information from parent to child takes time and therefore produces a delay in treatment response. However, it is of note that at 1-year
follow-up, child CBT was again more effective and cost-effective than family CBT (Bodden, Dirksen, et al., 2008).
Dysfunctional family processes like parental psychopathology and parenting inefficacy can have a profound negative effect on child development and child psychopathology (Jones & Prinz, 2004; Leinonen, Solantaus, & Punamäki, 2003).
Consequentially, these processes may retain positive treatment outcomes for children’s mental health problems (Berman et al., 2000; Liber et al., 2008). For example, parental
psychopathology and parenting inefficacy may lead to certain maladaptive rearing styles that maintain children’s mental health problems (Leinonen et al., 2003). It is therefore argued by some researchers that parents with high levels of psychopathology and parenting inefficacy should be involved in their child’s treatment. This would enable parents to acquire effective parenting practices and it would enhance their feelings of competence, as a result of which positive treatment outcomes are established (Beauchaine et al., 2005; Diamond & Josephson, 2005; Hinshaw et al., 2000). In addition, learning about their child’s problems might give them insight in their own psychopathological problems, as a result of which their own problems may also diminish (Kazdin & Wassell, 2000). In contrast to these ideas, the results in the study of Bodden, Bögels, et al. (2008) suggest that anxious parents may find it difficult to transfer the tools they get in treatment to their child, and that therefore, child CBT is more efficacious than family CBT in treating clinically anxious youth who have clinically anxious parents. Taken together, there is disagreement about the effect of parental involvement in child treatment when parents have psychopathological problems and/or feel incompetent in parenting their child.
Additionally, the child’s age may be an important factor to consider in deciding whether or not parents should be involved in their child’s treatment. As children grow older,
clinical point of view, parental involvement in treatment may be undesirable for adolescents, because this may cause them to attribute the effectiveness of therapy to their parents’ skills instead of their own (Barrett, Dadds, & Rapee, 1996). Contrary, parental involvement may be of crucial importance for young children, because these children often lack the cognitive capacities to engage in treatment alone (Freeman et al., 2003). However, Bodden, Bögels, et al. (2008) found that both child CBT and family CBT were more efficacious in treating clinically anxious children than adolescents.
In conclusion, several studies found an effect of parental characteristics like parental involvement, parental psychopathology, and parenting efficacy on treatment outcomes for youth with mental health problems. However, results have been inconsistent. In addition, many studies were conducted under highly controlled circumstances that differ from real life clinical settings. Therefore, effectiveness studies are needed in order to predict treatment response based on parental variables.
The primary research question of the current study is: Which parental characteristics predict treatment effectiveness in a clinical setting for youth with mental health problems? This question is particularly relevant as studies within clinical settings are scarce and therefore the results of previous studies might be unrepresentative for the clinical youth population. Moreover, the present study provides more insight into the role that parents play in the effectiveness of their child’s mental health treatment. This information could be useful for clinical practice. For example, the results may lead to a better understanding of what drives a positive or negative treatment outcome and this may eventually lead to more effective treatments and possibly lower societal costs. Based on previous studies, it is expected that a positive treatment outcome is predicted by frequent parental involvement in treatment, low parental psychopathology, and high parenting efficacy. In addition, it is expected that the negative effects of parental psychopathology and low parenting efficacy on children’s
treatment outcome diminishes when parents are involved in treatment. Lastly, from a clinical point of view, it is expected that parental involvement in treatment is more important for young children than for adults. However, scientific results are inconsistent. Therefore, with some caution, it is hypothesized that parental involvement in treatment produces more beneficial treatment outcomes for young children than for adolescents.
A second relevant question – which will be examined exploratory – is why some children respond favourably to treatment, whereas others do not. In other words: Which characteristics of the client (system) and treatment define responders and non-responders? The findings of this question might inform clinical practice about the conditions that are important in order to promote clinically meaningful changes. In addition to the parental characteristics stated above, variables like child gender and age, treatment duration, and primary diagnosis might be relevant in defining (non-)responders. With respect to child gender and age, results have been inconsistent, ranging from no effect to superior effects for girls and children or adolescents (Bodden, Bögels, et al., 2008; Weisz, McCarty, & Valeri, 2006; Weisz, Weiss, Han, Granger, & Morton, 1995). As regards to treatment duration, some studies found a positive effect on treatment efficacy, whereas other studies found no effect (Shadish, Matt, Navarro, & Phillips, 2000; Weisz et al., 2006). Lastly, with respect to primary diagnosis, a poorer outcome was predicted by a diagnosis of conduct disorder (Pfeiffer & Strzelecki, 1990). In contrast, Weisz, Weiss, et al. (1995) found no difference in treatment efficacy between undercontrolled (i.e., aggressive, externalizing, acting out, and conduct disorder) and overcontrolled (i.e., inhibited, internalizing, shy-anxious, and personality disorder) behavioral problems.
Method Participants
The data contained information about 196 clinically referred children and adolescents (55.1% boys), as reported by their parents (190 mothers and 145 fathers). At inclusion, children’s age ranged from 6.42 to 18.48 years (M = 11.30, SD = 2.56). The participating mothers had a mean age of 43.97 (SD = 5.29) and the fathers a mean age of 45.92 (SD = 5.24). The occupational status of mothers and fathers respectively was (a) working (n = 154, 81.0% and n = 136, 93.8%), (b) not working (e.g., housewife/man, sick leave, unfit for work, or unemployed) (n = 33, 17.4% and n = 9, 6.2%), or (c) studying (n = 3, 1.6% and n = 0, 0.0%). Based on the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.) (APA, 2000), children’s primary disorder was established: (a) pervasive developmental disorder (n = 26, 13.3%), (b) attention-deficit hyperactivity disorder (n = 77, 39.3%), (c) disruptive behavior disorder (n = 6, 3.1%), (d) anxiety disorder (n = 44, 22.4%), (e) mood disorder (n = 4, 2.0%), (f) adjustment disorder (n = 21, 10.7%), (g) disorder of infancy, childhood, or adolescence not otherwise specified (n = 16, 8.2%), (h) learning disorder not otherwise specified (n = 1, 0.5%), or (i) parent-child relational problem (n = 1, 0.5%). Comorbid disorders were found in 31.1% (n = 61) of the children. These DSM-IV-TR diagnoses were established by a multidisciplinary team of psychologists, therapists, and psychiatrists at the mental health care center. No standardized protocol was used. Instead, diagnoses were based on diagnostic assessments, psychiatric consults, interviews with parent(s) and/or child, and observations. Received treatment can be categorized into (a) cognitive behavioral therapy (n = 113, 57.7%), (b) parent management training (n = 32, 16.3%), (c) individual parent and/or child support (n = 96, 49.0%), (d) medication (n = 41, 20.9 %), and/or (e) mindfulness (n = 55, 28.1%). Multiple type of treatments were received by
52% (n = 102) of the children and therefore, the above presented percentages do not add up to 100.
Measures
Child problem behavior. The Dutch version of the Child Behavior Checklist 6-18 (CBCL 6-18; Achenbach & Rescorla, 2001) is a questionnaire that measures problem
behaviors of children between six and 18 years old, as reported by caregivers. It consists of 113 items that are rated on a 3-point Likert scale (0 = not true, 1 = somewhat or sometimes true, 2 = very true or often true). In this study, the total score obtained at pre- and posttest, as reported by one or both parents, was used as an outcome measure. Reliability and validity of this questionnaire are good (Achenbach & Rescorla, 2001). In this study, Cronbach’s alpha for the mothers and fathers respectively was .93 and .94 at pretest, and .95 and .96 at posttest.
Parental involvement. Parental involvement was operationalized as the percentage of the total number of therapy sessions that one or both biological parents were actively participating. A therapy session is defined as a session in which the client and/or parent(s) are present at the mental health care centre and is directed towards improving the problem
behavior of the child or helping the client (system) understand and cope with it. Not included as therapy sessions were (pre)intakes, diagnostics, advice conversations, treatment
evaluations, homework assignments, and telephone/e-mail contacts.
Parental psychopathology. The Adult Self-Report Form for Ages 59 (ASR 18-59; Achenbach & Rescorla, 2003) was used to operationalize parental psychopathology. It is a self-report questionnaire that measures adult functioning. It consists of 123 items that are rated on a 3-point Likert scale (0 = not true, 1 = somewhat or sometimes true, 2 = very true or often true). The total score obtained at pretest, as reported by one or both parents, was used in this study both as a continuous score and to assign parents to one of two categories: non-clinical or (sub)non-clinical. These categories are based on cut-off scores as reported by
Achenbach and Rescorla (2003). Reliability and validity of this questionnaire are good (Achenbach & Rescorla, 2003). In this study, Cronbach’s alpa for the mothers and fathers respectively was .87 and .89.
Parenting efficacy. The competence subscale of the Nijmeegse Ouderlijke Stress Index (NOSI; De Brock, Vermulst, Gerris, & Abidin, 1992) – an adapted Dutch version of the Parenting Stress Index (Abidin, 1983) - was used at pretest to measure whether parents feel competent in parenting their child. It consists of 13 items (e.g., “Parenting my child is more difficult than I thought it would be”). Parents rated each item on a 6-point Likert scale (1 = I totally disagree, 2 = I rather disagree, 3 = I disagree a little bit, 4 = I agree a little bit, 5 = I rather agree, 6 = I totally agree). The total score on the subscale was used as a continuous score. In addition, based on the cut-off scores of a non-clinical norm group, as reported by De Brock et al. (1992), parents were categorized as non-clinical (≤ an average score of 31 for mothers and 30 for fathers) or clinical (≥ an above average score of 32 for mothers and 31 for fathers). Reliability and validity of the subscale are good (De Brock et al., 1992). In this study, Cronbach’s α for the mothers and fathers respectively was .87 and .89.
Procedure
The data were part of a larger general effectiveness study that were collected at UvA minds, a Dutch academic treatment centre that provides ambulant mental health care for children aged 4 to 23 years. The data were collected between the beginning of 2010 and the end of 2012. Caregivers that registered their child for diagnostics and treatment were approached by e-mail to participate in the study. They were asked to separately fill out an online questionnaire about their child’s and their own behavior, both at intake and after finishing treatment. They could do this at home or at UvA minds. When requested, caregivers received the questionnaire on paper. Inclusion criteria for the study were (a) at least one pre and one post measurement per child available, (b) respondents were the biological parents of
the child, (c) parents of the child were a heterosexual couple, (d) parents did not have another (steph)child that was registered at UvA minds, and (e) families received at least one treatment session. In addition, the children had to be diagnosed with at least one of the following DSM-IV-TR clinical disorders (APA, 2000): (a) pervasive developmental disorder, (b) attention-deficit disorder, (c) disruptive behavior disorder, (d) anxiety disorder, (e) mood disorder, (f) adjustment disorder, or (g) disorder of infancy, childhood, or adolescence not otherwise specified. All parents agreed to participate in the study, and ethical permission was given by the university’s ethical committee.
Analyses
The primary research question was investigated through a multilevel approach to repeated measures, using the maximum likelihood estimation procedure. This method takes into account the hierarchical structure of the data: Measurement occasions are nested within parents, which are nested within families, as a result of which the assumption of independent observations is violated. Furthermore, incomplete data can be analyzed with this method. In the present study, there were missing scores on the CBCL pretest (11 mothers and 57 fathers) and posttest (10 mothers and 76 fathers), on the ASR pretest (13 mothers and 59 fathers), and on the competence subscale of the NOSI pretest (14 mothers and 61 fathers). It is of note that part of these missing scores cannot be considered as true missing data, because some parents were not involved in the process of their child and therefore did not fill out the questionnaire. Children with and without missing data did not differ with respect to their gender and primary diagnosis: χ²(1) = 0.33, p = .57 for gender and χ²(7) = 7.32, p = .40 for primary diagnosis. In contrast, children with missing data had a higher mean age than children without missing data, F(1, 194) = 43.28, p = .01 (M = 11.77, SD = 2.71 vs. M = 10.83, SD = 2.32).
Prior to analyses, the assumptions were assessed. The assumptions of normality, linearity, and the absence of multicollinearity and singularity were satisfactory. Although
some variables deviated from normality, it was decided to leave them untransformed, because a multilevel approach to repeated measures is expected to be robust against this type of violation with a large sample size. Six univariate outliers and three multivariate outlier were identified. However, since the results of the analyses with and without these outliers were similar, it was decided to report the results of the total sample.
Subsequently, the analyses were performed. Child problem behavior at pre- and posttest was treated as the repeated dependent variable. Child gender and age, medication use for their mental health problems (no versus yes), and treatment duration (i.e., number of therapy sessions; see the description of parental involvement under measures for the definition of a therapy session) were treated as covariates. Parental involvement, parental psychopathology, and parenting efficacy were included as predictors. First, all covariates were included in the model. Non-significant effects were removed (except for child age, because it was hypothesized that this variable would interact with parental involvement). Second, all predictors were included as direct effects. Parental psychopathology and parenting efficacy were treated either as categorical or continuous variables, resulting in two final models. Third, these predictors were specified to be interacting with measurement occasion (two-factor interactions) in order to examine whether the change in child problem behavior from pre- to posttest was affected by these predictors. Lastly, parental psychopathology, parenting efficacy, and child age were specified to form a three-factor interaction with measurement occasion and parental involvement in order to examine whether including parents in the treatment process of their child is helpful when parents have psychopathological problems, parenting inefficacy, and when their child is young. For the final models, the
covariance matrix was specified to be unstructured. All continuous variables were
of 1), in which case parameter estimates than can be interpreted as a measure of effect: Cohen’s d for dichotomous variables and Pearson’s r for continuous variables.
The exploratory research question was investigated through a multivariate analysis of variance (MANOVA) for continuous outcome variables and chi-squared tests of
independence for dichotomous and categorical outcomes. For every child a reliable change index (RCI) of treatment effectiveness was calculated based on the total scores on the CBCL pre- and posttest. The RCI’s were based on the standard errors and Cronbach’s alphas of demographically matched referred and non-referred samples, as presented by Achenbach and Rescorla (2001). When both parents reported about their child’s problem behavior at pre- and/or posttest, their scores were averaged (correlation between mothers’ and fathers’ reports: pretest, r = .38, p < .01; posttest, r = .58, p < .01). According to Bögels and Van Melick (2004), combining information of multiple informants gives more reliable information than using single informants. Based on the RCI’s, children were categorized as responders (i.e., a clinically significant improvement in child problem behavior from pre- to posttest; RCI > 1.96) or non-responders (i.e., stability or a clinically significant deterioration in child problem behavior from pre- to posttest; RCI ≤ 1.96). The created treatment response groups were compared on several variables: child gender and age, medication use for their mental health problems (no versus yes), treatment duration, parental involvement, parental
psychopathology, parenting efficacy, and primary diagnosis. In contrast to the primary research question, parental psychopathology and parenting efficacy were only treated as continuous variables. With respect to the primary diagnosis, in order to obtain at least five observations in every cell, children were categorized based on the following primary diagnosis categories: (a) pervasive developmental disorder, (b) externalizing problem behavior (i.e., attention-deficit hyperactivity disorder and disruptive behavior disorder), (c) internalizing problem behavior (i.e., anxiety disorder and mood disorder), (d) adjustment
disorder, or (e) another disorder (e.g., disorder of infancy, childhood, or adolescence not otherwise specified).
Results Preliminary analyses
Children received an average of 13.65 (range 1-48) therapy sessions, with parents being involved in an average of 7.09 (54.2%; range 0-32) sessions. With respect to parental psychopathology (mothers: M = 30.25, SD = 23.72; fathers: M = 28.57, SD = 23.12), 282 parents (88.1%) were categorized as non-clinical (88.0% of the mothers and 88.3% of the fathers) and 38 (11.9%) as (sub)clinical (12.0% of the mothers and 11.7% of the fathers). As regards to parenting efficacy (mothers: M = 26.56, SD = 10.21; fathers: M = 25.36, SD = 9.77), 234 parents (73.8%) were categorized as non-clinical (73.1% of the mothers and 74.8% of the fathers) and 83 (26.2%) as clinical (26.9% of the mothers and 25.2% of the fathers). On average, mothers reported more child problem behavior at pretest compared to fathers, F(1, 322) = 6.37, p = .01, d = 0.28 (M = 46.80, SD = 22.97 vs. M = 40.32, SD = 22.73); at posttest there was no significant difference, F(1, 304) = 0.28, p = .60 (M = 31.94, SD = 22.51 vs. M = 33.38, SD = 23.55). With respect to treatment effectiveness, an average reduction in the child problem behavior total score from pre- to posttest was reported by both parents, F(1, 369) = 39.60, p < .001, d = 0.65 for mothers and F(1, 257) = 5.71, p = .02, d = 0.30 for fathers.
Intercorrelations of the measures at pretest are presented in Table 1. Parental
psychopathology and parenting efficacy were positively correlated, which indicates that the more psychopathological problems parents experience, the less likely it is that they feel competent about their parenting practices. Furthermore, mothers reported a positive
correlation between child problem behavior on the one hand and parental psychopathology and parenting efficacy on the other hand. For fathers, this positive correlation was only found
when parental psychopathology and parenting efficacy were treated as continuous variables, which might be due to a loss of information when categorizing variables.
Table 1
Pearson Intercorrelations for Scores on the CBCL pretest, ASR, Competence Subscale of the NOSI, Treatment Duration, and Parental Involvement as a Function of Parental Gender
Measure 1 2 3 4 5 6 1. CBCL pretest - -.08 .50** -.07 .34** -.18* 2. ASR categoricala .34** - .75** .98** .45** -.10 3. ASR continuous .38** .78** - .07 .53** -.01 4. NOSI-C categoricalb .35** .38** .46** - .81** -.09 5. NOSI-C continuous .38** .45** .53** .81** - -.07 6. Parental involvement .07 .05 .07 .16* .13 -
Note. Intercorrelations for the mothers are presented below the diagonal, and for fathers above the diagonal. Results are presented for the untransformed measures. CBCL = Child Behavior Checklist 6-18. ASR = Adult Self-Report Form 18-59. NOSI-C = competence subscale of the Nijmeegse Ouderlijke Stress Index.
a Non-clinical = 0/(Sub)clinical = 1. b Non-clinical = 0/Clinical = 1.
* p < .05. ** p < .01.
Parental predictors of treatment outcome
The fixed parameter estimates of the final models are presented in Table 2. Since child gender, medication, and the three-factor interactions did not significantly predict scores on the CBCL, they were excluded from the final model. The results presented in the final models show that parents reported significantly more child problem behavior at pretest than at
posttest (M = 44.02, SD = 23.05 vs. M = 32.50, SD = 23.09; estimates of -0.53 in model 1 and -0.52 in model 2). In addition, there was a positive association between treatment duration and child problem behavior in model 1 (i.e., higher levels of child problem behavior were related to a higher number of therapy sessions; estimate = 0.09). Furthermore, in both models, there was a positive association between child problem behavior on the one hand and parental psychopathology and parenting efficacy on the other hand. In model 1, this result indicates that parents reported significantly more child problem behavior when they had (sub)clinical psychopathological problems (estimate = 0.73) or parenting inefficacy in the clinical range (estimate = 0.49) compared to non-clinical parental psychopathology and non-clinical parenting inefficacy. In model 2, this result indicates that higher levels of child problem behavior were related to higher levels of parental psychopathology and parenting inefficacy (estimates of 0.32 and 0.18, respectively). In contrast, child age and parental involvement did not predict child problem behavior (ps > .05).
The interaction effect between measurement occasion and parental involvement was found to be positive significant in both models, which indicates that parental involvement is positively associated with child problem behavior at posttest but not at pretest, as the main effect of parental involvement was not found to be significant. This finding suggests that the more parents are involved in their child’s treatment process, the more child behavior problems parents report at posttest. In other words, parents who are more involved in their child’s treatment process, report a less steep decline in child problem behavior from pre- to posttest. However, the size of this effect was small: estimates of 0.10 in model 1 and 0.11 in model 2 (interpretable as r).
As regards to the interaction effects of measurement occasion with parental
psychopathology and parenting efficacy, there were no significant results in model 1. This suggests that the change in child problem behavior from pre- to posttest was not predicted by
Table 2
Fixed Effect Estimates of the Predictors of Child Problem Behavior
Parameter
Child Problem Behaviora
Model 1 Model 2
Estimate (SE) t Estimate (SE) t
Measurement occasionb -0.53 (0.06)** -8.53 -0.52 (0.05)** -10.07 Child agea 0.03 (0.05) 0.56 0.02 (0.04) 0.47 Treatment durationa 0.09 (0.04)* 2.00 0.08 (0.04) 1.81 Parental involvementa -0.03 (0.05) -0.58 -0.05 (0.05) -1.02 Parental psychopathologyc 0.73 (0.16)** 4.43 0.32 (0.06)** 5.63 Parenting efficacyd 0.49 (0.12)** 4.02 0.18 (0.06)** 3.13 Measurement Occasion × Parental Involvement 0.10 (0.05)* 2.03 0.11 (0.05)* 2.16 Measurement Occasion × Parental Psychopathology -0.28 (0.17) -1.62 -0.16 (0.06)* -2.56 Measurement Occasion × Parenting Efficacy 0.14 (0.12) 1.10 0.09 (0.06) 1.38
Note. Parental psychopathology and parenting efficacy were treated either as categorical (model 1) or continuous (model 2) variables. Child problem behavior was measured with the total score on the Child Behavior Checklist 6-18. Parental involvement was operationalized as the percentage of the total number of therapy sessions that one or both biological parents were actively participating. Parental psychopathology was measured with the total score on the Adult Self-Report Form for Ages 18-59. Parenting efficacy was measured with the competence subscale of the Nijmeegse Ouderlijke Stress Index.
a Continuous variables were transformed into standard normal scores (with an overall mean of 0 and a standard
deviation of 1). Their estimates can be interpreted as a measure of effect: Cohen’s d for dichotomous variables and Pearson’s r for continuous variables.
b Pretest = 0 / Posttest = 1.
c Model 1: Non-clinical = 0/(Sub)clinical = 1. d Model 1: Non-clinical = 0/Clinical = 1.
* p < .05. ** p < .01.
having (sub)clinical or non-clinical parental psychopathology or parenting inefficacy. In addition, there was no significant interaction effect of measurement occasion with parenting
efficacy in model 2. In contrast, model 2 did show a negative significant interaction effect of measurement occasion with parental psychopathology. This suggests that parents who have less psychopathological problems at pretest, report a less steep decline in child problem behavior from pre- to posttest.
Exploratory research question
Due to more than 5% missing scores on the ASR and the competence subscale of the NOSI, the analyses were performed over the sample of participants with complete data on the outcome variables (N = 123). However, it is of note that part of these missing scores cannot be considered as true missing data, because some parents were not involved in the process of their child and therefore did not fill out the questionnaire. Children with and without missing scores on the outcome variables did not differ with respect to their gender and primary diagnosis: χ²(1) = 0.28, p = .60 for gender and χ²(7) = 9.67, p = .21 for primary diagnosis. In contrast, children with missing scores had a higher mean age than children without missing scores, F(1, 194) = 9.47, p < .001 (M = 12.02, SD = 2.65 vs. M = 10.88, SD = 2.42). The assumptions of normality, linearity, homogeneous variance-covariance matrices, and the absence of multicollinearity and singularity were satisfactory. Although some variables deviated from normality, it was decided to leave them untransformed, because a MANOVA is expected to be robust against this type of violation with large sample sizes. Six univariate outliers were identified. However, since the results of the analyses with and without these outliers were similar, it was decided to report the results of the total sample.
Based on the RCI’s 46 children (37.4%) were categorized as responders and 77 children (62.6%) as non-responders (54.5% stable, 8.1% deterioration). Demographics and means of the outcome variables for the responders and non-responders are displayed in Table 3. Multivariate results showed that the combined continuous dependent variables were not significantly different between the responders and non-responders, F(7, 115) = 0.77, p = .62.
Table 3
Demographics and Means of the Outcome Variables for the Responders and Non-Responders Responders (n = 46) Non-responders (n = 77) Gender Boys (n, %) Girls (n, %) 25 21 54.3 45.7 41 36 53.2 46.8 Child Age (M, SD) 10.79 2.38 11.02 2.50 Treatment duration (M, SD) 13.86 8.74 13.93 9.73 Parental involvement (M, SD) 60.47 31.14 51.84 30.98 ASR Mother (M, SD) Father (M, SD) 26.17 28.04 18.18 25.43 31.11 29.50 25.70 20.06 NOSI-C Mother (M, SD) Father (M, SD) 24.79 24.90 9.11 9.06 26.96 26.11 8.97 10.57 Medication use No (n, %) Yes (n, %) 38 8 82.6 17.4 54 23 70.1 29.9 Primary diagnosis
Pervasive developmental disorder (n, %) Attention-deficit hyperactivity disorder (n, %) Anxiety disorder (n, %)
Mood disorder (n, %) Adjustment disorder (n, %)
Disorder of infancy, childhood, or adolescence NOS (n, %) Learning disorder NOS (n, %)
3 14 16 1 5 5 0 6.5 30.4 34.8 2.2 10.9 10.9 0.0 11 40 14 1 4 6 1 14.3 51.9 18.2 1.3 5.2 7.8 1.3 Note. Descriptive statistics are presented for the untransformed variables. Based on the reliable change index,
children were categorized as responders (i.e., a clinically significant improvement in child problem behavior from pre- to posttest; RCI > 1.96) or non-responders (i.e., stability or a clinically significant deterioration in child problem behavior from pre- to posttest; RCI ≤ 1.96). ASR = Adult Self-Report Form 18-59. NOSI-C =
Univariate results were also not significant (ps > .05). This indicates that responders and non-responders are not different with respect to child age, treatment duration, parental
involvement in treatment, parental psychopathology, and parenting efficacy. Furthermore, treatment non-responders and responders did not differ with respect to child gender,
medication use for their mental health problems, and primary diagnosis: χ²(1) = 0.01, p = .91 for gender; χ²(1) = 2.38, p = .12 for medication use; χ²(4) = 8.13, p = .09 for primary
diagnosis.
Discussion
In the present study, it was investigated which parental characteristics predict treatment effectiveness in a clinical setting for youth with mental health problems. It was found that parents reported a less steep decline in child problem behavior from pre- to post-treatment when parents were more often involved in their child’s post-treatment process. In
contrast, having (sub)clinical or non-clinical parental psychopathology or parenting inefficacy did not predict treatment effectiveness. However, when parental psychopathology was treated as a continuous variable, it was found that parents reported a less steep decline in child problem behavior from pre- to posttest when parents had less psychopathological problems. Additionally, possible differences between treatment responders and non-responders were explored. It was found that responders and non-responders did not differ with respect to child gender and age, medication use for their mental health problems, treatment duration, parental involvement, parental psychopathology, parenting efficacy, or primary diagnosis.
Contrary to expectations, it was found that parents reported a less steep decline in child problem behavior from pre- to post-treatment when parents were more often involved in their child’s treatment process. One explanation for this finding is that - as suggested by Bodden, Bögels, et al. (2008) - parents, in comparison to therapists, are less skilled in transferring treatment information to their child. However, since the results of Bodden,
Bögels, et al. (2008) also showed that family CBT was equally efficacious as child CBT at 3-month follow-up, it could be expected that the transfer of information from parent to child takes time and therefore produces a delay in treatment response. Note however that at 1-year follow-up in the study of Bodden, Dirksen, et al., child CBT was again more effective and cost-effective than family CBT. A second explanation for the results could lie in the fact that the amount of parental involvement was not randomly allocated to the participating families. It is possible that when treatment did not seem to reach success during the treatment process, parents were more involved by the therapist. So the results could indicate that children benefit more from treatment if their parents are less involved, but, as parental involvement was not randomly allocated, results may also indicate that if children respond less well to treatment, parents are more involved in treatment. In other words, the direction of the relationship is not clear. A third explanation for the results could be that parents who are more involved in treatment become more aware of their child’s problem behavior and are better able to recognize particular symptoms in comparison to parents who are less often involved.
Consequently, frequently involved parents might have a more realistic image of their child’s problem behavior after treatment and therefore report more child problem behavior at post-treatment. On the other hand, parents who are involved in treatment and put considerable effort in that treatment may report more improvement. Fourth, the preliminary analyses revealed a correlation between lower maternal parenting efficacy and more parental involvement, suggesting that mothers who have clinical scores of incompetence are more likely to be involved in their child’s treatment. It might be that mothers who feel less competent want to be more involved in treatment or are more likely to request for parental guidance. As a consequence, mothers who are often and less often involved differ
considerably with respect to parenting efficacy, and possibly also with respect to efficacy related factors that were not measured in the present study (e.g., rearing problems). This
might have led to a response bias when reporting about (changes in) child problem behavior. Of note is that the association between fathers’ parenting efficacy and parental involvement was not significant. This might be due to the fact that mothers are usually more involved in treatment and therefore, the parental involvement variable might be a better reflection of mothers’ involvement than of fathers’ involvement (Phares, Lopez, Fields, Kamboukos, & Duhig, 2005). Lastly, the present study did not control for parental adherence and motivation for therapy. These factors could have affected how much parents learned and were willing to learn from therapy, which in turn could have influenced treatment effectiveness (Nock & Ferriter, 2005).
Also in contrast to the hypotheses, having (sub)clinical or non-clinical parental psychopathology or parenting inefficacy did not predict treatment effectiveness. An explanation could be that the parents who were categorized as having (sub)clinical psychopathological problems or clinical parenting inefficacy did not have severe enough problems to find an effect: The mothers and fathers with (sub)clinical psychopathological problems had a mean score on the ASR of respectively 80.0 and 75.9, which fall just within the clinical range. In addition, the mothers and fathers with clinical parenting inefficacy had a mean score on the competence scale of the NOSI of respectively 40.1 and 38.9, which
represent an above average score for mothers and a high score for fathers (above average, high, and very high scores were considered clinical). Perhaps only extremely severe psychopathological problems and parenting inefficacy can predict treatment effectiveness. Furthermore, the ASR and NOSI do not measure whether parents experience interference of their problems with daily life, which might be a better indicator of the severity of parents’ problems and therefore a better predictor of treatment effectiveness. Another explanation could be that the amount of parental psychopathology and parenting inefficacy decreases considerably during treatment, which makes these variables unsuitable predictors of treatment
outcome. That is, during child therapy, parents might learn more about their own
psychopathological problems and their parenting skills might have improved due to parental guidance (Kazdin & Wassell, 2000). In addition, parents might feel more competent in parenting their child as a result of their child’s improvement. Indeed, post hoc analyses revealed that mothers (but not fathers) reported a significant decrease in their
psychopathological problems and parenting inefficacy from pre- to post-treatment. When parental psychopathology was treated as a continuous variable, it was found that, in contrast to the hypothesis, parents reported a less steep decline in child problem behavior from pre- to post-treatment when parents had less psychopathological problems. Perhaps children of parents with psychopathological problems have more to gain due to the maladaptive rearing environment that they are frequently brought up in (Jones & Prinz, 2004;
Leinonen, Solantaus, & Punamäki, 2003).
An account of the characteristics of responders and non-responders could not be provided. That is, responders and non-responders did not differ with respect to child gender and age, medication use for their mental health problems, treatment duration, parental involvement, parental psychopathology, parenting efficacy, or primary diagnosis. These results suggest that in general, a clinically meaningful change in child problem behavior does not depend on these variables. However, the sample was rather heterogeneous with respect to diagnosis and type of treatment. Therefore, within a certain subsample with a specific pattern of symptoms or a specific type of treatment, these variables might have an effect. For
example, in children diagnosed with an attention-deficit hyperactivity disorder (ADHD), psychosocial treatments were found to be less effective in reducing ADHD symptoms than stimulant treatments or a combination of both (Van der Oord, Prins, Oosterlaan, &
treatment fidelity may be more relevant in differentiating between responders and non-responders (Karver, Handelsman, Fields, & Bickman, 2006).
A strength of the present study is that a relatively large clinical sample with comorbid disorders was recruited. This may have resulted in outcomes that are more representative for the clinical youth population than results of previous efficacy studies. Furthermore, in addition to mothers, fathers were approached to participate in the study. Since fathers are often ignored in research related to child psychopathology, this may have provided a more complete picture (Phares et al., 2005). Lastly, parental psychopathology and parenting efficacy were included both as categorical and continuous variables, which may have provided a more differentiated picture of the role that these variables play in treatment effectiveness.
The present study also has several limitations. First, due to the inclusion of children with comorbid disorders who received different types of treatment, a rather heterogeneous sample was created. This hampers conclusion making about subsamples with a specific pattern of symptoms or a specific type of treatment. Second, parental involvement was not randomly assigned to the participating families. Therefore, it is difficult to conclude whether parental involvement produces less beneficial treatment effects, or whether parents are more involved when children respond less well to treatment. Third, due to parents’ lack of
motivation to participate in the study or difficulties in contacting parents, some parents did not fill out the questionnaire immediately after finishing treatment. Therefore, in some cases, the posttest could be contemplated as a follow-up, which might have influenced the results. Fourth, the absence of a follow-up makes it more difficult to provide a conclusive statement about the role that parental involvement plays in children’s treatment effectiveness. For example, perhaps parental factors become more important when treatment has finished and parents have to deal with their child’s behavior problems themselves without the help of a
therapist. Therefore, it would be interesting measure parents’ involvement in the child’s recovery process after treatment and what this does to the child’s behavior problems. Lastly, solely parental reports were used, which may have led to a response bias. For example, as stated above, often involved parents might have a more realistic image of their child’s
behavior problems compared to less often involved parents. In addition, parents tend to judge their own rearing behaviors more positive than their children (Bögels & Van Melick, 2004). Furthermore, research has shown that it is not the actual parenting that mainly influences child behavior, but that the child’s perception of parenting is dominant (Barry, Frick, & Grafeman, 2008). Therefore, future research should appeal to multiple and/or more objective informants, like teacher and therapist reports or observations (Bögels & Van Melick, 2004; Weisz, Weiss, et al., 1995).
Despite these limitations, the present study shows that even within a clinical setting with a rather heterogeneous group of children with various (comorbid) disorders that received different types of treatment, child mental health treatment produces beneficial effects
(Cohen’s d = -0.53, which is considered a medium effect). Although the effect size is somewhat lower than usually reported in efficacy studies (Westen et al., 2004), it is beyond expectations considering the fact that a general measure of child problem behavior was used. In addition, this study adds to the literature about the role that parental characteristics play in the effectiveness of child mental health treatment. The results might suggest that frequent parental involvement in children’s treatment process is undesirable. However, since parental involvement was not randomly allocated, results may also indicate that if children respond less well to treatment, parents are more involved by the therapist. Furthermore, these are short-term results that might look different at the long-term: Due to parental difficulties in transferring treatment information to their child, a delay in treatment response could have occurred. So parental factors might become more important when treatment has finished
(Kolko, Brent, Baugher, Bridge, & Birmaher, 2000). For example, after finishing treatment, parents’ involvement is important to guide the child in applying the learned techniques to real life situations (Walker, 2012). Besides, parents’ involvement is important to overcome
children’s treatment resistance and it might encourage parents’ empathy and sympathy for their child’s condition, which strengthens their relationship (Liddle, 2004). So excluding parents from their child’s treatment process cannot be advised based on the results of the present study. However, more research at the long-term is necessary in order to advice the clinical practice.
References
Abadin, R. R. (1983). Parenting stress index: Manual, administration booklet, and research update. Charlottesville, VA: Pediatric Psychology Press.
Achenbach, T. M., & Rescorla, L. A. (2003). Manual for the ASEBA Adult Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.
Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
Barrett, P. M., Dadds, M. R., & Rapee, R. M. (1996). Family treatment of childhood anxiety: A controlled trial. Journal of Consulting and Clinical Psychology, 64, 333-342. doi:10.1037/0022-006X.64.2.333
Barry, C. T., Frick, P. J., & Grafeman, S. J. (2008). Child versus parent reports of parenting practices: Implications for the conceptualization of child behavioral and emotional problems. Assessment, 15, 294-303. doi:10.1177/1073191107312212
Beauchaine, T. P., Webster-Stratton, C., & Reid, M. J. (2005). Mediators, moderators, and predictors of 1-year outcomes among children treated for early-onset conduct problems: A latent growth cure analysis. Journal of Consulting and Clinical Psychology, 73, 371-388. doi:10.1037/0022-006X.73.3.371
Berman, S. L., Weems, C. F., Silverman, W. K., & Kurtines, W. M. (2000). Predictors of outcome in exposure-based cognitive and behavioral treatments for phobic and anxiety disorders in children. Behavior Therapy, 31, 713-731.
Bodden, D. H. M., Bögels, S. M., Nauta, M. H., De Haan, E., Ringrose, J., Appelboom, C., . . . Appelboom-Geerts, K. C. M. M. J. (2008). Child versus family cognitive-behavioral therapy in clinically anxious youth: An efficacy and partial effectiveness study.
Journal of the American Academy of Child and Adolescent Psychiatry, 47, 1384-1394. doi:10.1097/CHI.0b013e318189148e
Bodden, D. H. M., Dirksen, C. D., Bögels, S. M., Nauta, M. H., De Haan, E., Ringrose, J., . . . Appelboom-Geerts, C. M. M. J. (2008). Costs and cost-effectiveness of family CBT versus individual CBT in clinically anxious children. Clinical Child Psychology and Psychiatry, 13, 543-564. doi: 10.1177/1359104508090602
Bögels, S. M., & Van Melick, M. (2004). The relationship between child-report, parent self-report, and partner report of perceived parental rearing behaviors and anxiety in children and parents. Personality and Individual Differences, 37, 1583-1596. doi:10.1016/j.paid.2004.02.014
Costello, E. J., Egger, H., & Angold, A. (2005). 10-year research update review: The epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 972-986. doi:10.1097/01.chi.0000172552.41596.6f
De Brock, A. J. L. L., Vermulst, A. A., Gerris, J. R. M., & Abidin, R. R. (1992). NOSI, Nijmeegse ouderlijke stress index. Lisse: Swets en Zeitlinger.
Diamond, G., & Josephson, A. (2005). Family-based treatment research: A 10-year update. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 872-887. doi:10.1097/01.chi.0000169010.96783.4e
Dowell, K. A., & Ogles, B. M. (2010). The effects of parent participation on child psychotherapy outcome: A meta-analytic review. Journal of Clinical Child and Adolescent Psychology, 39, 151-162. doi:10.1080/15374410903532585
Fazel, S., & Grann, M. (2006). The population impact of severe mental illness on violent crime. American Journal of Psychiatry, 163, 1397-1403.
doi:10.1176/appi.ajp.163.8.1397
Freeman, J. B., Garcia, A. M., Fucci, C., Karitani, M., Miller, L., & Leonard, H. L. (2003). Family-based treatment of early-onset obsessive-compulsive disorder. Journal of Child and Adolescent Psychopharmacology, 13, 71-80. doi:10.1089/104454603322126368 Hinshaw, S. P., Owens, E. B., Wells, K. C., Kraemer, H. C., Abikoff, H. B., Arnold, L. E., . . .
Wigal, T. (2000). Family processes and treatment outcome in the MTA:
Negative/ineffective parenting practices in relation to multimodal treatment. Journal of Abnormal Child Psychology, 28, 555-568. doi:10.1023/A:1005183115230
Hofstra, M. B., Van der Ende, J., & Verhulst, F. C. (2002). Child and adolescent problems predict DSM-IV disorders in adulthood: A 14-year follow-up of a Dutch
epidemiological sample. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 182-189. doi:10.1097/00004583-200202000-00012
In-Albon, T., & Schneider, S. (2007). Psychotherapy of childhood anxiety disorders: A meta-analysis. Psychotherapy and Psychosomatics, 76, 15-24. doi:10.1159/000096361 Jones, T. L., & Prinz, R. J. (2005). Potential roles of parental self-efficacy in parent and child
adjustment: A review. Clinical Psychology Review, 25, 341-363. doi:10.1016/j.cpr.2004.12.004
Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review, 26, 50-65. doi:10.1016/j.cpr.2005.09.001 Kazdin, A. E., & Wassell, G. (2000). Therapeutic changes in children, parents, and families
Academy of Child & Adolescent Psychiatry, 39, 414-420. doi:10.1097/00004583-200004000-00009
Kessler, R. C., Heeringa, S., Lakoma, M. D., Petukhova, M., Rupp, A. E., Schoenbaum, M., . . . Zaslavsky, A. M. (2008). The individual-level and societal-level effects of mental disorders on earnings in the United States: Results from the national comorbidity survey replication. The American Journal of Psychiatry, 165, 703-711.
doi:10.1176/appi.ajp.2008.08010126
Kolko, D. J., Brent, D. A., Baugher, M., Bridge, J., & Birmaher, B. (2000). Cognitive and family therapies for adolescent depression: Treatment specificity, mediation, and moderation. Journal of Consulting and Clinical Psychology, 68, 603-614.
doi:10J037//0022-006X.68.4.603
Leinonen, J. A., Solantaus, T. S., Punamäki, R. L. (2003). Parental mental health and children’s adjustment: The quality of marital interaction and parenting as mediating factors. Journal of Child Psychology and Psychiatry, 44, 227-241. doi:10.1111/1469-7610.t01-1-00116
Liber, J. M., Van Widenfelt, B. M., Goedhart, A. W., Utens, E. M. W. J., Van der Leeden, A. J. M., Markus, M. T., & Treffers, P. D. A. (2008). Parenting and parental anxiety and depression as predictors of treatment outcome for childhood anxiety disorders: Has the role of fathers been underestimated? Journal of Clinical Child and Adolescent
Psychology, 37, 747-758. doi:10.1080/15374410802359692
Liddle, H. A. (2004). Family-based therapies for adolescent alcohol and drug use research contributions and future research needs. Addiction, 99, 76–92. doi:10.1111/j.1360-0443.2004.00856.x
Nock, M. K., & Ferriter, C. (2005). Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review, 8, 149-166. doi:10.1007/s10567-005-4753-0
Phares, V., Lopez, E., Fields, S., Kamboukos, D., & Duhig, A. M. (2005). Are fathers involved in pediatric psychology research and treatment? Journal of Pediatric Psychology, 30, 631-643. doi:10.1093/jpepsy/jsi050
Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000). The effects of psychological therapies under clinically representative conditions: A
meta-analysis. Psychological Bulletin, 126, 512-529. doi:10.1037//0033-2909.126.4.512 Van der Oord, S., Prins, P. J., Oosterlaan, J., & Emmelkamp, P. M. (2008). Efficacy of
methylphenidate, psychosocial treatments and their combination in school-aged children with ADHD: A meta-analysis. Clinical Psychology Review, 28, 783-800. doi:0.1016/j.cpr.2007.10.007
Vander Stoep, A., Weiss, N. S., Kuo, E. S., Cheney, D., & Cohen, P. (2003). What proportion of failure to complete secondary school in the US population is attributable to
adolescent psychiatric disorder? The Journal of Behavioral Health Services & Research, 30, 119-124. doi:10.1007/BF02287817
Walker, J. V. (2012). Parental factors that detract from the effectiveness of
cognitive-behavioral treatment for childhood anxiety: Recommendations for practitioners. Child & Family Behavior Therapy, 34, 20-32. doi:10.1080/07317107.2012.654433
Weisz, J. R., Donenberg, G. R., Han, S. S., & Weiss, B. (1995). Bridging the gap between laboratory and clinic in child and adolescent therapy. Journal of Consulting Clinical Psychology, 63, 688-701. doi:10.1037/0022-006X.63.5.688
Weisz, J. R., McCarty, C. A., & Valeri, S. M. (2006). Effects of psychotherapy for depression in children and adolescents: A meta-analysis. Psychological Bulletin, 132, 132-149. doi:10.1037/0033-2909.132.1.132
Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of
psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117, 450-468.
doi:10.1037/0033-2909.117.3.450
Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004). Empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130, 631-663. doi:10.1037/0033-2909.130.4.631
Zimmer-Gembeck, M. J., & Collins, W. A. (2003). Autonomy development during adolescence. In G. R. Adams & M. Berzonsky (Eds.), Blackwell handbook of adolescence (pp. 175-204). Malden, Massachusetts: Blackwell Publishing.
