The effectiveness of Multisystemic treatment (MST):
A Meta-analysis
Masterthesis Forensic Child and Youth Care Services University of Amsterdam T. van der Stouwe Supervisors: Dr. J.J. Asscher, Prof. dr. G.J.J.M. Stams Amsterdam, April 2013
The effectiveness of Multisystemic treatment (MST):
A Meta-analysis
Masterthesis Forensic Child and Youth Care Services University of Amsterdam T. van der Stouwe Supervisors: Dr. J.J. Asscher, Prof. dr. G.J.J.M. Stams Amsterdam, April 2013
Contents Abstract 2 Introduction 3 Method 7 Selection of studies 7 Inclusion criteria 7
Coding the studies 8
Calculation and analysis of effect sizes 9
File drawer analysis 11
Results 13
Moderator analysis 16
Study design characteristics 17
Publication characteristics 17
Treatment characteristics 17
Sample characteristics 17
Correlations among moderators 17
Multiple regression analyses 19
Discussion 20
Abstract
Multisystemic treatment (MST) is worldwide considered one of the most effective evidence-based treatments for juveniles with social, emotional and behavioural problems. A meta-analysis of k = 22 studies and 4066 juveniles was conducted to examine the
effectiveness of MST. MST yielded small but significant effects on recidivism, number of offenses, violent offenses, out-of-home placement, internalising and externalising behavior, self-reported delinquency, psychopathology, family functioning and positive parenting. Moderator analysis showed that earlier studies, studies conducted by dependent (i.e.
associated with the developers of MST) researchers, published studies, efficacy studies and studies with a longer follow-up period reported larger effects. No effects were found when compared to multimodal control treatment. MST proved to be most effective with younger, male, Caucasian juvenile offenders. It was argued that the effectiveness of MST could be improved by focusing more on peer relationships and risks and protective factors in the school domain.
Keywords: multisystemic treatment, MST, effectiveness, meta-analysis, juvenile delinquency
Introduction
Multisystemic treatment (MST) is a multi-faceted, short-term, home- and community-based intervention, disseminated in fourteen countries (MST Services Inc., 2010), which is worldwide considered to be one of the most effective evidence-based treatments for juveniles with social, emotional and behavioural problems.
MST is based on the premise that adolescent delinquency is associated with an accumulation of criminogenic risk factors (e.g., Loeber, Farrington, Stouthamer-Loeber, & Van Kammen, 1998), in particular individual, family, peer, school and neighbourhood characteristics (Henggeler, 2011; Henggeler, Schoenwald, Borduin, Rowland, &
Cunningham, 2009b; Henggeler, Schoenwald, Rowland, & Cunningham, 2002a). The idea that these factors should be targeted simultaneously finds its base in Bronfenbrenner’s (1979) bio-ecological-systems approach, which assumes that human behaviour develops within and across contexts . MST mainly focuses on improving family functioning, because it is
theorised that improvements in family functioning mediate improvements in peer
relationships, school functioning and participation in the community (MST theory of change, Henggeler, 2011). Furthermore, the implementation of MST is highly flexible and designed to address specific individual risk factors. This is in line with the Risk-Need-Responsivity (RNR)-model (e.g. Andrews & Bonta, 2010; Andrews, Bonta & Hoge, 1990; Andrews, Bonta, Wormith, 2006), which states that judicial interventions should take into account the recidivism risk, and be matched to the criminogenic needs and learning style and capabilities of the individual.
Therapists visit the juveniles and their families at home and/or in their community to reduce drop-out rates, provide treatment exactly where and when it is needed and to increase generalizability. Moreover, the therapist is available twenty-four hours a day, seven days a week and therapeutic sessions take place up to every day. MST uses well-established treatment strategies derived from strategic family therapy, structural family therapy,
behavioural parent training and cognitive-behavioral therapy (Borduin, 1999). Finally, MST is accompanied by training and supervision, organisational support and adherence measures to monitor treatment adherence (Henggeler, 2011).
Since the first efficacy trial (Henggeler et al., 1986) and subsequent implementation of MST, the treatment has been implemented in a growing number of teams, regions and
countries. The target population has expanded from delinquent and antisocial juveniles to abused and neglected juveniles (e.g., Brunk, Henggeler, & Whelan, 1987), sex offenders
(e.g., Borduin, Henggeler, Blaske, & Stein, 1990), youth with psychiatric emergencies (e.g., Henggeler et al., 1999b), substance-abusing and –dependent juveniles (e.g., Henggeler, Pickrel, & Brondino, 1999a), youth with poorly controlled type I diabetes (e.g., Ellis et al., 2004), and juveniles with obesity (e.g., Naar-King et al., 2009). All of these variants have been examined at least once, resulting in a total of 20 published randomised controlled trials in 2012 (MST Services inc., 2012).
With the growing number of randomized controlled studies, a meta-analysis of the effectiveness of MST became possible. To date, two of such meta-analyses have been conducted: one by dependent researchers, associated with the developers of MST (Curtis, Ronan, & Borduin, 2004) and the other by independent researchers (Littell, Campbell, Green, & Toews, 2005). Notably, Petrosino and Soydan (2005) reviewed 50 meta-analyses of social interventions and conducted a meta-analysis of 300 randomized field trials of interventions targeting recidivism in order to examine the impact of dependency of researchers on study outcomes. They found that research conducted by dependent researchers yielded consistently and substantially larger effect sizes.
The first meta-analysis by Curtis et al. (2004) included outcome studies published in peer reviewed journals between 1987 and 2002. Studies were included if they focused on MST, used random assignment, included juveniles or parents with antisocial behaviour and/or psychiatric symptoms, used pre- and post-treatment and/or follow-up assessment measures, and described test statistics suitable for meta-analysis. These criteria resulted in eleven
eligible studies, consisting of seven independent (non-overlapping) samples and a total of 708 juveniles. Youth received MST because they were classified as juvenile offenders (59%), substance abusers (17%), required emergency psychiatric hospitalization (16%), were
classified as abused or neglected (6%) or as a sexual offender (2%). Four out of seven studies compared MST with a wide array of usual services (mixed treatment), one with parent
training and the remainder with individual therapy.
Curtis et al. (2004) reported a moderate overall effect (d = .55) of MST compared to the control group. Furthermore, larger effect sizes were found specifically for family relationships compared to individual adjustment and peer relationships. The type of target population did not moderate treatment effects, although optimal conditions of delivery yielded larger effects than clinical representative conditions (i.e., efficacy versus real-world
conditions ,Flay et al., 2005).
Shortly after publication of the Curtis et al. review, Littell et al. (2005) published their Cochrane systematic review of MST. Littell and colleagues used different selection criteria
and methods of analysis than Curtis et al. did, which yielded different outcomes than the latter meta-analysis. In contrast to the first meta-analysis, which only included peer review
published articles, Littell et al. did not use publication or language restrictions, thus including non-published studies as well. Other inclusion criteria were roughly the same as those used by Curtis et al. (2004), with the exception that outcome data with the latter had to be provided for the full sample with a response rate of at least 60% (no such criterion was reported in the Curtis et al. study). Furthermore, instead of the Cohen’s d effect size used by Curtis et al., Hedge’s g – an effect size that adjusts for small sample sizes, was used. One study on abused/neglected juveniles was excluded because it provided insufficient data, and two additional studies in Canada (with juvenile offenders) and in Norway (with more general problem behaviours) were included. Consequently, the Little et al. meta-analysis consisted of 21 studies from eight independent samples, and a total of 1230 juveniles for whom they found inconclusive evidence for the effectiveness of MST.
More than these contradictive outcomes, an article by Littell (2005) about methods used in systematic reviews raised controversy about the evidence-base of MST. In this article Littell used the MST meta-analysis as an example to display issues and limitations in
assessing the available studies for the analyses and the way previous (selective) reviews reproduced results from individual studies. First, Littell explained that little of the 27 published reviews of research on MST included explicit inclusion/exclusion criteria, systematic searches of electronic databases, unpublished studies, explicit study quality criteria, intent-to-treat or attrition or blinding to treatment conditions. This implicates lower reliability of the outcomes of these studies, while the aforementioned key organisations base their judgement on the effectiveness of MST largely on these reviews.
Second, Littell elaborated on problems encountered with the encoding of selected studies in the meta-analysis. These problems included (1) inconsistent reports on the number of cases in the study, (2) unjust handling of yoked designs, (3) unclear randomization
procedures, (4) largely varying follow-up periods and (5) subjective definition of treatment completion. Furthermore, she stated that only one study – non-published, therefore not included in the Curtis et al. (2004) study – was of high quality based on study design and implementation criteria.
Finally, Littell indicated inter alia publication bias and conflicts of interests with researchers as causes for differences between the Curtis et al. (2004) and Littell et al. (2005) meta-analyses. In a response, Henggeler, Schoenwald, Borduin and Swenson (2006b),
however, stated that Littell did not appreciate the nature of and inherent issues with treatment research. Littell (2006) in her turn, refuted those allegations.
Obviously, the two meta-analyses and the subsequent discussion between Littell and Henggeler and colleagues contributed to a more critical view of the evidence-base for treatments, especially for MST. Since these publications the (international) research base for MST has grown. Almost a decade after the previous meta-analyses it therefore seems time to conduct a new meta-analysis. Littell et al. (2005) already expressed their hopes for this.
The main aim of the present meta-analysis therefore is to examine the impact of MST with juvenile offenders on both ultimate outcomes (i.e., crime and delinquency and out-of-home placements) and other behavioural and psychosocial outcomes. Another aim is to assess moderators, such as treatment and study characteristics, that possibly influence the outcomes. This meta-analysis differs from the previous meta-analyses with respect to several aspects, to account for disadvantages of the previous analyses and to make maximum use of the grown body of research on MST.
First, in contrast to the Curtis et al. (2004) meta-analysis, non-published studies will be included to account for publication bias. Published studies on average show larger treatment effect than non-published studies (McAuley, Pham, Tugwell, & Moher, 2000; Hopewell, McDonald, Clarke, & Egger, 2007). Excluding unpublished studies could therefore result in an overestimation of the actual effect size.
Second, in contrast to both meta-analyses, non-randomised (quasi-experimental) studies are included. Although randomised controlled trials are considered a more valid study design for causal inference, Shrier et al. (2007) found that non-randomised studies produce similar effects, and that the advantages of including non-randomised studies outweigh the disadvantages. Including these studies in the current analyses yields a larger number of studies to be analysed. The possible influence of study quality (Moher et al., 1998) will be accounted for by adding a study quality index.
Third, again in contrast to both previous analyses, only antisocial, conduct disordered and/or delinquent juveniles receiving regular MST are included. Juveniles receiving MST for abuse and neglect, diabetes, psychiatric emergencies and obesity were excluded, because these populations differ from the original target population, which could result in differences in the way MST is carried out and different treatment effects. Exclusion of these studies is thought to yield results that can better be generalized to the population of juvenile delinquents receiving MST.
Finally the larger number of studies compared to the previous meta-analyses makes it possible to (a) analyse more outcome measures and (b) conduct moderator analyses to assess the influence of study, treatment and sample characteristics on treatment effects and (c) analyse outcome measures over a longer follow-up period. Moreover, Littells’ (2005)
assumptions about publication bias and conflicts of interest can be tested. Finally, the present meta-analysis can shed a new, more conclusive, light on the effectiveness of MST.
Method
Selection of studies
All reports from 1985 until 2012 addressing the effectiveness of MST with antisocial or delinquent juveniles were included. This particular starting year was chosen because the first research on MST – not even named MST yet – was published in 1986 (Henggeler et al., 1986).
First, three electronic databases were searched using the search string “Multisystemic therapy” OR “Multisystemic treatment”: Science Direct, Web of Knowledge and Google Scholar. Furthermore, MST reviews and primary studies reference sections were searched for qualifying studies: both Curtis et al. (2004) and Littell et al. (2005) meta-analyses, the MST services inc. website including referrals to other website and databases and MST overview reports (e.g. MST Services inc., 2012; Henggeler, 2011). This search yielded 112 reports of which 51 studies met the inclusion criteria of our meta-analysis, which resulted in 22 independent (non-overlapping) samples.
Inclusion criteria
To be included in the current meta-analysis, studies had to meet the following criteria: (1) focus on regular MST, (2) treat antisocial, conduct disordered and/or delinquent juveniles, (3) assignment of participants to MST and one or more control group(s), (4) provide pre- and post-treatment assessment measures and/or follow-up assessment measures, (5) provide statistics suitable for meta-analysis, and (6) report in either Dutch or English.
Two studies were excluded because they consisted of more than one and other than usual treatment comparison group, and no usable statistics were reported to separate only MST from control treatment statistics (i.e. Glisson et al., 2010; Henggeler et al, 2006a). One study (i.e. Mitchell-Hertzfeld et al., 2007) reported about research in which assignment
methods differed over two inclusion periods. This study was therefore included as two separate samples (Pilot and Post-Pilot respectively).
Furthermore, with less than half of the included samples (k = 8), more than one article reported about the same sample. For these studies the report with the longest follow-up period, closest to four years for ultimate outcomes (i.e., delinquency and out-of-home placements), and closest to two years for other behavioural and psychosocial outcomes was used as primary outcome study. When necessary, the information from the primary outcome study was supplemented with information from the other – secondary – reports.
Consequently, the final sample of studies consisted of 22 independent samples from 51 reports.
Coding the studies
The samples included in the meta-analysis were coded for publication characteristics, research design characteristics, treatment characteristics, sample characteristics, and ultimate and other outcomes.
Coded publication characteristics were publication year, authors, secondary reports, author (in)dependence (i.e. associated with the developers of MST), whether or not the study was published in a peer reviewed journal and – if applicable – in which journal. Research design characteristics included the country where the research was conducted, the data collection period, the study design: randomized controlled trial or quasi-experimental and effectiveness or efficacy, in- and exclusion criteria and reported measures (official data, self-, parent-, teacher-report and/or observational measures). Study quality was assessed using the Quality Assessment Tool for Quantitative Studies (Thomas, Ciliska, Dobbins, & Micucci, 2004), which classifies study design based on selection bias, study design, confounders, blinding, data collection method and withdrawals an dropouts as weak, moderate or strong. For study quality, both the categorical variable (i.e. weak, moderate or strong) as well as the continuous variable (sum of components) was used.
Treatment characteristics were coded for both MST and the control group. For MST the proportion of treatment completers, treatment duration (in weeks and hours), therapist characteristics, including education, proportion of males and immigrants and treatment
adherence, were coded. Coded characteristics for the control group were name and description of the control condition, if it was treatment as usual, considered evidence-based treatment and if it consisted of different treatments (mixed). Hours of treatment and proportion of treatment completers were coded as well.
Sample characteristics were the target population as a string and as a categorical variable, number of juveniles assigned to the different conditions and number of juveniles used in analyses, proportion of males, proportion living with biological parents, proportion living with single parents, proportion immigrants, proportion with low social economic status and proportion of juveniles who had been previously arrested, and the number of previous arrests. The existence of and description of baseline difference between the MST and control group were also coded, together with a moderator indicating whether outcome measures had been corrected for these differences.
For the ultimate outcomes, the average follow-up period was coded in months. For one study, the follow-up period was an outlier (> 3.2 SD; Tabachnick & Fidell, 2003). Ultimate outcomes were measures of any offense, any violent offense, any non-violent offense and number of each of those offenses. Regarding out-of-home placement it was coded if the juvenile was placed out of home during follow-up and the number of days in out-of-home treatment. For the remaining outcome measures the average follow-up period and number of juveniles were coded again, because these often differed from the ultimate outcome follow-up. These outcomes were substance use, internalizing and externalizing behaviour,
aggression, delinquency, psychopathy, psychopathology, family functioning, positive parenting, positive peers, school functioning and skills and cognitions.
Calculation and analysis of effect sizes
For the effect size, the standardized mean difference, or Cohen’s d, was calculated based on means and standard deviations (SD), or – if means and SD’s were not available – F-test statistics using the Practical Meta-Analysis Effect Size Calculator (Wilson, 2010). With two studies (i.e. Henggeler et al., 1986; Stambaugh et al., 2007) Cohen’s d was estimated using the standardized difference between delinquents and non-delinquents using Mullen’s (1989) advanced basic meta-analysis program.
With dichotomous variables – when regular standardized mean differences could not be calculated – the effect size was calculated using the arcsine method. According to Lipsey and Wilson (2001) this is the preferred method to estimate the standardized mean difference, although it is a conservative underestimate of the actual standardized mean difference. The alternative probit and logit methods are not recommended, because those have the undesirable potential to overestimate the effect size extremely with non-normal underlying distributions.
If multiple follow-up periods were available, outcomes closest to a four year follow-up were used for ultimate outcomes and outcomes closest to a two year follow-up were used for
the other outcomes. Furthermore, if more than one outcome measure was reported for the same outcome variable – for instance: juvenile and parental reports on family functioning – the effect sizes for the different measures were averaged into a pooled effect size for the regarding outcome variable.
Analyses were conducted if an outcome variable consisted of over five studies. This was the case for all outcome measures, but not for non-violent offenses, substance use, aggression, psychopathy, school functioning and skills and cognitions. Moreover, any violent offense and number of violent offenses yielded insufficient studies to conduct a meta-analysis. Therefore a composite outcome variable was made consisting of an average of both variables, along with a variable about the composition of the effect size (i.e. any, number or composite) for the purpose of moderator analysis. The outcome variable for out-of-home placement and incarceration was calculated similarly, with a moderator variable consisting of four
categories: any or number of days in out-of-home placement, composite of out-of-home placement and incarceration (if incarceration was reported, no out-of-home placement was reported and vice versa).
Effect size and moderator analyses were conducted using the SPSS macros of Lipsey and Wilson (2001) based on the random effects model. The random effects model, in contrast to the fixed effects model, allows for generalisation of results to the population from which all studies were drawn, thus yielding maximum external validity (Lipsey & Wilson, 2001).
Homogeneity was tested with Q-within-statistics. If the Q-within-statistic is
significant, a heterogeneous distribution is assumed, and variance in the effect size therefore could be explained by other variables. Moderator analyses were conducted only for outcome variables with a heterogeneous distribution.
Categorical moderators were year of the study (before 2004 or after 2004),
independence of authors (yes or no), published in peer reviewed journal (yes or no), country (United States of America versus other), target population (conduct problems, offenders, or sex-offenders), randomized (yes or no), mixed control treatment (yes or no), age (under 15 or over 15 years of age), efficacy versus effectiveness, baseline differences between MST and controls (yes or no) and study quality (weak, moderate,or strong). For violent offenses and out-of-home placement the composition of the variable (any, number, or composite) was entered as a moderator as well. Continuous moderators were study year, proportion of males, proportion living with single parents, proportion of migrants, proportion previously arrested, number of previous arrests, hours of treatment and proportion of treatment completers for both MST and controls, duration of MST (in days), follow-up period and study quality.
After moderator analyses, the significant moderators for each outcome variable were entered into a multiple regression analysis to assess the unique influence of each moderator. Because the impression was that some moderators were highly correlated, first of all
correlation analysis of all significant moderators for all studies was conducted. Next,
multicollinearity was assessed. Highly correlating (i.e. Pearsons Correlation > .8) and highly multicollinear moderators (i.e. Variance Inflation Factor > 10) were excluded from the regression analysis.
File drawer analysis
It is commonly known that studies with no significant or negative results are less likely to be published than studies with positive and significant results. This phenomenon, referred to as the ‘file drawer problem’ (Rosenthal, 1995), was also highlighted by Littell (2005) with the previous meta-analysis. Although part of this problem should be resolved by the inclusion of non-published studies in the current meta-analytic study, file drawer bias was examined using the fail-safe number. The fail-safe number estimates the number of
unpublished studies, presumably reporting null results, needed to reverse the outcomes to non-significance (Lipsey & Wilson, 2001). There should be no file drawer bias if the fail-safe number is larger than 5*k+10.
Table 1 Study characteristics of the studies included in the meta-analysis
Follow-up (months)
Study Year N Age Man
(%)
Migrants (%)
Target population Random (yes/no) Independent (yes/no) Study quality Ultimate Other
Asscher 2012 256 16.0 .73 .45 Conduct problems Yes Yes Strong 24 0
Barnoski 2004 145 15.2 .77 Offenders Yes Yes Weak 18
Borduin 2009 48 14 .96 .29 Sex offenders Yes No Moderate 107 0
Borduin 1990 16 14.0 1.00 .37 Sex offenders Yes No Weak 37
Borduin 1995 176 14.8 .68 .30 Offenders Yes No Strong 47 0
Butler 2011 108 15.1 .82 .64 Offenders Yes Yes Strong 12 0
Cunningham 2002 409 14.7 .74 Offenders Yes No Moderate 36 0
Henggeler 2002b 118 15.7 .79 .53 Offenders Yes No Strong 48 6
Henggeler 1997 155 15.2 .82 .81 Offenders Yes No Strong 20 0
Henggeler 1993 84 15.2 .77 .58 Offenders Yes No Strong 23 0
Henggeler 1986 156 14.8 .84 Offenders No No Moderate 0
Letourneau 2009 131 14.6 .98 .85 Sex offenders Yes No Strong 6 6
Lofholm 2009 156 15.0 .61 .47 Conduct problems Yes Yes Strong 18 18
Mayfield 2011 252 14 .60 .17 Conduct problems No Yes Strong 6
Mitchell-Hertzfeld 2008 Pilot
269 16.1 .73 .94 Offenders No Yes Weak 30
Mitchell-Hertzfeld 2008 Post-Pilot
629 16.3 .80 .96 Offenders No Yes Weak 18
Ogden 2006 75 15.1 .64 .01 Conduct problems Yes Yes Strong 18 18
Painter 2009 174 11.9 .47 .46 Conduct problems No Yes Moderate 0 0
Rowland 2005 55 14.5 .58 .90 Conduct problems Yes No Strong 0 0
Stambaugh 2007 265 12.1 .74 Conduct problems No Yes Strong 12
Timmons-Mitchell 2006 93 15.1 .78 .22 Offenders Yes Yes Strong 18 18
Timmons-Mitchell 2004 296 .64 .57 Offenders No Yes Weak 6
Results
The current meta-analysis consists of k = 22 studies, reporting on 4066 juveniles of whom N = 1890 received MST treatment and N = 1835 constituted the control group. Table 1 shows a survey of the included studies and their characteristics. Because not all studies reported an intent-to-treat analysis (i.e., including drop-outs in their analyses) the reported number of participants included in the analyses was used to calculate effect sizes.
Cohen’s (1988) guideline was used to interpret effect sizes, with effect sizes of d = .20 considered small, d = .50 considered medium and d = .80 considered large. The overall distribution of effect sizes is presented in Table 2.
Table 2 Meta-analytical results of MST Outcome variable N Number of subjects K Number of studies Effect size d Random effects 95% Confidence interval Q statistic within studies Fail-Safe N Any offense 2938 15 .296*** .117-.474 309.075*** 43.518 Number of offenses 1723 8 .393* .018-.768 294.951*** 4.487 Violent offenses 980 6 .174* -.012-.359 41.073*** 1.473 Out-of-home placement 1326 9 .321*** -.160-.481 61.084*** 16.334 Internalising 670 6 .239** .058-.421 8.834*** 25.565 Externalising 1098 9 .413** .105-.720 199.578*** 13.969 Delinquency 1116 10 .256*** .113-.400 48.271*** 35.328 Psychopathology 945 7 .225** .064-.387 34.869*** 12.268 Family Functioning 1304 11 .340* .061-.619 246.702*** 12.141 Positive Parenting 930 6 .205*** .130-.280 6.556 57.121+ Positive Peers 1028 8 .167 -.035-.369 71.243*** -.223
* p < .05, ** p < .01, *** p < .001, + Fail-Safe > critical, i.e. no file drawer bias.
As can be derived from Table 2, all outcome variables except Positive Peers yielded small but significant effects. For Positive Parenting the Q statistic within studies was not significant (Q = 6.556, p = NS), indicating a set of homogenous effect sizes. With all other outcome variables, variance in effect sizes could therefore be moderated by other variables. Additionally, only Positive parenting yielded a Fail-Safe number large enough to rule out file drawer bias, which means the other effect sizes may be affected by a file drawer problem.
Table 3 Categorical Moderator Analyses for ultimate outcomes : Cohen’s d, 95% CI, Q Statistics N K d 95% Confidence interval Q between studies Q within studies Any offense Year 3.922* Before 2004 436 4 .595*** .215-.938 12.124** 2004 and later 2502 11 .195* -.001-.391 13.367 Independent authors 8.768** Yes 2205 10 .116 -.097-0.329 7.419 No 733 5 .694*** .376-1.012 10.374* Published 15.603*** Yes 1074 9 .533*** .343-.724 21.234** No 1864 6 -.052 -.273-.167 1.186 Target population 12.578** Conduct problems 810 4 .113 -.214-.440 1.497 Offenders 2064 9 .223* .004-.443 12.445 Sex-offenders 64 2 1.205*** .667-1.743 1.234 Control treatment is mixed 7.801**
Yes 2364 10 .135 -.061-.331 8.441 No 574 5 .639*** .345-.932 14.405**
Age 4.719*
Under 15 1075 6 .580*** .288-.873 12.238* 15 years and over 1703 8 .158 -.087-.402 6.555 Efficacy/Effectiveness 30.495***
Efficacy 240 3 1.068*** .759-1.376 2.501 Effectiveness 2698 12 .122 -.011-.254 17.604
Number of offenses
Control treatment is mixed 5.399*
Yes 969 5 .164 -.153-.481 1.617 No 304 3 .786*** .368-1.205 5.828 Efficacy/Effectiveness 24.390*** Efficacy 224 2 1.119*** .786-1.452 4.169* Effectiveness 1049 6 .162 -.022-.345 3.986 Violent offenses Year 11.659*** Before 2004 256 2 .443*** .254-.631 .914 2004 and later 724 4 .050 -.073-.174 3.522 Independent authors 11.659*** Yes 724 4 .050 -.073-.174 3.522 No 256 2 .443*** .254-.631 .914 Control treatment is mixed 11.659***
Yes 724 4 .050 -.073-.174 3.522 No 256 2 .443*** .254-.631 .914 Efficacy/Effectiveness 6.883** Efficacy 176 1 .520*** -.232-.808 .000 Effectiveness 804 5 .095 -.038-.228 4.902 Out-of-home placement Published 13.266*** Yes 665 7 .402*** .283-.520 10.440 No 661 2 .005 -.172-.183 .003 * p < .05, ** p < .01, *** p < .001. 14
Table 4 Categorical Moderator Analyses for other outcomes : Cohen’s d, 95% CI, Q Statistics N K d 95% Confidence interval Q between studies Q within studies
Internalising No significant moderators
Externalising
Year 20.099***
Before 2004 126 1 1.430*** .956-1.904 .000 2004 and later 972 8 .276** .103-.449 10.452 Control treatment is mixed 9.512**
Yes 810 6 .165 -.108-.469 1.503 No 288 3 .913*** .524-1.301 5.172 Efficacy/Effectiveness 38.994*** Efficacy 172 2 1.234*** .944-1.525 2.448 Effectiveness 926 7 .197** .049-.345 5.123 Delinquency Efficacy/Effectiveness 11.020*** Efficacy 46 1 .930*** .516-1.344 .000 Effectiveness 1070 9 .201*** .085-.318 10.646 Study quality 11.020*** Weak Moderate 46 1 .930*** .516-1.344 .000 Strong 1070 9 .201*** .085-.318 10.646 Psychopathology Target population 12.307** Conduct problems 595 3 .126 -.035-.288 1.151 Offenders 304 3 .171 -.010-.352 3.472 Sex-offenders 46 1 .870*** .487-1.253 .000 Control treatment is mixed 4.729*
Yes 725 5 .113 -.053-.289 2.441 No 220 2 .502*** .209-.795 4.145 Efficacy/Effectiveness 5.192* Efficacy 126 2 .519*** .226-.813 4.358* Effectiveness 819 5 .129 -.033-.292 2.872 Study quality 4.134* Weak Moderate 300 3 .414*** .176-.652 5.235 Strong 645 4 .097 -.095-.289 2.101
Family Functioning No significant moderators
* p < .05, ** p < .01, *** p < .001.
Table 5 Continuous moderator analyses: Standardized regression coefficients (Beta) and Z-values N K Beta Z Any offense Year 2938 15 -.442* -2.349 Proportion of males 2938 15 .526** 2.593 Migrants 2384 13 -.425* -1.981 Follow-up period 2938 15 .479* 2.446 Number of offenses
Proportion previously arrested 813 3 .998*** 14.427
Violent offenses Follow-up period 980 6 .743* 2.223 Out-of-home placement Proportion MST completers 835 6 .733* 2.051 Internalising Year 670 6 -.669* -2.142
Proportion living with single parents 554 5 -.998*** -5.043
Externalising
Year 1098 9 -.738*** -3.482 Proportion living with single parents 982 8 -.661* -2.080 Proportion previously arrested 726 4 .966*** 5.595 Duration of MST (days) 753 6 .812*** 3.288 Delinquency Proportion of males 1116 10 .583* 2.364 Study quality 1116 10 -.718*** -3.243 Psychopathology Proportion MST completers 282 3 .978*** 4.664 Proportion control treatment completers 282 3 .921* 2.363 Hours of treatment MST 384 4 .854* 2.535 Hours of treatment controls 300 3 .994*** 3.750 Duration of MST (days) 865 6 .733* 2.275
Family functioning
Proportion of migrants 1036 9 -.633* -2.317 Number of previous arrests 960 9 .595* 2.371
* p < .05, ** p < .01, *** p < .001.
Moderator analysis
Because sets of effect sizes proved to be heterogeneous, moderator analyses were conducted for each of the outcome variables with the exception of Positive Parenting.
A moderator is considered to have a significant moderating effect if the Q statistic between groups (categorical moderator) or the standardized regression coefficient (Beta) (continuous moderator) is significant. Table 3, 4 and 5 present the significant moderators for each outcome variable with their corresponding statistics. Overall, no moderating effects were found for country, randomized, baseline differences between MST and controls and the composite variables for violent offenses and out-of-home placement. As Tables 3, 4 and 5 show, no moderator had a significant moderating effect on all outcome variables.
Study design characteristics. Efficacy versus effectiveness had a moderating effect
on most outcome variables: efficacy studies reported larger effects for any offense, the number of offenses, violent offenses, externalising behaviour, self-reported delinquency and psychopathy. Furthermore, lesser quality studies reported larger effects on delinquency and psychopathy.
Publication characteristics. Larger effects on general recidivism, violent offenses,
externalising behaviour and internalising behaviour were reported for earlier studies. Furthermore, studies conducted by dependent researchers reported larger effects on any offense and violent offenses. Published studies reported larger effects on out-of-home placements and any offense.
Treatment characteristics. With regard to any offense, the number of offenses,
violent offenses, externalising behaviour and psychopathy, MST juveniles only did better than the control group if the control group received one singular type of treatment, as opposed to a mixed composite of control treatment. Other treatment characteristics yielded mostly only different effects for psychopathology: juveniles with MST showed less psychopathological symptoms when there were more MST and control treatment completers, with more hours of MST and control treatment, and when they received MST for a longer period of time.
Furthermore, more MST completion led to less out-of-home placements and a longer period of MST treatment yielded less externalising behaviour.
Sample characteristics. General recidivism (any offense) was moderated by sample
characteristics. Less recidivism was reported with all offenders, but not with non-offenders, younger juveniles, males, and non-ethnic minority youth. Improvements in family functioning were found less with ethnic minority youth as well. Furthermore, larger effects on recidivism in general and on violent offenses were reported for studies with a longer follow-up period, and self-reported delinquency showed effects only for males. Less psychopathology was only found for sex-offenders, offenders in general, whereas conduct troubled youth showed no improvements in psychopathology. Finally, internalising and externalising behaviour
improved less if juveniles were living in a single-parent household as opposed to a two-parent household.
Correlations among moderators
To be able to include the categorical variables target population and study quality in a multiple regression analysis, two dummy variables were created for each variable. These were conduct problem juveniles versus the other (coded resp. 1 and 0) and sex-offenders versus the
other (coded resp. 1 and 0) for target population. Study quality was recoded into strong design versus the other and weak design versus the other (coded resp. 1 and 0).
Before the moderators were included in a multiple regression analysis, first,
correlations between all moderators were examined. Obviously, a decision needed to be made with regard to categorical and continuous variables measuring the same construct. First, year of the study as a categorical moderator was excluded, because the continuous variable was considered more informative. Furthermore, the categorical variables pertaining to study quality were excluded, because the continuous study quality variable retained more information about study quality variance.
Hours of treatment for both MST and control treatment were only reported in published studies. These variables were excluded from the multiple regression analysis, because they could represent the quality of reporting in published versus non-published studies instead of actually representing the target constructs. The same decision was made for the proportion of juveniles with previous arrests, because this variable was not reported for sex-offenders, thus not able to discriminate between the two kinds of offenders. Finally, the proportion of treatment completers in the control group was highly correlated with MST duration (in days) (r = .802, p = .030, k = 7). Because the duration in days was reported for more studies and more relevant to the effectiveness of MST, it was decided to exclude the proportion of control group treatment completers from further analysis.
With the remaining moderators, multicollinearity between moderators was tested for each of the outcome variables. For moderators with a Variance Inflation Factor (VIF) larger than 10, one of the two collinear variables was excluded from analyses. Based on this criterion, no moderators were excluded for the following outcome variables: number of offenses, out-of-home placement, internalising behaviour and family functioning. Study design and publication characteristics in general proved to be highly multicollinear, and were therefore mostly excluded.
For any offense, dependency/independency of the researchers, publication status (published or not), efficacy versus effectiveness, control treatment is mixed, year of the study and proportion of migrants were highly multicollinear, and therefore excluded from the regression equation. For violent offenses, dependency/independency of the researchers, control treatment is mixed and efficacy versus effectiveness were excluded. For externalising behavior, efficacy versus effectiveness, control treatment is mixed and year of the study were excluded. Finally, for delinquency, efficacy versus effectiveness and study quality were
excluded, and for psychopathology, proportion MST completers, efficacy versus effectiveness, and control treatment is mixed were excluded.
Multiple regression analyses
With the remaining moderators per outcome variable, regression analyses were
conducted to assess the unique contribution of each moderator to the overall effect size (Table 6). These analyses yielded significant regression equations for all outcome variables but any offense. No significant predictors emerged for out-of-home placement and psychopathology.
Table 6 Multiple regression analysis Outcome variables
Moderator variables
Beta Z
Any offense, Q (5, 13) = 10.119, p = .072
Sex-offenders versus others .360 .634 Conduct problems versus others -.131 -.307
Age categories -.364 -.951
Proportion of males .258 .374
Follow-up period -.083 .805
Number of offenses, Q (2, 7) = 21.821, p = .000**
Control treatment is mixed .021 .076
Efficacy versus effectiveness -.848** -3.025
Violent offenses, Q (1, 5) = 4.944, p = .026* Follow-up period .743* 2.223 Out-of-home placement, Q (2, 5) = 9.264, p = .009** Published study -.489 -1.619 Proportion MST completers .549 1.819 Internalising, Q (2, 4) = 25.459, p = .000*** Year .059 .164
Proportion living with single parents -1.048** -2.878
Externalising, Q (2, 4) = 32.148, p = .000***
Duration MST in days .923* 2.381
Proportion living with single parents -.053 -.137
Delinquency, Q (2, 7) = 8.471, p = .014*
Proportion males .620* 2.132
Proportion living with single parents -.245 -.842
Psychopathology, Q (3, 5) = 11.403, p = .009***
Sex-offenders versus others -.171 -.173 Conduct problems versus others -.732 -1.025
Duration MST in days 1.038 1.106
Family functioning, Q (2, 7) = 17.468, p = .000***
Proportion of migrants -.626** -2.997
Number of previous arrests .547** 2.616 * p < .05, ** p < .01, *** p < .001.
For number of offenses, significantly larger effects were reported for efficacy than for effectiveness studies. For violent offenses follow-up duration was a significant moderator. The longer the follow-up, the larger the difference in violent offenses between juveniles receiving MST treatment and the control group. The proportion of juveniles living with single parents was a significant moderator for internalising behavior. If the treatment group
comprised of relatively few single parent households, the effects on internalising behavior were larger. Furthermore, only duration of MST was a significant predictor for externalising behavior: MST treatment yielded larger effects when the treatment period was longer. Effects for self-reported delinquency were significantly larger for studies with a higher proportion of males. Finally, family functioning yielded larger effects with samples consisting of less migrants and with juveniles with a higher number of previous arrests.
Discussion
The purpose of the present meta-analysis was to examine the effectiveness of MST and replicate the previous meta-analyses by Curtis et al. (2004) and Littell et al. (2005) with a larger body of research studies. This larger number of studies made it possible to examine moderators explaining the effectiveness of MST.
Small but significant treatment effects were found for recidivism in general, the number of offenses, violent offenses, out-of-home placement, internalising and externalising behavior, self-reported delinquency, psychopathology, family functioning and positive parenting. Only positive parenting proved to be homogenous, and had a fail-safe number that ruled out file-drawer bias. The positive effect for positive parenting is consistent with the fact that MST aims to improve juvenile functioning by improving parenting skills (e.g., Borduin, 1999; Henggeler, 2011). In contrast no significant effects were found for peer relationships.
As all combined effect sizes, except positive parenting, proved to be heterogeneous, moderator analyses were conducted. Efficacy studies reported larger effect sizes for almost all outcome variables than effectiveness studies. Furthermore, larger effect sizes were reported in earlier studies, published studies and studies conducted by dependent researchers for most of the ultimate outcomes (i.e., crime and delinquency and out-of-home placements). Recidivism in general (any offense) was moderated the most by sample characteristics. Effect sizes for offending were largest when juveniles were younger, male, Caucasian white and received MST for offending (not for conduct problems). Furthermore, effects were different for different comparison treatments. MST only yielded larger effects on offending when it was
compared to a singular type of treatment. No effects were shown if the control treatment was a mix of different kinds of treatment. As Andrews, Bonta, and Wormith (2006) already emphasised the importance of multimodal treatment approaches, it is well possible that the effects of MST can be attributed to the multimodal approach of the intervention, because no effects were found when compared to other multimodal interventions. Seven moderators uniquely explained the effectiveness of MST. For general recidivism, out-of-home placement and psychopathology, no significant moderators were found.
With regard to study characteristics, only efficacy studies still yielded larger effects on the number offenses than effectiveness studies did, which indicates that MST has larger effects on number of offenses under perfect treatment conditions. This finding differs from that of Shadisch et al. (2000), who in a meta-analysis found that the differences between efficacy and effectiveness were an artefact of selective assignment in non-randomized trials. Furthermore, of all treatment characteristics, only if MST was applied for a longer period of time, larger effects were found for externalising behavior. This finding is difficult to interpret, since Lipsey (2009) found no overall effect for treatment duration in his meta-analysis of effective interventions with juvenile delinquents. It is possible that longer treatment duration is related to the severity of pre-treatment problems, and therefore a higher pre-treatment recidivism risk. Longer, more intensive treatment would then fit the RNR-model by alignment with the recidivism risk, thereby enhancing effectiveness (i.e., the higher the risk, the more intensive treatment needs to be; e.g. Andrews & Bonta, 2010; Andrews, Bonta & Hoge, 1990; Andrews, Bonta, Wormith, 2006).
Finally, sample characteristics yielded the most unique moderating effects in the regression equations. First, for violent offenses, larger effects were found with a longer follow-up period, indicating that MST treatment effects are persistent and more distinctive over a longer period of time. Second, larger effects were found with two-parent households in contrast to single-parent households. This could be related to the fact that juveniles from single-parent household are in general more sensitive to behavioural problems (Amato, 2001; Amato & Keith, 1991). Third, for self-reported delinquency larger effects were reported if a larger proportion of the sample was male. This is consistent with the fact that recidivism in general is predicted by gender (Cottle, Lee, & Heilbrun, 2001) so that larger effect sizes could be more easily achieved with a male overrepresentation. That is, if girls in general do not reoffend (as much), treatment effects on delinquency could only be achieved with the male proportion of the sample. If this proportion is smaller, treatment effects for males would have
to be much larger to yield an effect for the overall sample. Another possible explanation might be that MST insufficiently addresses gender differences in treatment.
Fourth, the effectiveness of MST in terms of family functioning was larger in non-migrant families. The first is especially interesting given that MST studies that tested for moderating effects of ethnicity reported no moderating effects (Borduin et al., 1995; Henggeler et al., 2002b; Henggeler, Melton & Smith, 1992; Schaeffer & Borduin, 2005), which is probably due to smaller sample sizes in individual studies (lack of statistical power). Furthermore, these findings are in contrast to those by Wilson, Lipsey and Soydan (2003), who found no
differences in treatment effects between Caucasian white and minority juvenile delinquents in spite of the lack of cultural tailoring of treatments. They did state, however, that more
research was needed to specifically assess the treatment outcome differences between
minority and Caucasian white youth. The findings in the current study endorse that statement once more.
Furthermore, family functioning showed larger effects if juveniles had a higher number of previous arrests. The larger effects with a higher number of previous arrests could be an indication that the juveniles showed more severe delinquency. Family functioning thus improved more with more severe cases. These findings are in line with those by Lipsey (2009), who found larger treatment effects with juveniles with higher levels of delinquency risk for a variety of juvenile delinquent treatments.
As the primary goal of MST is to reduce recidivism (any offense) with juvenile offenders, it can be concluded that MST works best with offenders. Non-offenders, even if they have conduct problems, do not benefit from the treatment as much as offenders do. Furthermore, long-term effects of MST seem plausible, because the effects on recidivism have been shown to be larger with longer follow-up durations. Moreover, MST seems to only have added value compared to singular treatment alternatives. That is: if the alternative treatment consist of a variety of treatments, presumably fit to the different juveniles’ needs, MST juveniles do not offend particularly less. Finally, MST proved to be only effective for juveniles under age fifteen, and was the most effective with males. MST proved to be less effective with ethnic minority youth.
The findings of the current meta-analysis differ from the previous meta-analyses by Curtis et al. (2004) and Littell et al. (2005). The effect sizes in the present study were smaller than those reported in the Curtis et al. study (i.e., small versus moderate) and larger than those reported by Littell et al. (i.e., no effect versus small). This is probably caused by the fact that both non-published, non-randomised and more recent studies were added to the analyses in
the present study, and child abuse and neglect and psychiatric hospitalization studies were excluded. Larger average effect sizes for family relations over individual adjustment (Curtis et al., 2004) were not replicated, whereas the effect on peer relationships diminished altogether.
Contrary to previous meta-analyses (Curtis et al., 2004; Littell et al., 2005), no moderating effects were found for study design with respect to randomization. Less strictly assigned treatment and control groups did not yield larger effects, thereby validating the use of non-randomised studies in the present meta-analysis and confirming its generalizability (see Shadish, Matt, Navarro, & Phillips, 2000). Furthermore, no moderating effects were found for country where the study had been conducted. While MST is applied in 14 countries worldwide, the effectiveness of MST when transported to other countries than the United States of America was assessed in only a few countries. Overall, it now seems that this transportation did not have different effects on the outcome variables. Moreover, given that Van der Put et al. (2011) found that the prevalence of, and association between risk factors and recidivism are much the same across Europe and the United States of America, no other treatment outcomes were to be expected as MST aims at those risk factors.
Additionally, moderators were highly confounded. The moderators (in)dependence of authors, efficacy versus effectiveness, control treatment is mixed and year of publication were multicollinear for several outcome variable. Furthermore, although Littell (2006) implicated that some large effects could be due to conflict of interest of the involved researchers (i.e. researchers having personal or financial stakes in MST), it seems that the (in)dependence of researchers is linked to other study characteristics. MST developers examined more offenders than juveniles with conduct problems, examined younger and more male juveniles, were more often involved with efficacy studies, were more often involved with earlier studies and
reported about longer follow-up periods than independent researchers (see Table 1). Larger effects can thus not simply be attributed to the dependency of researcher. Still, findings of the current meta-analysis were consistent with those of Petrosino and Soydan (2005) in that dependent researchers reported larger effect sizes than independent researchers.
Unfortunately, treatment adherence could not be included in the moderator analyses. Precisely this meta-analysis would be an opportunity to assess the assumption that treatment effects are highly dependent of treatment adherence (e.g. Henggeler et al., 2006b). Although treatment adherence measures seem to be applied more often in the more recent studies, the way this adherence is reported is overall inconsistent, and adherence outcomes could not be coded for a sufficient amount of studies. Some studies reported an actual treatment adherence outcome (without referential information), other studies reported treatment adherence
categorically – i.e., low, moderate or high treatment adherence – if adherence had a moderating effect on the outcome variables (without outcome or category of treatment adherence), if adherence measures were administered, or if a measure of treatment adherence was used or not.
There are some limitations of this meta-analysis that need to be mentioned. First, although an attempt was made to include as much studies as possible by including both randomized and non-randomized studies, and both published and none-published studies, a possible file-drawer bias may exist for all outcome variables, except for positive parenting. Thus, the present outcomes could be an overrepresentation of the actual effect sizes. Furthermore, because of the inclusion of unpublished and non-randomised studies, some studies were of weak study design and therefore had questionable validity. Moreover, several moderators, mostly study design and publication characteristics, were excluded from multiple the regression analyses, because those were the most correlated with other moderators. The outcomes from the regression analyses should therefore be interpreted with some caution.
In spite of these limitations, this meta-analytic study provides the most up-to-date information on Multisystemic Therapy, which has been spread over the world since the previous meta-analyses, indicating that MST has a small but significant effect. MST shows a small, significant effect for general recidivism, the number of re-offenses, violent offenses, out-of-home placement, internalising and externalising behaviour, self-reported delinquency, psychopathology, positive parenting and family functioning.
In line with findings of Andrews, Bonta, and Wormith (2006), especially the
multimodal approach of MST appears effective. Although MST appears to achieve its main treatment goals, i.e. lowering recidivism rates and out-of-home placement and enhancing parenting skills and family functioning, no effects were found for peer relationships. A recent study by Van der Put, Deković, Stams, Hoeve, and Van der Laan (2012), however, shows that family risk factors are predictive of recidivism only for children up to age 12 years. With older juveniles, family factors lose their predictive value for recidivism and peer and school factors take over. The effectiveness of could MST could therefore presumably be improved by targeting school and peer related factors more with older juveniles.
References
References marked with one or two asterisks indicate primary studies or secondary analysis studies, respectively, in the meta-analysis.
Amato, P. R. (2001). Children of divorce in the 1990s: An update of the amato and keith (1991) meta-analysis. Journal of Family Psychology, 15(3), 355-370.
Amato, P. R., & Keith, B. (1991). Parental divorce and the well-being of children: A meta-analysis. Psychological Bulletin, 110(1), 26-46.
Andrews, D., & Bonta, J. (2010). Rehabilitating criminal justice policy and practice.
Psychology, Public Policy, and Law, 16, 39-55. doi: 10.1037/a0018362
Andrews, D. A., Bonta, J., & Hoge, R. D. (1990). Classification for effective rehabilitation: Rediscovering psychology. Criminal Justice and Behavior, 17(1), 19-52. doi: 10.1177/0093854890017001004
Andrews, D. A., Bonta, J., & Wormith, J. S. (2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52(1), 7-27. doi:
10.1177/0011128705281756
** Asscher, J. J., Deković, M., & Manders, W. A. (2012). MST onderzoek: Onderzoeksvraag:
Leidt MST tot meer afname in recidive bij adolescenten die antisociaal gedrag tonen dan treatment as usual (TAU)? Unpublished manuscript.
** Asscher, J. J., Deković, M., Van der Laan, Peter H., Prins, P. J. M., & Van Arum, S. (2007). Implementing randomized experiments in criminal justice settings: An evaluation of multi-systemic therapy in the Netherlands. Journal of Experimental
Criminology, (3), 113-129. doi: 10.1007/s11292-007-9028-y
* Asscher, J. J., Deković, M., Manders, W. A., Van der Laan, Peter H., Prins, P. J. M., & Dutch MST Cost-Effectiveness Study Group4. (2012). A randomized controlled trial of the effectiveness of multisystemic therapy in The Netherlands: Post-treatment changes and moderator effects. Journal of Experimental Criminology, doi: 10.1007/s11292-012-9165-9
* Barnoski, R. P., & Aos, S. (2004). Outcome evaluation of washington state's
research-based programs for juvenile offenders. Olympia, WA: Washington State Institute for
Public Policy.
Borduin, C. M. (1999). Multisystemic treatment of criminality and violence in adolescents.
Journal of the American Academy of Child & Adolescent Psychiatry, 38(3), 242-249.
* Borduin, C. M., Henggeler, S. W., Blaske, D. M., & Stein, R. J. (1990). Multisystemic treatment of adolescent sexual offenders. International Journal of Offender Therapy
and Comparative Criminology, 996(3), 105-113.
* Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M., & Williams, R. A. (1995). Multisystemic treatment of serious juvenile offenders: Long-term prevention of criminality and violence. Journal of Consulting & Clinical
Psychology, 63(4), 569-578.
* Borduin, C. M., Schaeffer, C. M., & Heiblum, N. (2009). A randomized clinical trial of multisystemic therapy with juvenile sexual offenders: Effects on youth social ecology and criminal activity. Journal of Consulting & Clinical Psychology, 77(1), 26-37. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and
design. Cambridge, MA: Harvard University Press.
** Brown, T. L., Henggeler, S. W., Schoenwald, S. K., Brondino, M. J., & Pickrel, S. G. (1999). Multisystemic treatment of substance abusing and dependent juvenile delinquents: Effects on school attendance at posttreatment and 6-month follow-up.
Children's Services: Social Policy, Research & Practice, 2(2), 81-93.
Brunk, M., Henggeler, S. W., & Whelan, J. P. (1987). Comparison of multisystemic therapy and parent training in the brief treatment of child abuse and neglect. Journal of
Consulting & Clinical Psychology, 55(2), 171-178.
* Butler, S., Baruch, G., Hickey, N., & Fonagy, P. (2011). A randomized controlled trial of multisystemic therapy and a statutory therapeutic intervention for young offenders.
Journal of the American Academy of Child & Adolescent Psychiatry, 50(12),
1220-1235.e2. doi: 10.1016/j.jaac.2011.09.017
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
Cottle, C. C., Lee, R. J., & Heilbrun, K. (2001). The prediction of criminal recidivism in juveniles: A meta-analysis. Criminal Justice and Behavior, 28(3), 367-394. doi: 10.1177/0093854801028003005
** Cunningham, A., & Centre for Children and Families in the Justice System. (2002).
Lessons learned from a randomized study of multisystemic therapy in Canada.
London, ON: Center for Children and Families in the Justice System.
* Cunningham, A., & Centre for Children and Families in the Justice System. (2002).
Randomized study of MST in Ontario Canada: Final results. London, ON: Center for
Children and Families in the Justice System.
** Cunningham, P. B., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (1999). Testing underlying assumptions of the family empowerment perspective. Journal of Child and
Family Studies, 8(4), 437-449.
Curtis, N. M., Ronan, K. R., & Borduin, C. M. (2004). Multisystemic treatment: A meta-analysis of outcome studies. Journal of Family Psychology, 18(3), 411.
** Deković, M., Asscher, J. J., Manders, W. A., Prins, P. J., & van der Laan, P. (2012). Within-intervention change: Mediators of intervention effects during multisystemic therapy. Journal of Consulting & Clinical Psychology, 80(4), 574-587.
Ellis, D. A., Naar-King, S., Frey, M., Templin, T., Rowland, M., & Greger, N. (2004). Use of multisystemic therapy to improve regimen adherence among adolescents with type 1 diabetes in poor metabolic control: A pilot investigation. Journal of Clinical
Psychology in Medical Settings, 11(4), 315-324.
Faw Stambaugh, L., Mustillo, S. A., Burns, B. J., Stephens, R. L., Baxter, B., Edwards, D., & Dekraai, M. (Fall 2007). Outcomes from wraparound and multisystemic therapy in a center for mental health services system-of-care demonstration site. Journal of
Emotional and Behavioral Disorders, 15(3), 143-155. doi:
10.1177/10634266070150030201
Fidell, L. S., & Tabachnick, B. G. (2003). Preparatory data analysis. Handbook of psychology. Hoboken, NJ: John Wiley & Sons, Inc. doi: 10.1002/0471264385.wei0205
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., . . . Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination.
Prevention Science, 6(3), 151-175.
Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting & Clinical
Psychology, 78(4), 537-550.
Henggeler, S. W. (2011). Efficacy studies to large-scale transport: The development and validation of multisystemic therapy programs. Annual Review of Clinical Psychology,
7, 351-381. doi: 10.1146/annurev-clinpsy-032210-104615
* Henggeler, S. W., Clingempeel, W. G., Brondino, M. J., & Pickrel, S. G. (2002b). Four-year follow-up of multisystemic therapy with substance-abusing and substance-dependent juvenile offenders. Journal of the American Academy of Child & Adolescent
Psychiatry, 41(7), 868-874. doi: 10.1097/00004583-200207000-00021
Henggeler, S. W., Halliday-Boykins, C. A., Cunningham, P. B., Randall, J., Shapiro, S. B., & Chapman, J. E. (2006a). Juvenile drug court: Enhancing outcomes by integrating evidence-based treatments. Journal of Consulting & Clinical Psychology, 74(1), 42-54.
** Henggeler, S. W., Letourneau, E. J., Chapman, J. E., Borduin, C. M., Schewe, P. A., & McCart, M. R. (2009a). Mediators of change for multisystemic therapy with juvenile sexual offenders. Journal of Consulting & Clinical Psychology, 77(3), 451-462. * Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997).
Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting &
Clinical Psychology, 65(5), 821-833.
** Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using multisystemic therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting & Clinical Psychology, 60(6), 953-961.
* Henggeler, S. W., Melton, G. B., Smith, L. A., Schoenwald, S. K., & Hanley, J. H. (1993). Family preservation using multisystemic treatment: Long-term follow-up to a clinical trial with serious juvenile offenders. Journal of Child and Family Studies, 2(4), 283-293.
** Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999a). Multisystemic treatment of substance-abusing and-dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1(3), 171-184.
** Henggeler, S. W., Pickrel, S. G., Brondino, M. J., & Crouch, J. L. (1996). Eliminating (almost) treatment dropout of substance abusing or dependent delinquents through home-based multisystemic therapy. (1996). American Journal of Psychiatry, 153(3), 427-428.
* Henggeler, S. W., Rodick, J. D., Borduin, C. M., Hanson, C. L., Watson, S. M., & Urey, J. R. (1986). Multisystemic treatment of juvenile offenders: Effects on adolescent behavior and family interaction. Developmental Psychology, 22(1), 132.
Henggeler, S. W., Rowland, M. D., Randall, J., Ward, D. M., Pickrel, S. G., Cunningham, P. B., . . . Hand, L. D. (1999b). Home-based multisystemic therapy as an alternative to the hospitalization of youths in psychiatric crisis: Clinical outcomes. Journal of the
American Academy of Child & Adolescent Psychiatry, 38(11), 1331-1339.
Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (2009b). Multisystemic treatment of antisocial behavior in children and adolescents.
treatment manuals for practitioners. (2nd ed.). New York, NY: The Guilford Press.
Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., & Swenson, C. C. (2006b). Methodological critique and meta-analysis as trojan horse. Children and Youth
Services Review, 28(4), 447-457. doi: 10.1016/j.childyouth.2005.07.001
Henggeler, S. W., Schoenwald, S. K., Rowland, M. D., & Cunningham, P. B. (2002a).
Serious emotional disturbance in children and adolescents: Multisystemic therapy.
New York, NY: The Guilford Press.
Hopewell, S., McDonald, S., Clarke, M., & Egger, M. (2007). Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of
Systematic Reviews 2007, (2), Art. No.: MR000010. doi: DOI:
10.1002/14651858.MR000010.pub3
Leschied, A. W., & Cunningham, A. (2001). Clinical trials of multisystemic therapy: 1997 to
2001: Evaluation update report april 2001. London, ON: University of Western
Ontario, Centre for Children & Families in the Justice System.
** Leschied, A., & Cunningham, A. (2002). Seeking effective interventions for serious young
offenders: Interim results of a four-year randomized study of multisystemic therapy in ontario, canada.london (CA): Centre for children and families in the justice system of the london family court clinic. London, ON: Centre forChildren and Families in the
Justice System (London Family Court Clinic Inc.).
* Letourneau, E. J., Henggeler, S. W., Borduin, C. M., Schewe, P. A., McCart, M. R., Chapman, J. E., & Saldana, L. (2009). Multisystemic therapy for juvenile sexual offenders: 1-year results from a randomized effectiveness trial. Journal of Family
Psychology, 23(1), 89-102.
Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims & Offenders, 4(2), 124-147. doi: 10.1080/15564880802612573
Lipsey, M. W., & Wilson, D. B. (2001). In Bickman L., Rog D. J. (Eds.), Practical
meta-analysis. Thousand Oaks, CA: SAGE Publications, Incorporated.
Littell, J. H. (2005). Lessons from a systematic review of effects of multisystemic therapy.
Children and Youth Services Review, 27(4), 445-463. doi:
10.1016/j.childyouth.2004.11.009
Littell, J. H. (2006). The case for multisystemic therapy: Evidence or orthodoxy? Children
and Youth Services Review, 28(4), 458-472. doi: 10.1016/j.childyouth.2005.07.002
Littell J. H., Campbell, M., Green, S., & Toews, B. (2005). Multisystemic therapy for social, emotional, and behavioral problems in youth aged 10-17. Cochrane Database of
Systematic Reviews 2005, (4), Art. No.: CD004797. doi:
10.1002/14651858.CD004797.pub4
Loeber, R., Farrington, D. P., Stouthamer-Loeber, M., & Van Kammen, W. B. (1998).
Antisocial behavior and mental health problems: Explanatory factors in childhood and adolescence. Mahwah, NJ: Lawrence Erlbaum Associates.
* Löfholm, C. A., Olsson, T., Sundell, K., & Hansson, K. (2009). Multisystemic therapy with conduct-disordered young people: Stability of treatment outcomes two years after intake. Evidence & Policy: A Journal of Research, Debate and Practice, 5(4), 373-397.
* Mayfield, J. (2001). Therapy outcomes in an evidence-based practice pilot. Document No. 11-04-3901. Olympia: Washington State Institute for Public Policy.
McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey
literature influence estimates of intervention effectiveness reported in meta-analyses?
The Lancet, 356(9237), 1228-1231. doi: 10.1016/S0140-6736(00)02786-0
* Mitchell-Herzfeld, S., Shady, T. A., Mayo, J., Han Kim, D. H., Marsh, K., Dorabawila, V., & Rees, F. (2008). Effects of multisystemic therapy (MST) on recidivism among
juvenile delinquents in New York state. New York: New York State & Office of
children and family services.
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., . . . Klassen, T. P. (1998). Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? The Lancet, 352(9128), 609-613. doi:
10.1016/S0140-6736(98)01085-X
MST Services Inc. (2010). MST multisystemic therapy: Breaking the cycle of criminal behavior by keeping teens at home, in school and out of trouble. Retrieved March 1, 2013, from http://mstservices.com/
MST Services Inc. (2012). Multisystemic therapy® (MST®): Research at a glance. (Research report). Mount Pleasant: MST Services Inc. Retrieved from
http://mstservices.com/outcomestudies.pdf
Mullen, B. (1989). Advanced BASIC meta-analysis, 1989. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
