UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)
UvA-DARE (Digital Academic Repository)
Routine outcome monitoring & learning organizations in substance abuse
treatment
Oudejans, S.C.C.
Publication date
2009
Link to publication
Citation for published version (APA):
Oudejans, S. C. C. (2009). Routine outcome monitoring & learning organizations in substance
abuse treatment.
General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.
—
—
ChAPTER 5
—
MEASURING
ThE LEARNING
CAPACITY Of
ORGANIZATIONS.
dEvELOPMENT
& fACTOR
ANALYSIS Of ThE
QUESTIONNAIRE
fOR LEARNING
ORGANIZATIONS
(QLO)
M E A S U R IN G L E A R N IN G C A P A C IT Y OUdEjANS, S.C.C. SChIPPERS, G.M. SChRAMAdE, M.h. kOETER, M.W.j. vAN dEN BRINk, W. SUBMITTEd fOR PUBLICATION—
ChAPTER 5 / ABSTRACT / PAGE 76
Abstract
Aims: To investigate internal consistency and factor structure of a questionnaire
measuring organizational learning capacity based on Senge’s theory of the five disciplines of a learning organization: Personal Mastery, Mental Models, Shared Vision, Team Learning, and Systems Thinking.
Design: Cross-sectional study.
Setting: Substance abuse treatment centers (SATCs) in the Netherlands. Participants: A total of 293 SATC employees: 213 treatment professionals (TPs),
and 80 employees working in supportive departments (SPs).
Main outcome measures: Psychometric properties of the Questionnaire for Learning
Organizations (QLO), including factor structure, internal consistency, and inter-scale correlations.
Findings: A five-factor model representing the five disciplines of Senge showed
good fit. The scales for Personal Mastery, Shared Vision, and Team Learning had good internal consistency, but the scales for Systems Thinking and Mental Models had low internal consistency.
Conclusions: The proposed five-factor structure was confirmed in the QLO,
which makes it a promising instrument to assess learning capacity in teams. The Systems Thinking and Mental Models scales have to be revised. Future research should be aimed at testing criterion and discriminative validity.
—
—
Introduction
Organizational learning is essential for quality improvement processes and it is thought to be crucial for increasing performance and effectiveness of health care. The concept of organizational learning emphasizes expanding capacity at a personal and organizational level. Treatment teams are key components in the delivery of high-quality patient care, and learning capacity is a charac-teristic of individuals and teams in maintaining and improving quality of pro-fessional care. Teams that acquire learning characteristics are able to handle complexity, organizational change, and rising expectations in today’s health care (Jeong et al., 2007; Kelly et al., 2007; Rushmer et al., 2007; Senge, 1992).
Theory on organizational learning
Peter Senge introduced the term “learning organization” with his book The Fifth
Discipline (Senge, 1992) to a bigger audience. He defined a learning organization
as one that possesses five core-learning disciplines: Personal Mastery, Mental Models, Shared Vision, Team Learning, and Systems Thinking (see Chapter 1, p.11 for a more detailed description of the theory and the five disciplines). The five disciplines for organizational learning were originally conceptualized for busi-ness organizations, but they are also applied in organizations in the public sector (Jeong et al., 2007; Kelly et al., 2007; Pang, 2004; Rushmer et al., 2007).
Empirical studies and instruments
Pang (Pang, 2004) developed a questionnaire for learning organizations in the context of elementary schools and showed that, in accordance with Senge’s fifth discipline discourse, Systems Thinking has the strongest associations with the other four scales in a learning community (Model 1). In addition, Pang suggested another model in which Systems Thinking derives from Team Learning, which, in turn, derives from the other three disciplines (Model 2), but this model was not empirically tested. Figure 1 (p.78) depicts the two models. Jeong ( Jeong et al., 2007) developed the Learning Organization Scale (LOS). The LOS was used in a health care setting and revealed positive associations between learning capacity and organizational commitment and job satisfaction of nurses in South Korean medical hospitals. However, no data was presented about the factor structure of this questionnaire, and therefore it is unknown whether factor structure of the questionnaire fits the original model of Senge. Moreover, both questionnaires are limited to Asia, and to our knowledge no translations exist. Recently, the Learning Practice Inventory (LPI) was
—
ChAPTER 5 / MEASURING LEARNING CAPACITY / PAGE 78
developed (Kelly et al., 2007; Rushmer et al., 2007). The LPI is a diagnostic tool for primary care settings in the United Kingdom and assesses practice life from the point of view of practice members. The LPI is based on learning organizations theory, but the empirical relation of the LPI with these theories and its psychometric properties are not known. This situation prompted us to develop and test the Questionnaire for Learning Organizations (QLO).
Figure 1: Model 1; five correlated factors
Figure 1: Model 2; three independent factors and two dependent factors
Aim of the study
The primary aim was to develop an instrument, based on the theory on organiza-tional learning of Senge, that assesses the learning capacity in individuals and teams using a sample of employees working in substance abuse treatment centers (SATCs).
—
MethodSubjects
Subjects were 720 employees of four Dutch SATCs in urban and suburban areas, working in different departments (e.g. outpatient and inpatient treatment departments, financial, and human resources). Of these subjects, 46% filled in and returned the questionnaire, resulting in 331 respondents. In 38 cases, one or more answers were missing and these respondents were excluded, leaving questionnaires of 293 respondents for analysis. Excluded respondents did not differ from the remaining sample in terms of age, gender, educational level, and SATC. Based on self-reported information on their position in the organization, 213 of the 293 respondents (73%) were assigned to the group of treatment professionals (TPs: counselors, psychologists, group therapists, doctors, nurses, and managers of treatment departments), and 80 respondents (27%) were assigned to the group of supportive professionals (SPs: human resources specialists, financial and quality staff, and receptionists). TPs and SPs were similar in terms of gender (61% female) and age (mean = 43.5; sd = 10.9). The proportion of highly educated respondents (a bachelor or masters degree)
was 87% percent in TPs and 63% in SPs (χ2
(2) = 21.1; p = 0.00).
Instrument and procedure
In the first development stage of the QLO, a team of psychologists, method-ologists and a management scientist generated 44 items according to the five disciplines of Senge’s theory. Finally, 38 items were chosen for the concept ques-tionnaire. Subsequently, two meetings were organized, one in which potential respondents commented on the clarity and formulation of items, and another after 80 respondents filled out the questionnaire after which deviant scores were discussed with respondents. During both meetings items were reformulated or deleted, resulting in the QLO with 36 items that is analyzed in this study.
The scales of the QLO include: Personal Mastery (PM; 9 items), Mental Models (MM; 4 items), Shared Vision (SV; 5 items), Team Learning (TL; 13 items), and Systems Thinking (ST; 5 items). The items employ a 6-point Likert-type scale ranging from 1 to 6, denoting “not applicable at all” for 1 and “very applicable” for 6. Higher scale scores represent a larger capacity on the repre-sented discipline, and a higher total score represents a larger learning capacity.
Respondents, who could remain anonymous, were asked to rate items accord-ing to their own work situation. The questionnaire contained items about the employee’s individual situation as well as the employee’s department. As some
—
ChAPTER 5 / MEASURING LEARNING CAPACITY / PAGE 80
employees carried out several tasks in different positions and departments, respondents were asked to fill-in their primary position and department. They were instructed to complete the department-related items about this primary department.
Data preparation and analysis
Skewness and kurtosis coefficients, means and standard deviations of items were
assessed, together with internal consistency (Cronbach’s α) of each of the five scales.
Next, by deleting items with the lowest inter-item correlation, the five scales were reduced to a maximum of five items in each scale (items with the lowest inter-item correlations were deleted) resulting in the final QLO for further analysis.
To investigate the factor structure of the QLO, confirmatory factor analysis (CFA) was conducted using LISREL 8.8 for Windows. The response format of the QLO items is regarded as an ordered categorical response format. Since cate-gorical scales with more than five categories can be treated as continuous in CFA, we analyzed the asymptotic covariance matrix using the Maximum Likelihood estimation procedure with the Satorra Bentler Scaled Chi-Square (Byrne, 1998; West, Finch, & Curran, 1995) for non-normal data. The two models presented in Figure 1 were tested. Although the chi-square statistic and degrees of freedom
are reported by LISREL, the absolute chi-square (χ2) value is not evaluated for
fit because the chi-square statistic is not particularly useful. Large samples (gen-erally above 200) will almost invariably produce significant chi-square values (Schumacker & Lomax, 1996). The present study used the 90% confidence interval for the root mean square error of approximation (RMSEA) with the sug-gested cutoff value of 0.05 indicating a close fit. Next, it is sugsug-gested that the p-value to test the interval should be > 0.50 (Byrne, 1998). In addition, the Adjusted Goodness of Fit Index (AGFI) was evaluated, ranging from zero to 1.00, with values close to 1.00 being indicative of good fit. Finally, Bentler’s Comparative Fit Index (CFI) was also calculated where values > 0.90 indicate an acceptable fit.
In addition we want to investigate whether the QLO can differentiate between groups of employees. Therefore, multiple regression was conducted using the general linear model (GLM) procedure, with comparisons for TPs and SPs and taking educational level into account, since groups differ in this respect.
Results
Descriptive statistics, reliability and data reduction
Table 1 presents descriptive statistics of all items and all scales of the initial QLO and the reliability of the scales. Most items were right skewed and had significant
—
Table 1:
Psychometric properties of the items and the five subscales of the QLO for the TP sample (n = 293)
Item Subscales (bold) and items; Cronbach’sα in parentheses Mean sd Skewness Z Kurtosis Z
Personal Mastery (0.75) 4.8 0.6 -4.8 3.1
a3* My professional expertise is essential for the achievement of good results. 5.2 0.9 -10.9 12.9
a5* I feel personally responsible for the achievement of good results. 5.2 0.8 -7.6 5.9
a6 I have spent time in the past month updating my knowledge and skills (for example, I read professional journals, attend conferences or am taking
a professional training course). 3.9 1.7 -2.9 -4.0
a8 I can exert a great deal of influence on improving how I do my job. 4.4 1.2 -4.4 -0.2
a10* My present job is extremely important to me. 4.9 1.0 -6.9 3.7
a13* I feel I am personally responsible for a job well done. 5.5 0.6 -9.2 11.3
a15* My professional expertise is an important part of how I do my job. 5.4 0.8 -14.9 29.1
a19 I take initiative when it comes to my job. For example, I introduce matters at the team consultations and present new ideas. 4.8 1.1 -6.1 2.0
a22 I envision a career for myself in substance abuse treatment. 3.8 1.5 -2.2 -3.1
Mental Models (0.65) 4.5 0.8 -4.8 2.0
a12* I adjust my standpoints and ideas about my job on the basis of quantitative results such as reports on outcomes, the annual report or research reports in my field. 3.7 1.4 -1.8 -2.4
a16* My standpoints and ideas about my job have a strong effect on how I do my job. 5.0 0.9 -8.2 6.7
a18* I often adjust my standpoints and ideas about my job via consultations with experts, for example colleagues, about concrete cases or incidents. 4.4 1.1 -6.2 3.4 a20* I discuss my standpoints and ideas with other experts in my field, for example colleagues or groups of experts. 4.8 1.1 -8.4 5.3
Shared Vision (0.59) 3.7 0.9 -1.3 -1.6
a1* My opinion matters when the joint policy line on treatment and so forth is formulated at [name of SATC]. 2.9 1.5 1.9 -4.2 a4* I think [name of SATC] has a clear perspective, for example on the treatment of clients or on quality care. 3.7 1.3 -2.1 -2.6
a11* I feel closely involved with [name of SATC]. 4.4 1.2 -4.3 -1.4
a17* My standpoints and ideas about my job are in keeping with the policy line on treatment and so forth at [name of SATC]. 3.9 1.2 -3.2 0.0
a23 I regularly work longer hours than I should. 3.7 1.7 -1.2 -4.3
Team Learning (0.88) 3.8 0.9 -1.5 -0.1
b1 The members of my team regularly talk about the statistical results of the work. 3.6 1.6 -1.0 -3.9
b2* Improvement actions are carefully carried out and monitored by my team. 3.7 1.2 -1.4 -1.7
b3* The results in papers or annual reports provide information my team can concretely utilize. 3.1 1.4 1.5 -3.3 b4* My team has formulated aims regarding the results of the work, for example waiting times, drop-out percentages and client satisfaction. 3.6 1.5 -0.4 -3.6
b5* The staff members of my team support each other in learning new skills. 4.7 1.1 -7.6 3.7
b6* The members of my team regularly stipulate how we can achieve better results and this leads to concrete improvement actions. 4.1 1.3 -3.1 -1.6
b7 The members of my team often discuss how the job is done. 4.6 1.1 -5.8 1.0
b8 The members of my team evaluate each other’s way of working. 3.3 1.3 0.4 -2.4
b9 My team’s statistical results are evaluated. 3.6 1.7 -0.3 -4.5
b10 My team is skilled at jointly studying problems at work, such as an excessively high drop-out rates or logistic issues. 3.7 1.5 -1.3 -3.1
b11 The members of my team have a lot of time for each other. 3.8 1.3 -2.1 -2.8
b12 Every three months my team has access to statistical results of the work. 3.0 1.7 3.0 -4.1
b13 The members of my team give each other open and honest feedback at official meetings such as team talks, case discussions or training sessions. 4.1 1.4 -4.0 -1.2
Systems Thinking (0.32) 4.4 0.6 -2.9 1.9
a2* My work results are partly determined by the efforts of staff members on my team. 4.5 1.2 -5.9 0.5
a7* My work results are partly determined by the efforts of staff members outside my team. 4.0 1.4 -3.5 -2.3 a9* As a professional I have the skills to distinguish the cause and effect of a problem. 5.1 0.8 -9.2 13.2
a14* As a professional I solve a problem by approaching it from various angles. 5.2 0.8 -7.7 7.6
a21 As a professional I usually solve problems alone. 3.1 1.3 1.9 -2.6
Total score# (0.89) 4.2 0.6 -3.0 0.8
—
ChAPTER 5 / MEASURING LEARNING CAPACITY / PAGE 82
peaked kurtosis, indicating deviations from normality. The high mean and low standard deviations of three out of five scales (PM, MM and ST) and the total score suggest ceiling effects. Reliability coefficients for PM and TL are satisfying, whereas the internal consistency of the other scales needs improvement. Eliminating items with lowest inter-item correlation resulted in the final 22-item QLO – the asterisks in Table 1 pertain to the items that were kept in the QLO – with higher internal consistencies, leaving only two scales with low internal consistency: MM and ST (Table 2). However scale scores still showed deviations from normality as expressed in the skewness and kurtosis coefficients and the mean scores PM, MM and ST became even higher.
Confirmatory factor analysis (CFA)
For Model 1, factor loadings and correlations between the factors were posi-tive and significant. For Model 2, non-significant factor loadings were found, indicating an improper model. Therefore, the latter model, with PM, MM, and SV as independent latent factors, and TL and SV as dependent latent factors, does not seem to fit the data. Further model fitting was performed for Model 1, i.e. the model with five correlated factors. The CFI for Model 1 indicates a satisfying fit, but AGFI and RMSEA and the associated confidence interval suggest a less satisfying fit. Modification indices provided by LISREL suggest that permitting associated measurement error between items a10: “My pres-ent job is extremely important to me” and a11: “I feel closely involved with the organization” share some variance over and above the measured concepts.
Permitting this association results in a significantly better fit (Model 1b; ∆χ2
(1)
42.85; p < 0.05). However, the squared multiple correlations for two items of the ST scale are very low (0.04) for item a2: “The results of my work are de-termined in part by the efforts of staff members on my team”; as well as for item a7: “The results of my work are determined in part by the efforts of staff members outside my team”, indicating that almost no variance of these items
Table 2:
Psychometric properties of the five subscales of the shortened QLO (Total n = 293)
α Mean sd Skewness Z Kurtosis Z
Personal Mastery 0.75 5.2 0.6 -7.8 10.3 Mental Models 0.65 4.5 0.8 -4.8 2.0 Shared Vision 0.70 3.7 0.9 -1.7 -1.1 Team Learning 0.85 3.8 1.0 -1.1 -0.9 Systems Thinking 0.51 4.7 0.7 -3.9 1.6 Total score 0.84 4.4 0.6 -3.3 1.8
—
is explained by the factor. These items should, therefore, be rephrased in order to produce a scale with better internal consistency.Table 4 presents the correlations between the five scales, showing weak as-sociations between TL and all other factors, suggesting that a two factor model with an individual learning factor (integrating PM, MM, SV, and ST) and a team learning factor (the original TL factor) might fit the data as well. Although correlations between PM, MM, SV and ST are not high, we chose to test this model as well (Model 3). This resulted in a model with inferior fit (Table 3), and with associated measurement errors for items originally designated to the PM and the SV scale, justifying the postulated model with the five correlated factors. Therefore, Model 1b is the best alternative.
To test whether the QLO differentiates between employee groups, the GLM procedure was applied. Multiple regression showed main effects for group membership
(i.e. SP or TP) and educational level (Wilks’ Lambda = 0.96; F5,278 = 2.57; p = 0.03 and
Wilks’ Lambda = 0.93; F5,278 = 3.92; p = 0.00 respectively), as well as an interaction effect
for group membership and educational level (Wilks’ Lambda = 0.96; F5,278 = 2.44;
p = 0.03). For Mental Models main effects exist for both group membership (F1 = 5.24;
p = 0.02) and educational level (F1 = 5.87; p = 0.01) with higher means for TPs and
Table 3:
Goodness of fit indices for Models 1, 2 and 3
χ2/df# RMSEA CI (90%) P value RMSEA CFI AGFI Δχ2/df$ ratio χ2/df
Model 1a 506.91/199 0.065 – 0.081 0.00 0.94 0.79 2.6
Model 1b 464.06/198 0.060 – 0.076 0.00 0.95 0.80 42.85/1* 2.3
Model 2 592.03/202 0.074 – 0.089 0.00 0.92 0.76 2.9
Model 3 734.72/208 0.086 – 0.10 0.00 0.89 0.72 3.5
Model 1a: five correlated factors # Satorra Bentler Chi-Square Model 1b: five correlated factors and one item pair with correlated measurement errors * significant better fit (p < 0.05) Model 2: three x two factor model $ compared to Model 1a Model 3: two factor model
Table 4:
Correlations between factors*
PM MM SV TL PM MM 0.62 SV 0.41 0.53 TL 0.26 0.31 0.37 ST 0.52 0.51 0.28 0.20
—
ChAPTER 5 / MEASURING LEARNING CAPACITY / PAGE 84
respondents with higher education. A main effect for educational level on Systems
Thinking was also found (F1 = 5.11; p = 0.03), with higher means for respondents with
higher education. Finally, an interaction effect was present for Team Learning (F1 =
4.53; p = 0.03), with highly-educated TPs scoring lower than highly-educated SPs, and less than highly educated TPs scoring slightly higher than less than highly educated SPs. Table 5 presents means for scales with significant results. It should be noted that error variances of groups were different for Mental Models and Systems Thinking, with the bigger variances in the smaller groups, indicating somewhat liberal statistical tests (Stevens, 1996). Differences in means between groups are not big rela-tive to their range and standard deviation, indicating moderate to small effect sizes.
Conclusions and discussion
This is the first study on the QLO and one of the very few empirical studies ex-amining the operationalization of the concept of learning organizations. This study was performed for treatment professionals and supportive professionals in a SATC. The presence of the five disciplines proposed by Peter Senge was identified in a confirmatory factor analysis, although some fit indices suggest improvements are possible. Improving psychometric properties – i.e. low internal consistency and the presence of ceiling effects – of Mental Models and Systems Thinking might result in better fit.
We also showed that the model in which Personal Mastery, Mental Models, and Shared Vision would predict Team Learning and Systems Thinking as dependent variables (Model 2) did not fit, as well as a model with one individual
Table 5:
Means and standard deviations for TPs and SPs and different educational levels
High education Group Mean SD n
Mental Models no TP 4.3 0.8 27 SP 4.1 1.1 29 yes TP 4.6 0.6 181 SP 4.3 1.0 49 total TP 4.6 0.7 208 SP 4.2 1.0 78 Team Learning no TP 4.0 1.1 27 SP 3.8 1.2 29 yes TP 3.7 0.9 181 SP 4.2 1.1 49 total TP 3.7 0.9 208 SP 4.1 1.1 78 Systems Thinking no TP 4.6 0.9 27 SP 4.4 0.9 29 yes TP 4.7 0.6 181 SP 4.8 0.7 49 total TP 4.7 0.7 208 SP 4.6 0.8 78
—
learning factor and one factor for team learning. This implies that learning capacity consists of five correlated factors and that the hierarchical ordering in which Personal Mastery, Mental Models and Shared Vision leads to Team Learning which in turn leads to Systems Thinking is not supported by our data.Finally, for different groups of employees, one different score pattern was found, suggesting treatment professionals being more flexible in beliefs, as reflected in different scores for Mental Models. However, educational level seems to play a role as well, as is suggested by an effect both in Mental Models and Systems Thinking – with highly educated employees having the higher capacity – and an interaction effect for both variables in Team Learning. However, effect sizes were small to moderate and the test statistic became liberal due to violated model assumptions, so these findings are preliminary and should be interpreted with caution.
Research efforts should be put into the further development of the QLO, with special attention to the Mental Models and Systems Thinking scales. Adding one extra item to both scales might raise internal consistency and in addition, for Systems Thinking, revision of items is necessary. Next, research is needed to establish criterion and discriminative validity of the QLO by comparing QLO scores of teams or organizations known to be different in learning capacity measured with an alternative criterion, like assessments by organizational experts. In addition, associations between learning capacity (measured with an improved version of the QLO) and characteristics like educational level, years of employment or gender and age and should be es-tablished in order to get a more detailed picture of the concept of people in learning organizations and learning organizations as such. Effort should be put into creating enough variability between subjects (i.e. educational level in this study was fairly homogeneous).
One limitation of this study is sample size. To apply CFA on ordinal or non-normal data, the use of asymptotic distribution-free (ADF) estimators is recommended. A practical limitation of using ADF estimators is the need for very large sample sizes of 1,000 subjects or more. Most researchers, therefore, treat the data as continuous by using ML estimators and correct this by using the Satorra Bentler scaled statistic (Byrne, 1998; Hoyle, 1995; West et al., 1995). However, a suggestion for future research would be testing the QLO in bigger samples in order to apply the correct procedures in CFA.
One conceptual issue has to be addressed here. Senge stresses that teams form the core of learning, and not individuals. In this context it is question-able if the measurement of Team Learning should be performed on individual subjects. Although we tackled this issue by making the team the object of evaluation in the scale Team Learning, it is still an individual assessment and
—
ChAPTER 5 / MEASURING LEARNING CAPACITY / PAGE 86
remains questionable if it is appropriate to calculate and summarize individual scores on a shared object of evaluation.
Treatment teams are key components in the delivery of high-quality patient care, and learning capacity is a characteristic of teams in maintaining and im-proving quality of addiction care. Results of the QLO can be used in assessing this capacity of teams. Next, it is interesting to see how the concept of orga-nizational learning is related to team climate, as is conceptualized and opera-tionalized by several researchers (Mathisen & Einarsen, 2004), with the Team Climate Inventory being most widely used (Anderson & West, 1996, 1998; Mathisen, Einarsen, Jorstad, & Bronnick, 2004; Ouwens et al., 2008). A posi-tive team climate is assumed to result in creaposi-tive and innovaposi-tive behavior that benefits team effectiveness, mainly in terms of job satisfaction and consumer satisfaction (Mathisen et al., 2004; Proudfoot et al., 2007). Organizational learning has a larger emphasis on personal development and flexibility. These features are more of concern for organizations to deal with considering the changes that are taking place in the addiction treatment field. Nevertheless, it is of interest to investigate how concepts relate since both play an important role in organizational science. In addition, comparisons with several other available instruments – based on several other learning theories like those of Argyris or Pedlar – developed in other fields can be worthwhile. However, the number of instruments with known psychometric properties is low (Moilanen, 2001).
Finally, the next step in research on organizational learning is to estimate its usefulness as a predictor of outcome of care, and to see whether the QLO is useful for the measurement of learning capacity in other branches of health care.
Acknowledgements
We would like to express our gratitude to the management and treatment professionals at the various centers for taking part in the study. The Jellinek (now the Jellinek division of Arkin), Brijder Substance Abuse Treatment Center (now the Brijder Substance Abuse Treatment Division at ParnassiaBavo Group), and Novadic-Kentron made it possible for us to conduct the study. We also thank Masha Spits at the AIAR for research assistance, and Angela Buchholz for her advice.
