Author: E. Kommers
Institutions: University of Twente and Gezondeboel Amsterdam
Faculty of Behavioural, Management and Social Sciences Department Health Psychology and Applied Technology
Enschede, August 20th, 2019
Insight into an online mental health improving intervention for employees –
a Mixed Methods Study of Gezondeboel
Abstract
Background. Absenteeism due to work stress has become very common recent years and has serious consequences for the individual employee, organisations and society. Optimizing the mental health of employees, is increasingly acknowledged by organisations. However, most mental health improving interventions developed for work places have three shortcomings: These interventions (1) have not been tested on their (potentially negative) effects, (2) cannot be considered evidence-based, or (3) lack an emphasis on positive psychology. eMental Health is an example of a mental health improving intervention and is in various ways increasingly implemented at work past decade. In practise eMental Health shows multiple advantages and disadvantages. However, it is rarely evaluated and not considered evidence-based. Furthermore, the implementation of eMental Health reveals several challenges, which have a negative impact on (long-term) usage. The current study evaluates the eMental Health platform Gezondeboel.
Methods. A mixed methods study has been conducted with quantitative- and qualitative data. A quantitative study with a pre-test-post-test design was used including four questionnaires to examine the effectiveness of the programs of Gezondeboel on employees. Furthermore, during the pre-test the work characteristics of employees were examined. After the pre-test was conducted (n=29), employees were given four months to use eight eMental Health programs of Gezondeboel. They could choose how many and which programs they worked on, and how much time they invested in these programs. The post-test was completed by sixteen participants. Five of these sixteen participants did not use the programs.
Eleven did use the programs. Therefore only this group was asked about usage, appreciation, effect, and implementation of eMental Health. In the qualitative study, semi-structured interviews (n=6) were conducted, to gather in-depth insight into employees’ usage, appreciation, and self-perceived effects. In addition, the perspective of employees about implementing Gezondeboel at work was investigated.
Results. It appeared that, although employees were enthusiastic about the eMental Health programs of Gezondeboel, only a small group of employees completed the baseline and worked on the programs for 1-6 weeks. Despite, most programs were not totally finished for multiple reasons, participants positively appreciated the programs of Gezondeboel. Some suggestion for improvement were mentioned to better suit the preferences of a more diverse group of employees (e.g. improve access to make contact with a coach or make the programs less time-consuming), and to increase program usage (e.g. implement reminders or something to keep you on track). Participants mentioned they improved their self-compassion and awareness, and had a better balance between work and personal life. Furthermore, they gained more knowledge and insight about themselves as employee and about how they could improve their mental health. No negative or harmful effects were mentioned. The pilot study showed a significant effect on mental wellbeing (p <0.03) and the subscale psychological wellbeing (p <0.03). The effects on perceived stress (p <0.06), work engagement (p <0.66), and general self-efficacy (p <0.06) could not be considered significant. Participants prefer working on the programs at home. Forgetting about the programs and the type of employee (e.g. not intrinsically motivated) were mentioned as main barriers for implementation. Participants thought eMental Health programs should be provided by the employer, and can best be used preventively for employees (who are at risk) at developing mental complaints.
Conclusion. The current study suggests that employees are enthusiastic about the eMental Health programs of Gezondeboel and experience multiple positive effects. However, only a small group of employees started with the programs, and the time that was spent on the programs weakened over time. As a result, almost none of the programs was fully completed. There are careful indications that the eMental Health programs improve the mental well-being of employees. To improve the implementation of the programs at work, Gezondeboel is advised to take into account the suggestions for improvement of the current and new programs. All these conclusions should be used with caution, since this study included a very small research group. Future research with a larger sample size is needed to confirm the results of the current study.
Keywords: eMental Health, mental health employees, positive psychology, wellbeing at work, stress, work engagement, self-efficacy.
Samenvatting
Achtergrond. Ziekteverzuim door werkstress komt de laatste jaren steeds meer voor en heeft serieuze gevolgen voor de individuele werknemer, bedrijven en de samenleving. Bedrijven erkennen steeds vaker het belang om de mentale gezondheid van hun werknemers te vergroten. Echter, de meeste interventies die worden aangeboden om dit te bewerkstelligen, schieten op drie vlakken te kort. Zo zijn de interventies (1) niet eerder getest op effectiviteit en (mogelijke) negatieve gevolgen, (2) kunnen ze niet als effectief worden beschouwd of (3) maken ze geen gebruik van positieve psychologie. eMental Health is een voorbeeld van zo’n interventie die de laatste jaren in verschillende vormen steeds vaker wordt geïmplementeerd binnen organisaties. In de praktijk komen verschillende voor- en nadelen aan het licht. Echter zijn ook veel eMental Health niet eerder onderzocht of konden niet als effectief worden beschouwd. Daarnaast brengt de implementatie van eMental Health verschillende uitdagingen met zich mee, welke resulteren in een negatief effect op het (lange termijn) gebruik van de interventie. In dit verslag wordt de evaluatie van het eMental Health platform Gezondeboel beschreven.
Methoden. In dit onderzoek is een mixed-methods studie uitgevoerd met kwantitatieve- en kwalitatieve data. Een pre-test-post-test design werd ingezet voor de kwantitatieve studie. Hierbij werden vier vragenlijsten gebruikt om het effect van Gezondeboel op werknemers te onderzoeken.
Daarnaast werden werk-gerelateerde eigenschappen van participanten geëvalueerd. Nadat de pre-test was afgenomen (n=29) kregen participanten vier maanden de tijd om acht eMental Health programs van Gezondeboel te gebruiken. Zij konden zelf kiezen aan hoeveel en aan welke programma’s zij wilden werken en ook hoeveel tijd zij hieraan wilden besteden. De post-test werd door zestien participanten ingevuld. Vijf van hen had geen gebruik gemaakt van de programma’s. Elf hadden dit wel gedaan.
Daarom werden alleen zij gevraagd naar hun gebruik, waardering, effect en de implementatie van eMental Health. In de kwalitatieve studie werden semi-gestructureerde interviews (n=6) afgenomen om inzicht te krijgen in gebruik, waardering en ervaren effecten van de programma’s bij werknemers.
Daarnaast werd de visie van werknemers onderzocht op het implementeren van Gezondeboel op werk.
Resultaten. Uit de resultaten bleek dat ook al waren de werknemers enthousiast over de Gezondeboel, enkel een kleine groep werknemers de pre-test invulde en aan de programma’s werkte voor een kleine 1-6 weken. Ondanks dat de meeste programma’s niet volledig waren afgerond, waren de participanten positief over Gezondeboel. Wel gaven zij suggesties om de programma’s bij een bredere groep werknemers aan te laten sluiten (bv. het verbeteren van het contact met een coach of door programma’s minder tijdrovend te maken), en om het programmagebruik te blijven stimuleren (bv.
toevoegen van een herinnering of iets anders wat stimuleert om aan de programma’s te blijven werken).
Participanten gaven aan dat zij meer zelf-compassie en bewustzijn hadden ontwikkeld en ook een betere balans tussen werk en privé konden houden. Daarnaast ervaarden zij een vergroot kennis en inzicht in zichzelf als werknemer en over hoe zij hun eigen mentaal welzijn konden verbeteren. Er werden geen negatieve of schadelijke effecten opgemerkt. De pilot studie liet een significant effect op mentaal welzijn (p <0.03) en de subschaal psychologisch welbevinden (p <0.03) zien. De effecten op waargenomen stress (p <0.06), werkbetrokkenheid (p <0.66) en algemene zelf-effectiviteit (p <0.06) waren niet significant. Participanten spraken de voorkeur uit om thuis aan de programma’s te werken. Het vergeten van de programma’s en het type werknemer (bv. niet intrinsiek gemotiveerd) werden gezien als implementatie barrières. De participanten waren van mening dat eMental Health programma’s door de werkgever aangeboden dient te worden, en dat deze het best preventief werken om te voorkomen dat werknemers (met een verhoogd risico) mentale klachten ontwikkelen.
Conclusie. Dit onderzoek toont aan dat werknemers enthousiast zijn over de eMental Health programma’s van Gezondeboel en dat zij hierdoor verschillende positieve effecten ervaren. Echter maakte enkel een kleine groep gebruik van de programma’s en zwakte de tijd die zij aan de programma’s besteedde sterk af. Dit had als gevolg dat bijna geen van de programma’s volledig was afgerond. Met veel voorzichtigheid kan worden gesteld dat de eMental Health programs het mentaal welzijn van werknemers verbetert. Om de implementatie van de programma’s op de werkvloer te bevorderen wordt Gezondeboel geadviseerd om de genoemde suggesties voor verbetering te verwerken in de huidige en nieuwe programma’s. Deze conclusies dienen met voorzichtigheid te worden gebruikt, omdat de onderzochte aspecten op een kleine onderzoeksgroep zijn geëvalueerd. Vervolgonderzoek met een grotere onderzoeksgroep is noodzakelijk om de uitkomsten van dit onderzoek te bevestigen.
Table of Content
Abstract ... 2
Samenvatting ... 3
Introduction ... 6
1.1 Consequences of absenteeism due to work stress ... 6
1.2 Interventions for improving mental health at work ... 7
1.3 eMental Health as intervention at work, and the lack of evidence ... 8
1.4 Advantages, disadvantages and challenges of eMental Health ... 9
1.5 Positive Psychology ... 10
1.6 Gezondeboel ... 10
1.7 Study aims and research questions ... 12
Method ... 13
2.1 Design ... 13
2.2 Participants... 13
2.3 Sampling method and procedure ... 13
2.4 Intervention ... 15
2.5 Additional questions and questionnaires ... 16
2.6 Interview scheme ... 18
2.7 Data Analysis ... 18
Results pre- and post-test study ... 20
3.1 Characteristics of the participants ... 20
3.2 Usage ... 22
3.3 Appreciation ... 23
3.4 Self-perceived effect ... 24
3.5 Effect according to questionnaires ... 25
Results interviews ... 26
4.1 Participants... 26
4.2 Usage ... 26
4.3 Appreciation ... 28
4.4 Effect ... 35
4.5 Implementation on the work floor ... 37
Discussion ... 42
5.1 Usage ... 42
5.2 Appreciation ... 43
5.3 Effect ... 44
5.4 Implementation on the work floor ... 45
5.5 Strengths ... 46
5.6 Limitations and recommendations for follow-up research ... 47
5.7 Recommendations for Gezondeboel and further research ... 48
Conclusion ... 49
References ... 50
Appendix A. Information for participants about the research and participation ... 55
Appendix B. Online informed consent ... 57
Appendix C. Email for participants who filled out the pre-test ... 58
Appendix D. Description of the programs participants could choose ... 59
Appendix E. Last mail for participants to fill out the post-test ... 61
Appendix F. Visual representation of Gezondeboel ... 62
Appendix G. Interview ... 64
Appendix H. Overview final codes, sub codes and variations ... 66
Appendix I. Broad overview positive aspects ... 67
Appendix J. Broad overview suggestions for improvement ... 69
Introduction
Dutch philosopher Achterhuis (1984, p. 36) defined labour as a ‘fancy medicine’: you can become sick of working, feel loaded and live a very structured life with constantly taken into account other peoples’
agendas and preferences. Besides, you can feel better by working, enjoy interesting challenges, and receive the opportunity to develop yourself. This paradox is also visible when looking back at the two Latin translations: ‘labor’ which is defined as work as effort, tax or trouble. In contrast to ‘opus’ which is defined as work as result, challenge and self-fulfilment (Schaufeli & Bakker, 2007). Unfortunately, these days a considerable number of people feel the pressure and the negative sides of working.
1.1 Consequences of absenteeism due to work stress
Absenteeism due to work stress has become very common recent years and has serious consequences for the individual employee, for organisations and for society. Especially the last months of 2017 and the first quarter of 2018 show a strong peak in absenteeism. The latest statistics of the Central Bureau for Statistics showed that 49 working days out of thousand were skipped due to illness (CBS, 2018a).
Most absence is seen in the healthcare and social service, public governance, industry, and education sectors (CBS, 2018b). Twenty-five percent of the absence is caused by mental complaints. The greatest culprit is ‘experienced work stress’, which caused almost 36% of absenteeism (Bakker & Demerouti, 2014). Ganster and Rosen (2013) define work stress as the process by which workplace psychological experiences and demands (stressors) produce both short-term (strains) and long-term changes in mental and physical health. It is a negative consequence which can be experienced by employees who have a high workload. In The Netherlands, 2.700.000 employees reported high workload and one million employees are at increased risk to develop a burnout (TNO, 2014). Burnout is defined as a persistent, negative, work-related state of mind in ‘normal’ individuals that is primarily characterised by exhaustion, which is accompanied by distress, a sense of reduced effectiveness, decreased motivation, and the development of dysfunctional attitudes and behaviours at work (German Sociey for Psychiatry, 2016; Shaufeli & Enzmann, 1998). According to Maslach, Schaufeli and Leiter (2001), burnout is a syndrome consisting of three dimensions: exhaustion, cynicism, and reduced professional efficacy.
Exhaustion refers to feelings of reduction in emotions. Due to high workload, occurrence of depression and anxiety are frequent (Harvey et al., 2017; Knudsen, Harvey, Mykletun & Overland, 2013; Knudsen, Overland, Aakvaag, Harvey, Hotopf & Mykletun, 2010). Unfortunately, mental complaints may go unnoticed in the workplace, as they can often be characterized as ‘normal’ stress during work (Knudsen, Harvey, Mykletun & Overland, 2013).
Not only employees are suffering. Employers are faced with high costs caused by employee absence. Firstly, according to the Dutch Organisation for Applied Scientific Research average absenteeism through psychological complaints can cost organisations 7.555.000 days of absence, which make a total of €1,8 milliard each year. Despite this, inspections of the Ministry of Social Business and
Employment have concluded most employers pay the significant costs of absenteeism (TNO, 2014;
Arbo Rendement, 2016). Secondly, productivity loss, also called presenteeism, results in costs for the employer. Examples of productivity loss are a declining ability to concentrate, decreasing quality of interpersonal communication, working more slowly, and the need to repeat a work task. Mental illness most often appears as the top-ranked condition associated with productivity loss (Goetzel et al., 2004).
Tan et al. (2014) and National Institute for Health and Clinical Excellence (2008) state that organisations have increasingly recognised the importance of optimizing the mental health of employees for a variety of reasons, including: ethical reasons, to improve productivity, meet legislation changes, decrease the risk their employees develop a burnout, and to reduce their own cost burden.
1.2 Interventions for improving mental health at work
Interventions to improve the mental health of employees are increasingly used on the work floor.
However, most of the established workplace mental health interventions have not been evaluated on their (potentially negative) effects or could not be considered evidence-based (Joyce et al., 2016; Stratton et al., 2017). According to LaMontagne et al. (2014) workplace mental health interventions have three aims: (1) to protect mental health by reducing work-related risk factors for mental health problems, (2) promoting mental health by developing the positive aspects of work as well as worker strengths and positive capacities, and (3) address mental health problems among people regardless of cause. This is partly in line with the statement of Busch and Meijer (2019). They state that there are two kinds of interventions that improve mental health in general. First, ‘health-promotion’ which focuses on strengthening factors and reducing risk factors for psychological (un)health(iness) (what kind of factors depends on the experienced complaints). Second, ‘disease prevention’, which aim at reducing psychological complaints to prevent a mental disorder. These interventions are focussed on resilience and stress resistance in order to prevent employees to develop a burnout, anxiety or depression. Mainly interventions are offered to people with larger mental complaints (people who are at risk), and are focussed on disease prevention (Busch & Meijer, 2019). However, according to Bakker and Demerouti (2014) interventions should not only focus on health-promotion or disease prevention. They state that to increase the mental health of employees or to keep them mentally healthy it is important to provide resourceful work environments, since such environments increase job performance and facilitate work engagement. Work engagement is used to describe the extent to which employees are involved with, committed to, and enthusiastic and passionate about their work (Macey and Schneider, 2008). The concept has emerged from research showing that certain employees find pleasure in work notwithstanding stressors and job demands (Bakker, Hakanen, Demerouti and Xanthopoulou, 2007). It is a positive and continuous work mind-set, consisting of three dimensions: dedication (i.e. taking pride in their work and enjoying it), vigor (i.e. being energetic and resilient), and absorption (i.e. being focused and engrossed in work) (Schaufeli, Salanova, González-Romá, & Bakker, 2002). Work engagement is very important for a positive mental health of employees, because employees who are most committed
to their organization outperform others by 20% on the job (Attridge, 2009). To improve the mental health of employees organisations should offer their employees resourceful work environments, sufficient job challenges, and job resources, including assignments in which different skills can be used, offer social support and feedback. Organizations can decide to invest in training their employees to better be able to stay mentally healthy.
1.3 eMental Health as intervention at work, and the lack of evidence
Harrison et al. (2011) stated that the past decade has been an explosion in the delivery of eMental Health interventions at the workplace. Either by creating their own programs for internal use or through buying commercially available products (Birgit, Horn & Maekers, 2013; Ahthes, 2016). Current electronic distribution outlets enable organizations, not affiliated with health organizations, to easily create and distribute their own eMental Health technologies to improve the (mental) health of their employees (Van Gemert-Pijnen, Peters & Ossebaard, 2013). However, interventions are rarely evaluated for employees.
This paragraph describes eMental Health in general. eMental Health includes a wide variety of technological tools (e.g. Apps) to improve mental health. These tools are designed to effectively communicate the right information needed by different public at the right time, and in the right place, to guide mental health promotion (Kreps & Neuhauser, 2010). Components of eMental Health interventions are Cognitive Behavioural Therapy (CBT), Stress Management, Mindfulness approaches and Cognitive training. Some eMental Health interventions offer self-help programs, without any human guidance or supervision. While others involve supporting professional guidance (e.g. a coach), and therefore are more costly (Andersson et al., 2013). Both guided and unguided eMental Health interventions have been examined on short- and long term effects. For treating anxiety and depressive conditions, eMental Health interventions with CBT deliver most benefits (Andrews, Cuijpers, Craske, McEvoy & Titov, 2010; So et al., 2013; Gilbody et al., 2015). Interventions that included guidance throughout the intervention period showed greater positive improvement on the effects of an intervention than unguided interventions (Andrews, Cuijpers, Craske, McEvoy, & Titov, 2010;
Andresson, Cuijpers, Carlbring, Riper, & Hedman, 2014). Evidence has also emerged supporting Mindfulness based eMental Health interventions. However, those studies did not test the effects on employees as separate group (e.g. on profile, risks, function and response) (McIntyre et al., 2017).
Therefore, this evidence may not be generalizable to the occupational setting (Stratton et al., 2017). Also because of the lack of evidence, there is no indication whether eMental Health interventions are effective in the occupational setting, or even if they may cause harm (Ahthes, 2016). In general it is clear that the use of eMental Health can be of value to improve the mental health. However, more research about the use and effects of eMental Health in the workplace is needed.
1.4 Advantages, disadvantages and challenges of eMental Health
Despite the lack of studies which look into the effectiveness of eMental Health interventions, various advantages and disadvantages emerge in practice. Furthermore, the implementation contains several challenges. An advantage of eMental Health is that it improves education, self-management, and it greatly enhances the possibility for monitoring users (e.g. through a therapist or coach). Besides, it enables to reach general public, which can enhance the quality and usage of health promotion efforts (Mosa, Yoo & Sheets, 2012). Also eMental Health can reach various public with information that matches the health needs, and health information can be easily updated and adapted to changing health conditions. It can foster greater participation between interdependent health care providers and consumers and insure users have access to timely and accurate information to guide their decisions (Kreps & Neuhauser, 2010). Another advantage is that people can use eMental Health at any time they want, and that it can be used preventively (Gezondeboel.nl, 2018). Furthermore, eMental Health can be used to mitigate the costs of mental health improving interventions (Birgit, Horn & Maekers, 2013;
Ahthes, 2016).
On the other hand, eMental Health brings disadvantages and challenges. Firstly, the content of health apps is not always reliable, as it is often written by non-expert writers (O’Neill & Brady, 2012).
Secondly, depending on the national situation, mobile medical apps may be considered to be medical devices. As such, they would be subject to specific safety and security regulations that apply. Thirdly, when eMental Health technology is used to collect medical information and to share it with health professionals, privacy is a delicate issue (Kotz, 2011). Furthermore, there is a never-ending need to accommodate for people’s privacy preferences, since new technologies raise new ethical concerns for all involved (Van Gemert-Pijnen, Peters & Ossebaard, 2013). Also users of eMental Health are depending on internet to access health information (Kreps & Neuhauser, 2010). Another challenge is implementation. eMental Health can only be effective if it is used, and used in the right way. The interventions must be designed to maximize interactive communication with users to encourage their active involvement in health promotion. Also it must be designed to work effectively and with a various population of users. It must be designed to have broad reach across diverse populations, while at the same time adapting to the specific interests of different users. Interventions must be designed to personally engage the interests and emotions of users to promote maximum message exposure and influence (Kreps & Neuhauser, 2010). Another disadvantage is the lack of evidence eMental Health is safe and delivers significant positive effects. With some exceptions, eMental Health is showing improved, but not stunning, results. Many questions remain about how eMental Health can be effectively used to influence health behaviours (Neuhauser & Krep, 2010).
A challenge which needs to be named is the implemention of eMental Health. The study of Folker et al. (2018) made clear that integration in employees life can be difficult because of the perceived skepticism towards it. Another important issue for implementation concerns the long-term usage. The transitions from enthusiastic beginning to permanent usage on the long-term can be difficult. Feijt, De
Kort, Bongers and Ijsselsteijn (2018) stated that not having sufficient time to invest in eHealth during working hours significantly hinders the implementation of eMental Health. More research is necessary to get clarity about specific barriers of implementing eMental Health.
1.5 Positive Psychology
Next to the lack of evidence-based interventions for employees, accessible interventions do not emphasize positive psychology. There are parallels between recovery in mental health and positive psychology (Sheldon, Kasser, Smith & Share, 2002). The positive psychology focusses on personal growth, purpose in life, positive relations with others (all indicated as strong or weak), self-acceptance, environmental mastery and autonomy (all indicated as high or low) (Ryff, 2006). These constructs are also necessary for high employee wellbeing. Employee wellbeing is important, because it is a meaningful predictor of: (1) employee and organizational productivity, (2) absenteeism, and (3) turnover (Wright & Cropanzano, 2004; Wright, Cropanzano & Bonett, 2007).
Before the turn of the century, scientific psychology focused on the negative mental states (e.g.
headache, tired, extremely emotional). As a result, there is lots of knowledge about psychological illnesses and limitations and only little knowledge about potentials and vigour of the human psyche (Linley, Joseph, Harrington & Wood, 2006). Seligman and Csikszentmihalyi (2000) introduced positive psychology. They state that the field of positive psychology at the subjective level is about valued subjective experiences, wellbeing, contentment and satisfaction (in the past), hope and optimism (for the future), and flow and happiness (in the present). Moreover, they see the positive psychology as an alternative to the one-sided psychopathology, disorders, and troubles related to psychological science (Seligman & Csikszentmihalyi, 2000). Positive psychology makes use of the scientific study of human strength and optimal functioning. In contrast to only cure employees who dropped out of work (in Dutch called ‘curatie’), positive interventions are aimed to improve the positive conditions (in Dutch called
‘amplitie’) of employees who are still working, like wellbeing, pathos and health (Ouweneel, Schaufeli
& Le Blanc, 2009). Besides, positive psychology focuses on personal qualities and optimal functioning (Schaufeli & Bakker, 2001). Emphasizing positive aspects of wellbeing makes it possible to shift from prevention and treatment-oriented approaches, towards development and optimization in companies (Ouweneel, Schaufeli & Le Blanc, 2009).
1.6 Gezondeboel
Gezondeboel has been developed specially for employees to provide guidance in keeping employees mentally healthy or to improve the mental health of employees. Furthermore, in the long term Gezondeboel aims to decrease the absenteeism costs for organisations. Gezondeboel picks up modern innovative technology and the use of positive psychology. “Gezondeboel” is developed by Therapieland. Therapieland is an online platform originated from the positive psychology, which provides eMental Health programs for healthcare patients. The programs of Gezondeboel also provides online programs, however these are focussed on business healthcare. It provides almost thirty programs
with different themes which aim to prevent employees develop mental complaints or to decrease experienced mental complaints of employees before they drop out of work. Self-efficacy is one of the most important constructs of Gezondeboel. Self-efficacy is the mechanism which enables people to take control over their thoughts, feelings and act, and to steer behaviour by making choices (Bandura, 1997).
According to Bandura (1997) behaviour, personal factors (e.g. knowledge, skills, attitude) and environmental factors (e.g. social support, presence of others) are seen as interlocking and mutually influencing factors. The influence of each factor can differ per situation. Self-efficacy is defined as the personal judgment of an individual about his or her own skills, to achieve a certain result in a given situation. The concept is expressed in the degree of confidence in the possibilities to perform certain behaviour. Self-efficacy affects a variety of things: (1) actions that someone performs, (2) the effort that someone makes to achieve the proposed result, (3) the resilience that someone has in the setbacks and the perseverance, and (4) thinking patterns and emotional reactions (Bandura, 1986). The concept is seen as a predictor of health behaviour (Aljasem, Peyrot, Wissow & Rubin, 2001). The programs of Gezondeboel aim to improve self-efficacy in multiple ways, including theories about mental wellbeing and possible complaints (both written and spoken by a video therapist/ coach), homework assignments and a library with informative documents and videos of experts from the field, and a group chat to talk with peers (Gezondeboel, 2018). Gezondeboel does not only want to improve the mental wellbeing of employees who already dropped out of work. It also aims to have a preventive effect. This corresponds closely to the study of Ouweneel, Schaufeli and Le Blanc (2009). They conclude that work associated interventions need to focus on both employees who dropped out (e.g. through a burnout), and employees who still function well but of which the potential is not used yet. Gezondeboel adds the third group ‘at risk’; which is about people who are not dropped out yet, but are at risk to drop out because of increased mental complaints. Together these groups are named: @home, @risk and @work. The programs of Gezondeboel aim to be valuable for each group (Gezondeboel, 2018). By means of four claims Gezondeboel purposes to indirectly reduce absenteeism and the associated costs (within organisations), and increases the mental wellbeing and productivity of employees. The claims are as follows:
Gezondeboel…
1. … maintains and / or improves the mental wellbeing of employees who are at risk; to prevent they develop a stress related disease.
2. … makes employees more aware of their mental wellbeing.
3. … makes employees aware they can take the lead in order to improve their own mental wellbeing.
4. … provides employees tools to work on their own mental wellbeing.
1.7 Study aims and research questions
As mentioned before, like many other eMental Health interventions which are promoted in workplaces, Gezondeboel is not previously evaluated and tested on its effect. To be sure Gezondeboel can be used to increase the mental health of employees, and it does not cause any harm, more evidence is needed on the effectiveness and potential risks (Stratton et al., 2017). Furthermore, it is important to get in-depth insight into employees’ usage, appreciation and self-perceived effects of the platform, to investigate whether Gezondeboel works as intended. Also employees’ opinion about implementing the intervention at the work floor must be examined, to improve the uptake in practice. Therefore, one pilot study and multiple semi-structured interviews are conducted among employees. This study contains four leading questions:
(1) Usage: What are reasons of employees to work on the programs, what is the way employees chose the programs, and which choices are made about how to (not) proceed with the programs?
(2) Appreciation: Which aspects of Gezondeboel are positively appreciated by employees, which aspects are negatively appreciated, and how do employees appreciate the ‘coach’ feature?
(3) Effect: What is the effect of the Gezondeboel programs on the mental health (stress, mental wellbeing, work engagement, and self-efficacy) of employees, and which effects do participants experience themselves?
(4) Implementation: What are barriers and opportunities for implementing Gezondeboel on the work floor, what type of employees can benefit from Gezondeboel, and how and when can Gezondeboel be provided?
Method
2.1 Design
In order to answer the research questions this study applied a mixed methods approach consisting of an one group pre-test-post-testdesign (n= 29) and semi-structured interviews (n= 6). Four questionnaires and additional questions were taken to examine the usage, appreciation, and effects of the programs of Gezondeboel on employees. Besides, interviews were used to receive in-depth insight about employees’
point of view on these constructs, and how they think about implementing Gezondeboel at work. The method section presents the quantitative- and qualitative data integrated. In the results section, the two will be separately discussed.
2.2 Participants
It was planned to include at least hundred employees in this study. The participants of this study were employees from the general population who wanted to improve their mental wellbeing or gain more knowledge about mental health. Inclusion criteria for the participants were that they (1) were employee of a company based in The Netherlands (there was no recommended sector or province employees had to work in, and also self-employed without employees were allowed to participate), (2) mastered the Dutch language very well, both spoken and written, (3) had basic knowledge and experience with using a computer and internet, (4) had an own computer with internet or the possibility to borrow one on daily basis (since the intervention and tests were provided on the internet), and (5) were eighteen years or older. No criteria were applied regarding experienced work stress or mental complaints.
2.3 Sampling method and procedure
The current study has received ethical approval from the Ethics Committee of the University of Twente.
After this approval, participants were recruited by means of snowball sampling (Biernacki and Waldorf, 1981). In Figure 1 a flowchart of the procedure and participants is displayed. During the month of data collection (December 2018), potential participants were assembled in two ways. Firstly, all employees of Gezondeboel asked other employees of their network to participate in the study by the use of an email with extensive information about the research aims (Appendix A describes the content of the email).
These new employees asked potential participants too. Secondly, a short video was made which promoted the study on Social Media (LinkedIn, Twitter and Facebook). Interested employees could send an email to Gezondeboel. Potential participants who self-applied received an email with extensive information about the research, the informed consent (Appendix B), and the questions of the pre-test.
The informed consent explained the participants’ rights and guaranteed anonymity and confidentiality.
It could be accepted by filling in the check mark. All participants were asked if they were willing to participate in an interview. They were informed that the interviews were recorded with a voice-recorder.
Responding to this question could be done by filling in the second check mark. After accepting the informed consent and returning the filled in questionnaires, the participants received an internet link by
email (Appendix C). By the use of this link a free-account could be made which gave permission to the programs of Gezondeboel. Participants could choose between eight different programs (Appendix D provides an extended description of each program). Participants engaged in the intervention from the first of December 2018 till 4 March 2019. This period was chosen, to allow participants to work on multiple programs in a row, since most programs would take around four till six weeks according to the authors of the programs. Besides, the speed of completing the programs depended on participants’ own speed. The accounts could be made between the first of December 2018 and 14 January 2019 only, since there was no time left to engage in the intervention if participants joined after this period. One week before the end date of the intervention all participants received a reminder to fill out the post-test (Appendix E). After the post-test was conducted, the interviews were executed with participants who agreed to participate.
In total twenty-nine participants filled-out the baseline. They were aged between twenty-two and sixty-five. Twenty-five of them were female and four were male. Thirteen participants dropped-out because they did not completed the post-test. The eleven participants who did filled out the post-test and made use of the intervention, were named ‘adherence group’. The five participants who filled-out the post-test but did not used the intervention, were named ‘non-adherence group’. In the analyses section a broader description is given about the analyses where the non-adherence group was exclude. Six of the sixteen participants who filled-out the pre- and post-test were interviewed. The characteristics of all four groups were presented in the result section (Table 2).
Figure 1. Flowchart of procedure and participants
2.4 Intervention
The intervention began directly after the participants made their own account. By logging in the participant had access to eight Gezondeboel programs: mindfulness, well sleeping, happiness, balance, StressLess, resilience (in Dutch: veerkracht), self-compassion and worrying (in Dutch: piekeren). In Appendix F a visual representation of how Gezondeboel looks like is displayed. Participants could choose which program they wanted to work on. It was allowed to change program if the first chosen program did not meet the expectations or when another program seemed to fit better. Besides, it was allowed to work on more programs at the same time. The authors of Gezondeboel stated that the programs would take between six till ten weeks, and advised to log in at least twice a week. However, participants could work on the program(s) at their own pace. Therefore they might needed less or more time to complete a program. By clicking on a theme, the participant opened the program consisting of several lessons. Firstly, a video coach introduced the program and gave information about how Gezondeboel worked, and where everything was located in de program. After this short introduction the participant continued by listening and reading theoretical information about the chosen theme.
Subsequently, several practical homework assignments were presented, which could translated theory into practice. On the right side in-depth information could be found by the use of videos, literature and explanatory images. Participants could click on every feature they wanted to see, which made it possible to go back, skip or continue to other features of interest. Since the participants worked on the program(s) on their own and therefore possibly experienced troubles (e.g. because of confrontation or working on their mental complaints), it was possible to contact one of the psychologist or the help-desk of Gezondeboel during every working day between 09.00 AM and 17.00 PM. Participants who filled out both pre- and post-test were granted access to the programs for another year as thanks for their participation.
2.5 Additional questions and questionnaires
During the pre-test participants were asked about their demographics and work characteristics (Table 1, result section). This was evaluated to gain insight into what kind of employees were interested in the eMental Health programs. Also four questionnaires were filled out: (1) Mental Health Continuum-Short form (MHC-SF), (2) Perceived Stress Scale (PSS), (3) Utrecht Work Engagement Scale (UWES), and (4) General Dutch Self-Efficacy Scale (SES). During the post-test the same four questionnaires were used. Furthermore, participants were asked additional questions during the post-test. Firstly, they were asked if they made a Gezondeboel account and if they used it. The participants who positively answered this question became the ‘adherence group’. The participants who negatively answered this question became the ‘non-adherence group’. The additional questions were about how participants used and appreciated the programs, and which effects they self-perceived. To evaluate the self-perceived effects, the claims Gezondeboel states were used. (See Tables 4-7 in the result section for the exact wording of items.) Only during the interviews participants were asked about the implementation of eMental Health programs on the work floor, since it was preferred to gather broad information about this aspect, and there was no time to do this during the post-test.
2.5.1 Mental Health Continuum- Short Form
To examine the influence of the programs on mental health and wellbeing the Mental Health Continuum- Short Form (MHC-SF) was used. This questionnaire is derived from the long form which include forty items measuring emotional wellbeing (Keyes, 2002). The short form consists of fourteen items which are most prototypical for representing the three facets of wellbeing: three items measure emotional wellbeing (e.g. ‘In de afgelopen maand, hoe vaak had u het gevoel dat u tevreden was?’), six items represent psychological wellbeing (e.g. ‘In de afgelopen maand, hoe vaak had u het gevoel dat u zelfverzekerd uw eigen ideeën en meningen gedacht en geuit hebt?’), and five items social wellbeing (e.g. ‘In de afgelopen maand, hoe vaak had u het gevoel dat u deel uitmaakte van een gemeenschap?’).
Answers can be given on a six-point scale which was ranged as follows: 0 = Never, 1 = Once or twice, 2 = About one time a week, 3 = Two or three times a week, 4 = Almost every day, and 5 = Every day.
Totals can be calculated by averaging the scores on items. Higher scores indicate higher levels of
wellbeing (Lamers, Westerhof, Bohlmeijer, & Ten Klooster, 2010). In this study the Cronbach’s α were 0.91 (total scale), 0.86 (emotional wellbeing), 0.82 (psychological wellbeing), and 0.75 (social wellbeing).
2.5.2 Perceived Stress Scale
To examine how much stress the participant perceived the Perceived Stress Scale (PSS) was used (Cohen, 1983). The questionnaire consists of ten items which ask the participant how often he has had certain feelings and thoughts within the last month (e.g. ‘In de afgelopen maand, hoevaak was u boos omdat dingen buiten uw controle waren?’; and ‘In de afgelopen maand, hoevaak voelde u zich zelfverzekerd over uw vermogen om met persoonlijke problemen om te gaan?’ reverse coded). Answers are given on a five-point Likert Scale which is ranged from never (0) to very often (4). Four items (4, 5, 7 and 8) have to be reversed, and the total score can be calculated by summing up the ten items for each individual. The total sum score can range from zero till forty. The higher the score, the higher the perceived stress (Cohen, 1983). In this study the Cronbach’s α was 0.93.
2.5.3 Utrecht Work Engagement Scale
The Utrecht Work Engagement Scale (UWES) is currently the most used measure to assess work engagement (Shaufeli & Bakker, 2003). The results provide support for a four-factorial model of work- related wellbeing consisting of job satisfaction, occupational stress, burnout, and engagement (Rothman, 2002). The questionnaire consists nine, fifteen or seventeen items. In this study the nine-item scale was used, because of the large number of total questions the participants needed to answer. The nine-item scale consists of three subscales: vigor (three items: e.g. ‘Op mijn werk bruis ik van energie.’), dedication (three items: e.g. ‘Ik vind het werk dat ik doe nuttig en zinvol.’), and absorption (three items: e.g.
‘Wanneer ik heel intensief aan het werk ben, voel ik mij gelukkig.’). Participants rate their levels of employee engagement on a seven-point Likert scale which was ranged from never (0) to always (6). The subscale scores are calculated by summing up the scores per scale, and then divide it through the number of items of the scale. For calculating the total score it can be done the same way. The total sum score can range from zero till fifty-four. The higher the score, the more the participants owns the subject of the scale (e.g. work engagement, vigor, dedication or absorption) (Shaufeli & Bakker, 2003). In this study the Cronbach’s α were 0.91 (total scale), 0.86 (vigor), 0.89 (dedication), and 0.62 (absorption).
2.5.4 Dutch General Self-Efficacy Scale
The Dutch General Self-Efficacy Scale (GES) is an unidimensional questionnaire which measures how someone in general deals with stressors and difficult situations in life (Teeuw, Schwarzer & Jerusalem, 1994). Ten statements (optimistic self-beliefs) are about how people generally think and act. In contrast to other scales that measure optimism, the GES is explicitly about someone's self-confidence that his or her actions are responsible for successful outcomes, or that they have control over the challenging demands of the environment (e.g. ‘Het lukt mij altijd moeilijke problemen op te lossen, als ik er genoeg
moeite voor doe.’; and ‘Als ik geconfronteerd word met een problem, heb ik meestal meerdere oplossingen.’). Participants rate on a four-point scale to what extent the proposition matches their personal opinion at the moment. They can chose: Completely untrue (1 point), Barely true (2 points), Rather true (3 point), and Completely true (4 points). The total score can be calculated by summing up the points of the items. The sum score can range from ten till forty. The average in The Netherlands is twenty-nine points with a standard deviation (SD) of four. The higher the score, the more self-efficacy the participant has in general (Schumacher, Klaiberg & Brähler, 2001). In this study the Cronbach’s α was 0.83.
2.6 Interview scheme
The interviewer (E.K.) used a semi-structured interview scheme to receive in-depth understanding into participants’ experience with Gezondeboel. Participants were asked multiple open-questions about: (1) usage (e.g. ‘What was your reason to participate and did or did you not continue with the programs?’);
(2) appreciation (e.g. ‘Which aspects of Gezondeboel or the programs did you like or did you positively appreciate and why?’); (3) effect (e.g. ‘Can you describe what you have learned by the use of Gezondeboel?’); and (4) implementation on the work floor (e.g. ‘Do you think Gezondeboel is useful for the organisation you work for? Can you explain?‘). In Appendix G the content of the interview is presented. Throughout the interview participants were encouraged to give their own honest opinion, and to elaborate and motivate their answers. The interviews took between 23 and 53 minutes, with an average duration of 38 minutes.
2.7 Data Analysis
2.7.1 Data analysis additional questions and questionnaires
Descriptive data (mean, standard deviation, and frequencies) were applied to explore the demographic and work characteristics of all participants who filled out the pre-test (n=29). The additional questions during the post-test about usage, appreciation, and self-perceived effect were only analysed on the adherence group (n=11). The non-adherence group (n=5) was excluded for this part, because they did not use and experience the programs themselves. Only by making the account and using the programs participants could be able to form their opinion about these aspects. The t-test and chi-square were performed to test whether there was selective non-adherence. The four questionnaires were analysed on all participants who filled out both pre- and post-test (n=16). The non-parametric Wilcoxon Signed Rank test was performed as assumptions for a paired sample t-test were not met (the n was rather small).
Significance levels were set at p < 0.05.
2.7.2 Data analysis interviews
After the interviews (n=6) were conducted, the recordings were rewound multiple times to transcribe accurately. Both a deductive- and inductive analyses was conducted. Deductive analysis means that predefined categories are used to search relevant fragments (Kenneth, 2000). In this study relevant
frequencies were first coded into one of the predetermined categories: usage, appreciation, effect, and implementation on the work floor. Inductive analyses means that themes are derived from data, instead of predefined categories (Kenneth, 2000). Firstly, all transcripts were read and reread to familiarize with the data. Secondly, relevant fragments were selected by one coder (E.K.). Then, all fragments were deductively coded into one of the four main categories. Fourthly, all fragments within one category were transformed into sub codes and variations using inductive analysis. Subsequently, the codes schedule was discussed with one supervisor (M.S.). The labels for the final codes, sub codes and variations can be found in Appendix H.
Results pre- and post-test study
This section describes the results of the questionnaires, in which participants were asked about their demographics, work characteristics, mental health, perceived stress, work engagement, and self- efficacy. Furthermore, participants were asked how they used and appreciated Gezondeboel, and what effects they experienced by themselves.
3.1 Characteristics of the participants
In total twenty-nine participants filled out the pre-test. They were aged between 22 and 65 years, with an average of 36 years (SD: 13.0). The largest part was female (n=25). Thirteen participants dropped- out, because they did not filled out the post-test. Eleven participants made a Gezondeboel account and worked on the programs of the intervention (adherence group). Three participants did not made an account, and two participants did not use their account (non-adherence group). See Table 1 for the characteristics of each group. Selective failure for the adherence- and non-adherence group was tested on age, gender, and the results of the four questionnaires of the pre-test. It could be concluded there was no selective failure. All participants were higher or scientifically educated, and most of them did not study besides work. The largest number of participants was employed in Utrecht. The sector participants worked in varied.
Table 1.
Demographics and work characteristics of the baseline, drop-out, adherence- and non-adherence group Baseline
(n = 29)
Drop-out (n = 13)
Adherence group (n = 11)
Non-adherence group (n = 5)
Age, M (SD), min-max ᵃ 36 (13), 22-65 37 (14), 22-65 37 (14.0), 23-58 31 (3.0), 26-35
Gender, n (%) ᵇ
Female 25 (86) 11 (85) 10 (91) 4 (80)
Male 4 (14) 2 (15) 1 (9) 1 (20)
Education, n (%)
Higher Vocational Education 16 (55) 8 (62) 6 (55) 2 (40)
Scientific Education or higher 13 (45) 5 (38) 5 (45) 3 (60)
Study work, n (%)
More study than work 1 (3) 1 (8) 0 (0) 0 (0)
More work than study 6 (21) 2 (15) 2 (18) 2 (40)
No study besides work 22 (76) 10 (77) 9 (82) 3 (60)
Province, n (%)
Friesland 2 (7) 1 (8) 0 (0) 1 (20)
Gelderland 1 (3) 0 (0) 1 (9) 0 (0)
Noord-Brabant 1 (3) 1 (8) 0 (0) 0 (0)
Noord-Holland 5 (17) 3 (23) 2 (18) 0 (0)
Overijssel 3 (10) 0 (0) 2 (18) 1 (20)
Utrecht 13 (45) 6 (46) 5 (55) 2 (40)
Zuid-Holland 4 (13) 2 (15) 1 (9) 1 (20)
Work sector, n (%)
Financial services 3 (10) 1 (8) 2 (18) 0 (0)
Information and communication 6 (21) 3 (23) 3 (27) 0 (0)
Education 2 (7) 1 (8) 1 (9) 0 (0)
Public administration 1 (3) 1 (8) 0 (0) 0 (0)
Business services 2 (7) 1 (8) 1 (9) 0 (0)
Care 8 (28) 2 (15) 4 (36) 2 (40)
Different 7 (24) 4 (31) 0 (0) 3 (60)
Number organisations, n (%)
Works for one organisation 25 (86) 12 (92) 9 (82) 4 (80)
Works for two organisations 4 (14) 1 (8) 2 (18) 1 (20)
Number of employees of organisation, n (%)
1 – 30 11 (38) 6 (46) 3 (27) 2 (40)
31 – 50 3 (10) 1 (15) 2 (18) 0 (0)
51 – 100 3 (10) 2 (15) 0 (0) 1 (20)
101 – 200 2 (7) 2 (15) 0 (0) 0 (0)
200 or more 9 (31) 2 (15) 5 (46) 2 (40)
Self-employed without employees
1 (3) 0 (0) 1 (9) 0 (0)
Number of managing employees, n (%)
1 – 5 4 (14) 3 (23) 0 (0) 1 (20)
6 – 15 2 (7) 2 (15) 0 (0) 0 (0)
16 – 30 2 (7) 2 (15) 0 (0) 0 (0)
Managed no employees 21 (72) 6 (46) 11 (100) 4 (80)
Working hours per week, n (%)
0 – 8 1 (3) 0 (0) 1 (9) 0 (0)
9 – 16 1 (3) 1 (8) 0 (0) 0 (0)
17 – 24 2 (7) 0 (0) 0 (0) 2 (40)
25 – 32 11 (38) 4 (31) 5 (45) 2 (40)
33 – 40 14 (48) 8 (62) 5 (45) 1 (20)
Hours overlabour, n (%)
1 – 3 11 (38) 5 (39) 4 (36) 2 (40)
4 – 8 5 (17) 4 (31) 1 (9) 0 (0)
17 – 24 1 (3) 0 (0) 1 (9) 0 (0)
No overlabour 12 (41) 4 (31) 5 (45) 3 (60)
ᵃ t-test ᵇ chi-square
3.2 Usage
For the following paragraphs of this chapter only the adherence group (n=11) is evaluated, since only this group used the programs. The use of Gezondeboel variated between one till six weeks (Table 2).
The amount of logging in variated between one till four times a week. Most participants (n=5) logged in two times a week, as was recommended by the authors of Gezondeboel.
Table 2.
Number of weeks the adherence group (n=11) used Gezondeboel, and how often they logged in Usage Gezondeboel
Weeks Gezondeboel is used, n (%)
1 3 (27)
2 2 (18)
3 2 (18)
4 1 (9)
5 1 (9)
6 2 (18)
Number of logins in per week, n (%)
1x 3 (27)
2x 5 (46)
3x 2 (18)
4x 1 (9)
There was a variety in chosen programs and how far these programs were finished (Table 3).
‘Mindfulness’ was chosen most often. ‘Happiness’ and ‘Balance’ were chosen least, since those were only quickly scanned by two participants. ‘StressLess’ and ‘Well sleeping’ were the only programs who were totally finished.
Table 3.
Overview chosen programs by the adherence group (n=11, ordered from most chosen till least chosen program)
Participants were asked about their program usage. Table 4 displays an overview of the statements they could (dis)agree with. Positive statements were ordered from highest amount of ‘agree’ till least ‘agree’.
Negative statements were ordered from highest amount of ‘disagree’ till least ‘disagree’. In general the adherence group was positive about the investigated aspects regarding the usage of the Gezondeboel programs.
Chosen program, n (%) Content scanned quickly
Made a big start
Finished biggest part
Totally finished
Total chosen
Mindfulness 2 (18) 3 (27) 0 (0) 0 (0) 5
StressLess 1 (9) 2 (18) 0 (0) 1 (9) 4
Well sleeping 0 (0) 1 (9) 1 (9) 1 (9) 3
Resilience 1 (9) 1 (9) 1 (9) 0 (0) 3
Self-compassion 0 (0) 2 (18) 1 (9) 0 (0) 3
Worrying 1 (9) 1 (9) 0 (0) 0 (0) 2
Happiness 1 (9) 0 (0) 0 (0) 0 (0) 1
Balance 1 (9) 0 (0) 0 (0) 0 (0) 1
