Factors associated with the persistence of medically unexplained symptoms in later life
van Driel-de Jong, Dorine
DOI:
10.33612/diss.136429372
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date: 2020
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
van Driel-de Jong, D. (2020). Factors associated with the persistence of medically unexplained symptoms in later life. University of Groningen. https://doi.org/10.33612/diss.136429372
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
Feasibility of an internet-based Cognitive
Behavioural Therapy for older treatment
seeking patients
van Driel, D., van der Vaart, R., te Boekhorst, S.,
Hilderink, P., Oude Voshaar, R., & Rosmalen, J. (2019)
Abstract
Background – Internet-based Cognitive Behavioural Therapy (iCBT) is a promising approach for
the treatment of psychiatric disorders. However, although computer and internet use steadily increases among older persons, it remains unknown whether older treatment seeking patients are motivated for and able to use iCBT .
Research design and Methods – Among older adults (60+) referred for psychological treatment,
reasons for (non)acceptance of iCBT were explored. Among patients who accepted iCBT, digital health literacy was assessed by administering the self-report Digital Health Literacy Instrument (DHLI) and by systematically observing eight performance tasks in an iCBT program.
Results – Out of 52 treatment seeking patients (mean age 72 years), 12 would accept iCBT
and participated in the usability study. The most important reasons for refusing were lack of self-perceived computer skills and no familiarity with computer or internet use at all. The participating patients scored low on the DHLI subscales measuring evaluating reliability, and adding content to the web. The performance tasks revealed problems with logging in and out, sending messages via the mailbox in the program, filling out the registration diary, and completing assignments. The problems were due to a lack of operating skills and navigation skills.
Discussion and Implications – Older patients encounter multiple problems to accept and use
iCBT, particularly due to a self-perceived lack of computer skills among non-users, and to gaps in digital health literacy skills among users. Feasibility of iCBT for this population could be improved by training operational skills and by increasing usability of the iCBT program.
Introduction
Within e-Mental Health, considerable research has been done on Internet-based Cognitive Behavioural Therapy (iCBT) 1. Cognitive Behavioural Therapy (CBT) is very suitable to deliver
via the internet. Interventions based on CBT principles are often well-defined, enabling the presentation of psycho education, homework assignments and diary registrations in a structured online format 2,3. Systematic reviews and meta-analyses have indicated that iCBT
can be as effective as face to face CBT in both psychiatric and somatic disorders 4, 5,.
ICBT offers several potential benefits relative to regular face to face therapy. First, iCBT may be cost-effective by lowering therapist burden. This is important because of the discrepancy between the needs of psychological therapy and the limited supply of therapy in the population 6.
Secondly, accessibility of iCBT is easier for somatically ill patients or persons with mobility problems 7. Thirdly, the anonymity of iCBT may be attractive for patients who feel embarrassed,
stigmatized, or easily judged by others 7. Finally, iCBT may increase self-management and
empowerment, since patients are expected to work on their treatment independently, with therapist support on a distance. This emphasis on self-management is also assumed to increase the involved of patients in their own treatment 8.
Despite the potential internet interventions have, successful use of iCBT is still dependent on both the application and its user. A first prerequisite for the user is internet access. In the Netherlands, 88.3% of the population above the age of 65 has internet-access and 77.8% of them also use the internet sometimes or regularly 9. Secondly, to minimize the need for
assistance, it is essential that the application is easy to use 10. Simultaneously, possible users
should have sufficient digital health literacy skills, defined as the ability to seek, find, understand, and appraise health information from electronic sources and to apply the knowledge gained to address or solve a health problem 11. Higher self-perceived digital health literacy is associated
with better performance of internet use for health related purposes 12.
Whereas the potential benefits of iCBT are particularly relevant for older persons (mobility, stigmatization), older patients (>60) have not been included in most previous large effect studies 2, 3. Thus far, only a few and merely small clinical trials have been conducted among
older patients 13-19. Although these studies have suggested positive effects of iCBT, interpretation
is hampered by defining “old” as 50 years or over 17 or including a selected group of older patients experienced in the use of a computer and Internet 14, 16.
Whether internet-based therapy is feasible for older treatment seeking patients in general therefore remains unknown. Prior research has indicated that increasing age negatively interferes with iCBT use, due to age-related sensory disability or cognitive impairment 20, 21
as well as a lack of awareness that Information and Communication Technologies (ICT) in general may improve quality of life 22.
The objective of this study was to assess the feasibility of using iCBT in the population of older treatment seeking patients. The first aim was to examine if older treatment seeking patients would potentially agree to be treated with iCBT, and were willing to participate in a usability study. The second aim was to examine the digital health literacy skills among the older patients that would agree to participate in an iCBT program, using a self-report questionnaire and an actual performance test.
Methods
Design and patients
The feasibility assessment was performed among patients referred to two different mental health care institutions. The first institution was specialized in treatment of older patients with affective disorders and Medically Unexplained Symptoms (MUS) and was located near the Eastern border. The second institution was specialized in treating patients with fibromyalgia and was located near the Western border of the Netherlands. Patients were included between June and December 2017 and had already completed the diagnostic and assessment phase. All patients aged 60 years or above, who suffered from a depressive, anxiety or somatic symptom disorder, were asked: 1) whether they used the internet at home, and 2) whether they would potentially agree to be treated with iCBT, and if so, 3) to participate in an iCBT usability study concerning digital health literacy. Exclusion criteria for the study were 1) cognitive impairment, 2) a (known) history of mental retardation (IQ< 70), and 3) a psychiatric disorder other than a depressive, anxiety or somatic symptom disorder. Cognitive impairment was defined as an established diagnosis of dementia or a score below 23 points on the Mini Mental State Examination (MMSE) 23. In addition to the in- and exclusion criteria, we assessed age, sex,
and level of education.
When patients potentially agreed to be treated with iCBT and were willing to participate in the usability study, patients received further written information about the goal of the study, its procedure, location, time investment, compensation, and their rights during participation. Participants were invited at a specified date and time and were asked to sign an informed consent form before the start of the study. To execute the usability study, participants visited the outpatient clinic once. First, they were asked to fill in a self-report questionnaire on digital health literacy 24 and subsequently they were asked to take a seat behind a PC (using Windows
10) and conduct eight performance tasks in the iCBT program ‘Master Your Pain’ 25 (see
measurements). During the tasks, participants were asked to state their thoughts by ‘thinking aloud’ 26. This is a verbal report method which provides insight in understanding a person’s
decision making and problem solving.
Measurements
Digital Health Literacy Instrument (DHLI)
with good internal consistency and validity 24. The DHLI has 21 items to be rated on a four-point
response scale with response options ranging from ‘very easy’ to ‘very difficult’ and from ‘never’ to ‘often’. Scores are reversed, so that higher scores represent a higher level of digital health literacy. The seven skills (subscales) are 1) operational skills, 2) navigation skills, 3) searching information, 4) evaluating reliability, 5) determining relevance, 6) adding content to the web, 7) protecting privacy. The 3 items on the subscale protecting privacy were excluded, because respondents did not have any experience with posting messages on social media or other communication portals.
Performance tasks
To study performance when using an iCBT program, participants were systematically observed when performing eight tasks within the iCBT ‘Master Your Pain’ application 25. Master Your
Pain is an internet-based treatment program for patients suffering from chronic pain. The eight performance tasks were 1) log in to the treatment website, 2) navigate to a specific assignment and back to the home page, 3) send a message to the therapist via the mailbox in the program, 4) search a specific assignment, 5) complete a specific assignment, 6) read a specific psycho- educational text and recall the core message, 7) fill in a registration diary, and finally 8) log out from the treatment website. Based on these eight performance tasks, the seven categories of digital health literacy skills were rated 12. Each performance task required at least one digital
health literacy skill; some tasks required multiple skills. Except for performance task 1, 2, and 8 (starting and logging out of the program), performance tasks were offered in a random order to avoid learning effects.
After instruction by the researcher, participants had to perform each task while talking aloud about the steps they were taking, following the thinking out loud method 26. A freely available
screen activity program (Flashback Express) recorded the online performance test. The video files with screen activity and webcam recordings were used to rate the level of completion of a task, the level of performance of a task, and the specific problems encountered within each task. The level of completion was categorized as completed independently (all steps finished correctly), completed with help (researcher gave a hint which contributed to the completion of the task), or not completed (not all steps were finished correctly, irrespective of help). The level of performance was categorized as good (task completed with minimal mistakes and without help), reasonable (task completed with help or with a number of problems, but in an acceptable manner, e.g., sufficient understanding of the task) or poor (task not finished or insufficiently, e.g., poor understanding of the task). The encountered problems were marked and when there was overlap between them, two researchers (DvD, StB) merged the problems. The ratings of the performance tasks were independently coded by these two researchers using inductive analysis 27. Inter-rater agreement was calculated with Cohen’s kappa and was .87. Disagreements
were resolved by discussion and in case no consensus could be reached, a third researcher (RV) took the final decision.
Statistical analysis
potentially agree to be treated with iCBT and were willing to participate in the usability study was reported. Differences on age, sex and educational level between participants and non-participants were compared by Students T-tests (normality was tested with a Shapiro-Wilk test considering the small sample size) and chi-square tests.
Scores on the DHLI and the rating of the seven performance tasks were expressed as means (with standard deviations) or proportions. All analyses were performed using SPSS (IBM SPSS Statistics 23).
Results
Acceptation of iCBT in older treatment seeking patients and willingness to participate in a usability study.
Table 1 lists the characteristics of the invited eligible patients (N=52), stratified by participation in the usability study. Of these 52 eligible patients, 24/52 (46.1%) said that they used the internet regularly. Most older treatment seeking patients (40/52, 76.9%) reported not to (potentially) accept internet-based therapy, and were not willing to participate the usability study. Self-perceived lack of computer skills was the most important reason why patients would not participate (23/40, 57.5%). Another 8 patients (20.0%) indicated to have sufficient computer skills, but were not motivated to participate in the usability study; 2 of these patients would potentially accept iCBT. Finally, 9/40 (22.5%) would not participate because of mental or physical impairment, i.e. low vision: n=1 (2.5%), cognitive impairment: n=3 (7.5%), mental retardation: n=2 (5.0%), physical illness: n=3 (7.5%). As shown in Table 1, the older treatment seeking patients who (potentially) agreed to be treated with iCBT, and were willing to participate in the usability study, were significantly younger compared to non-participants, but did not differ significantly with respect to sex or educational level.
Table 1
Socio-demographic characteristics of (non) participants in the usability study
Participants Non-participants Statistics
Characteristics (n=12) (n=40)
• Female sex, n (%) 8 (66.7) 31 (77.5) Chi2=.58, df=1,
p=.45
• Mean (SD) age (years) 66.7 (4.8) 73.6 (6.6) t=-4.01, df=50,
p<.001
• Higher education, n (%) 6 (11.5) 3 (7.5) Chi2=2.8, df=1,
p=.096
Self-reported digital health literacy
Table 2 presents the scores on the DHLI. As 10 out of 12 participants lack experience in public forums or social media, no mean score is presented for the skill ‘protecting privacy’. The level of digital health literacy was lower (mean 2.6; SD 0.8) compared to the general population (mean 3.1; SD 0.5) 24.
Table 2
Self-reported digital health literacy skills (n=12)
Mean (SD)
Specific skill Study population Validation sample
(n=12) (n=200)2
• Total digital health literacy 2.6 (0.8) 3.1 (0.5)
• Operational skills 2.9 (1.0) 3.7 (0.6) • Navigation skills 2.9 (0.8) 3.3 (0.5) • Searching information 2.7 (0.7) 3.0 (0.6) • Evaluating reliability 2.2 (0.7) 2.7 (0.6) • Determining relevance 2.5 (0.8) 2.8 (0.6) • Adding content 2.3 (0.8) 3.0 (0.7)
• Protecting privacy n.a.1 3.5 (0.5)
1 10/12 persons had no experience with items concerning privacy
2 Van der Vaart, et al. (2017), including 41/200 (20.5%) persons aged 65 years and over.
Performance tasks
Table 3 lists an overview of the level of completion and performance for each performance task and the number of problems per performance task.
The performance task ‘fill in a registration diary’ was most difficult. Only one person completed this task without help and with good performance; 33% could not complete the task and 73% performed badly. The performance task ‘send a message to the therapist via the mailbox’ could not be performed without assistance by three participants, and was completed by 64% of the participants. The performance tasks ‘log in to the treatment website’ and ‘log out from the treatment website’ could not be completed without help by two people. Almost half of the participants completed the task without help, but the performance was poor for more than 50%. The performance task ‘complete an internet-based assignment’ could not be completed by three persons. On the other hand, the performance was good or reasonable for eight persons. Less difficulties, i.e., completion without help and with good performance, were found in the performance tasks ‘navigate to a specific assignment and back to the homepage’, ‘search for a specific assignment’, ‘, and ‘read a specific psycho-educational text and recall the core message’. When looking at the number of problems per performance task, log in to the treatment website (mean 2.4), send a message to the therapist via the mailbox at the website (mean 2.5), fill in a
Table 3 Completion, performance and number of pr
oblems for each performance task (n, %)
Performance tasks 1 1 2 3 4 5 6 7 8 Number of participants n=12 n=11 n=11 n=11 n=11 n=11 n=11 n=11 Level of completion • W ithout help 4 (33.3) 6 (54.5) 4 (36.4) 8 (72.7) 5 (45.5) 8 (72.7) 1 (9.1) 5 (45.5) • W ith help 6 (50.0) 4 (26.4) 4 (36.4) 3 (27.3) 3 (27.3) 2 (18.2) 6 (54.5) 4 (36.4) • Not completed 2 (16.7) 1 (9.1) 3 (27.3) 0 (0) 3 (27.3) 1 (9.1) 4 (36.4) 2 (18.2) Performance • Good 2 (16.7) 2 (18.2) 3 (27.3) 2 (18.2) 5 (45.5) 6 (54.5) 1 (9.1) 1 (9.1) • Reasonable 3 (25.0) 5 (45.5) 4 (36.4) 3 (27.3) 3 (27.3) 2 (18.2) 2 (18.2) 4 (36.4) • Poor 7 (58.3) 4 (36.4) 4 (36.4) 6 (54.5) 3 (27.3) 3 (27.3) 8 (72.7) 6 (54.5) Specific problems 26 19 27 0 4 4 18 21 • Mean (SD) 2.4 (1.75) 1.7 (1.0) 2.5 (1.7) 0 (0) .36 (.92) .36 (.50) 1.6 (.67) 1.9 (.30) • Minimum-maximum 0-6 0-3 0-3 0 0-3 0-4 0-2 0-1 1 1) log in to the tr
eatment website; 2) navigate to a specific assignment and back to the homepage; 3) send a message to the therapist via the mai
box at the website;
4) sear
ch for a specific assignment: 5) complete an inter
net-based assignment; 6) r
ead a specific psycho-educational text and r
ecall the cor
e message; 7) fill in a
registration diary; 8) log out fr
om the tr
registration diary (mean 1.6), and log out from the treatment website (mean 1.9), caused the most problems.
Table 4 provides an overview of the number of participants that had problems per performance task and the underlying skills that were insufficient. Often, problems in performance tasks were due to a lack of multiple underlying skills. Some problems were more general (for example, operating the computer and internet browser), while others were more specific to the iCBT (for example, navigating and orientating skills). The basic skill of logging in to the treatment website was problematic for five participants (46%). The problems that occurred regarding the basic skills were: operating the computer and internet browser, in particular limited knowledge of the keyboard, inadequate slider usage, and problems with the monitor. Other observed problems were more related to the specific iCBT program. Problems with navigating and orientating skills, in particular inadequate usage homepage button in website, and problems finding the new message button occurred in 3/8 performance tasks. Seven participants (64%) had problems to send a message via the mailbox at the website. Eight participants had problems to fill in a registration diary (73%). They had problems with locating and proper use of buttons on the platform, selecting the correct assignment, and correct application of information. The problems that seven patients (64%) experienced to complete an internet based assignment were related to correct application of the information and not using relevant fields in program. All participants (n=11) had problems to log out from the treatment website. None of the participants was able to log out from the treatment website adequately. None of the participants had problems with searching for a specific assignment, or with reading a specific psycho-educational text and recalling the core message.
Table 4
Frequency of problems per performance task split by specific skills required
Performance tasks1 Skills 1 2 3 4 5 6 7 8 • Operational skills 5 1 1 • Navigation skills 6 7 8 • Searching information 2 4 • Evaluating reliability2 -• Determining relevance 3 1 1 1 1 • Adding content 5 7 • Protecting privacy 11
1 1) log in to the treatment website; 2) navigate to a specific assignment and back to the homepage; 3) send
a message to the therapist via the mailbox in the website; 4) search for a specific assignment: 5) complete an internet-based assignment; 6) read a specific psycho-educational text and recall the core message; 7) fill in a registration diary; 8) log out from the treatment website.
2 Reliability is not registrated because the content of the program ’Master Your pain’ is provided by Leiden
Discussion
The objective of this study was to assess the feasibility of using iCBT in the population of older treatment seeking patients. In contrast to rising figures of older persons using the computer and internet, only a quarter (12/52, 23.1%) of the older treatment seeking psychiatric patients in the outpatient treatment centre would potentially participate in an internet-delivered CBT and was willing to participate in a usability study. Moreover, our findings showed that those willing to accept an Internet-delivered treatment program were more or less hampered by a lack of digital skills. Some problems were due to a lack of general operating skills as using the keyboard and limited knowledge of tabs. Other problems were due to a lack of skills more related to the specific iCBT program. These problems would have prevented the older patients to fully benefit from internet-based treatment in its current form.
Willingness to participate
Out of all patients who were not willing to participate, for a minority this was due to age-related physical restraints, such as visual or cognitive impairments or physical disabilities. The most important reason not to participate (half of the eligible patients), however, was a lack of computer skills or no usage of the internet at all. The percentage of internet-use (54%) was lower in our sample than the percentage of internet-using older adults (65+) in the general population the Netherlands. These findings could suggest that older psychiatric patients are over represented in the minority group of non-internet users. This corresponds with studies of Heart & Kalderon 22 and Gell, et al.28 who concluded that health status is a strong
moderating factor affecting internet-use. Therefore, unhealthy older adults may be less inclined to adopt ICT to improve their quality of life. This is partly due to perceived behavioural control as a result of physical or cognitive problems, which is claimed as the main reason 22.
Digital competences
Older treatment seeking patients with some computer and internet skills, who were willing to participate, self-reported significant difficulties in evaluating reliability, and adding content to the web. For most performance tasks, help was needed and performance was poor. The problems were due to a lack of operating skills and navigating and orientation skills.
To the best of our knowledge, this is the first study that investigated the feasibility of internet-based therapy for older adults referred for specialized mental health care. The conclusions of our study are more cautious than the positive conclusions of other internet-delivered treatment studies for older patients 17, 18, 29. These studies concluded that internet-based treatment is effective and acceptable
for older adults. The main explanation is probably the fact that previous treatment studies recruited highly selective patient populations, specifically those with good internet skills. A second explanation for this discrepancy is that the dropout rate for all effect studies was relatively high. It is possible that older patients with low digital health literacy skills belonged to the dropout group and were not included in ratings regarding usability. Finally, it is unknown to what degree the older patients were helped using the iCBT program in these studies.
A lack of digital health skills might prevent older treatment seeking patients to benefit from iCBT in its current form. There are different solutions to solve the observed problems. For participants with operating problems, an introduction to the platform, together with the therapist of other support staff, could probably improve operational skills. Showing patients how to log in and log out, where to find all the specific parts of the treatment, and how the platform is structured, could help them in getting started 30. Furthermore, a combination of
internet treatment and face to face contact 31 for a telephone call, and replacing written text by
videos, pictures and audiotapes might help older patients with limited more complex digital health skills 32.
Finally a senior-friendly design is a possibility to minimize the generation gap by increasing access to internet-based treatments for older treatment seeking patients. For example, changes in the interface and navigation to the assignments can solve age-related problems as low vision, cognitive decline and physical inability, thereby improving the usability of the iCBT for older treatment seeking patients. Intergenerational educational settings with older and younger participants may help designing a senior-friendly iCBT 32.
Methodological issues
The findings in this study need to be interpreted in the context of some limitations. Firstly, the sample size is small. Moreover, one person of the twelve participants could not fulfil all assignments. Secondly, we recruited patients to participate in the usability study, but not with the intention to include them in an actual internet-based treatment. This might explain motivation problems of non-participants and may have led to underperformance of partici-pants. On the other hand, those accepting the usability study were potentially motivated for internet-delivered treatment, but some of them would probably refuse when they would have received an internet-delivered treatment instead of face-to-face treatment.
The strength of this study is that we included older adults with a mean age of 72 years old, while previous studies among older patients included patients from a much lower age group 17.
In addition, our sample consisted of patients with high care needs who were referred to specialized mental health care. This contrasts with most studies that have recruited participants through the community and via mass media 3, 15 in whom symptom severity is probably less
as these persons did not need specialized mental health care. Moreover, individuals who are responding to community recruitment are in general familiar with the internet and probably very motivated to participate.
Conclusion
Taken together, the findings of current study suggest that older patients seeking psychological treatment experience serious obstacles to benefit from the advantages of internet-based interventions in its current form. They encounter multiple problems to accept and use iCBT, particularly due to a self-perceived lack of computer skills among non-users, and to gaps in digital health literacy skills among users. Feasibility of iCBT for this population could be improved by training operational skills and by increasing usability of the iCBT program.
References
1. Anderson, G. (2009). Using the internet to provide cognitive behaviour therapy. Behaviour Research
and Therapy, 47, 3, 175-180.
2. Anderson, D. (2016). Internet-delivered psychological treatments. Annual Review of Clinical
Psychology, 12, 157-179.
3. Cuijpers, P., van Straten & Andersson, G. (2008). Internet-administrated cognitive behaviour therapy for health problems: a systematic review. Journal of Behavioral Medicine, 31, 2, 169-177.
4. Anderson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014). Guided Internet-based vs. face to face cognitive therapy for psychiatric and somatic disorders: A systematic review and meta-
analysis. World Psychiatry, 133, 288-295.
5. Van Beugen, S., Ferwerda, M., Hoeve, D., Rovers, M., Spillekom-van Koulil, S., van Middendorp, H., & Evers, A. (2014). Internet-based cognitive behavioural therapy for patients with chronic somatic conditions: A meta-analytic review. Journal of Medical Internet Research, 16, 3, e88.
6. Lal, S., & Adair, C. (2014). E-mental health: A rapid review of the literature. Psychiatric services, 65,
1, 24-32.
7. Griffiths, F., Lindenmeyer, A., Powell, J., Lowe, P. & Thorogood, M. (2006). Why are health care interventions delivered over the internet? A systematic review of the published literature. Journal of
Medical Internet Research, 8, 2, e10.
8. Putters, K., Janssen, M., van der Wel, T. & Kelder, M. (2012). E-health: Face to Facebook over e-health
en zelfredzaamheid van patienten in de GGZ. Rotterdam: Erasmus Universiteit.
9. Statline.cbs.nl/StatWeb/publication/?VW=T&DM=SLNL&PA=83429NED, march 2017.
10. Jaspers, M. (2008). A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. International Journal of Medical Informatics, 78, 340-353.
11. Norman, C. & Skinner, H. (2006). E Health literacy: Essential skills for consumer health in a networked
world. Journal of Medical Internet Research, 8, 2, e9.
12. Van der Vaart, R., Drossaert, C., de Heus, H., Taal, E., & van de Laar, M. (2013). Measuring actual eHealth literacy among patients with rheumatic diseases: a qualitative analysis of problems encountered using Health 1.0 and Health 2.0 applications. Journal of Medical Internet Research, 15, 2, 227. 13. Dear, B., Zou, J., Titiov, N., Lorian, C., Johnston, L., Spence, J.,…& Knight, R. (2013). Internet-delivered
cognitive behavioural therapy for depression: A feasibility open trial for older adults. Australian &
New Zealand Journal of psychiatry, 47, 169-176.
14. Dear, B., Zou, J., Ali., S., Lorian, C., Johnston, L., Sheehan, J.,…& Titov, N. (2015). Clinical and cost-effectiveness of therapist guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety: A randomized controlled trial. Behavior Therapy, 46, 206-217.
15. Silfvernagel, K., Westlinder, A., Anderson, S., Bergman, K., Hernandez, R., Fallhagen, L.,… & Andersson, G. (2017). Individually tailored internet-based cognitive behaviour therapy for older adults with anxiety and depression: a randomised controlled trial. Cognitive behaviour therapy, 1-12. 16. Spek, V., Cuijpers, P., Nyklicek, I., Smits, N., Riper, H., Keyzer., J., & Pop, V. (2008). One-year follow-up results of a randomised controlled clinical trial on internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years. Psychological medicine, 38, 635-639.
17. Spek, V., Nyklicek, I., Smits, N., Cuijpers, O., Riper, H., Keyzer, J., & Pop, V. (2007). Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: A randomized controlled clinical trial. Psychological medicine, 37, 1797-1806.
18. Titov, N., Dear, B., Ali, S., Zou, J., Lorian, C., Johnston, L.,…..& Fogliati (2015). Clinical and cost- effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: A randomized controlled trial. Behavior Therapy, 46, 193-205.
19. Zou, J., Dear, B., Titov, N., Lorian, C., Johnston, L., Spence, J.,…& Sachdev, P. (2012). Brief internet- delivered cognitive behavioural therapy for anxiety in older adults. A feasibility trial. Journal of
Anxiety Disorders, 26, 650-655.
20. Cornett, S. (2006). The effects of aging on health literacy.
Retrieved from http://medicine.osu.edu/sitetool/sites/pdhfs/ahecpublic/HL_Module_Elderly.pdf 21. Melenhorst, A., Rogers, W. & Bouwhuis, D. (2006). Older Adults’ Motivated Choice for Technological Innovation: Evidence for Benefit-Driven Selectivity. Psychology & Aging, 21, 190-195.
22. Heart, T. & Kalderon, E. (2013). Older adults: Are they ready to adopt health-related ICT? International
journal, of medical informatics, 82, 209-231.
23. Folstein, M., Folstein, S., & Mc Hugh, P. (1975). “Mini –mental state”. A practical method for grading the cognitive state of patients for the clinician, Journal of psychiatric research, 12, 3, 189-198. 24. Van der Vaart, R. & Drossaert, C. (2017). Development of the Digital Health Literacy Instrument: Measuring a Broad Spectrum of Health 1.0 and Health 2.0 Skills. Journal of Medical Internet Research,
19, 1.
25. Terpstra, J., Van der Vaart, R., Spillekom-van Koulil, S., van Dam, A., Rosmalen, J., Knoop, H., … & Evers, A. (2018). Becoming an e Coach: Training therapists in online cognitive-behavioral therapy for chronic pain. Patient Educ Couns., 101, 9, 1702-1707.
26. Ericsson, K. & Simon, H. (1980). Verbal reports as data. Psychological Review, 87, 3, 215-251. 27. Thomas, D., (2006). A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation, 27, 2, 237-246.
28. Gell, N., Rosenburg, D., Demiris, G., LaCroix, A., & Paterl, K. (2013). Patterns of technology use among older adults with and without disabilities. Gerontologist, 55, 33, 412-421.
29. Mewton, L., Sachdev, P., & Andrews (2013). A naturalistic study of the acceptability and effectiveness of internet-delivered cognitive behavioural therapy for psychiatric disorders in older Australians. PLoS One, 8, e71825.
30. Van der Vaart, R., van Driel, D., Pronk, K., Paulussen, S., te Boekhorst, S., Rosmalen, J. & Evers, A. (2019). The role of age, education, and digital health literacy in usability of Internet-based cognitive behavioural therapy for chronic pain: a mixed-methods sstudy. JMIR, Form Res, 3, 4: e12883.
31. Wentzel, J., Van der Vaart, R., Bohlmeijer, E., & Van Gemert,-Pijnen, J. (2016). Mixing online and Face to Face therapy: How to benefit from blended care in mental health care. Journal of Medical
Internet Research Mental Health, 3, 1, e9.
32. Xie, B. (2011). Effects of an eHealth literacy intervention for older adults Journal of Medical Internet
