5.4 Implementations & recommendations
5.4.5 Ethical concerns: disclaimers & data
Last but not least, chatbots that cater to emotional needs also pose ethical challenges. A paper by Kretzschmar et al. (2019) addresses some of the ethical concerns when it comes to designing chatbot therapists: for example, what happens when the treatment actually hurts the user?
What information about the user is stored, and who can access what parts of that data? These questions are especially relevant when chatbots are implemented on third-party platforms with their own policies on how to store and potentially share your data, such as Facebook.
A proper discussion of all ethical concerns surrounding therapeutic chatbots is beyond the scope of this work, but it does warrant caution in developing chatbots that work with sensitive information. Practically, Kretzschmar et al. (2019) suggest standards regarding three important ethical aspects of mental health chatbots, which we will discuss briefly here with regards to our study. First, the users’ privacy should be ensured: data should be kept confidential, if stored at all, and users need to be made aware of what happens to their data and who can see what.
Our study included a detailed informed consent form that required approval before participants could continue with the survey, but since these forms contain a lot of text, it is recommended to include information about data security and privacy in the actual chatbot conversation as well (Kretzschmar et al., 2019).
Second, users should be kept safe: they need to be reminded that a bot is no replacement of a human therapist and there should be systems in place to deal with emergency situations.
Although Vincent never pretended to be human, he did not have safety nets in place in case the user felt unsafe or otherwise negatively impacted: this impact was deemed unlikely to result from our experiment. However, as one participant pointed out to us, revisiting a moment when you felt bad about yourself may be potentially damaging for certain users, for example those who suffer from PTSD. Hence, we follow Kretzschmar et al. (2019)’s recommendation and strongly suggest to include a disclaimer about potentially triggering content - either in the introduction of the chatbot, or as part of the initial messages in the conversation.
Last, bots should have evidence to back up their efficacy and inform their users of what they can expect by interacting with the bot (Kretzschmar et al., 2019). Such a disclaimer was not included in our experiment, but our results provide exactly that evidence that chatbots for self-compassion can refer back to as soon as they are deployed in the real world.
6 Conclusion
This work set out to discover what effect a single, short intervention, delivered by a chatbot that either gives or asks for help, has on self-compassion. We expected both the caregiving and care-receiving bot to work, with the care-receiving one outperforming the caregiving one. To test this, we let participants chat with a version of Vincent that gave care, asked for care, or simply proceeded to chat about something else.
The first, main takeaway from this work is that all three conditions worked equally well, against what was expected. We suggest that the attachment to the chatbot plays a crucial role in whether or not these conditions can have an effect. Our participants did not get to know Vincent well enough to care - neither about his advice or his struggles: he remained a stranger. As a result, we saw that our conditions had no effect on whether or not self-compassion improved.
However, the second takeaway is that chatbots can effectively be deployed to stimulate self-compassion and support mental health: Vincent successfully improved self-self-compassion in our participants after one short (10 min) interaction, with an effect size of Cohen’s dz= 0.22, coming close to the effect of chatting with Vincent for two weeks of dz = 0.35 (Lee et al., 2019). We argue that this is a result of the fact that Vincent remained a stranger on a train (Rubin, 1975) to participants, thereby forming a safe space for them to vent their problems.
Last but not least, this work provides strong, qualitative insights regarding the levels of anthropomorphism needed in text-based chatbots for emotional needs. We find that these bots do not have to pretend to be human in order to be effective. In fact, as long as they talk in a human-like way, for example by expressing their feelings, they can be - and should be - transparent about their chatbot nature: people do not have to believe that they really have emotions, they only have to accept that they show them. We suggest future research to investigate the types of relationships that can form between people and chatbots without aiming for the same sort of relationships that form between humans: after all, bots are not human and we might befriend them in a very different way.
References
Abu Shawar, B. & Atwell, E. (2007). Chatbots: are they really useful? LDV-Forum: Zeitschrift f¨ur Computerlinguistik und Sprachtechnologie, 22 (1), 29–49. doi:10.1.1.106.1099
Arditte, K. A., C¸ek, D., Shaw, A. M. & Timpano, K. R. (2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological Assessment, 28 (6), 684–
691. doi:10.1037/pas0000217
Bonaccio, S. & Dalal, R. S. (2006). Advice taking and decision-making: An integrative literat-ure review, and implications for the organizational sciences. Organizational Behavior and Human Decision Processes, 101 (2), 127–151. doi:10.1016/j.obhdp.2006.07.001
Brandtzaeg, P. B. & Følstad, A. (2017). Why People Use Chatbots. In I. Kompatsiaris, J. Cave, A. Satsiou, G. Carle & A. Passani (Eds.), Internet science. insci 2017 (Vol. 342, pp. 377–
392). Thessaloniki, Greece. doi:10.1007/978-3-319-70284-1 30
Brave, S., Nass, C. & Hutchinson, K. (2005). Computers that care: Investigating the effects of orientation of emotion exhibited by an embodied computer agent. International Journal of Human Computer Studies, 62 (2), 161–178. doi:10.1016/j.ijhcs.2004.11.002
Breines, J. G. & Chen, S. (2013). Activating the inner caregiver: The role of support-giving schemas in increasing state self-compassion. Journal of Experimental Social Psychology, 49(1), 58–64. doi:10.1016/j.jesp.2012.07.015
Calvo, R. A. & Peters, D. (2014). Positive Computing: Technology for Well-Bein and Human Potential. Cambridge, MA: The MIT Press.
Ciechanowski, L., Przegalinska, A., Magnuski, M. & Gloor, P. (2019). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems, 92, 539–548. doi:10.1016/j.future.2018.01.055
Corrigan, P. (2004). How stigma interferes with mental health care. American Psychologist, 59(7), 614–625. doi:10.1037/0003-066X.59.7.614
Dale, R. (2016). The return of the chatbots. Natural Language Engineering, 22 (5), 811–817.
doi:10.1017/S1351324916000243. arXiv: 0910.0834 [hep-th]
De Angeli, A. & Carpenter, R. (2006). Stupid computer! Abuse and social identities. Abuse:
The darker side of Human-Computer Interaction, 19. Retrieved from http : / / scholar . google . com / scholar ? hl = en % 7B % 5C & %7DbtnG = Search % 7B % 5C & %7Dq = intitle : Stupid+computer!+Abuse+and+social+identities%7B%5C#%7D0
de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A., McKnight, P. E., Krueger, F.
& Parasuraman, R. (2016). Almost human: Anthropomorphism increases trust resilience in cognitive agents. Journal of Experimental Psychology: Applied, 22 (3), 331–349. doi:10.
1037/xap0000092
Donovan, E., Rodgers, R. F., Cousineau, T. M., McGowan, K. M., Luk, S., Yates, K. & Franko, D. L. (2016). Brief report: Feasbility of a mindfulness and self-compassion based mobile in-tervention for adolescents. Journal of Adolescence, 53, 217–221. doi:10.1016/j.adolescence.
2016.09.009
Falconer, C. J., Rovira, A., King, J. A., Gilbert, P., Antley, A., Fearon, P., . . . Brewin, C. R.
(2016). Embodying self-compassion within virtual reality and its effects on patients with depression. British Journal of Psychiatry Open, 2 (1), 74–80. doi:10.1192/bjpo.bp.115.
002147
Falconer, C. J., Slater, M., Rovira, A., King, J. A., Gilbert, P., Antley, A. & Brewin, C. R. (2014).
Embodying compassion: A virtual reality paradigm for overcoming excessive self-criticism.
PLoS ONE, 9 (11). doi:10.1371/journal.pone.0111933
Finlay-Jones, A., Kane, R. & Rees, C. (2017). Self-Compassion Online: A Pilot Study of an Internet-Based Self-Compassion Cultivation Program for Psychology Trainees. Journal of Clinical Psychology, 73 (7), 797–816. doi:10.1002/jclp.22375
Fitzpatrick, K. K., Darcy, A. & Vierhile, M. (2017). Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health, 4(2), e19. doi:10.2196/mental.7785
Følstad, A. & Brandtzæg, P. B. (2017). Chatbots and the new world of HCI. Interactions, 24 (4), 38–42. doi:10.1145/3085558
Fulmer, R., Joerin, A., Gentile, B., Lakerink, L. & Rauws, M. (2018). Using Psychological Arti-ficial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety; A Randomized Controlled Trial. JMIR Mental Health, 5 (4). doi:10.2196/mental.9782
Go, E. & Sundar, S. S. (2019). Humanizing chatbots: The effects of visual, identity and con-versational cues on humanness perceptions. Computers in Human Behavior, 97 (January), 304–316. doi:10.1016/j.chb.2019.01.020
Grudin, J. & Jacques, R. (2019). Chatbots, Humbots, and the Quest for Artificial General Intelligence. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, 1–11. doi:10.1145/3290605.3300439
Hill, J., Randolph Ford, W. & Farreras, I. G. (2015). Real conversations with artificial intel-ligence: A comparison between human-human online conversations and human-chatbot conversations. Computers in Human Behavior, 49, 245–250. doi:10.1016/j.chb.2015.02.026 Kim, Y. & Sundar, S. S. (2012). Anthropomorphism of computers: Is it mindful or mindless?
Computers in Human Behavior, 28 (1), 241–250. doi:10.1016/j.chb.2011.09.006
Kirby, J. N. (2016). Compassion interventions: The programmes, the evidence, and implications for research and practice. Psychology and Psychotherapy: Theory, Research and Practice, 90(3), 432–455. doi:10.1111/papt.12104
Kontogiorgos, D., Pereira, A., Koivisto, M. & Rabal, E. G. (2019). The Effects of Anthropo-morphism and Non-verbal Social Behaviour in Virtual Assistants. In Acm int’l conference on intelligent virtual agents (pp. 133–140). Paris, France.
Kretzschmar, K., Tyroll, H., Pavarini, G., Manzini, A. & Singh, I. (2019). Can Your Phone Be Your Therapist? Young People’s Ethical Perspectives on the Use of Fully Automated Con-versational Agents (Chatbots) in Mental Health Support. Biomedical Informatics Insights, 11, 117822261982908. doi:10.1177/1178222619829083
Krieger, T., Martig, D. S., van den Brink, E. & Berger, T. (2016). Working on self-compassion online: A proof of concept and feasibility study. Internet Interventions, 6, 64–70. doi:10.
1016/j.invent.2016.10.001
Kroenke, K., Spitzer, R. & Williams, W. (2001). The Patient Health Questionnaire PHQ-9:
Validity of a brief depression severity measure. Jgim, 16, 2–3. doi:10 . 1097 / 01 . MLR . 0000093487.78664.3C. arXiv: arXiv:1011.1669v3
Lakens, D. (2017). Equivalence Tests. Social Psychological and Personality Science, 8 (4), 355–
362. doi:10.1177/1948550617697177
Lakens, D., Scheel, A. M. & Isager, P. M. (2018). Equivalence Testing for Psychological Research:
A Tutorial. Advances in Methods and Practices in Psychological Science, 1 (2), 259–269.
doi:10.1177/2515245918770963
Leary, M. R., Tate, E. B., Adams, C. E., Allen, A. B. & Hancock, J. (2007). Self-compassion and reactions to unpleasant self-relevant events: The implications of treating oneself kindly.
Journal of Personality and Social Psychology, 92 (5), 887–904. doi:10.1037/0022-3514.92.
5.887
Lee, M., Ackermans, S., van As, N., Chang, H., Lucas, E. & IJsselsteijn, W. (2019). Caring for Vincent. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, 1–13. doi:10.1145/3290605.3300932
Lucas, G. M., Gratch, J., King, A. & Morency, L. P. (2014). It’s only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior, 37, 94–100. doi:10.1016/
j.chb.2014.04.043
Ly, K. H., Ly, A. M. & Andersson, G. (2017). A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods. Internet Interventions, 10(October), 39–46. doi:10.1016/j.invent.2017.10.002
MacBeth, A. & Gumley, A. (2012). Exploring compassion: A meta-analysis of the association between self-compassion and psychopathology. Clinical Psychology Review, 32 (6), 545–
552. doi:10.1016/j.cpr.2012.06.003
Mori, M., MacDorman, K. F. & Kageki, N. (2012). The uncanny valley. IEEE Robotics and Automation Magazine, 19 (2), 98–100. doi:10.1109/MRA.2012.2192811
Muresan, A. & Pohl, H. (2019). Chats with Bots: Balancing Imitation and Engagement, 1–6.
doi:10.1145/3290607.3313084
Nass, C. & Moon, Y. (2000). Machines and Mindlessness: Social Responses to Computers.
Journal of Social Issues, 56 (1), 81–103. doi:10.1111/0022-4537.00153
Necka, E. A., Cacioppo, S., Norman, G. J. & Cacioppo, J. T. (2016). Measuring the prevalence of problematic respondent behaviors among MTurk, campus, and community participants.
PLoS ONE, 11 (6), 1–19. doi:10.1371/journal.pone.0157732
Neff, K. D. (2003a). Self-Compassion: An Alternative Conceptualization of a Healthy Attitude Toward Oneself. Self and Identity, 2, 85–101. doi:10.1080/15298860390129863
Neff, K. D. (2003b). The Development and Validation of a Scale to Measure Self-Compassion.
Self and Identity, 2 (3), 223–250. doi:10.1080/15298860309027
Neff, K. D. (2019). Test how self-compassionate you are. Retrieved June 13, 2019, from https:
//self-compassion.org/test-how-self-compassionate-you-are/
Neff, K. D. & Germer, C. K. (2013). A Pilot Study and Randomized Controlled Trial of the Mindful Self-Compassion Program. Journal of Clinical Psychology, 69 (1), 28–44. doi:10.
1002/jclp.21923
Pounder, J., McFaul, R., Barton, S., Brauer, C., Gerson, Y., Rumlova, D., . . . Leizaola, R. (2016).
Humanity in the machine. Mindshare UK. Retrieved from https://www.mindshareworld.
com/sites/default/files/MINDSHARE HUDDLE HUMANITY MACHINE 2016 0.pdf Rubin, Z. (1975). Disclosing oneself to a stranger: Reciprocity and its Limits. Journal of
Exper-imental Social Psychology, 11, 233–260.
Seering, J., Luria, M., Kaufman, G. & Hammer, J. (2019). Beyond Dyadic Interactions: Consid-ering Chatbots as Community Members. In Chi conference on human factors in computer systems proceedings (May 4-9, pp. 1–13). Glasgow, Scotland UK. doi:10.1145/3290605.
3300680
Seligman, M. E., Steen, T. A., Park, N. & Peterson, C. (2005). Positive psychology progress:
empirical validation of interventions. The American psychologist, 60 (5), 410–421. doi:10.
1037/0003-066X.60.5.410. arXiv: 0803973233
Shah, H., Warwick, K., Vallverd´u, J. & Wu, D. (2016). Can machines talk? Comparison of Eliza with modern dialogue systems. Computers in Human Behavior, 58, 278–295. doi:10.1016/
j.chb.2016.01.004
Shapira, L. B. & Mongrain, M. (2010). The benefits of self-compassion and optimism exercises for individuals vulnerable to depression. Journal of Positive Psychology, 5 (5), 377–389.
doi:10.1080/17439760.2010.516763
Spitzer, R. L., Kroenke, K., Williams, J. B. W. & Lowe, B. (2006). A Brief Measure for Assessing Generalized Anxiety Disorder. Arch Intern Med, 166, 1092–1097. doi:10.1001/archinte.166.
10.1092
Spurgeon, J. A. & Wright, J. H. (2010). Computer-assisted cognitive-behavioral therapy. Current Psychiatry Reports, 12 (6), 547–552. doi:10.1007/s11920-010-0152-4
Waarlo, N. (2018). Mijn chatbot en ik: kun je bevriend raken met een computer? Retrieved from https://www.volkskrant.nl/wetenschap/mijn-chatbot-en-ik-kun-je-bevriend-raken-met-een-computer%7B∼%7Db90d6b7d/
Warwick, K. & Shah, H. (2016). The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective. AI and Society, 31 (2), 207–221. doi:10.
1007/s00146-015-0588-5
Weisband, S. & Kiesler, S. (2003). Self disclosure on computer forms, 3–10. doi:10.1145/238386.
238387
Weizenbaum, J. (1966). ELIZA - A computer program for the study of natural language com-munication between man and machine. Comcom-munications of the ACM, 9 (1), 36–45. doi:10.
1145/365719.366410
Yarnell, L. M., Stafford, R. E., Neff, K. D., Reilly, E. D., Knox, M. C. & Mullarkey, M. (2015).
Meta-Analysis of Gender Differences in Self-Compassion. Self and Identity, 14 (5), 499–
520. doi:10.1080/15298868.2015.1029966
Zamora, J. (2017). I’m Sorry, Dave, I’m Afraid I Can’t Do That: Chatbot Perception and Ex-pectations. Proceedings of the 5th International Conference on Human Agent Interaction - HAI ’17, 253–260. doi:10.1145/3125739.3125766
Zessin, U., Dickh¨auser, O. & Garbade, S. (2015). The Relationship Between Self-Compassion and Well-Being: A Meta-Analysis. Applied Psychology: Health and Well-Being, 7 (3), 340–
364. doi:10.1111/aphw.12051
Information form for participants
This document gives you information about the study “Interactions and Mood”. Before the study begins, it is important that you learn about the procedure followed in this study and that you give your informed consent for voluntary participation. Please read this document carefully.
Aim and benefit of the study
The aim of this study is to measure how interactions with someone else impact your mood.
This information is used to get a better understanding of how text based scenarios can influence their readers.
This study is performed by Nena van As, a student under the supervision of Wijnand IJsselsteijn and Minha Lee of the Human-Technology Interaction group.
Procedure
First, you will answer some demographic questions, and fill in some short questionnaires about how you are feeling. Afterwards, you will have a short conversation with a chatbot. Then you will answer a few questions about the interaction. The survey finishes with a short
questionnaire.
All questions as well as the scenario are in English.
Risks
The study does not involve any risks, detrimental side effects, or cause discomfort.
Duration
The experiment will take approximately 20 minutes.
Participants
You were selected because you are registered as an mTurk participant.
Voluntary
Your participation is completely voluntary. You can refuse to participate without giving any reasons and you can stop your participation at any time during the study. You can also withdraw your permission to use your data up to 24 hours after they were recorded. None of this will have any negative consequences for you whatsoever.
Compensation You will be paid 2 USD.
Confidentiality and use, storage, and sharing of data.
All research conducted at the Human-Technology Interaction Group adheres to the Code of Ethics of the NIP (Nederlands Instituut voor Psychologen – Dutch Institute for Psychologists), and this study has been approved by the Ethical Review Board of the department.
In this study personal data (participant ID, demographic information) and experimental data (your survey responses) will be recorded, analyzed, and stored. The goal of collecting,
7 Appendix
7.1 A: Informed consent form
analyzing, and storing this data is to answer the research question and publish the results in the scientific literature. To protect your privacy, all data that can be used to personally identify you will be stored on an encrypted server of the Human Technology Interaction group for at least 10 years that is only accessible by selected HTI staff members. No information that can be used to personally identify you will be shared with others.
The data collected in this study might also be of relevance for future research projects within the Human Technology Interaction group and researchers otherwise affiliated with the TU/e.
The coded data collected in this study and that will be released to the public will (to the best of our knowledge and ability) not contain information that can identify you. It will include all answers you provide during the study.
At the bottom of this consent form, you can indicate whether or not you agree with the use of your data for future research within the Human Technology Interaction group and by
researchers otherwise affiliated with the TU/e.
No video or audio recordings are made that could identify you.
Further information
If you want more information about this study, the study design, or the results, you can contact the survey requester via mTurk.
If you have any complaints about this study, please contact the supervisor, Minha Lee (m.lee
[at] tue.nl). You can report irregularities related to scientific integrity to confidential advisors of
the TU/e.
Informed consent form
Interactions and Mood
- I have read and understood the information of the corresponding information form for participants.
- I had sufficient time to decide whether I participate.
- I know that my participation is completely voluntary. I know that I can refuse to participate and that I can stop my participation at any time during the study, without giving any reasons. I know that I can withdraw permission to use my data up to 24 hours after the data have been recorded.
- I agree to voluntarily participate in this study carried out by the research group Human Technology Interaction of the Eindhoven University of Technology.
- I know that no information that can be used to personally identify me or my responses in this study will be shared with anyone outside of the research team.
- I give permission to the Human Technology Interaction group and researchers otherwise
affiliated with the TU/e to use this data for future research projects unrelated to this study.
INTRODUCTION Same for all conditions
1
2
3
4 3b
:) I m glad What about you?
name of participant
previous answer