When some messages are not ...
A master thesis on shadowbanning in multi-participant conversations through a mobile chat application.
Master thesis Master Computer-Mediated Communication, Communication- and Information science University of Groningen
Faculty Faculty of Arts
Supervisor dr. G.J. Mills
Author Anna Faber
Student Number S3020134
Foreword
This work is created as a master thesis for the master Computer-Mediated Communication of Communication- and Information Sciences at the University of Groningen. I would like to thank dr. G.J. Mills for guiding me through this thesis. Furthermore, I’d like to thank Janna Dolman and Pauline van Putten for the teamwork on the experiment, and my friends and family for the mental support.
Abstract
One of the core features of social media is that they have algorithmically curated timelines, meaning messages are filtered, which results in not all messages that are meant for a person are made visible for them. Curation is typically done to filter out irrelevant content and leave the content that drives engagement. A similar form of filtering is more intentional, when messages by a person are deliberately filtered by the system. This is called shadowbanning, and it is often used on discussion boards, by moderators, or on social media by algorithms.Shadowbanning is a technique where some posts by an individual are not visible for others, without any of the people knowing, including the sender. If someone has their messages deleted with them knowing, it might lead to them reposting the message. However, with a shadowban, the assumption of people or systems doing the shadowbanning is, that it will make people feel that their posts are not engaging, since nobody responds. This, hopefully, dissuades the person being shadowbanned from making further posts. So far, no research has addressed the question whether shadowbanning does in fact affect how people feel subjectively about their level of involvement in the interaction, and whether it actually does objectively affect their behavior. There has been research on ignoring in conversations, which shows some contradictory results. In Face-to-face conversations, people who are ignored show less involvement in the conversation. On the contrary, in Computer-Mediated Communication, people show an increase in their activity, to get back in the good graces of the others. In this thesis, a way of examining shadowbanning in Computer-Mediated Communication is introduced. The effect of shadowbanning on subjective and objective involvement is examined through a three-way online chat experiment where participants discuss moral dilemmas with each other and during this conversation, a selection of one participant’s turns are randomly shadowbanned. Results show no significant evidence of an effect on the objective or subjective involvement of participants. However, subjective involvement appears to decrease, while the objective involvement increases. A deeper additional analysis of the turns following the shadowban showed that shadowbanned participants have different responses for different types of turns that are shadowbanned. Shadowbanned questions show no effect, but whenever another type of turn is shadowbanned, participants repeat turns, give additional reasoning and show more activity.
Contents
1. Introduction ... 1
2. Theoretical Background ... 2
2.1. Expectations of responsivity ... 2
2.2. Involvement ... 3
2.3. Expectations in Computer-Mediated Communication ... 5
2.4. Shadowbanning ... 6
2.5. Research questions ... 6
3. Method ... 8
3.1. The Telegram setup ... 8
3.2. The Trolley dilemma ... 8
3.3. Shadowban manipulation ... 11 3.4. Participants ... 13 3.5. Material ... 13 3.6. Procedure ... 14 3.7. Hypotheses ... 14 4. Results ... 15 4.1. Data collection ... 15 4.2. Objective measures ... 17 4.3. Subjective measures ... 25 5. Discussion ... 28 5.1. Sequences of turns ... 28
5.2. Coding of shadowbanned turns ... 29
5.3. Limitations data collection ... 39
5.4. Final conclusion ... 41 Bibliography ... 45 Appendices ... 47 Appendix 1: Questionnaires ... 47 Appendix 2: Nicknames ... 49 Appendix 3: Logins ... 50
Appendix 4: Trolley problems ... 51
Appendix 5: Toelichting experiment ... 63
Appendix 6: Consentform ... 64
Appendix 7: Instructions logging in ... 65
Appendix 8: Reminder after not being active ... 67
Appendix 9: Final message to participants ... 68
Appendix 10: Example shadowban data file ... 69
Appendix 12: All data questionnaire ... 83 Appendix 13: All Shadowbans ... 86
1.! Introduction
Social media timelines can be seen as a big conversation between people (Young, 2013; Rashid, 2016), where users share their news, stories and pictures and interact with others. However, these social media can’t display all messages that are meant for you, as your timeline would become one big waterfall of content. This is why many social media filter messages for you. Facebook, for example, has an adapted timeline for every person, meaning some content that is created by a person with the intention of it being seen by a target audience is not in fact seen by this target audience. Still, there is the expectation from people that the things they write are seen, when in fact they might not be. Curating content is usually done in order to filter out content that is irrelevant, however there is another kind of filtering, where the filtering is intentional and messages are deliberately filtered by a system. This can be called shadowbanning, where some posts by an individual are not visible for others without any of the people knowing, including the sender (Deyahe, 2016). Not having your messages read and thereby not receiving any response, can feel like others are not involved in the conversation. Additionally, Facebook users have complained about their comment not showing up on public pages, and companies getting drastically less views on their content (Harsh, 2015; Hilarski, 2016). This suggests that Facebook users are in effect regularly shadowbanned, without them realizing it. At this point, the effect of this shadowbanning is unknown. The assumption of the person or system doing the shadowbanning is that the shadowbanned person will feel like their posts are not engaging, since nobody responds, and hopefully is dissuaded from making more posts. No research has addressed the question whether shadowbanning affects how people feel subjectively about their level of involvement in the interaction, and whether it actually objectively affects their behaviour. If it appears that people don’t respond to a specific message, it may feel like they are not involved in the conversation. After all, the lowest level of involvement is not responding at all. This research will examine the effect of being shadowbanned on the subjective and objective level of involvement.
2.! Theoretical Background
2.1. Expectations of responsivity Adjacency pairsIn everyday spoken conversation we have an expectation that when we speak, the other person hears and gives evidence they have heard us. This is part of the grounding process. Grounding is explained by Clark & Brennan (1991) as a joint effort, where, when people interact, they collect shared knowledge, which is called the common ground. Through a process of grounding their contributions they continually strive to reach mutual understanding. To show mutual understanding and creation of common ground in conversations, positive evidence of understanding, such as continued attention (i.e. eye gaze), acknowledgement responses or initiations of the next relevant turn, is needed. The process of grounding contributions consists of a presentation phase, where the turn is presented to the conversation partner and an acceptance phase, where evidence of understanding is given. Similar to the presentation and acceptance phases are adjacency pairs, consisting of a first pair part and second pair part (Sacks, Schegloff & Jefferson, 1974), where the second logically follows the first. For example, when a question is asked in the first pair part, there is a major expectation that a relevant answer will be given in the second pair part. A first pair part is very often also a second pair part, responding to a previous turn and, the other way around, a second pair part is very often also a first pair part, eliciting a response.
Silence
A second pair part can either be a preferred or a dispreferred action. When it is preferred, it is the response that is wanted or expected by the sender. For example, for an invitation you would want an acceptance. The dispreferred answer would be a decline or no answer at all. According to Boyle (2000), a dispreferred answer can either be noticeable, accountable, but not sanctionable, or it can be noticeable, accountable and sanctionable (figure 1).
Figure'1:'Structure'of'preference'(Boyle,'2000)'
When you invite someone to a party, and they decline without giving a reason, this would be seen as noticeable, accountable and sanctionable, since no legitimate reason is given for the dispreferred answer. When someone explains they have other business to attend to or that they are very sorry that they can't make it, this dispreferred answer will be noticeable, accountable but not sanctionable. Here is another example, where a question is asked about the length of a flight and, in the response, there is not a relevant next turn, but there is an explanation that they can not answer, because this information is not available. This response is seen as a dispreferred answer, since it’s not what the sender
“How long will the flight from KLM to Paramaribo take?”
“I’m sorry, this is Surinam Airways, we don’t have information on KLM flights”
Other forms of giving an account of why you are unable to answer, is through non-verbal communication by, for example, showing you can not answer, because you are occupied with something. However, it can also be the case that no second pair part is given, and there is just silence. As explained before, this silence is also seen as a dispreferred answer, and without account it’s sanctionable, usually with hostility involved (Seedhouse, 2004). Sometimes, these situations can be avoided by using, so called, communication breakdown strategies, which include repeating the first pair part, or adding some questions to it (Long, 1983). For example, with the greeting, they might greet again to make sure it was heard by the other person (Seedhouse, 2004). However, sometimes it’s unavoidable and when, for example, no response is given to an invitation, the person may no longer be ever invited to social events. Or when you greet a person and and there is no return greeting, there might be an irritated response. Not getting any response to a first pair part you create, can seem like your conversation partner is not involved in the conversation, which can affect your level of involvement as well (Coker & Burgoon, 1987; Tannen, 2007; Geller, 1974; Kendon, 1967; Williams et. al, 2000; Molden et. al, 2009). But how exactly does this affect the involvement? In order to look into the effect of a lack of involvement, we must first explain the concept of involvement.
2.2. Involvement
Being involved in a conversation is important for creation of mutual understanding (Coker & Burgoon, 1987; Tannen, 2007; Geller, 1974; Kendon, 1967; Williams et. al, 2000; Molden et. al, 2009). According to Tannen (2007), involvement plays a big part in the meaningfulness of a conversation and a lack of involvement can lead to misunderstandings or communication breakdown, which means people prefer others to be involved to limit this breakdown. Conversational involvement can be described as: “the degree to which participants in a communicative exchange are cognitively and
behaviourally engaged in the topic, relationship and/or situation.” (Coker & Burgoon, 1987, p.463). Similar to this,
Nguyen & Fussell (2014) explain that there are two aspects to involvement; the feeling of engagement and involvement by conversation partners and yourself (subjective), and the actual markers that show involvement in conversations, which are necessary to feel whether your partner is involved (objective).
Subjective
The feeling of involvement includes how people rate their own involvement in the conversation or how they rate that of their conversation partner. According to Goffman (1967), lack of involvement of a conversation partner can make people feel uneasy. You can even expect people to feel ignored when they get no response from their conversation partner, as this feeling can be evoked when the expectations with regards to attention are not met, such as not getting eye contact in a conversation or not getting a response to a first pair part in an adjacency pair (Geller et. al, 1974). In order to feel how involved your conversation partner is, and in order to express your own involvement, these objective types of behaviour are needed.
Objective
The objective aspect of involvement in face-to-face communication includes non-verbal cues such as eye-gaze and verbal lexical indicators, such as personal pronouns and assent words. In 2014, Nguyen and Fussell aimed to identify lexical cues of involvement in text-based IM in their research on expressing and interpreting involvement (Table 1). They discovered first of all that high involved communicators used more words than less involved communicators. Additionally, more involved communicators used less personal pronouns (I, Me), more definite articles (The, This, That) and more assent words (Accept, right etc.), whereas uninvolved communicators use more personal pronouns, less definite articles and less assent words.
Table'1:'Lexical'indicators'of'involvement'from'Nguyen'&'Fussell'(2014)'
Type Lexical indicators
Personal pronouns I, Me
Qualifiers Could, hardly, fairly, few etc.
Intensifiers Definitely, Absolutely, Certainly, Extremely etc. Definite articles The, This, That
Indefinite articles A, an, any
Assent words Accept, right, indeed, true, etc. Cognitive mechanism words Think, guess, assume, understand etc.
Effects of being ignored
The uneasy feeling people can get when their conversation partner is not involved, can also be described as a feeling of being ignored. According to Geller et. al. (1974), people who are being ignored in face-to-face conversations will not evaluate themselves or their conversation partners favourably compared to people who were not ignored. They explain that one of the reactions to being ignored is flight, but often this is not an option. Participants who were in an experimental group and were being ignored, by being looked at less than others and being talked to infrequently, participated less in the overall conversation. Similar result was found by Kendon (1967) in his research on social performance when being ignored through not getting eye contact in face to face interaction. When a person does not receive these visual cues, which is similar to when a person is not behaviourally/objectively involved, it leads to less speaking and less participation. This shows that in face-to-face conversations, the shadowbanned person ‘feels’ ignored, and thereby subjectively also feels less involved. Additionally, objectively they show less involvement by participating less. These results are based on face-to-face conversations, but in order to be able to investigate the effects of being shadowbanned, this research will use Computer-mediated communication (CMC). What are our expectations with regards to responsivity in CMC and what happens when people are ignored?
2.3. Expectations in Computer-Mediated Communication
In Computer-mediated communication, the expectations with regards to responsivity in conversations are weakened due to natural incoherence in the conversation. This incoherence is caused by lack of simultaneous feedback, which is a result of the fact that send messages can’t overlap, and disrupted turn adjacency, which is caused by messages that are posted in the order they were received in by the system. Most multi-participant CMC systems use ‘turn-by-turn’ transmission for messages. This means messages are sent in their entirety as soon as the sender presses ‘send’. It’s not possible for anyone to comment on the message while it’s being created, ergo the lack of feedback. Additionally, as the messages are posted to the recipients in the order in which they were received by the system, disrupted turn adjacency is not an uncommon thing. This means the adjacency pairs get intertwined (Herring, 1999). Especially in multi-participant interaction this is an issue. This is also visible in Figure 2, where there are five participants in the conversation. The lines show the adjacency pairs between these people and in this example, the different adjacency pairs are intertwined. The more people there are in the conversation, the bigger the possibility of turns being out of sequence. Therefore, the likelihood of phantom adjacency pairs increases; Usually, in face-to-face conversations, the next turn after a first pair part is the second pair part, or if no turn follows, it is seen as ‘absent’. As a result, people in CMC will often see the message that is posted after a first pair part, as the second pair part, even when this is not the case (Garcia & Jacobs, 1998). For example, when in line 3, D asks a question and in line 4, there is a turn by J, it might look like J answers the question by D, even though he actually responds to A in line 1.
Clark & Brennan (1991) explain these limitations to the conversation as part of constraints that media have for reaching common ground, such as whether the communication partners are co-present in the same physical environment, whether they can see each other or whether the turns from both communication partners can get out of sequence. As people get used to the incoherence in CMC, the expectations that a second pair part follows the first pair part immediately, are lowered significantly. Users adapt to the medium features by creating new methods to deal with these incoherence issues, for example, using cross-referencing to signal who they are referring to in their turn (Herring, 1999; 2013). Additionally, people even use the incoherence to their advantage and often cut up their utterance in CMC into multiple messages, to make the conversation more speech-like or to hold the floor and continue talking without interruptions (Herring & Androutsopoulos, 2015). These findings suggest that disrupted turn adjacency is common and the expectations for receiving a second pair part in sequence with the first pair part, are lowered.
Figure'2:'Instant'Messaging'
conversation'structure'with'disrupted' turn'adjacency.'J'(line'4)'responds'to' A'(line'1)'and'in'between'D'(line'3)' gives'a'first'pair'for'K'(line'5)'etc.
Ignoring in CMC
Because participants in CMC have the expectation of incoherence, it also appears that their responses in CMC to being ignored are different. What they do, when they are ignored, appears to be that they compensate objectively by actually participating more. Recent research on ignoring over the internet shows that after being ignored, participants feel bad, feel like they are not in control and will try to get back in the good graces of the others in the conversation, which might increase their activity in the conversation to fix the contact and thereby also increase their level of objective involvement (Williams et. al., 2000). Additionally, Molden et. al. (2009) state that passive, implicit and indirect ignoring leads to more promotion-focussed responses, meaning they would reengage in the interaction, increase the social contact in the conversation, but also have an increased feeling of dejection, confirming the results by Williams et. al (2000). This suggests that in CMC, ignored people will objectively get more involved, but subjectively the ignoring has a negative effect and they feel dejected, and not in control.
So the question arises; what will happen when adjacency pairs are not just intertwined, but parts of it are not send at all, like with shadowbanning? Will people show more objective involvement in the conversation, like research online suggests, and show less subjective involvement? Or will their involvement decrease, as is explained in face-to-face research on ignoring?
2.4. Shadowbanning
Shadowbanning is a form of censorship in CMC where, without any of the communicators knowing, not all posts by a person are visible for the conversation partners (Deyahe, 2016). This method can only be used in CMC, as in face-to-face communication, it’s impossible for nobody to know that one person is being shadowbanned. Shadowbanning in a CMC conversation can be simulated best using at least three participants per conversation, since this way it’s less noticeable that one person is being shadowbanned. The participants can be asked about how involved they felt their conversation partners were, and thereby give insight in the differences in the subjective involvement of both the non-shadowbanned user, and the shadowbanned user. This method can also show the the effect on the objective involvement of someone who is shadowbanned and others who are not, by being able to identify the objective lexical markers that are used by the participants. In addition to the lexical markers, simple measures like amount of turns and length of turns can show whether the shadowbanned person is more or less active in their contributions to the conversation. Additionally, the shadowban can have effect on all participants. When, for example, a question is asked, but this question never reaches the receiver, the sender might have the assumption that the question was not successful, as none of the other two participants respond, and they will not ask it in that way again. However, if the question that is send by one of the participants is visible to all, and the reply is shadowbanned, it might seem like this conversation partner does not care about the question and is not involved.
2.5. Research questions
So, how do people respond to shadowbans? Or to being ignored? On the one hand, work from communication study suggests that when people are ignored, they drop out of the conversation and speak much less than the others, which is clearly a decrease in objective involvement. On the other hand, work on CMC suggest that, when there is an expectation of noise or incoherence, people appear to be much more tolerant and might compensate in a different way by being more objectively involved in the conversation and participating more. With regards to subjective
feel dejected and demotivated, which suggests a decrease in subjective involvement to the conversation. This results in the following research questions:
•! RQ: Does shadowbanning lead to more involvement or less involvement? o! Is there an effect on the objective measurements of involvement? o! Is there an effect on the subjective measurements of involvement?
3.! Method
In this experiment, participants participate in a multiparty interaction using an instant messaging app, Telegram. All the messages they type pass through a server that runs on a laptop. The server is used to artificially manipulate the conversation. In this case, by introducing artificial shadowbans. The participants don’t know this. They have a three participant conversation where they try to solve the “trolley game”, which is a moral dilemma (created by Philippa Foot in 1967), by coming to a joint decision. The participants participate over four days.
3.1.!The Telegram setup
Each participant in the experiment is simultaneously participating in to two different chat conversations. They chat with two other participants; of whom they don’t know the identity. Each participant has a personal nickname, which is the same nickname in both conversations. This is assigned to them randomly by the experimenters before the start of the experiment. The chat messages, that are sent by the participants, are passed through a server that runs on a laptop, which also stores them.
Telegram allows users to connect to a bot. The way that this experiment functions, is that each of the participants connects to a bot. That bot, instead of being connected to a programmed chat bot, intelligently connects participants to other participants. Their messages are relayed between themselves and they participate in a conversation. What that means for the participants, they have two separate chat windows for two different bots and in each chat window they talk to two other people. The chat conversations are named ‘Communication125149Bot’ and ‘Communication641322Bot’. These names are chosen so the participants will not have preference of one of the conversations based on the name. Participants can join these conversations by searching for these names in the app and by typing in their login code. The bots are programmed with the rules of the trolley dilemma. After typing in their login code, the participant is connected to a conversation by the bot, and as soon as all participants are connected, they can start discussing the trolley dilemma.
3.2.!The Trolley dilemma
As soon as all participants are logged in, they are presented with a trolley dilemma. They have to come to an agreement on how to solve this dilemma in order to proceed. In Figure 3, an example of the original trolley dilemma is shown. The dilemma concerns deciding whether to get involved in a situation in which you might save lives, while taking others’. Besides this basic version, there are many alterations of this dilemma that the participants discuss.
During the conversation, the participants go through several stages with regards to the trolley dilemma:
•! Stage 1: The participants discuss the trolley dilemma. When they want to review the image, they can type ‘/t’; the bot will then send the image again, this time only to the person asking for a resend.
•! Stage 2: When participants have come to an agreement on how to solve the dilemma, they can initiate a decision process by typing ‘/b’. If any of the participants type ‘/b’ before all of the participants have typed anything in the conversation, all participants get the message: ‘Discussieer alle drie voordat jullie het beslissingsproces starten’ and it goes back to stage 2.
•! Stage 3: When the decision process starts, the participants get two buttons in their screen that show the two decision options for the trolley dilemma (Figure 4b). Each participant clicks the option of their choice. •! Stage 4: When all participants click the same option, the bot sends a new trolley dilemma image and they
start again from stage 1. When they do not make a unanimous decision and one of the participants chooses a different option from the others, the decision fails and they go back to stage 2.
Figure 4a and Figure 4b show a typical chat window that the participants see while chatting on the app Telegram. In Figure 4a, the experiment has just started and they are at stage 1. In the top of the image, the name of the conversation is shown and at the bottom of the screen the participant can type their message. In Figure 4b the conversation is just transitioning to stage 4, as the decision process has been initiated by a participant and the options are visible. The participant speaks to two people: Pela and Tohi. Only the nicknames of the other participants are visible for the participant. They are deciding on the dilemma: ‘Anonieme rijkdom door moord’, which they received at the start in Figure 4a.
Figure'3:'Original'trolley'dilemma:'A'train'is'heading'towards'a'track'where'five'people' are'tied'down.'You'do'not'have'time'to'free'the'five'people,'but'you'are'near'a'switch' that'will'lead'the'train'towards'another'track. However, on this track one person is tied. What do you choose?
Figure'4a:'Stage&1:'A'chat'window,'as'seen' by'the'participant.'After'the'participants'are' all'logged'in,'they'receive'their'first'trolley' problem'image'as'well'as'instructions'on' how'to'review'the'image'or'start'the'decision' process.' Figure'4b:'Stage&4:'A'chat'window,'as'seen' by'the'participant.'After'discussing,'the' decision'process'is'initiated'and'the' participants'can'choose'their'option.'
3.3.!Shadowban manipulation
During the experiment, one of the two conversations that the participants have is manipulated with a shadowban manipulation. The independent variable in this research is shadowbanning, where there is either a shadowban manipulation in the conversation or there is not. The dependent variables measure the effect of the shadowban on the involvement in the conversation.
In order to test the conversation bots and the manipulation, a pilot experiment was conducted. This pilot experiment lasted two days, consisted of nine participants and all the settings for the bot with regards to the trolley dilemma were tested. The participants in this pilot study are gathered through convenience sampling. In this pilot study, the rules for the manipulation were tested to check how often the manipulation takes place in the conversation in order to make sure the participants were manipulated enough but not too much to notice. The chance for the manipulation to be triggered was tested at 10%, 15% and 20%. Based on the pilot study, the settings are altered for the final experiment, such as some functions of the bot and the manipulation settings.
3.3.1.! Shadowbanning
Shadowbanning is the manipulation of this research and consists of turns by one person, the so called ‘victim’ (V), not being sent to the other two participants (P) in the conversation. In order to examine this concept in the experiment, different possibilities for the manipulation are considered, such as when the shadowban takes place in the conversation and what the rules are. After careful consideration, the final manipulation includes the following rules:
•! Rule 1: During stage 1, when the participants start discussing the trolley problem, all participants should send at least one turn before the manipulation can be triggered. This is to make sure they are aware that everyone is present in the conversation before turns are immediately shadowbanned.
•! Rule 2: After all participants have sent one turn, the next turn that the victim (V) sends, has a 15% chance of being shadowbanned.
•! Rule 3: If the shadowban is not triggered, every next turn by V has a 15% chance of being shadowbanned. This is to make sure there are enough manipulations during the conversation.
•! Rule 4: After a shadowban happens, all participants should, again, send at least one turn before the next turn by V has a chance of being shadowbanned. This is to limit the chance of the participants noticing the manipulation, and for a possible recovery to take place in the conversation, before any turns are shadowbanned again.
•! Rule 5: Command messages to the bot, such as deciding (/b) and resending (/t), should not be shadowbanned. These turns are important for the decision process and should not be shadowbanned, which will also limit the participants noticing the manipulation.
3.3.2.! Dependent measures
This research includes two types of measures with matching dependent variables in order to measure the involvement of the participants. It consists of objective measures, which include game performance measures and other qualitative measures with regards to the conversations, and subjective measures, in the form of a questionnaire.
3.3.2.1.! Objective measures
The objective measures include game performance measures, such as the amount of games played and amount of decision initiations, and the conversational performance, which include qualitative measures with regards to the conversation, such as amount of turns and lexical cues of involvement.
Game performance:
•! Amount of games: The number of trolley problems the participants made a decision about.
•! Amount of decision initiations: The amount of times a participant sends ‘/b’ to start the decision process.
Conversational performance:
•! Amount of turns: All turns that are sent by participants, excluding ‘/b’ and ‘/t’.
•! Length of the turns: The amount of characters in the turn. The average length is measured by taking the total length of the turns of each participant in characters and dividing this through the amount of turns they made. •! Contribution in characters: The contribution of a participant in the conversation compared to the others in
the same conversation. This was measured by calculating the amount of characters the participants used and dividing this by the total amount of characters typed within the conversation.
•! Lexical cues of involvement: In order to describe the level of objective involvement by the participants, the lexical indicators of involvement from Nguyen & Fussell (2014) are used. The analysis includes the personal pronoun indicators ‘Ik’ and ‘Mij’.
3.3.2.2.! Subjective measure - Questionnaire
In order to measure the feeling of involvement of the participants, a questionnaire will be filled in by the participants. The full questionnaire can be found in Appendix 1. The questionnaire consists of two parts. Participants participated over four days. Ten triads of participants participated in this experiment. In order to make sure the server didn’t crash, there were multiple starting moments of triads during the two weeks of running the experiment. In the first week, on Monday, the first triad started, and on Tuesday another two triads were added to the experiment. In week two of the experiment, on Monday another three triads started the experiment and on Tuesday the remaining four triads. The first part of the questionnaire was asked right after each triad of participants finished with the experiment, but we wanted to wait until every triad finished the experiment before revealing the purpose of the research in part two to make sure they could not communicate the purpose to participants who were still in the experiment.
Part 1 Conversation
These questions are asked to figure out which of the conversations the participants preferred. •! In which group did you think the conversation was most fluent?
•! In which group did you feel like participating more in the conversations? •! In which group was the decision process easier?
Participants
These questions in the questionnaire focus on specific people in the experiment. In order to find out which person the participants preferred speaking to, or, to find out whether the participants would reward themselves for their involvement in the experiment.
•! Who did you enjoy talking to the most?
•! Which pair in the three-way conversation was better in communicating with each other?
•! What if you can give a gift coupon of 25 euro to the person that was involved most in the conversation. Who do you believe earned the gift coupon?
Other
Finally, in the first part, some open questions are asked to see whether the participants have an idea about the goal of the experiment:
•! What do you think the purpose of this research was?
•! We are working on optimizing the bots of the conversations and to eliminate any flaws. Have you discovered anything that you think we should know of?
•! Is there anything else you would like to share about the experiment or the questions?
Part 2
The second part of the questionnaire is very short, and consists of two main questions to figure out if they noticed the manipulation, but to still limit the feeling of being misled by telling them they are network errors. Finally, they are told which manipulation they were in and asked which role they think they had in the experiment.
•! We tested three typed of network errors. Which one do you think happened in the conversation?
•! In Communication641322Bot there was a manipulation where some utterances by one person were not sent to the other two. Which role do you think you had?
3.4.!Participants
30 participants participate in two conversations at the same time. One of these is manipulated with the shadowban manipulation, which is called the manipulation condition, and the other has no manipulations, which is the baseline condition. Each participants speaks to two others in each conversation. The participants in the baseline condition are linked to two participants who are not in any of the conversations in the manipulation condition, to make sure the participants are never speaking to the same person twice. In total, there are ten triads in the manipulation condition. The gathering of the participants was based on convenience sampling in our direct network. After the participants are divided into the conversation triads they will participate in, these triads are put into blocks. These blocks then start on different days, to make sure the server, that runs on the laptop, can handle the simultaneous conversations and will not crash. This is also why the experiment takes place during two weeks and there are two starting point in every week.
3.5.!Material
Each participant has a personal nickname, which is the same in both conversations, and which was created by taking consonants for the 1st and 3rdletters and vowels for the 2nd and 4th letters and to make sure they sound neutral. The
list of nicknames is in Appendix 2. The logins that the participants use to login to the bots, were created with the use of a random password generator (Appendix 3). In total, 120 trolley problems were created by three experimenters for this experiment, of which 60 were actually discussed during the experiment. These trolley problems include very serious as well as humorous topics to give the participants some variety. The complete set of trolley problems are in Appendix 4. The trolley problems were also divided into two groups: baseline trolleys and manipulation trolleys, to make sure participants do not get trolleys twice. For the communication with the participants, a specific email address is created.
3.6.!Procedure
Before the start of the experiment, possible participants are asked briefly if they would be interested to participate and if they may be invited. Shortly after, the participants are invited through email and sent the background information on the topic (Appendix 5) as well as a consent form (Appendix 6). To motivate the participants to participate actively in the experiment, four 25-euro gift cards are promised to the most active participants. After all consent forms are in, the evening before the start of the experiment, the participants get the instructions on how to login (Appendix 7). These instructions are unique for every participant, as every participant needs a personal login code. From that moment on, the participants are allowed to login to the two conversation bots. As soon as all three participants in a conversation are logged in, the bot sends the first trolley problem and the participants can begin. At 6:00 am every morning, the bot automatically sends a new trolley problem in order to trigger the conversation and make sure the participants don’t forget they are part of the experiment. During the experiment, the conversations are monitored. If, in the first two days, a participant is not active for 6 hours, the participant gets a reminder via email (Appendix 8). The morning after the experiment, the participants get the first part of the questionnaire. After all the experiments have finished, the second part is sent to all participants. As soon as all the questionnaires are filled in, the participants are thanked through email and the winners of the gift certificates are informed (Appendix 9).
3.7.!Hypotheses
•! RQ: Does shadowbanning lead to more involvement or less involvement? o! Is Is there an effect on the objective measures of involvement?
!! The game performances are affected by the shadowban
!! The conversational performances are affected by the shadowban o! Is there an effect on the subjective measures of involvement?
4. Results
4.1.!Data collectionDuring the data collection there were some errors with the setup of the experiment and some limitations with regards to the use of Telegram, the trolley problems, the manipulation and the participants. In two manipulation conversations there was no manipulation due to participants not interacting much (i.e. producing few turns), resulting in the final data of the experiment consisting of sixteen chat conversations; eight from the shadowban manipulation and eight matching baseline conversations. From the Telegram conversations, a large data file is created which includes timestamps, bot messages, commands from the participants and all the turns with corresponding nicknames and login ID’s. In this data file, the length of each turn in characters is added and the questions are labelled. An example of the data file can be found in Appendix 10. All SPSS Data can be found in (Appendix 11)
4.1.1.! Errors
In total, there were four system resets. This means that at those moments there was something wrong in the experiment, and the laptop with the server had to be reset for either both groups or for just the baseline group. During a reset, the participants can sent turns, but nobody will receive them. The bot will also not sent any messages. After the system is reset, a new trolley problem is sent to all conversations.
1.! In week 1, on experiment day 2 at 6:00 am a new trolley was sent to the participants, but there was an error where it was not just sent at 6:00 am, but every hour a new trolley was sent in all conversations. This lasted for 7 hours, which means in the chats that were not very active, they weren’t able to decide on a trolley before a new one was send. This eventually resulted in a reset for both the baseline and the manipulation conversations at 12:30pm.
2.! In week 1, on experiment day 4, it appeared that the conversations went through the available trolley problem images very quickly. Also because of error 1. New trolley problem images were created and this resulted in a reset of both groups at 11:30am.
3.! In week 2, on experiment day 3 it appeared that the baseline chats were receiving old trolley problem images from an earlier test that were not grammar checked and did not have the two choices for the dilemma visible in the image. This resulted in the servers for all conversations needing to be turned off and being offline during 2 hours and eventually it led to a reset at 19:45pm. Additionally, this day was a holiday, which was unforeseen and resulted in little activity on this day.
4.! In week 2, on experiment day 4 the issue from day 3 with the old trolley images was fixed, but now the baseline chats were receiving trolleys they had already discussed. This resulted in a reset at 1:30pm for only the baseline conversations.
Additionally, when a reset was necessary for the system, the chats went back to stage 2, where they start discussing a new trolley image. This also means that rule 1 for the manipulation, where everyone has to have sent at least one turn, was repeated, meaning less turns had the chance of being shadowbanned.
4.1.2.! Telegram
Because of using Telegram bots, there can be some delay in the messages being stored on the laptop that runs the server. Because of this delay, it is possible that some messages are not shown in the correct order in the data for this experiment. Additionally, Telegram offers the option to use a web-version, meaning participants could have typed turns on their computer. Even though the participants were asked not to use this function, it might have happened, as it’s impossible to monitor this at a distance.
4.1.3.! Manipulation
As rule 2 shows, there is a chance of 1/15 of a turn being shadowbanned. This chance resulted in there only being 25 shadowbans in the 8 conversations that were used in this experiment. Table 2 shows the amount of shadowbans per conversation. Figure 5 shows the average amount of shadowbans that occurred per triad per day. In Figure 6, the average percentage of turns that were shadowbanned per triad is displayed. When 15% of 1/3 of messages is shadowbanned, this percentage should be on average around 5%. On day one 4,5% of the turns in each triad were shadowbanned, on day two this was 3%, on day three 2,5% of the turns were shadowbanned and on day four this went up again and around 4,2% of the turns were shadowbanned.
'
Table'2:'Amount'of'shadowbans'per'conversation' Conversation name Amount of shadowbans
during experiment Shadowban02 4 Shadowban03 4 Shadowban04 1 Shadowban05 5 Shadowban06 2 Shadowban07 3 Shadowban08 2 Shadowban10 4
4.1.4.! Participants
From the 30 participants, there were five participants that were not very active during the experiment, which resulted in there not being enough data in two manipulation conversations, which resulted in not using them in the dataset.
4.2.!Objective measures
4.2.1.! Game performance - Amount of games
Ha: There is a difference in the amount of games played between the manipulation and baseline conditions.
In total the participants finished 89 trolley games. On average, the manipulation condition finished 1.5 games per day. For the baseline condition, this is 1.28 games per day. Figure 7a shows the average games finished per triad per day. This shows that on day one and day two the participants in the manipulation condition finished more games on average than in the baseline condition. On day three it is almost equal and on day four, the baseline condition have finished much more on average than the manipulation condition. In Figure 7b, it’s also visible that on day one there is a big outlier in the manipulation condition. On day four there is a big outlier in the baseline condition. However, considering all days, with P=0.363 (N=64, α= 0.05), this is not significant and Ha can not be accepted.
Figure'5:'Average'amount'of'shadowbans'per'triad'per'day Day Av er ag e am o u nt o f s h ad o w b an s p er t ri ad Figure'6:'Average'percentage'of'turns'that'were' shadowbanned'per'triad'per'day Day Av er ag e p er cen tag e o f tu rn s th at w er e sh ad o w b an n ed p er t ri ad
4.2.2.! Game performance - Amount of decision initiations Victim vs. other participants in manipulation condition:
Ha: There is a difference in the amount of decision initiations from the victim compared to the other participants in manipulation condition.
Looking at the decision making process, the victims were not very active in deciding. Figure 8a shows that on day one, the decisions were initiated equally by the victims and the participants, around 30/33%. However, after the first day, the victims became less and less active in deciding in the manipulation condition compared to the other two participants in the conversation, where on day two only 20% of the decisions came from the victims and on day three less than 10%. On day four however, the decisions are back to being equally divided. Overall, with P=0.248 (N=81, α= 0.05) it is not significant, meaning there was no evidence to support the hypothesis.
Victims in baseline vs victims in manipulation condition:
Ha: There is a difference in the amount of decision initiations from the victim in the baseline condition compared to the victims in the manipulation condition.
In Figure 8b, the decision initiations by the victim in both conditions is visualized, and shows a similar graph as Figure 8a. The victim is more active in deciding in the baseline condition compared to the manipulation condition on day two, three and four. On day two over 30% of the decisions are from the victim in the baseline condition. On day three 20%, but in the manipulation condition this is only 5%. Taking all days into account, with P=0.692 (N=50, α= 0.05) this result Figure'7:'Average'amount'of'games'finished'per'triad'per'
day'in'the'experiment
Figure'7b:'Boxplot'of'average'amount'of'games'finished'per' triad'per'day'in'the'experiment'showing'the'outliers'on'day'1' and'4.
4.2.3.! Conversational performance - Amount of turns Manipulation condition vs baseline condition:
Ha: There is a difference in the amount of turns in the manipulation condition compared to the baseline condition.
The average amount of turns that were sent per day per person, is visualized in Figure 9a and Figure 9b. Figure 9a shows that on day one and two, the people in the manipulations condition sent more turns than in the baseline condition. However, on day three, this changed and for the remainder of the experiment, people in the baseline condition sent more turns per person per day than the people in the manipulations condition. Over these four days, with P= 0.602 (two-tailed) (N=192, α= 0.05), the results are not significant, which means Ha can not be accepted.
Victim vs other participants in manipulation condition:
Ha: There is a difference in the amount of turns sent by the victim compared to the other participants.
Within the manipulation condition (Figure 9b), there is a different situation, as on day one, three and four the participants sent more turns per day than the victims and only on day two the victims sent more turns per person than the participants. However, over these four days, with P= 0.720 (two-tailed) (N=96, α= 0.05), the results are not significant. This means there is no evidence to support the hypothesis.
Figure'8a:'Average'percentage'of'decisions'initiated' per'person'per'conversation'compared'to'the'other' two'conversation'partners'in'each'day'(2'participants' +'1'victim'make'1.0'in'total)' Figure'8b:'Average'percentage'of'decisions'initiated'per' person'per'conversation'compared'to'the'other'two' conversation'partners'in'each'day'(comparing'victims'in' baseline'to'victims'in'manipulation'condition)
These results show an overall decrease in participation in the amount of turns sent. This could also explain the decrease in absolute shadowbans that occurred.
4.2.4.! Length of turns in characters
Manipulation condition vs. Baseline condition:
Ha: There is a difference in the length of the turns in the manipulation condition compared to the baseline condition.
The average length of the turns in characters per person per day, comparing the baseline condition to the manipulation condition, is visualized in Figure 10a. It appears that, after the first day, which shows a drop in both conditions, the average length of turns in the manipulation condition goes up with a largest difference on day three. In the first two days, most of the shadowbans took place, which could explain the big difference on day three. However, overall, with P= 0.134 (two-tailed) (N=181, α= 0.05), this result is not significant, which means Ha can not be accepted.
Figure'9a:'Average'amount'of'turns'per'person'per'day' (Comparing'baseline'to'the'manipulation'conversations). Figure'9b:'Average'amount'of'turns'per'person'per'day' (Comparing'the'participants'to'the'victims'in'de'manipulation' conversations).
Victims vs. other participants in manipulation condition:
Ha: There is a difference in the length of the turn typed by the victim compared to the other participants in the manipulation condition.
If we look closer at the manipulation condition in Figure 10b, the difference between the participants and the victim in the manipulation looks even bigger. Especially on day three. However, overall, with P= 0.369 (two-tailed) (N=91, α= 0.05) this is not signficant, meaning Ha can not be accepted.
Victims in manipulation condition vs victims in baseline condition:
Ha: There is a difference in the length of the turn sent by the victim in the manipulation condition compared to the victims in the baseline condition.
When looking at the victims in both conditions, in the average length of turns per person (Figure 10c), it seems that the turns by the victim are the longest in the manipulation condition. Again, on day 3 this difference is the biggest. However, overall, with P=0.470 (two-tailed) (N=58, α= 0.05) this is not significant. Therefore Ha can not be accepted.
Figure'10a:'Average'amount'of'characters'per'turn'per'person' per'day'(Comparing'baseline'to'manipulation)
Figure'10b:'Average'amount'of'characters'per'turn'per'person'per' day'(comparing'participants'and'victims'in'manipulation)
Even though there are no significant results, it appears that there was some effect on the manipulation condition and the victims. In all of these graphs, it’s visible that the average length of the turns for the manipulation, or the victims gets bigger and bigger untill day three, and then goes down again. For the baseline and the participants, the average length starts high but gets lower and lower.
4.2.5.! Contribution in characters
Victim vs other participants in manipulation condition
Ha: There is a difference in the contribution in characters of the victim compared to the other participants in the manipulation condition.
In all situations, if there would be no effect, the average contribution per person would be 33.3%. However, Figure 11a shows that in the manipulation condition, the victims were more active than both of the other participants on day two and day three. Again, on the third day there was a big difference between the victims and the participants. However, overall, with P=0.345 (Two-tailed) (N=96, α= 0.05), this was not significant, therefore Ha can not be accepted.
Victim in manipulation condition vs victim in baseline condition
Ha: There is a difference in the contribution in characters of the victim in the baseline condition compared to the manipulation condition.
If we view the contribution of the victim in both conditions, it is a similar result (Figure 11b). The lines are alike, but the victim shows more contribution in characters in the manipulation condition on the second and third day compared to the baseline condition. but with P=0.550 (N=63, α= 0.05), it’s not significant, meaning Ha can not be accepted.
Figure'10c:'Average'amount'of'characters'per'turn' per'person'per'day'(comparing'victims'in'both' conditions).
4.2.6.! Lexical indicators of involvement Victims vs participants in manipulation condition
Ha: There is a difference in the use of lexical cues of involvement between the victims and other participants in the manipulation condition.
Counting of the Dutch personal pronouns ‘Ik’ and ‘Mij’ showed that on average the participants used 3.11 personal pronouns per day and the victims around 2.88 personal pronouns per day, which would suggest victims being more objectively involved with their use of lexical cues. However with P=0.859 (N=96, α= 0.05) this result is not significant. Figure 12 shows the average amount of personal pronouns per person per turn in the manipulation condition for the victims and the participants. It shows an interesting result. On day one, the victims used personal pronouns in 30% of their turns. The participants used personal pronouns in around 42% of the turns. On day two, the use of personal pronouns for the victims increased and they used personal pronouns in over 40% of their turns. The participants use of personal pronouns increased as well, with over 52% of the turns on day 3 containing personal pronouns. On day three and four, however, the use of personal pronouns per turn drastically lowers for the participants to only around 32%. For the victims the use of personal pronouns per turn stays about equal on day 2, 3 and 4 around 45%. Overall, with P=0.771 (N=91, α= 0.05) this result is not significant, meaning there is no evidence to support the hypothesis.
Figure'11a:'Average'contribution'to'the'conversation' in'characters'per'person'per'day'in'the'manipulation' conversation'(1 victim and 2 participants makes 1.0).
Figure'11b:'Average'contribution'to'the'conversation'in' characters'per'person'per'day'(comparing'victims'in'both' conditions).
Day Figure'12:'Average'amount'of'personal'pronouns'per' person'per'turn'(comparing'participants'to'victims' within'the'manipulation'conversations) Av er ag e am o u nt o f p er so n al p ro no un s p er pe rs on pe r tu rn
4.3.!Subjective measures 4.3.1.! Questionnaire Part 1 Conversation
With regards to a preference for a conversation, three questions were asked. Both the victims and the participants 62.5% thought the manipulation conversations were most fluent (Table 3). With P=1.00 (N=24, α= 0.05) this is not significant. Looking at the conversation where they felt like participating more, 75% of the victims chose the baseline condition and only 56.3% of the participants chose the baseline condition (Table 4). With P=0.657 (N=24, α= 0.05) this is not significant. Additionally, from both the victims as the participants, 62.5% thought the decision process was easier in the baseline condition than in the manipulation condition (Table 5). However, with P=1.00 (N=24, α= 0.05) this is not significant. All data for the questionnaire can be found in Appendix 12.
Table'3:'Question'1:'In'which'group'did'you'think'the'conversation'was'most'fluent?'
Most fluent Baseline Manipulation
Victim (N=8) 37.5% 62.5%
Participant (N=16) 37.5% 62.5%
Table'4:'Question'2:'In'which'group'did'you'feel'like'participating'more'in'the'conversations?'
Participating Baseline Manipulation
Victim (N=8) 75% 25%
Participant (N=16) 56.3% 43.8%
Table'5:'Question'3:'In'which'group'was'the'decision'process'easier?'
Easier Decisions Baseline Manipulation
Victim (N=8) 62.5% 37.5%
Participant (N=16) 62.5% 37.5%
Participants
The other questions in the questionnaire focused on specific people in the experiment to show whether people preferred speaking to the victim in the conversation or the other participant or whether they would reward themselves for the experiment. In the manipulation condition, the participants mainly preferred talking to the other participants (68.8%) instead of the victims (31.3%) (Table 6). When the participants were asked who they thought earned the gift coupon, in the manipulation condition 37.5% of the participants thought they should get the coupon themselves and only 25% of the victims thought the same and chose themselves to earn the coupon (Table 7). However with P=0.667 (N=24, α= 0.05) this result is not significant. In the baseline condition 31.2% of the participants thought they should receive the coupon and 12.5% of the victims thought they should receive the coupon (Table 8). However with P=0.621 (N=24, α= 0.05) this result is not significant.
Table'6:'Question'5:'In'Communication641322Bot,'who'did'you'enjoy'talking'to'the'most?'
Enjoy speaking to Participant Victim
Participant (N=16) 68.8% 31.3%
Table'7:'Question'10:'What'if,'in'Communication125149Bot,'you'can'give'one'of'the'Bol.com'gift'coupons'of'25'euro' to'the'person'that'was'involved'most'in'the'conversation.'Who'do'you'believe'earned'the'gift'coupon?'
Gift Coupon Other Me
Victim (N=8) 87.5% 12.5%
Participant (N=16) 68.8% 31.2%
Table'8:'Question'11:'What'if,'in'Communication641322Bot,'you'can'give'one'of'the'Bol.com'gift'coupons'of'25'euro' to'the'person'that'was'involved'most'in'the'conversation.'Who'do'you'believe'earned'the'gift'coupon?'
Gift Coupon Other Me
Victim (N=8) 75%% 25%
Participant (N=16) 62.5% 37.5%
Other questions
To check whether the participants had an idea what the experiment was about, we asked some open questions.
Question 12: What do you think the purpose of this research was?
18 people thought the research was about how people make decisions with strangers about difficult ethical topics on a chat medium. One participant thought it would have to be about group pressure and making decisions. One participants thought it was a ‘Turing test’ for the bot. This participant also believed he was not chatting with people but with two bots. Five people didn’t know or didn’t fill in this question. Nobody guessed something about messages not being sent to others.
Question 13: We are working on optimizing the bots of the conversations and to eliminate any flaws. Have you discovered anything that you think we should know of?
Most of the participants (12) didn’t answer this question or didn’t have anything to say. Eight people thought the new trolleys, that came in at 6:00 am or after a reset, were annoying. One person said he thought that the dilemmas were a bit unclear sometimes and they would mostly discuss the meaning of the dilemma instead of the options. Other participants said they thought they were talking to bots, they sometimes couldn’t decide or that they sometimes got an error message.
Question 14: Is there anything else you would like to share about the experiment or the questions?
Again, most of the participants (15) didn’t answer this question or just answered with ‘no’. Five people said they really enjoyed the experiment. One participant suggested the experiment should be shorter the next time and two others said they had other things on their mind towards the end of the experiment or they resisted the experiment a bit, because of the duration.
4.3.2.! Questionnaire Part 2
These questions were asked after the end of the entire experiment, which also revealed the true purpose of the experiment. Of both the victims and the participants, 25% of the people thought they were in an experiment where messages were deleted (Table 9). However, with P=0.869 (N=24, α= 0.05) this result is not significant. After reveiling the purpose of the experiment, only 12.5% of the victims and participants thought their own messages were deleted (Table 10). With P=1.00 (N=24, α= 0.05) this result is not significant. Most people thought there was a delay in the messages and just a few thought the nicknames were swapped in the experiment.
Table'9:'Question'1:'We'tested'three'typed'of'network'errors.'Which'one'do'you'think'happened'in'the'conversation?'
Type of network error Delay in messages Deleted messages Nickname swap
Victim (N=8) 62.5% 25% 12.5%
Participant (N=16) 68.8% 25% 6.3%
Table'2:'Question'2:'In'Communication641322Bot'there'was'a'manipulation'where'some'utterances'by'one'person' were'not'send'to'the'other'two.'Which'role'do'you'think'you'had?'
Role Other messages weren’t send My messages weren’t send
Victim (N=8) 87.5% 12.5%
5.! Discussion
We found no statistical evidence that the manipulations had an effect on the interaction. However, there is a hint of effect in the objective measurements. We found that victims were less active in initiating decisions, they used more characters per turn, from day 2, and they had a slightly higher contribution to the conversations on day 2 and 3. Additionally, in the subjective measurements, it appears that the victims preferred participating in the baseline conversation, where they also thought the deciding was easier. Since the dataset is only eight conversations, this allows us to examine in detail each intervention, and determine whether participants did in fact notice the shadowbans, and whether they were affected by them.
5.1.!Sequences of turns
As explained before, a shadowban can have two different effects; from the perspective of the victim, they don’t receive a response to their first pair part of an adjacency pair and from the perspective of the others in the conversation, they don’t see the second pair part of the adjacency pair that the victim has produced. There is a slight difference between both perspectives. For the victim, the turn is not responded to because the turn is never sent to the others. In the perspective of the others, the participants might be given the impression that the victim hasn’t responded, even if they have. In the data, different types of sequences were identified and they are analysed separately. Figure 13 shows all the shadowbans that occur and categorizes them into a number of distinct sequence types. Each block in Figure 13 represents a turn. S1 through S23 corresponds with the numbers of the shadowbans, which are also visible in Appendix 13. The sequences are identified in the data by finding a shadowban ( V ) with it’s previous and subsequent turns. In order to see whether the shadowban is part of multiple turns by V and how many turns previous to the shadowban, and after the shadowban, one of the non-shadowbanned participants was active, the sequences show the preceding turn by one participant (P), and the subsequent turn by the same participant (P), or the other non-shadowbanned participant (Po). Everything that comes before these sequences can be any turn by V or P, but without a shadowban, which is why these turns are not included in this figure.
For example, in S1, the shadowban is in between turns by P and Po, so in between turns from both participants. The victim has two turns between the turns by P and Po and one of these is shadowbanned. The types of sequences are sorted by the amount of turns by V that the shadowban is part of. In some cases, the shadowban took place right after a new game was started or right before a new game. On the far left, in S16, for example, there was a new trolley game, the next turn was by the victim, and this was immediately shadowbanned, and after this, there was already a new game. In S14, there was a turn by P, previous to the shadowbanned turn by V, and after the shadowban the people decided on the current trolley and a new game was started. On the far right, in S17, the shadowban was part of many turns by the victim. The victim sent eight turns in a row and the fifth turn was shadowbanned.
In the next section, the shadowbans are coded and categorized in order to examine the shadowbans in more detail.
5.2.!Coding of shadowbanned turns
Based on the shadowbans, a coding scheme was created to identify factors that amplify or dilute the effect of the manipulation (Table 11). First, the question is asked whether the shadowban is a single turn or if it is part of multiple turns by V. One might expect that a shadowban that is part of multiple turns should dilute any effect, since the victim is very active in their turns and having one of them missing would perhaps make the turns vague, but since there are still turns by the victim, it will not objectively show that the victim is less involved.
Then it is checked if the shadowban is a question. You might expect that when a question is shadowbanned, there is an expectation that an answer should follow. However, this answer will not come and there might be ways of the victim to get the others to answer the question.
Finally, it’s checked whether any of the preceding turns by P is a question. If this is the case, the shadowbanned turn could be a possible answer or possible second pair part, which will be missing. It can be expected that the victim seems less objectively involved, as they do not answer to the question, while, probably, the other participant does answer.
Table 11: Coding scheme shadowbans Coding scheme shadowbans
Coding category Characteristic
Is it a single turn or part of multiple turns? Single / Multiple Is it a question by the victim (QbV)? QbV / No QbV Is any of the preceding turns by P a question (QbP)? QbP / No QbP
This gives the following possible categories and the corresponding shadowbans from the data in that category (Table 12). All the shadowbans can be found in Appendix 13.
Table 12: Final categories of shadowbans
Category Shadowban numbers
Single, QbV, No QbP 7, 15 Single, QbV, QbP - Single, No QbV, QbP 23 Single, No QbV, No QbP 2, 3, 8, 9, 11, 12, 14, 16, 25 Multiple, QbV, No QbP 6, 19, 21 Multiple, QbV, QbP - Multiple, No QbV, QbP 1, 20 Multiple, No QbV, No QbP 4, 5, 10, 13, 17, 18, 22, 24
Most shadowbans are either a single turn or part of multiple turns by V, they are non-questions by the victim and there is no preceding question by P. However, since there are also shadowbans in the ‘Question by V’ category and the expectations for a reply are high with questions, the next part will first examine these situations.
5.2.1.! Shadowbanned questions (QbV)
There were five instances where the shadowban was a question; in two cases it was a single turn and in three cases it was part of multiple turns. It is interesting that in these sequences, the victim did not try to get the other participants to answer the question that was shadowbanned.
Table 13: Example of single question turn without answer from ban 7 (Single, QbV, No QbP) (S7)
Line Time Role Nickname Turn
1 6:00:04 New trolley is send:
‘029_ontdekker-dilemma_geen-chips-meer_geen-ijs-meer.png’ 2 7:38:45 P Lemi Ik zou niets doen. Dus dan zou de uitvinder van chips doodgaan.
3 8:48:22 V Haro Ik zou ook niks doen, maar dat heeft niets met mijn liefde voor chips of ijs te maken.
4 9:05:45 P Dazu /b
5 9:06:10 P Dazu Clicks option: geen-ijs-meer 6 9:07:40 V Haro Clicks option: geen-chips-meer
Voting is stopped due to participants not choosing the same option 7 9:08:07 V Haro Dazu we hebben jouw mening helemaal nog niet gelezen? 8 10:04:04 P Lemi Nee voor mij geldt dat ook
9 10:04:07 P Lemi /b
10 10:04:10 P Lemi Clicks option: geen-chips-meer 11 10:22:51 P Dazu Clicks option; geen-ijs-meer
