Name: F.R. de Jongh (Felix) July 22nd, 2016 StudentID: 10124470
Assignment: Master’s Thesis, Graduate School of Communication Course: Master’s programme Communication Science
Track: Political Communication Supervisor: Y. Lelkes
Political Discussion on Reddit
2 Abstract
Does the internet offer too many opportunities for users to filter out dissenting opin-ions? This question has been asked and answered by researchers many times over the past two decades, often leading to conflicting results. Using Habermas’ typology of the public sphere, Freelon (2010) created three models to which online discussion can adhere. Using his models, this research explores political discussion taking place on eight subreddits during the prelimi-nary elections leading up to the 2016 US general election. With 240 content submissions and 1137 comments quantitatively analyzed on each of the features Freelon conceptualized, it was found that users on reddit create their own communitarian or liberal-individualist echo cham-bers, but are also enticed to hold deliberative discussion on neutral ground. However, this neutral ground was also found to become the frontline of opposing viewpoints, with heated discussion devolving into flaming, name-calling and mud flinging. Evidence points toward proper moderation in order to keep discussion orderly and constructive.
Introduction
In discussing matters of general interest, citizens unconsciously create a public sphere able to formulate a public opinion, as Habermas stated in 1964 (Habermas, Lennox & Len-nox, 1974). This public sphere transcends the marketplace of ideas, by at the same time gen-erating and distributing culture (Boeder, 2005). At the time of Habermas’ publication, public bodies had grown to the extent specific media were necessary in order to deliver political news and current events to as many people as possible for them to become involved on an informed basis (Habermas, Lennox, & Lennox, 1974). These media (such as newspapers, tel-evision and radio) are slowly being replaced by the internet as it is a better medium to suit public needs such as variety and convenience (Gaskins & Jerit, 2012). These positives are not
3 without drawbacks, as they also make it easier for users to filter out dissenting opinions, creat-ing the conditions for an echo chamber if the user wants to (Sunstein, 2001).
By only reading slanted news, from which struggling media in a very competitive cli-mate gain profit, users only receive one perspective from which they derive their understand-ing of and attribute meanunderstand-ing to current events (Converse, 2000; Iyengar, Sood & Lelkes, 2012). Since both liberal and conservative media put out increasingly inflammatory articles, a divide is created strengthening party identification and increasing polarization (Baldassarri & Bearman, 2007). This polarization diminishes the public sphere’s ability to formulate a coher-ent public opinion and punishes those politicians willing to cut across party lines (Harbridge, 2015-05-04). The resulting gridlock leads to political alienation, opening the way for populist candidates promising impossible solutions to societies’ problems (Barr, 2009; Mudde, 2004), a process which is intensifying in the run up towards the 2016 US General Elections (Mans-bridge, 2016-11-03). Does online political discussion bring people closer together through constructive deliberation, or does it increase political polarization through selective exposure?
Theoretical Framework
A healthy democracy relies on five aspects: effective participation, voting equality, en-lightened understanding, control of the agenda, and the inclusion of (all) adults (Dahl, 1998). While these aspects require an institutional framework, it also requires those participating to be informed in order to achieve the best possible form of the public sphere through media, causing effective participation (Habermas, Lennox & Lennox, 1974). The knowledge gained from traditional media was enhanced and broadened by discussing political news with others, such as neighbors, co-workers, friends and/or family (Eveland, 2004). With the internet offer-ing people the opportunity to reflect upon political news on the spot, the need to discuss events ‘offline’ is satisfied. The internet opened up the status quo, enabling a multitude of
4 new initiatives to compete with established news outlets. In this competitive climate, outlets found that offering slanted and politically-biased news gained more attention and thus ad-revenue ensuring the survival of the outlet (Iyengar, Sood & Lelkes, 2012). In order to re-spond to people’s confirmation bias and preference for entertainment over information (Prior, 2005), outlets began reinforcing existing frames (Iyengar & Hahn, 2009) more focused on scandals and drama, making Republicans look ineligible to ‘liberal’ media and their audience, and vice versa (Bernhardt, Krasa & Polborn, 2008). And, as options for reading political news grow, people paradoxically stick with fewer outlets paralyzed by the amount of choice pre-sented to them (Iyengar & Hahn, 2009).
Selective exposure through partisan preference causes existing frames to deepen, facil-itating social extremism and political polarization (Bennett & Iyengar, 2008; Iyengar & Hahn, 2009). Whether the ensuing discussion enhances or destroys the deliberative aspect of online discussion is a disputed debate (Wright & Street, 2007). Sunstein envisions online communi-ties drifting apart, creating fierce ideological rivalries culminating in what he calls a ‘cyber-cascade’ (2001), while others see online discussion as an optional tool for deliberation from which democracy only stands to gain (Coleman & Gotze, 2001).
Freelon’s models of online discussion
Based on Habermas’ concept of the public sphere, a model of online political discus-sion has been created by Deen Freelon, an associate professor in the School of Communica-tion department at American University of Washington, DC. His model, devised in 2010, con-sists of three distinct types of online deliberation: the ‘liberal individualist’-, ‘communitari-an’- and ‘deliberative’-models of discussion. While these models have no overlap themselves, Freelon acknowledges that realistically no forum will specifically resemble only one type but will be a mixture of multiple models, the deciding factor to which model a forum adheres is
5 the amount and frequency of features found within discussion (Freelon, 2010).
Discussion within the liberal individualist model mostly out of personal expression and the pursuing of self-interest, combined with disinhibition caused with the anonymous and impersonal nature of the internet (Lapidot-Lefler & Barak, 2012). Four features pertain to this model: monologue, personal revelation, personal showcase and flaming. A monologue, Free-lon argues, is the antithesis to a healthy deliberative discussion. Psychologically, it stems from the desire to be heard. It entails an individual going on a rant about his personal preferences, often derailing the discussion into off-topic mud-flinging. The second feature, the personal revelation, is a participant in the discussion revealing personal information used as a source for argumentation. This could mean the disclosure of one’s profession, nationality, race, gen-der, age, or any other personal attribute which could be used in the context of the discussion to empower his contribution. The ‘personal showcase’ means a user is advertising his creation to the discussion platform, for instance promoting a blog post or video in order to receive acknowledgement from others. Flaming, the fourth feature of the liberal individualist model, are words written with a hostile intention of inflicting harm to an individual or organization to express discontent, fueled by frustration over one’s powerlessness (Freelon, 2010; Lapidot- Table 1.
Models of online democratic communication and their features (Freelon, 2010).
Model: Liberal individualist Deliberative Communitarian Indicator: Monologue
Personal revelation Personal showcase Flaming
Rational-critical argument Public issue focus
Equality
Discussion topic focus Inter-ideological response Inter-ideological question-ing Ideological homophily Mobilization Community identifica-tion Intra-ideological re-sponse Intra-ideological ques-tioning
6 Lefler & Barak, 2012).
The ‘communitarian’-model encompasses discussion focused on social cohesion and group identity over individual desires. The communitarian model is most at risk of becoming an echo chamber, since members within such a forum of discussion are encouraged to become part of the ‘hivemind’ (Sunstein, 2001). Five features are presented by Freelon and attributed to this model: ‘ideological homophily’, ‘mobilization’, ‘community identification’, ‘intra-ideological questions’ and ‘intra-‘intra-ideological responses’. Ideological homophily is best de-scribed by the hypothetical question “I just don’t understand why people could vote for X”. Lack of interaction with dissenting opinions causes a rift which removes any insight into why people with different preferences have these preferences, since ‘theirs are clearly better’. The second feature, mobilization, encourages users to take action small (‘retweet this’) or large (‘join us in protest’). A comment expressing community identification sees his identity as part of the larger community, exemplified by the use of ‘we’ when talking about the community. An intra-ideological question is a question directed inwards to the own ideological group, while an intra-ideological response is a response to said question (Freelon, 2010).
The third and final model is the one Habermas would agree is the best form of online discussion for the public sphere. The deliberative model values any argument from any ideol-ogy, as long as it is well thought out, coherent and contributing to the overall discussion and atmosphere. It’s six features are the ‘rational-critical argument’, ‘public issue focus’, ‘equali-ty’, ‘discussion topic focus’, ‘inter-ideological questions’ and ‘inter-ideological responses’. The rational critical argument is seen as the evaluation of content free from ideological pref-erences. The public issue focus is seen when users overcome their ideological differences in order to serve a purpose they all support. The third feature, equality, pertains to the level play-ing field the platform offers to users, and whether all users receive the same exposure. A dis-cussion topic focus means that, whenever disdis-cussion threatens to derail, a user directs
atten-7 tion back to the original point of the discussion. Inter-ideological questions are questions aimed at people from other ideologies posed to learn from other’s viewpoints, while inter-ideological responses are the answers given (Freelon, 2010).
With online news-use becoming more widespread and the supply more diffuse, link-aggregating websites such as Yahoo! News, Google News and reddit have risen in popularity. These websites pose a possible solution to the abundance of choice by offering a wide range of sources and perspectives on one website (Lee & Chyi, 2015). While the former two offer non-curated results from pre-approved sources, on reddit the users themselves submit content found elsewhere which is then ‘curated’ by the users by way of voting on its worth. Content that receives lots of ‘upvotes’ rises to the top, increasing visibility to other users. It also offers ‘redditors’ the opportunity to discuss the content, which could then also be voted upon by the community (“the conversation starts on Reddit”, n.d.; Ovadia, 2015). Users are incentivized to post quality content since the upvotes received for content and comment submissions adding up to a ‘karma’-count, visible to everyone providing a sense of progression and worth to users (Ovadia, 2015).
Reddit’s success had led to it becoming the biggest ‘forum’ for online discussion, with eight billion monthly page views from 234 million unique visitors (“reach millions”, n.d.). In 2013, six percent of all American internet-using adults visited reddit (Duggan & Smith, 2013). More recent numbers state seventy percent of Redditors get their news directly from Reddit and 45% learn about new developments in the lead up to the 2016 US General Election on a weekly basis (Barthel, 2016-05-26). During the preliminary elections, polarization ramped up both between and within parties, an effect which was multiplied in discussion online (Richey, 2016-06-07).
8 Though reddit alleviates a user’s need to look for content themselves, it still qualifies for the same pitfalls. Reddit is not one big forum, but a multitude of ‘subreddits’ created by users dedicated to a topic or purpose to which users can freely subscribe or unsubscribe (Ova-dia, 2015). This makes it easier to create a small community of likeminded people, and, with the karma incentive, users are more enticed to submit content or write a statement in agree-ment with the popular consensus within a particular community. Reddit’s versatility means it has become a microcosm of everything the internet has to offer within the confines of a single website and community. This has led to many different political subreddits, with some meant to discuss general political events and trends, while others are dedicated to specific candidates, ideologies, or political parties. The first hypothesis will investigate whether this has an effect on the type of online discussion a subreddit can be classified under, since a ‘general’ type of discussion is meant to attract a multitude of viewpoints;
H1: A subreddit aimed at general political discussion is more deliberative in nature
than other subreddits.
The second hypotheses supposes whether right-wing subreddits score higher in the liberal individualist category, while left-wing subreddits achieve higher scores for discussion of communitarian nature than their right-wing counterparts, based on the findings that liberals place more value on horizontal relations (working together) and conservatives on vertical re-lations (authority) (Janoff-Bulman, 2009);
H2: Liberal subreddits are more communitarian in discussion than conservative
sub-reddits, which are more liberal-individualistic in nature.
The third hypothesis is derived from Pew Research categorization of subreddits. In a 2015 automated content analysis consisting of 350,000 comments, it was found that 60% of prelim-inary candidate’s names were mentioned in political subreddits. Pew categorized these
politi-9 cal subreddits in four categories: general political discussion (37% of mentions), candidate-specific discussion (18%), ideology-candidate-specific (5%), and party-candidate-specific (1%) (Barthel et al, 2016-02-25). Using this categorization, the following hypothesis will look for differences in discussion;
H3: The difference in discussion purpose (general, candidate-, ideological- or
party-specific) of subreddits predicts the type of discussion found within the subreddit.
While the creator of a subreddit has the freedom to create a distinct appearance and dictate ruleset for their subreddit, they are still subjected to Reddit’s general rules and operate on the same voting algorithm, making the core experience the same site-wide (Ovadia, 2015). Free-lon acknowledges that his model only reflects on what is written, hinting that “the forums’ name, ‘about’ page, or mission statement (if applicable), may also hold relevance for the dis-cussion model it tends toward” (2010: p.15). Wright & Street firmly state that the design and purpose of the discussion forum is deciding on its contributing factor for deliberation (2007), providing the basis for the third hypothesis:
H4: Moderator-created characteristics for the subreddit encourage users to adhere to a
specific type of discussion.
While in the early days of the internet developments happened at a fast pace, with new web-sites quickly replacing older webweb-sites, Michels ‘iron law of oligarchy’ (1911) eventually got a grip on the internet as well. Facebook has become the Western norm for social profiles, Insta-gram for photo-sharing. Twitter for news-sharing. Tumblr for micro-blogging, and reddit, it seems, for news-discussion. Since websites’ popularity a decade ago was often no more than a few years, creating a case study did not hold a lot of relevance. But the past decade has prov-en that some websites can withstand the test of time and actually increase in popularity, mak-ing research on these websites scientifically relevant. Researchmak-ing how news is received and
10 discussed on a website like reddit holds value for society as well, politicians and media may find the results of the hypotheses in this thesis useful in creating content and dealing with an audience over ten million strong. While some research has been done to analyze voting be-havior on reddit, the discussion itself remains unexplored. This thesis aims to thread the water on this topic, paving the way for further research.
Method
The unit of analysis is content posted to reddit, on various subreddits. These are cate-gorized into two separate groups: the first being the links published on reddit by its users, the second being the comments posted under these links. Of each subreddit, the top three highest upvoted submissions in a week (from Sunday 21:00h UTC to Sunday 21:00 UTC) were se-lected for a quantitative content analysis. Of these sese-lected submissions, the five highest upvoted top level comments were then analyzed in the same manner. This was repeated ten times, with the first week of analysis starting March 20th, 2016, and the last week of analysis ending on May 29th, 2016. This resulted in 240 content submissions in the dataset, and a fur-ther total of 1137 top-level comments to these submissions.
The first subreddit selected for analysis is the /r/politics subreddit. This is the main discussion board for US politics, and the largest political subreddit by number of subscribers on reddit itself. This subreddit will represent the ‘general’ political discussion category. Three further subreddits selected for analysis are subreddits pertaining to the three main candidates at the time of data-gathering, Bernie Sanders (/r/sandersforpresident), Donald Trump (/r/the_donald), and Hillary Clinton (/r/hillaryclinton). While at the time both Cruz and Kasich still ran for the Republican nomination, their respective subreddits had less than a thousand subscribers and lacked enough weekly content to be included for analysis. Two ide-ological subreddits were selected to reflect the liberal/conservative dichotomy, with r/social-
11 lism representing the liberal (left) camp, and /r/libertarian the conservatives (right). Finally, two subreddits bearing the names of the main political parties in the US’ two-party system were selected, /r/democrats and /r/republican. Some exploratory facts about these subreddits can be found in table 2.
Data gathering was done by first copying the URL of the submission, selecting the week the submission was made in, the day of the week, the position of the link (1st – 3rd), the subreddit it was posted to, and whether it was a content or a comment submission. The first variables measured for content submissions were exact measures, such as the overall karma score, the username of the user submitting the content, the total amount of comments on the content, and the overall upvote percentage. Next, the tone of the submission was selected from three options: negative (-1), neutral (0) or positive (1). A negative submission would use negatively associated frames and words, while a positive submission would use positive ones. A neutral submission was purely factual, or used the title of the article being posted, meaning the user had no input on the title.
Table 2.
Subreddit specifics for survey (reddit, 2016).
Category Subreddit Established Leaning Subscribers* Growth** General Politics 06/08/2007 Center 3,095,660 0.18% Candidate Sanders 05/12/2013 Liberal 236,292 1.35%
Trump 27/06/2015 Conservative 141,983 11.83%
Clinton 04/05/2012 Liberal 17,122 19.01%
Ideology Socialism 29/06/2008 Liberal 59,913 2.43% Libertarian 25/01/2008 Conservative 132,281 2.08%
Party Democrats 04/10/2008 Liberal 18,876 1.17%
Republican 27/10/2012 Conservative 20,621 0.77% *data taken from respective subreddits on 11-5-2016, not indicative of active users. **growth is calculated in a one month period, between 11-4-2016 and 11-5-2016.
12 The type of content was coded from five categories, all with two sub-categories. These categories were discussions (ideological discussion, or event discussion), articles (news or opinion), pictures (entertainment or informative), questions (intra-ideological or inter-ideological) and video’s (entertainment or informative). These categories were decided upon within the limits of possibilities offered by reddit, but one submission could have multiple content-types. A picture showing a political convention could be both an informative picture and an event discussion, and an inter-ideological question can also be an ideological discus-sion. In such cases, both content types were selected. Then, content was controlled for the topic it discusses. The first three selectable topics are the three main candidates: Clinton, Trump and Sanders. Secondly, both parties (DNC and GOP) were given as separate options to select from. If a piece of content could not fit in either of these topics but was still about the elections, a catch-all topic ‘election’ was created. Within this topic, content about third-party candidates could still be coded. Lastly, ‘other’ was created as an option in order to code for any content that did not relate to the election in any way.
Comments were first coded for their position (1st – 5th), sorted by the highest upvoted comments first. The karma score and reddit username were then logged, as well as the word count. Then, tone was coded to reflect the user’s sentiment while writing his comment, whether he was positive (1), neutral (0) or negative (-1). Another variable measured the con-tribution to the overall discussion, whether it was disruptive (-1), neutral (0), or constructive (1). A disruptive comment would derail discussion, while a constructive comment would con-tribute, such as by posting a personal reflection on the content or by providing additional in-formation. Freelon’s three models (liberal individualist, communitarian and deliberative) were included, with each of the features selectable per model. Only ‘equality’ was not coded as such, since the level of involvement could not be ascertained from the five comments coded per submission. Finally, an option was included to code ‘deleted’ comments, whether the
13 comment was removed by moderators (indicated with ‘removed’ and hidden username), re-moved by the user (‘deleted’ comment, hidden username) or if the user had deleted his ac-count (username ‘deleted’) in order to code for missing comments.
While some comments fell under clear categories and were being coded as such, some adjustment of the model was necessary to effectively code the discussion taking place on red-dit, especially pertaining to specific often recurring comments. An inside joke, for instance, was coded as both an intra-ideological response (since it repeats an often heard narrative that is known by the user to be appreciated by other users) and community identification (since it acknowledges the existence of a specific ‘culture’ within the subreddit). And, on the basis of its nature, the inside joke can also be coded as flaming if it is mean-spirited, and as mobiliza-tion if it encourages users to start a chain of comments.
The results of the individual characteristics coded from the comments were first com-bined for each model and divided by the number of characteristics, giving a 0 (‘no feature of the model was found’) to 1 (‘every feature of the model was found’) creating three dependent ratio-variable’s, one for each model which could then be measured for each subreddit. The choice of dividing by the number of features was made to minimize the effect the difference the four-point scale causes in context with the five-point scales.
For the fourth hypothesis, a more qualitative approach was needed. Each subreddit was subjected to a qualitative analysis. This analysis analyzed anything but the content posted by the users. The subreddit’s visual appearance and pictures used, the mission statement, the specific ruleset created by the moderators, and any extra information posted in the sidebar. Every element would then be judged on the basis of the three categories as imagined by Free-lon. Some of these elements are harder to code then others, so only elements that are undenia-bly adhering to a certain feature would be coded as such. In order to find whether the creator’s
14 vision when creating the sub influences the debate found, will be tested with the data found through the quantitative analysis of content and comments.
Results
After finishing data collection, a quick exploration of the dataset was performed be-fore starting on the analysis for the hypotheses by running a descriptive statistics test through SPSS, the results of which can be found in table 3. Trump’s subreddit performed high in the liberal individualist category (M = .235), while Sanders’ and Clinton’s subreddits achieved high scores for the communitarian model (M = .243, M = .224). The deliberative score for the general /r/politics subreddit was strongly above average (M = .248), while Trump’s was far below average (M = .049) (avgM = .135).
Table 3.
Descriptive statistics of the scores for each model per subreddit.
Liberal Individualist Communitarian Deliberative
Subreddit n M SD M SD M SD Politics 150 .153 .171 .171 .169 .248 .213 Sanders 150 .158 .149 .243 .183 .112 .165 Clinton 149 .148 .148 .224 .177 .122 .148 Trump 150 .235 .179 .188 .158 .049 .111 Socialism 150 .143 .157 .216 .153 .124 .153 Libertarian 150 .155 .173 .203 .178 .124 .162 Republican 132 .136 .173 .177 .139 .168 .180 Democrats 106 .172 .156 .196 .172 .138 .184 Total 1137 .163 .166 .203 .168 .135 .174
15
Hypothesis 1
To test the first hypothesis, the ‘politics’ subreddit (‘general’) and all other subreddits were coded into a new independent variable consisting of two categories. Since this gives a large disparity between results (‘Politics’ n = 150, ‘Other’ n = 987) and the data being non-normally distributed, an independent samples t-test cannot be performed as it would fail the assumptions of Levene’s test of equality of variances between these two groups due to their sample size difference. Instead, the non-parametric Mann-Whitney U test was used to test the results for significant differences in discussion taking place in both categories. Distribution of the deliberative score of discussion for general and other were not similar, as was assessed trough a visual inspection. There was, however, a statistically significant difference in scores for deliberative discussion between general (mean rank = 743.47) and other (mean rank = 542.49), U = 47,855, z = -7.74, p < 0.01.
To test these results, a weighed stratified sample was created from the dataset, using all 150 cases from the /r/politics subreddit and 150 randomly selected cases from the ‘other’ category. Descriptive statistics of this sample can be found in Appendix A. Using an inde-pendent samples t-test to find differences between these groups in the indeinde-pendent variable, it was found that the mean deliberative score was .11 higher for politics, 95% CI [.07 to .16] than it was for ‘other’. The difference for deliberative discussion was still statistically signifi-cant in this sample, with t(289,984) = 4.97, p < .01. Again, however, Levene’s test for the homogeneity of variances was not met for this sample, with p = .04, which is under the re-quired p = .05 necessary to interpret the data. By performing the same analysis for the com-munitarian model, Levene’s test was met (p = .09) with a mean difference of -.03 for politics, 95% CI [-.07 to .00] lower than other subreddits. This means that the politics subreddit has less communitarian features than the other subreddits, but due to its level of significance (p = .07) this effect is close to, but not significant.
16 Looking at each feature of the deliberative model individually with this weighed strati-fied sample using the independent t-test, all but one fail Levene’s test. The discussion topic focus scores a p = .07 in the test for equality of variances, and is significant with p < .01. The mean difference for discussion topic focus is .173, 95% CI [0.61 to 0.29]. For the communi-tarian model, neither of the five features passed either the Levene’s test or showed a statisti-cally significant relation. The same result was found for the liberal individualist model.
Hypothesis 2
In order to test the first part of the second hypothesis, whether liberal or conservative subreddits differ in their discussion, another weighed stratified sample was created since the liberal subreddits outnumber the conservative subreddits in the dataset four to three. /r/Politics was left out of this sample, and the 555 cases of the liberal subreddits were brought down to the 432 of the conservative, creating a sample of 864 cases of which the descriptives can be found in Appendix A.
Subjecting this sample to another independent samples t-test showed a significant re-sult for the liberal individualist model (p = 0.046) with a mean difference of -0.023 at a 95% confidence interval (lower -0.05, upper 0.00), but this test failed Levene’s test meaning that this significant result cannot be interpreted as such. Results for the communitarian model gave a Levene’s score of p = .25 passing Levene’s test, and a significance of p = .01. The mean difference was 0.03 (95% CI), with the lower value being 0.01 and the upper 0.05. The delib-erative model did not show any significant difference between liberal and conservative sub-reddits. Looking at each feature of each model individually, only mobilization (p = 0.045, Mdif = 0.051, 95% CI at 0.00 to 0.10) and monologue (p = 0.016, Mdif = 0.065, 95% CI at -0.12 to -0.01) showed significant results, but neither of these features passed Levene’s test.
17
Hypothesis 3
To test whether the discussion topic causes a difference in the discussion model, a third sample was created. Using four categories, ‘general’ (/r/politics), ‘candidate’ (/r/sandersforpresident, /r/Hillaryclinton and /r/the_donald), ‘ideology’ (/r/socialism and /r/libertarian) and ‘party’ (/r/democrats and /r/republican), a sample was created consisting of 600 cases, each category being represented by 150 randomly selected cases. Sample charac-teristics of this sample can also be found in Appendix A. Using a data exploration it was found that this sample had 27 outliers, four in both the general and party categories, and elev-en in both candidate and ideology categories. These outliers were kept within the dataset since the data collection method did not leave room for errors in collecting.
Using a One-Way ANOVA analysis with ‘subreddit type’ as independent variable with four categories and the three cumulative model scores as dependent variables, it was found that for both liberal individualist and communitarian models, the assumption of homo-geneity of variances was met (p = 0.13 and p = 0.97), while the deliberative model did not meet the assumption. Liberal individualist discussion was not significant (p = .64), but the communitarian discussion type showed significance between groups (p = .047). Using Tuk-ey’s analysis within the ANOVA to discover any significant results within the particular sub-reddit types, no significant results were found within the liberal individualist and communitar-ian models (closest to significance being the communitarcommunitar-ian level of significance for ‘general x ideological’, with p = .10), while in the deliberative model nearly all (except ‘candidate x ideology’ and ‘ideology x party’) combinations showed significant results but since there was no homogeneity of variances, these results cannot be interpreted as such. Performing the same analysis for each features individually netted the same results, many features showed a signif-icant difference, but neither met the assumption of homogeneity of variances.
18
Hypothesis 4
Before analyzing the subreddits itself, a non-parametric Kruskal-Wallis H test was run for all eight subreddits as the independent variables, to measure their score for each model (dependent variable). Median CWWS scores were similarly distributed for the communitarian model (Mdn = .20), while for the liberal individualist model only /r/socialism and /r/republican deviated from the common median (Mdn = .13, Mdn = .00, and Mdn = .25 re-spectively). For the deliberative model, /r/politics and /r/republican similarly deviated from the common median (both Mdn = 0.20 versus Mdn = 0.00). Differences between subreddits were all significant, with the liberal individualist model achieving χ2(7) = 30,016, p < .001, the communitarian model had χ2(7) = 18,356, p = .01, and the deliberative model had χ2(7) = 86,982, p < .001.
Pairwise comparisons as a post hoc analysis revealed statistically significant differ-ences in CWWS scores in the Liberal individualist model for every subreddit compared to /r/the_donald (mean rank = 692.76), except /r/democrats (mean rank = 593.01) (p = .225). For the communitarian model, only the difference between /r/politics (mean rank = 558.88) and /r/sandersforpresident (mean rank = 634.02) showed a significant result (p = .008). The delib-erative model, however, showed significant results for every subreddit compared to /r/the_donald (mean rank = 566.29) (p < .05), and any subreddit compared to /r/politics (mean rank = 743.47) (p < .05).
Looking at the subreddit’s characteristics, /r/the_donald has four distinctly liberal in-dividualist features, two communitarian, and one deliberative. /r/politics on the other hand had four distinct deliberative features, and none of the other models, while /r/sandersforpresident had three communitarian features and none pertaining to the liberal individualist or delibera-tive models. Finally, /r/democrats had three communitarian, two deliberadelibera-tive and no liberal
19 individualist features. With the exception of /r/democrats, all subreddits are congruent with their subreddit specific look-and-feel when compared to their respective mean ranks.
Discussion
As Freelon (2010) stated, no forum perfectly aligns with only one model of discussion. This was found in the data as well. Some subreddits had high scores for one specific type of discussion, while other’s had low scores for one specific type with scores generally the same for the other two models. As expected, the /r/politics subreddit (where all manner of prefer-ences come together to discuss political events and news) scored significantly highest in the deliberative model of discussion and scored below average for both the communitarian and liberal individualist models, but these results showed no level of significance. /r/sandersforpresident, /r/hillaryclinton and /r/socialism scored high for the communitarian model, but /r/democrats did not. This could be explained by the /r/democrats becoming a plat-form where Sanders and Clinton supporters came together to plat-form a battlefield over the ques-tion which of the two candidates should receive the nominaques-tion to run as the democratic can-didate in the 2016 general election, showing instead above average scores in the liberal indi-vidualist category caused mainly by the level of flaming found in the discussion (Appendix B).
As for the liberal/conservative-dichotomy, conservative subreddits were significantly more liberal individualist in nature. A big part, however, is the inclusion of /r/the_donald within the conservative category. Donald Trump’s subreddit, /r/the_donald, had by far the highest score for the liberal individualist model. This was caused in most part because of the level of flaming encountered within the subreddit (Appendix B), but also the level of ‘mono-logue’. Many users wrote in all-capitalization letters and bold text, expressing the desire to be heard. This was amplified by the moderator’s use of flaming in the sidebar, such as calling
20 Sanders’ supporters ‘BernieBots’. Dissenters are actively discouraged from posting within the subreddit by the subreddit rules, preventing deliberative discussion by creating a bubble filled with like-minded individuals. If this subreddit were subtracted from the conservative catego-ry, discussion would not have showed significant difference with the liberal subreddits.
The third hypothesis showed that candidate-specific discussion had a significant lack of deliberative discussion, instead showing a significant higher level of communitarian dis-cussion. Since the purpose of this discussion on these subreddits is the most specific (one per-son), this can be explained through common sense. People who heavily favor a specific can-didate are more likely to discuss said cancan-didate in a specific subreddit. Furthermore, the sub-reddit is used a means to mobilize each other for voting in preliminary elections, and to coor-dinate ‘phone-banking’ (calling voters to cast their ballot). This was especially true for /r/sandersforpresident, with each state caucus being accompanied with attempts to mobilize users to contribute to the campaign.
This leads to the final conclusion based on the last hypothesis. Significant results can best be explained by the purpose of the subreddit. ‘Sanders for President’ as a name for a sub-reddit has an inherent mobilizing message purposefully created in order to lead a grass-roots campaign for Bernie Sanders. The /r/politics subreddit was created as a place for general US political discussion, and the resulting deliberative discussion found can be in part explained by the subreddit rules, but also the legion of moderators (38 in total) enforcing those rules. Furthermore, the subreddit has existed for a long time, so any newcomers are expected to ad-here to the rules. Newer, or less strictly moderated subreddits, are still developing their specif-ic subreddit culture. /r/the_Donald’s culture, however, more or less reflects Trump’s ‘crusade’ against political correctness, making the subreddit a screaming-contest of ever increasing lev-els politically incorrect statements in order to get attention through shock-value.
21 In order to have a healthy discussion on reddit, clear rules should be created and should be enforced by the moderators to prevent discussion spiraling out of control. While users do not have to interact with dissenting opinions on reddit if they so desire, they will eventually represent and defend their viewpoints in a deliberative environment if they feel attacked. While reddit is not the best platform for quality discussion since popular opinions will eventually receive the highest visibility, its fragmented nature through subreddits offers any user to find a political subreddit that suits their preference. There does, however, lie a danger in this in that if a user excludes himself from dissenting opinions, he/she will unknow-ingly gain a warped vision on reality. Whether reddit creates an echo chamber is up to the users themselves. Reddit creates a framework for discussion, what the user does with it is up to him or her.
Limitations
As became apparent during analysis, many of the variables failed to pass Levene’s test for the assumption of homogeneity of variances, which meant that most significant results were caused by variances within the dataset. This posed a problem, which could be solved with non-parametric tests, but this solution would diminish the explanatory power of the re-sults. Instead, random samples were used to control for population sizes, but this meant that the population size was much smaller than originally intended.
Another limitation came from the way the dataset was created while coding. Since posts and comments were coded separately, there was no way to link comments to posts di-rectly without manually entering post identity into comment cases. While manually entering the data was possible, this conclusion was made too late and was undoable due to time con-straints. Hypotheses could have gone more in-depth into the dataset if, for instance, the type of post (whether it is a discussion of an event, or an informative video) elicited different forms
22 of discussion. Or, whether the tone of the post (negative, neutral or positive) caused com-ments to significantly alter their tone as well. Further research could manually enter this data to see whether this is the case or not.
Coding itself was done within a time frame of seven days, during eight hour shifts by one coder. Coder fatigue became an issue, with some missing data having to be added retroac-tively. Furthermore, the time frame gave a learning effect which meant that the way cases were coded on the first day might not have been coded the same way on the last. An intercod-er reliability test proved a meagintercod-er .71, but this could also be explained by the complexity of reddit’s inner workings. The person contacted to perform intercoder reliability had no previ-ous knowledge of Freelon’s model, nor the same level of hands-on experience as the author with reddit. Intercoder-reliability could have been improved upon by finding a more experi-enced reddit user, but due to the aforementioned time constraints this luxury was not applica-ble to this study.
This exploratory study of discussion on reddit, with a model which is yet to be gener-ally adopted among the scientific community, could be used as a starting point for future re-search. Whether this happens is up to future technological developments, and whether reddit is able to maintain its continuous rise in use among the public. While their statistics show a stable rise in usage, if a better alternative arises offering users the same features and more, popularity could quickly fall. Were that to happen, this study is nothing more of snapshot of the past, without any implications for future online discussion. The value of this study, while initially not very strong to begin with due to the lack of homogeneity of variances, is depend-ent on as of yet unknown future developmdepend-ents. Only time will tell.
23 References
Baldassarri, D., & Bearman, P. (2007). Dynamics of political polarization. American
socio-logical review, 72(5), 784-811.
Barr, R. R. (2009). Populists, outsiders and anti-establishment politics. Party Politics, 15(1), 29-48.
Barthel, M. (2016, May 26th) How the 2016 presidential campaign is being discussed on Red-dit. Retrieved from http://www.pewresearch.org/fact-tank/2016/05/26/how-the-2016-presidential-campaign-is-being-discussed-on-reddit/
Barthel, M., Stocking, G., Holcomb, J., & Mitchell, A. (2016, February 25th) 2. Discussion mentioning the 2016 presidential candidates was vigorous in run-up to primaries, but did not dominate Reddit. Retrieved from http://www.journalism.org/2016/02/25/discussion- mentioning-the-2016-presidential-candidates-was-vigorous-in-run-up-to-primaries-but-did-not-dominate-reddit/
Bernhardt, D., Krasa, S., & Polborn, M. (2008). Political polarization and the electoral effects of media bias. Journal of Public Economics, 92(5), 1092-1104.
Bennett, W. L., & Iyengar, S. (2008). A new era of minimal effects? The changing founda-tions of political communication. Journal of Communication,58(4), 707-731.
Boeder, P. (2005). Habermas’ heritage: The future of the public sphere in the network society. First Monday, 10(9).
Coleman, S., & Gotze, J. (2001). Bowling together: Online public engagement in policy de-liberation (pp. 39-50). London: Hansard Society.
Converse, P. E. (2000). Assessing the capacity of mass electorates. Annual Review of Political
Science, 3(1), 331-353.
Dahl, R.A. (1998). On Democracy. Yale University Press.
Duggan, M., & Smith, A. (2013). 6% of online adults are reddit users. Pew Internet &
Ameri-can Life Project, 3.
Eveland, W. P. (2004). The effect of political discussion in producing informed citizens: The roles of information, motivation, and elaboration. Political Communication, 21(2), 177-193.
24 Freelon, D. G. (2010). Analyzing online political discussion using three models of democratic
communication. New Media & Society, 20(10), 1-19.
Gaskins, B., & Jerit, J. (2012). Internet News Is It a Replacement for Traditional Media Out-lets?. The International Journal of Press/Politics, 17(2), 190-213.
Habermas, J., Lennox, S., & Lennox, F. (1974). The public sphere: An encyclopedia article (1964). New German Critique, (3), 49-55.
Harbridge, L. (2015, April 5th). Congress is more bipartisan than you think. Retrieved from https://www.washingtonpost.com/blogs/monkey-cage/wp/2015/05/04/congress-is-more-bipartisan-than-you-think/
Iyengar, S., & Hahn, K. S. (2009). Red media, blue media: Evidence of ideological selectivity in media use. Journal of Communication, 59(1), 19-39.
Iyengar, S., Sood, G., & Lelkes, Y. (2012). Affect, not ideology a social identity perspective on polarization. Public opinion quarterly, 76(3), 405-431.
Janoff-Bulman, R. (2009). To provide or protect: Motivational bases of political liberalism and conservatism. Psychological Inquiry, 20(2-3), 120-128.
Lapidot-Lefler, N., & Barak, A. (2012). Effects of anonymity, invisibility, and lack of eye-contact on toxic online disinhibition. Computers in human behavior, 28(2), 434-443. Lee, A. M., & Chyi, H. I. (2015). The rise of online news aggregators: Consumption and
competition. International Journal on Media Management, 17(1), 3-24.
Mansbridge, J. (2016, March 11). Three reasons political polarization is here to stay. Re-trieved from: https://www.washingtonpost.com/news/in-theory/wp/2016/03/11/three-reasons-political-polarization-is-here-to-stay/
Michels, R. (1911). Zur Soziologie des Parteiwesens in der modernen Demokratie. Untersu-chungen uber die oligarchischen Tendenzen des Gruppenlebens. Leipzig: Klinkhardt. Translated in 1915 as Political Parties: A Sociological Study of the Oligarchical Tendenci-es of Modern Democracy.
Mudde, C. (2004). The populist zeitgeist. Government and opposition, 39(4), 542-563.
Ovadia, S. (2015). More Than Just Cat Pictures: Reddit as a Curated News Source. Behavior-al & SociBehavior-al Sciences Librarian, 34(1), 37-40.
25 Richey, H. (2016, June 7th). Biased Online Media Drives Polarization. Retrieved from
http://www.cw.ua.edu/article/2016/07/biased-media
Sunstein, C. R. (2001). Republic.com. Princeton University Press.
The conversation starts on Reddit. (n.d.). Retrieved July 08, 2016, from https://about.reddit. com/
Wright, S., & Street, J. (2007). Democracy, deliberation and design: the case of online discus-sion forums. New media & society, 9(5), 849-869.
Appendix
Appendix A.
Descriptive statistics of samples used of the scores for each model per subreddit.
Lib. Individualist Communitarian Deliberative
Sample Category n M SD M SD M SD 1 General 150 .153 .171 .171 .169 .248* .213 Other 150 .157 .155 .205 .165 .135 .180 Total 300 .155 .163 .188 .168 .191 .205 2 Liberal 432 .155* .151 .217** .166 .123 .158 Conservative 432 .177* .180 .190** .160 .112 .160 Total 864 .166 .167 .204 .164 .118 .159 3 General 150 .153 .171 .171** .169 .248* .213 Candidate 150 .173 .171 .212** .156 .083* .147 Ideology 150 .150 .169 .215** .162 .124* .155 Party 150 .162 .169 .184** .158 .136* .169 Total 600 .160 .170 .195 .162 .148 .183 * statistically significant at p = < .05
* statistically significant at p = < .05 and passed the assumption of homogeneity of variances at p > .05
Appendix B.
Descriptive statistics of each feature of each model individually per subreddit.
Liberal
indi-vidualist monologue
pers.
revela-tion pers. showcase flaming
Subreddit n M SD M SD M SD M SD Politics 150 .11 .310 .13 .341 .08 .272 .29 .457 Sanders 150 .17 .380 .11 .318 .06 .238 .29 .454 Clinton 149 .09 .293 .18 .386 .03 .181 .28 .451 Trump 150 .31 .463 .11 .318 .05 .225 .47 .501 Socialism 150 .17 .374 .11 .310 .07 .250 .23 .424 Libertarian 150 .17 .380 .23 .424 .02 .140 .19 .396 Republican 132 .19 .393 .11 .319 .04 .192 .20 .405 Democrats 106 .23 .420 .07 .250 .02 .137 .38 .487
Communi-tarian ideological h. mobilization Com. Ident. Intra-id. Resp. Intra-id. Quest.
Subreddit n M SD M SD M SD M SD M SD Politics 150 .13 .341 .33 .473 .14 .348 .14 .348 .11 .310 Sanders 150 .19 .391 .33 .471 .21 .406 .33 .471 .17 .374 Clinton 149 .23 .421 .14 .349 .23 .425 .43 .497 .09 .283 Trump 150 .20 .401 .15 .362 .09 .282 .38 .487 .12 .326 Socialism 150 .13 .341 .19 .396 .09 .292 .52 .501 .14 .348 Libertarian 150 .11 .318 .19 .391 .13 .334 .45 .499 .14 .348 Republican 132 .14 .344 .08 .266 .13 .336 .36 .481 .19 .393 Democrats 106 .23 .420 .14 .350 .08 .280 .37 .485 .16 .369
Deliberative Rat.Crit. Ar-gument.
Pub.Issue fo-cus
Disc.topic
focus. Inter-id. Resp. Inter-id. Quest.
Subreddit n M SD M SD M SD M SD M SD Politics 150 .26 .440 .20 .401 .56 .498 .09 .292 .13 .334 Sanders 150 .13 .341 .06 .238 .35 .478 .01 .115 .01 .082 Clinton 149 .13 .335 .03 .162 .36 .480 .09 .283 .01 .115 Trump 150 .03 .180 .02 .140 .17 .380 .02 .140 .00 .000 Socialism 150 .11 .318 .05 .212 .46 .500 .00 .000 .00 .000 Libertarian 150 .16 .368 .01 .115 .42 .495 .02 .140 .01 .082 Republican 132 .22 .416 .04 .192 .54 .500 .04 .192 .01 .087 Democrats 106 .16 .369 .06 .232 .38 .487 .06 .232 .04 .191 * statistically significant at p = < .05
27 Appendix C.
Descriptive statistics of each feature of each model individually per sample.
Liberal individualist monologue pers. revela-tion pers. show-case flaming sample category n M SD M SD M SD M SD 1 General 150 .11 .310 .13 .341 .08 .272 .29 .457 Other 150 .21 .406 .13 .334 .05 .212 .25 .433 2 Liberal 432 .16* .367 .12 .328 .04 .195 .30 .457 Conservative 432 .22* .418 .16 .362 .04 .189 .29 .455 3 General 150 .11 .310 .13 .341 .08 .272 .29 .457 Candidate 150 .18 .385 .09 .282 .07 .262 .35 .480 Ideology 150 .18 .385 .20 .401 .07 .250 .15 .362 Party 150 .21 .406 .09 .292 .04 .197 .31 .463 Communitarian ideological
h. mobilization Com. Ident.
Intra-id. Resp. Intra-id. Quest. sample Subreddit n M SD M SD M SD M SD M SD 1 General 150 .13 .341 .33 .473 .14 .348 .14 .348 .11 .310 Other 150 .15 .362 .22 .416 .16 .368 .40 .492 .09 .292 2 Liberal 432 .18 .385 .19* .394 .15 .358 .43 .495 .14 .344 Conservative 432 .15 .358 .14* .349 .11 .317 .40 .490 .15 .356 3 General 150 .13 .341 .33 .473 .14 .348 .14 .348 .11 .310 Candidate 150 .18 .385 .17 .380 .17 .380 .37 .485 .16 .368 Ideology 150 .13 .341 .22 .416 .13 .334 .45 .499 .15 .355 Party 150 .19 .391 .09 .292 .11 .310 .37 .484 .17 .374 Deliberative Rat.Crit. Argument. Pub.Issue focus Disc.topic focus. Inter-id. Resp. Inter-id. Quest. sample Subreddit n M SD M SD M SD M SD M SD 1 General 150 .26* .440 .20* .401 .56** .498 .09* .292 .13* .334 Other 150 .18* .385 .07* .262 .39** .489 .02* .140 .01* .115 2 Liberal 432 .13 .334 .04 .200 .40 .490 .04 .195 .01 .107 Conservative 432 .13 .341 .02 .151 .37 .483 .03 .158 .00 .068 3 General 150 .26 .440 .20 .401 .56 .498 .09 .292 .13 .334 Candidate 150 .12 .326 .03 .180 .23 .420 .02 .140 .01 .115 Ideology 150 .14 .348 .02 .140 .44 .498 .01 .115 .01 .082 Party 150 .18 .385 .04 .197 .41 .493 .04 .197 .01 .115 * statistically significant at p = < .05
