Analysis of Gender and Ethnicity Measurements in News Consumption
Courtney D. Tabor (11359536) Courtney.Tabor@student.uva.nl
Universiteit van Amsterdam Graduate School of Communication Research Master’s Program: Communication Science
Master’s Thesis
Supervised by Prof. Dr. Jochen Peter
28 January 2019 Word Count: 9,7361
1 As this thesis uses two studies with two different methods, a word limit of 10,000 has been allotted by the thesis supervisor.
Abstract
Over the last decade, increasing attention has been placed on diversity in
communication science, particularly regarding gender and ethnicity. However, little is known about the present state of these measurements in published literature, as well as the potential effects of introducing new measurements. When looked at alongside the rapidly changing societal views and increasing news coverage of minority people, a pressing need becomes apparent to understand how gender and ethnicity are conceptualized. As such, this thesis uses two studies to address how the two have been treated as variables in the context of news consumption and how unique measurements can illuminate new results that would otherwise be obfuscated. First, a content analysis of academic articles (N = 76) was conducted to analyze the diversity of measurements, language used, sampling, and relationships between variables. Results of the first study showed that research employs a rudimentary
understanding of gender and ethnicity, with restricted vocabulary, ambiguous concepts, oversampling of White people, and limited available statistics. Then, a series of three
otherwise identical surveys were randomly distributed to respondents (N = 257) that varied in their operationalizations of gender and ethnicity (i.e., binary, orientation, and
self-identification). These were then compared to see what differing results arose. Each group indicated unique consumption habits with different implications, showing that different measurements can indeed provide distinctive results. Based on the findings, future research should put more consideration into the decisions regarding the operationalization of gender and ethnicity. It seems that the previously relied upon measures are not always relevant and can limit the information researchers obtain.
Keywords: gender, ethnicity, news consumption, demographics, content analysis, survey
Analysis of Gender and Ethnicity Measurements in News Consumption
The American Psychological Association (APA; 2010) recommends that researchers “Detail the sample’s major demographic characteristics, such as age; sex; ethnic and/or racial group; level of education; socioeconomic, generational, or immigrant status; disability status; sexual orientation; gender identity; and language preference as well as important topic-specific characteristics…” (p. 29). However, there is little to no information about the relevancy or importance of each characteristic in relation to the others, the differences between concepts such as sex and gender identity, or accountability in the sampling process. Ansara and Hegarty (2014) pinpointed several of these flaws by identifying hypocrisies in the guide. For example, the manual encourages researchers to be socially conscious by
distinguishing between gender identity and sex, but then conflates the two by discussing “both sexes” (Ansara & Hegarty, 2014; APA, 2010). It is concerning that the manual (APA, 2010) widely regarded as the standard for writing, poses requirements that are both
ambiguous and contradicted within the text.
The manual also discusses collecting this information to make claims about
generalizability (APA, 2010), implying the necessity of a diverse sample. However, there is little to no consensus of what constitutes “diverse.” Recent definitions discuss the broad concept of diversity as fluid and contextual (Davidson, 2016; Deaux, 2018; Fernandes & Polzer, 2015; Garces & Jayakumar, 2014), but rigid structures remain the norm (Baker, Schmaling, Fountain, Blume, & Boose, 2016). Of the characteristics presented by the APA manual (2010), sex/gender identity and ethnic and/or racial group stand out for this reason. Although these characteristics are frequently used, they suffer from inconsistencies in conceptualization, unlike more formulaic variables, such as age.
The conflation of terms such as sex and gender or race and ethnicity is a primary inconsistency to be explored. By using distinct terms interchangeably, groups of people are
systematically excluded. For example, using sex to mean gender enforces a binary idea (Butler, 1986; Foucault, 1980) and excludes nonbinary people, or those who do not identify exclusively as a man or woman (Losty & O’Connor, 2018). As such, the term gender, referring to a chosen identity dislodged from biological sex (Butler, 1986) will be employed in this paper. Just as sex and gender are regularly conflated, so too are the concepts of race and ethnicity. This thesis embraces the work of Eisenhower, Suyemoto, Lucchese, and Canenguez (2014) who defined race as a political construction that uses physical appearance as an excuse for oppression, whereas ethnicity refers to a shared cultural background. As such, ethnicity is more appropriate than race.
In addition to the conceptual and reporting issues, there are also pressing social issues which make this topic relevant. Recent legal matters—for example, New York City’s decision to allow for a gender-neutral option on birth certificates (Simko-Bednarski, 2019) and the Netherlands issuing its first gender neutral passport (Barr, 2018)—have brought changing social identities to global attention. Additionally, nonbinary people have recently been gaining attention in the media, in large part due to social networking sites (Losty, &
O’Connor, 2018; Oakley, 2016). Similarly, minority ethnic groups have seen an increase in news coverage over the last few years, which is often negatively charged and fear-based (Jacobs, 2017; Ogan, Pennington, Venger, & Metz, 2018). When Great Britain voted to leave the European Union in June 2016, the country saw a drastic rise in bigotry and racism with 2,300 hate crimes reported in the first 38 days following the referendum (Burnett, 2017; Eddo-Lodge, 2017). Essentially, the changing political, legal, and news landscapes are reflective of novel representations of identity.
Accordingly, this thesis is situated in a news consumption context. Understandings of gender and ethnicity have been developing over the last decade in both academia and the media (Eisenhower et al., 2014; Losty & O’Connor, 2018; Westbrook & Saperstein, 2015),
although it is presently unknown whether researchers studying news consumption have adapted to these variations. Ultimately this research focuses on addressing the call made by Chakravartty, Kuo, Grubbs, and McIlwain (2018):
For sustained accountability to representational disparities in our field, future research must be able to rely on nuanced methodological approaches to account for race, caste, ethnicity, gender, sexuality, religion, and other globally racialized markers of
majority-minority difference. Our ability to understand why existing and potential disparate patterns exist depends on our ability to produce additional forms of data that allow us to do so (p. 262).
This is done first through a quantitative content analysis of the existing literature on news consumption to determine its present understanding of diversity. It is additionally important to consider how different measurements can affect the way groups both identify themselves and how they are represented in research. This thesis thus examines how operational decision making can suppress latent demographic groups (Magliozzi, Saperstein, & Westbrook, 2016; Westbrook & Saperstein, 2015). By comparing three surveys where each uses a different measure of gender and ethnicity, the emergence of unique groups that otherwise could go unnoticed, is explored. These studies provide an answer to the main research questions: Research Question 1: Looking at published journal articles of news consumption, how are gender and ethnicity presently treated as variables?
Research Question 2: What variability arises when using different measures of gender and ethnicity in the same survey seeking to explain news consumption?
Theoretical Framework Explorations into Diversity in Academia
Efforts have been made to move towards inclusivity, but existing research on diversity in academia shows that rudimentary understandings remain the norm. A content analysis of
four of the largest surveys in the United States (US) proved that researchers using nationally representative data tend to rely on outdated concepts of gender (Westbrook & Saperstein, 2015). All four surveys addressed gender as binary and treated sex and gender as
interchangeable (Westbrook & Saperstein, 2015). The conflation of sex and gender is apparent in the terminology used; many surveys used both words interchangeably or used gender while only having “male” and “female” options (Westbrook & Saperstein, 2015). The authors (Westbrook & Saperstein, 2015) further claimed that stagnation in surveys regarding gender conceptualization and terminology will reinforce statistics that exclude variations from the norm. While this content analysis provides a rudimentary idea of how gender is being measured in survey research, it focuses only on four US surveys and only on gender. There is little to no existing information about smaller studies, studies conducted outside the US, the measurement of ethnicity, and quantitative studies conducted that are not surveys. As such, this thesis includes a quantitative content analysis of academic articles that emphasizes many of the same points as Westbrook and Saperstein (2015), especially regarding the limiting measurements, conflation of concepts, and the use of certain terminology. These are addressed in the first two sub-questions:
SQ1: How reliant are researchers on restrictive measures of gender and ethnicity? SQ2: What terminology are researchers using to describe gender and ethnicity? A recent publication revealed the domination of communication science by White and Western scholars, and indicated that disadvantages are often “normalized and institutionally rearticulated” (Chakravartty et al., 2018, p. 255) because of narrow understandings of ethnicity. The study found that despite the increase of minority scholars, communication science remains predominately led by White men (Chakravartty et al., 2018). While this study conducted a thorough and necessary content analysis of communication science
by Chakravartty et al. (2018) by looking at sampling diversity in news consumption papers, moving from who is writing the research to who is being studied.
In addition to these two empirical works, social identity theory proposes that people categorize themselves and others into in- and outgroups, with the ingroup being people with whom they share common characteristics (Fernandes & Polzer, 2015; Tajfel & Turner, 1979; Turner, Hogg, Oakes, Reichter, & Wetherell, 1987). Individuals tend to favor people of their own ingroups, rather than pursuing an inclusive and diverse environment (Fernandes & Polzer, 2015). It is thus possible that researchers would subconsciously do the same when making decisions about their research; therefore, research may not always be designed with wholly inclusive measures. Furthermore, while researchers may implement measures that are not implicitly geared towards their ingroups, they may still show bias in the practice of recoding variables. Often done to fit the assumptions of statistical analyses, recoding can mean, for example, taking a sample of seven or eight ethnic groups, and condensing it to the ingroup (e.g., White) and all other outgroups (e.g., non-White). Inspired by the findings of Chakravartty et al. (2018) and social identity theory, two more sub-questions are explored. These concern the diverse representation of people in sampling and the recoding of their identities in analysis methods:
SQ3: In academic articles about news consumption, who makes up the sample in terms of gender and ethnicity?
SQ4: To what extent are researchers recoding the genders and ethnicities of respondents?
Existing News Consumption Claims
News consumption literature has claimed, practically as a fact, that men consume more news than women (Benesch, 2012) and that White people consume more news than non-White people (Callanan, 2012). Despite the vast amount of research using demographics
as a predictor or covariate of news consumption (e.g., Chyi & Lee, 2013; Jordan, Pope, Wallis, & Iyer, 2015; Lee, 2013; Lee & Chyi, 2015; Molyneux, 2018; Strömbäck & Kiousis, 2010), little has been done to compare these relationships and their effect sizes. Thus, this thesis includes a systematic review of the literature on demographics and news consumption to answer two additional sub-questions:
SQ5: Do the relationships presented in existing literature bolster or contradict the claims being made about White people/men consuming more news than others? SQ6: How strong are the relationships regarding gender and ethnicity as predictors for news consumption?
News consumption literature can be further divided into specific topics. Research from Benesch (2012) indicated that men gravitate more towards sports, environmental, and science and technology news whereas women tend to consume more news focused on religion, local news, entertainment and celebrities, arts and culture, education, and health. Additionally, Condit (1997) claimed that there are broader topics such as health and labor that are not typically associated with a certain gender. Divisions of topic preference is
supplemented by policy literature, as news consumption often dictates policy preference (Jordan, 1993), and little research exists dividing news consumption by ethnicity. In this vein, Page and Shapiro (1992) claimed that White people prefer policy positions that emphasize foreign policy and the military whereas Black people lean more towards those that highlight treatment of minorities, education, housing, jobs, welfare, and health. Unfortunately, many of the studies that have investigated these relationships rely on measures that are focused on the dominant groups—binary genders and White people (Carrotte, Vella, Bowring, Douglass, Hellard, & Lim, 2016; Chakravartty et al., 2018); very little research exists examining broader consumption preferences. This inspires the first sub-questions for the empirical
testing of diverse measurements. Using different measurements of gender and ethnicity, tested with respondents, this study seeks to answer two more sub-questions:
SQ7: When using different measures of gender, what differences are seen between gender groups regarding news consumption?
SQ8: When using different measures of ethnicity, what differences are seen between ethnic groups regarding news consumption?
Attempts at Changing Gender & Ethnicity Measurements
Little empirical work has been done in communication science to change the way gender and ethnicity are measured. In other disciplines, however, researchers have been testing and retesting new scales.
Regarding gender, Lesbian, Gay, Bisexual, and Transgender (LGBT) activist groups have advised that researchers consider, among other things—differentiating between sex and gender, excluding biological sex, allowing respondents to place their gender orientation on sliding scales, and/or using open-ended answers (The GenIUSS Group, 2014; Miller & Weingarten, 2005). In turn, researchers have applied these to a variety of contexts. Hunt, Lewars, Emslie, and Batty (2007), for example, researched social factors impacting coronary heart disease by including a measure of gender role orientations. Respondents were given scores related to masculinity, androgyny, and femininity, and the results showed that men with higher feminine scores had a lower risk of coronary heart disease than others, indicating that gender role orientation can play a factor in physical health (Hunt et al., 2007). In a study analyzing discrimination faced by transgender and nonbinary people in the workplace, gender was measured with a write-in option, of which 24.62% of respondents indicated identifying with a nonbinary gender (Davidson, 2016). Similarly, a study designed to test representations of gender diversity in social science research showed that
although more than 99 percent of both samples would be classified as cisgender2
based on their categorical responses, femininity and masculinity responses suggest that significantly fewer than 99 percent of respondents see their gender in traditionally dichotomous, categorical terms (Magliozzi et al., 2016, p. 4)
Each of these examples provides evidence that varying measures of gender can
produce more information about the identities of respondents, as well as novel findings. Thus, this thesis tests three measures of gender to be compared—a binary man/woman option, masculinity and femininity orientation scales, and an open-ended option for
self-identification. These will be used to answer a sub-question regarding identification variation: SQ9: To what extent does a change in measurement lead to variation in the way people identify their gender?
Looking at ethnicity, some studies, such as Lee’s (2013) analysis of patterns in news consumption, have stuck to a binary conceptualization that exclusively looks at White versus non-White people. Others have focused their measurements on a few local minority groups relative to the study, such as Callanan’s (2012) focus on highlighting differences between White, Latino, and African-American respondents when looking at perceptions of crime in California. However, these measures suffer the same pitfalls as binary conceptualizations of gender in that they could possibly exclude other groups and mask potential insights.
Alternatively, sociologists and psychologists have also studied new measurements of ethnicity, with Davis, Gurin, and Engel (2011) compiling a book of scales focused on different elements related to the concept. One such measurement is the Multi-Group Ethnic Identity Measure (MEIM; Davis et al., 2011; Phinney, 1992). The MEIM assesses two
2 Cisgender refers to someone who identifies a match between their gender identity and their sex assigned at birth (Magliozzi et al., 2016).
primary elements—feeling of belonging and commitment to a community, and involvement behaviors within the community (Davis et al., 2011; Phinney, 1992). Using this scale, researchers can learn more about individual relationships with ethnicity, which could affect how they navigate the world and provide previously missing social information (Deaux, 2018).
Further exploration into ethnicity measures has included the use of open-ended questions (Eisenhower et al., 2014). Results of this measure, in a health services context, indicated that rates of missing demographic data dropped from 26-43% to 11-18%
(Eisenhower et al., 2014). These measurements, however, have been minimally used in the social sciences. As such, this thesis will test and compare the binary white/non-White conceptualization, the MEIM orientation scale, and an open-ended measure of ethnicity. In doing so the final sub-question is addressed:
SQ10: To what extent does a change in measurement lead to variation in the way people identify their ethnicity?
Study 1—Methods
A quantitative content analysis was used to answer sub-questions 1-6. A content analysis is chosen over a meta-analysis for two main reasons. First, where a meta-analysis collects statistics to compare and combine for a main result (Cheung, Ho, Lim, & Mak, 2012; Cheung & Vijayakumar, 2016; Cooper & Hedges, 2009), this study synthesizes information about the literature without making a statistical claim about the relationships, similar to a research synthesis or systematic review (Cheung & Vijayakumar, 2016; Cooper & Hedges, 2009). Second, since there are few statistics upon which to draw conclusions, meta-analysis techniques are not appropriate (Borenstein, Hedges, Higgins, & Rothstein, 2009). The final codebook is provided in Appendix A.
Separate search strings for gender and ethnicity were used to find potential coding units. For gender, the following search string was used—((news consumption) AND gender) OR ((news consumption) AND sex) OR ((news use) AND gender) OR ((news use) AND sex) OR ((news usage) AND gender) OR ((news usage) AND sex). The search string for ethnicity read—((news consumption) AND race) OR ((news consumption) AND ethnic*) OR ((news use) AND race) OR ((news use) AND ethnic*) OR ((news usage) AND race) OR ((news usage) AND ethnic*). “News consumption,” “news use,” and “news usage” were chosen due to their relative similarity and ability to broaden the scope of the search while remaining feasible. For gender, both gender and sex were used to capture papers that included either a gender identity or biological sex measure. Similarly, the ethnicity string uses both race as well as “ethnic*” to include papers using phrases such as ethnicity, ethnic group, or ethnic minority.
The Web of Science Social Science Citation Index was used to search for articles that were published in English between January 2008 and September 2018. Web of Science was chosen because it is a reputable database that is human- rather than algorithm-curated and because it searches across disciplines using citation indexing and Boolean driven keyword searches. The decision to use a 10-year period was based both on feasibility and existing literature that indicates diversity is a more recently prevalent topic (Eisenhower et al., 2014; Losty & O’Connor, 2018; Westbrook & Saperstein, 2015). This resulted in 403 articles for gender and 302 articles for ethnicity.
The screening process used to narrow the sample is depicted in Figures 1-3. First, all abstracts were read to search for obvious removals (e.g., qualitative method, political “race,” or “sex”-ual content). Next, the articles were downloaded, removing any titles that were unavailable or written in another language. Then, the keywords “gender,” “sex,” “race,” “ethnicity,” and “news” were used to remove any paper that did not contain the appropriate
terms in the main body text. Subsequently, the methods section was read to determine whether the measures used were appropriate (e.g., using a news article as a stimulus was removed and measuring frequency of reading news was kept). Further details are provided in Appendix B.
The final number of articles retained was 42 (10.42%) for gender and 39 (12.91%) for ethnicity. Separate studies were treated as unique entries, increasing the sample size from 81 coding units to 84. However, three papers appeared in both samples and, as the codebook allows for the simultaneous coding of gender and ethnicity, the repeated articles were removed. Thus, the sample size consisted of 81 coding units.
Reliability Testing
Reliability tests were executed between two coders by calculating the Krippendorff’s alpha for each variable, a commonly used measure for quantitative content analysis
agreement (de Swert, 2012). Three tests were conducted, with additional coder training and codebook rewrites each time. Nevertheless, a minimum threshold of .60 (de Swert, 2012) could not be met for all variables. As of the third test, two methodology variables (
= .26-.57), two gender variables ( = -.07-.00), one news variable ( = .35), and all but two statistics variables ( = -.52-.41) were still not reliable. However, intra-coder reliability with a two-week lag period proved reliable (all levels = .73-1.00). A table with the results of both tests is provided in Appendix C.
Study 1—Results Descriptive Information
Five studies were removed during coding (see Figure 3), which made the final sample size 76 studies. Most came from communication science and media studies journals (n = 43),
Figure 1: Flow Chart for Gender Sample Selection.
Figure 1: Flow chart for gender sample selection (content analysis).
Figure 2: Flow chart for ethnicity sample selection (content analysis).
followed by political science (n = 12), miscellaneous3 (n = 9), sociology (n = 6), behavioral
science (n = 4), and criminal justice/criminology (n = 2). Research was conducted
overwhelmingly by surveys (n = 66), but experiments (n = 7) and content analyses (n = 3) were also present. Similarly, the research was largely cross-sectional (n = 60) with far fewer longitudinal designs (n = 16). Regarding news consumption, 59 studies used a generic measurement (e.g., frequency of media exposure), whereas 17 were topic-specific (e.g., attention to media stories about Iraq). Most served as independent (n = 46) or dependent variables (n = 24), but with a few moderators (n = 2) and mediators (n = 3) as well (a complete list of the news variables is provided in Appendix D).4
Sub-question One
The first sub-question asked “how reliant are researchers on restrictive measures of gender and ethnicity?” To answer this, both the presence of gender and ethnicity were coded, as well as the diversity level of the measurement. Of the 76 studies, gender was present in 74 (97.37%) and ethnicity in 48 (63.16%). An item that evaluated the level of diversity in
measurements was developed for this codebook. This used a seven-point Likert scale, ranging from not at all diverse (e.g., binary) to wholly diverse (e.g., open-ended). Results showed that binary measurements for gender were relied upon, with only three studies reporting
nonbinary options (M = 1.03, SD = 0.16). Studies including ethnicity showed slightly more openness, with an average score of 2.98 (SD = 1.77) on the diversity scale. Essentially, measures for gender and ethnicity are quite restrictive as both scored below even the middle point of the scale.
3 The miscellaneous journals which did not fit into another category were Applied Economics; Armed Forces & Society; Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science; BMC Public Health; Bulletin of the American Meteorological Society; Educational Research; Health Education Research; Risk Analysis; and Women’s Health Issues.
Sub-question Two
The second sub-question asked “what terminology are researchers using to describe gender and ethnicity?” In answering this question, it is clearer what labels are used and how often researchers rely on the familiar. A word count was employed to address this sub-question where each coded word referring to these concepts was singularized (e.g. “woman” and “women” were counted as the same word) and counted once per study.
“Gender” (n = 65) appeared far more often than “sex” (n = 14) and these terms were used interchangeably in 11 studies (14.86%). Binary concepts of gender were prevalent throughout, with the most common words being “female” (n = 63), “male” (n = 55), “woman” (n = 44), and “man” (n = 37). Only in six cases did any phrase come up that
referenced nonbinary genders (“did not identify with their gender,” n = 1; “gender-neutral,” n =3; and “no gender,” n = 2). Additional gender words (e.g., “boy” and “girl”) further reflected the binary (see Appendix D for a complete list of coded words).
The words used to describe ethnicity were more complicated than gender (see Appendix D for a complete list of coded words). “Race” was the most commonly used term (n = 30), closely followed by “ethnicity” (n = 21). Seventeen additional phrases were also used to describe the concept. Ethnicity and race were more frequently used interchangeably than gender, with 16 of the 48 studies (33.33%) conflating the two. For individual-level characteristics, “White” was the most common label (n = 36), followed by “Black” (n = 22), “Asian” (n = 15), “Hispanic” (n = 15), and other (n = 13). However, this only scratches the surface of the labels used; 45 total labels were coded to describe ethnicity, with some showing overlap with others (e.g., “Black/African American”).
Overall, these results show how prevalent conflating concepts is and that many of the familiar labels are still frequently used. For gender, researchers rarely expanded beyond
commonly understood concepts. Further, the ambiguity of ethnicity was ubiquitous, as seen by the great number of words used to describe both the concept itself and its factors.
Sub-question Three
Sub-question three asked “in academic articles about news consumption, who makes up the sample in terms of gender and ethnicity?” This question addressed how diverse groups of respondents have been over the last decade and investigated whether samples reflect the same biases towards White men that publications have shown (Chakravartty et al., 2018).
Sampling was primarily probability based (n = 41), with several quota (n = 14) and non-probability (n = 15) samples as well.5 The sample sizes themselves deviated greatly with
an average of 3601.37 respondents, but a standard deviation of 7777.50. Fifty-nine studies (79.73%) included the gender distribution of respondents. On average, 45.20% of the sample was male (SD = 14.94) and only three articles included nonbinary genders; these reported anywhere from 1.00-4.00% of the sample as nonbinary. Regarding ethnicity, the sample makeup was reported in 36 out of 48 studies (75.00%). There was a reported average of 66.92% White respondents (SD = 25.24), but with greater variability in describing other ethnicities (see Appendix D). Additionally, one article was unique in that it reported no White people whatsoever; instead, the study, which took place in Singapore, reported “Chinese,” “Malay,” and “other” groups (Lee & Thien, 2014).
These findings show that while White people still make up most sampled respondents, men and women are almost evenly split in their representation. However, nonbinary genders and non-White ethnicities are still underrepresented in the sampling process. Furthermore, ethnicity samples can be contextual, with studies reporting distributions relative to the location of the study (e.g., Lee & Thien, 2014).
Sub-question Four
The next sub-question asked “to what extent are researchers recoding the genders and ethnicities of respondents?” To answer this, whether and how researchers changed the
identities of respondents to fit certain statistical analyses, as well as what constitutes these new groups, was coded.
Two articles in the gender sample (2.70%) recoded nonbinary genders to fit dummy variables. Recoding of ethnicity occurred in 21 articles (43.75%), with many of these recoding greater distinctions between groups into a dummy variable (e.g., White and non-White people). However, it sometimes meant using different labels in the results section than what was reported in the sample characteristics. For example, one article reported that “Chinese,” “Latino,” and “Caucasian/non-Hispanic White” people were sampled, but then used “Asian,” “Hispanic,” and “Anglo” labels in the results (Liu, Chen, Ognyanova, Nah, & Ball-Rokeach, 2018). In this case, using a sample of exclusively Chinese people to make claims about all people from Asia (Liu et al., 2018), brings up issues of reliability. These findings, overall, show that research still relies on recoding gender and ethnicity. Doing so changes the representation of identities, sometimes with the unintended consequence of falsely reporting results (e.g., Liu et al., 2018).
Sub-questions Five and Six
The last two sub-questions for the content analysis are (SQ5) “do the relationships presented in existing literature bolster or contradict the claims being made about White people/men consuming more news than others?” and (SQ6) “how strong are the
relationships regarding gender and ethnicity as predictors for news consumption?” Twenty-five articles reported an influence of gender on news consumption, with 16 of these
significant at the 95% confidence interval. Of the significant relationships, five bivariate effect sizes and 14 multivariate were reported. Fifteen articles reported an influence of
ethnicity on news consumption. Eight of these were significant at the same level, and the reported effect sizes were bivariate in three studies and multivariate in five. All reported relationships are provided in Appendix D.
In general, it seems that men and White people do not unilaterally consume more news than other groups. Although men significantly read more news online, read newspapers more often, and listened to the radio more, women surpassed men in several categories; these include watching more television news, using more diverse sources, getting more news from Facebook, and learning more from news. For ethnicity, White people did not consume more news than any other group. Rather, other ethnicities (primarily Black and Hispanic people) watched more television news, listened to the radio, and read news more often than White people. However, effect sizes were never more than moderately sized. Effect sizes for the gender relationships ranged from .00-.43 and for ethnicity, they ranged from .00-.11. Across all gender and ethnicity effect sizes, the average was just .10 (SD = .15). Given that the highest effect sizes are multivariate, and the average effect size is quite small, the substantial size of the effect is questionable.
These findings show that, while it has been previously believed that minority groups consume less news than men and White people (e.g., Benesch, 2012; Callanan, 2012), this is not necessarily true. The relationships between demographics and news consumption are far more complex than presently understood in literature. The effect sizes also show that these relationships are not necessarily very strong.
Study 2—Methods
Three comparative surveys were used to analyze sub-questions 7-10. The complete questionnaire is provided in Appendix E.
Using Qualtrics software, all survey questions were input into the same project. Then, the gender and ethnicity questions were randomized across the three surveys such that each respondent would see either binary, orientation-based, or self-identifying answers (as seen in Table 1). This study opted to use three surveys rather than an experiment to avoid potential priming effects (Rivers & Sherman, 2018) caused by presenting varying measures of gender and ethnicity (i.e., the only measure that differs systematically across the surveys) prior to news consumption measures (e.g., Cassino & Erisen, 2010; Koopmans & Veit, 2014).
Likewise, as the aim of this study is not to establish any causal claims but rather to illuminate the possibilities afforded by varying measurements, an experimental design is unnecessary. Instead, this study asks news consumption questions first and then randomly assigns gender and ethnicity questions to respondents. In this way, the focus remains on naturally drawing out new groups and thus seeks to address to what extent operational decision making suppresses pertinent information.
Furthermore, the implementation of randomization strengthens the study. Often seen in experimental designs, randomization is used to subdue spurious effects (Horiuchi, Imai, & Taniguchi, 2007) by ensuring every respondent has an equal chance of being allocated to a group (Manly, 2010). Thus, as little bias as possible regarding the distribution of respondents to groups (e.g., oversampling nonbinary respondents in the self-identifying group), is ensured with randomization. As such, any differences found should be attributed to the measures rather than uniquely different characteristics of respondents.
Online surveys were disseminated from December 4th-27th, 2018. Distributing the
surveys online was chosen to obtain a larger and more heterogeneous sample. Diversity and snowball sampling techniques were used; contacts from varying backgrounds were sent the questionnaire, and then these connections were encouraged to share the survey with others. The target sample size for each survey was determined with a G*Power ver. 3.1 (Faul,
Erdfelder, Buchner, & Lang, 2009; Faul, Erdfelder, Lang, & Buchner, 2007) analysis and resulted in a total predicted sample size of 93 respondents.6
At the start of the survey, respondents were presented with a fact sheet that provided information about their rights7 and said that the study would be focused on news
consumption8. Respondents were then asked to give their informed consent to participate.
Due to changes in European Union privacy regulations, potentially identifying characteristics (e.g., IP address) were not collected. However, respondents were permitted to withdraw their response within seven days and thus created a unique code to provide the researcher when requesting to withdraw (n = 0).
Once consent was given and a code created, respondents completed their assigned questionnaire. A total of 398 responses were initially collected. One hundred seventeen respondents did not answer the first question following informed consent. Another 17 began the survey, but did not make it to the gender and ethnicity questions.9 Ultimately, 64.57% of
the sample was retained (detailed steps provided in Appendix F; N = 257, see Table 1). Table 1:
Variations in Gender/Ethnicity Across Survey Groups
6 G*Power test conducted used the following parameters—a priori X2 variance two-tailed model with a
1.5 ratio, 95% confidence interval, and 0.80 power coefficient.
7 Ethical approval for this survey was provided by the thesis supervisor and meets the general requirements for ethical research set forth by the APA (2010).
8 A nonbinary LGBT activist was consulted about the ethics of potentially knowingly misgendering respondents (e.g., seeking out nonbinary people and then exposing them to binary questions). Their
recommendation was to include an extensive debriefing (C. Wright, personal communication, September 17, 2018).
Measures
Four key measures were used in conducting this study—news consumption (frequency and attention), gender, and ethnicity.
News consumption. Frequency and attention were used as news consumption
measures because they were two of the most common variables found in the content analysis. Frequency was measured by asking respondents whether they retrieved news yesterday from any of four sources (i.e., newspaper, TV, radio, or online; Benesch, 2012). This was answered with a scale ranging from 0-120 minutes (Wen, Xiaoming, & George, 2013). Attention paid towards each topic was measured with a 10-point sliding scale from very little to very close, indicating how closely respondents follow the news (Hoffman & Eveland, 2010). Both measures included a battery of topics inspired by previous literature —Treatment of Minority Groups, Education, Health/Healthcare, Social Programs, Economy & Labor Market, Foreign Policy, Military & War, Sports, Environment, Science & Technology, Local News,
Entertainment & Celebrity, and Arts & Culture (Benesch, 2012; Herrnson, Lay, & Stokes 2003; Page & Shapiro, 1992).
Gender and ethnicity. The three different gender and ethnicity measures were based on those previously tested in communication, feminist, medicine, and social science research (Carrotte et al., 2016; Davidson, 2016; Davis et al., 2011; Hunt et al., 2007; Lee, 2013; Magliozzi et al., 2016; Phinney, 1992). The binary questions asked respondents to choose between man/woman and White/non-White options, whereas the self-identification questions offered open-ended answers.
The orientation questions used three scales to measure gender and ethnicity. First, respondents were asked to identify their gender orientation on two 10-point scales, one for masculinity (M = 4.02, SD = 3.01) and one for femininity (M = 6.60, SD = 3.09). The
to create one measure for gender orientation, where a higher value indicates very masculine and a lower value, very feminine (M = 4.21, SD = 2.86, = .87).
Then, respondents answered a battery of 14 questions from the MEIM that asked about their level of connection with their ethnic groups on a seven-point scale (see Appendix E). Three components emerged from an initial principal component factor analysis (PCA) with varimax rotation (see Appendix G for factor loadings). While previous research has also shown the MEIM to load on more than one factor (e.g., Davis et al., 2011), this study requires one component so that it is consistent with the other measures of ethnicity. Thus, inspired both by the factor loadings of the first PCA and an existing extension of the MEIM (Phinney & Ong, 2007), six items were tested in a second PCA. All items in this test load on one factor with an eigenvalue of 3.37 (56.13% of the variance is explained) and with factor loadings (see Appendix G) that range from .53-.88. The scale with these six items was reliable ( = .84).
It is worth reporting that a question was included to ensure respondents understood the gender and ethnicity measures correctly. Respondents were asked to evaluate the openness of the answer options for both variables on seven-point scales. ANOVA tests indicated that, indeed, all groups responded as intended. For gender, the self-identification group reported the most openness (M = 6.19, SD = 1.52), followed by the orientation (M = 5.05, SD = 1.88) and binary groups (M = 3.01, SD = 2.14), F(2, 249) = 63.56, p < .001. The same is true for ethnicity—the self-identifying group reported the most openness (M = 6.04, SD = 1.62), then the orientation group (M = 5.22, SD = 1.52), and finally, the binary group (M = 2.67, SD = 2.09), F(2, 249) = 84.11, p < .001.
Due to the limited sample and the need for significance testing, recoding is inevitable. Although this is not ideal, statistical tests do not allow, for example, groups of just one
respondent. As such, gender and ethnicity were both adjusted to fit the analyses.
First, the gender scale in the orientation group was split into three groups where any score below four was coded as feminine (n = 47), scores of four to seven were androgynous (n = 21), and any score above seven was masculine (n = 16; see Hunt et al., 2007 for a similar split and grouping). Next, ethnic identity was split at the midpoint of the scale (i.e., all scores below four indicate a low connection to ethnic identity and all remaining scores indicate a high connection). This was chosen over a median split as that technique has been previously proven to increase a study’s chance for error and decrease the power (McClelland, Lynch, Irwin, Spiller, & Fitzsimons, 2015), and it could mean manipulated levels of diversity. Thus, the orientation groups consisted of either (a) respondents with little connection to their ethnic group (n = 47) or (b) respondents with a high connection (n = 37). Finally, self-identifications for both gender and ethnicity were open coded to create new groups (see Appendix H). Gender was divided into women (n = 61), men (n = 16), and nonbinary (n = 9) and ethnicity was divided into White (n = 51) and non-White (n = 34). This process met the standards for open coding validity as set forth by Guba (1981) and Shenton (2004).
Using these groups, t-tests, Analysis of Variance (ANOVAs), and X2 tests of
association are used to answer sub-questions 7-10. Effect sizes for all significant relationships are reported; these include Cohen’s d (see Becker, 2000 for calculator), 2, and Cramer’s V.
As the content analysis found few reported effect sizes with which to compare, effect size interpretation is instead based on the thresholds indicated by Sullivan and Feinn (2012).
Study 2—Results Randomization
Though the study is not an experiment, randomly allocating respondents to survey groups provided the added benefit of controlling for various demographics and preventing researcher bias (Manly, 2010). Respondents did not differ across groups in terms of age, F(2, 249) = 0.01, p = .988; level of education, X2 (4, N = 254) = 5.54, p = .236; employment
status, X2 (8, N = 254) = 5.18, p = .738; political interest, F(2, 250) = 1.11, p = .331; or
political ideology, F(2, 250) = 0.34, p = .715. Effects should thus be attributed primarily to demographic measurement differences.
Sub-questions Seven and Eight
The first two sub-questions for the survey study are (SQ7) “when using different measures of gender, what differences are seen between gender groups regarding news consumption?” and (SQ8) “when using different measures of ethnicity, what differences are seen between ethnic groups regarding news consumption?” Each topic for frequency and attention is treated as a dependent variable with the respective demographic measures as the independent variables. Tables with results from all six tests are provided in Appendix I.
Gender and news consumption. T-tests were run for the binary survey group and ANOVAs with post-hoc Bonferroni tests were run for the remaining two groups.
Binary gender survey group. Men and women served as the independent groups for
the binary gender survey group and news consumption t-tests (see Table 2). Men reported significantly more time consuming science and technology news than women, and this effect is moderately sized, d = .56. Similarly, men reported paying significantly more attention to sports news than women with a moderate effect size, d = .55. Finally, men reported paying significantly more attention to science and technology news than women with a large effect size, d = .90. This is in line with previous findings that claim science and technology news and sports news are more strongly associated with men (Benesch, 2012). However, these
results do not reflect any previous findings (Benesch, 2012) regarding women’s news preferences.
Table 2:
Significant T-Test Statistics for Binary Gender and News Consumption
Note: T-tests with df that differ from 84 do not meet Levene’s test of equal variances. *Indicates significance at the 95% level. ^Indicates significance at the 99% level.
Gender orientation survey group. Six relationships were significant for the gender
orientation group (see Table 3). Androgynous people reported significantly more
consumption of treatment of minority group news than feminine people, but this effect was weak, 2 = .08. Masculine people did not significantly differ from either group for this topic.
Androgynous people also spent significantly more time consuming education news than both other groups, and the difference between feminine and masculine people was not significant, with a small effect, 2 = .13. No effect was found between feminine and masculine groups,
but androgynous people reported significantly more minutes than both for environmental news consumption, with a small effect, 2 = .15. Finally, androgynous people spent
significantly more time consuming science and technology news than feminine people, with a small effect, but no differences were found for masculine people, 2 = .09.
Feminine people reported significantly less attention paid towards economy and labor market news then the other groups, with no significant differences between masculine and androgynous people. This effect is small, 2 = .13. Alternatively, feminine people reported
a small effect, 2 = .09. No significant differences for this topic were found for androgynous
people.
Only the results for arts and culture news support Benesch (2012)’s claim that this topic is more commonly consumed by women. The rest of the results, however, are contrary to previous findings. Whereas Benesch (2012) claimed environmental and science and
technology news were topics men primarily consume, these results show androgynous people consumed more of both. Further, although Benesch (2012) categorized education as a
women’s topic, these findings show that androgynous people consumed significantly more of this topic. Finally, whereas Condit’s (1997) findings would lead one to believe health and labor news would be associated with androgynous people, neither emerged as significantly related to androgynous people.
Table 3:
Significant ANOVA Statistics for Gender Orientation and News Consumption
*Indicates significance at the 95% level. ^Indicates significance at the 99% level.
Self-identifying gender group. Three relationships were significant for the gender
self-identification group (see Table 4). First, although there were no significant differences between men and women, nonbinary people reported significantly more time consuming
news about the treatment of minority groups than both, though this effect is small, 2 = .16.
Similarly, there were no significant differences between women and men, but nonbinary people reported a significantly higher amount of attention paid towards news about treatment of minorities than both other groups, with another small effect size, 2 = .16. Lastly, men
reported significantly less attention to news about social programs than nonbinary people, although no differences were significant regarding women’s attention to the topic. This effect is small, 2 = .09.
These results are opposed to the findings of Benesch (2012) and Condit (1997). Whereas all findings from Benesch (2012) relate to men and women specifically, all findings in this study involved nonbinary people. Furthermore, the topics which emerged as relevant for nonbinary people were contrary to Condit’s (1997) health and labor topic assumptions. Table 4:
Significant ANOVA Statistics for Self-identified Gender and News Consumption
*Indicates significance at the 95% level. ^Indicates significance at the 99% level.
Ethnicity and news consumption. Since all categories for ethnicity were recoded as binary variables, all analyses for ethnicity and news consumption are run with t-tests.
Binary ethnicity group. Only one relationship was significant for the binary ethnicity
group (see Table 5). White people reported paying significantly more attention to environmental news than non-White people, with a strong association, d = 1.00. These
findings do not reflect those found by Page and Shapiro (1992) and rather, introduce a new topic completely.
Table 5:
Significant T-Test Statistics for Binary Ethnicity and News Consumption
*Indicates significance at the 95% level.
Ethnic identity orientation group. The orientation group, as with gender, produced
the most significant relationships of all the survey groups (see Table 6). People with a high connection to their ethnic group reported significantly more consumption of foreign policy news than those with a low connection, and this was a moderately sized effect, d = .47. The same relationship is true for frequency of consuming arts and culture news, with highly connected people reporting significantly more consumption than their counterparts. This effect is also moderate, d = .47.
Four relationships are present between connection to ethnicity and attention. First, highly connected people reported more attention paid towards health news than those less connected, with a moderate effect, d = .58. Second, like frequency, highly connected people reported a greater amount of attention paid towards foreign policy news than people with less of a connection to their ethnic group, with a moderate effect size, d = .52. Third, highly connected people reported significantly more attention paid towards local news than their counterparts, with a very strong effect, d = .93. Finally, highly connected people reported paying significantly more attention to arts and culture news than less connected people, which is reflected in the frequency results for the same topic. This effect is moderate, d = .44.
These results are more difficult to compare with previous findings, as Page and Shapiro (1992) compared White and Black people and these findings focused on levels of
connection to ethnic groups. However, comparing the dominant groups (i.e., White people and people with a low connection) with the smaller groups (i.e., Black people and people with a high connection) should provide some insights. First, results were found that
contradicted the previous literature (Page & Shapiro, 1992) in that the smaller group showed both more attention towards and frequency of consuming foreign policy news. On the other hand, Page and Shapiro’s (1992) findings were reflected in attention paid towards health and healthcare news, in that the smaller group reported higher numbers than the dominant group. New topics also emerged, these being local news and arts and culture news; highly connected people reported higher scores on both elements. For arts and culture news, this could be because people more highly connected to their ethnic groups are more invested in the topic, or vice versa (i.e., people who consume more culture news become more connected to their ethnicity).
Table 6:
Significant T-Test Statistics for Connection to Ethnicity (Orientation) and News Consumption
Note: T-tests with df that differ from 82 do not meet Levene’s test of equal variances assumed. *Indicates significance at the 95% level. ^Indicates significance at the 99% level.
Self-identifying ethnicity group. The self-identifying group produced only one
significant relationship (see Table 7). White people reported having consumed nearly twice as much news about the treatment of minorities than non-White people, with a moderately strong effect, d = .60. This finding is in direct opposition with Page and Shapiro’s (1992) findings.
Table 7:
Significant T-Test Statistics for Self-Identified Ethnicity and News Consumption
Note: T-test does not meet Levene’s test of equal variances assumed. *Indicates significance at the 95% level.
Sub-questions Nine and Ten
The last two sub-questions asked “to what extent does a change in measurement lead to variation in the way people identify their (SQ9) gender and (SQ10) ethnicity?” While the tests run in these analyses are correlational—and correlation is not causation—the causal ordering in this case can be inferred (Van der Stede. 2014). Indeed, this study meets requirements that have been previously established for inferring causality from survey research (Van der Stede, 2014). These include a theoretically backed and logical relationship between the variables and the establishment of controls through randomization (Van der Stede, 2014). As it is almost certainly impossible for a person’s identification to change the measurement they receive, there is a logical ordering to these variables, that is supported by the randomization process of the survey.
Gender was categorized as “man,” “woman,” and “other” for each group. Table 8 indicates the percentages for each group and categorization. The differences between these distributions are significant, as indicated by a X2 test of association, X2 (4, N = 256) = 27.45,
p < .001. This shows that a change in gender measurement can lead to significant variability among reported gender identity, and this relationship is moderately strong, V = .23. This variability includes (1) a greater dispersion of people across categories for the orientation
group, (2) a greater amount of men in the binary group than in others, and (3) the emergence of androgynous and nonbinary people in the orientation and self-identifying groups.
Table 8:
Percentage Distributions of Gender Categorizations Across Survey Groups
For ethnicity, two categories were used with each survey group. Table 9 indicates the distribution of people across categories and survey groups. A X2 test showed this association
was also significant, with a moderate effect size, X2 (2, N = 255) = 20.75, p < .001, V = .29.
As such, we can be reasonably sure that a change in ethnicity measurement leads to a change in reported ethnic identity. These changes lead to such variabilities as (1) a greater dispersion of people across connections to their ethnic identities and (2) a much lower number of White people in the self-identifying group than in the binary group.
Table 9:
Percentage Distributions for Ethnicity Categorizations Across Survey Groups
Conclusion & Discussion
Using two studies, this thesis analyzed gender and ethnicity measures in a news consumption context by examining literature and the potential for change. The research questions “looking at published journal articles of news consumption, how are gender and ethnicity presently treated as variables?” and “what variability arises when using different measures of gender and ethnicity in the same survey seeking to explain news consumption?” were answered by using both a content analysis and survey design. In answering these research questions (to be discussed further), this thesis addressed calls made by previous
researchers to examine measures of demographics and encourage the use of diverse measurements (Chakravartty et al., 2018; Magliozzi et al., 2016; Westbrook & Saperstein, 2015).
The State of News Consumption Literature
The state of news consumption research regarding gender and ethnicity measures is presently underwhelming. The content analysis showed very similar results to that of
Westbrook and Saperstein (2015), in that the sample reflected outdated concepts and limited vocabulary. However, the language used to discuss ethnicity more closely reflected the discussions put forth by Deaux (2018) and Fernandes and Polzer (2015), as the terms were often ambiguous, similar but not precisely the same, and contextual in nature. This indicates a need for more precise definitions of ethnicity and updated definitions of gender.
Furthermore, though the gender samples were often well split between men and women, nonbinary genders were largely forgotten and non-White people were regularly underrepresented. This echoes the work of Chakravartty et al. (2018), with both their content analysis and this study’s indicating that the dominant groups receive most of the attention in research. Social identity theory is not fully supported since some research showed hints of incorporating outgroups. For example, three of the studies in the gender sample accounted for nonbinary people, a typical outgroup. What is not known, though, are the demographics of the authors and their personal ingroups. It is possible that only nonbinary authors wrote the papers which included nonbinary genders. As such, it would be interesting to see who
conducts more inclusive research and to what social identity groups these researchers belong. The final focus of the content analysis was on the relationships and effect sizes for gender and ethnicity as predictors of news consumption. Whereas the existing research has regularly claimed that White people consume more news than non-White people (Callanan, 2012), the relationships found in the content analysis showed directly opposing results.
Moreover, claims about the way men and women consume news were less concrete than the previously accepted “men consume more news than women” principle (Benesch, 2012). While this is sometimes true, it is dependent on factors such as the medium or topic; in other ways, women consumed more news than men. Although effect sizes were small, the
significant results indicated that men and women, overall, vary in the way they consume news, and that non-White people consume more news in general than White people. This has implications for research which may incorrectly apply an absent relationship. Based on these findings, there is uncertainty as to whether the previous assumptions were mistaken, or if a shift has occurred since they were made. As such, there is a need for meta-analyses to establish stronger claims about the predictive value of demographics on news consumption. Diversity of Measures
Using three surveys, this thesis showed that there is significant variability in how people identify themselves, dependent on the implemented measure. These findings show the potential for future communication science researchers to explore gender and ethnicity in more open ways. As previously mentioned, the legal parameters for some demographic groups are changing (e.g., gender-neutral passports), and by utilizing different measurements communication science research could become more reflective of this. Likewise, the research question is an essential part of choosing the measurements; for example, although few studies aim to test biological factors, many still use “sex” where perhaps “gender identity” is more accurate.
Expanding on this, studies that aim to evaluate the social aspects of gender or
ethnicity lose opportunities by relying on measures that serve exclusively as a label. Previous literature on gender orientation and ethnicity scales has shown that these measures allow researchers to evaluate unique elements of the demographics (The GeniUSS Group, 2014; Hunt et al., 2007), such as performativity of gender (Butler, 1988) or a respondents’ feelings
of connection to their ethnic background (Davis et al., 2011; Phinney, 1992). This study reflected the previous literature in that the gender orientation scale signified a person’s level of masculinity or femininity (Hunt et al., 2007), and the revised MEIM was an indicator of closeness to the ethnic group, the same concept identified by Phinney and Ong (2007). Both move beyond a one-track understanding of the concept.
Were communication scientists to begin using diverse scales more frequently, researchers could likely better answer socially-driven questions. Such an example relates to Condit’s (1997) work on gendered and gender diverse topics. While news research can be clouded by “male and female topics” (e.g., the military or abortion), changing the
measurement can illuminate additional information (e.g., androgynous people showing interest in treatment of minority groups and education). Future research should ask questions such as “what news preferences are associated with androgyny?” or “how does a sense of belonging with an ethnic group affect a person’s interest in news?” Distinctions such as these are worth careful consideration when choosing measurements for demographic variables. Expanding Differences in News Consumption
The surveys used in this thesis showed that depending on the measurement used, different news topics became relevant; the varying measures brought forth unique
information. This is especially interesting when considering a topic as relevant to the study as treatment of minority groups. A primary interest for choosing gender and ethnicity is the growing news coverage of the treatment of minority groups in the media (Davidson, 2016; Jacobs, 2017; Losty, & O’Connor, 2018; Oakley, 2016; Ogan, et al., 2018), and while it did not rise to prominence in the binary groups, it was a significant topic in both
self-identification groups and the gender orientation group. Correspondingly, using the MEIM showed that topics such as foreign policy and arts and culture news are more relevant when predicted by levels of closeness to one’s ethnic group, an entirely unique relationship
compared to those measured in previous research (e.g., Page & Shapiro, 1992). There is much to learn about the nuances of news consumption, among other possible topics, that are only seen when new measures are introduced.
Limitations
For the content analysis, limitations arose with regards to the search string and coding process. First, the search string was developed with feasibility in mind, which greatly
narrowed the search. For example, words which did not fit the terms “gender,” “sex,” “race,” or “ethnic*” were excluded, and, as the content analysis showed, several other phrases were used to describe these phenomena. Second, the string only looked for news consumption and the demographics to appear within the same article, not within a causal relationship; this limited the statistics available for coding. Third, the content analysis did not code three possibly noteworthy variables—study location, researcher demographics, and assumed language. As some areas are more homogenous than others, the location of the study could affect the ethnicity measurement and level of diversity. As previously discussed, the
demographics of the researchers themselves may be relevant, as it could provide additional support or dissent for social identity theory. Finally, coding assumed language (e.g. “the average media user has many outlets at his disposal”) could shed light on the pervasiveness of simplistic language.
Regarding the survey, there are two prominent limitations (i.e., the sample size and recoding). The study did not meet the requirements for sample size based on the power analysis. Due to the limited time and scope of the thesis, it was not possible to recruit enough respondents for each survey group, although all sample sizes were within 10 respondents of the necessary amount. Regardless, the study results should be interpreted with caution. Furthermore, the diversity of the small sample sizes meant that some groups (e.g., self-identified Black people) had too few respondents to run meaningful significance tests. The
statistical tests needed to run the analyses for this paper do not match the diverse amount of possibilities that respondents present when identifying themselves. To solve both the issue of power and recoding, this study should be replicated on a much larger scale.
Contributions, Implications, & Concluding Remarks
Despite the limitations, this thesis offers a lot to the literature on measuring
demographics. Taking the first steps towards analyzing new measures of gender and ethnicity opens the door for further questions about presently underrepresented groups (Carrotte et al., 2016). The content analysis conducted addressed calls to evaluate the status of academic literature’s understanding and application of the measures (Chakravartty et al., 2018; Westbrook & Saperstein, 2015), while empirically testing and comparing showed what potential information these measures hold (Magliozzi et al., 2016; Westbrook & Saperstein, 2015). Together, these studies contribute new knowledge to both communication science and general demographics-related literature.
Decision making at every step in the research design is of great importance. This thesis has indicated that gender and ethnicity measures require more attention in this process. In the context of communication science, this thesis has tested six measures (i.e., three for gender and three for ethnicity) and proven the potential each has to garner new information. Academics should consider employing these and other similar measures (e.g., Davis et al., 2011; Miller & Weingarten, 2005) more frequently when formulating their research design. As with all research-related decision making, the research question must drive the process.
The choices researchers make regarding these measurements can have consequences; these relate to both understanding the science academically, as well as how researchers translate it to society (Chakravartty et al., 2018; Losty & O’Connor, 2018; Westbrook & Saperstein, 2015). Beyond potentially misrepresenting relationships, researchers using simplistic measures of gender and ethnicity systematically exclude people from their
research. Doing so has larger-scale consequences, such as the exclusion of minority people from social spheres and normalizing marginalization (Chakravartty et al., 2018; Foucault, 1980). The time to reflect on the use of demographic variables is now. This thesis has shown clearly that different measures of gender and ethnicity can have an impact on the results, and so academics should keep this in mind when developing future research.
References
References marked with one asterisk (*) indicate papers included in both the content analysis sample and the main body text. References marked with two asterisks (**) indicate papers used only in the content analysis sample.
**Allen, M., Wicks, R. H., & Schulte, S. (2013). Online environmental engagement among youth: Influence of parents, attitudes and demographics. Mass Communication and Society, 16(5), 661-686. doi:10.1080/15205436.2013.770032
American Psychological Association. (2010). Publication Manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological
Association.
Ansara, Y. G., & Hegarty, P. (2014). Methodologies of misgendering: Recommendations for reducing cisgenderism in psychological research. Feminism & Psychology, 24(2), 259-270. doi:10.1177/0959353514526217
**Appiah, O., Knobloch-Westerwick, S., & Alter, S. (2013). Ingroup favoritism and outgroup derogation: Effects of news valence, character race, and recipient race on selective news reading. Journal of Communication, 63, 517-534. doi:10.1111/jcom.12032 **Armstrong, C. L., & Collins, S. J. (2009). Reaching out: Newspaper credibility among
young adult readers. Mass Communication and Society, 12(1), 97-114. doi:10.1080/15205430701866592
**Armstrong, C. L., & McAdams, M. J. (2009). Blogs of information: How gender cues and individual motivations influence perceptions of credibility. Journal of Computer-Mediated Communication, 14, 435-456. doi:10.1111/j.1083-6101.2009.01448.x Baker, D. L., Schmaling, K., Fountain, K. C., Blume, A. W., & Boose, R. (2016). Defining
diversity: A mixed-method analysis of terminology in faculty applications. The Social Science Journal, 53(1), 60-66. doi:10.1016/j.soscij.2015.01.004
