Personality and gamification effectiveness for idea crowdsourcing, an experimental study

(1)

Personality and gamification effectiveness for

idea crowdsourcing, an experimental study

Author: Harro Engelen

Student number: 11998660

Date of submission: 22-06-2018

Version: Final

Qualification: Msc. Business Administration – Entrepreneurship and Innovation

Institution: University of Amsterdam

(2)

2

Statement of originality

This document is written by Student Harro Engelen who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

3

Abstract

The aim of this study is to gather empirical evidence regarding gamification effectiveness in the context of crowdsourcing creative ideas. An experiment is conducted in which an online idea crowdsourcing platform is set up aimed at generating creative ideas for fictional stories. On this platform 274 participants actively engaged by posting ideas and comparing the ideas of others. Gamification elements, which rewarded the participant’s performance with game-points with which they could level up, had been integrated into the platform. Yet, the gamifications elements were only visible to the treatment group. The treatment group compared significantly more ideas, compared to the control group. However, they did not post significantly more ideas. Furthermore, because the personality traits of the participants were measured using a questionnaire, conclusion could be made about how personality relates to crowdsourcing ideas and gamification effectiveness. It turned out that high levels of openness positively related to the number of ideas posted and neuroticism positively moderated the gamification effectiveness.

(6)

6

Motivation

Although the web has enabled great collaborative writings such as Wikipedia, a collaborative fictional story of similar scale and impact has never been achieved. Peer production often splits up the work into independent parts, yet a story is difficult to untangle. In contrast to a Wikipedia page, a story is not a representation of modularly separable pieces of information, instead, it is a multi-layered work with many interdependent characters, story elements, and settings. Furthermore, when collaborating on developing a single product, it is very likely that a conflict of opinion arises, especially with an experiential product such as a fictional story. The quality of a story is mostly assessed subjectively, it is often a matter of taste and opinion. There are no objective metrics on which people can immediately agree on. Unstructured attempts at collaborative creative writing, such as the experimental Wiki-novel A million Penguins (Mason and Thomas, 2008) eventually resulted in vandalism and uncertainty about the direction of the story. On the other hand, highly structured approaches such as Foldingstory (foldingstory.com) and Story Wars (storywars.net) perform better at generating stories. Here, the participants are constrained to contributing only one sentence or paragraph a time. This does, however, unable the people to collaborate at the level of the overall plot. Having constrictions may allow a consistent creative trajectory, but too many constrictions may suppress creativity. Examples of hybrid forms of collaborative writing are Ensemble (Cheng and Bernstein, 2014) and Scrittura Industriale Collettiva (scritturacollettiva.org). These projects are set up in such a way that a select group of individuals, the leaders, are responsible and have the power over the overall plot and story line. They carefully divide the story into independent parts and manage the development of the story. The crowd is able to contribute via a contest-mechanism where the best written pieces are selected and integrated into the story by the leaders. Hence, the leaders come up with the ideas for the story and set out the main story line enabling them to effectively instruct and coordinate the crowd in writing down the story in a coherent way.

(7)

7 The motivation behind this study is to make the first steps in exploring a more elaborate collaborative writing approach. One in which the crowd is both able to create and develop the plot and story line as well as the process of eventually writing down the whole story. Instead of considering the whole development project, this study only focusses on the first ideation phase. Thus, the process of crowdsourcing ideas for a fictional story.

Introduction

Companies increasingly apply crowdsourcing practices to collect new ideas for eventual innovation or new product development purposes. These can be open calls for new ideas from ordinary users which can potentially result in thousands of suggested ideas and perspectives. The process of generating ideas is also referred to as the Fuzzy Front End and is seen as the most important stage for a large number of new products (Koen et al., 2001). This is the starting point where opportunities are identified and creative ideas are generated before entering the formal product development process. It is important to notice the difference between creativity and innovation. According to Amabile (2000) innovation is: “… the successful implementation of creative ideas by an organization”. Hence, it distinguishes between generation of new ideas and their implementation. In this case, the focus is solely on using crowdsourcing as a tool for the generation of ideas. This process is often facilitated by an online platform. On this platform, a community of individuals is able to voluntarily share and collaborate on generating and developing their ideas. Such an Idea Crowdsourcing Platform (ICP) enables the organization to integrate their customers or end-users into the ideation processes for new product development conforming to the open innovation paradigm of Chesbrough (2003). Furthermore, empirical evidence showed that ordinary users in some cases are actually better able to come up with original and radical ideas compared to professional developers or technology experts. Results from the study of Schemmann et al. (2016) show that the popularity of ideas (based on input from the crowd) was positively related to the likeliness of the idea being judged as valuable by professionals from the requesting company. Certainly, not every ordinary user is able to come up with

(8)

8 valuable ideas, but in this case the crowd of ordinary users was collectively capable to both generate and identify valuable ideas and solutions.

The success of such crowdsourcing initiatives is strongly dependent on the commitment of the participants. These people must be self-motivated or at least driven by an external incentive (Amabile et al., 1994). Amabile et al. recognized two types of motivations; intrinsic and extrinsic. Intrinsic motivations come from within the person, performing a task provides satisfaction and enjoyment.The extrinsic motivations origins from outside the person, it is based on receiving direct or indirect economic advantage by performing a task. For example, monetary rewards, free final products, service or discounts (Ryan and Deci, 2000). One common method of motivating participants to engage in crowdsourcing projects is by implementing gamification elements.

The concept of gamification refers to the “use of game design elements within non-game contexts” (Deterding, Dixon, Khaled & Nacke, 2011, p. 1). The essential purpose behind gamification is to increase the user’s engagement, enjoyment and loyalty, which should result in better user performance. Gamification literature suggest that participants can have very different approaches towards gamification elements and how they interact with them, especially between different contexts. Yet, in most studies gamification is often treated as a uniform concept, the effects of certain gamification mechanisms in specific crowdsourcing contexts are not measured separately. While in practice, the gamification environments and context can be quite diverse. Morschheuser et al. (2017) concluded from their extensive literature review on gamification in crowdsourcing contexts that there are only a few studies which consider the moderating effects of personal factors of the participants on the impact of gamification. There is still little understanding of how different affordances affect motivational and behavioural outcomes. Hence, when designing gamification elements for an ICP, the personal factors as well as the context and overall aim of the ICP should be considered. Therefore, the goal of this study is to empirically investigate gamification elements through an experiment and to measure the impact of an isolated gamification effect by using isolated experiment groups.

(9)

9 Additionally, the personal factors and context are taken into account. It particularly focuses on implementing gamification elements on an ICP aimed at creative writing.

First, a literature review will provide the theoretical background regarding this topic. Based on this literature, the research questions are determined along with a conceptual model and hypotheses. After that, the research design aimed at testing the hypotheses and answering the research questions is described. All the steps taken to analyse the data and to test the hypotheses are then carefully documented. The paper eventually ends with a discussion, conclusion and limitations of the study.

Literature review

Crowdsourcing creative ideas

By actively crowdsourcing ideas, organisations are able to surpass their boundaries and to see their end-users or customers as a key resource. End-users and customers often have high product expertise and experience gained from regular product usage (Scheiner et al., 2017). Well-known companies, such as DELL, IBM, Startbucks, Google, Intel and BMW, have set up online platforms through which they effectively organize and build a community of product enthusiasts and creative minds. Studies from Dell and IBM, pioneers in the field of online idea crowdsourcing, have recognised different participant behaviours (Bayus, 2013; Stewart et al., 2010). They recognised that one of the biggest challenges in crowdsourcing is that only a few participants contribute ideas while some only comment on these ideas and the majority only reads the content without active participation. Based on literature, six main reasons for people to join and participate on such platforms have been found. The order at which the motivations are presented here has nothing to do with their relevance.

Starting with the first possible motivation; enjoyment. Participating could be experienced as fun and enjoying. It may offer people an opportunity to express their creativity and to challenge them intellectually. They may also be curious about the development process and want to be a part of it. Zheng (2011) concluded from their study that intrinsic motivation is more important than extrinsic

(10)

10 motivation. They suggested that in order to encourage participation, the platform should provide enough autonomy and variety and the work has to be challenging enough to keep the participant engaged. (Jokisch, 2007; Motzek, 2007 ; Antikainen et al., 2010)

Secondly, motivation could be based on altruism, especially when the platform is built for a particular social cause. People may feel they are able to help enhance a product or service or to help achieve a bigger goal shared by all the participants. They may act out of a feeling of social responsibility and contributing on the platform gives them a sense of efficacy. (Jeppesen and Frederiksen, 2006; Schattke et al., 2012)

Third motivation to participate may be to demonstrate capabilities and skills in order to get recognition for it. By showing competence to the public, firms or other participants, the user is able to build a reputation. When receiving positive feedback, it makes them feel proud which is likely to enhance their motivation, so it is self-reinforcing. The platform can also be used as a self-marketing tool. Some people may even feel obliged to be part of it, as it is their field of expertise or interest. (Jeppesen and Frederiksen, 2006; Motzek, 2007; Battistella and Nonino, 2013)

The fourth motivation could be that participants consider it to be effective way to learn or gain knowledge. By working on tasks on the platform the participant is able to practice certain skills in order to acquire or enhance them. Receiving feedback from other participants helps them improve their selves. (Jokisch, 2007; Battistella and Nonino, 2013)

The fifth motivation comes from the social needs of participants. Through the platform they may find and interact with like-minded people with whom they can build informal or formal relationships. Frequent contact with peers may lead to a shared a sense of companionship with the other participants or team members. (Bretschneider et al., 2015; Battistella and Nonino, 2013)

The last motivation refers to extrinsic based motivations. The participant may see the platform as an opportunity to receive monetary rewards, free products or services. But it could also be used as

(11)

11 a way to develop future potential returns. These are indirect results which may eventually provide the participant with certain benefits. For example, people may expect a form of reciprocity, so by helping others on the platform they expect to receive help in return at some point. (Battistella and Nonino, 2013)

Gamification

Research on gamification has greatly increased since 2010, it has shown to be a useful tool in changing user behaviour without having to use extrinsic incentives such as monetary rewards. There are many definitions of gamification, a more elaborate definition than the one mentioned in the introduction would be: “Gamification refers to a process of enhancing a service with affordances for gameful experiences in order to support user’s overall value creation.” (Huorari and Hamari, 2012, p20). This definition also refers to the use of game design elements, which may enhance the user’s experience and outcome. Additionally, it acknowledges the utilitarian aspect which is supporting value creation by the user. In the case of an ICP, the organization can implement gamification elements in such a way that the experience of contributing is enjoyable and fun. Morschheuser et al. (2017) indicate that the less complex crowdsourcing platforms are in most common cases complemented with rather simple point-based and leader board-based designs. When looking at more complex platforms the gamification methods used were often also more complex and more elaborate. Furthermore, they recommend considering the crowd diversity and task characteristics when choosing and designing suitable gamification elements. This is in line with the study of Star et al. (2016), they concluded from their literature review that the greatest increase in user performance due to the implementation of gamification is mostly dependent on the social design and meaningfulness of the gamification elements and mechanisms. The needs and goals of the participants as a part of the community should be well integrated and should provide a meaningful game-based experience. The underlying non-game aspects and goals should be well connected and in line with the gamification design.

Gamification elements can take many forms and sizes and can combine game design elements in many different ways. Commonly used game mechanisms are game points, social points, redeemable

(12)

12 points, levels, leaderboards and high scores, stories, virtual identities and collecting. Here follows a short description for each element and how it could be implemented on an ICP:

Game points; Participants receive game points for achievement of objectives. They provide the

participant with direct feedback of their performance, in case of an ICP, this could be for submitting, commenting or rating an idea. Game points allow participants to compare their performance with others and can foster competitive behaviour.

Social points can also support competitive behaviour, these points are rewarded by other users

(e.g. community rating). This represents direct qualitative feedback from others on for example the quality of an idea or comment. Thumbs up/thumbs down buttons or five-star rating scales are common examples of social point mechanisms often seen on community platforms.

Redeemable points can be seen as an in-game currency. Participants can spend these points to

purchase virtual or real goods. An economic mechanism gives the participants autonomy in development and differentiation.

Levels; by exceeding certain point thresholds or achieving certain objectives the participants can

level up. A participant’s level indicates their past performance and allows for inter-participant comparison as a user ranking mechanism. Thus, increasing the competitive nature of the platform. Levels can be sections where the game is divided into smaller subtasks where difficulty stays the same, or as stages, where difficulty increases per level.

Leaderboards and high scores enable participants to compare each other and themselves

regarding past performance. They can increase the visibility of their performance and give them special attention for it.

Stories can form a background narrative behind the platform giving it meaning and helps to focus

(13)

13

Virtual identities; Having an avatar can increase one’s self-esteem and confidence in overcoming

hierarchy levels. It enables participants to create anonymous identities expressing individuality and differentiation between participants.

Collecting; Rare items which can be collected by participants can motivate them to achieve

objectives. (e.g. badges for submitting the fifth/tenth idea or rating 20 ideas). It provides them with an extra opportunity to gain social recognition and supports the competitive behaviour.

(Source: Schneider et al., 2017)

Personality factors

Regarding personal factors of the participants, the personality traits are argued to be especially interesting in relation to gamification impact. In non-game contexts, such as education and workplace, personality has shown to moderate task performance and learning styles (Barrick, Mount & Strauss 1993, Kolb, Kolb 2005). Furthermore, gaming preferences are also affected by personality traits (Jeng and Teng 2008, Yee 2006). Gamification combines these domains and although it, in general, improves user performance, there may be significant differences between people due to different personality traits. Different people will be differently motivated. Furthermore, measuring personality offers a relatively stable measure to explain behaviour.

The widely accepted Five-Factor Model (FFM) (McCrae and Costa, 1989) offers a taxonomy of personality traits composed of: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Backed by plenty of empirical research, FFM shows consistency in measurements when compared across interviews, self-reports, and observations considering people from different ages and cultures (Barrick et al., 1993; Schacter et al. 2009). Here follows a short description for each trait.

Openness (or Open-mindedness): refers to the intellectual curiosity, imagination, creativity,

originality, perceptiveness, and joy of variety. People who score high on openness have the tendency to come up with novel ideas, hold unconventional values and don’t hold back in

(14)

14 questioning authority. Individuals who score low have a more conventional and conservative perspective.

Conscientiousness: refers to being orderly, consistent, decisive, punctual, and reliable. People

who score high on Conscientiousness prefer to be in control, they actively plan, organize and carry out tasks. A low score suggests the individual to be less precise and less directed towards a goal.

Extraversion: refers to the degree of sociability, assertiveness, adventurousness and active

engagement. Someone with a high score seeks out new opportunities, stimulation in the company of others, and is full of energy, enthusiasm and excitement. A low score means the individual is more reserved, self-reflective and independent.

Agreeableness: refers to the degree of being sympathetic to others, being cooperative, being

modest, showing affection and generosity. A high score means the individual is more likely to help others. A low score refers to people who are more competitive, egocentric and sceptical towards others.

Neuroticism; refers to the emotional stability of the individual. A high score means a low

emotional stability, and therefore the individual shows stronger emotional reactions towards events and has higher irritability and moodiness. These people experience fear, sadness, embarrassment, disgust, anger, and guilt more than those with a relatively lower Neuroticism score. People with lower scores are even-tempered, more relaxed and calm.

(source: McCrae and Costa, 1989)

Star et al. (2016) examined evidence of correlations between overall job performance and FFM Traits from a collection of studies through a most cited meta-analysis in the educational and business domains. For both domains, there is compelling evidence that there is a significant relationship between FFM measured personality traits and performance. See the table 1.

(15)

15 Table. 1: Meta-analysis results from Star et al. (2016)

Task Performance Job Performance Academic Performance Professional Performance Openness .06** .07* .12*** .13** Conscientiousness .24* .24* .24*** .29** Extraversion .09** .15** .17*** .11** Agreeableness .12** .11* .07*** .35* Neuroticism .-14* -.15* -.02*** -.09** Meta-Analysis Study Reference Hurtz, Donovan (2000) Barrick et al. (2001)

Poropat (2009) Peeters et al. (2006) Reported R2_{values based on true-score correlations.}

*p <.05, **p <.01, ***p <.001 Source; Star et al. (2016)

Personality and creativity

The meta-analysis from Feist (1998) showed that creative people tend to be more autonomous, introverted and open to new experiences. They are in general more self-confident, self-accepting, impulsive, driven and ambitious. Regarding creative scientists, conscientiousness and closed mindedness were negatively related to their overall creativity and artists were relatively more non-conformist than creative scientists. Furthermore, creativity is often associated with intuitive thinking and the ability to handle ambiguity.

Costa and McCrea (1985) found that openness, positively correlated with cognitive fluence and divergent thinking ability. Regarding creativity, researchers often refer to the definition of divergent thinking which is the thought process that leads to generating new creative ideas by exploring many possible solutions. Furthermore, McCrae (1987), Sen and Hagtvet (1993), and Stavridou and Furnham (1996) found positive relations between extraversion and divergent thinking. Yet, Götz and Götz (1979) clarified that his relation is only measurable with highly creative individuals.

(16)

16 They also showed that creative scientists and artists have a tendency towards neuroticism. Additionally, Kaufman (2001) found significant higher levels of psychopathology among poets, playwrights, journalist, and fiction writers compared to other groups. Yet, this relation was not found within other studies (McCrae, 1987). Agreeableness has an unclear relation to creativity. Some researchers suggested that creative persons are less agreeable and more independent, with their own taste and opinion. McCrae (1987) found no significant correlation. McCrae (1987) found strong correlation between conscientiousness and the likeliness of people to follow through with creative undertakings. Yet, this was only the case with self-report measures, peer-raters didn’t agree with this. Eventually, Wolfradt and Pretz (2001) confirmed in their study the positive relation between openness and creativity. They measured creativity over three measures: ratings of written stories, lists of personal hobbies and scores on the Creative Personality Scale (Cough, 1979). They also found extraversion to be positively related to creativity and neuroticism wasn’t a significant predictor of creativity.

Personality and crowdsourcing

There is very limited research regarding personality and crowdsourcing. Zhu et al. (2014) Studied the concept of internal crowdsourcing, this is the process of companies allowing all their employees to participate in the ideation process via an online platform (Hippel, 2005; Jeppesen and Lakhani, 2010). According to Simula and Vuori (2012) employees need to be open-minded in order for them to participate and early-adaptors are needed to convince other employees of the usefulness of the platform. Zhu et al. (2014) concluded from their study results on internal crowdsourcing that a person’s creativity and proactivity are important factors determining the origin of idea contribution in online idea crowdsourcing. These personality factors determine whether a person is likely to have a creative idea and whether he or she is willing to actively share and pursue the development of their ideas on a crowdsourcing platform.

Two other studies, both from Kazai et al. (2011 and 2012), focused on the relationship between participant characteristics and the quality of their work. For their crowdsourcing experiments

(17)

17 they payed people to perform relevance labelling tasks. This is highly repetitive work where creativity doesn’t play a role. They found openness and conscientiousness to be related to the labelling accuracy of the participants.

Gamification and creativity

It is shown that playing supports divergent thinking which can lead to creativity and new ideas. It is seen as the opposite of studying, working or other aspects of everyday life and it leads to free associations and floating thoughts as well as the ability to unlearn well-established associations (Dansky, 1980; Dansky and Silverman 1973; Simões et al., 2013). Other known psychological effects from gamification which are beneficial for generating new and better ideas are emotional reactions (Russ, 2004), self-efficacy (Bandura, 1976) and group affiliation, loyalty and equality (Simões et al. 2013). Feelings of competence and autonomy that occur when playing makes them feel responsible for their outcomes. Therefore, competitive gamification elements such as points and levels especially support these kinds of motivations and feelings.

Personality and Gamification

Regarding gamification, there are only a handful of published empirical studies on FFM in relation to Gamification effects. Codish and Ravid (2014) performed many studies in this field, two of them considered personality which are both published in 2014. One of them was about the impact of FFM traits on the perceived playfulness of gamification in educational and academic context. In this study, they implemented game points, leaderboards and badges to an academic course and found a significant moderating effect of personality traits. Yet, the effect sizes were rather small (𝑄2= 0.12 for leaderboards and 𝑄2= 0.14 for rewards). In their second study, they used the same setting but particularly focussed on the differences between Extroverts and Introverts. They found a significant difference in how certain gamification elements are related to playfulness between Extroverts and Introverts. For Extroverts, the enjoyment from leaderboards had a negative effect on the playfulness of the entire system. For Introverts, this relation was opposite yet not significant. Furthermore, Extroverts enjoyed badges significantly more than introverts.

(18)

18 Karanam et al. (2014) also studied gamification elements in relation to personality traits. Their study focused on game points, rewards, feedback, challenges and progress in the context of daily health habits (mood, fitness, diet, sleep). They found correlations between some personality traits and specific reward and challenge preferences. Yet, their sample size was rather small (n=36). Another study, from Jia et al. (2016), resulted in the statement that extraverts tend to be motivated by Points, Levels and Leaderboards and people with high levels of Openness are less likely to be motivated by Avatars. Furthermore, they found that high scores of Neuroticism negatively correlates with being motivated by Gamification elements.

The most elaborate study found on personality and gamification is from Star et al. (2016), they conducted a quasi-experiment in which they build a platform where the participants could collaborate between small groups in the collection, creation and sharing of digital artefacts for educational purposes. They made the distinction between competitive and cooperative gamification elements. Their study yielded significant evidence that incorporating gamification elements increased participant performance and that some personality traits had a moderating effect on the performance depending on which kinds of gamification elements were present. Extraversion positively moderated performance with competitive gamification elements and Openess positively moderated performance with cooperative gamification elements.

Gamification or Exploitationware?

There are also opponents towards gamification who label it as ‘exploitationware’. They argue that gamification methods are mainly used to extract real value from users and employees in return for worthless virtual tokens or artefacts. While playing and participating may be voluntary, gamification is often applied as a strategy to captivate attention and propel people towards certain behaviour in order to generate more revenue. They argue that the activity or product is only altered in how it is represented rather than fundamentally changed in its underlying purpose. (Bogost, 2011; Rey, 2015) However, it can also be argued that just like with any other medium, technology, form of persuasion or cognitive manipulation, it can be used positively or exploitatively (Anderson and Rainie, 2012). It is

(19)

19 also argued that well-designed gamification mechanisms which are aligned to its underlying purpose could provide people with better product or activity experiences and could also help them reach their own personal goals (Schneider et al., 2017).

Research question

The central aim of this study is to gain insights in how participant’s personality traits relate to their willingness to submit ideas and to help select the best ideas that are posted on a crowdsourcing platform. Additionally, the effect of gamification elements is part of this study. Based on literature, they appear to have positive effects on participant performance in crowdsourcing contexts. However, it cannot be expected that every individual will respond with the same behaviour towards the presence of a gamification element. It is not likely that the gamification element provides the game that everyone wants to play, given that the participants are unlikely to have homogeneous personalities. Some individuals may perform better due to the gamification element where this may be less for others or it may even be negatively affected. Therefore, this thesis also aims at gaining insights in how personality may moderate the effect of a gamification element. The research questions are;

1. How do individual personality traits relate to participant performance in crowdsourcing creative ideas?

2. To what extent does gamification influence participant performance on an idea crowdsourcing platform?

3. Are there moderating effects from the participant’s personality traits on the gamification effectiveness?

To answer the research question, an online experiment environment is set up. The website simulates an idea crowdsourcing platform with the aim to generate ideas for fictional stories. Participants can upload ideas and are able to assess the quality of the ideas by comparing them to each other. Gamification elements are integrated into the website, some of these elements can only be seen by

(20)

20 the treatment group. More information about the experimental setup will follow in the research design.

Conceptual model

The conceptual model (see figure 1) contains the independent variable which refers to the presence of a gamification element on a crowdsourcing platform. This can be true or false, it does not say anything about the quality or level of complexity of the gamification element. Other independent variables are the Personality traits of the Participants based on the FFM (McCrae, Costa 1989). The dependent variables are the Participant performance indicators; the Number of ideas posted, and the Number of comparisons made. The Personality traits are also expected to moderate the impact of the Presence gamification element on Participant performance.

Figure 1: Conceptual model

Hypotheses

Based on the literature multiple hypothesis are formed about the relations in the conceptual model.

1. Personality and Participant performance

As people with high scores in conscientiousness are shown to perform better at overall jobs it could be argued that they would also perform better in this context. Yet, this may not be case because this

(21)

21 study is about collecting creative ideas. While performance of tasks that require planning and coordination appear to be positively related to conscientiousness, task performance that require high levels of decision making or creativity are shown to be negatively related. (Neuman and Wright 1999, Barry and Stewart 1997). Therefore, hypothesis 1a is as follows:

H1a. Conscientiousness has is negatively related with Participant performance.

Because Openness is strongly associated with intellectual curiosity, imagination and creativity it is hypothesised that this personality trait will have a positive relation with Participant performance:

H1b. Openness has a positive relation with Participant performance.

Extraverted people are shown to be more assertive and more likely to show active engagement. Furthermore, Extraversion is often found to be positively related to creativity. Therefore, hypothesis 1c is as follows:

H1c. Extraversion has a positive relation with Participant performance.

Agreeableness is associated with being cooperative and generous. A high score means someone is more likely to help others. Yet, some researchers have suggested that creative people are less agreeable and more independent. Nevertheless, Mount et al. (1988) found that Agreeableness was a positive predictor of job performance, but only in jobs involving cooperation or dyadic interaction. Therefore, agreeableness is expected to have a positive relation with participant performance:

H1d. Agreeableness has a positive relation with Participant performance.

Studies have shown that creative scientists and artist have a tendency towards Neuroticism (Götz and Götz, 1979). Yet, Fang (2002) found Neuroticism to negatively affect the willingness to share knowledge with others. People with high scores on Neuroticism tend to be more insecure about themselves, are afraid to be embarrassed and tend to be more hostile towards other people. These characteristics negatively impact creativity and willingness to share ideas and cooperate. Hence, hypothesis 1e is as follows:

(22)

22 H1e. Neuroticism has a negative relation with Participant performance.

2. Gamification and Participant performance

Based on the rich body of literature on gamification it is hypothesized that Participant performance is higher when the gamification element is present. Although, it should be noticed that the effectiveness of the gamification element strongly depends on whether it is well designed or not. Nevertheless, hypothesis two is as follows:

H2. Participant performance is higher when a gamification element is present.

3. Moderating effects Personality traits on impact of Presence gamification element

Because gamification elements provide participants with feedback on how well they are performing, the performance of people with high Conscientiousness is expected to be more affected by the presence of a gamification element. Their high level of Conscientiousness may cause them to feel more responsible for their contribution. Therefore, hypothesis 3a is defined as:

H3a. Conscientiousness has a positive moderating effect on the relation between gamification and user performance.

Gamification elements may cause the participants to feel challenged and to be curious about what happens when they receive game points or achieve a level. People with high levels of Openness are in general more curious and creative. The effect from the presence of gamification elements is therefore expected to be higher for people with higher levels of Openness:

H3b. Openness has a positive moderating effect on the relation between gamification and user performance.

The behaviour of people with high levels of Extraversion is also expected to be more affected by the presence of gamification elements. These people are known to be more adventurous and to seek for external stimulation. Hypothesis 3c is therefore as follows:

(23)

23 H3c. Extraversion has a positive moderating effect on the relation between gamification and user performance.

Gamification elements often add a competitive element to an activity or project and less agreeable people are often more competitive and enjoy to challenge others. On the other hand, more agreeable people are known to be more cooperative. When receiving feedback from a gamification element, they may be more complaisant to this feedback and therefore more likely to alter their behaviour. Therefore, it the following is hypothesised:

H3d. Agreeableness has a positive moderating effect on the relation between gamification user performance.

High levels of Neuroticism can result in self-destructive behaviour and can decrease motivation, it is therefore often negatively correlated with performance (Shebaya, 2011). Furthermore, Jia et al. (2016) found Neuroticism to negatively moderate the effect of gamification elements. Based on this knowledge the following hypothesis is defined:

H3e. Neuroticism has a negative moderating effect on the relation between gamification and user performance.

Research design

Experiment setup and procedure

To test the hypotheses an online experimental study was conducted. Compared to observational studies, an experiment is a better fit as it allows you to draw conclusions about causal relationships. Doing an online experiment, where a real crowdsourcing platform is simulated, gives some of the benefits of both a lab experiment and field experiment. It is an in-between solution. Nevertheless, it does not give full control over who is actually active behind an account. It is possible that an account is shared by multiple people or that one person creates multiple accounts.

(24)

24 The experimental website (www.writealong.io) simulated a creative writing platform with the aim to generate ideas for fictional stories. Participants could upload ideas and make comparisons between the posted ideas. When making a comparison, two randomly picked ideas were presented and the participant were asked to guess which idea is more popular. They were correct when they chose the idea which is also relatively more chosen by others in other comparison sessions. Hence, the participants were rewarded for their similarity/agreement with other participants. Such scoring mechanism, which depends on the extent of agreement, motivated them to emulate others and to think and act like the community (Morscheuser, 2017). The gamification element contained a game point mechanism which rewarded a game point for each correct comparison and five game points when an idea was uploaded. When incorrect, a game point was retracted. By exceeding certain game point thresholds, the participant was able to level up, their game point-count was then set to zero and an icon which comes with their nickname would change. A progress bar showed how many points they needed to level up and it showed which icon they would get when they would level up again. When levelled up, the participant could not lose the achieved level, game points couldn’t go beneath 0.

As people entered the website they first saw a message about the experiment and the goal of the research. Then they could decide to participate by creating an account or not. When creating an account, participants were randomly assigned to the control or treatment group. After they had created an account they were able to fill in the FFM questionnaire and to play the game. The control group could not see the gamification element and the treatment group can (see figure 2 and 3).

(25)

25 Figure 2: What the treatment group sees:

Figure 3: What the control group sees.

In the control condition, the only activated game design elements, which were also shown in the treatment group, were the correct or incorrect message after a comparison was made. Also, all the participants were able to see how many comparisons they had made in total and how many of them were correct (as a percentage). Furthermore, all the participants were able to see how well their ideas were performing. Hence, how many comparisons their ideas had gone through and in how many of them their idea was chosen (as a percentage).

The participants were able to decide to stop at any moment and they could even come back to the website at a later time. With their account, they were able to log in again and continue playing the game. In the meanwhile, the website kept track of all the variables of each account created.

(26)

26

Sample

320 participants took part in the experiment. As a rule of thumb, a minimal of 40 participants are needed per treatment. Hence, the minimum was 80 participants. Yet, in order to also detect the small and moderating effects, it is of course better to have more. As mentioned before, the participants were randomly assigned to the experiment groups to prevent biased data. It is always a challenge to find participants for crowdsourcing initiatives and it is preferable to attract people who are intrinsically motivated to participate. Paid crowdsourcing options (e.g. Amazon Mechanical Turk) allows direct engagement with users in exchange for monetary rewards. However, the results from such an approach will probably be significantly different compared to when the participants are intrinsically motivated. Therefore, paid ads were used to attract people to participate. This is a good in-between option, the people who arrive at the site are not being paid, and when they choose to create an account and participate they do this out of intrinsic motivation. The added benefit of using ads was the possibility to target the right audience. For this study, Facebook ads were used which allowed the ads to be targeted to people who are interested in creative writing.

Measurement

Because this study was highly dependent on the number of intrinsically motivated participants. The questionnaire used to measure the Personality traits, couldn’t contain too many items, or else, people would have been more likely to abandon the experiment. The solution for this was the Mini International Personality Item Pool (IPIP) scales from Donnellan et al. (2006). This is a short form of the 50-item IPIP from Goldberg (1999). The Mini IPIP has been developed and validated across five studies, it has four items per Big Five trait and showed consistency and acceptable internal consistencies across the five studies with Cronbach’s Alpha values at or well above 0.60. See appendix A for the items and the Cronbach’s Alpha for each scale.

The independent variable, the presence of a gamification element is controlled and is only shown to the treatment group. The gamification element contains two of the most common game elements:

(27)

27 points and levels. The dependent variables, Number of ideas posted, and Number of comparisons made, are automatically kept track of for each registered account on the website.

Data analysis

Preparing the data and descriptive analysis

Eventually 320 people, who all found the website via a Facebook ad, took part in the experiment. In order to analyse the data, first some cases had to be deleted. All data collected from friends, family and relatives who took part in testing the website were removed. Only intrinsically motivated people who reached the website via a Facebook advertisement are relevant. For an overview of frequencies, dichotomous variables have been made from the ComparisonCount and IdeaCount showing if a participant has at least made one comparison or posted at least one idea. See the crosstab tables in appendix B. The next step was to remove cases with missing information. Those who didn’t make any comparisons, didn’t post any ideas, and didn’t fill in the survey have been removed, assuming they had left the experiment directly after creating an account. These cases were rather evenly distributed over the experiment groups. See table 2 for the relevant frequencies and relative percentages.

Table 2: Crosstabs after removing missing data cases

Treatment Total 0 1 Count 140 134 274 % of Total 51.1% 48.9% 100.0% Filled in the Survey No Count 9 5 14 % within Trtmnt 6.4% 3.7% 5.1% Yes Count 131 129 260 % within Trtmnt 93.6% 96.3% 94.9%

(28)

28 Posted at least one idea No Count 97 103 200 % within Trtmnt 69.3% 76.9% 73.0% Yes Count 43 31 74 % within Trtmnt 30.7% 23.1% 27.0% Made at least one comparison No Count 10 11 21 % within Trtmnt 7.1% 8.2% 7.7% Yes Count 130 123 253 % within Trtmnt 92.9% 91.8% 92.3%

In the treatment group, 23,1% of the participants posted at least one idea. This is less compared to the control group where 30.7% posted at least one idea. Also, in the treatment group relatively less participants made at least one comparison (91,8%) compared to the control group (92,8%) but this is just a minor difference.

Both the Number of ideas posted (IdeaCnt) and the Number of comparisons made (CompCnt) are not normally distributed, so a non-parametric test should be used for analysing the data. Furthermore, both variables show some outliers with Z values larger than 3. See the histograms in appendix C.

Many crowdsourcing initiatives are for a large part dependent on the effort of such outliers. Often the ‘90-9-1 ′ participation rule is recognised in crowdsourcing projects, meaning that only 1% of the users perform almost all of the actions needed to create value (Tinati et al., 2016). In this case, eight out of ten outliers lie on the treatment side. To get more insight on the effect of the outliers, different descriptives tables are generated, see Table 3.

(29)

29 Table 3. Descriptives of the data including and excluding the outliers

N Minimum Maximum Mean Std. Deviation

IdeaCnt 274 0 7 .39 .836

CompCnt 274 0 364 20.31 38.789

Valid N (listwise) 274

Descriptives from Independent T-Test including the outliers:

Trtmnt N Mean SD SE CompCnt 0 140 14.94 19.515 1.649 1 134 25.92 51.266 4.429 IdeaCnt 0 140 .39 .707 .060 1 134 .38 .956 .083

Descriptives from Independent T-Test without the outliers:

The descriptives tables show that the outliers have a big impact on the means and standard deviations. Instead of trimming the outliers another option is to winsorize the data. Winsorizing is the transformation of statistics by limiting extreme values to reduce the effect of possibly spurious outliers. To winsorize the data, all the outliers (Z > 3) have been given another value. These values are set three SDs away from the mean:

(30)

30 Mean (total) + 3 * SD (total) = new value for outliers

CompCnt: 20.31 + 3 * 38.789 = 136,677 (rounded off to 137) IdeaCnt: .39 + 3 * .836 = 2.898 (rounded off to 3)

See table 4 for the descriptives and figure 4 and 5 for the histograms of the data after it has been winsorised.

Table 4: Descriptives from Independent T-Test when the four outliers are winsorised

(31)

31 Figure 5: Histograms of the winsorised data, CompCnt

Survey validation and reliability analysis

To correctly analyse the results from the questionnaire, some items had to be recoded first because they were counter-indicative items. In total, eleven items had to be recoded. After that, the reliability of each scale was analysed regarding their Cronbach’s Alpha values. From this analysis it turned out that the Extraversion, Agreeableness, and Openness scale had high reliability with Cronbach’s Alpha values all above .626. The corrected item-total correlation for each item in the scales had a good correlation with the total scores of the scales (all above .30).

However, the Conscientiousness and Neuroticism scale showed low reliability. The Conscientiousness scale had a Cronbach’s Alpha of .610 which is acceptable but not very convincing. By leaving out the second item “Like order.”, which had a low corrected item-total correlation (.250), the Cronbach’s Alpha was slightly improved to a value of .634. The Neuroticism scale had a Cronbach’s Alpha of .321, after deleting the fourth item “Seldom feel blue.”, which had a negative corrected item-total correlation of -1.89, the Cronbach’s Alpha increased to an acceptable .614. The third Neuroticism item could also have been removed to increase the Cronbach’s Alpha to .638 but is preferable to have at least 3 items per scale. See appendix D for more information about the reliability analyses.

(32)

32 To further analyse and evaluate the scales, the shared variances between the items and underlying factors are determined by performing a principal axis factoring analysis (PAF). See appendix E for a table with the correlated factor loadings for each item. The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis, KMO = .685 and the Bartlett’s test of sphericity χ² (190) = 1287.008. p < 0.001, indicated that correlations were sufficiently large for the PAF analysis. As expected, five components with eigenvalues above 1 were found, which combined explained 54.553% of the variance. The five factors were retained and rotated with an Oblimin with Kaiser normalization rotation. The items do not cluster on the same factors, see the table 6.

Table 6: Oblimin with Kaiser normalization rotation table

Component Ex Ne Co Op Ag Ex 1.000 -.112 .080 -.078 -.162 Ne -.112 1.000 .018 .106 .066 Co .080 .018 1.000 -.012 -.016 Op -.078 .106 -.012 1.000 .153 Ag -.162 .066 -.016 .153 1.000

Eventually, after thoroughly validating the scales, the items were computed into single variables. With these variables and the Trtmnt, IdeaCnt and CompCnt variables a correlation analysis is applied which resulted in table 7.

(33)

33 Table 7: Means, Standard Deviations and Correlations

Mean Std. Deviation ExTOT AgTOT CoTOT NeTOT OpTOT Trtmnt IdeaCnt CompCnt

ExTOT 2.9452 0.91619 (.772) AgTOT 3.9606 0.78175 .289** (.744) CoTOT 2.7808 0.88762 .139* 0.073 (.634) NeTOT 3.2744 0.85496 -.159* -0.081 -.240** (.614) OpTOT 4.2558 0.63858 .188** .266** 0.053 -.147* (.626) Trtmnt 0.49 0.501 .123* 0.021 -0.012 -0.016 0.066 - IdeaCnt 0.35 0.665 -0.03 0.015 -0.035 0 .149* -0.038 - CompCnt 18.09 25.044 -0.075 -0.016 -0.029 -0.059 0.05 .129* .401** -

* Correlation is significant at the 0.05 level (2-tailed). ** Correlation is significant at the 0.01 level (2-tailed).

No significant correlations is found between the Personality traits and CompCnt. There is however a significant correlation between the Personality trait Openness and IdeaCnt (.149) which is significant at the 0.05 level. Furthermore, there is an unwanted significant correlation between Extraversion and the Treatment. Yet this is only a small effect (.123). Nevertheless, this correlation has to be taken into account when testing the hypotheses.

Hypothesis testing

Hypothesis 1

To test the first hypothesis, two multiple regression analyses were performed with the CmpCnt and IdCnt as dependent variables and the Personality traits as independent variables. With a multiple regression analysis the predictive values of the Personality traits for CompCnt and IdCnt are determined. See the results in table 8 and 9.

(34)

34 Table 8: Results from multiple regression analysis with the dependent variable CompCnt

Regarding the CompCnt, all variables except OpTOT, had a negative Beta-value meaning that as the personality traits are higher, the CompCnt is lower. When OpTOT is higher, the CompCnt is also higher. Nevertheless, none of these predictive values are statistically significant. This means that based on this data, regarding the Number of comparisons made, the Participant performance cannot be predicted based on these Personality traits.

Table 9: Results from multiple regression analysis with the dependent variable IdCnt

Statistical significance: *p <.05 R R2 _B _SE _β _t Model .124 .015 CoTOT -1.041 1.856 -.036 -.561 ExTOT -2.504 1.850 -.090 -1.354 AgTOT -.325 2.182 -.010 -.149 NeTOT -2.195 1.946 -.074 -1.128 OpTOT 2.428 2.619 .061 .927

Dependent variable: IdCnt

R R2 _B _SE _β _t Model .165 .027 CoTOT -.026 .049 -.035 -.539 ExTOT -.038 .049 -.052 -.785 AgTOT -.010 .057 -.011 -.170 NeTOT .005 .051 .006 .097 OpTOT .175 .069 .165* 2.533

(35)

35 The results from the multiple regression with IdCnt as the dependent variable lead to the conclusion that only hypothesis 1b is partly confirmed. OpTOT is the only independent variable that has a significant predictive value (β = .165) on the dependent variable IdCnt. This means that based on this data, regarding the Number of ideas posted, the Participant performance can (for a small part) only be predicted based on the Personality trait Openness. Although, the predictive value is very limited because the effect of Openness is very small. The model explains only 2.7% of variance in IdCnt.

Hypotheses 2

To test hypothesis 2, the non-parametric Mann Whitney analysis is applied. In this analysis all the cases are ranked and the difference in mean ranks shows the difference in means between the groups. CompCnt has a higher mean rank in the treatment group compared to the control group. And the analysis shows that this is a significant difference. IdCnt is actually lower in treatment group but this difference is not significant. See table 10.

Table 10: Results Mann Whitney analysis

Ranks

Trtmt N Mean Rank Sum of Ranks

CompCnt 0 140 128.73 18022.50 1 134 146.66 19652.50 Total 274 IdCnt 0 140 142.03 19884.50 1 134 132.76 17790.50 Total 274 ComparisonCount IdeaCount Mann-Whitney U 8152.500 8745.500 Wilcoxon W 18022.500 17790.500

(36)

36

Z -1.874 -1.247

Monte Carlo Sig. (1-tailed) .033b _.101b

99% confidence interval Lower bound .028 .093

Upper Bound .037 .109

a. Grouping Variable: Treatment

b. Based on 10000 sampled tables with starting seed 2000000.

Because there is an unwanted correlation between Extraversion and Treatment, a regression analysis is run to see if the effect of the treatment still stands after controlling for Extraversion. See table 11.

Table 11: Results regression analysis with dependent variable CompCnt

Statistical significance: *p <.05;

In the first step of hierarchical multiple regression, the predictor Extraversion (ExTOT) was entered. This model only explained 0.6% of variance in CompCount and was not statistically significant F (1, 258) = 1.45; p > .05. After entry of Trtmnt at step 2 the total variance explained by the model as a whole was 2.6% F (2, 257) = 3.447; p < 0.05. The introduction of Trtmnt explained an additional 2.1% variance in CompCnt, after controlling for Extraversion (R2_{Change = 0.021; F (1, 257) = 5.423; p <}

0.05). This means that the difference in CompCnt between the two groups due to the treatment is still valid. R R2 _R2 _Change _B _SE _β _t Step 1 .075 .006 ExTOT -2.081 1.730 -.075 -1.203 Step 2 .162 .026 0.021 ExTOT -2.575 1.729 -.092 -1.489 Trtmnt 7.362 3.162 .144* 2.329

(37)

37 This leads to the conclusion that hypothesis 2 is partly confirmed. Participant performance is, regarding the Number of comparisons made positively related to the Presence of a gamification element.

Hypotheses 3

To test the moderation effect of Personality traits on the impact of Gamification elements on Participant performance, regression analyses are applied to determine the interaction effects. See tables 12 and 13 for the coefficient values for each interaction effect and their p-values.

Table 12: Interaction effects, dependent variable IdCnt

Interaction Coefficient P-value

Trtmnt x ExTOT -.0684 .4293

Trtmnt x AgTOT .0187 .8226

Trtmnt x CoTOT -.1206 .2699

Trtmnt x NeTOT .1397 .1553

Trtmnt x OpTOT -.0995 .4128

Table 13: Interaction effects, dependent variable CompCnt

Interaction Coefficient P-value

Trtmnt x ExTOT -4.7294 .1909

Trtmnt x AgTOT -1.5783 .7632

Trtmnt x CoTOT -1.6261 .6274

Trtmnt x NeTOT 7.4435 .0382

Trtmnt x OpTOT -7.6305 .1042

The only significant moderating effect found in this data is the Trtmnt x NeTOT effect with CompCnt as dependent variable (c3 = 7.4435, p-value = .0382). The interaction term (or coefficient) determines

(38)

38 how much the effect of Trtmnt on CompCnt is different between people with a high and low score in Neuroticism. This interaction is significant, meaning that the effect of the Gamification element on the Number of comparisons made depends on the Neuroticism score of the participant. More specifically, by being in the treatment group, the difference in CompCnt between low and high scoring participants on Neuroticism increases by 7.4435 units.

Moreover, the analysis results show that this model only accounts for 3.64% of the variance in CompCnt. And a closer inspection of the conditional effects indicate that the relationship between Treatment and CompCnt is only significant when Neuroticism is around or above the mean. See table 14.

Table 14: Regression analysis results of interaction effect NeTOT x Trtmnt with DV CompCnt

Coefficient SE T p Intercept i1 18.56 1.58 11.72 <.001 Trtmnt (X) c1 6.75 3.18 2.12 .0347 NeTOT (M) c2 -1.48 1.79 -.83 .4069 NeTOT*Trtmnt c3 7.44 3.57 2.08 .0382 R2 _{= 0.0364 p = 0.0046} F(3,256) = 4.4415

Conditional effect of Trtmnt (X) on CompCnt (Y) at values of NeTOT (M)

NeTOT value Effect SE t p

Minus 1 SD .3834 4.4323 .0865 .9311

Mean 6.7473 3.1783 2.1230 .0347

(39)

39 The graph in figure 6 clearly depicts the moderating effect and the histogram in figure 7 gives some more insights about the data from which the moderated effect is determined. From these results it can be concluded that the data contrasts with what was hypothesised in hypothesis 3e, Neuroticism does not have a negative moderating effect on the relation between gamification and user performance. Instead, it positively moderates the relation between gamification and user performance.

Figure 6: Graph of interaction effect of NeTOT x Trtmnt with DV CompCnt

(40)

40

Overview hypotheses

Table 15 provides an overview of all the hypotheses and whether or not they are confirmed based on the results of this study. Only hypothesis 1b, regarding the Number of ideas posted, and hypothesis 2, regarding the Number of comparisons made, are confirmed. Hypothesis 3e, regarding the Number of comparisons made, is actually contradicted by the results of the analysis.

Table 15: overview of hypothesis and outcome

Hypothesis Participant performance

indicator

Outcome

H1a. Conscientiousness has is

negatively related with Participant performance.

Number of comparisons made Not confirmed Number of ideas posted Not confirmed

H1b. Openness has a positive relation with Participant performance.

Number of comparisons made Not confirmed Number of ideas posted Confirmed

H1c. Extraversion has a positive relation with Participant performance.

H1d. Agreeableness has a positive relation with Participant performance.

H1e. Neuroticism has a negative relation with Participant performance.

H2. Participant performance is higher when a gamification element is present.

Number of comparisons made Confirmed Number of ideas posted Not confirmed H3a. Conscientiousness has a

positive moderating effect on the relation between

gamification and user performance.

Number of comparisons made Not confirmed

Number of ideas posted Not confirmed

H3b. Openness has a positive moderating effect on the relation between gamification and user performance.

(41)

41 H3c. Extraversion has a

positive moderating effect on the relation between

H3d. Agreeableness has a positive moderating effect on the relation between

gamification user performance.

H3e. Neuroticism has a negative moderating effect on the relation between

Number of comparisons made Not confirmed (contradicted)

Discussion, limitations and future research

The two gamification elements; game-points and levels have been implemented in an online platform aimed at crowdsourcing creative ideas for fictional stories. The game elements were only visible to the treatment-group in the experiment. This made it possible to assess the effects that the presence of the gamification elements had on participant performance. Their performance was measured by the number of ideas posted and number of comparisons made between the ideas that were posted on the platform. By doing this, the study addressed the lack of empirical evidence from experimental studies regarding gamification and crowdsourcing (Morschheuser, 2017). The outcome, derived from 274 participants who actively engaged with the platform, provided statistically significant differences in performance, which could be assessed in relation to their personality traits and whether they were able to see the gamification element or not. Based on this data, the following research questions can be answered:

1. How do individual personality traits relate to participant performance in crowdsourcing creative ideas?

Of the five measured personality traits, only Openness showed a positive relation with Participant performance. Yet, this was only regarding their performance in posting ideas. Hence, people with

(42)

42 higher levels of openness appear to be more likely to share their ideas on a crowdsourcing platform. This is in line with the hypothesis that was formulated. People with higher levels of openness are more open to new experiences, are better able to understand abstract ideas and also better at coming up with new creative ideas. The number of comparisons made was not significantly related to any of the personality traits. Perhaps this is due to the simple nature of just comparing two ideas with each other. No specific skills or characteristics are needed to be ‘good’ at this.

2. To what extent does gamification influence participant performance on an idea crowdsourcing platform?

The participants in the treatment group, who were able to see the gamification elements, made on average 43% more comparisons compared to the control group, who couldn’t see the gamification elements. Apparently, the points and level elements prompted participants to compare more ideas. This was a statistically significant difference between the two groups. Yet this increase was not the case for the number of ideas posted. This was actually lower for the treatment group (-13%). Perhaps this is due to relatively simple nature of comparing ideas, which makes it easier to influence through gamification.

3. Are there moderating effects from the participant’s personality traits on the gamification effectiveness?

All but one personality trait showed a significant moderation effect. It was hypothesised that neuroticism would negatively moderate the impact of gamification effectiveness. Yet, this was not the case. Instead it positively influenced the gamification effectiveness. This was only the case for the influence on the number of ideas compared not on the number of ideas posted. The positive moderation effect from neuroticism may be explained by the fact that gamification elements can help people to motivate themselves and promote self-improvement. Buunk et al. (2005) found a significant correlation between Neuroticism and social comparison and gamification enables people to compare their score with others. Furthermore, receiving positive feedback from gamification elements may increase one’s self-esteem and confidence.

Personality and gamification effectiveness for idea crowdsourcing, an experimental study

Personality and gamification effectiveness for

idea crowdsourcing, an experimental study

Statement of originality

Contents

Abstract

Motivation

Introduction

Literature review

Crowdsourcing creative ideas

Gamification

Personality factors

Personality and creativity

Personality and crowdsourcing

Gamification and creativity

Personality and Gamification

Gamification or Exploitationware?

Research question

Conceptual model

Hypotheses

1. Personality and Participant performance

2. Gamification and Participant performance

3. Moderating effects Personality traits on impact of Presence gamification element

Research design

Experiment setup and procedure

Sample

Measurement

Data analysis

Preparing the data and descriptive analysis

Survey validation and reliability analysis

Hypothesis testing

Hypothesis 1

Hypotheses 2

Hypotheses 3

Overview hypotheses

Discussion, limitations and future research