Author
H.C. van Akkeren
Oude Kijk in’t Jatstraat 40a 9712 EL Groningen
Student number 1322044
University of Groningen
Faculty of economics and business MSc BA Business Development Supervisors #1: dr. C. Reezigt Supervisors #2: dr. J.D. van der Bij
Connaxis S.A.
drs. P. van Grinsven Malabia 1865
C1414DMK Buenos Aires; Argentina
Master Thesis Business Administration – Business Development
Learning the rules of
crowdsourcing contests
An explorative case study into the world of crowdsourcing contestsCrowdsourcing, together with similar applications that tab into the collective intelligence of groups, garnered considerable interest with organizations, the academic world and the general public (Bonabeau, 2009). Although the term crowdsourcing was only coined in 2006, it is already used for a wide diversity of applications. A fair amount of crowdsourcing platforms/ websites have emerged but until now little is known about the functioning of these platforms. This explorative study delves into the field of crowdsourcing contests which is the most popular form of crowdsourcing.
These crowdsourcing contests have major influence on several industries like the (graphic) design industry on which this study focuses. Aim of the research is to get insight into the crowdsourcing contest mechanism of these platforms and their main challenges. Subsequently this study will come with propositions for further research on crowdsourcing contests.
2 Master Thesis Business Administration – Business Development
Learning the rules of
crowdsourcing contests
An explorative case study into the world of crowdsourcing contestsAuthor
H.C. van Akkeren
Oude Kijk in’t Jatstraat 40a 9712 EL Groningen
Student number 1322044
University of Groningen
Faculty of economics and business MSc BA Business Development Supervisors #1: dr. C. Reezigt Supervisors #2: dr. J.D. van der Bij
Connaxis S.A.
drs. P. van Grinsven Malabia 1865
3
Preface
Crowdsourcing was one of the many topics that were discussed during my final year (MSc BA) Business Development. The concept of crowdsourcing has intrigued me ever since, so when I got the chance to do research after this phenomenon I didn’t hesitate to take this chance. At first, I was both curious as skeptic about this new business model which is facilitated by the possibilities that internet brings. I experienced that the possibilities of the internet are indeed endless and give rise to new ways of doing business. Crowdsourcing is clearly one of them and raised a lot of discussion on the web which made it even more interesting for me to do research after this concept.
My initial skeptical and critical stance towards crowdsourcing made me choose a comprehensive view on crowdsourcing contests for my research to ensure that all the motives, advantages and disadvantages for all stakeholders were captured. Together with an extended literature review this case study has lead to clear insights in the crowdsourcing (contest) mechanism and provides readers with a fair view on things.
Thanks to Connaxis S.A., especially Peter van Grinsven and the Guerra Creativa team, I got the opportunity to explore this new business model of which I’m very grateful. The research findings have offered them tools and insights to improve their platform and create more value for all stakeholders. This thought of creating value for all stakeholders is an important element to legitimate the concept of crowdsourcing contests.
Next to the people at Connaxis I’d like to express my gratitude to Kees Zoethout and Cees Reezigt for the continuous support and advice. Their help has lead to a paper which is very interesting to read for everyone who is somehow attracted to the world of crowdsourcing contests.
The amount of knowledge and talent dispersed among the human race has always outstripped our capacity to harness it. Crowdsourcing corrects that – but in doing so, it also unleashes the forces of creative destruction.
4
Executive Summary
Crowdsourcing is a newfound business model that tabs into the collective intelligence of groups and garnered considerable interest with organizations, the academic world and the general public (Bonabeau, 2009). The concept can be defined as: “the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call" (Howe, 2007).
This explorative study delves into the field of crowdsourcing contests which is the most popular form of crowdsourcing. A typical process of crowdsourcing (contests) is described by Brabham (2008a) as follows:
1. A company posts a problem/ task online;
2. A vast number of individuals offer solutions to the problem/ task (enabled by the internet); 3. The winning ideas are awarded some form of a bounty (often prize money);
4. The company mass produces the idea for its own gain.
This study contributes to the scientific research on crowdsourcing contests by offering a more comprehensive view on crowdsourcing contests and absorbs the current available knowledge. By exploring the motives and interests of all stakeholders (contest holders, intermediary organization and the community) as well as the functioning of the crowdsourcing contest mechanism a greater insight in the concept of crowdsourcing contests is gained.
The research consisted of two parts with the first part being a literature study. The literature study concludes with a theoretical model that was used as a guide for a case study. Research object for this case study was a popular Latin American crowdsourcing contest platform for low-end design work, Guerra Creativa. The case study included a wide-ranging survey research among all stakeholders, a document study as well as direct and participation observation. The findings from the case study were then confronted with the earlier found principles from theory. This has lead to the following four propositions that can serve as clear starting points for future crowdsourcing research.
P1: For participants of crowdsourcing contests with relatively uncomplicated tasks, direct compensation is more important than social motives.
P2: The interests and motives of contest holders and the (semi) professional contest participants are conflicting.
P3: With crowdsourcing contests, the amount and form of the direct compensation is the most important predictor for the amount and idea quality of the ideas.
P4: Intermediary organizations can significantly increase the quality of ideas by facilitating tools that foster the creativity and learning of the creatives.
The theoretical model which is presented in this paper can be very helpful for future and existing crowdsourcing contest platforms to test the design of their platform (e.g. as a checklist).
5
Table of contents
Chapter 1: Introduction ...8
1.1 Introduction to Crowdsourcing ...8
1.2 Crowdsourcing contests ... 10
1.3 Crowdsourcing in the design industry ... 10
Chapter 2: Research Design ...12
2.1 Research Strategy... 12
2.2 Research model... 12
2.3 Methodology... 13
2.3.1 Literature review ... 13
2.3.2 Case study ... 13
2.3 Explorative Data Analysis and Testing ... 16
2.4 Introducing Guerra Creativa ... 17
Chapter 3: Theory ...18
3.1 Crowdsourcing contest mechanism ... 18
3.1.1 Collective Intelligence ... 19
3.1.2 Fostering (collective) creativity ... 19
3.1.3 Organizing (idea) contests ... 20
3.2 Stakeholder requirements – Community ... 21
3.3 Stakeholder requirements – Clients ... 23
3.4 Stakeholder requirements – Intermediary organization ... 24
6
Chapter 4: Case study – Guerra Creativa ...27
4.1 Crowdsourcing contest mechanism ... 27
4.1.1 Organization of contests ... 27
4.1.2 Crowd composition ... 28
4.1.3 Prize money vs number of designs ... 29
4.2 Community ... 30
4.2.1 Motives to participate ... 30
4.2.2 Fostering creativity ... 31
4.3 Clients ... 32
4.4 Intermediary Organization ... 34
Chapter 5: Results – Confronting theory and practice ...35
5.1 The crowdsourcing mechanism ... 35
5.2 Stakeholder requirements – Community ... 36
5.3 Stakeholder requirements – Clients ... 37
5.4 Stakeholder requirements – Intermediary organization ... 37
5.5 Wrap-up confronting theory and practice... 38
Chapter 6 - Conclusion & Discussion ...39
6.1 Conclusion ... 39
6.2 Discussion ... 40
7
Appendix A - Overview of several crowdsourcing sites for graphic design...49
Appendix B – Questionnaire Creatives (English) ...50
Appendix C – Interviews Intermediary organization ...57
Appendix D – Questionnaire Contest Holders (English) ...63
Appendix E – Test of Normality – Creatives survey ...68
Appendix F – Test of Normality – Contest Holders Survey ...71
Appendix G - Overview of requirements/ implications identified in literature. ...72
Appendix H – Creative Brief ...73
Appendix I – Contest gallery ...74
Appendix J – Crowd composition ...75
Appendix K – Contention uniformity ...77
Appendix L – Professionals vs Hobbyists ...80
Appendix M – Regression Analysis ...83
Appendix N – Family ties at Guerra Creativa? ...84
Appendix O – Accomplishment ...85
Appendix P – Winning creatives are active contenders ...86
Appendix Q – Fostering Creativity ...87
Appendix R – Contest Holders positively rate proposed benefits ...88
Appendix S – iStockphoto versus Guerra Creativa ...89
8
Chapter 1: Introduction
Crowdsourcing, together with similar applications that tab into the collective intelligence of groups, garnered considerable interest with organizations, the academic world and the general public (Bonabeau, 2009). Although the term crowdsourcing was only coined in 2006, it is already used for a wide diversity of applications. A fair amount of crowdsourcing platforms/ websites have emerged but until now little is known about the functioning of these platforms.
This explorative study delves into the field of crowdsourcing contests which is the most popular form of crowdsourcing. These crowdsourcing contests have major influence on several industries like the (graphic) design industry on which this study focuses. Aim of the research is to get insight into the crowdsourcing contest mechanism of these platforms and their main challenges. Subsequently this study will come with propositions for further research on crowdsourcing contests.
The structure of this paper is as follows, chapter 2 will describe the research design and subsequently a literature review of crowdsourcing (related) literature which concludes with a theoretical model (chapter 3). This theoretical model will be used as a guide for a case study performed at a crowdsourcing platform (chapter 4) and is confronted with the findings from this case study (chapter 5). This results in interesting findings which lead to propositions that give direction to future research (conclusion).
This chapter will start by giving an introduction to the term crowdsourcing and its applications (1.1). This will give more insight into the rise of crowdsourcing and the potential of this new form of collaboration. Although there are different types of crowdsourcing this research focuses on crowdsourcing contests (1.2) in the design industry. The way crowdsourcing has reshaped the graphic design industry is described in 1.3.
1.1 Introduction to Crowdsourcing
Corporations are looking more and more outside their firm’s boundaries for ideas and intellectual property (Chesbrough, 2003, 2007). In combination with rise of the internet and its possibilities, a new social and business environment is created (Benkler, 2006) where new forms of collaboration with consumers and the general public arise (Molane, 2010). Crowdsourcing is such a new form of collaboration and enables companies to tap of the collective intelligence of individuals who couldn’t be reached through traditional business models.
The term crowdsourcing was first coined in 2006 (Howe, 2006) and can be defined as “the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call" (Howe, 2007).
9
This business model has clear links with outsourcing, but instead of outsourcing specific tasks to a professional workforce these tasks are performed by a group of unknown individuals which are (often) unpaid and carry out tasks in their spare time (Boutin 2006). This group (community) consists of amateurs, volunteers, experts or small businesses which are generally unknown to the initiating organization (McAfee, 2006). These “Pro-Am” workers are, when networked, working to professional standards and thereby playing an important economic role (Leadbeater & Miller, 2004; Repo et al., 2006). It is proposed that amateurs can provide services similar to those provided by professionals (Pantzar & Shove, 2005) but as well are able to create non-standard products.
This increasing importance of amateurs is in line with the work of Von Hippel (1988, 2005) in where amateurs (e.g. in the form of innovation communities, lead users) represent an important source of innovative solutions.
However, critical voices exist about the negative effects of the rise of amateurism (Keen, 2007). Especially the lack of trust in the quality and uncertainty in production time can lead to dissatisfaction among companies with their adoption of the crowdsourcing principle.
Main principle is that under the right circumstances this group of motivated individuals, the crowd/ community, performs better than the smartest people in them by not averaging solutions but aggregating from them (Surowiecki, 2004; Libert & Spector, 2007)). The applications of the crowdsourcing model are not limited to solely the computing sector where this form of labor organization was pioneered (Whitla, 2009,) but ranges across industries and include most functions in contemporary business (Viitamaki, 2008).
Crowdsourcing could not exist without the coming of the internet. The web provides the technology to aggregate ideas from a vast number of individuals and networks (Lévy, 1997 [1995]; Terranova, 2004) from all over the world. Not only the barrier of distance but also the cost of collaboration has become insignificant and moreover people are not solely passive users of information. New internet complications make two-way communication easier to manage which has lead to changes in how people use and perceive the internet, a phenomena named “Web 2.0” (Constantinides & Fountain, 2008). Malone (2009) acknowledged that the internet has enabled this new form of organizing work which has been described with a variety of terms -radical decentralization, crowdsourcing, wisdom of crowds, peer production, and wikinomics. While these communication capabilities continue to improve, the applications of crowdsourcing seem myriad.
Kleemann et al. (2008) confirm that the tasks that lend themselves to crowdsourcing are numerous and include product design, advertising, quality monitoring, and the solution of specific technical problems. These authors have furthermore identified and classified the types of crowdsourcing that are used by organizations as shown in table 1.
10
Crowdsourcing type Illustration of application
1) Participation in product development and configuration
Dell’s “idea storm” - call for comments and suggestions regarding the company's entire product palette.
2) Product design Fiat - Call announced by the auto manufacturer Fiat for its new Fiat 500 resulted in ten million clicks, 170,000 designs from (potential) consumers, and 20,000 specific comments on things like particular exhaust pipe forms.
3) Competitive bids on specifically defined tasks or problems (crowdsourcing contests)
Innocentive.com –Over 80,000 Independent scientists solve Fortune 500 companies R&D challenges.
4) Permanent open calls CNN - use of "amateur reporters," who submit photos or short articles for publication or broadcast
5) Community reporting Trendwatching.com individuals are asked to notify the company regarding any observable changes in market supply or consumer demand. Information is commercially used for market reports.
6)Product rating by consumers and consumer profiling
Amazon.com - Customers submit unpaid reviews of products it sells.
7) Customer-to-customer support Nike - users can upload their running times via their iPods and then use this data to engage in various competitions with other users.
Table 1. Crowdsourcing types and examples, based on Kleemann et al. (2008).
1.2 Crowdsourcing contests
As highlighted by Kleemann et al. (2008), crowdsourcing is a rather broad concept which can be categorized by their application forms. Since the concept itself is only defined in 2006 and the study area is within a multidisciplinary field, empirically-based analytical studies of crowdsourcing websites are lacking (Yang et al., 2008).
This research will explore crowdsourcing platform design of the most popular form of crowdsourcing: a crowdsourcing contest (Archak & Sundararajan, 2009). The typical process of crowdsourcing (contests) is described by Brabham (2008a) as follows:
1. The company posts a problem/ task online;
2. A vast number of individuals offer solutions to the problem/ task (enabled by the internet); 3. The winning ideas are awarded some form of a bounty (often prize money);
4. The company mass produces the idea for its own gain.
The type of tasks that can be outsourced can be categorized into routine, complex and creative tasks (Schenk & Guittard, 2009). To avoid generalities, this paper will only focus on the creative tasks that are crowdsourced by means of a crowdsourcing contest.
1.3 Crowdsourcing in the design industry
The potential of the crowdsourcing business model to reshape an industry is demonstrated in the graphic design industry by crowdsourcing contest websites which offer a platform for low-end design tasks of clients. These low-end design services include the graphic design of logos, websites, stationeries, illustrations and other graphic design.
11 The crowdsourcing contest platforms are mainly intermediaries which act as markets for clients to interact with the crowds. These marketplaces for (design) ideas (Tapscott & Williams, 2006) have their own specialized crowd in the form of an online community. The community typically consists of so-called "low-end designers" which are a mix of amateurs, recent grads and the like (Howe, 2009).
The intermediary facilitates the crowdsourcing process by providing a platform where clients and the crowd come together as shown in figure 1. These crowdsourcing platforms can be seen as hybrid models between open innovation and open source (Trompette et al., 2008).
Figure 1: Crowdsourcing intermediary as a facilitator
The concept is rather straight forward: clients post a briefing on the crowdsourcing contest platform of what they want to have designed, the community submits designs which fit this briefing and a winner is picked. The designer of the winning design receives a fee (often in the form of prize money), and the client receives the design and the IP-rights.
In recent years many crowdsourcing contest websites have emerged which are dispersed all over the world. An overview of several of these crowdsourcing contest platforms is given in appendix A which also includes community size.
The influence of this business model becomes clear when looked at the turnover of the largest crowdsourcing site, 99designs, which has a monthly payout to designers of over more than half a million dollar. Since these platforms can offer designs at a much lower rate than with a professional design agency the increase in crowdsourcing sites has lead to much turmoil. The proposed negative side effects on the design industry with topics like the quality/ originality of designs have been openly debated (Carpenter, 2009; Dunn, 2009; Cass, 2009). The traditional graphic design industry sees the disruptive nature of the new business model as an obvious threat which is underlined by a statement by the trade group AIGA (representing around 22,000 designer members): "AIGA strongly discourages the practice of requesting that design work will be produced and submitted on a speculative basis in order to be considered for acceptance on a project."
Submitting design work on a speculative basis (spec work) implies that most of the designers don’t get (financially) compensated for their work and this is the essence of the crowdsourcing contest model. In literature the ethics of this type of crowdsourcing has therefore been questioned due to its taking advantage of the creativity of the user community for commercial gains (Bruns, 2007).
Clients (In search for graphic design) Crowdsourcing intermediary (Platform) Community (Designers)
12
Chapter 2: Research Design
As proposed in the introduction, the aim of this research is to gain insight into the crowdsourcing contest mechanism of crowdsourcing contest platforms and their main challenges. Current studies mostly focus on the rise of the concept and possible applications for future business but not on the functioning and working of the mechanism of the concept itself. This explorative study therefore initiates the road towards a further insight in this new concept called crowdsourcing contests. Scientific contribution
This research aims to encapsulate all the existing knowledge on crowdsourcing contests and adjoining theories on the crowdsourcing contest topic. Current studies use different terminology/ definitions, investigate isolated research topics and are scattered across a diversity of media. Like within most topics in the field of management science, existing knowledge is insufficiently used by practitioners (Van Aken, 2004). Therefore, an extensive literature review is part of the research to ensure that this study builds on the (fragmented) existing knowledge on crowdsourcing contests. This study contributes to the scientific research on crowdsourcing contests by offering a more comprehensive view on crowdsourcing contests and absorbs the current available knowledge. This approach ensures that the propositions that are formed after confronting the current available crowdsourcing contest theory with practice, can serve as a clear starting point for future research.
2.1 Research Strategy
The research strategy used for this research is based on grounded theory (Strauss & Corbin, 1998) with the goal to formulate propositions based on conceptual ideas. Verschuren & Doorewaard (2004) state that this research strategy is characterized by the different matters that are observed in practice and are compared with theoretical assumptions/ principles.
With the help of crowdsourcing literature, a (preliminary) theoretical model was constructed which was used as a guide to highlight and identify important aspects/ elements of the functioning of crowdsourcing platforms. Because the body of literature on crowdsourcing (contests) is relatively small, literature on relevant topics that are related to the functioning of crowdsourcing platform was used as well.
The different elements from the theoretical model are then researched in a case study. A Latin-American crowdsourcing website was used as a research object to confront already known theory with practice.
2.2 Research model
Figure 2 shows the research model which visualizes the research process. The process starts with a casual explorative phase in which the relatively unknown field of crowdsourcing is explored. This creates insight in the research object and important themes/ topics are identified that help focusing the literature review. This explorative search consists of reading articles, blogs etc. on crowdsourcing and clearly serves as a starting point.
13 Figure 2: Research model
The second phase is a literature search in where (relevant) crowdsourcing literature and crowdsourcing related literature are described (chapter 3). This theory chapter concludes with a theoretical model that is used to guide the case study research, in line with the research strategy of grounded theory.
The way the elements of the theoretical model manifest in practice are described in the case study (chapter 4). Findings from both theory and practice are then confronted with each other and the differences and similarities are discussed in chapter 5. Subsequently, propositions are derived in the conclusion.
2.3 Methodology
As described in the research model, the heart of the research consists of two parts: a literature review and a case study.
2.3.1 Literature review
The books of Howe (2008) and Surowiecki (2005) served as a starting point for the literature review. These two authors have pioneered the field of crowdsourcing and are the most referred to authors within this field. Main source for the literature review are scientific articles from several journals found via databases like Business Source Premier and Science Direct. Besides the scientific articles, information is obtained from conference proceedings, reports and public opinion journals (e.g. The Economist).
The internet is a vast resource of information when it comes to crowdsourcing and many articles are found on forums, blogs etc. However, it is difficult to rely on the information retrieved from the internet. Information from the internet is therefore solely used for examples or supplemented with non-web sources to increase validity.
2.3.2 Case study
The decision to include a single case study in this explorative research is because of the reason that it provides the (rare) ability to access the needed information. Case study research is suitable for explanatory research and is a useful approach for theory development (Blumberg et al., 2005).
The case selection has been based on pragmatic grounds since the number of cases that are available are limited (Van Aken, et al. 2009). The crowdsourcing platform chosen for the case study is Guerra Creativa, a Latin American crowdsourcing contest platform which is described in 2.4. Since the owner
Crowdsourcing literature Theoretical model Case study Guerra Creativa Propositions (Conclusion) Crowdsourcing related literature Explorative phase
14 of the organization that has launched the website/ platform has a Dutch nationality, it increased the chance of receiving authorization for this unique study of the platform up close.
The case study (inquiry) method, as described and defined by Yin (1994), will be used which:
‘- copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result,
- benefits from the prior development of theoretical propositions to guide data collection and analysis.’ See also the research model, the literature review (phase 2) guides the data collection and
analysis of the case study (phase 3).
The sources of evidence which are used for this case study consist of sources that are identified by studies of Stake (1995) and Yin (1994):
• Survey research • Documents • Archival records • Direct observation • Participant-observation
With using these different types of methods as well as different sources in the same study, the validity was increased (Brewer & Hunter, 1989; Stake, 1995). A case study database was created to ensure evidence is stored and can be linked to the research questions.
Survey research
Self-administered surveys make up a large part of the research in this case study. To collect the required data from all three stakeholders, survey research has been evaluated as the best method as will be elaborated on shortly. The theoretical model helped to define a clear outline of what information was needed from each stakeholder/ informant.
1) Platform users
Surveys were used to discover the motives of the community members of Guerra Creativa and how they rate the website. An online survey (trilingual, see appendix B) was created because it’s an economical and efficient method to collect the needed information. Adjacent to this, this online survey has the ability to reach the large number of the geographical dispersed users of Guerra Creativa. The online survey was posted at the platform as a regular (design) contest which consists of a description of the research and a link to the online survey. Together with a monetary incentive of $150, which is raffled among the participants, the participation motivation was increased and thereby attracted a lot of platform users to the survey. As a follow-up procedure, a (trilingual) newsletter was sent to all the platform users who have submitted a design at least once (active users) with information and an invitation to the online survey. The number of newsletters that have been sent is distributed as follows: Portuguese 223, Spanish 2158 and English 572. The online survey was also promoted through social media amongst the platform users.
Although the total community consists of 12.000+ registered users, the amount of active users is much lower, namely 2953. This group can be seen as the core group and these users are competing in the contests. In total, 358 community members completely filled in the questionnaire. This resulted in a response rate of 12% which is a perfect result for this online survey. The response rate is in practice probably higher due to false and inactive accounts.
15 The respondents could select their background: professional, student or hobbyist. Because the variable ‘student’ was not specified in the questionnaire, a follow-up was sent to ensure these were design students. The data was corrected when needed but it turned out that the respondents did understand the question and had a design background (e.g. studying graphic design, multimedia design, Design & Communication etc).
Background number of responses
Professional 198
Student 135
Hobbyist 25
Total 358
Figure 3: Distribution of respondents by background.
2) Intermediary organization
A self-administered survey was used as well to collect data from some employees of the intermediary organization. These participants were geographical dispersed but are well experienced with this form of communication since the organization is active in the outsourcing industry. Employees who work in this industry need to use electronic means of communication on a daily basis. The survey consisted of multiple open-ended questions which touches elements of the theoretical model. When answers were not considered complete or misunderstood, a process of additional questioning followed in a more face-to-face form (via Skype telephone calls) to collect the data that was needed.
The persons that are selected to fill in the questionnaire are three members of the organization that are closely involved to the Guerra Creativa project, namely:
Director of Connaxis who is the director of the intermediary organization and founded Guerra Creativa. He looks after the strategy of the platform and initiates projects to support this strategy.
Project Manager of Guerra Creativa who guides projects from start-up to completion and has strong links to the programming and design department. Other tasks include the guiding of (three) support agents, the writing of newsletters and creation of other content for the website.
Ex-Project Manager of Guerra Creativa who was involved during the launch of the platform and the creation of several applications on the website to meet the encountered demands. This person is an important source of knowledge because there was a lot of uncertainty during the start-up phase of the platform which had to be coped with. Goals and applications had to be altered in order to make the platform function.
A report on the questions and answers of this survey can be found in appendix C.
3) Clients
The clients form a difficult group to gain information from since they are relatively less engaged to the platform than can be expected on forehand (based on observations in the explorative phase). Responsiveness on emails from Guerra Creativa to clients was experienced to be relatively low. However, an email with an invitation to an online survey was send to all the contest holders (exactly 200) whose email address was correct in the database. After a follow-up email, a total of 31 contest holders have completed the questionnaire (response rate of 16%). Because the respondents were
16 not compensated for their effort and to make the survey not to (time) demanding, a short questionnaire was created to mainly unveil the motivations of the clients to participate (appendix D). Documents & Archival records
To ensure the validity of the research, documents and archival records were studied next to the surveys. The sources of information include screenshots, emails, (support) tickets and organizational records. All the information that is used in this research is stored in a case study database. False leads are minimized through a process of double checking and confronting the findings from other information sources.
Direct & Participant-observation
With the use of observations, a greater amount of data was collected. During a time period of six months of observation and casual data collection, activities were performed in a variety of roles. Both the role as a member of the organization (Connaxis S.A.) as well as the role of community member of Guerra Creativa was explored to retrieve information.
2.3 Explorative Data Analysis and Testing
The quantative research which was used to retrieve information from the community resulted in a rich collection of data. In this explorative study there is a need to discover patterns in data and search of hypotheses to test. Based on this need, Explorative Data Analysis (EDA) was used as an approach to analyze the data instead of a more traditional approach like Confirmatory Data Analysis. EDA is especially found to be useful when there is little explicit theoretical background to guide prediction and the first stages of model building are desired (Behrens, 1997). The EDA therefore forms an important tool in the process of theory development like with this current research on crowdsourcing where little (empirical) theory is available.
The first step in the EDA was a global analysis of the available data that was retrieved from the surveys. When using EDA, there is an emphasis on graphic representations of data to discover patterns in data. Several graphical representations of the data were therefore rendered in SPSS:
Frequency distribution histograms
Stem & leaf plots
Boxplots
Normal probability plot (QQ plot)
After an initial graphical analysis was performed, the descriptive statistics (mean, median, range, variance, standard deviation, etc) were explored. Additionally, the data from both surveys (contest holders and creatives) was tested on normality.
The data from the ‘creatives survey’ was tested on normality by using the Kolmogorov-Smirnov test (also categorized by background, see appendix E). This test was chosen over the Shapiro-Wilk test because of the large sample size (n=358). On contrary, the contest holder survey responses were tested on normality by using the Shapiro-Wilk test (see appendix F) because of the small sample size (n=31).
17 Important finding here was that none of the variables from both surveys (contest holders and creatives) were normally distributed (p<0.01) and the data was therefore characterized as distribution free. Non-parametric methods therefore had to be used to analyze the data. Unfortunately, these methods are less flexible in practice and less powerful than parametric tests but do have led to interesting and fair results.
The Kruskal–Wallis one-way analysis of variance was used to identify possible inequality of population medians among groups. Significant results that came from this analysis, together with the graphical analysis, were further examined by testing two of the three groups separately (hobbyist, professional or student) by using the Mann-Whitney U test. Another non-parametric test, the binomial test, was used to test variables from one sample to analyze the ratings of respondents towards platform characteristics.
2.4 Introducing Guerra Creativa
Research object of this study into the functioning of (intermediary) crowdsourcing contest platforms is the website Guerra Creativa. This platform represents a typical crowdsourcing initiative in the graphic design industry and is based in Buenos Aires, Argentina. Launched in 2009, the website has a fast growing community of mostly Latin American “creatives” who submit designs in competitions of clients facilitated by the website. Currently, the community consists of more than 10.000 designers who registered at the website and there has been a payout of nearly 60.000 USD of prize money through 400 design contests. The process of the design contests is consistent with other crowdsourcing contest platforms as shown in figure 4.
Figure 4: Banner at the Guerra Creativa website describing the contest process.
The organization which created the platform Guerra Creativa is Connaxis S.A., a multimedia company serving the market by more traditional means. Guerra Creativa is a project in which Connaxis invests resources from all the different departments, including the marketing, design and programming department. One project manager is working full time on the project to improve the platform and to delegate tasks to the operational departments (design & programming). Next to the project manager, three (trilingual) support agents take care of questions of clients and answer emails/ messages from the community 24 hours a day.
18
Chapter 3: Theory
The explorative phase in the field of crowdsourcing resulted in a clear starting point for the literature review. The main finding was that if crowdsourcing is to be successful, all participants should clearly experience the benefits of the platform. Many articles and blogs on the internet express the importance of this requirement from the viewpoint of the community, the client as the intermediary organization.
This finding is in line with a study of Reichwald and Piller (2006) who stress that crowdsourcing is only possible when the requisite interaction brings benefits to all participants. This points out the importance of the enclosure of all the stakeholder motives and requirements toward the crowdsourcing contest platform. Figure 5 illustrates a typical relation between the direct stakeholders of the platform including an intermediary organization (like Connaxis) which facilitates resources to the platform.
Figure 5: Crowdsourcing contest platform and relationship with stakeholders
Based on this model, the books of Howe (2008) & Surowiecki (2005) and the perception that a platform should benefit all stakeholders, a list of associated concepts and theories was formed. These theories and concepts highlight and describe important implications for the functioning of the crowdsourcing mechanism (3.1) and furthermore the requirements of the stakeholders (3.2, 3.3, 3.4).
This comprehensive view on the crowdsourcing platform ensures the viability of the theoretical model (3.5) that will serve as a guide for confronting the findings in theory with the practices from the case study (chapter 4).
3.1 Crowdsourcing contest mechanism
The crowdsourcing contest mechanism can be considered the motor of the crowdsourcing contest platform. It is the way how stakeholders interact and the process of how crowdsourcing works that leads to the eventual value creation for all stakeholders. Although crowdsourcing contest theory is
Intermediary organization (founding organization)
crowdsourcing contest platform
Clients
(design seeking companies) Community
19 scarce, theories on collective intelligence, facilitating creativity and the organization of (idea) contests are closely related. Design implications that are derived from these theories should give insight in the functioning of the crowdsourcing contest platform.
This paragraph describes theories on collective intelligence, facilitating creativity, organization of (idea) contests and their links with the crowdsourcing contest mechanism which will be used for the theoretical model.
3.1.1 Collective Intelligence
The term crowdsourcing in literature is directly linked with the concept of “wisdom of the crowd theory”, popularized by Surowiecki (2004). This theory proposes that decisions of a diverse and large enough group of people will, over time, be intellectually superior to those of isolated expert individuals. This idea of collective intelligence (Malone, 2010) is not new, its development, application and evaluation have been described by various studies (Lykourentzou et al. 2010). Levy (1997) defined collective intelligence as a “form of universally distributed intelligence, constantly enhanced, coordinated in real time, and resulting in the effective mobilization of skills”.
Surowiecki (2004, 2005) sets clear conditions to assure that the aggregate contributions of a crowd are of high quality:
1) Diversity: The crowd should be diverse by backgrounds and perspectives enable the crowd to conceptualize problems in novel ways (West & Dellana, 2009).
2) Independence: Members of the group need to have relative freedom from each others’ influence. 3) Decentralization: Decentralized individuals are able to make better decisions based on their own local and specific knowledge rather than on an omniscient or far reaching planner.
4) Crowd size: Large number of contributors (Arazy et al., 2006).
These conditions need to be met by any crowdsourcing initiative in order to reach the full potential. 3.1.2 Fostering (collective) creativity
The positive effects of crowd diversity, independence, decentralization and size are in line with the work of Amabile (1983, 1995) on individual creativity in a group and organizational context. Acadamic literature recognizes creativity as an important tool and starting point for innovation (Amabile, 1988; Woodman, Sawyer, & Griffin, 1993). Kozinets, Hemetsberger & Schau (2008) point out that the context in where an online community solves client’s posted problems is one manifested by “cultural, yet optionally anonymous, noncommittal, experimental, playful, educational,
collaborative, visual, filled with richness and diversity of content and social and cultural depth”.
Therefore an online community context overcomes most of these environmental constraints for creativity that are in attendance with a traditional business model. The same requirements identified for tapping from collective intelligence consequently seem to be applicable for fostering creativity. In this particular case study of a platform which hosts contests for graphic design, sustaining creativity is an essential element as well since creativity is an important skill in the design industry (Bureau of Labor Statistics, 2010).
Leimeister et al. (2009) have done research on how activation-enabling functionalities can be systematically designed and implemented in an IT-based ideas competition for enterprise resource planning software. This paper contributes greatly since the field of research on online ideas
20 competitions is new and no theoretical and empirical insights exist, as they point out. Although their explorative study is largely focused on requirements of users, it provides useful insights and models which can be incorporated in this study as they appeared to be of value. In their study they make use of one of the most established models for supporting creativity and developing user interfaces/ software tools that support creativity (Shneiderman, 2002). The Genex framework identifies four activities during the process of creativity for evolutionary ideas:
Collect: learn from previous works stored in libraries, the Web, etc.
Relate: consulting with peers and mentors.
Create: explore, compose, evaluate possible solutions
Donate: disseminating the results elaborated during the creativity process
These activities can be assisted by the crowdsourcing contest platform by technical and organizational components.
3.1.3 Organizing (idea) contests
Besides the specific form of crowdsourcing contests, other theory than merely collective intelligence can contribute constructive knowledge about the functioning of the platform. Morgan & Wang (2010) presented an interesting and helpful framework for organizing tournaments for ideas in the internet era and create awareness of the compatibility between the two concepts of crowdsourcing contests and idea tournaments. Although their propositions are not tested, they form useful input for the theoretical model since the propositions are based on empirical research.
Based on their framework and the specific characteristics of a crowdsourcing contests, the following design implications can be wished-for in a crowdsourcing contest platform.
Use a tournament platform
The authors argue that the use of a tournament platform is only useful when the participants performances are independent. This facilitates competition among the participants which benefits the client who is only interested in the value of the best submission. The contest should therefore be open to all corners to maximize total bandwidth devoted to the problem (Konrad, 2009). The crowdsourcing contest must encourage open participation and thus should not restrict entry in any way.
Offer flattened incentive structure
The incentive structure depends largely on the type of innovation and the variance in ability of the contestants. Since the crowdsourcing contest studied consists of low-end design tasks a flattened incentive structure is proposed as fitting to encourage all participants. This incentive structure invites participation from contestants who might not see their submission as the very best. To preserve diversity in the crowd and the quantity of submissions, their participation is needed.
The proposition that the incentive structure is an important element of the crowdsourcing contest architecture is underlined by Archak (2010) who states that designing efficient crowdsourcing mechanisms is not possible without deep understanding of incentives and strategic choices of all participants. Although the client is only interested in one best solution, the optimal number of prizes can be strictly greater than the number of solutions desired (Archak & Sundarajan; 2009b). An example of this contest architecture can be seen at Topcoder.com. This is a crowdsourcing contest
21 platform for software components, where the client awards two prizes for each software contest, although it only uses the winning submission. In their articles Archak & Sundarajan (2009a) propose a rule of thumb in where each new prize should have approximately twice higher marginal utility than the prize immediately above it.
Organize multiple rounds
To motivate the contestants with different level of abilities, Morgan & Wang (2010) bring in the idea of multiple rounds. By doing this they want to prevent that the probability of talented contestants winning the tournament is diluted by contestants that are unlikely to contribute much to the contest outcome. It is also suggested that weaker contestants are discouraged by the participation of strong contestants. Multiple rounds, with larger prizes at each round, should solve these problems because it eliminates contestants in earlier rounds. However, the jeopardy of implementing multiple rounds is that it can form a barrier for contestants to participate.
Leverage social motivators
Next to monetary incentives, social acceptance is a dominant interest which explains the human behavior (Harsanyi, 1969). The role of social acceptance as a motive for participation is elaborated on in the section ‘community requirements’ but it concerns the crowdsourcing contest mechanism as well.
Within online communities, the status and recognition that users receive represent very important motivators for contributing on the platform (Lampel & Bhalla, 2007). Users will even give up monetary compensation for obtaining this social acceptance (Huberman et al., 2004). This is an important finding since most of the work of the participants is done “on-spec” which is heavily criticized as pointed out earlier on. Social appreciation is not only received from peers but obtaining appreciation by the organizers of the crowdsourcing contest is an important motive factor (Leimeister, 2009).
The crowdsourcing contest platform should therefore facilitate the social acceptance by peers and organizers of the contest. Part of this social acceptance is formed by attention, which has shown to have a strong positive dependence with the productivity exhibited in crowdsourcing (Wu et al, 2009; Huberman et al., 2009).
3.2 Stakeholder requirements – Community
Compared to the other stakeholders much research has been done after online communities, its motive, composition and behavior. A crowdsourcing community typically involves experts, amateurs or any mix of those (Archak & Sundararajan, 2009b). When mapping the crowd at iStockphoto, a royalty-free stock photography agency using the crowdsourcing principle, it has been found that the emblematic user fit the “Elite” web user profile of a white, middle and upperclass, higher educated web user (Brabham, 2008). This could be due to the ability this group has above others to access to the technology to participate (Ross et al. 2010).
An interesting view in the composition of the crowd could be the influence of emerging markets. Since crowdsourcing has an outsourcing element, a link to these emerging markets would only be logical. Mike Samson, one of the founders of the design crowdsourcing contest platform
22 Crowdspring, has indicated in May 2009 that users from these countries make up an important part of the community by declaring the following:
"IMF has a list of Emerging/Developing countries and we had visits from 61 of these. Collectively they accounted for over 19% of our traffic and over 14% of our uniques!"
Whether the motive of this group has focus on financial gain or other motivation factors could become evident in the results since Guerra Creativa positioned as a creative platform for Latin America.
The motives to participate in online communities have been studied by several authors over time. It seems to be a remarkable and interesting phenomenon that users put time and effort by creating and submitting content on these platforms without the of financial compensation. Motives that trigger participation in crowdsourcing contests can be categorized in extrinsic motive and intrinsic motive. While extrinsic motive works through indirect satisfaction of needs, most importantly through monetary compensation, intrinsic motive works through immediate need satisfaction (Deci & Ryan, 2000). The motives to participate in online communities which are applicable to crowdsourcing contests are summarized in table 2. This table is adapted from Antikainen and Väätäjä (2008) and complemented with other motives identified in open source research.
Motives to participate in online communities Authors
Care for community and attachment to the group Kollock (1999)
Career related benefits Robles et al. (2001)
Enjoyment and fun von Hippel & von Krogh (2003); Nov (2007); Torvalds & Diamond (2001)
Firm recognition Jeppesen & Frederiksen (2006)
Friendships, relationships and social support Hagel & Armstrong (1997); Rheingold (1993); Ridings & Gefen (2004)
Interesting objectives and intellectual
Stimulations Ridings & Gefen (2004); Wasko & Faraj (2000) Knowledge exchange, personal learning and
social capital
Antikainen (2007), Gruen et al. (2005), von Hippel & von Krogh (2003); Wasko & Faraj (2000); Wiertz & Ruyter (2007); Gosh et al. 2002
Peer recognition Lerner & Tirole (2002); Hargadon & Bechky (2006)
Recreation Ridings & Gefen (2004)
Reputation and enhancement of (professional) Status
Bagozzi & Dholakia (2002); Hargadon & Bechky (2006); Lakhani & Wolf (2005); Lerner & Tirole (2002); Wasko and Faraj (2005); Hars & Ou (2002) Sense of efficacy, influencing Bandura (1995), Constant et al. (1994); Kollock (1999)
Table 2: Modified from Antikainen and Väätäjä (2008)
Although a crowdsourcing contest platform hosts a community which have all the same motives to participate as with the regular online communities, the contest element provides additional motives to participate. Leimeister et al. (2009) include motives to participate in crowdsourcing contest derived from the field of sport motive research which seems suitable because of the competitive factor.
The extrinsic motives to compete are divided in two classes:
Direct compensation, in the form of trophies and prizes for the (three) best participants.
Social motives, including the expected reactions of significant others, friends, or the audience.
23 These social motives identified in sport competitions and their importance is in line with the theory on social acceptance described under crowdsourcing contest mechanism. Positive reactions from the peers, organizers and clients seem to be important for motivating participants.
From possessing motives to actual participation
The motives of users to participate in a crowdsourcing contest are identified but motives don’t directly lead to actual participation. To understand the workings of how to trigger these users to participate, Leimeister et al. (2009) used the MIAB model of Briggs (2006).
Figure 6: The research approach derived MIAB Model
This model helps to visualize the process from having motives to actual participation. The relevant motives that have been identified are influenced by incentives to activate the users. These incentives can be derived from these motives and should be supported by incentive supporting components in the crowdsourcing platform design.
3.3 Stakeholder requirements – Clients
In academic literature on crowdsourcing contests and related concepts there has been a strong focus on the community and the crowdsourcing mechanism for determining the optimal architecture of the crowdsourcing contest platform. Although these two elements are essential, the requirements of the clients should be included as well as major stakeholder. The clients financially support the platform and their needs should be fulfilled and their requirements satisfied in order for the crowdsourcing contest platform to exist as a sustainable business models.
In line with outsourcing where the concept of crowdsourcing is developed from, cost reduction seems to be an important motive (Kleemann et al., 2008). Crowdsourcing contest platforms for graphic design (see appendix A) uniformly present the key benefits for the clients to be:
Low cost of design (supported by Schenk 2009);
Large quantity of designs to choose from (supported by Schenk 2009);
High quality through the creative potential of their community (supported by Schenk 2009);
Low risk (supported by Schenk 2009);
Fast and easy process with direct communication with designers.
Whether these (specific) benefits are important motives for clients to choose crowdsourcing sites is not confirmed in literature. Pisano & Veranti (2008) contributed to the subject of collaboration
Motives
Incentive
Activation Participation in the Crowdsourcing Contest Incentive-
24 modes by taking a clients perspective and elaborate on structure and organizing principles. Since more companies are investing in these web collaboration tools (Bughin & Manyika, 2007) their study which proposes a framework with advantages and challenges is helpful. The crowdsourcing contest can be seen as an open hierarchical collaboration mode as categorized in their framework. Most important challenge for the client, due to the openness, is the screening of all the ideas. This can be time-consuming and expensive so both the task as the platform mechanisms should fit this mode. Another remark is that the best idea generators prefer closed networks where their ideas are more likely to be implemented. The non-participation of the group of best idea generators (designers at Guerra Creativa) could be contradictory with the idea that crowdsourcing leads to the best solution.
Because the client posts the contest and picks a winner, the client should have the capabilities and knowledge needed to define the problem and evaluate proposed solutions. With a complicated task or a task which is outside the area of knowledge of the client, this is more difficult.
3.4 Stakeholder requirements – Intermediary organization
The intermediary organization is the organization that exploits the platform and has the responsibility of managing the platform for clients and the community. Under the design crowdsourcing contest platforms, the intermediary organization is often a subsidiary of another organization, e.g.:
design studio (12designer.com, designonclick.com);
web applications developers (Guerra-creativa.com);
media companies (99designs.com).
From their experience in the industry they launched these platforms but there are also examples of sole design crowdsourcing contest start-ups (crowdspring.com, logotournament.com). The mission of most of these crowdsourcing contest platforms is to deliver clients good value for their money and offer creatives the change to build their portfolio, enhance their skills, build-up relations with customers and additionally make some money.
The intermediary organization is wedged between the design community and the clients where the platform has a high level of control (Banabeau & Lakhani, 2009). The platform needs to achieve a fine balance between value creation for clients and community values. Although the participants have strong competitive relationships among one another, a strong feeling of community seems necessary to enhance engagement and foster the social motives participants have. The conflicts that can arise between the setting of competitive markets and collaborative communities can lead to considerable risks. The intermediary company should therefore deploy the crowdsourcing contest platform with caution and attention to governing mechanism (Banabeau & Lakhani, 2009).
Community management
The importance of finding the right balance to optimize the result for all stakeholders requires a closer look to community management. Online communities typically have a large percentage of between 45-90 percent of non-contributing users (Nonnecke & Preece, 2000). This is supported by research after online community of open source developers (Mockus et al., 2002; Xu et al., 2005) were they identified a (small) group of core members that are responsible for guiding and coordinating the development of the community.
25 A valuable contribution regarding crowdsourcing contest platforms and how the intermediary company can actively undertake action is offered by the research of Yang et al. (2008a, 2008b) who have studied the Chinese Witkey website Taskcn. These "Witkey" websites have ‘gathered thousands of tasks, hundreds of thousands of participants, and millions of users’ and are clear examples of crowdsourcing contest platforms. Around half of all the tasks performed at the studied platform are in the design category and are performed in the same way as with Guerra Creativa. In line with preceded literature, the participation rate is low since only 11% of all registered users have participated at least once. Since only 3.2% of these users who participated won a contest, this exposed excessive entry has created market inefficiencies. These market inefficiencies in order to tap from the crowd are typically related to “wisdom of the crowd theory”.
Interesting findings concerning the community characteristics are given by the authors:
Most users become inactive after only a few submissions;
Users make strategic choices: they select tasks where they are competing against fewer opponents to increase their chances of winning;
There is a very small core of successful users: they manage to win multiple tasks and additionally increase their win-to-submission ratio over time.
An important design implication for the crowdsourcing contest platform that is proposed to optimize the result in the contests is to incentive the core group of winners in order to maintain their presence on the platform. The intermediary organization could guide users to less popular contests or ones that suit their expertise. The active core of users provides 80% of the solutions and therefore it is important to maintain attracting large numbers of users who could become part of this core group.
3.5 Theoretical model
The literature review has identified several design implications for the crowdsourcing contest platform which could optimize the functioning of the platform for all stakeholders. Figure 7 visualizes these requirements/ implications in the form of a theoretical model and in appendix G an overview is given which links the proposed implications with the corresponding authors.
The theoretical model consists of four blocks, the crowdsourcing platform and the three direct stakeholders. In the ‘Crowdsourcing contest platform’ block implications are listed that optimize the functioning of the platform. These concern the organization of the contests and the crowd composition. Both the community and clients have requirements that need to be fulfilled by the crowdsourcing platform and these lead to the design implications as shown in the block. Finally, the intermediary organization has clear tasks that need to be executed in order to run a successful crowdsourcing contest platform.
26 Figure 7: Theoretical model with implications for the crowdsourcing contest platform
In the next chapter these implications for the crowdsourcing contest platform which are derived from theory will be studied in practice. The case study will show whether these implications can be observed at a crowdsourcing contest platform in a real life environment. It will be interesting to see where there are differences between theory and practice and if these can be explained (chapter 5).
Crowdsourcing contest platform Functioning – Organization of contests:
- Open tournament
- Offer flattened incentive structure - Organize multiple rounds
- Leverage social motivators Functioning - Crowd composition: - High internal diversity
- Relative freedom from each others’ influence
- Crowd should be decentralized
- Large number of contributors (crowd size)
Intermediary Organization
Functioning – Tasks that need to be executed: - Balance between value creation for clients and community values
- Incentive of the core group of winners - Guide users to less popular contests or ones that suite their expertise
- maintain attracting large numbers of users
Community
Requirements – Platform characteristics/ benefits : - Incentive-supporting components for motives (for all motives to participate, see table 2) - Ability to collect, relate, create & donate
Clients
Requirements – Service characteristics/ benefits: - Low cost of design
- Large quantity of designs to choose from - High quality
- Low risk
- Fast and easy process
- Direct communication with designers - Knowledge available to define the problem and evaluate proposed solutions
27
Chapter 4: Case study – Guerra Creativa
Guerra Creativa is a typical crowdsourcing contest platform and therefore forms a perfect research object for this case study. Since the launch of the website, the platform has a fast growing community and experienced rapidly increasing website activity. In the course of time the platform management has developed and implemented several functionalities to improve the website’s performance for all stakeholders.
This case study makes use of information gathered from several sources (as described in 2.3 Methology) in the period from March 2010 – November 2010. The theoretical model has identified important aspects that will steer the information search in this case study. All elements and findings from this theoretical model are discussed in this chapter and the structure of this case study chapter is identical to that of the theory chapter.
The structure is as follows: first the functioning of the crowdsourcing contest mechanism is explored, followed by the requirements of the community and clients. In the last part of this chapter, the role of the intermediary organization is examined.
4.1 Crowdsourcing contest mechanism
The literature study has resulted in clear insights on how crowdsourcing contests can be organized to benefit both the client as the community as well as insights on the platform’s crowd composition. In this section it is described how these design applications are applied in a real life environment, namely at the Guerra Creativa website.
4.1.1 Organization of contests
The organization of a contest is a standardized process which consists of three basic phases as shown in figure 8.
Figure 8: Contest process flow chart
The process starts with the contest creation phase in where the client, also called contest holder in the GC environment, goes through the following steps:
1. The contest holder registers at the website as a user;
2. defines the contest setup (amount of prize money, privacy level & pay-out options etc.); 3. completes the payment transaction (prize money + handling & listing fee);
4. writes the creative briefing for the designers (called creatives in the GC environment). Contest Holder picks a winner Contest Holder rates designs Community posts designs Contest Holder launches contest
