Master Thesis
Master of Science Business Administration – Strategic Innovation Management
The changing domain of Open Innovation:
A Systematic Literature Review
__________________________________________________
A
bstract
Open innovation has become one of the most well-known strategies for organizing innovation. Accordingly, a growing body of literature is published over the past years. Prior research addresses that due to the rapid technological change, the literature on open innovation might also experience changes. We lack understanding of how the research field of open innovation has developed from 2010 onwards. Hence, this study provides a descriptive analysis of the literature within this period of time, in which 1,504 articles were identified. Following the descriptive analysis, we find the main subject within the open innovation literature. This main subject concerns the term crowdsourcing. Due to the importance of this subject, a systematic literature review is performed to get insight into how firms organize for crowdsourcing. Using the findings from the systematic literature review, we develop a five-step approach for firms organizing crowdsourcing. Building on the findings, we propose future research directions. For instance, we argue that future research regarding crowdsourcing needs to shift from theory-building to theory testing research.
Author: Roeleke Staal
Student number: S3662675
Supervisor: dr. K.R.E. Huizingh
Co-assessor: dr. J.D. van der Bij
Date of submission: July 13, 2020
2
1. Introduction
The term “open innovation” has, since its introduction in 2003, rapidly become one of the most well-known imperatives for organizing innovation (Bogers, Chesbrough, Heaton & Teece, 2019). Thus, few themes in innovation management received more attention during the last years than open innovation (West, Salter, Vanhaverbeke, & Chesbrough, 2014). Since the introduction in 2003, firms’ traditional innovation model has gradually shifted towards the adoption of open innovation practices (Cappa, Rosso & Hayes, 2019). Open innovation is described by Chesbrough (2006, p. 1) as “the use of purposive inflows and outflows of knowledge to accelerate internal innovation and expand the markets for external use of innovation, respectively.”
Since the introduction of open innovation, many authors have studied the term. Many of these studies conducted systematic literature reviews on the topic, to give an overview and synthesis of the literature. However, the vast majority of the articles available, consider the term in a specific field of interest. Examples of these fields can be within the context of SME’s (Kraus, Krailer, Dorfer & Jones, 2019) or open innovation performance (Greco, Grimaldi & Cricelli, 2015). In these specific fields of interest, researchers use different definitions, which makes building a coherent body of knowledge challenging (Di-Benedetto, 2010). Few authors have responded to this research gap by providing a systematic literature review on the research field of open innovation without addressing a specific context. One of these authors is Huizingh (2011). His study provides an overview of the state of art and future perspective on open innovation, by focusing on three main questions: what, when and how for open innovation. Looking at the high frequency of citations of Huizingh’s (2011) article, the overview has been considered providing new insights and therefore used repeatedly by other authors. Though, the study of Huizingh (2011) only covered a period of ten years, ranging from the years 2000 to 2010. Due to the fast pace of technological change and the increasing importance of subjects as sustainability, many papers have been published after 2010. These papers include special issues on New Product Development through open innovation in leading journals like the ‘Journal of Product Innovation Management’ (Barczak, 2012). This rate of technological change, implies that the research field of open innovation may have also changed compared to the period of 2000 to 2010. Therefore, our study acknowledges the gap by initiating the following research question: (1) How has the research field of open innovation developed from 2010 onwards?
3
Following the descriptive analysis in the first part of the study, we found ‘crowdsourcing’ to be the most frequently studied term within open innovation. Besides being the most frequently studied term, we have additional motives to perform a literature review on crowdsourcing in the second part of our study. Crowdsourcing is a broad term that is used be used in a wide variety of contexts. Merely a relatively small number of articles published on crowdsourcing, approach the term in the context of open innovation. In fact, the articles that do consider crowdsourcing in the context of open innovation often explain the term by a specific focus. An example of such a focus can be the crowd’s motivation or the differences between specific forms of crowdsourcing (Kleemann, Voss & Rieder, 2008; Kohler, 2015). As crowdsourcing is increasingly used by firms, it is important to understand the term from an open innovation perspective and how it can be applied by firms that want to crowdsource. For instance, what needed to perform crowdsourcing and when is it effective? The authors Devece, Palacios & Ribeiro-Navarrete (2019) plea for generalizability on factors that influence crowdsourcing effectiveness. Consequently, to use crowdsourcing as the way to go, a firm has to choose an appropriate form. The majority of the literature studies approach crowdsourcing by innovation contests (Acar, 2019; Brabham, 2008), but is this the only form of crowdsourcing firms can use? We address this gap in the crowdsourcing literature through the following research question: (2) How should crowdsourcing for innovation be organized by seekers? This research question is answered through a systematic literature review, in which 31 studies were included that solely addressed crowdsourcing from an open innovation perspective.
The contributions of this study are twofold. The descriptive analysis findings contribute to a better understanding of the open innovation research field and the developments within this field over the past decade. The findings of the systematic literature review on crowdsourcing contribute to a better understanding of the steps that seekers have to take into account when wanting to shift from a more closed innovation model to crowdsourcing.
4
2. Methodology
In this section, the research approach, design, setting, and data collection and analysis used in this research, is described. First, a descriptive analysis of open innovation literature is performed to be able to answer our first research question. Relevant literature regarding open innovation is reviewed to answer this research question. Prior studies are used as an inspiration for this study’s design, such as the studies of Meier (2011) and Tranfield, Denyer & Smart (2003). Both prior studies use a methodological approach that can be partially extended, applied to, and thereby used within this study. Tranfield et al. (2003) give a three phase-step approach for systematic review, that is applied to this study in the following steps:
1. At the first stage (i.e., planning), the research objective is defined as well as the source of the primary data.
2. In the second stage (i.e. execution), we follow three systematic steps: (1) inclusion and exclusion criteria and search terms, (2) assembly of a ‘consideration’ set and (3) grouping of the articles. These three steps apply to data collection.
3. In the third and last step, (i.e., results), we classify the results and synthesize the findings in order to give a comprehensive overview of the literature as well as the research field. 2.1 Planning
The systematic review process of this study is twofold: first we map the research field of open innovation from 2010 onwards. This is done by giving the number of articles on this topic published per year and the journals in which these articles were published. Hereafter, we give an analysis on the most frequently used keywords, to get an insight into the popular and emerging subjects within the research field of open innovation. The most frequently used keyword will be used for the second step of this study; namely performing a systematic literature review on this topic, to map the present literature on this specific topic and get insight in the state of art and future perspectives. So, the specific subject within the open innovation field that we review in the second step, is not known beforehand and thus follows from the results of the analysis in the first step.
5
years of 2010 to 2020, against 2,587 articles in Web of Science and 2,774 in Business Source Premier. Lastly, Scopus gives the ability to do a search query solely based on the indexed keyword, both Web of Science and Business Source premier do not offer this option.
2.2 Execution: initial selection criteria, key search terms & consideration set Following our first research question, we focused only on articles from 2010 onwards. We furthermore focused solely on articles. Also, the publication stage of the article had to be “Final”, since in Scopus, there is also a possibility to extract articles that are still under review. Lastly, all articles had to be written in English. Following these inclusion criteria, we selected a large set of articles using Scopus. We used the search term “open innovation”, to search only for articles that included the indexed keyword of “open innovation” given by Scopus. The decision to merely perform a search based on the indexed keywords was based on the avoidance of incorporating articles in this study that may not have open innovation as their primary focus. This search has resulted in 1,504 articles. The dataset was entered into Mendeley to detect possible duplicates. No duplicates were found. The total dataset comprised a number of 1,504 articles.
2.3 Results: Data analysis and synthesis
A dataset was extracted from Scopus in which the authors, article title, year of publication, journal name, and all the author-supplied keywords are given. We chose to merely extract, and thus study, the author-supplied keywords, since we have no insight into how the indexed keywords in Scopus are established.
During the analysis of the keyword data, different keywords were clustered. This applies, for example, to two keywords that have the same meaning, but of which both singular and plural form was used. Examples include “network” and “networks, but also abbreviations such as “living lab” and “living laboratory”. Abbreviations were also clustered, such as the keywords “Research and Development” and “R&D”, and “Mergers and Acquisitions” and “M&As”, as they mean the same. However, it is important to note that keywords were not always straight away clustered. If an article contained both the keywords “R&D” and “Research and Development”, these two keywords were clustered into the keyword “R&D” and counted as a frequency of one, instead of two. The final set of clustered keywords, can be found in Appendix A for transparency reasons.
6 These exclusion criteria are as follows:
Exclusion criteria Description/motivation
Company or industry descriptions
Keywords that, for example, described the size of a firm researched in a particular study, such as Small and Medium-Sized Enterprises (SME) or the industry the firm is in, were not incorporated in the final keyword set. Study methods Keywords regarding the study methods of a particular study, such as “Living
Labs” and “Case study” were not incorporated as keywords in the final set. General innovation terms Since this keyword analysis is performed with the eventual goal of
identifying the main (emerging) concepts in open innovation, keywords as “Innovation Management” and “Open Innovation” theirselves should not be incorporated as keywords in the final set.
Country/region of study performed
Also, the country or region where a specific study was performed, such as “China”, was not incorporated in the final keyword set.
7
3. Descriptive analysis of literature
In this section, we first present the results addressing the descriptive analysis of the open innovation literature. Hereafter we address the results of the descriptive analysis of the crowdsourcing literature. 3.1 Descriptive analysis of open innovation literature
In this subsection, we provide the results addressing the years of publication, number of articles per journal and the frequency of the author-supplied keywords belonging to the articles.
Publications per year
Figure 1. Breakdown of the number of articles per year
We first present the analysis of the number of publications per year, see Figure 1. The number of articles increases per year; therefore, we can speak of an overall increasing trend. Whereas the initial year, 2010, shows a low number of articles, the number of published articles started increasing and reached its highest point in 2019, in which 229 articles were published. We see a small dip after 2014 and 2017. Though, after this dip the increasing trend continues. The low frequency of published articles in 2020, compared to other years, can be explained by the fact that we have only analyzed articles up until the beginning of May 2020.
Journals
8
Journal name Frequency
articles
Percentages
Technological Forecasting and Social Change 66
Sustainability 53
International Journal of Innovation Management 52
Research Policy 47
International Journal of Technology Management 44 Journal of Open Innovation Technology Market and Complexity 37
Research Technology Management 34
Technology Analysis and Strategic Management 32
Technovation 32
International Journal of Business Innovation and Research 31 European Journal of Innovation Management 26 Journal of Technology Management and innovation 25 International Journal of Innovation and Technology 18
Industry and Innovation 17
Journal of Product Innovation Management 17
California Management Review 16
Journal of Knowledge Management 15
Management Decision 15
Business Process Management Journal 14 Innovation Management Policy and Practice 14 International Journal of Entrepreneurship and Innovation Management 14 International Journal of Technology Intelligence and Planning 13
Journal of the Knowledge Economy 13
Journal of Business Research 12
Journal of Engineering and Technology management 11
Journal of Cleaner Production 10 45,08%
‘Other’ (468 journals) 826 54,92%
Total 1,504 100%
Table 2. Breakdown of articles per journal
9
that, not surprisingly, the journals in which is the highest frequency of articles is published, mostly remain innovation or business management journals.
Keywords
10
Rank # Keywords 2010 2020 # Keywords 2010 – 2013 # Keywords 2014 – 2016 # Keywords 2017 - 2020
1 86 Crowdsourcing 20 Crowdsourcing 28 Crowdsourcing 38 Crowdsourcing
2 55 Collaboration 18 R&D 16 Collaboration 25 Collaboration
3 52 R&D 15 Intellectual Property 14 R&D 25 Absorptive capacity 4 47 Absorptive capacity 15 Network 13 New Product Development 25 Internet of Things 5 40 New Product Development 14 Collaboration 11 Co-creation 21 Innovation performance 6 36 Innovation performance 12 Absoprtive capacity 10 Absorptive capacity 20 R&D
7 30 Intellectual Property 12 New Product Development 10 Innovation performance 17 Entrepreneurship
8 30 Network 10 Technological transfer 9 User innovation 15 New Product Development 9 29 Internet of Things 10 Open source 9 Social media 14 Co-creation
10 28 Co-creation 8 Motivation 8 Radical innovation 14 Social media
11 28 Entrepreneurship 7 Knowledge management 7 Knowledge management 14 Knowledge management 12 28 Knowledge management 7 User innovation 7 External knowledge 12 Network
13 27 User innovation 7 Organizational change 6 Intellectual Property 12 Platform
14 24 Social media 7 Organizational learning 6 ICT 10 Sustainable development 15 20 Sustainability 6 Entrepreneurship 6 Patents 9 Innovation ecosystems
Table 3. Keyword frequencies per total period of time, 2010 to 2020, 2010 to 2013, 2014 to 2016, and 2017 to 2020 ‘#’ shows the number of articles in which the certain keyword is used
11
Table 3 identifies ‘crowdsourcing’ as the most frequently used keyword in both the period of time, as in the three different time blocks. For this reason, we point out ‘crowdsourcing’ as the main topic in the research field of open innovation over the past decade, as the keyword is used in 86 different articles. We analyze these keywords by addressing three aspects: (1) a topic that is emerging within the open innovation field, (2) a topic that has remained the same importance, and (3) a topic that has become less relevant over the past years.
The keyword that can be considered as becoming increasingly important is the ‘Internet of Things’. This keyword is one of the most frequently used keywords over the past decade, with a total frequency of 29. By way of contrast, in the time block of 2017 to 2020, we see that this keyword has a frequency of 25. Consequently, these frequencies imply that the keyword of ‘Internet of Things’ has only become relevant within the research field since 2017. The keyword of ‘Intellectual Property’ can be considered as receiving less attention as time goes by and thereby being less relevant. With a total frequency of 30, it is one of the most frequently used keywords over the past decade. Nonetheless, from 2017 on, it has not been again identified as being frequently used. This implies that the keyword has become less relevant. Additionally, although crowdsourcing has experienced an increase in frequency of publications as time goes by, the term remained the most important subject within the whole period of time and in all three-time blocks.
As time progresses, keywords can be subjected to change. For instance, the name of the keyword may change over time, although the definition will remain the same as in the previous application of the initial keyword. If we apply this given to Table 3, we see that the keyword ‘Network’ might have also experienced a shift in towards a different keyword (in terms of name). In the past decade, the keyword was used 30 times. In the last time block, we see that the keyword again has a high frequency. Even so, we see a new keyword emerging in this time block: the keyword ‘Innovation ecosystem’. This new keyword can possibly be seen as a new name for what was previously called ‘Network’. A possible explanation is that as time goes by, more research is performed. Hence, more knowledge is acquired within the research field. Accordingly, names of keywords might change or be adjusted, due to a possible new and better fitting name.
3.2 Descriptive analysis of crowdsourcing literature
12 Keywords
Besides the keywords ‘open innovation and ‘crowdsourcing, a third keyword might present, which is often used in combination with these two keywords. Accordingly, we analyze the keywords that used in combination with ‘open innovation’ and ‘crowdsourcing’, see Table 4.
Table 4. Keyword used together with ‘crowdsourcing’
Following Table 4, we can conclude that there is no third term present that is often used together with the terms ‘open innovation’ and ‘crowdsourcing’. A possible explanation is that crowdsourcing is a specific form of open innovation, and therefore a fairly self-contained term.
Rank # Keyword
13 Methodologies
In this next step, we analyze the methodologies that were employed in our set of articles. The articles are classified into one three possible methodologies, consisting of qualitative methodology, quantitative methodology and mixed methodology.
Figure 2. Breakdown of methodologies
Figure 2 shows that a small share of the articles, 10,47%, uses a mixed methodology. Additionally, Figure 2 indicates that the largest share of articles uses a qualitative methodology, with 59 articles. A quantitative methodology was used in 17 articles, which is a share of 19,77%.
14
Table 5. Breakdown of methodologies per publication year
From Table 5, we identify that the number of qualitative studies has increased whilst for the quantitative number of studies, it is very difficult to identify a trend. To fully understand these numbers, we constructed a scatterplot for both quantitative studies and qualitative studies
Figure 3. Scatterplot quantitative studies
Year of publication Quantitative Qualitative Total
15
Figure 4. Scatterplot of qualitative studies
16 Data collection method
Following the methodology used per study, here we address the type of data collection used per study.
Figure 5. Breakdown of data collection methods
17
4. Literature review Crowdsourcing
Since the descriptive analysis in the previous chapter identified ‘crowdsourcing’ as the most frequently studied topic within the research field of open innovation, we will zoom in on this term through a systematic literature review. In the descriptive analysis, 86 papers were found to address the term crowdsourcing. Therefore, all 86 articles were evaluated. Due to the exclusion criteria set of the descriptive analysis, all 86 articles already comprise English, peer-reviewed articles in final stage of publication and published in the past decade. For the evaluation of these 86 articles we used the customized approach of Meier (2011). Articles were first scanned based on paper title and abstract to see if they met the inclusion criteria (Table 6). This resulted in 45 seemingly useful papers. Consequently, these articles were scanned based on the introduction and conclusions, which led to 31 articles meeting the inclusion criteria. The reviewed papers were placed in an analytical review scheme, see Appendix C.
Inclusion criteria Description/motivation Context of open
innovation
Only articles were selected addressing crowdsourcing in the context of open innovation.
Explaining the concept of crowdsourcing
The article had to address an explanation to the term of crowdsourcing (solely stating a definition was not enough), and had to address one of the following topics: the content of crowdsourcing, the factors influencing crowdsourcing, the process of crowdsourcing and the forms of crowdsourcing.
Research types Both quantitative, qualitative and mixed methodologies were selected.
Table 6. Inclusion criteria for selection of crowdsourcing articles
To answer our second research question, we first have to elaborate on the content of crowdsourcing to explain the term and its boundaries. Additionally, if a firm decides to use crowdsourcing, they want the crowdsourcing to be effective. But how can effective crowdsourcing be measured, and what influences might there be on this effectiveness? Lastly, we address the steps towards crowdsourcing as well as crowdsourcing practices.
4.1 Content of crowdsourcing
In this subsection, we will address the definition of crowdsourcing and the purpose for which it is used. Also, we describe the relevance it has relative to open innovation and other relevant terms.
Definition and classification of crowdsourcing
18
uniform definition. Although this paper addresses crowdsourcing solely in the context of open innovation, many authors have defined the term crowdsourcing in a broader context. According to Marjanovic, Fry, & Chataway (2012), the term crowdsourcing is known for, like its parent (open innovation), accusable of being an umbrella term itself. The first definition is the definition of Howe (2006a), which states that crowdsourcing “is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call” (Howe, 2006a). The definition of Howe (2006a) is still the most often used definition to date (Schemman, Herrmann, Chappin, & Heimeriks, 2016; Gupta & Sharma, 2012; Foege, Lauritzen, Tietze, & Salge, 2012; Muhdi, Friesike, Dalber, & Boutellier, 2011; Malhotra, & Majchrzak, 2013). However, two years later, Howe (2008) added to this definition that the designated agent outsourcing this job must be a company or an institution (Feller, Finnegan, Hayes & O’Reilly., 2012). Another add-on to the definition of crowdsourcing comprises the use of the word ‘online’, implying that crowdsourcing can only be done through access to the Internet. According, Brabham (2008. p. 76) states that “. . . a company posts a problem online, a vast number of individuals offer solutions to the problem, the winning ideas are awarded some form of a bounty, and the company mass produces the idea for its own gain.” (Cappa et al., 2019; Callaghan, 2014; Cappa, Rosso & Hayes, 2019). The literature on crowdsourcing has, over the past decade, indeed emphasized on the use of digital technology, in order to reach far beyond organizational boundaries (Boons & Stam, 2019; Mladenauer, Bauer & Strauss, 2014). Bauer & Gegenhuber (2015) state that while crowdsourcing does not necessarily rely on the Internet, it nowadays always does.This implies that access to the Internet has become a prerequisite for implementing crowdsourcing.
We can conclude that the term has many definitions and variations (Chiu, Liu & Turban, 2014). In short, crowdsourcing can be considered as a way to facilitate open innovation (Foege et al., 2019), as a means to use the talents of the crowd (Gupta & Sharma, 2012). The definition used in this paper, is the definition given by Howe (2008), combined with the addition of the ‘online’ characteristic of Brabham’s definition (2008). This definition is as follows: “is the act of taking a job traditionally performed by a company or an institution and outsourcing it online to an undefined, generally large group of people in the form of an open call” (Howe, 2008; Brabham, 2008).
19 Crowdsourcing, outsourcing and open source
20
Crowdsourcing Open source Outsourcing
Actors Seeker Public, private, third-sector players with an innovation need
Public, private, third-sector players with an innovation need
Public, private, third-sector players with an innovation need
Solver Anyone can be a potential solver Large pool of independent, decentralized solvers
Anyone can be a potential solver Defined contractor, either identified through open-competition for a contract, or without open competition Roles Seeker Specifying challenges, and making
them known to public
Selecting ‘winners’ and decide reward (directly or via a broker)
A publicly owned question; question not submitted by a specific company or institution*
Every actor within the community can simultaneously fulfill the role of the seeker and the solver*
Problem definition
Setting out contractual agreement Sometimes are also involved in solving process (in collaboration with supplier to whom problem is outsourced) Solver Solves specified problem A publicly owned question; question
not submitted by a specific company or institution*
Every actor within the community can simultaneously fulfill the role of the seeker and the solver*
Solves specified problem
Risk and risk management
Seeker No guarantee of a solution Risk of exposing competitive intelligence
Seeker can use a broker to manage risk, and also pays-for-performance only
Formal terms of engagement and legal frameworks
Potential risks to authenticity and reliability
Risks managed through self-regulation and self-management by community norms
New IPR give a level of attribution to author *
Opportunistic behavior and trust issues can arise, but contractual specifications tend to mitigate these*
Reputation of solver, experience and prior relationships between seeker and solver often play a role in mitigating risk
Clear format for compensating
contributors specified at onset, and who will be compensated clear at onset Solver Risks in terms of upfront investment
of time without guarantee of reward for solver
Potential risks to authenticity and reliability
21
Clear format for compensating contributors specified at onset, but who will be compensated unclear at onset
Risks managed through self-regulation and self-management by community norms
New IPR give a level of attribution to author*
Reputation of solver, experience and prior relationships between seeker and solver often play a role in mitigating risk
Clear format for compensating
contributors specified at onset, and who will be compensated clear at onset Reward/incentives Seeker Solution which can drive competitive
advantage
Improved product to use/community good/collaborative spirit
Solution which can drive competitive advantage
Solver Financial and/or reputational Improved product to use/community good/collaborative spirit
Financal and/or reputational Intellectual
Property
Seeker Almost always yes Depends on the open source platform used**
Generally, yes (depends on agreements with contractor)
Solvers No Depends on the open source
platform used**
Maybe (depends on agreement with contractor)
Table 7. Comparison of features of crowdsourcing, open source and outsourcing
Note. Adopted from “Crowdsourcing based business models: In search of evidence for innovation 2.0”, by Marjanovic, S., Fry, C. & Chataway, J. (2012), Science and Public Policy, 36, p. 323.
The sign * was used for small changes that we have made, adjusting a sentence or adding a few words, and therefore were not incorporated in the original comparison framework of Marjanovic et al. (2012).
22
The comparison between crowdsourcing and open source, show many differences. This comparison thus implies that the two concepts are often mistakenly confused. The most striking difference is that in open source actors can simultaneously fulfill the role of solver and seeker, whereas in crowdsourcing there is a strict boundary between the job of the seeker and the job of the solver. Furthermore, comparing crowdsourcing to outsourcing, a lot of similarities arise. Though, the most striking difference is the number of solvers. In crowdsourcing, the problem is outsourced to a vast number of individuals, whereas in outsourcing the problem is outsourced to a few parties or even one party. The comparison between crowdsourcing and outsourcing also includes another difference between, namely the format for compensation. In both concepts the format for compensating is clear at onset. However, in outsourcing, it is also known at onset who will be compensated. In crowdsourcing, the selected solver is not yet known beforehand.
Crowdsourcing process
So far, we have learned that both a solver and a seeker participate in crowdsourcing. But what activities do the seeker and solver undertake? The process of crowdsourcing can be illustrated as follows: an organization, the seeker, has a problem that it needs to solve. It then presents the problem online, either on its own website or on a dedicated platform, thus inviting the individuals, the solvers, to propose solutions. The seeker then selects the solution it prefers, pays a reward to the inventor, and uses the solution for his/her own purposes (Schenk, Guittard & Penin, 2019). As seekers might experience difficulties in how to crowdsource, a third actor can be identified. This third actor is called ‘platforms’ (Paik, Scholl, Sergeev, Randazoo, & Lakhani, 2010), which are web-based intermediaries that support the seeker in leveraging the power of crowds. The role of intermediary platforms is to support the seeker in problem-solving by connecting the seeker with the solver (Colombo, Buganza, Klanner & Roiser, 2013).
This process of crowdsourcing raises the question of who is to be considered the solver in crowdsourcing; does the term ‘solver’ concern all potential solvers or merely the eventually selected solver with the winning solution? According to Chiu et al. (2014), solvers include different populations, such as experts, non-experts, informal members, customers, etc. From this statement, we conclude that the ‘solver’ indicates all potential individuals than can take part in crowdsourcing. The specific solver whose idea is eventually selected is called the ‘inventor’, following the statement of Schenk et al. (2019).
Boundaries of crowdsourcing
23
and selection of solutions), (2) the seeker selects the solvers based on qualifications or context and also decides if and what incentives will be offered to the inventors (Cai, Gippel, Zhu, & Singh., 2019). Though, the first element of the seeker having control over the process does not always apply. Hence, the word ‘almost’ was added. Total control of the process is on the condition that a seeker does not use a platform. Schemman et al. (2016) identify the third element, by stating that the commercialization of the new products or services should be carried out by the seeker. However, this is not included in the process of crowdsourcing; it rather is part of New Product Development. Crowdsourcing itself ends by selecting one or more winning idea(s), possibly combining these ideas and eventually paying the inventors.
4.2 Crowdsourcing effectiveness
To answer our second research question on how seeker should organize crowdsourcing for innovation, we first address the effectiveness of crowdsourcing. This effective crowdsourcing is addressed to get an insight as to which factors have to be taken into account when organizing for crowdsourcing. Hence, this subsection addresses the way in which effective crowdsourcing can be measured. Second, we list and explain the factors influencing crowdsourcing effectiveness.
Measuring crowdsourcing effectiveness
Many different approaches exist regarding the measurement of crowdsourcing effectiveness: some studies review the effectiveness by the quantity of ideas submitted, while other studies review it by the quality or novelty of the ideas. According to Acar (2019), crowdsourcing should be measured by the quality of the ideas. The quality of ideas concerns the appropriateness of the solutions generated in crowdsourcing. Also, crowdsourcing literature suggests that a positive quantity-quality association exists for submissions of solutions, implying that the likelihood of high-quality solutions increases with the level of solution quantity (Pollok, Lüttgens & Piller, 2019; Lee, Chan, Ho, Choy, & Ip, 2015). Concluding, as the number of participants increase (and so the number of solutions submitted), the likelihood of receiving high-quality solutions increases accordingly. Crowdsourcing should be measured by the quantity of ideas.
Factors influencing crowdsourcing effectiveness
24
received, whereas intrinsic motivation influences the quality of these solutions. Acar (2019) found that both intrinsic and extrinsic motivation were positively related to solution appropriateness. This finding implies that the higher the motivation of the solvers, the more appropriate the solution is for the seeker. However, there is a limit to this compensation. According to the study of Cappa, Rosso & Hayes (2019), monetary compensation brings a positive impact on the number of contributions. Nevertheless, large monetary rewards can be detrimental to attracting participants, resulting in a crowd-out effect. This negative effect can be explained by the fact that when a high monetary reward is promised, crowds tend to think that only experts can make insightful contribution, be compensated, and thereby be discouraged from participating. Besides monetary rewards, non-monetary rewards can also benefit the number of participants in crowdsourcing (Cappa, Rosso & Hayes, 2019). These non-monetary rewards comprise social benefits such as contributing to reducing CO2 emissions.
The second factor concerns the number of solvers. The number of solvers participating in crowdsourcing influence the effectiveness of crowdsourcing. In the context of high uncertainty problems, the number of solvers influence the outcome of a crowdsourcing challenge. Hence, a greater number of potential entrants increase the likelihood of a good solution (Cai et al., 2019). However, Kohler & Nickel (2017) state that a large user base might negatively influence the value-creation experience. This can be explained by the fact that when the number of solvers is high, seekers encounter substantial problems with the presence of oversized communities. For instance, the seeker might not have sufficient capacity to communicate with solvers during the time when seekers are working on their solution.
The third factor flows from the beforementioned communication between seeker and solver, and thus concerns the level of communication. Boons & Stam (2019) found that it is important for seekers to take a more active approach for selecting potential contributors. An active approach consists, for example, of increasing the level of communication between the seeker and solver, thereby improving the ratio of ideas that are valuable compared to those that are not.
Another factor concerns the diversity of the solvers. The presence of a great diversity of solvers allows original solutions to be proposed (Malhotra & Majchrzak, 2014). Concluding, although problems might arise in the value-creating process due to oversized communities, a high number and great diversity of solvers are needed for a sufficient amount and diversity of solutions to be submitted.
Moving beyond characteristics of the solvers that influence the crowdsourcing effectiveness, Lee et al. (2015) underscore the task complexity in crowdsourcing, by addressing that seekers should realize the intellectual level and involvement of their seekers and try to avoid complex tasks that can deter the participation of the solvers. If a task is too complex, this might increase the cognitive demand of the solvers and thereby adversely affect the quality or number of solutions received.
25
time to communicate with solvers during the idea submission process, leading to a higher number and quality of solutions submitted. Though this does not automatically imply that only larger firms can use crowdsourcing strategies. For instance, small-and medium sized enterprises (SME) precisely participate in crowdsourcing, since it enables these firms to reach a global pool of skilled problem solvers quite quickly and cost-efficiently (Feller et al., 2012). However, SMEs often note that they lack detailed knowledge on which platforms are suitable for their business and on the availability of tools (de Mattos, Kissimoto & Laurindo, 2018). Firm size accordingly influences the way in which the crowdsourcing is organized, by means that a smaller firm is more likely to crowdsource through an intermediary platform. Larger firms generally have more resources, which increases the likelihood of these firms to crowdsource on their proprietary platforms. Schenk et al. (2019) accordingly found that intermediary platforms are more relevant for seekers that do not possess a large brand or user community, which is often the case for smaller firms.
5.3 Organizing for crowdsourcing
In this subsection, two subjects are addresses. The first subject addresses the shift from traditional innovation practices towards crowdsourcing. The second subject explains the forms that seekers can use for organizing crowdsourcing.
Towards crowdsourcing
Applying crowdsourcing as a means of opening up innovation practices is seen as the answer to fast-changing user needs, and an increasingly competitive climate (Kohler, 2015). But how does a firm actually shift from its traditional way of searching solutions to crowdsourcing? According to Bauer & Gegenhuber (2015), the process towards crowdsourcing ranges from a local to a distant search. When a seeker has chosen crowdsourcing as a way to search for innovative solutions, they thus went through four steps of search options. These steps are as follows: internal sourcing, external sourcing to local specialized contractors, external sourcing to distant specialized contractors, and lastly, crowdsourcing (Bauer & Gegenhuber, 2015). An important note here is that the seeker will only extend this search radius (and thus continue to the next step of sourcing), if previous searches have not resulted in a satisfying outcome.
In internal sourcing, the seeker will source the problem in-house. In external sourcing to local dedicated contractors, the problem is sourced to actors that are familiar to the seeker. In step three, external sourcing to distant dedicated contractors, the seeker does not know the contractor; and merely selects the contractor based on the knowledge they possess on the specific problem. In the fourth and last step, crowdsourcing is used. In crowdsourcing, the sources are identified through a global search. This global search refers to covering the complete search space, whereby all potential solvers can be reached.
26
to develop new products and services, which can be understood in a number of different ways (Malhotra & Majchrzak, 2014). Well-defined problems will often be understood straight away by the solvers. Sourcing this problem to the solvers happens through a request for proposal. Defining the problem and thereby writing the request for proposal might seem easy, but this is often the most challenging part of crowdsourcing (Pollok et al., 2019; Lee et al., 2015). In sourcing the problem to the solvers, the seeker deals with the “paradox of openness”. The paradox of openness means that, to capture the solvers’ attention and enable solution proposals, sufficient information has to be revealed. This sufficient information has to be revealed on the problem itself, but also on the opportunities that follow by solving it. Though, at the same time, the seeker has to be very strict on not disclosing technological deficiencies and future development projects to current customers and potential competitors. The dilemma that follows is that when too little or the wrong information is disclosed to the solvers, these solvers might not be willing to contribute and submit a solution (Pollok et al., 2019). Concluding, when considering crowdsourcing as the way to go, the seeker has to have a specified problem that needs to be solved (Marjanovic et al., 2012). Also, the seeker does not know upfront who the inventor will eventually be, which leads to the task needing to be outsourced to a large pool of independent solvers, instead of a defined contractor (Marjanovic et al., 2012).
Crowdsourcing practices
In order to use crowdsourcing for inventive activities, the way of organizing crowdsourcing has to be determined. According to Lee et al. (2015), using crowdsourcing requires planning, organization, and coordination. The authors state that implementing crowdsourcing for innovative problem-solving involves the selection of solvers expertise and the way of communicating with participants, implying that crowdsourcing comprises more than merely asking solvers to submit solutions. The authors address the importance of carefully selecting the solvers, determining the way of communication between the seeker and the solver, and the design of the problem that is sourced to the solvers. Hence, applying crowdsourcing for innovation practices, requires managers to rethink certain firm aspects such as managerial and governance structures which enable the knowledge flow across the boundaries of the firm (Bogers et al., 2017).
27
comprise external intermediary platforms as multiple seekers can use them. Examples of intermediary platforms are specialized online platforms such as InnoCentive (for science problem-solving challenges) and TopCoder (for software challenges) (Boons & Stam, 2019; Callaghan, 2014). When using an intermediary platform, the intermediary sources a problem-solving task of the seeker to a solver in the form of an open call for proposals (Diener, Luettgens, & Piller, 2020). A characteristic of an intermediary platform concerns the diversity of the solvers(Boons & Stam, 2019). Instead of the seeker, the intermediary performs the support activities in crowdsourcing, such as problem formulation or solution evaluation.
After choosing the platform that the seeker will use, a specific form of crowdsourcing has to be chosen. Paik et al. (2010) adopted the forms of crowdsourcing given by Boudreau & Lakhani (2013), implying that there are four different forms of crowdsourcing. The first form is called ‘crowdsourcing challenges’, also referred to as crowdsourcing contests or tournament-based crowdsourcing. In this approach, solvers compete against each other to generate the best outcome for an innovation problem (Acar, 2019). The following steps consist for crowdsourcing challenges: the seeker formulates a problem, offers a monetary reward and publishes an invitation for solvers to submit solutions (Boudreau & Lakhani, 2013). Crowdsourcing challenges are increasingly used by seekers, for the identification of new ideas for better servicing their customers (Armisen & Majchrzak, 2015). Though this does not directly imply that the degree to which these challenges have provided new ideas is also increasing. According to Armisen & Majchrzak (2015) the latter has been rather disappointing. The second form of crowdsourcing is called ‘crowdsourcing communities’, also referred to as collaborative communities. In crowdsourcing communities, interaction between solvers take place and solvers thereby collaborate for the generation of creative outcomes. Crowdsourcing communities often take place on an ongoing basis (Acar, 2019; Schemman et al., 2016), whereas crowdsourcing contests is a short-term competition for a specific problem. A prominent example that is often used to describe a crowdsourcing community with, is Wikipedia (Paik et al., 2020). Boudreau & Lakhani (2013) states that, besides crowdsourcing and contests, two more forms of crowdsourcing exist. The third form is called ‘complementors’, in which the core product of a firm facilitates a market for third-party offerings from the solvers, thereby enhancing the core product value. The fourth and final form is called ‘open labor market’, which helps seekers to find solvers who are paid for their work on the problem rather than the solution. Though, in this study we merely consider the first two forms, crowdsourcing challenges and crowdsourcing communities, as forms used for inventive activities.
28
Figure 6. Steps for organizing crowdsourcing from seekers’ perspective
5. Discussion
In this section, we discuss the findings of our study by addressing the research objectives. Furthermore, the main implications of our thesis are discussed, and directions for future research are given.
The objective of this study is twofold: first to map the open innovation literature of the past decade, hence the research question: “How has the research field of open innovation developed from 2010 onwards?”. In effort to answer this question, a descriptive analysis was performed. The second objective of this study was to describe forms that seekers can use in organizing crowdsourcing, hence the research question: “How should crowdsourcing for innovation be organized by seekers?”. In order to answer this second research question, a systematic literature review was performed.
5.1 Main findings and implications
In the first part of our study, we first performed a descriptive analysis based on the consideration set of 1,504 articles. This analysis leads to several main findings. First, we observed an overall increasing trend regarding the number of published articles from 2010 to 2019. This increasing trend implies that open innovation is increasingly being worldwide (Cappa, Rosso & Hayes, 2019). Thus, the term has not yet faded away to due fully being incorporated in innovation management practices, contradicting the prediction of Huizingh (2011). Another interesting finding is the number of journals in which the articles were published. This number consists of a total of 494 journals. 45,08% of the articles were published in only 26 journals, which means that the remaining 54,92% of articles were published in 468 journals. These percentages imply that open innovation has received attention from a high number and a wide variety of different journals, which go beyond the boundary of solely innovation or business management journals. Furthermore, our findings indicate crowdsourcing as the most studied topic within the field of open innovation. As crowdsourcing is a fairly new term, we found that most articles written on the subject concern qualitative, for example case studies, and thus theory-building studies. Consequently, as case studies describe the early adopters of a concept (Davis, Richard & Keeton, 2015; Schlagwein & Bjorn-Andersen, 2014), not all findings may be generalizable and thus applicable for following firms.
The systematic literature, the second part of the study, provides the synthesis for the set of crowdsourcing papers. First, crowdsourcing is, aside from a few similarities, very different from the concepts of ‘open source’ and ‘outsourcing’ (Marjanovic et al., 2012). Second, the role and motivation
29
of the crowd is the most often researched topic within crowdsourcing (Kohler & Nickel, Cai et al., 2019; Malhotra & Majchrzak, 2014; Lee et al., 2015; Acar, 2019; Cappa, Rosso & Hayes, 2019), thereby denying the role other factors play in determining the outcome of crowdsourcing. These ‘other factors’ are addressed in our findings chapter. We addressed the steps from a closed business model to crowdsourcing, based on the study of Bauer & Gegenhuber (2015). A few prerequisites exist in to participate in crowdsourcing. Furthermore, many authors address several forms in which crowdsourcing can be organized, varying from short term use to long term use and from using a proprietary or an intermediary platform (Schemman et al., 2016; Boons & Stam, 2019; Schenk et al., 2019; Callaghan, 2014; Ettlinger, 2017). However, though there are forms of crowdsourcing, according to Schenk et al. (2019) crowdsourcing challenges are most often used. Besides crowdsourcing challenges, many different forms can possibly be suitable for organizing crowdsourcing. As these forms exist, further research is needed. We synthesized the literature on forms of crowdsourcing, and platforms as well, and therefore gave a five-step approach for crowdsourcing.
The main contribution of this study is twofold. First, this study contributes to an overview of the open innovation literature and developments within this research field. Second, available literature regarding organizing for crowdsourcing is synthesized, leading a few implications for further research.
5.2 Future research directions
We introduce four research directions that resulted from our study. One research direction has resulted from our descriptive findings, whereas three other research directions have resulted from our systematic literature review.
(1) As the descriptive analysis of this research has been performed using the Scopus database, the findings might differ when using another database. Although Scopus one of the leading databases when it comes to scientific publications, performing the analysis using a different database might result in different findings. Therefore, we argue for further research on the development of the open innovation research field by using a different database.
(2) This study furthermore argues for more quantitative, theory testing, research. As crowdsourcing is still a fairly new concept, the concept has mainly been studied through qualitative research. However, in order to test the theory on crowdsourcing, a quantitative research focus is needed.
(3) Scholars have identified several factors influencing the effectiveness of crowdsourcing. Though more research is needed; for example, through empirical testing of certain variables. In this manner, relationships between the factors and the outcome can be described.
30
platforms might make it easier for seekers to use crowdsourcing, such as through the existence of a multi-seeker and internal platform (a more hybrid form), we acknowledge this urgency from Schenk et al. (2019) and argue for more research on intermediary platforms.
5.3 Limitations
Although the systematic review was conducted in a disciplined manner, we do have to acknowledge potential limitations. The search process was limited to English-written articles through using the Scopus database. Another inclusion criterion was the presence of the indexed keyword of “Open Innovation” in this database. Thus, this review does not involve open innovation literature that goes beyond the boundary of this keyword or was published in different language than English. To identify a main important topic within the open innovation literature, a keyword analysis was performed. However, due to the great number of keywords we set up a few exclusion criteria. These exclusion criteria, such as general innovation management terms and industries, led to the corresponding keywords not being incorporated in the final keyword list. The third limitation is that of the clustering of keywords within the dataset. To ensure that this list consisted of as little researcher subjectivity as possible, we only clustered the keywords for abbreviations and the use of multiples and singulars of the same keyword. The last limitation is the coding process of the systematic literature review, as different coding may lead to different results. We believe, though, that we have done our best to follow a consistent and proper coding procedure.
31
References
Acar, O.A. (2019). Motivations and solution appropriateness in crowdsourcing challenges for innovation. Research Policy, 48, 1-13.
Afuah, A., & Tucci, C.L. (2012). Crowdsourcing as a solution to distant search. The Academy of Management Review, 37 (3), 355-375.
Armisen, A., & Majchrzak, A. (2015). Tapping the innovative business potential of innovation contests. Business Horizons, 58 (4), 389-399.
Barczak, G. (2012). The Future of NPD/Innovation Research. Journal of Product Innovation Management, 29 (3), 355-357.
Bauer, R.M., & Gegenhuber, T. (2015). Crowdsourcing: Global search and the twisted role of consumers and producers. Organization, 22 (5), 661-681.
Bogers, M., Chesbrough, H., Heaton, S. & Teece, D. (2019). Strategic Management of Open Innovation: A Dynamic Capabilities Perspective. California Management Review, 62 (1), 77-94. Boons, M. & Stam, D. (2017). Crowdsourcing for innovation: How related and unrelated perspectives interact to increase creative performance. Research Policy, 48 (7), 1758-1770.
Boudreau, K.J., & Lakhani, K.R. (2013). Using the Crowd as an Innovation Partner. Harvard Business Review, 91 (4), 60-69.
Brabham, D.C. (2008). Crowdsourcing as a Model for Problem Solving: An Introduction and Cases. Convergence: The International Journal of Research into New Media Technologies, 14 (1), 76. Brabham, D.C. (2008). Crowdsourcing as a Model for Problem Solving: An Introduction and Cases. Convergence: The International Journal of Research into New Media Technologies, 14 (1), 75-90. Cai, C.W., Gippel, J., Zhu, Y. & Singh, A.K. (2019). The power of crowds: Grand challenges in the Asia-Pacific region. Australian Journal of Management, 44 (4), 551-570.
Callaghan, C.W. (2014). R&D Failure and Second-Generation R&D: New Potentialities. Mediterranean Journal of Social Sciences, 5 (3), 11-24.
Cappa, F., Rosso, F. & Hayes, D. (2019). Monetary and Social Rewards for Crowdsourcing. Sustainability, 11 (10), 1-14.
Cappa, F., Oriani, R., Pinelli, M. & De Massis, A. (2019). When does crowdsourcing benefit firm stock market performance? Research Policy, 48, 1-11.
32
Chiu, C.-M., Liang, T.-P., Turban, E. (2014). What can crowdsourcing do for decision support? Decision Support Systems, 65, 40-49.
Colombo, G., Buganza, T., Klanner, I.M., & Roiser, S. (2013). Crowdsourcing Intermediaries and Problem Typlogies: An Explorative Study. International Journal of Innovation Management, 17 (2), 1-24.
Davis, J.R., Richard, E.E. & Keeton, K.E. (2015). Open Innovation at NASA: A New Business Model for Advancing Human Health and Performance Innovations. Research-Technology Management, 58 (3), 52-58.
Devece, C., Palacios, D., & Ribeiro-Navarrete, B. (2019). The effectiveness of crowdsourcing in knowledge-based industries: the moderating role of transformational leadership and organizational learning. Economic Research-Ekonomska Istrazivanja, 32 (1), 335-351.
Di-Benedetto, A. (2010). Comment on ‘Is open innovation a field of study or a communication barrier to theory development?’. Technovation, 30, 557.
Diener, K., Luettgens, D., & Piller, F.T. (2020). Intermediation for open innovation: comparing direct versus delegated search strategies of innovation intermediaries. International Journal of Innovation Management, 24 (4), 1-20.
Ettlinger, N. (2017). Open innovation and its discontents. Geoforum, 80, 61-71.
Feller, J., Finnegan, P., Hayes, J., & O’Reilly, P. (2012). ‘Orchestrating sustainable crowdsourcing’: A characterization of solver brokerages. Journal of Strategic Information Systems, 21, 216-232.
Foege, J.N., Dragsdaghl Lauritzen, G., Tietze, F., & Salge, T.O. (2019). Reconceptualizing the paradox of openness: How solvers navigate sharing-protecting tensions in crowdsourcing. Research Policy, 48 (6), 1323-1339.
Greco, M., Grimaldi, M., & Cricelli, L. (2015). Open innovation actions and innovation performance: A literature review of European empirical evidence. European Journal of Innovation management, 18 (2), 150-171.
Gupta, D.K., & Sharma, V. (2013). Exploring crowdsourcing: a viable solution towards achieving rapid and qualitative tasks. Library Hi Tech News, 2, 14-20.
Howe, J. (2006). The rise of crowdsourcing. Wired Magazine, 14 (6), 1-4.
33
Howe, J. (2008). Crowdsourcing: Why the Power of the Crowd is driving the Future of Business. Crown Publishing Group, New York, NY.
Huizingh, K.R.E. (2011). Open innovation: State of the art and future perspectives. Technovation, 31 (1), 2-9.
Kleemann, F., Voss, G., & Rieder, K.M. (2008). Un(der)paid Innovators: The Commercial Utilization of Consumer Work through Crowdsourcing. Science, Technology & Innovation Studies, 4 (1), 5-26. Kraus, S., Kailer, N., Dorfer, J., & Jones, P. (2019). Open innovation in (young) SMEs. International journal of Entrepreneurship and Innovation, 21 (1), 47-59.
Kohler, T. (2015). Crowdsourcing-Based Business Models: How to Create and Capture Value. California Management Review, 57 (4), 63-84.
Kohler, T. (2018). How to Scale Crowdsourcing Platforms. California Management Review, 60 (2), 98-121.
Lee, C.K.M., Chan, C.Y., Ho, S., Choy, K.L. & Ip, W.H. (2015). Explore the feasibility of adopting crowdsourcing for innovative problem solving. Industrial Management & Data Systems, 115 (5), 803-832.
Malhotra, A., & Majcrzak, A. (2014). Managing Crowds in Innovation Challenges. California Management Review, 56 (4), 103-123.
Marjanovic, S., Fry, C., & Chataway, J. (2012). Crowdsourcing based business models: In search of evidence for innovation 2.0. Science and Public Policy, 39, 318-332.
Marjanovic, S., Fry, C., & Chataway, J. (2012). Crowdsourcing based business models that last: In search of evidence for innovation 2.0. Science and Public Policy, 39, 323.
Meier, M. (2011). Knowledge Management in Strategic Alliances: A Review of Empirical Evidence. International Journal Management Review, 13 (1), 1-23.
De Mattos, C.A., Kissimoto, K.O., & Laurindo, F.J.B. (2018). The role of information technology for building virtual environments to integrate crowdsourcing mechanisms into the open innovation process. Technological Forecasting & Social Change, 129, 143-153.
Mergel, I. (2015). Opening Government: Designing Open Innovation Processes to Collaborate with External Problem Solvers. Social Science Computer Review, 33 (5), 599-612.
Mladenauw, A., Bauer, C., & Strauss, C. (2014). Social Crowd Integration in New Product
34
Muhdi, L., Friesike, S., Dalber, M., & Boutellier, R. (2011). The crowdsourcing process: an intermediary mediated idea generation approach in the early phase of innovation. International Journal of entrepreneurship and Innovation Management, 14 (1), 315-332.
Paik, J.H., Scholl, M., Sergeev, R., Randazzo, S., & Lakhani, K.R. (2020). Innovation Contests for High-Tech Procurement. Research-Technology Management, 63 (2), 36-45.
Pollok, P., Lüttgens, D., & Piller, F.T. (2019). Attracting solutions in crowdsourcing contests: The role of knowledge distance, identity disclosure, and seeker status. Research Policy, 48, 98-114.
Randhwawa, K., Wilden, R., & Hohberger, J. (2016). A Bibliometric Review of Open Innovation: Setting a Research Agenda, 33 (6): 750-772.
Schemmann, B., Hermmann, A.M., Chappin, M.M.H., & Heimeriks, G.J. (2016) Crowdsourcing ideas: Involving ordinary users in the ideation phase of new product development. Research Policy, 45, 1145-1154.
Schenk, E., Guittard, C., & Pénin, J. (2019) Open or proprietary? Choosing the right crowdsourcing platform for innovation. Technological Forecasting and Social Change, 144, 303-310.
Schlagwein, D. & Bjorn-Andersen, N. (2014). Organizational Learning with Crowdsourcing: The Revelatory Case of LEGO. Journal of the Association for Information Systems, 15, 754-778. Tranfield, D., Denyer, D. & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British Journal of Management, 14, 207-222.
35
Appendixes
Appendix A: Clustered keywords
Final keyword Keywords that were clustered Frequencies before clustering Number of duplicates Frequency after clustering
SMEs Small and Medium Sized Enterprises SMEs
Small and Medium Enterprise 9 90 4 8 95 R&D R&D Research and Development Research Development Research & Development R and D 35 15 1 2 1 2 52
Case study Case study Case studies Case study analysis Case study method Case study design Single case study Multiple case study Cross case study
39 9 1 1 1 1 2 1 5 50
Absorptive capacity Absorptive capacity ACAP 47 1 1 47 New Product Development New Product Development NPD 39 9 8 40
Living labs Living lab Living laboratory
25 3
1 27
36 Appendix B: Articles consideration set ‘crowdsourcing’
Author and title of article Methodology
Malhotra, A. & Majcrzak, A. (2014) Managing Crowds in Innovation Challenges.
California Management Review, 56 (4), 103-123.
Qualitative
Kohler, T. (2015). Crowdsourcing-Based Business Models: How to Create and Capture Value. California Management Review, 57 (4): 63-84.
Qualitative
Paik, J.H., Scholl, M., Sergeev, R., Randazzo, S. & Lakhani, K.R. (2020). Innovation Contests for High-Tech Procurement. Research-Technology Management, 63 (2), 36-45.
Qualitative
De Mattos, C.A., Kissimoto, K.O. & Laurindo, F.J.B. (2018). The role of information technology for building virtual environments to integrate crowdsourcing mechanisms into the open innovation process. Technological Forecasting & Social Change, 129, 143-153.
Qualitative
Kohler, T. (2018). How to Scale Crowdsourcing Platforms. California Management
Review, 60 (2), 98-121.
Qualitative
Kohler, T. & Nickel, M. (2017). Crowdsourcing business models that last. Journal of
Business Strategy, 38 (2), 25-32.
Qualitative
Armisen, A. & Majchrzak (2015). Tapping the innovative business potential of innovation contests. Business Horizons, 58, 389-399.
Qualitative
Wilson, K.B., Bhakoo, V. & Samson, D. (2018). Crowdsourcing: A contemporary form of project management with linkages to open innovation and novel operations. International
Journal of Operations & Production Management, 38 (6), 1467-1494.
Qualitative
Wagner, E.B. (2011). Why Prize? The Surprising Resurgence of Prizes to Stimulate Innovation. Research-Technology Management, 54 (6), 32-36.
Qualitative
Schlagwein, D. & Bjorn-Andersen, N. (2014). Organizational Learning with Crowdsourcing: The Revelatory Case of LEGO. Journal of the Association for
Information Systems, 15, 754-778.
Qualitative
Pedersen, K. (2020). What can open innovation be used for and how does it create value?
Government Information Quarterly, 37, 1-13.
Qualitative
Muhdi, L., Daiber, M., Friesike, S. & Boutellier, R. (2011). The crowdsourcing process: an intermediary mediated idea generation approach in the early phase of innovation.
International Journal of Entrepreneurship and Innovation Management, 14 (4), 315-332.
Qualitative
Mortara, L., Ford, S.J. & Jaeger, M. (2013). Idea Competitions under scrutiny: acquisition, intelligence or public relations mechanism? Technological Forecasting & Social Change, 80, 1563-1578.
Qualitative
Martinez, M.G. & Walton, B. (2014). The wisdom of crowds: The potential of online communities as a tool for data analysis. Technovation, 34, 203-214.
Qualitative
Gupta, D.K. & Sharma, V. (2013). Exploring crowdsourcing: a viable solution towards achieving rapid and qualitative tasks. Library Hi Tech News, 2, 14-20.
37
Feller, J., Finnegan, p., Hayes, J. & O’Reilly, P. (2012). ‘Orchestrating’ sustainable crowdsourcing: A characterization of solver brokerages. Journal of Strategic Information
Systems, 21, 216-232.
Qualitative
Felin, T., Lakhani, K.R. & Tushman, M.L. (2017). Firms, crowds, and innovation.
Strategic Organization, 15 (2), 119-140.
Qualitative
Ettlinger, N. (2017). Open innovation and its discontents. Geoforum, 80, 61-71. Qualitative Davis, J.R., Richard, E.E. & Keeton, K.E. (2015). Open Innovation at NASA: A New
Business Model for Advancing Human Health and Performance Innovations.
Research-Technology Management, 58 (3), 52-58.
Qualitative
Cummings, S., Daellenbach, U., Davenport, S. & Campbell, C. (2012). “Problem-sourcing”: a re-framing of open innovation for R&D organisations
Qualitative
Colombo, G., Buganza, T., Klanner, I.M. & Roiser, S. (2013). Crowdsourcing intermediaries and problem typologies: an explorative study. International Journal of
Innovation management, 17 (2), 1-24.
Qualitative
Callaghan, C.W. (2014). A New Probabilistic Problem-Solving Paradigm: A Conceptual Critical Reflection. Mediterranean Journal of Social Sciences, 23 (5), 2070-2079.
Qualitative
Callaghan, C.W. (2014). R&D Failure and Second-Generation R&D: New Potentialities.
Mediterranean Journal of Social Sciences, 3 (5), 11-24.
Qualitative
Bogers et al. (2017). The open innovation research landscape: established perspectives and emerging themes across different levels of analysis. Industry and Innovation, 24 (1), 8-40.
Qualitative
Ettlinger, N. (2016). The governance of crowdsourcing: Rationalities of the new exploitation. Environment and Planning A, 48 (11), 2162-2180.
Qualitative
Lenart, Gansiniec, R. (2016). Relational capital and Open Innovation – In search of interdependencies. Acta Universitatis Agriculturae et Silviculturae Mendelianae
Brunensis, 64 (6), 2007-2013.
Qualitative
Mustafa, S.E. & Adnan, H.M. (2017). Crowdsourcing: A Platform for Crowd Engagement in the Publishing Industry. Publishing Research Quarterly, 33, 283-296.
Qualitative
Almiralli, E., Lee, M. & Majchrzak, A. (2014). Open innovation requires integrated competition-community ecosystems: lessons learned from civic open innovation. Business
Horizons, 57, 391-400.
Qualitative
Franzoni, C. & Sauermann, H. (2014). Crowd science: the organization of scientific research in open collaborative projects. Research Policy,43, 1-20.
Qualitative
Chanal, V. & Caron-Fasan, M.L. (2010). The Difficulties involved in Developing
Business Models open to Innovation Communities: the Case of a Crowdsourcing Platform.
M@n@gement, 13 (4), 316-341.
Qualitative
Gustetic, J.L., Cursan, J., Rader, S. & Ortega, S. (2015). Outcome-driven open innovation at NASA. Space Policy, 34, 11-17.
Qualitative
Gustetic, J.L., Friedensen, V., Kessler, J.L., Jackson, S. & Parr, J. (2018). NASA’s Asteroid Grand Challenge: Strategy, Results and lessons Learned. Space Policy, 44-45, 1-13.
