Synthesizing Antecedents of Idea Evaluation and Selection -
The Bottleneck of Innovation
A systematic literature review
Master thesis, MScBA, specialization Strategic Innovation Management
University of Groningen, Faculty of Economics and Business
22/6/2015
Michal Vojta
Student number: s2559501
vojta.michal@gmail.com
Abstract
Generating large amount of ideas has proven to be a rather simple task, but being able to select the one breaking through towards ultimate success seem to be a privilege of few. Even though the shift of focus from idea generation towards idea evaluation and selection is present in the literature, comprehensive theoretical body has yet to be developed. This study employs a systematic literature review approach in effort to identify relevant antecedents and variables influencing this process in an exhaustive set of literature. Synthesis of identified antecedents - mainly psychology and process setting related - enabled to draw propositions, forming a comprehensive multi-dimensional framework capturing forces influencing idea evaluation and selection process. This framework fulfills the ultimate objective to provide guidance for further research and allows practitioners to implement the effects of antecedents in the innovation process management.
Word count: 13284
Keywords: idea selection, idea evaluation, antecedents Supervisor: dr. René Van Der Eijk
1 INTRODUCTION ... 1
1.1 Ideas ... 1
1.2 Idea Evaluation and Selection ... 1
1.3 Research Relevance ... 2 2 METHODOLOGY ... 3 2.1 Choice of Methodology ... 3 2.2 Data Collection ... 3 2.3 Data analyzing ... 4 3 ANALYSIS ... 5 3.1 Theoretical perspectives ... 5
3.2 Idea Evaluation and Selection Antecedents ... 7
4 SYNTHESIS ... 8 4.1 Setting dimensions ... 8 4.2 Psychological antecedents ... 13 5 DISCUSSION ... 18 6 CONCLUSION ... 22 6.1 Implications ... 22
6.2 Limitations and Future Research ... 23
1 INTRODUCTION 1.1 Ideas
In 2001 there has been a record number of ideas generated at one place when 8000 people generated nearly half million ideas in a single hour1. One can be amazed by this tremendous power of
human brain as well as with the ingenuity and imagination embedded in sketches which participants of another experiment created when told to think of as many innovations as possible on as simple thing as a toaster (Kudrowitz & Wallace, 2013). Similarly impressive numbers can be found in professional setting as well. In design company IDEO, employees generate over hundred ideas per hour (Kelly, 2001). Due to this power of human imagination, past research has heavily focused on idea generation and has brought multiple tools to improve this process (e.g. brainstorming), that are widely used to this day.
1.2 Idea Evaluation and Selection
However, the focused attention on the idea generation is rather myopic. Statistics show, that 3 000 ideas need to be generated in order to achieve one commercial success (Stevens & Burley, 1997). Although another research provides a more conservative ratio of 62 ideas to one successful (Booz, Allen, & Hamilton, 1982), it still illustrates that the idea generation is not the most important part. The poor new idea success rate discourages innovators to invest into new ideas. The cost of new product failure is prohibitive. Even though companies fail to deliver success from various reasons (Ozer, 2005), ineffective evaluation and selection of generated ideas is one of the main hurdles (Chan & Ip, 2010). This illustrates the importance of and the need for thorough idea evaluation and selection to leverage the tremendous amount of ideas some of the world-class brains generate.
Contemporary research (Jones & Samalionis, 2008; Ozer, 2005; Rietzschel, Nijstad, & Stroebe, 2010) suggests that rather than increasing the level of idea creativity and originality in idea generation, improving the quality of idea evaluation and selection is the key to success. Without the ability to select the breakthrough idea from a set of generated ideas, it will never reach the implementation phase (Sharma, 1999).Hence, idea selection is reflected as one of the most critical decisions in new product development (NPD) (Buyukozkan & Feyzioglu, 2004). Furthermore, the effectiveness of idea evaluation and selection influences not only the subsequent phases of NPD process, but it has a backward impact on idea generation as well. As Bothos, Apostolou, and Mentzas (2012) propose, the generation of ideas is to a certain extent determined by the next step, as the evaluation apprehension – people being afraid of future judgment – negatively influences creativity.
Although idea evaluation and selection is an important part of innovation in companies (Urban, Hauser, & Urban, 1993) and both theory and practice has made several attempts to improve the success rate, it is still associated with high risk and uncertainties (Foo, 2010; Ozer, 2005). In this light, idea evaluation and selection seems to be the bottleneck of innovation. In order to improve the idea evaluation and selection step, it is first important to understand factors influencing its effectiveness. There has been quite extensive research devoted to study the antecedents of idea evaluation and
selection. However, despite the Ozer’s (Ozer, 2005) sporadic attempt to summarize the knowledge on this issue in management disciplines, the literature is still scattered over multiple fields and no comprehensive study has been introduced to provide grounds for thorough research taking into account multiple antecedents of idea evaluation and selection and influences among them.
Therefore, the goal of this study is to identify the antecedents of idea evaluation and selection and their influence on this process.
In order to meet the goal of this study, the scope is set across multiple disciplines. Although other disciplines are involved in studying idea evaluation and selection (medicine for example), the domain of this research is set on management and psychology. The aspiration of this study is not to test the relationships, models, or conclusions from the extant literature, which is left to the future research. Literature is not unified in how authors refer to the studied processes. While some authors see idea evaluation and idea selection as synonyms (e.g. D. T. Campbell, 1960), others view it as two consecutive processes, where an idea has to be evaluated first, in order to be selected (e.g. Rietzschel, Nijstad, & Stroebe, 2010). As meeting the goal of this study requires the inclusion of as much relevant literature as possible, the term idea evaluation and selection is adopted.
1.3 Research Relevance
This research picks up the call of many authors for more attention towards idea evaluation and selection rather than idea generation (e.g. Ekvall, 1996; Ozer, 2005; E. F. Rietzschel, Nijstad, & Stroebe, 2010). Extant research has addressed this issue, however, the authors focus only on a single or few antecedents of the idea evaluation and selection. Some antecedents, such as the ability of people accurately evaluate originality of ideas (e.g. Licuanan, Dailey, & Mumford, 2007; Mumford et al., 2006; Runco, Okuda, & Thurston, 1987), receive a substantial amount of attention, while others, for example the evaluation of originality in a team setting after a brainstorming session (despite its importance in NPD process), have not been in appropriate focus (Putman & Paulus, 2009). Reviewing and visualizing the present literature on idea evaluation and selection is therefore relevant theoretical interest.
Despite decision-making processes are made use of in most companies, the initial idea selection still seems to be political and champion-based activity (Barczak, Griffin, & Kahn, 2009). The application of feasibility criteria either from formal, working group, or managerial perspective is usually a cause of failure even though Rietzschel, Nijstad, and Stroebe (2010) advocates that the feasibility and originality are compatible if enough attention to idea selection is granted. Careful selection process reduces risk and uncertainties, which is highly desirable effect (Ozer, 2005). To move towards being able to take into account, predict and manage many influences and interaction of antecedents on idea evaluation and selection forms the managerial interest in this study.
drawn and embedded into a comprehensive framework. This study concludes with implications and recommendations for further research.
2 METHODOLOGY
2.1 Choice of Methodology
As argued earlier, literature on idea evaluation and selection is scattered across multiple disciplines and has not been linked together yet, hence qualitative literature review (van Aken, Berends, & Van der Bij, 2012) has been adopted to systematically evaluate the given body of literature (Ginsberg & Venkatraman, 1985). Methodological process proposed by Crossan and Apaydin (2010) has been followed to remove personal bias and subjectivity from data gathering process. By employing a predetermined search algorithm as transparent and reproducible procedure, quality improvement of review process and outcome is attained (Tranfield, Denyer, & Smart, 2003). This paper follows a three-stage approach, used by Crossan and Apaydin (2010), which begins with data collection process reporting, followed by comprehensive data analysis and concludes with data synthesis, from which the propositions are derived.
2.2 Data Collection
Data collection process consisting from sample search and narrowing the initial sample to a final data set is now reported.
2.2.1 Sample search
review is of multidisciplinary nature. This search led to initial sample of 643 articles. Additional backward citation search during screening phase allowed to capture complementary papers related to the topic. This second search yielded 47 papers, resulting in initial data sample of 692 papers.
2.2.2 Data Converging
Paper citation counts indicate paper to be peer-reviewed, in other words the quality of the paper (Saha, Saint, & Christakis, 2003). As it has been chosen to include only peer reviewed articles The Web of Science was used to seek citation counts and exclude papers with no citation and Google Scholar database was employed as a secondary source for this process. Journal’s impact factor of journals has been accepted as an indicator of journal’s quality as it is based on citation frequency of articles published in particular journal (Garfield, 2006). The ISI Web of Knowledge Journal Citation Reports were employed to remove articles published in journals with no impact factor or lower impact factor than 0.50. Further, duplicates were removed as well as non-English written papers.
As a last step of conversion the abstracts were analyzed to examine the relevancy of each paper. Papers which are not dealing with or do not have direct connection to the idea evaluation and selection were eliminated. Among excluded papers, articles dealing with exclusively NPD or idea management in general were mostly represented. This convergence process resulted in the final sample of 120 papers (Table 1.), which has been fixed for further analysis.
Table 1: Data sample
Electronic
database Limitations Keywords
Initial sample No citation, impact factor <0.50, errors Duplicates Abstract not relevant Final sample Web of Science Core Collection article and review; English
topic search keywords: idea selection, idea evaluation, innovation selection, innovation
evaluation, idea management, selection evaluation 116 33 - 51 32 EBSCO Host Business Source Premier Scholarly (Peer Reviewed) Journals
abstract search keywords: : idea selection, idea evaluation, innovation selection, innovation
evaluation, idea management, selection evaluation, idea choice
104 26 4 55 19 Google scholar not “include patents”; not “include citations”
tittle search keywords: idea selection, idea evaluation, innovation selection, innovation
evaluation, idea management
423 309 25 67 22
Backward citation
tittle keywords search: idea selection, idea evaluation, innovation selection, innovation
evaluation, idea management
49 2 - - 47
Total 692 370 29 173 120
2.3 Data analyzing
of evaluation by experts or non-experts (O’Quin & Besemer, 1999). Another categories then evolved logically during the reading of articles, for instance the connection or separation of idea generation and evaluation in the studies. Antecedents from each paper were then extracted, followed by grouping and organizing them into a conceptual model which will be now reported in detail. Although this research is of qualitative nature, review and conclusions have been supported by basic calculations in effort to increase the quality of the research by employing principle “count the countable” suggested by Seale (1999). These simple calculations on share of the articles dealing with particular antecedent and indicating nature of variable’s influence were used where applicable, taking into account other important attributes of a research. For example stronger importance has been given to recent studies as those can build on broader previous research.
3 ANALYSIS
In analysis section, the macro view on the idea evaluation and selection framing is firstly provided as this process is a matter of many different science field and views, which have significant overlap. Secondly antecedents are organized to a simplifying conceptual model. This multidimensional view attempts to address the motivation of this study to capture the complexity of idea evaluation and selection in an understandable way (Table 1).
Table 1: Idea evaluation and selection research
Discipline Field % of Data Set Authors Framing Psychology Cognitive psychology 13%
Bink and Marsh (2000); C. Merle Crawford (1980); Herman and Reiter-Palmon (2011)
Idea evaluation as cognitive process based on cognitive regularities and traditional inventive behaviors.
Creativity 19%
Dailey and Mumford (2006); Glover, Ronning, and Reynolds (2013)
Idea evaluation as part of creativity, critical component of creative though together with ideation and problem finding skills.
Social
Psychology 21%
Amabile (1983); Stroebe, Nijstad, and Rietzschel (2010)
Idea evaluation as task influenced by personality characteristics, social factors and group interaction.
Management Problem
solving 4%
Herman and Reiter-Palmon
(2011); Mumford et al. (1991)
Idea evaluation associated with problem solving process consisting of problem identification and construction, idea generation and idea evaluation.
Decision
making 14%
Edwards (1954); Montazemi and Gupta (1997); Mousavi, Torabi,
and Tavakkoli-Moghaddam
(2013)
Idea evaluation as decision making process taking into account multiple criteria, idea attributes and contextual variables to choose among multiple ideas as alternatives for further development.
NPD 22% Buyukozkan and Feyzioglu
(2004); Chan and Ip (2010)
Idea evaluation critical part in NPD to a large extent deciding about the success of the project.
Idea
management 8%
Boeddrich (2004); Bothos,
Apostolou, and Mentzas (2009);
Buyukozkan and Feyzioglu
(2004); Flynn et al. (2003)
3.1 Theoretical perspectives
The theoretical perspectives of idea evaluation an selection are now discussed, divided in two main domains of this research – psychology and management -, providing the framing for further analysis and synthesis. The overview is summarized in Table 1, including share of the literature in data set framing idea evaluation and selection into particular field.
Extant part (about 53%) of the articles were positioned in domain of psychology as idea evaluation and selection process arguably takes place in human brain. On the most basic level of general psychology, evaluation is studied as a cognitive process based on cognitive regularities and inventive behavior (Bink & Marsh, 2000). Further, idea evaluation as a component of creative though (Blair & Mumford, 2007) has been proven to be more difficult than idea generation in psychological research (Glover, Ronning, & Reynolds, 2013). Process models then contain the idea evaluation specifically. The componential model of creative thinking includes ideation (divergent thinking), problem finding skills, and evaluative accuracy (Runco, 2004). Lastly, social psychology looks at the idea evalution from perspective of personality charateristics, social factors influencing the idea evalution (Amabile, 1983) and interpersonal interactions in group setting of idea evaluation and selection (Stroebe, Nijstad, & Rietzschel, 2010).
Management studies (about 47% of the data set) mostly framed idea evaluation and selection into problem solving, decision making, NPD process and idea management. Mumford et al. (1991) see idea evaluation as one of the core processes of problem solving consisting of problem construction, information encoding, category selection, category combination and reorganization, idea generation, idea evaluation, implementation planning, and solution monitoring supported by Herman and Reiter-Palmon (2011).
How people make choices among alternative ideas is a view of decision making literature (Edwards, 1954). This field looks at the idea evaluation and selection as a comprehensive process which has to take into account multiple criteria, idea attributes and contextual variables (Mousavi, Torabi, & Tavakkoli-Moghaddam, 2013), however the decision making may be studied on multiple levels including fractional go/kill decisions of individual criteria or idea evaluation and selection as complex process (Swink, Talluri, & Pandejpong, 2006). Idea evaluation as a critical part of NPD process (Buyukozkan & Feyzioglu, 2004) has to a large extent influence on the success of a project. Moreover, this selection decision is rarely reversed, having strong implications for further phases of NPD and company performance consequently (Chan & Ip, 2010), which adds to the importance of this process. Common practice is the extension of ideation to the whole company, which increases the scale of evaluation and selection leading to a need of idea management (Flynn et al., 2003). Extensive part of the literature is then devoted towards research on functioning models, algorithms and software for managing and accelerating idea evaluation and selection (Bothos, Apostolou, & Mentzas, 2012; Westerski, Dalamagas, and Iglesias (2013)). This field has overlap with IT disciplines proving the multidimensionality of idea evaluation and selection.
finding is then a supporting the motivation for this study, particularly in light of article on the nature of theoretical contribution to the research by Shapira (2011), revealing that some research fields only seem to have a developed theoretical body. Following, conceptual model of identified antecedents is introduced.
3.2 Idea Evaluation and Selection Antecedents
As has been shown, relevant literature on the idea evaluation and selection is in the interest across different research fields. Researchers, however, link the papers across fields and patterns can be observed. The extracted variables have been grouped under antecedents and these were organized into setting and psychology part. From a logical perspective setting of the evaluation and selection influences the psychology of the actors who are in charge of the act of evaluation and selection. On the other side, psychological processes, when known, can initiate the change of setting to obtain different outcomes. Hence the influence goes both ways as arrows in the conceptual model (Figure 1.) illustrates. This division of influencers may be also seen in light of creativity research, which recognizes personal, process and product components (Runco, 2004).
4 SYNTHESIS
After the conceptual model was provided, zooming into two main parts (setting & psychology) by synthesizing the literature on antecedents follows. Main findings for each antecedent are provided, taken further towards the level of variables (e.g. complexity). Furthermore, concluding remarks on what seem to be the main findings of the literature about the influence of particular variable on idea evaluation and selection are given.
4.1 Setting antecedents
The idea evaluation and selection is always taking place under certain setting, which influences the process itself. Influences of particular setting antecedents are now synthesized (Table 2).
Table 2: Setting Antecedents
Level /
Antecedents
Individual Team Organization Undetermined (U) /
Multilevel (M) / Other (O)
Problem Nature Mark A Runco and Dow (2004)
Kavadias and Sommer (2009)
Mohanty et al. (2005); Rochford (1991)
Buyukozkan and Feyzioglu (2004) (O) Hsu, Tzeng, and Shyu (2003)(O)
Generation & Selection
Faure (2004); Rietzschel,
Nijstad, and Stroebe
(2010)
Faure (2004); Rietzschel,
Nijstad, and Stroebe
(2010)
Buyukozkan and Feyzioglu (2004)
Girotra, Terwiesch, and Ulrich (2010) (M)
Evaluation Criteria
Kudrowitz and Wallace (2013), Dean et al. (2006)
Mousavi, Torabi, and
Tavakkoli-Moghaddam (2013)
Blair and Mumford (2007);
Montoya-Weiss and
O'Driscoll (2000); Ozer (2005))
Dean et al. (2006) (M) Amabile (1983) (M)
Evaluator E. F. Rietzschel, Nijstad, and Stroebe (2010a)
M. D. Foo (2010b) Bothos, Apostolou, and
Mentzas (2009); Diehl and Stroebe (1987); Mousavi,
Torabi, and
Tavakkoli-Moghaddam (2013)
Olshavsky and Spreng
(1996) (O)
Idea Source Rietzschel et al., (2010) Rietzschel, Nijstad, and Stroebe (2010)
Xiao (2014)
Company Culture
Buyukozkan and Feyzioglu (2004)
4.1.1 Problem Nature
The nature of the problem influences the idea evaluation and selection (Buyukozkan & Feyzioglu, 2004). Kavadias and Sommer (2009) found that the idea generation and selection teams perform better then nominal groups – individual ideas grouped for idea selection – under certain levels of complexity. In terms of the idea evaluation and selection, teams proved to be more suitable for either very simple or very complex problem structures. This finding contradicts the conclusions of earlier research suggesting that nominal groups perform better in idea generation and evaluation in general (Putman & Paulus, 2009; Rietzschel, Nijstad, & Stroebe, 2006).
lectures of psychology, or dealing with drug-dealing roomate), which are different from ideas for products in terms of problem complexity. Therefore, general conclusions on idea selection and evalution cannot be drawn from the studies with these different outcomes. Interestingly enough (Runco, 2004) found that the evaluative accuracy of realistic problems is low, whereas it tends to be higher for creative problems, which is also supported by (Rietzschel, Nijstad, & Stroebe, 2010a).
Idea evaluation depends on information availability within the company as well (Buyukozkan & Feyzioglu, 2004). The lack of information or low reliability of information is one of the main difficulties of the go/kill decision for an idea(Mohanty et al., 2005; Rochford, 1991).
Findings of recent studies emphasize the imporatance of complexity of a problem which derives also from its type – situational problem, or product idea. The complexity, in turn, determines the appropriatness of setting the evalution to a team or to a nominal group, which then affects idea evaluation and selection. Additionaly, the realisticity of the problem has a positive effect. This variable, however, is not particularly interesting in the light of reality, where unrealistic problems are hardly in the focus. Finally, the information availability on a concrete problem has an influence on idea evaluation and selection, however, it is not discovered yet under which circumstances its influence is positive or negative.
4.1.2 Idea Generation and Evaluation
Idea evaluation naturally follows the idea generation, however combining or separating these two processes influences the outcomes of both. From the idea generation point of view, separation is demanded by literature as of preventing the negative influence of expected criticism and judgment (Amabile, 1979; Diehl & Stroebe, 1987), called evaluation apprehension. People fear, their ideas will be judged and criticized in front of their peers or bosses, which then lowers their courage to submit or present more creative, potentially revolutionary ideas (Miranda & Bostrom, 1997; Nagasundaram & Bostrom, 1994). This further influences the idea evaluation and selection, as lower quality ideas will be in the consideration set. Recent studies focus on the outcome of both phases and the influence of their separation or integration. The findings suggest that the two processes are synergic, hence should be studied together (Khandwalla, 1993). Nevertheless, in nominal groups – individual generation and idea pooling for idea selection – the task separation leads to slightly better selection performance (Faure, 2004; Rietzschel, Nijstad, & Stroebe, 2006).
On the team level, the task integration increases the selection performance, which offsets the negative effect on idea quantity in idea generation phase (Faure, 2004; Rietzschel, Nijstad, & Stroebe, 2006). Hence more ideas available don’t necessarily lead to better selection outcome (Rietzschel, Nijstad, & Stroebe, 2010).
The most effective option might be a hybrid setting, in which the ideas are generated individually and evaluated in team (Girotra, Terwiesch, & Ulrich, 2010). Arguments against the team structure in favor of hybrid structure in regards of idea selection performance are limited engagement of team members, and path dependence (Girotra, Terwiesch, & Ulrich, 2010).
The research shows the interactions between idea generation and selection are strong. Setting of these processes (individual vs. team, and performed together vs. separated) is then in the end an important antecedent of the outcome of idea evaluation and selection. Team performed joint task leads to evaluation apprehension and has then a negative influence. Joint task performed by nominal group doesn’t influence significantly the idea evaluation and selection. Separated tasks on individual level seem to have positive effect and hybrid solution seem to have positive influence as well.
4.1.3 Selection criteria
Naturally, to select an idea, selection criteria have to be chosen and applied, having influence on the idea evaluation and selection. The research and practice either set some of the criteria or simply just intend to choose the “best” idea. The latter however doesn’t mean no criteria are applied, as every individual or team then assigns own criteria to complete the task of the best choice (Amabile, 1983). Research by Amabile (1983) suggests not to assess attributes of the ideas, as creativity cannot be determined objectively by using metrics and rather make use of subjective assessment of creativity. Most of the research, however, demands accurate assessment of relevant attributes needed for accurate new idea evaluation (Licuanan, Dailey, & Mumford, 2007; Scott, Lonergan, & Mumford, 2005; Ward, Patterson, & Sifonis, 2004) by making use of assessing creativity attributes (Kudrowitz & Wallace, 2013).
For proper selection, the way to rate each idea must be reliable and the ratings have to be aggregated to a score (Briggs et al., 1997). In research experiments, selectors usually rate some attributes of the idea (e.g. novelty or usefulness) on a scale, or according to a standard scoring key (e.g. agreeing what score 1, 3 or 5 means) (Kudrowitz & Wallace, 2013). MacCrimmon and Wagner (1994) have specified four main idea dimensions – novelty, workability, relevance, and specificity. These dimensions are, however, used inconsistently in literature, which makes it difficult to synthesize results of the studies (Dean et al., 2006). This raises a problem of assigning values to inappropriate selection criteria. When choosing the most creative ideas, such ideas are often described simply as novel, or in other case, as novel plus other quality attributes. The different definitions of creative ideas fall mostly into novelty-centric or quality centric definitions (Dean et al., 2006). Due to misalignment between selectors’ own definitions and the actually desired definitions, the non-clarity of the attribute definitions may hinder the evaluation performance, even if selection criteria are specifically instructed (Rietzschel, Nijstad, & Stroebe, 2010). To address this issue, Dean et al. (2006) provide unified clear definitions for dimensions with sub-dimension, allowing researches to choose one or some of them to evaluate an idea’s quality, novelty, and creativity. If adopted, this may lead to comparability improvement.
To the NPD selection process many criteria are incorporated (Chan & Ip, 2010). In general, when selecting the high quality ideas, innovative standards shall be used and when choosing highly original ideas, operative standards should be used to obtain better results (Blair & Mumford, 2007). The literature deals with many criteria that are scattered. In this study, criteria from the data set were organized into four following categories specified by Montoya-Weiss and O'Driscoll (2000). (1) Marketing criteria asses the idea from the point of end user or market needs and trends, marketing synergy, competitive advantage, potential market size and growth, competition, concept development, product superiority and uniqueness, corporate synergy (De Brentani & Droge, 1988; Montoya-Weiss & O'Driscoll, 2000; Mousavi, Torabi, & Tavakkoli-Moghaddam, 2013; Ozer, 2005; Swink, Talluri, & Pandejpong, 2006). (2) Technology criteria deal with technological feasibility, skill set and resource requirements, development strategy synergy, novelty, technology complexity and magnitude of the idea, production synergy (Montoya-Weiss & O'Driscoll, 2000; Mousavi, Torabi, & Tavakkoli-Moghaddam, 2013; Swink, Talluri, & Pandejpong, 2006). (3) Business criteria deal with opportunity, time to market, customer and strategic alignment, expected performance (De Brentani & Droge, 1988; Montoya-Weiss & O'Driscoll, 2000; Ozer, 2005) as well as market share maintenance and sunk costs (Mousavi, Torabi, & Tavakkoli-Moghaddam, 2013). (4) Human factors address resource compatibility, resource requirements, usability assessment, productivity enhancement, and interface competitiveness of the ideas. These categories are supported by other studies as strongly associated with new product success (Cooper & Kleinschmidt, 1986, 1995; Charles Merle Crawford & Di Benedetto, 2008),
Project risks (e.g. financial, managerial, envisioning, design, execution) stand as another important attribute related to the further investments in to the product development (Mousavi, Torabi, & Tavakkoli-Moghaddam, 2013).
As we can see, research criteria in the data set are either set simply as “best idea”, or as a combination of attributes for particular case. The first option is not recommended by literature as “safe” setting and tends to have rather distorting effect on idea evaluation and selection process due to unclear and inconstant influence. This inconsistency jeopardizes the desired outcome. Researchers use many criteria and there is not a unified belief which of the criteria and definitions should be used to obtain planned outcome.
4.1.4 Evaluator
The idea evaluation and selection is greatly influenced by the evaluators themselves. Whether the evaluator of an idea is a non-expert or an expert, influences the outcome of the process. Moreover, including customers or employees has a significant influence on the process outcomes.
(MacCrimmon & Wagner, 1994). There are different methodologies (Yiching & Minder, 2011), algorithms (Kempe et al., 2012) and readymade software (MacCrimmon & Wagner, 1994) present in the literature. It is out of scope of this study to review the influence and effectivity of systems assisting to the idea evaluation in companies, hence this is left to other researches.
Secondly, customers are let to evaluate the new product ideas mainly in marketing-driven companies, using outside-in approach towards innovation source (Day & Moorman, 2010). As Olshavsky and Spreng (1996) introduced, the evaluation process of customer is based on a different thinking than company innovators, as they are strongly influenced by cognitive processes. The decision making process includes evaluative criteria, forming expectations about the innovative concept, assessing satisfaction with an old product, and comparing the new and old products (Olshavsky & Spreng, 1996). Furthermore, customers may make use of “word of mouth” or imitate peers’ opinions (Rogers, 2010). Important note is however that this “voice of customer” is considered applicable in case of incremental innovation or as indicator market acceptance in the future (Kelley, 2007) as when confronted with highly innovative product, customers find it hard to create own evaluative criteria and expectations (Olshavsky & Spreng, 1996).
Thirdly, the need to include as many employees as possible into innovation processes, not only in idea generation has been emphasized (Diehl & Stroebe, 1987). As discussed, in case of NPD process, usually experts are involved, however employees in general may be involved in the initial screening phase, where they “vote” for ideas which are then sent to another rounds, incorporating employee, or partner feedback on the ideas (usually via idea management platform) might increase the idea selection performance (Bothos, Apostolou, & Mentzas, 2009).
Complexity seems to be, similarly as by problem nature antecedent (chapter 4.1.1.), the underlying influencing variable when putting experts or non-experts in charge of evaluation and selection. If problem complexity is low, non-experts seem to have positive and experts negative influence on idea evaluation and selection. On the other hand, when complexity is high, the influences tend to be opposite. Involving customers into the process seem to be an extra input into the process as of different decision making processes of customers with no certain influence on evaluation and selection outcome.
4.1.5 Idea Source
Whose ideas are the people choosing form has an effect on the selection outcome. People tend to prefer familiar ideas (their own mostly), therefore, familiar ideas might be chosen over unfamiliar, instead of desirable more original or higher quality ideas (Rietzschel, Nijstad, & Stroebe, 2010). When choosing from own ideas people tend to simplify the decision making in selecting and that could hinder the beneficial effect of setting explicit selection criteria (Rietzschel, Nijstad, & Stroebe, 2010). Further, the selection ideas which selectors are personally involved in has negative influence on the selection outcome (Rietzschel, Nijstad, & Stroebe, 2010). Earlier research by Basadur, Runco, and Vega (2000) concludes that person who is accurately evaluating own ideas will also accurately evaluate someone else’s ideas, suggesting that evaluating non familiar ideas is more accurate task.
4.1.6 Company culture
Naturally, company culture has certain influence on the idea selection (e.g. in terms of NPD) (Buyukozkan & Feyzioglu, 2004). On a macro level, the local culture influences selection criteria, same as later market acceptance (Chi-yue & Kwan, 2010). Risk aversion influences the approach towards idea selection on the company level. If the firm is risk-averse, new product ideas will be more likely also assessed with intend to minimize risk (Tyagi, 2006). Less risk-averse companies, therefore, will reach a better performance in idea evaluation under the assumption that the goal is to find the most original or novel idea.
4.2 Psychological antecedents
As there have to be always people involved in the evaluation and selection, there are many psychological antecedents influencing the outcome of this stage. The ones found in the data set are now being synthetized (Table 4).
Table 4: Psychological antecedents
4.2.1 Creativity
As evaluation and selection has been introduced as part of a creative thought, creativity has an important influence on this process.
Researches (Dörner & Schaub, 1994; Girotra, Terwiesch, & Ulrich, 2010; Rietzschel, Nijstad, & Stroebe, 2006, 2010) suggest, people are not better in selecting the best ideas than a chance. However, series of studies by Runco and colleagues (Basadur, Runco, & Vega, 2000; Runco & Smith, 1992; Runco & Vega, 1990) suggests that people do have the ability to distinguish among the attributes of the ideas and this ability is positively correlated with creative capacity. Nonetheless, people are easily influenced and biased. In one single study, there has been identified 35 different types of psychology based errors influencing one or more processes of creative thought (Mumford et al., 2006).
Level /
Antecedents
Individual Team Organizational Undetermined (U)
Other (O)
Creativity Furst, Ghisletta, and
Lubart (2012)
Perry-Smith and Coff (2011)
Herman and Reiter-Palmon (2011)
Evaluator Armor and Taylor (2003);
Campbell and Fairey
(1985); Dailey and
Mumford (2006); Runco and Basadur (1993)
Dailey and Mumford
(2006); Furst, Ghisletta, and Lubart (2012)
Stokes and Fisher
(2005)
Schwenk and Cosier (1980); Stokes and Fisher (2005); Suri and Monroe (2003)
Evaluation Criteria
Licuanan, Dailey, and
Mumford (2007),
Mumford et al. (2006)
Rietzschel, Nijstad, and Stroebe (2006)
Rogers (2010)
Appearance of idea
Blair and Mumford
(2007)
Blair and Mumford
(2007)
Westerski,
Dalamagas, & Iglesias (2013)
Boeddrich (2004) (O)
Group Influences
Blair and Mumford
(2007); Karau and
Williams (1993)
Perry-Smith and Coff (2011)
Bothos, Apostolou,
Creativity of a person is influenced by creative processes, domain specific skills and knowledge, traits, and motivational factors (Herman & Reiter-Palmon, 2011). The discussion in extant research turns around the appropriateness of convergent and divergent thinking for idea generation and evaluation. Intuitively, divergent thinking, researched mostly by Runco and his colleagues (2000; 2004; 2012; 1987) is associated with ideation as is allows to produce a wide scale of ideas and has a positive effect on choosing highly original ideas. Divergent thinking is then spelled in relation to evaluation as the careful and realistic choice needs to be performed (Furst, Ghisletta, & Lubart, 2012). Literature is, however, not unified. While some studies on the relation between divergent thinking and evaluation performance revealed significant relation (e.g. Runco & Smith, 1992), others suggest that the high ability in divergent thinking doesn’t ensure accurate evaluations of ideas (e.g. Runco & Acar, 2012).
To a certain extent, a similar classification of individual characteristics influencing idea generation and evaluation is the promotion or prevention focus of cognitive processes (Herman & Reiter-Palmon, 2011). Promotion focus is more dynamic (associated to advancement, growth, taking chance etc.) and prevention focus may be, on the other hand, characterized as a conservative focus (associated with protection, safety, responsibility etc.) (Herman & Reiter-Palmon, 2011). Intuitively, research shows that in idea generation phase, people with promotion focus are more creative and generate more original ideas then prevention focused (Lam & Chiu, 2002; Paulus, 2000; Valacich, Jung, & Looney, 2006). Researchers on other creative processes, including idea evaluation, implicated these findings. However recent study by Herman and Reiter-Palmon (2011) shows that people with promotion focus might be disadvantaged in evaluating quality as this rather optimistic focus blinds people in evaluating quality. On the team level the creativity mind-set of an individual projects into team, which has then influence on idea evaluation and selection (Perry-Smith & Coff, 2011).
Promotion focus and divergent thinking seem to agree on the positive effect on idea generation while having similar characteristics of one’s approach towards the task. This positive effect may be brought into evaluation as well, however due to different nature of the process, it is not certain. Convergent thinking and prevention focus appear to have a positive effect on idea evaluation and selection, due to need for thorough, realistic analysis of ideas.
4.2.2 Evaluator
As already synthesized in setting section (chapter 4.1.4), evaluator has an influence on idea evaluation and selection, however in this section, the personal psychological effects are captured.
suppression of the positive approach to an idea might lead to waste of effort invested by selectors and towards worse selection performance (Dailey & Mumford, 2006). Research by Furst, Ghisletta, and Lubart (2012) decomposed the influence of appropriateness of an individual for evaluation and selecting task on team level to people with different level of energy and inspiration. Authors suggest that the evaluation and selection is not beneficial for people with low energy and inspiration as they might prematurely reject ideas and might be favorable for those with high energy and inspiration to steer their thinking process.
As already discussed, in companies, experts are usually involved in idea evaluation. However, domain familiarity or expertize has been proven to be source of errors in idea evaluation as well (Finke, Ward, & Smith, 1992; Mumford et al., 2006; Sternberg & Lubart, 1996).
Another personal characteristics have also an effect on accuracy of evaluations (Dailey & Mumford, 2006). Interestingly, age has been found as one of the influencers of evaluation, when Mark Runco and Basadur (1993) found that evaluative skills tend to increase by age, which may have interesting implications for human resources management. Stress influences each individual in different way, however, the research is somewhat unified in the overall negative influence on evaluation and selection performance of (1) too many ideas to choose from, as people then choose ideas with short term benefits, rather than original ideas with long term effect (Blair & Mumford, 2007), (2) time pressure causing superficial analysis and preference for rapid closure (Stokes & Fisher, 2005; Suri & Monroe, 2003).
In conclusion, literature seems to agree that the prediction inaccuracy is negatively influencing the idea evaluation and selection performance, especially in resource and outcomes forecasting. Implementation intention has then influence which’s character consensus was not established. Literature is also not unified on what effect has the expertise of evaluators. Energized and inspirational team members may have positive influence on the evaluation and selection performance. Another personal characteristics, such as age, influence idea evaluation, for example increasing age has positive influence, however further studies have to confirm this influence. Stress caused by either size of the idea set or time pressure seem to have negative effect on idea evaluation and selection accuracy. 4.2.3 Selection Criteria
As discussed in the setting section (chapter 4.4.3.), although explicit criteria are not set up, people apply standards or criteria which are influenced by certain situational variables (Stokes & Fisher, 2005). People consider number of attributes embedded in the idea (Sternberg & Lubart, 1996). Interesting insight is, that ideas are first evaluated according to appropriateness and relevance and only then the focus is turned to originality (Runco, Okuda, & Thurston, 1987).Another research found that appropriateness is then considered according to contextual aspects of the environment as perceived fit and practical benefit (Bink & Marsh, 2000).
This might be caused by focus on operative goals or by low relevant information availability (Licuanan, Dailey, & Mumford, 2007). This bias against original ideas might be strengthened with rising complexity of the idea development (Licuanan, Dailey, & Mumford, 2007). This bias against original ideas might be however averted by explicitly instructing the selectors to choose according to specified criteria (Rietzschel, Nijstad, & Stroebe, 2010) or by better task structuring (Lonergan, Scott, & Mumford, 2004). Associated bias is then risk averseness (Gouda, 1999), which translates into idea evaluation, where people disregard risky ideas (Blair & Mumford, 2007).
Ideas consistent with prevailing social norms are preferred (Blair & Mumford, 2007), as people feel being more responsible by this choice. Selectors also prefer short term before long term benefits of the idea (Blair & Mumford, 2007), which is an argument correlating with general economic research on preference of present benefit consumption (Samuelson, 1958). Related criteria is then time consumption where people also disregard highly time consuming ideas (Blair & Mumford, 2007) and ideas demanding high implementation costs (Rogers & Adhikarya, 1979).
To conclude, research is divided on the question of ability of people to choose original ideas. However strong biases against original and risky ideas have been found, which shall be considered when dealing with idea originality. Overall conservatism of idea evaluators negatively influencing idea evaluation and selection can be observed.
4.2.4 Group Influences
The idea evaluation and selection in companies usually takes place in a team setting, which bears considerable amount of influences.
Evaluation in cross functional teams allows to increase the amount of information available and to evaluate ideas from different perspectives (Bothos, Apostolou, & Mentzas, 2012). Members’ experience has been found as a positive determinant on idea evaluation (Foo, 2010). Another researched factor influencing the team’s ability to evaluate the business ideas is the presence of a member with founding experience, which is, however, beneficial only in small teams as in larger teams this members’ domination might suppress the views of others (Foo, 2010). Social loafing (Karau & Williams, 1993) may occur in groups when individuals do not feel as accountable or identifiable to external evaluators for their performance in groups as they would if they performed as individuals (Putman & Paulus, 2009).
One contextual factor naturally influencing team or even organizational idea evaluation and selection is leadership (Shin & Zhou, 2003). More specificly, the emotional intelligence has an influence on creative processes, including idea evaluation (Zhou & George, 2003). The creative process is facilitated by leaders, hence they are in charge of moving towards the evaluation phase. At that moment the emotional intelligence of the leaders is important factor influencing idea evaluation performance of the employees, as the creativity may be activated during evaluation, modification and feedback process even more then during idea evaluation itself (Zhou & George, 2003). Emotionally intelligent leaders can revise and modify ideas in informational and encouraging manner and allow the employees to evaluate ideas without bias (Zhou & George, 2003).
lead to low quality ideas (Kaufmann & Vosburg, 1997). After the idea generation, it is important to avoid strong criticism, which would demotivate employees and protect them from being creative (Finke, Ward, & Smith, 1992). When feedback is provided in a controlling sense, leaders may destroy the employee’s intrinsic motivation to think creatively, which has also negative influence on idea evaluation (Zhou, 1998).
Feedback related and previously discussed evaluation apprehension is especially relevant in team setting. It has detrimental influence on people during the creative process, hence it is beneficial to avoid any unnecessary judgment (Furst, Ghisletta, & Lubart, 2012). Another effect of apprehension is that people who expect to be compared to the others, prefer to choose risky and more original ideas (Blair & Mumford, 2007), which is in contrast to the previous section (4.1.3).
One of the psychological determinants of the team performance is also the team mood (Baron, 2008). It has been found that a certain type of mood is fitting either idea generation or idea selection, hence the specific task should be assigned to different teams (Perry-Smith & Coff, 2011).
In conclusion, group related antecedents have an important influence on the idea evaluation and selection as often used setup in practice. The presence of expertise appears to have a positive effect on idea evaluation and selection, however caution may be dedicated to the size of the team when considering this effect. Additionally, high emotional intelligence of the leaders seems to have a positive influence the whole creative process, especially in transition between generation and evaluation. Feedback, depending on its nature, can have either negative or positive influence on idea evaluation and selection. Finally, group pressure projected to individual evaluation apprehension has negative effect on the creative performance of the team members.
4.2.5 Appearance of the idea
Once an idea has been evaluated and selected it has to be presented in some form, this form and attributes of this idea influence the decision making of the evaluator.
Ideas which are easy to understand are more likely to be chosen which distorts the selection performance (Blair & Mumford, 2007). Intuitively, it has been confirmed that inaccurate depiction and description of the idea has negative effect on idea evaluation (Westerski, Dalamagas, & Iglesias, 2013). The selection performance may be in case of product ideas improved by using sketches of the products (Kudrowitz & Wallace, 2013). However, the difference in ability of product sketching among idea originators might cause bias among selectors towards better sketches which degrades selection reliability (Licuanan, Dailey, & Mumford, 2007). On the other hand ideas, which have a detailed description, might not be selected as they are too obvious and simple (Blair & Mumford, 2007). A related problem is the non-relevant information attached to ideas, for example story, inspiration of the idea, etcetera. To better assure the complete description on an organizational level, companies are using software (Boeddrich, 2004) to unify the form and information richness of the idea to give “equal chance to all ideas”. Research then also turns the attention towards developing unified taxonomy (Dean et al., 2006; Westerski, Dalamagas, & Iglesias, 2013).
5 DISCUSSION
The synthesis of the literature on antecedents and variables present in data set has been provided in the previous chapter and each of the antecedent has been concluded with synthetizing view on the aiming of the literature. In this part, propositions on the specific influence of these variables from these conclusions are derived and framed to the present literature. These propositions are then integrated to a framework (Figure 2) following the dimension structure of general conceptual model (Figure 1).
Based on the data synthesis, this paper draws following proposals for the setting related variables previously attributed to six antecedents.
(1) Important variables connected with problem nature are the complexity and information availability. Complexity has a U-shape (in case of nominal groups) and inverted U-shape (in case of teams) influence respectively on the appropriateness of nominal or team setting, which then according to the level of complexity has a positive or negative effect on idea evaluation and selection. In other words teams are more appropriate for very simple or very complex problems and nominal groups for medium complex problems (Kavadias & Sommer,2009). The information availability can have either positive (relevant information) or negative influence (too much or non-relevant information) as literature haven’t researched this influence in depth and further empirical research is needed (Blair & Mumford, 2007).
(2) Joint or divided task of idea generation and evaluation has a significant influence on the idea evaluation and selection. When joint task (generation and evaluation) performed in team, positive influence on evaluation apprehension comes in effect and causes final negative influence on idea evaluation and selection. This influence, in case of performing joint task in nominal group is proposed to be non-significant. Separated task (generation and evaluation executed separately or by different people) performed by nominal group has a positive effect on idea evaluation. Even stronger positive effect will then have hybrid solution (individual generation and team evaluation). These propositions incorporate findings mostly by Faure (2004); Rietzschel, Nijstad, and Stroebe (2006, 2010) and Girotra, Terwiesch, and Ulrich (2010).
(3) Criteria instruction to choose the “best idea” have a negative effect on idea evaluation and selection as of its non-clarity and high subjectivity of evaluators (Girotra, Terwiesch & Ulrich, 2010). In case of choosing specific criteria, clarity has a positive effect on the usage of these criteria. The effect of different or combination of selection criteria is unclear, as it may be positive or negative according to the literature. Findings mostly by Amabile (1983) and Dean et al. (2006) are mainly followed by these propositions.
(5) In case of evaluating idea set including own ideas of the evaluator, negative effect will be in place (as of the familiarity bias). When evaluating someone else’s ideas, the effect will be positive (due to reduced familiarity bias) as the focus will be directed towards choosing the original or high quality idea (Rietzschel, Nijstad, & Stroebe, 2010).
(6) Finally, following Tyagi (2006), risk averseness of companies is proposed to have negative effect on idea evaluation and selection, especially in combination with originality and quality selection criteria. This connection of variables (risk aversion and specific selection criteria) may be another opportunity for further research.
Moving towards synthetized five psychological antecedents, following proposals on the nature of their variables are deduced.
(1) Research connects creativity to personal characteristics and cognitive processes. Divergent thinking and promotion focus may have both, positive or negative influence on the idea evaluation and selection as research is not unified. Positive influence will then have the prevention focus and convergent thinking as characteristics more fitting the nature of idea evaluation and selection. These findings are in line with research done by Herman and Reiter-Palmon (2011) and Perry-Smith and Coff (2011).
(2) Evaluator is, same as in setting part, an important antecedent. Here are however psychological influences in question. Evaluator’s prediction inaccuracy has negative influence and implementation intentions can have both either positive or negative influence (Dörner & Schaub, 1994). Energized and inspirational nature of evaluators will have positive effect on the quality of idea evaluation and selection (Armor & Taylor, 2003). Increasing age has than positive effect on the process (Runco, 2004), which may have interesting implications for human resources management. Intuitively, stress will then have distorting, negative effect on the idea evaluation and selection (Blair & Mumford, 2007; Stokes & Fisher, 2005; Suri & Monroe, 2003).
(3) Selection criteria have also important influences in the psychological dimension. The ability to choose original idea is dividing the research field (e.g. Herman & Reiter-Palmon, 2011; Blair & Mumford, 2007; Licuanan, Dailey, & Mumford, 2007; Mumford et al., 2006), so in this study either positive or negative effects are proposed as possible and resulting influence is let to further empirical research. Peoples overall conservatism towards new things (McClosky, 1958) and consequent application conservative idea criteria (e.g. consistency with prevailing social norms) is then causing negative influence on the idea evaluation and selection (Rietzschel, Nijstad, & Stroebe, 2010).
(4) Group influencers are important antecedents on the psychological dimension (Anderson & West, 1998). Presents of expertise in team will have a positive influence (Foo, 2010), same as high emotional intelligence of the leader. Feedback can have both - either negative or positive – effect, depending on its nature and timing (Kaufmann & Vosburg, 1997; Zhou & George, 2003; Finke, Ward, & Smith, 1992; Zhou, 1998). Finally the group pressure itself causes evaluation apprehension, which is a negative effect which may be studied in connection with proposal from setting dimension in regards of the very same variable (see chapter 4.1.2).
Inaccurate depiction and information will have negative influence on idea evaluation and selection (Kudrowitz & Wallace, 2013) as of decreasing the probability of ideas to be chosen.
6 CONCLUSION
The goal of this study was to identify the antecedents of idea evaluation and their influence on this process. A data set of 120 selected articles has been analyzed and synthetized to fulfill this goal.
First, the multidimensionality of the relevant literature have been shown by providing systematic overview. Scientific discipline psychology contains view of cognitive psychology, creativity and social psychology. Management then approaches idea evaluation and selection from problem solving, decision making, NPD and idea management perspective. It has been found, that theoretical body on idea evaluation and selection is rather underdeveloped as dataset contains mostly models and hypothesis without grounding theories. In light of augmented idea evaluation and selection importance, this finding represents first valuable contribution of this study.
Second, data set has been analyzed and simplifying conceptual model introduced. Two main parts of this model are setting - overarching the different contextual setting dimensions influencing idea evaluation and selection from process point of view, and psychology - including all identified antecedents impacting the people involved in the process of idea evaluation and selection.
Third, under these antecedents, literature finding have been synthesized and 32 variables identified. Finally, the nature of influence of these variables has been drawn from the literature, resulting influence proposed and incorporated into a comprehensive framework following the structure of previously introduced conceptual model.
The objective of this research was to answer the question - what are the antecedents of idea evaluation and selection. Findings indicate that these antecedents are mainly related to two dimensions - idea evaluation and selection setting and psychology. Specific identified antecedents in setting dimension are problem nature, idea generation and selection, evaluation criteria, evaluator, idea source and company culture. Psychology dimension includes creativity, evaluation criteria, evaluator, appearance of the idea and group influences antecedents. Further, 32 specific variables attributable to these antecedents’ influence idea evaluation and selection.
6.1 Implications
The finding revealing underdeveloped theoretical body of idea evaluation and selection, together with importance of idea evaluation and selection augmented by the literature (see chapter 1.3.), call for further research of idea evaluation and selection directed towards building comprehensive theory. Provided framework fills the literature gap of missing systematic overview of idea evaluation and selection antecedents and serves as a platform to be used for developing hypothesis based on given propositions, choosing antecedents to study while “seeing the bigger picture”, and most importantly include interactions among antecedents and variables into the research. This last point on interactions between extracted variables has some importance as individual studies usually don’t incorporate influence of more antecedents, which can be however constructed on the basis of this study.
the provided framework to incorporate influences relevant in their context into management of the NPD, as this is still not reflected by the practice, causing ineffective outcomes (see chapter 1.3).
6.2 Limitations and Future Research
Several limitations of this research are now being addressed, including those related to the qualitative nature of this study. Even though pre-determined selection process was employed to limit subjectivity and increase reliability, personal bias in the phase of paper selection based on abstract screening could not been entirely removed. Another limitation stems from the propositions drawn from simplified synthesis of the selected articles approaching grouped articles from a homogenous perspective, although studies differ in context, scope or date of research. Lastly the causality of specific influences, which is illustrated in the conceptual model, was not researched profoundly, as this research is based on findings of other studies (which mostly do not reflect on causality) and this depth of the research was out of scope of a master thesis.
REFERENCES
Amabile, T. M. (1979). Effects of external evaluation on artistic creativity. Journal of personality and social psychology, 37(2), 221.
Amabile, T. M. (1983). The social psychology of creativity: A componential conceptualization. Journal of personality and social psychology, 45(2), 357.
Anderson, N. R., & West, M. A. (1998). Measuring climate for work group innovation: development and validation of the team climate inventory. Journal of organizational behavior, 19(3), 235-258.
Armor, D. A., & Taylor, S. E. (2003). The effects of mindset on behavior: Self-regulation in deliberative and implemental frames of mind. Personality and Social Psychology Bulletin, 29(1), 86-95.
Barczak, G., Griffin, A., & Kahn, K. B. (2009). PERSPECTIVE: Trends and Drivers of Success in NPD Practices: Results of the 2003 PDMA Best Practices Study. Journal of Product Innovation Management, 26(1), 3-23.
Baron, R. A. (2008). The role of affect in the entrepreneurial process. Academy of management review, 33(2), 328-340.
Basadur, M., Runco, M. A., & VEGAxy, L. (2000). Understanding how creative thinking skills, attitudes and behaviors work together: A causal process model. The Journal of Creative Behavior, 34(2), 77-100.
Bink, M. L., & Marsh, R. L. (2000). Cognitive regularities in creative activity. Review of General Psychology, 4(1), 59.
Blair, C. S., & Mumford, M. D. (2007). Errors in idea evaluation: Preference for the unoriginal? Journal of Creative Behavior, 41(3), 197-222.
Boeddrich, H.-J. (2004). Ideas in the Workplace: A New Approach Towards Organizing the Fuzzy Front End of the Innovation Process. Creativity & Innovation Management, 13(4), 274-285.
Booz, Allen, & Hamilton. (1982). New products management for the 1980s: Booz, Allen & Hamilton. Bothos, E., Apostolou, D., & Mentzas, G. (2009). Collective intelligence for idea management with Internet-based information aggregation markets. Internet Research, 19(1), 26-41.
Bothos, E., Apostolou, D., & Mentzas, G. (2012). Collective intelligence with web-based information aggregation markets: The role of market facilitation in idea management. Expert Systems with Applications, 39(1), 1333-1345. doi: 10.1016/j.eswa.2011.08.014
Briggs, R. O., Reinig, B. A., Shepherd, M. M., Yen, J., & Nunameker, J. (1997). Quality as a function of quantity in electronic brainstorming. Paper presented at the System Sciences, 1997, Proceedings of the Thirtieth Hawaii International Conference on.
Buehler, R., Griffin, D., & Ross, M. (1994). Exploring the" planning fallacy": Why people underestimate their task completion times. Journal of personality and social psychology, 67(3), 366.
Campbell, D. T. (1960). Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological review, 67(6), 380.
Campbell, J. D., & Fairey, P. (1985). Effects of self-esteem, hypothetical explanations, and verbalization of expectancies on future performance. Journal of personality and social psychology, 48(5), 1097. Chan, S. L., & Ip, W. H. (2010). A Scorecard-Markov model for new product screening decisions. Industrial Management & Data Systems, 110(7), 971-992.
Chi-yue, C., & Kwan, L. Y. Y. (2010). Culture and Creativity: A Process Model C-Y. Chiu and L. Y-Y. Kwan Culture and Creativity. Management & Organization Review (Wiley-Blackwell), 6(3), 447-461. Cooper, R. G., & Kleinschmidt, E. J. (1986). An investigation into the new product process: steps, deficiencies, and impact. Journal of Product Innovation Management, 3(2), 71-85.
Cooper, R. G., & Kleinschmidt, E. J. (1995). Benchmarking the firm's critical success factors in new product development. Journal of Product Innovation Management, 12(5), 374-391.
Crawford, C. M. (1980). THE IDEA EVALUATION FUNCTION IN SMALLER FIRMS. Journal of Small Business Management, 18(2), 31-40.
Crawford, C. M., & Di Benedetto, C. A. (2008). New products management: Tata McGraw-Hill Education.
Crossan, M. M., & Apaydin, M. (2010). A multi‐dimensional framework of organizational innovation: A systematic review of the literature. Journal of Management Studies, 47(6), 1154-1191.
Dailey, L., & Mumford, M. D. (2006). Evaluative aspects of creative thought: Errors in appraising the implications of new ideas. Creativity Research Journal, 18(3), 367-384.
Day, G., & Moorman, C. (2010). Strategy from the outside in: Profiting from customer value: McGraw Hill Professional.
De Brentani, U., & Droge, C. (1988). Determinants of the new product screening decision A structural model analysis. International Journal of Research in Marketing, 5(2), 91-106.
Dean, D. L., Hender, J. M., Rodgers, T. L., & Santanen, E. L. (2006). Identifying quality, novel, and creative ideas: Constructs and scales for idea evaluation. Journal of the Association for Information Systems, 7(10), 646-698.
Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of personality and social psychology, 53(3), 497.
Dörner, D., & Schaub, H. (1994). Errors in planning and decision‐making and the nature of human information processing. Applied psychology, 43(4), 433-453.
Edwards, W. (1954). The theory of decision making. Psychological bulletin, 51(4), 380.
Ekvall, G. (1996). Organizational climate for creativity and innovation. European journal of work and organizational psychology, 5(1), 105-123.
Faure, C. (2004). Beyond brainstorming: Effects of different group procedures on selection of ideas and satisfaction with the process. The Journal of Creative Behavior, 38(1), 13-34.
