Strategy testing as an interplay between strategy formulation and implementation?

(1)

University of Twente

School of Management and Governance Track: Innovation & Entrepreneurship

Business Administration

Thesis paper

Master Thesis BA - 194100040 2015-2016

Strategy Testing as an Interplay between Strategy Formulation and Implementation?

Keywords: strategy testing, strategy validation, ex ante strategy evaluation, strategic management, start-ups

Submitted by: Ven, Dennis van der s1533630

1

^st

supervisor: Dr. ir J. Kraaijenbrink 2

^nd

supervisor: Dr. K. Zalewska-Kurek

Contact e-Mail: d.vanderven@student.utwente.nl

Enschede, August 26

^th

2016

(2)

2

Acknowledgements

“With your mind power, your determination, your instinct, and the experience as well, you can fly very high”

Ayrton Senna da Silva (1960 – 1994)

This thesis paper marks the end of my study in Business Administration at the University of Twente.

Realizing that education should not be taken for granted, I will always cherish the moments along the way, both, the good ones, and the difficult ones. Although my journey at the University has come to its final end, the learning process will not, it will never end. I would like to take this opportunity to thank the people that played a part in the final phase of my educational journey.

First of all, I would like to thank Dr. ir. J. Kraaijenbrink for supervising my research study. I’m grateful for his critical assessment and provided feedback, which guided me through my research process.

Secondly, I would like to thank Dr. K. Zalewska-Kurek for her assessment and advice as this research study’s secondary supervisor. In addition, I sincerely would like to thank all the interviewees who have been participating in this research study.

Finally, my special thanks goes out to my parents and brother, for their unconditional support. Even though there were some difficult times, I always gained great energy from you which kept me going.

Rotterdam, August 22^th2016 Dennis van der Ven

(3)

3

Abstract

The evolution of the competitive playground in which organizations are operating in has resulted into more complexity, shorter product life-cycles, and markets that appear and disappear at an alarming rate. Several studies have demonstrated pessimistic prospects about the chances of survival for start- ups. For this paper, the premise is made that (successfully) executing on a plan that does not work is at the heart of strategic failures. The study elaborates on the thought that strategy testing serves the goal of making better strategic decisions based on proven strategic initiatives, which ultimately enhance business success. An elaborated literature review identified five approaches which can be used for the purpose of strategy testing, i.e. checklist assessment, Pre-mortem analysis, interactive simulations, Scrum, and the Lean Startup methodology. However, academic research is limited on the actual application of these approaches in practical settings. The aim of this paper was to identify whether organizations test their strategies in advance of (full) implementation, how they perform this activity, and what challenges and benefits they experience.

Due the exploratory nature of this study, a qualitative research approach has been chosen. The data was collected from eight early stage IT-ventures by means of semi-structured interviews with executives. The audio-recorded interviews where translated into transcripts, analysed, and processed by usages of coding technique.

The findings of this study demonstrate that the majority of the case ventures aren’t systematically testing their strategies. In fact, most of the ventures pay limited attention to their strategy development process. Only one case venture applies the Lean Start-up methodology (LSM) in accordance to the process identified in the literature review of this paper, whereas the other cases are merely using the element of a minimum viable product. The Scrum framework is used partly or loosely for product development purposes by five case ventures and only applied strictly by one venture. In addition, none of the other three validation approaches are used for strategy validation purposes, or used at all. The reasons for not applying checklist assessments, Pre-mortem analyses, and interactive simulations are because of: a lack of a formalized strategy, unfamiliarity with approaches, and a low perceived rivalry within the business environment. With exception of two ventures, all cases initially have put a premium on product aspects, rather than validating their customer’s real problem in advance. Moreover, the majority of the case ventures demonstrated a poor application of systematic measurements. Finally, the case analyses identified the following key challenges: product-market fit, confusion about the concept of a MVP, managing the workload, and finding the right pricing model.

There are three main points of attention which are derived from this study’s findings: 1) applying elements of an approach is not the same as applying the approach, 2) strategy is more than a product, and 3) meaningful testing requires measurements. Firstly, as identified in the case analyses, the majority of the case ventures are applying certain elements of Scrum and the LSM. However, by using loose elements of these approaches the essence becomes (partially) obsolete since of the elements of these frameworks are interconnected with other. Secondly, the research findings identified a predominant emphasis on the product offering by the case ventures. However, other elements of strategy should be taken into consideration as well. Thirdly, strategy testing involves measuring. This implies that measurement instruments need to be in place to make testing meaningful.

Furthermore, this research paper can assist managers and executives to gain insights in strategy testing methods. In addition, strategy testing prior to fully implementation might provide new insights in factors the organization should take into consideration and makes problems visible, which make strategy makers redefine their strategy into a more feasible one. Moreover, this study invites academics to position strategy testing as an intermediating element between strategy formulation and implementation, rather than performing strategy evaluation as a final phase in the strategic management process.

Future scholars are encouraged to elaborate on though of strategy testing prior to (full) implementation and making additional contributes towards a general framework for strategy testing.

(4)

4

List of abbreviations

BW Business Wargame BM Business Model

CD Customer Development PD Product Development e.g. exempli gratia—for example i.e. id est—that is

LSM Lean Startup Methodology MLP Minimum Lovable Product MMP Minimum Marketable Product MVP Minimum viable product TPM Toyota Production System

(5)

5

List of tables

Table 1: Checklists for strategy testing………..…………15-16

Table 2: Advantages and disadvantages of checklist assessments……….………17

Table 3: Advantages and disadvantages of pre-mortem analyses……….….…19

Table 4: Advantages and disadvantages of BWs……….…24

Table 5: Advantages and disadvantages of Scrum……….28

Table 6: Advantages and disadvantages of LSM………..33

Table 7: Comparison of key advantages and disadvantages………..34

Table 8: Overview key characteristics of the five strategy testing approaches………..35

Table 9: Data collection sample.………...39

Table 10: Key findings case venture one……….…43

Table 11: Key findings case venture two……….46

Table 12: Key findings case venture three……….49

Table 13: Key findings case venture four……….………51

Table 14: Key findings case venture five………..54

Table 15: Key findings case venture six……….…………57

Table 16: Key findings case venture seven……….……59

Table 17: Key findings case venture eight………..…62

Table 18: Opportunity Identification……….…63

Table 19: Case comparison strategic approaches……….…64

Table 20: Strategy formulation………..…64

Table 21: Case comparison validation approaches………..65

Table 22a: Strategy Validation………..……65

Table 22b: Strategy Validation………..…66

Table 23: Non-applies validation approaches……….………67

Table 24: Case comparison measurement areas………68

Table 25: Case comparison measurement approaches………68

Table 26: Measurement areas and approaches……….………69-70 Table 27: Experienced benefits………..………70-71 Table 28: Case comparison of strategy validation challenges………72

Table 29: Experienced challenges with Product-Market fit………72

Table 30: Experienced challenges Product-Market fit………73

Table 31: Experienced challenges Minimum Viable Product…….………73

Table 32: Experienced challenges with pricing model………74

Table 33: Experienced challenges with managing workload………..………..74

Table 34: Taken & planned actions……….………75

Table 35: Interview Guide……….……90-92

List of figures

Figure 1: coding interview transcripts………..………39

figure 2: categorizing codes……….………39

(6)

6

1. Introduction

1.1 Situation & Complication

Understanding how and why some firms outperform others is at the heart of strategic management.

A compact definition of strategic management is stated as “the analyses, decisions, and actions an organization undertakes in order to create and sustain competitive advantages” (Dess et al., 2012, p.

48). According to Dess et al. (2012), there are three interdependent basic elements in the strategic management process of decisions and activities which lead to the development and practice of strategies: strategy analysis, strategy formulation and strategy implementation. Wheelen and Huger (2012) added a fourth element of evaluation & control within the domain of strategic management.

However, there are three flaws in the strategic management domain that has triggered to address the element of ‘strategy testing’.

Imbalance of academic attention to strategic management elements

To date, there is definitely no shortage of (scientific) literature within the field of strategic management, and strategy in general. Strategy formulation concerns “decisions made by firms regarding investments, commitments, and other aspects of operations that create and sustain competitive advantage” (Dess et al., 2012, p. 54). The formulation of strategy is often viewed from the mechanistic perspective as a rational plan; i.e. “a posture between mutually supporting organizational elements, such as activities and organizational structure, and environmental elements, such as customer groups” (Farjoun, 2002, 563). In their work, De Wit and Meyer (2010) describe the formulation process as: 1) identification of opportunities and risk; 2) determining the company’s material, technical, financial and managerial resources; 3) personal values and aspirations of senior management to decide what to do; and 4) acknowledgement of non-economic responsibility to society. Whereas strategy formulation deals with the development of long-range plans, strategy implementation actually puts the strategy into action (Wheelen and Hunger, 2012). A concise definition of strategy implementation states that it is “the communication, interpretation, adoption, and enactment of strategic plans” (Noble, 1999, p. 120). However, strategy implementation has attracted notable less attention than strategy formulation (Aaltonen and Ikävalko, 2002; Alexander, 1985; Al-Ghamdi, 1998; Raes et al., 2011). Some authors suggest that this imbalance between the aforementioned two elements is because of: formulating or designing a strategy is according to Alexander (1991) more of a glamorous top management activity (Aaltonen and Ikävalko, 2002), some strategists assume that strategy formulation is the only necessary element for strategic success and implementation is just a relatively straightforward operationalization of a clearly articulated strategic plan (Noble, 1999), strategy implementation is “often seen as something of a craft, rather than a science” (Blahová and Knápková, 2011, p. 61), strategy implementation is more complex (Siciliano, 2002), and MBA programs are more focused on strategy formulation and planning (Hrebiniak, 2006).

Moreover, a keyword analysis indicated that substantially less academic attention has been given to the element of strategy testing.

Strategic failures in practice

Various authors ascribe strategic failure in practice to two main streams of causes. Firstly, different authors claim that it is not bad strategy formulation, but rather bad strategy implementation, that can be allocated to strategic failures–the formulation-to-implementation gap–and/or poor performance–

strategy-to-performance gap (Allio, 2005; Crittenden and Crittenden, 2008; Mankins and Steele, 2005;

Miller, 2001; Speculand, 2009). Elaborating on this thought, strategy implementation failure rates are reported up to 70 percent (Beer, 2000; Miller, 2001; Nutt, 1999; The Economist, 2013). However, it is worth noting that strategy implementation failure rates reported by a variety of authors are often based on evidence that is “outdated, fragmentary, lacks scientific rigour or is just absent” (Cândido and Santos, 2015, p. 254). For this reason, Cândido and Santos (2015) concluded that the actual rate

(9)

9 of strategy failure might be difficult to determine. Secondly, various authors were triggered by assumingly high failure rates of start-ups. Shane (2008) illustrates the survival of US start-ups originated in 1992 with a survival rate of respectively 45 percent after five years and 30 percent after ten years. He further states that “[…] no matter which developed country you look at, it appears that only half of new firms started remain in business for five years, and less than one-third last ten years”

(Shane, 2008, p. 98). This statement is supported by Bangma and Snel’s (2009) study about the survival rate of Dutch start-ups in the period from 1987 till 2008, i.e. ten percent was out of business after one year, and about half of the start-ups was out of business after five years after their foundation. Feinleib (2012) is even more pessimistic about start-ups’ survival rates by claiming that about eight of ten new businesses fail within three years after their foundation. Although these numbers should be interpreted with caution as Shikhar Ghosh—a senior lecturer at Harvard Business School—mentions,

“If failure means liquidating all assets, with investors losing all their money, an estimated 30% to 40%

of high potential U.S. start-ups fail […], if failure is defined as failing to see the projected return on investment—say, a specific revenue growth rate or date to break even on cash flow—then more than 95% of start-ups fail” (Gage, 2012). Instead of ascribing start-ups failures as a poor execution of intended plans, some authors like Blank (2012), Ries (2011), and Maurya (2012) imply that it’s rather the (business) plan itself that could be ascribed as the root cause of these failures. These authors addressing this dilemma by elaborating on the thought of strategy validation, a terminology this paper intertwines with strategy testing.

New business environments

The practice of doing business during the previous century was predominantly influenced by streams of practitioners who embraced the philosophy of Fordism and Taylorism with a focus on rationalisation, standardization and elimination of uncertainty (McCarthy & Tsinopoulos, 2003).

However, the growing intensity and consequent increase in complexity of competition during the 1980s and 1990s within business environments had profound implications for the evolution of strategic management and the way organizations are managing their businesses (Grant, 1996). The

‘rules of the game’ for todays markets require anticipation on different dimensions, which have demonstrated to change frequently (Bessant et al., 2002). Current global competitive business environments are characterised by demand for greater product variety (Bessant et al., 2002; McCarthy

& Tsinopoulos, 2003), changes in demand (Eisenhardt, 1989), changes in technology and competition (Bourgeois & Eisenhardt, 1988; Silverblatt & Korgaonkar, 1987), shorter product life-cycles (Bassant et al., 2002), unstable economic conditions (Silverblatt & Korgaonkar, 1987), and shortening shelf-life of business opportunities (Trimi & Berbegal-Mirabent, 2012). This evolution of the competitive playground has resulted into a higher degree of complex, dynamic and unpredictable business environments (Acur & Englyst, 2006), thus making succeeding and survival within these business environments more difficult (Agarwal et al., 2006; Silverblatt & Korgaonkar, 1987). In fact, these evolvements result in markets to appear, mutate and disappear at an alarming rate (Goldman et al., 1994). As stated by Acur and Englyst (2006), “for most organizations, the dynamic process of adjusting to environmental change and uncertainty—of maintaining an effective alignment with the environment while managing internal interdependencies—is enormously complex […]” (p. 547), but nevertheless should be projected into the organization’s strategy. Trimi and Berbegal-Mirabent (2012) argue that organizations within technology-intensive industries might be confronted with even higher degrees of uncertainty, since new know-how technologies are often characterised by volatility and unpredictability themselves. The degree of volatility marks the change and predictability within an organization’s business environment (Judge & Miller, 1991) and results in information that “is often inaccurate, unavailable, or obsolete” (Bourgeois & Eisenhardt, 1988, p. 816). Indeed, Eisenhardt (1989) stresses that the disruptions in technology enforces rapid decision-making. In her study, she concluded that fast decision-makers used more information, considered more alternatives, and made centralized decisions supported by counsellors. One might argue that organizations confronted with volatile business context, and a higher degree of uncertainty, are putting a premium on strategy testing prior to fully implementation.

(10)

10

1.2 Theoretical background

The three above-mentioned issues, both in the academic base of knowledge and practical settings, formed the trigger for this study to emphasis on the thought of strategy testing—validation. In the strategic management process, evaluation and control is placed after strategy formulation and implementation (Wheelen and Huger, 2012). This after-the-fact rationalization, strategic hindsight, aims to enhance organizational learning by seeking for explanations from both failure and success (McKenna, 2011; Mitchell et al., 1989). However, this implies that an event first has to occur—or partially occur—what might obsolete the process of evaluation when an organization has faced a catastrophic downturn. Different approaches have already been proposed to evaluate intended strategic plans, to act upon the learnings from those activities, and thereby increasing the likelihood of strategic success by bringing forward the evaluation phase, rather than doing this activity as a final step in the strategic management process. Klein (2007) for example, introduced a technique that is based on the principle of prospective hindsight, i.e. explaining future events as past (Mitchell et al., 1989). Kraaijenbrink (2015) has included strategy assessment into his strategy generation process and reasons that checklists can be used for both, assessing an organization’s current strategy, or a newly generated strategy before execution has started. As argued by Schwarz (2011), “managers need to prepare for change and look for new constellations for better ways to reallocate their resources and to position their companies in the market” (Schwarz, 2011, p.122). In order to deal with this challenge, managers must develop sufficient ex ante strategy evaluation processes to avoid corporate failure, a domain which appears to be underrepresented (Schwarz, 2011). The overall goal of ex ante evaluation is to assess the consequences of a particular strategic decision in respect to the organizational environment and future impact of the own strategy and those of competitors, prior to implementation (Schwarz, 2011). A relatively young approach to deal with the practical problematics which were mentioned in previous paragraph is popularized by authors such as Blank (2013a), Ries (2011) and Maurya, 2012), who call for validation of a business idea in the earliest stage of the organization. These authors propose a hypothesis-driven approach as an antidote for the lethal problem of successfully executing a plan that leads nowhere, by seeking for a business model that works (Ries, 2011; Maurya, 2012). Although their different characteristics, focus areas, and strengths and weaknesses, all these approaches share the same purpose, i.e. strategy testing—validation. However, academic research is limited on the actual application of these approaches in practical settings. As a result, limited insights are available about which approaches are favoured and what challenges and benefits are experienced by applying them in practice.

1.3 Research aim; central research question and methodological approach

This study elaborates on the thought by authors like Blank (2012), Ries (2011), and Maurya (2012) who imply that it’s the (business) plan itself that could be ascribed as the root cause of strategic failures.

For this reason, this paper addresses the subdomain of strategy testing and will elaborate on the thought that testing a strategy before (full) implementation might contribute to avoid disappointing results from strategic decisions afterwards. The aim of this paper is to identify whether organizations test their strategies in advance of (full) implementation, how they perform this activity, and what challenges and benefits they experience. Derived from the aforementioned research aim, the following central research question is formulated.

Which approach can early stage IT-ventures use to test their strategies prior to implementation?

In order to understand the underlying motivation of organizations’ testing approaches, the following sub questions are formulated:

 Which strategy testing approaches are documented within the academic literature?

 Which strategy testing approach(es) are used by early stage IT-business ventures?

 Why do organizations adapt certain strategy testing approaches in favour of others?

 What are the perceived challenges and benefits from their approaches?

(11)

11 In this research study its context, testing is defined as “the process of executing a program with the intent of finding errors” (Myers et al., 2011, p. 11). However, as mentioned earlier, strategy testing and validation are used intertwined in this paper, where validation implies “the process of evaluating the logical arguments and scientific evidence that support claims” (Taylor, 2013, p. 2). The reason for considering both of these concepts is to broaden it applicability and arguably, because they complement each other well for the purpose of this study.

Due the explorative nature of this research, an inductive research strategy has been chosen. Since the underrepresentation of existing theory on strategy testing approaches—in contrast to strategy formulation and implementation—and the ability to identify the units of analysis, a multiple-case study design will form the foundation of this study. A multiple-case study design is chosen above a cross- sectional design, because the focus is on the unique context of the cases–IT-business–, thereby making a multiple-case study design more appropriate (Bryman and Bell, 2011). As units of analysis, organizations were selected on two criteria. Firstly, this study selected start-ups or young venture in their formation or early growth stage. Secondly, the organizations should had a comparable business context. For this study, organizations operating in the e-business industry were selected because of the expectation, based on literature study, that these ventures are able to conduct strategy testing activities more easily, thereby increasing the likelihood to gather more relevant data and insights for this study. Also, the expectation was that these ventures where more easily to identify and approach.

The data has been gathered through semi-structured face-to-face interviews with (senior) executives or founders. Although, an interview guide was used as a broad guideline, the interviewees still had enough leeway in how to reply to the questions (Brymann & Bell, 2011). More specific, the laddering technique during interviews with their repeating ’’why’’ questions on prior answer were performed to grasp the underlying logics and motives behind answers. Subsequently, the interviews were audio- recorded and translated into interview transcripts. Finally, the coding technique has been used for translating the data into interpretative research results.

1.4 Academic and practical contribution

This research contributes to the base of knowledge within the strategic management domain for the following reasons. Firstly, it aimed to provide exploratory insights in the way organizations approach strategy testing, a subdomain which is currently not addressed in in the same extent as strategy formulation and strategy implementation within the strategic management literature. Secondly, based on the research findings, it targets to set an additional step towards the develop of a conceptual framework, i.e. theory building. The generalizability of the study is presumably low, however this was not a pre-condition since the exploratory nature of the study. Thirdly, positioning strategy testing as an interplay between formulation and implementation may contribute to bridge the formulation-to- implementation gap. Finally, this research aimed to gather insights for other academics to (empirically) elaborate on. In addition, this research has practical relevance as well. Firstly, strategy testing prior to fully implementation might provide new insights in factors the organization should take into consideration and makes problems visible, which makes strategy decision-makers redefine their strategy into a more feasible one. Secondly, strategy testing ought to contribute to more effective strategy implementation, which in turn, can be a competitive advantage itself. And last, strategy testing brings together strategist and implementers in an early stage of the strategic management process.

1.5 Thesis outline

The next section of this paper, the literature review, will discuss theory concerning strategy testing methods. The up following section will present the methodological approach which was adapted for this research. Subsequently, the results of the interview data collection has been translated into interpretative results. The conclusion section will link this papers’ overall findings back to the predetermined research question. Finally, a discussion on the contribution, limitations, and suggestions for future research will be discussed.

(12)

12

2. Literature Review

2.1 Increasing the likelihood of strategic success by strategy testing

2.1.1 The cumbersome of estimating strategic success

Making the correct strategic decisions remain a key challenge for executives and often don’t turn out in a satisfying way. Authors like Allio (2005), Beer (2000), and Miller (2001) for example, attribute strategic failures to poor implementation, rather than the formulated strategy itself. In contrast, Blank (2013a), Blank & Dorf (2012), Ries (2011), and Maurya (2012) ascribe strategic failure to bad business plans instead. This ambiguity about what causes strategies to fail in practice forms a prevalent discussion within the research field of strategic management (Childress, 2012). The dilemma for many executives is that to find out whether a strategy works, it first needs to be implemented (Childress, 2012). This after-the-fact rationalization, strategic hindsight, aims to enhance organizational learning by seeking for explanations from both failure and success (McKenna, 2011; Mitchell et al., 1989), and forms a subsequent phase in the strategic management process after strategy implementation (Wheelen & Hunger, 2012). However, rather than discovering whether a strategy will succeed after being implemented, this study positions this activity prior to the (full) implementation stage, i.e.

strategy testing—validation.

“The logic of validation allows us to move between the two limits of dogmatism and scepticism”—

Ricoeur (1991)

According to Githens (2011), having the right strategy is perceived as the key success factor for strategic initiatives by CEOs. This author argues that a valid strategy must meet the following two criteria: 1) a distinct problem must be matched with an effective solution, and 2) the organization must have the necessary resources and organizational structure in place to act upon this solution.

Nowadays, organizations are confronted by high velocity business environments resulting in markets to appear, mutate and disappear at an alarming rate (Goldman et al., 1994), thus shortening the shelf- life of business opportunities (Trimi & Berbegal-Mirabent, 2012). These business evolvements are argued by this research study to put a premium on strategy validation before (full) implementation and commitment. As pointed out by Ricoeur (1991), “the logic of validation allows us to move between the two limits of dogmatism and scepticism. It is always possible to argue for or against an interpretation, to confront interpretations, to arbitrate between them, and to seek for an agreement […]” (p. 160). This study elaborates on the thought that strategy testing—validation—serves the goal of making better strategic decisions based on proven strategic initiatives, which ultimately enhanced business success.

2.1.2 Definition

Three key terms were identified that cover the focal unit of analysis of this study, i.e. (strategy) testing, validation, and ex ante strategy evaluation. Testing is defined as “the process of executing a program with the intent of finding errors” (Myers et al., 2011, p. 11). In addition, Taylor (2013) defines validation as “the process of evaluating the logical arguments and scientific evidence that support claims” (p. 2).

And thirdly, an ex ante strategy evaluation is defined by Schwarz (2011) as “testing strategies prior to their implementation” (p. 122). For the purpose of this research study, two terminologies are used intertwined, strategy testing and strategy validation. For the remaining of this research paper, when these terms are used, it refers to the following conceptualization:

“The process of finding errors and support for an intended strategy, due evaluation of logical arguments and scientific evidence, prior to (full) implementation”

(13)

13 2.2.3 Literature review process

For the purpose of answering the first sub-question, identifying strategy testing approaches within the academic literature, and building a sound foundation for the operationalization of this study’s empirical research, the following process has been followed for the literature review.

Orientation phase

In this first phase of this study, an orientation has been done into the topic of interest, i.e. strategy testing. For this purpose, various books and online data-bases were consulted such as Google, Google Scholar, Scopus, University of Twente, and sEURch—library of Erasmus University of Rotterdam. Firstly, identification of the position of strategy testing within the strategic management literature was done.

Secondly, the online data-bases were used for identifying key-words. This initially resulted into the following key-words: strategy testing, strategy validation, ex ante strategy validation, and stress- testing. Secondly, these key-words were used for key-word analyses and refined the searching criteria for the online data-bases. Since the inclusion of strategy resulted in an overwhelming amount of results, two actions were taken. The first action was a refinement of the search filters by searching on the exact phrases. This reduced the amount of searching results substantially. The second refinement was made by searching on title. As a result of these refinements, the amount of search results had significantly reduced. This allowed for analyses of the abstracts in order to identify relevant papers. As a result, additional key-words were identified, i.e. Business Wargames, Lean Startup, Agile, and Scrum.

The process of analysing the abstracts was repeated for these additional key-words.

Collection phase

The relevant papers which were identified from the key-word and abstract analyses, where collected, categorized on topic, placed in a literature chart, and ranked on their amount of references by other researchers. This reference-analysis was conducted by usage of Google Scholar.

Reviewing phase

Unfortunately, no review paper on strategy testing was found during the orientation and collection phase. For this reason, the literature chart served as a guide for the actual literature review. To guide this process, the following criteria were set. Firstly, the order of reading the papers was based on their ranks in the literature chart. Secondly, papers from the year 2000 or later were favoured. Thirdly, journal paper were selected. The papers were reviewed and short summaries were made. Moreover, those papers which were assessed as valuable were used as sources of references to other papers by using their reference lists. By means of this ‘snowball effect’, the base of literature expanded and eventually led to an additional strategy testing approach, i.e. Pre-mortem analysis.

Processing phase

As a result of the aforementioned literature review process, the results were translated into the literature review which will be discussed in the next section of this research paper, i.e. section 2.3—

strategy testing approaches.

(14)

14

2.3 Strategy testing approaches

The following sections provide an in-depth discussion about the five strategy testing approaches which were selected for this research study; 1) checklist assessment, 2) Pre-mortem analysis, 3) interactive simulations, 4) Scrum, and 5) hypothesis-driven experimentation.

2.3.1 Checklist assessment

Strategists are confronted with complex business dilemmas created by their unique business environment. For this reason, Simons (2010a) expresses his scepticism about one-size-fits-all solutions by arguing that the only approach to add value to all businesses is by asking the right questions. By probing tough questions, strategists can unveil ambiguity, inefficiencies or shortcomings in their strategies (Simons, 2010b). The usage of checklists can guide strategic decision-makers in a more systematic way through sets of questions categorized for different elements of a strategy.

Kraaijenbrink (2015) for example, has included strategy assessment due checklists into his strategy generation process for “judging and testing the quality of the organization’s strategy against relevant criteria” (p. 113). He reasons that checklists can be used for assessing an organization’s current strategies as well as newly generated strategies before execution has started. By means of assessing whether a strategy holds itself again tough self-assessment questions, an activity also referred to as

‘stress-testing’, strategic decision-makers can test the strategy’s quality early on (Bradley et al., 2011).

For this reason, checklist assessment has been included within this research study with the premise of testing whether a strategy holds against (basic) criteria prior to its implementation. For the purpose of this research paper, strategy checklist assessment is defined as “subjecting an intended or current strategy to a subset of questions which are incorporated into a checklist to systematically assess the strategy’s quality against relevant criteria”. The next paragraph will discuss criteria which can be incorporated into a checklist assessment for strategy testing.

Checklists

The four checklists which will be discussed in this paragraph all serve the purpose of strategy testing, although their focus areas differ. Bradley et al. (2011) have composed a list of ten general tests which address various topics that were acknowledge by senior executives as being part of the strategic dialogue within their organizations. These ‘pressure tests’ focus on the strategy as a whole and generate deeper strategic dialogues, which helps to unveil where the strategy needs additional work.

The second checklist assessment is provided by Simons (2010a,b), who has translated a stress-test into seven questions which can be used to assess whether the strategy is ready for implementation. These questions focus on determination of strict priorities, designating key performance indicators (KPIs), techniques to enhance creative tension and commitment, and the ability to adapt the strategy overtime (Simons, 2010a,b). Next, Kraaijenbrink (2015) offers a more comprehensive list of nine strategy checks, each addressing specific criteria to assess a strategy on. Six checks contain subsets of questions which are sub-categorized for all of the ten elements of Kraaijenbrink’s (2015) strategy sketch; resources & competencies, partners, customers & needs, competitors, value proposition, revenue model, risks & costs, values & goals, organizational climate, and trends & uncertainties. The remaining three checks provided by Kraaijenbrink (2015) focus on the strategy as a whole. The final checklists this paper discusses are provided by Kahneman et al. (2011), and focus on assessing whether (strategic) decisions are effected by biases. Elaborating on a McKinsey study by Lovallo and Sibony (2010), that reports empirical evidence of higher returns as a result of reducing the effect of biases in the decision-making processes, Kahneman et al. (2010) developed twelve checks, each addressing certain biases. All of the checklists can be used to systematically test whether the strategy meets the criteria, and which elements of the strategy require adjustments or additional work. Table 1 gives an overview of all four strategy checklist assessments and the specific focus areas they are addressing.

The next paragraph discusses the purpose of conducting strategy checklist assessments.

(15)

15 BRADLEY ET AL. (2011) KAHNEMAN ET A. (2011) KRAAIJENBRINK (2015) SIMONS (2010A,B)

BY MEANS OF:

Ten general tests which are acknowledged by CEOs as being part of the dialogue within their organizations.

Twelve checklist to assesses whether the initiative—plan—is prone to decision-making biases.

Nine strategy checks, each addressing specific criteria to assess a strategy’s quality on

Seven questions to stress-test whether the strategy is ready for implementation.

CHECKS

&

TESTS:

T1: will the strategy beat the market? Emphasis on being different, or just playing along?

C1: self-interest of (individual) decision-makers?

C1: coherence; is there coherence between the different elements of the strategy sketch?

Q1: is the primary customer defined?

Are all efforts maximized to generate value for the primary customer?

T2: sources of competitive advantage; positional advantage

& scarcity of capabilities.

C2: affect heuristic; is there an exaggeration of the benefits of a certain proposal?

C 2: efficiency; will each of the ten elements be employed to their full potential? Are areas for improvement identified?

Q2: is there a clear prioritization of whose interests are core valued—

customers, employees, shareholders, others?

T3: clarity about where to compete; market-segment

C3: group-thinking; were dissenting opinions take into consideration, or limited?

C3: effectiveness; are criteria in place for measuring the strategy’s effectiveness against a predetermined benchmark?

Q3: are critical performance indicators in place to monitor and control progress during implementation? Do these indicators actually measure what is crucial for the strategy to meet its intended goal?

T4: will the strategy put the organization ahead of trends, or merely elaborating on the status quo?

C4: saliency bias; are decisions based on heroic analogies?

C4: flexibility; is the strategy capable of matching the complexity and dynamics of the organization’s industry?

Q4: is there a clear scope? Are boundaries set for the strategic initiative(s) in order to avoid strategy implementers get side-tracked?

T5: is the strategy based on privileged—not easy available or proprietary—insights, which competitors don’t have access to?

C5: conformation bias; is evidence exclusively gathered for one solution instead of considering credible alternatives?

C5: robustness; will the strategy remains its stability when it is subjected to changes within its elements?

Q5: are there mechanisms in place to foster creative tensions among employees?

T6: has uncertainty been taken into consideration and is the strategy gardened against uncertain events?

C6: availability bias; is the decision based on the evidence or information that was currently available rather than seeking for new information—data?

C6: scalable for growth purposes or for achieving the same revenue with less effort?

Q6: is there a culture in place to support the strategic initiative(s)?

T7: is the balance between flexibility and commitment in line with the business environment?

C7: anchoring bias; where the numbers are based on guesses, extrapolations of historical trends or deliberate manipulation

C7: uniqueness; can the strategy generate sustainable value creation due the source, degree and sustainability of its uniqueness?

Q7: are there systems, mechanisms or plans in place to cope with (hypothetical) uncertainties

(16)

16 T8: is the strategy influenced by

biases of decision-makers?

C8: halo effect; are only certain aspects of stories of others used to support the plan, or were multiple examples taken into consideration which are comparable with the own business context?

C8: responsibility; is responsibility taken for the interests of other parties such as shareholder and stakeholders?

T9: is there support within the organization for the strategy?

C 9: sunk-cost fallacy; are (individual) decision-makers attached to the plan, due efforts of the past?

C9: pros and cons; is a systematically recapitulation of the previous discussed checks performed?

T10: is the strategy translated into clear, unambiguous objectives and actions?

C10: optimistic bias; is there overconfidence in the strategic plan?

C11: disaster-neglection; is the occurrence and impact of potential future events taking into consideration?

C12: loss-aversion; is there existence of extreme conservatism that might hinder fully exploitation of the plan’s potential?

Table 1: checklists for strategy testing

(17)

17 Purpose

Strategy checklists serve the overall purpose to assess a current or new strategy against critical criteria (Kraaijenbrink, 2015) and to (quickly) identify ambiguity, inefficiencies or shortcomings in the strategy (Simons, 2010b). Based on the information and learnings from a checklist assessment, strategist can prioritize on which areas to focus on (Bradley et al., 2011; Kraaijenbrink, 2015). Furthermore, checklists can function as a catalyst for stimulating engagement of participant in the decision-making process (Simons, 2010a,b). In addition, performing a checklist assessment and react upon it can be used as a vehicle to increase confidence in the strategic plan (Simons, 2010b). Finally, checklist assessments can be used as a tool to improve the strategy-development process itself (Bradley et al., 2011).

Advantages & Disadvantages

The primary advantage of using an assessment checklist lies in its usability for all businesses (Simons, 2010a), although not all items within a checklist might apply to a specific organization. Moreover, checklists can be used to test a strategy prior to its implementation. It can provide detailed insights into the quality of a strategy and its likeliness to succeed (Kraaijenbrink, 2015). Notwithstanding, this approach for strategy assessment—or testing—has its limitations as well. Kahneman et al. (2011) argue that checklists are prone to be used partially or selectively, while the benefits from checklist assessments can only be achieved by going through them completely. However, this argument is partially weakened by Kraaijenbrink (2015) who reasons that even performing a couple of test—which are most critical to an organizations specific business complexity—can contribute to (quickly) assess whether the strategy meets basic criteria. Nevertheless, the underlying message is that checklists should not be used merely as a check-or-no-check exercise without thorough assessment. Finally, Gary Klein argues in a McKinsey (2010) interview that checklists are only useful for high-validity environments, rather than complex environments, because the later requires a higher level of judgement. He proposes an alternative approach for assessing an intended plan which is discussed in the next section of this paper, i.e. a Pre-mortem analysis.

ADVANTAGES CHECKLIST ASSESSMENT

AUTHOR(S) DISADVANTAGES CHECKLIST ASSESSMENT

AUTHOR(S)

Usability for all businesses Simons, 2010a It is prone to be used partially or selectively

Kahneman et al., 2011 Can provide detailed insights into

the quality of the strategy

Kraaijenbrink, 2015

It requires discipline to go through all relevant checks

Kahneman et al., 2011 Assess the strategy’s elements

through different lenses—

perspectives

Bradley &

Matson, 2011

Going through a complete set of checklists requires much effort in information gathering and analyses

Kraaijenbrink, 2015

Can highlight the areas for improvement

Kraaijenbrink, 2015; Simmons, 2010b

In order to make a checklist assessment meaningful, a clear distinction should be made between the decision-makers and those performing the assessment

Kahneman et al., 2011

Can be used as a tool for a more systematic strategy assessment

Kraaijenbrink, 2015

It requires a culture and mind-set within the organization for open debate and where critical judgement is encouraged

Kahneman et al., 2011

Enforce engagement Simons 2010a,b Checklists are only useful for high- validity environments, rather than complex environments

McKinsey (2010)

Tackles (individuals) biases Kahneman et al., 2011

Better chance of developing a market-beating strategy

Bradley &

Matson, 2011 Table 2: advantages and disadvantages of checklist assessments

(18)

18 2.3.2 Pre-mortem analysis

As illustrated by Wheelen and Huger (2012), strategy evaluation often forms a subsequent phase in the strategic management process after strategy implementation. This after-the-fact rationalization, strategic hindsight, aims to enhance organizational learning by seeking for explanations from both failure and success (McKenna, 2011; Mitchell et al., 1989). However, this implies that an event first has to occur—or partially occur—,which might obsolete the process of evaluation when an organization has faced a catastrophic downturn. In contrast to a post-mortem learning-session where an event already occurred, Gary Klein (2007) introduced the pre-mortem technique which comes at the beginning of a project or initiative and makes the premise that an event already occurred and failed.

The pre-mortem technique is based on the principle of prospective hindsight, i.e. explaining future events as past (Mitchell et al., 1989). This theory argues that the decision-making of participants concerning future events will improve whenever decisions are reflected (Gross, 2014). Indeed, Mitchell et al (1989) found evidence that providing information about the outcome of an event, leads to a higher degree of correct reasons for this outcome by their research participants. In contrast to foresight which is drowned in uncertainty about the eventual outcome of an event, prospective hindsight artificially removes this uncertainty, making it a more effective approach to reason about the future (Gross, 2014). As emphasized by Klein in a McKinsey (2010) interview, “the pre-mortem technique is a sneaky way to get people to do contrarian, devil’s advocate thinking without encountering resistance” (p. 5). For this reason, it enforces participants to consider the initiative from another perspective and actively seek for flaws in the intended plan, rather than seeking for conformation (Serrat, 2012). By means of assuming that a plan has failed, it frees participants from being reserved and invites them to challenge the intended plan (Klein, 2007) and identifying potential impediments (McKenna, 2011). This approach is different from scenario planning which focuses on imagining how the future might evolve, by seeking for reasons why a future plan might has failed (Veinott et al., 2010). The team, guided by a facilitator, brainstorms about all the reasons that could have led to failure and develops actions to pro-actively mitigate these failures and strengthen the initial plan (Klein, 2007). The process how a pre-mortem analysis work towards this goal is described in the following paragraph of this section.

Process

In its essence, the pre-mortem technique is a brainstorm session consisting of a five-step process (Klein, 2007; Veinott et al., 2010): 1) preparation, 2) declaring the failure of a plan, 3) elicit reasons for failure, 4) prioritize the list of reasons, 5) revisit and strengthen the initial plan. Firstly, the facilitator must arrange a face-to-face, time-boxed session which can be attended by all key participants with a stake in the problem (Gross, 2014). During the kick-off, the facilitator starts with a recap of the initial plan and continues with the announcement that the plan has failed for currently unknown reasons (Klein, 2007). During the third phase, each team members writes down all the reasons for the possible failure of the plan within a short time period (Klein, 2007; McKenna, 2011; Veinott et al., 2010). The logic to perform this exercise individually is to avoid team members influencing each other and because each of the members have their own experiences, mental models and perceptions.

Consequently, during several rounds, each team member reveals one reason at a time until all reasons are gathered (Klein, 2007; Serrat, 2012; Veinott et al., 2010). Next, all the items are reviewed and collectively prioritized concerning their perceived impact or probability (Armbruster et al., 2014;

Serrat, 2012; Veinott et al., 2010). During the final phase, the team elaborates on the top three items on the collectively established list and brainstorms how to proactively mitigate or eliminate these potential failure reasons (McKenna, 2011; Serrat, 2012; Veinott et al., 2010). Based on the collective knowledge, the facilitator incorporates the mitigation activities in a project action plan (Gross, 2014).

This process can be repeated periodically to completely exhaust all items on the list and strengthen the plan in its early stage.

(19)

19 Purpose

The main purpose of conducting a pre-mortem analysis is to identify potential flaws or vulnerabilities in an intended plan before implementation (Armbruster et al., 2014; McKenna, 2011; Sullivan et al., 2008). The logic behind this methodology accompanies the dilemma this research paper addresses, i.e. testing a strategic initiative before implementation. By adapting this approach, managers are provided with the opportunity to pro-actively prevent, mitigate, or correct shortcoming in the plan and enhancing the probability for success (McKenna, 2011; Sullivan et al., 2008). In addition, a pre-mortem analysis can heighten the sensitivity on certain areas and enforcing extra attention on these areas (Gross, 2014; Klein, 2007; McKenna, 2011). Furthermore, by including multiples participants, the pre- mortem technique challenges individual conformation biases and encourage participants to abandon reservation and speak freely (Klein, 2007). The remaining of this section will summarize the advantages and disadvantages of performing a pre-mortem analysis.

Advantages & disadvantages

The main benefit of conducting a pre-mortem analysis lies in the identification of potential shortcomings of the intended plan, which otherwise might have been revealed in a later stage (Klein, 2007; McKenna, 2011; Serrat, 2012). By combining individual reasoning and collective brainstorming, the technique can overcome group thinking (Gross, 2014), cognitive biases by individuals (Serrat, 2012), and conformation biases (Gross, 2014; Serrat, 2012). Although the literature is very limited on addressing the disadvantages of performing a pre-mortem analysis, the following shortcomings can be argued. There is a significant need for an objective facilitator (Klein, 2007) to avoid the pre-mortem session becomes victim of being a battlefield where team members primarily criticise each other or get side-tracked from the initial purpose of the session. In the quarterly interview held by McKinsey (2010), Gary Klein argues that a pre-mortem won’t result in abandoning an initial plan, but rather adjusting it. One might argue that performing a pre-mortem analysis solely for adjusting an initial plan might result in strengthening a plan which itself is subordinated compared to alternatives.

Furthermore, since a pre-mortem is a periodic event at most, its lessons might be forgotten soon.

Finally, the knowledge from a pre-mortem analysis relies on assumption instead of empirical evidence.

The same counts for research studies which lack empirical evidence, questioning the proposed advantages of this technique in practical settings. Table 3 gives an overview of the advantages and disadvantages of conducting a Pre-mortem analysis.

ADVANGTAGES PRE-MORTEM ANALYSIS AUTHOR(S) DISADVANTAGES AUTHOR(S) Identification of potential shortcomings

of the intended plan, which otherwise might have been revealed in a later stage

Klein, 2007;

McKenna, 2011;

Serrat, 2012

A pre-mortem won’t result in abandoning an initial plan, but rather adjusting it

McKinsey, 2010

Reducing the tendency for conformation bias and consider alternatives

Gross, 2014, Serrat, 2012

There is a significant need for an objective facilitator

Klein, 2007

Can overcome cognitive biases from individuals with a significant stake in the development of the initiative

Serrat, 2012

Team members can challenge the intended plan, rather than being reserved for the fear of opposing their executive

Klein, 2007;

Serrat, 2012;

Veinott et al., 2010

Can overcome ‘group thinking’ Gross, 2014 Reducing overconfidence of individuals in

the intended initiative

Veinott et al., 2010

Can guide the team to pay extra focus on certain areas

Klein, 2007;

McKenna, 2011 Conducting a pre-mortem analysis is

relatively easy and low in costs

Serrat, 2012 Table 3: advantages and disadvantages of pre-mortem analyses

(20)

20 2.3.3 Interactive simulations: Business Wargames

“For centuries, the military–including its most famous leaders–have used the wargaming process to prepare for battle” (Frost et al., 2012, p. 86). By the end of the 18^th century, dr. C. L. Helwig developed a game that introduced three principles which are still considered of vital importance to today’s wargame simulations, i.e. game elements representing contingent instead of individual forces, multiple terrains, and a referee to run and control the game (Oriesek & Schwarz, 2008). McCarty Little, member of the US’s Naval War College which was founded in 1884, introduced the concept of two- sided wargaming, i.e. a focus on one’s own and one’s adversaries deliberating process (Brightman &

Dewey, 2014). Rather than adapting the notion of reductionism by viewing a system or problem as a dissociated collection of individual parts, McCarty took a holistic view and considered the pieces as a whole system–a view which is argued to be the foundation of modern wargaming (Brightman &

Dewey, 2014). Moreover, McCarty argued that the value of wargaming lies neither in its ability to predict or generate tangible outcomes, but rather lies in the dialogue that occurs within the wargaming process itself (Brightman & Dewey, 2014). “The application of wargaming in a business context, also called business wargaming is, in comparison to the overall evolution of the methodology, a recent development […]” (Oriesek & Schwarz, 2008, p. 6). Nowadays, a wargame is increasingly perceived as a useful process for gaining competitive actionable intelligence and decision-support by testing predetermined plans (Kurtz, 2003). There are important differences between educational business games–instructional games–, (computer) simulations, and business wargames used as practical assistance for strategic dilemmas.

Educational business games are often referred to as ‘total (business) enterprise simulations’, a term for “a descriptive and mathematical model of the general activities associated with operating a company in its totality” (Goosen et al., 2001, p. 23). These games, focusing on contingencies of multiple enterprise disciplines, predominantly ought to train players in business skills and evaluate players performances, e.g. the Business Strategy Game where (small) groups of students run a simulated company and individually are assigned to play the role of different departmental managers (Doyle &

Brown, 2000; Greco et al., 2013). Furthermore, there are functional simulation games which emphasis on one particular organizational discipline such as marketing or production, and concept simulation games, which further narrow their orientation within a certain organizational discipline, e.g. online brand marketing (Faria et al., 2009). Thus, business wargames differ from educational business simulation games by assisting in knowledge acquisition and accumulation rather than transferring knowledge. In addition to the distinction between educational business games and wargames, the second issue that needs clarification is the relation of wargames and simulations. “Since its introduction in academia and professional practice during the 1950s, gaming have been linked to simulation. Although, both fields have a few important characteristics in common, they are distinct in their form and underlying theories of knowledge and methodology” (Klabbers, 2009, p. 446). Rubel (2006) stresses that the difference between computer simulations and wargames lies in the usage of a computer model, where wargames require human players who may use a computer model to assist them. Thus, a wargames may be assisted by a computer model, rather than driven by a model such as simulations are (Schwarz, 2011). Moreover, Klabbers (2009) argues that gaming has much more potential in dealing with social and political issues because computer simulations are closed–

functionalistic–models representing a formal approach with a focus on the rules and resources domains where the user is operating outside the model as a spectator, whereas a wargame is an artificial representation–simulation–using an open–interactive–model where actors form a basic component of the model, thus adding the domain of social actors and human intervention (Klabbers, 2009; Rubel, 2006). Finally, simulations can run up to thousands of simulation runs, whereas wargames are limited to about 10 to 40 cycle runs since the inclusion of actors in the model makes more cycle runs too time consuming (Meadows, 2001). For this research paper, the view of Frost et al. (2012), Kurz (2003), Oriesek & Schwarz (2008), and Schwarz (2011, 2013) is followed by perceiving a wargame as a tailor-made, dynamic, strategic-simulation.

Strategy testing as an interplay between strategy formulation and implementation?

University of Twente

School of Management and Governance Track: Innovation & Entrepreneurship

Business Administration

Thesis paper

Master Thesis BA - 194100040 2015-2016

Strategy Testing as an Interplay between Strategy Formulation and Implementation?

Submitted by: Ven, Dennis van der s1533630

1

supervisor: Dr. ir J. Kraaijenbrink 2

supervisor: Dr. K. Zalewska-Kurek

Contact e-Mail: d.vanderven@student.utwente.nl

Enschede, August 26

2016

Acknowledgements

Abstract

List of abbreviations

List of tables

List of figures

Table of Contents

1. Introduction

1.1 Situation & Complication

1.2 Theoretical background

1.3 Research aim; central research question and methodological approach

1.4 Academic and practical contribution

1.5 Thesis outline

2. Literature Review

2.1 Increasing the likelihood of strategic success by strategy testing

2.3 Strategy testing approaches