A contingency approach towards a better understanding of SaaS Governance effectiveness

MASTER THESIS

A Contingency Approach Towards a Better Understanding of

SaaS Governance Effectiveness

Final version: 16-07-2014

Supervisor: Arjan Vreeken

Drs. Arjan Vreeken, signature:

Drs. Toon Abcouwer, signature:

Prof. Dr. Tom van Engers, signature:

AUTHOR: JAN-MARTIN SIJS

MASTER THESIS - INFORMATION STUDIES: BUSINESS

INFORMATION SYSTEMS

University of Amsterdam Faculty of Science

Internship at Ebicus

A Contingency Approach Towards a Better

Understanding of SaaS Governance Effectiveness

Jan-Martin Sijsa a_{University of Amsterdam}

Abstract. With the increasing use of SaaS (Software as a Service) by enterprises, there is a need for more

understanding on how SaaS should be governed effectively. SaaS governance in this paper is operationalized by the allocation of decision rights and task responsibilities relevant to a SaaS application in a business. This study aims to address the existing gap in scientific literature in the area of SaaS governance effectiveness by expanding and validating the contingency model of Winkler et al. (2011). The main goal of this study is to explore to what extent this contingency model explains SaaS governance effectiveness, and how this model can be improved. A case study approach was deployed to research three objectives. Firstly, the impact of SaaS security concerns on the allocation of decision rights and task responsibilities was studied. Secondly, we explored whether an outsourcing perspective could be a useful addition to the contingency model. Thirdly, the claim that the contingency model can measure SaaS governance effectiveness was tested. The results show that most SaaS security concerns are of low importance to businesses, but do indicate that data security is an important issue. In this study, we were not able to fully determine to what extent these security concerns are an addition to the contingency model, although we argue that the concerns may affect the mode of governance. Furthermore, SaaS governance is outsourced partly in practice. Task responsibilities, such as the responsibility for change implementation, were found to be outsourced to SaaS providers and intermediaries in one case, but decision authority was not outsourced in any of the studied cases. The findings also suggest that the contingency model is not fully capable of accurately predicting the effectiveness of SaaS governance. For a better assessment of SaaS governance effectiveness, we recommend that the contingency model is complemented by an adapted version of Weill and Ross’ (2004) governance effectiveness score. Future work could include a large-scale longitudinal study to gain further understanding in the dynamic concept of SaaS governance effectiveness.

Keywords. Contingency approach, IT governance, Outsourcing, SaaS, SaaS governance, SaaS governance

effectiveness, SaaS security concerns.

Introduction

Since the emergence of the internet, many traditional ways of doing business have changed. The rise of cloud computing is an example of a technological innovation that has allowed businesses to outsource their IT applications as well as their IT infrastructure to some extent (Böhm et al., 2011). As many companies are moving towards the use of cloud applications in their business and the cloud services market continues to grow (Gartner, 2013), there is a need for businesses to understand how these cloud applications should be governed. Organizations that employ effective IT governance are believed to perform better than those with poor IT governance (Weill & Ross, 2004). With the rise of SaaS (Software as a Service) applications and its widespread adoption in businesses, it is therefore essential to know how these cloud services should be governed as well.

The IT governance of cloud applications is, however, still a novel area of research. Only few papers have been written on the subject of SaaS governance (Winkler et al., 2011; Winkler & Günther, 2012; Winkler & Brown, 2013; Zainuddin, 2012). There is a need for more understanding on how SaaS governance differs from the more traditional (on-premise oriented) IT governance, and how effective SaaS governance can be achieved.

In order for organizations to improve their current mode of SaaS governance, they need reliable methods of measuring the effectiveness of this governance. Presently, there is a lack of tools available for companies to measure the effectiveness of their adopted SaaS governance. Although the contingency approach of Winkler et al. (2011) can be used to measure application governance effectiveness to some extent, it has a number of limitations. In this paper, we will address some of these limitations and suggest some additions to the contingency model. The purpose of this research is to improve our understanding of the aspects that influence the effectiveness of the governance of SaaS for businesses as well as for intermediaries and suppliers of SaaS. This understanding can help in the creation of tools which can be used to pin-point problems in businesses’ IT governance structures and modes

.

1. Research Question

1.1. Main question

In this thesis, the main goal is to get a better understanding as to when application governance is effective using the contingency model of Winkler et al. (2011) as a basis. Therefore, our main question is:

To what extent does the contingency model explain SaaS governance effectiveness? 1.2. Structure

In order to be able to answer the main question, we need more knowledge in the area of SaaS governance. Firstly, the concept of SaaS needs to be clearly defined as well as the parties that play a role in the SaaS ecosystem. As the role of intermediaries in governance research is often neglected, we will look at SaaS from an intermediary perspective as well. Besides this, the aspects that influence the adoption of SaaS will be elicited. Furthermore, we will investigate the security related issues regarding SaaS, as we believe these aspects can influence the mode of SaaS governance in an organization.

To understand SaaS governance, we will first explore IT governance literature in detail, because we believe that SaaS governance is one aspect of overall IT governance. Therefore it is also relevant to investigate what makes IT governance effective before addressing our main question. After determining what aspects make IT governance effective, we will review methods of measuring this effectiveness and analyze whether these methods are also applicable to SaaS governance.

After establishing a clear view on IT governance, we will continue by examining the concept of SaaS governance. More specifically we will firstly identify the differences between IT governance and SaaS governance. Then, the contingency model of Winkler et al. (2011) will be discussed and the manner in which it claims to measure SaaS governance effectiveness. Next, we will find opportunities for improving the contingency model of Winkler et al. (2011), with special attention to the explanatory power of the model regarding the effectiveness of SaaS governance.

We will proceed to investigate how SaaS security concerns influence the governance of SaaS, and to what extent these concerns may affect the effectiveness of SaaS governance.

Because there is a lack of literature that addresses the role of outsourcing with regards to governance, we will investigate this aspect as a possible manner of allocating IS authority for SaaS applications. We will look at how SaaS governance is currently being outsourced in businesses, and what the roles of intermediaries and suppliers are in the outsourcing of governance.

Finally, we will analyze to what extent the contingency model is a predictor of SaaS governance. We will use the validated method of measuring IT governance effectiveness (Weill & Ross, 2004) to measure the effectiveness of SaaS governance and compare the results to the Winkler et al. (2011) measured fit-effectiveness. Each sub question we formulated based on this argumentation can be seen below.

1.3. Sub-questions

1. What is SaaS?

1.1. What parties play a role in the SaaS ecosystem? 1.2. What is the role of intermediaries?

1.3. What are the reasons for the adoption of SaaS? 1.4. What are the security-related issues surrounding SaaS? 2. What is IT governance?

2.1. What makes IT governance effective?

2.2. How can IT governance effectiveness be measured? 3. What is SaaS governance?

3.1. What are the differences between IT governance and SaaS governance?

3.2. How does the contingency model explain the effectiveness of SaaS governance?

3.3. How can the contingency model be improved in order to explain SaaS governance effectiveness?

4. To what extent are SaaS security concerns an addition to the contingency model?

4.1. How do SaaS security concerns influence the allocation of decision rights and task responsibilities?

5. How can an outsourcing perspective be incorporated in the contingency model? 5.1. To what extent is SaaS governance outsourced in practice?

5.2. What is the current role of intermediary parties regarding the outsourcing of SaaS governance? 5.3. What is the current role of SaaS suppliers regarding the outsourcing of SaaS governance? 6. To what extent is the contingency model a predictor of SaaS governance effectiveness?

6.1. To what extent does the Weill and Ross (2004) method measure SaaS governance effectiveness?

6.2 What is the difference in measured SaaS governance effectiveness between the contingency method and the Weill and Ross (2004) method?

Each question specified above will be answered in the following sections.

2. What is SaaS?

In order to find out how SaaS governance effectiveness can be measured, the concept of SaaS must first be clarified. Software as a Service (SaaS) has gained a lot of attention in recent years due to the rising popularity in cloud services (Manro, Singh, & Joshi, 2013; Gartner, 2013). Cloud computing can be described as the applications that are delivered through the use of the internet, and the hardware and software in datacenters that provide these services (Armbrust et al., 2009). SaaS can be seen as an extension of the ASP (Application Service Provider) model developed in the 1990s (Cho & Chan, 2013). SaaS is one of the service models of cloud computing, alongside Platform as a Service (PaaS), and Infrastructure as a Service (IaaS) (Mell & Grance, 2011). SaaS is defined by the capability of a consumer to use software applications running on a cloud infrastructure provided by a cloud provider over a network (Mell & Grance, 2011; Cho & Chan, 2013). There are two major characteristics of cloud computing which traditional computational models lack: multi-tenancy and elasticity (Almorsy, Grundy, & Müller, 2010). Multi-tenancy involves the sharing of computational resources, the storage of data, services, and applications with other parties. Elasticity is a characteristic that makes it possible for tenants to scale up or down resources assigned to services based on current demand (Almorsy, Grundy, & Müller, 2010). Other characteristics of SaaS are web based mass customization of applications (Chong & Carraro, 2006; Xin & Levina, 2008), and the little to no control of the consumer over the cloud’s IT infrastructure.

Armbrust et al. (2009) argue that there are three levels in the SaaS paradigm (see figure 1). These levels are: the Cloud Provider, the SaaS provider / Cloud user, and the SaaS user. The difference between the cloud provider and the SaaS provider is that the cloud provider deals with the hardware needed for the SaaS application to run on, while the SaaS provider actually delivers web applications to end users (Armbrust et al., 2009).

Figure 1: The cloud paradigm of Armbrust et al. (2009).

The cloud paradigm as seen in figure 1 is not complete. Nowadays, it is quite common to see intermediary parties between the SaaS provider and the SaaS user (Kulkarni, 2012). As we will further elaborate on the role of intermediaries in the SaaS governance modes, it is important to specify who these intermediaries are and what they do.

2.1. SaaS Intermediaries

Intermediaries bring customers and suppliers together by providing a multitude of services. There are different roles that an intermediary might adopt. Dai and Kauffman (2002) stated that, in the context of B2B e-commerce, intermediaries may perform basic market functions, provide management support for

outsourcing, or serve as technology adapters. In the section below, we will briefly explore some of the roles that SaaS intermediaries may adopt.

Scientific literature about the ‘SaaS ecosystem’ (Hilkert et al., 2010) expands upon the cloud paradigm of Armbrust et al. (2009) by introducing an intermediary party. The SaaS ecosystem is a model that describes the roles of different parties in providing a client with a specific SaaS solution. Hilkert et al. (2010) present three roles in this ecosystem: The platform provider, the integrator, and the independent software vendor. We could say that the integrator fulfills a special role, as it does not actually provide a SaaS application or any infrastructure. The integrator can be thus seen as an intermediary party between the providers of SaaS. The client organization is not a role in the ecosystem, as it does not contribute to creating a SaaS solution (Hilkert et al., 2010). Other authors also have considered the role of the intermediary in SaaS. Böhm et al. (2010) created the generic value network for cloud computing in which they distinguished different roles for intermediaries. They categorize these parties into the integrator-, consultant-, and aggregator roles. Additional business models for intermediaries are the brokerage model and affiliate marketing (Lyons et al., 2009). Here, brokerages are seen as intermediaries making a profit by bringing buyers and sellers together and taking a percentage of each transaction that happens. A SaaS intermediary involved in affiliate marketing tries to forward potential customers to the SaaS provider.

Furthermore, Willcocks, Venters, and Whitley (2011) predicted that the dynamic cloud market would be organized along three value trajectories. According to them, there is a small group of bulk providers of utility and platform services supplemented with a larger group of specialist providers and service integrators. We regard these last two parties as intermediaries between the actual SaaS providers, and the SaaS users. We can thus make a distinction between two types of SaaS intermediaries:

 The Specialist Provider: This is the largest group of intermediaries surrounding SaaS. These specialize by having expertise in a certain branch of industry or functional know-how.

 The Service Integrator: This is a small group of intermediaries that uses their consulting and technology skills to integrate, manage, and optimize services for their partner enterprises. In this section, we have explored how intermediaries fit in the SaaS ecosystem and what roles they fulfill. Along with a number of business models, the two primary roles of intermediaries we identified are the specialist provider and the service integrator.

2.2. Adoption of SaaS

SaaS can provide businesses (SaaS users) with many benefits and there are thus a lot of drivers for adoption (Lee, Chae, & Cho, 2013). A few of the advantages of SaaS as compared to on premise software are: low costs of entry, easy implementation, low infrastructure maintenance costs, and scalability (Ju et al., 2010). SaaS applications usually make use of subscription based pricing models compared to the perpetual-use licensing with on premise applications (Choudhary, 2007). SaaS providers may have a pay-per-use or timely rental-charging model in place, which make the budget for implementing SaaS more predictable to the consumer (Cho & Chan, 2013). Also, the total cost of ownership with SaaS arrangements is generally lower compared to in-house software arrangements (Benlian, 2009; Bibi, Katsaros, & Bozamis, 2012).

2.3. SaaS Security Concerns

While there are many reasons to adopt SaaS, there are some disadvantages – or risks – that come with it too. Distrust in the security of SaaS applications was found to be the most influential inhibitor of SaaS adoption for businesses in Korea (Lee, Chae, & Cho, 2013), and is generally regarded to be a major barrier for the adoption of SaaS (Benlian, Hess, & Buxmann, 2009; Benlian & Hess, 2010). Willcocks, Venters and Whitley (2011) also found that over 60% of IT executives believed that data security and data privacy of SaaS are a bigger concern to the business compared to non-SaaS applications. We can therefore state that security concerns are a very large issue in the area of SaaS adoption and implementation. In the last few years, many security concerns of SaaS have been recognized in literature. Subashini and Kavitha (2011), for instance, identified 13 key security issues related to SaaS. Hashizume et al. (2013) also studied the security issues of cloud computing. In relation to SaaS, they diagnosed four important areas of security concerns. Furthermore, Almorsy, Grundy, and Ibrahim (2012) recognized five key SaaS security requirements as well which can be addressed with their SaaS security management architecture. Furthermore, Almorsy, Grundy, and Muller (2010) found two vital concerns of SaaS applications. These were more focused on the vulnerabilities of the application itself. We summarized the findings from the preceding literature in table 1. Because we found considerable overlap in the findings, we categorized each

concern into one of five security categories: data security, identity, availability, application security, and network security. These categories were based on common themes we found in the study of literature regarding SaaS security. We are aware that there are many other security related issues surrounding cloud computing, but decided to only focus on SaaS in particular, as this is our topic of interest.

From the short literature review above, we conclude that there are many security concerns around SaaS, such as: data security, identity, availability, application security, and network security. We could also add data ownership issues to this list, as organizations own and manage data in traditional IT environments, but may decide to outsource this partly to cloud service providers when adopting SaaS (Chen & Zhao, 2012). Although much attention has been given to the topic of data security in SaaS, few links have been made to the governance of SaaS.

Table 1. Categorization of SaaS related security concerns. Security Category Subashini & Kavitha (2011) Hashizume et al. (2013) Almorsy, Grundy, & Ibrahim (2012) Almorsy, Grundy, & Muller (2010)

Data security Data security Multi-tenancy (data storage)

Cryptography Data locality Data security

Data integrity Data segregation Data confidentiality Data breaches Backup

Identity Authentication & authorization Logging Identity management and sign-on process Authentication Virtualization vulnerability Identity management Availability Availability Accessibility

Application security Web application security Application security Web application vulnerability scanning Web application security miss-configuration and breaking Network security Network security

2.4. Governance of SaaS

SaaS applications, thus, have their advantages and potential issues. Just like any on premise applications, SaaS needs to be governed to control the issues, and make the most of the opportunities it offers. To comprehend how SaaS can be governed, however, we first need to understand the more generic concept of IT governance.

3. What is IT governance?

3.1. Definitions

IT governance is a concept that arose due to the ever increasing IT investments of companies and large scale failures in IT-related projects (Weill & Woodham, 2003). Governance is a term which stems from the Greek word of ‘κυβερνάω’, meaning ‘to steer’ (Rampersad & Hussain, 2014). The term has since been used in many different ways and contexts. IT governance has emerged from corporate governance and strategic information systems (Webb, Pollard & Ridley, 2006), and can be considered to be a subset of corporate governance (Korac-Kakabadse & Kakabadse, 2001). IT governance has been interpreted in many different ways by researchers throughout the years. Weill and Woodham (2003), for instance, defined IT governance as: “specifying the decision rights and accountability framework to encourage desirable behavior in the use

of IT” (p. 1). The IT Governance Institute (2003) described the concept as a structure of relationships and processes used to control the enterprise in order to achieve the business’ goals by adding value while balancing risk. IT governance, according to the IT Governance Institute (2007), also encompasses foundational mechanisms in the form of leadership, organizational structures and processes that ensure that the enterprise IT maintains and extends organizational strategies and objectives. Webb, Pollard, and Ridley (2006) used existing literature to give a comprehensive definition of IT governance: “IT Governance is the strategic alignment of IT with the business such that maximum business value is achieved through the development and maintenance of effective IT control and accountability, performance management and risk management” (p. 7).

IT governance can take many forms and can take place in different parts of an organization. Governance integration mechanisms can exist on a structural, procedural, and relational level in a business (Peterson, O'Callaghan & Ribbers, 2000). Structural integration deals with the formal integration structures and staff professionalization within an organization (Peterson, O’Callaghan & Ribbers, 2000), procedural integration mechanisms are described as the system of IT-decision making and communication processes (Luftman & Brier, 1999), and social integration mechanisms promote the shared understanding between key stakeholders and their active participation in IT decision making (Reich & Benbasat, 1996; Sambamurthy et al., 1994).

3.2. IT Governance as IT Authority

Much research on IT governance has focused mainly on the allocation of IT authority in organizations (Weill & Woodham, 2003). This authority refers to IT related decisions and activities which are normally not outsourced (Lee et al., 2003). Decisions regarding the overall IT function are seen as the core element of IT governance (Brown, 1997; Sambamurthy and Zmud, 1999; Weill & Ross, 2004). IT decision rights can be divided horizontally or vertically (Weill & Ross, 2004). Horizontal allocation of authority usually happens between the business- and IT departments within an organization, whereas vertical allocation is concerned with dividing the decision rights between different levels of staff (i.e. C-level, senior level, mid-level, and staff level). The horizontal allocation of IT authority is usually classified into the centralized, decentralized, and federal modes (Sambamurthy & Zmud, 1999; Weill and Ross, 2004). A centralized allocation of IT authority usually implies that the internal IT department or corporate IT has most decision authority and task responsibility, while a decentralized allocation suggests that line business units or smaller IT divisions have more authority (Sambamurthy & Zmud, 1999). Weill and Woodham (2003) extended Sambamurthy and Zmud’s (1999) model and added features of vertical allocation. They proposed a framework of six different governance archetypes that can be identified in organizations. These archetypes are: business monarchy, IT monarchy, feudal, federal, and anarchy. Each of these archetypes can be found in any organization and the archetype can differ based on many company specific aspects.

In the rest of this thesis, we will regard IT governance as the allocation of IT authority and use Sambamurthy and Zmud’s (1999) archetypes to explain the distribution of this authority in organizations. We have chosen this approach, because it is a widely applied and well-defined way of examining IT governance.

3.3. Changes in the Allocation of IT Authority

Traditionally, IT authority is centralized in many businesses. This is illustrated by Cramm (2005), who states that in many organizations “business executives have little authority over IT funding” (p. 1). The introduction of SaaS is, however, hypothesized to change this landscape significantly. Because of the earlier stated advantages of SaaS application over on premise applications, less involvement of the IT department is required to implement and support the application (Winkler et al, 2011). A business manager may, for instance, need a specific application which is time consuming to produce and only needed for a few employees. In this case, the manager could decide to bypass the IT department and order a SaaS application that fulfills her needs directly from a SaaS provider. SaaS, in contrast to on premise applications, requires less up-front capital investment and implementation efforts, which makes it easier for business managers to fund and deploy these applications (Xin & Levina, 2008; Xue et al., 2008). This may lead to ‘stealth adoption’ of SaaS by business units. With stealth adoption, the adoption of a certain innovation (e.g. a SaaS application) is conducted without the knowledge of important internal stakeholders such as top management (Zainuddin, 2012).

In short, SaaS facilitates the implementation of enterprise applications by business units. Winkler and Brown (2013) therefore argue that more decision authority moves towards business units with the introduction of SaaS applications, and centralized IT departments may lose some of their authority. The question is, though, if this change in the allocation of IS authority towards the business side is beneficial to the organization.

There seems to be no definitive ‘right’ way to allocate IT authority. The most appropriate allocation depends on a number of contextual factors (Brown & Magill, 1994; Sambamurthy & Zmud, 1999). With this in mind, a contingency approach seems most suited to determine the optimal distribution in order to increase the effectiveness of the organization’s IT governance. In the next section, we will explore what makes IT governance effective with a special focus on the distribution of IT related authority.

3.4. What makes IT Governance Effective

There have been a number of studies that have examined how effective IT governance can be achieved. Recently, Ferguson et al. (2012) found that there are three important determinants of effective IT governance. The existence of IT steering committees, senior management involvement, and a corporate performance measurement system are mechanisms that can increase the effectiveness of IT governance within a firm. The paper of Weill and Ross (2004) is another influential paper regarding the assessment of IT governance and the measurement of governance effectiveness. They believe that the assessment of IT governance should be about how well the governance arrangements encourage desirable behaviors and ultimately how the firm achieves its desired goals (Weill & Ross, 2004, p. 119). They identified five important factors that should be taken into consideration when evaluating IT governance: 1) enterprise setting, 2) governance arrangements, 3) governance awareness, 4) governance performance, and 5) financial performance. Enterprise setting describes the industry, size, the number of business units and the relationships between these business units, or the “level of synergy desired between business units” (Weill & Ross, 2004, p. 119). Governance arrangements capture which archetypes of the IT governance allocation are used for each IT related decision and which mechanisms are used for implementation. Governance awareness is about how well people throughout the firm understand governance and is able to identify communication approaches to engaging management. Governance performance assesses the effectiveness of IT governance using a number of factors, which we will discuss in more detail below. Financial performance is used to assess the extent of the impact of IT governance on business performance metrics.

3.5. Measuring IT Governance Effectiveness

Weill and Ross (2004) claim that their ‘governance performance’ – aspect can be used to assess the effectiveness of IT governance in delivering four objectives weighed by their importance to the business:

 Cost-effective use of IT

 Effective use of IT for asset utilization

 Effective use of IT for growth

 Effective use of IT for business flexibility

Senior managers should first identify the relative importance of each of these objectives in their business and then rate enterprise performance on each factor. With a weighted average formula, a score out of 100 can be calculated. An example of this way of assessing IT governance effectiveness can be found in the paper by Simonsson, Johnson, and Ekstedt (2010). Here, the authors used Weill and Ross’ method in order to find the effect of IT governance maturity on IT governance effectiveness. Bowen, Cheung, and Rohde (2007) expanded the governance performance assessment tool of Weill and Ross (2004) with a fifth aspect: compliance with legal and regulatory requirements.

Although Weill and Ross (2004) produced a popular method of evaluating IT governance effectiveness, there have been a number of other authors who looked at this problem from other perspectives. Damianides (2005), for instance, created a checklist for IT governance including 44 diagnostic questions divided into three categories: 1) questions to ask to uncover IT issues, 2) questions to ask to find out how management addresses these issues, and 3) questions for a self-assessment of IT governance practices. For each of the questions, the extent to which it relates to IT value delivery, IT strategic alignment, risk management, and performance is specified. This method, however, has not been validated in further research. Ali and Green (2005; 2007; 2012) based their IT governance effectiveness measurement approach on Goodhue and Thompson’s (1995) Technology to Performance Chain. The TPC is based on the assumption that there can only be a positive impact of IT on performance when the information system is utilized and there is a good

fit between the technology and the task it supports (Goodhue & Thompson, 1995). Their theoretical approach focuses more in individual performance impacts of IT than on the organizational impact (Goodhue & Thompson, 1995, p. 213). Ali and Green adapted two questions from Goodhue and Thompson related to measuring the individual performance impact of an IT tool and made these questions applicable to measuring IT governance effectiveness. This method of measuring IT governance effectiveness does not seem suited, because the TPC-framework on which it is based is oriented around the impact of technology on individual performance. IT governance is a much broader concept that has organization wide consequences, and cannot easily be captured in these two adapted questions. Yanoski and Caruso (2008) also measured IT governance effectiveness using two questions. These, however, are very abstract and generic questions aimed at measuring overall perceived governance effectiveness. These questions fail to capture important aspects of governance, such as the goals that governance should achieve Weill and Ross (2004).

In sum, we can conclude that IT governance effectiveness is a difficult construct. There are many aspects that contribute to effective governance, but not all of these can be measured. The Weill and Ross (2004) method, supplemented by Bowen, Cheun, and Rohde (2007) seems to be the most suited available method for assessing the effectiveness of IT governance.

4. SaaS Governance

4.1. IT Governance and SaaS Governance

Although there is quite a substantial amount of knowledge on IT governance, the more specific IT governance of SaaS is still a relatively uncovered area of research. Because SaaS-applications nowadays are part of many organizations’ enterprise IT-systems, these applications also need governing. But due to the unique features of SaaS, the governance of these applications may require a different approach than the governance of traditional on-premise software. In this study, we view SaaS governance as an application specific form of IT governance. Winkler et al. (2011) made an attempt to investigate how firms allocate SaaS governance by employing a contingency perspective. They focused on establishing the accountability framework as a form of achieving IT governance. Sambamurthy and Zmud (1999) define this accountability framework as “the patterns of authority for key IT activities in business firms” (p. 261). Winkler et al. (2011) operationalized the concept of IT governance on an application level (SaaS) with a set of variables. They believe that these variables can be divided into the decision authority and task responsibility categories. The most important decisions need to be taken about SaaS changes (e.g. choosing requirements), financial decisions (e.g. annual spend), and architecture decisions (e.g. integration with other applications). This is congruent with the main scope of overall IT related decision making (Weill & Ross, 2004). The other category of variables – task responsibility – is mainly concerned with a number of key operational activities surrounding SaaS applications. These task responsibilities refer to change implementation (e.g. implementing new workflows and customizations), first level end user support (e.g. incident handling), and second level end user support (e.g. non-routine technical disturbances). For an overview of the operationalization of SaaS governance, see table 2. Accordingly, Winkler et al. (2011) defined the IT governance of SaaS as “the locus of authority for key decision and task responsibilities related to a SaaS application” (p. 15). We could state that SaaS governance is essentially IT governance on application level, not on enterprise level. We will use Winkler et al.’s (2011) definition and operationalization of SaaS governance in the rest of this thesis. The terms SaaS governance, IS authority, and application governance are used synonymously throughout this paper.

Table 2. Overview of the operationalization of application governance according to Winkler et al. (2011). Application Governance Description Constructs

Decision Authority Deals with the allocation of authority to make decisions.

Change decision authority Financial decision authority Architecture decision authority Task Responsibility Deals with the allocation of

responsibilities for key operational activities to SaaS applications.

Change implementation responsibility

First-level end user support Second-level end user support

4.2. A Contingency Approach towards SaaS Governance

Winkler et al. (2011) studied organizational- and SaaS application contingencies with respect to their effect on the mode of governance (i.e. centralized, federal, decentralized). The contingency approach that Winkler et al. (2011) took towards examining this effect on the mode of governance is based on the multiple contingencies model of Sambamurthy and Zmud (1999).Contingency theory finds its roots in organization theory, and wasused primarily in studies of organizational design and performance (Drazin & Van De Ven, 1985). The central assumption of contingency theories is that an organization needs to be structured in a way that fits with its context in order to perform well (Drazin & Van De Ven, 1985; Fiedler, 1964). Winkler et al. (2011) made the distinction between organization- and application level contingencies. According to contingency theory, if there is a fit between these contingencies and the present mode of application governance, the governance is more effective and the firm performs better. An overview of the contingencies as established by Winkler et al. (2011) can be seen in figure 2 below.

Figure 2. The contingency model of Winkler et al. (2011).

Winkler et al. (2011) found that there were a few contingencies that were especially salient in explaining the mode of governance. Absorptive capacities and the origin of initiative were found to be most determining of the allocation of authority in a firm, but no support was found for strategic IS goals. According to contingency theory, if the theoretically induced mode of authority has a fit with the actually present mode of authority in a certain organization, the application governance mode is seen as effective. Winkler et al. (2011) claim that “governance effectiveness describes a positive outcome if there is a fit between the SaaS governance arrangements, the organizational context and the properties of the SaaS application itself” (p. 11). Their contingency approach can thus be seen as a tool that measures and predicts the effectiveness of SaaS governance arrangements in a company.

A later study of Winkler and Brown (2013) compared the distribution of authority between SaaS and on-premise applications. Here, the relation between application governance and the aspects IT governance,

origin of initiative, application specificity, scope of use, and absorptive capacities were empirically tested.

The researchers found that only a few contingencies were significantly associated with application governance. These were: the origin of initiative, the scope of use, and absorptive capacities (in particular the IT business knowledge). IT governance - defined as the horizontal distribution of decision rights on the organizational level - was related to neither decision authority nor task responsibility. The specificity of an application was not significantly associated with application governance either.

We can conclude that there are certain contingencies that affect the mode of application governance. Not all contingencies have proven to have the same level of impact on decision authority and task responsibilities surrounding SaaS applications. The most important contingencies to keep in mind are the origin of initiative, absorptive capacities, and the scope of use.

4.3. Limitations of Contemporary Research

There are, however, some limitations to the contingency approach of Winkler et al. (2011). Firstly, only 4 cases were studied in their research using a limited number of interviews. In two of the four cases, managers from the IT department and from the business units were interviewed. Ideally, in all cases interviews should be taken from both the business and the IT. Secondly, the measurement of the consequences of application governance arrangements is problematic. Winkler et al. (2011) measured the successfulness of SaaS governance by a limited number of relevant aspects. They primarily based their evaluation on the impact that the present governance arrangements had on the alignment between business and IT, which is in itself an important aspect of governance effectiveness (Brown & Magill, 1994; Sabherwal & Chan, 2001). IT governance is, however, a much more elaborate concept. There are more measures of the effectiveness of governance arrangements than just focusing on business-IT alignment, as we have discussed above. Further, in one case Winkler et al. (2011) were unable to determine if the governance was effective (p. 12). This may also indicate that their method of measuring governance effectiveness was insufficient. Therefore it is still unclear whether a fit between the theoretically induced mode of governance based on certain contingencies and the present mode of governance leads to an effective application governance.

4.3.1. SaaS Security Concerns and Governance

Another limitation is that Winkler et al. (2011) do not explicitly address the role of SaaS security and privacy concerns as a contingency. In fact, the concepts of privacy and security concerns were excluded from analysis, because they did not seem to “exhibit a logical link to the theme of application governance” (p. 6). We, on the other hand, do believe that this is an important contingency to take into account. It is related to application governance – operationalized in decision authority and task responsibility – in multiple ways.

We will illustrate this relation with an example. Consider the following fictional situation: company X has adopted a SaaS application and there are many security related issues surrounding that application. Business units use the application extensively, and have purchased the SaaS application themselves. When the business units only have a few security concerns, they will most likely do without a lot of involvement of the IT department. When they have many concerns, it would seem wiser to ‘outsource’ the handling of these concerns to the IT department. The IT department has specific knowledge on IT security, and probably knows more about the ways to address the security issues of the SaaS application than the business units. There is more knowledge in the IT department related to security, so it is beneficial for company X to let them handle security operations such as monitoring the SaaS application, updating it, and making sure all kinds of organization wide security measures have been taken to decrease the risk of data breaches. Also, the IT department usually has a good overview of the application landscape of the organization. This makes the IT department more capable of dealing with security concerns in the context of the whole enterprise than a business representative, who would mostly be focused on a specific part of the business. We would therefore expect that the business units source these security issues to the IT department to optimize efficiency and save costs, effectively increasing the task responsibility of the IT department.

To summarize, we expect a more centralized governance (IT department) when there are many SaaS security concerns, and a more decentralized allocation of IS authority (business units) when there are only few concerns. Consequently, we believe that the perceived security concerns towards a SaaS application do affect application governance. How this construct affects the mode of governance, however, is still unclear from existing research.

4.3.2. Outsourcing Governance

Additionally, a gap in application governance literature lies in its failure to include the important position of intermediaries and providers of SaaS. Winkler et al. (2011) mainly focus on the internal governance arrangements in a company, although the outsourcing of task responsibilities to providers is briefly mentioned (p. 7). As intermediary parties are offering more services to SaaS users, we could also argue that a part of SaaS governance may move towards external parties outside the adopting organization. It is debatable whether any form of governance can be outsourced. If decisions concerning the organization are outsourced, the business is basically outsourcing itself and giving a part of its autonomy away. To avoid a philosophical debate about whether an organization can outsource governance at all, we will refer back to the definition of application governance that we have used. In this definition, only decision authority and task responsibilities regarding SaaS applications are included.

Some businesses could wish to outsource a part of IT authority, such as some task responsibilities (e.g. first line support for an application). This also makes sense when we analyze this form of sourcing in terms of Transaction Cost Economics (TCE) (Commons, 1931; Williamson, 1985). In the case of a SaaS related helpdesk, for example, users of the application can call or e-mail in order to report a problem they experience. When the application is used on a large scale, there is a high frequency of transactions between users and the helpdesk, because an increasing number of users inherently means an increase in incidents that need to be solved. In the case of simple user-related problems (e.g. forgotten password) the asset specificity can be regarded as quite non-specific. In this case, according to TCE and ceteris paribus, it can be beneficial to outsource this activity. The task responsibility of first-level support, an aspect of application governance, can thus shift from inside the adopting organization towards an external party such as a SaaS intermediary. Although the outsourcing of task responsibilities seems plausible, we believe that in practice no organization would outsource their decision authorities. That would imply handing over decisions (e.g. financial decisions) to an external party, which again would mean giving away a part of the organization. Nonetheless, it would be interesting to further investigate if and how some aspects of SaaS governance are outsourced towards external parties.

4.4. Conclusion

With these limitations of contemporary research in mind, we can conclude a number of things. Firstly, there is a lack of proof that the contingency model of Winkler et al. (2011) is an accurate predictor of SaaS governance effectiveness. Further research is needed to uncover the influence of the fit between organization- and application level contingencies with governance arrangements on SaaS governance effectiveness. Also, although SaaS security concerns seem like an important contingency, its influence has not yet been examined on the governance arrangements. Thirdly, an outsourcing perspective is lacking in the area of SaaS governance. Our contribution to governance literature is threefold. We will expand the contingency model of Winkler et al. (2011) by including SaaS security concerns. Furthermore, we will incorporate an outsourcing perspective in looking at the allocation of IS authority. Besides this, we will validate the model of Winkler et al. (2011) as a tool that measures SaaS governance effectiveness.

5. Methodology

5.1. Introduction

To research the sub-questions mentioned in section 1.3 and find an answer to the main question, we need a solid methodological approach. For this study, we have chosen for a qualitative approach, making use of case studies to explore and describe SaaS governance. In the next sections, we will look into our research methodology in more detail.

5.2. Research Design/Type

We chose a qualitative research approach for this study for a multitude of reasons. One of the most important drivers to use qualitative research, according to Creswell (2012), is because a researcher needs to explore a relatively unknown area. In our case, the field of SaaS governance is novel and only few researchers have studied the concept. In general, we can state that there is relatively little known about the impact of SaaS on governance arrangements, both in literature as in practice. Qualitative research is suited for gaining a complex and detailed understanding of the issue at hand (Creswell, 2012). Furthermore, qualitative research empowers individuals to share their stories and takes context into account (Creswell, 2012). Also, because we want to extend the model of Winkler et al. (2011), we are essentially examining the relations in this model and extending it. Creswell (2012) claim that a qualitative approach should be applied when researchers wish to explain the mechanisms in theories and models, and develop theories when existing theories “do not adequately capture the complexity of the problem” (P. 48). Therefore the qualitative research approach seems like an appropriate means to investigate SaaS governance.

5.2.1. Theory Testing & Theory Developing

This study can be regarded as a theory-oriented research (Verschuren & Doorewaard, 2010). Theory-oriented research deals with “solving a problem encountered in the theory development in a particular

scientific area, and within this area, with regard to a specific issue” (p. 42). According to Verschuren and Doorewaard (2010), two types of theory-oriented research can be distinguished: Theory testing-, and theory developing research. Theory developing research often has the goal of addressing a so-called gap in existing theory. A part of theory still needs to be constructed (Verschuren & Doorewaard, 2010). Theory testing research differs in that its goal is not to create or complement theories, but to test existing views. Because we are testing if the model of Winkler et al. (2011) predicts SaaS governance effectiveness, but also exploring whether the model can be expanded with SaaS security aspects and an outsourcing perspective, our study employs both types of theory-oriented research.

5.2.2. Multiple Case Study

In order to test and develop theory in a qualitative way, a multiple case study method was used. Case studies can be seen as research strategies in which the researchers try to gain a profound and full insight into one or several objects or processes that are confined in time and space (Verschuren & Doorewaard, 2012, p. 178). The case study is well suited for examining phenomena in their natural context and can be used for theory testing as well as theory developing research (Flyvbjerg, 2006). Because we are testing and constructing a theoretical mode, generalizability of the results is relatively important. In contrast to prior belief, the results from a carefully constructed case study can be generalized (Flyvbjerg, 2006). We will be doing a comparative case study in order to compare variables such as the governance effectiveness of a company. Other characteristics of the case study are that it usually relies on small number of research units, intensive data generation, much depth, a strategic sample selection, an assertion concerning the object as a whole, and an open observation on site (Verschuren & Doorewaard, 2012). Taking these characteristics and the research questions and problem domain into account, the case study methodology seems like an appropriate approach.

5.3. Case Selection

A crucial aspect of conducting a good case study is the selection of the cases to be studied (Verschuren & Doorewaard, 2012). According to Flyvbjerg (2006), there are two general types of case (or sample) selection: random sampling and information oriented selection. In this study, we opted for the information oriented selection to maximize the utility of information from single cases (Flyvbjerg, 2006). There are four types of information oriented selection, each with its own objective. We chose for the maximum variation

cases, which is useful for obtaining significance of various circumstances for case process and outcome

(Flyvbjerg, 2006). By selecting cases in this way, we can improve the generalizability of our conclusions and examine a variety of cases with different characteristics. In the current study, we maximized case variety in the dimension of industry type (see table 3).

5.3.1. Requirements for Selection

As we want to find out if the contingency model predicts SaaS governance effectiveness, the cases are naturally organizations using SaaS applications. These organizations all had some form of IT governance in place. For the sake of case comparability, we decided to focus on one SaaS application called CRM On Demand. CRM On Demand is a well-known and used customer relationship management (CRM) application. CRM applications are a good representation of SaaS, as they are among the most common types of SaaS applications used by businesses (Benlian, Hess, & Buxmann, 2009).

5.3.2. Opportunistic variables

Some opportunistic and contextual factors also need to be taken into account when selecting cases. For instance, it was only possible to contact (former) clients of a SaaS intermediary who use CRM On Demand as a SaaS application. Furthermore, not all organizations were willing to cooperate in the research for a variety of reasons.

5.3.3. The Cases

Table 3. Summary of the selected cases. The number of employees is rounded for anonymity. Case A Case B Case C Case D Industry Services Retail Transport Government

Business model Service Product Service Service

Playing field World-wide World-wide World-wide The Netherlands

Number of employees 500,000-1,000,000 200,000-500,000 10,000-50,000 10,000-50,000 Number of CRM On Demand users 1000+ 10-50 50-100 50-100 Number of years using CRM On Demand 5-10 0-2 2-5 2-5 5.3.3.1. Case A

Case A is a large support service provider operating on a global scale. It has over 500,000 employees, with more than 1000 users of CRM On Demand. In this organization, CRM On Demand is used for more than five consecutive years and the primary purpose of its use is for sales and retention. The organization would like to approach CRM On Demand more as a strategy than as a database, and this change of approach is currently in progress. It is also important to note that organization A is in the middle of a transition from a very decentralized IT organization to a more centralized IT department.

5.3.3.2. Case B

Case B is another very large organization, with over 200,000 employees situated all over the world. This organization focuses mainly on retailing. As of now, there are 10-50 users of CRM On Demand, although not everyone uses the application very actively. CRM is mostly used for storage of franchise data, such as the size and location of certain shops. Contracts and other agreements with franchise stores are loaded into On Demand as well. Organization B has plans of expanding the use of On Demand to different business units. Also, they would like to further encourage employees to use the application and increase the number of active users.

5.3.3.3. Case C

Case C is an organization active in the port management industry. It is active all over the world and has around 10,000-50,000 employees. CRM On Demand is used primarily for pipeline management and forecasting by the commercial department. There are between 50-100 users of On Demand, although the number of active users is about half of that. The organization has been using On Demand for two to five years and is planning to improve the use of CRM to maximize the value they can get from it.

5.3.3.4. Case D

Case D is a Dutch government executive organization. It has around 10,000-50,000 employees providing services throughout the country. There are about 100 users of CRM On Demand, but there are currently only around 70 active users. CRM is used in a number of business units, largely for storing client data such as reports of company visits. The organization is presently busy constructing a vision around CRM and ways of expanding the use of CRM throughout multiple business units.

5.4. Operationalization of the Variables 5.4.1. Semi-Structured Interviews

In case study research, it is common to make use of qualitative methods of data gathering, such as interviews. In our study, the goal is to compare multiple cases on a number of variables concerning application governance contingencies (including our predicted contingency of SaaS security concerns), and the effectiveness of this governance. The interviews should therefore be quite structured so that the wording and sequence of questions are the same for each respondent. In this way, we can be more confident that the results vary based on the differences in respondents, and not because of differences in the questions (Gordon, 1975). This enhances the comparability of the interviews. But because the topic of SaaS governance is quite abstract and may contain more aspects than we have foreseen, the interviews should contain some room for discussion. The semi-structured interview is ideal for this purpose. The semi-structured interview is typically used when a researcher will not get more than one chance to interview (Bernard, 1988), and when one wishes to obtain reliable and comparable qualitative data with opportunities of identifying new ways of understanding the topic at hand (Cohen & Crabtree, 2006).

In order to get reliable and comparable data from the interviewees, it is important that each concept is well defined beforehand. Also, each variable that is measured should be operationalized in a way that the interviewees can provide sensible answers. In the following paragraphs, we will elaborate on how each variable was operationalized.

5.4.2. Contingencies

Measuring the influence of contingencies as specified by Winkler et al. (2011) is central to answering the main research question. We will address how we operationalized the contingencies in our study below.

5.4.2.1. Organization Level

The organization level contingencies of Winkler et al. (2011) are adapted from the contingent influences on IT governance as identified by Sambamurthy and Zmud (1999). The organization level contingencies can be divided into the corporate governance and absorptive capacities categories. Firm size, managerial autonomy, and strategic IS goals are contingency factors of the corporate governance category, while line IT knowledge and IT business knowledge belong to the absorptive capacities category. As we have discussed before, absorptive capacities are considered to be an especially salient contingency, while corporate governance contingencies were not regarded as very influential (Winkler et al., 2011). Therefore, in our study, we will disregard the corporate governance category of contingencies and focus on the absorptive capacity as an organization level contingency. We will, however, measure some organization level aspects (i.e. firm size, business model), because we generally believe these may prove helpful in explaining the findings and placing them into context. As we have argued above, we think that SaaS security concerns should also be taken into account as a possible contingency that affects the governance mode. Although it may perhaps seem logical to see this as an application level contingency, because security issues are partly inherent to a specific application, we argue that these concerns also exist on the organizational level. Concerns are often very dependent on contextual factors, such as the type of business a company is in. In banking, for instance, there may be many more security concerns than in a food production company. It also very much depends on the importance of the data that is stored in the application by the business. Therefore we believe SaaS security concerns can be seen as an organization- and an application level contingency, since it may be dependent on some application specific characteristics as well.

5.4.2.2. Application Level

Besides organization level contingencies, Winkler et al. (2011) also specified application level contingencies, as they found that organization level factors did not suffice to explain the phenomenon of SaaS governance. Within the application level contingencies, Winkler et al. (2011) created three categories: Scope, specificity, and initiative. The scope category contains factors related to application usage (i.e. to what extent is the SaaS application used in the organization). The specificity category is constructed of the integration complexity, ease of customization, and training needs factors. The final category of application contingencies, initiative, contains the origin of initiative (e.g. where did the initiative to purchase a SaaS application come from, a business unit or the IT department?). From the review of literature we found that some application level contingencies are less important than others. The origin of initiative (Winkler et al., 2011; Winkler & Brown, 2013), and the scope of use (Winkler & Brown, 2013) were found to be salient contingencies. Specificity was found to not significantly influence application governance (Winkler & Brown, 2013). Therefore we will not include specificity as a contingency that we will investigate. We did, however, measure scope of use and origin of initiative as application level contingencies. The contingencies that were measured in this study can be viewed in table 4.

5.4.2.3. Interview Questions Related to Contingencies

The organization level and application level contingencies were measured by adapting interview questions from Winkler et al. (2011), and adapting the questionnaire of Winkler and Brown (2013). All questions were translated to Dutch, as all cases are Dutch organizations or organizations with Dutch divisions. The interview questions can be viewed in appendix A.

Table 4. Measured contingencies in this paper. Contingencies marked with an asterisk are hypothesized to be contingencies

affecting SaaS governance.

Contingency Level Contingency Category Measured Contingency Factors

Organization level Absorptive capacities Line IT Knowledge Business IT Knowledge Organization

level/Application level

SaaS Security Concerns* Data Security* Identity Security* Availability* Application Security* Network Security*

Application level Scope Application Usage

Initiative Origin of Initiative

5.4.2.4. SaaS Security Concerns

From the literature study above, we concluded that there are many security concerns related to SaaS and that these can be divided into five categories: data security, identity, availability, application security, and network security. We have also argued that SaaS security concerns should be researched as a contingency that influences application governance. To measure if and to what extent security concerns influence SaaS governance, we constructed a number of interview questions. These questions addressed each of the security concern categories. The interviewees were asked to what extent they were concerned about each category of security issues. These questions can be viewed in appendix A.

5.4.3. SaaS Governance

As we have stated earlier, Winkler et al. (2011) operationalized SaaS governance as the decision authority and task responsibilities related to a SaaS application. Decision authority includes decisions related to changes to the application, SaaS expenditures, and architecture. Task responsibility covers the implementation of changes and first and second level support. These two constructs were measured using interview questions. The questions were formulated based on Winkler and Brown’s (2013) survey questions. These can also be consulted in appendix A. With the answers to these questions, we were able to determine the mode of application governance in the organization. In our questions, we placed a special emphasis on outsourcing in order to find out whether organizations can outsource a part of their SaaS governance and to what extent this might be beneficial.

5.4.3.1. SaaS Governance Effectiveness

To measure the effectiveness of the current SaaS governance arrangements, an accurate tool is needed. Because we argue SaaS governance is a more specific form of IT governance (i.e. IT governance aimed at a single application), we can apply ways of assessing IT governance to this form of governance too. In the literature study above, we have identified some of the most popular methods of measuring IT governance effectiveness. The methods of Weill and Ross (2004) and Bowen, Cheung, and Rohde (2007) seemed most suited for measuring the effectiveness of governance arrangements. In our study, we adapted the methods of Weill and Ross (2004) and Bowen, Cheung, and Rohde (2007) in order for them to measure the more specific application governance effectiveness. An extra question was added as a way to check the validity of this method of measuring SaaS governance effectiveness. The above-mentioned questions can be viewed in appendix A. To calculate the governance effectiveness, we used the following formula (Weill & Ross, 2004):

SaaS Governance effectiveness = ∑ (Q1 ×Q2)×100

5 1

∑ (Q1 × 5)5 1

Here, Q1 represents the values retrieved from the first set of questions, and Q2 stands for the values retrieved from the second set of questions (see appendix A). The value retrieved from this formula is a governance effectiveness score between 20 and 100, with 20 being the lowest possible score and 100 being a perfect score. According to Weill and Ross (2004), the mean score that companies had on IT governance effectiveness was 69 with the top one third of enterprises scoring over 74. In our research, the average score of Weill and Ross (2004) is solely used as a rough guide of determining the SaaS governance effectiveness of a company, as we believe these numbers cannot directly be translated from IT governance to SaaS governance. The score can, however, be seen as an indication. A very low score probably indicates some problem in the current mode of SaaS governance, whereas a high score indicates a good fit between the

current governance arrangements and the contingencies. An open question was added to determine the overall perceived SaaS governance effectiveness. This was done to validate the use of the Weill and Ross (2004) way of measuring SaaS governance effectiveness. The agreement between the governance effectiveness score and the answer to the open question provide an indication of the validity of how well the Weill and Ross (2004) method translates to SaaS governance. Here we must also add a critical note, as a confirmation bias might be present when the interviewees were asked about their opinion on the overall SaaS governance effectiveness. This point will be discussed further in section 7.3.2.

5.5. Data Collection

Data was collected from four different organizations. General characteristics of the organizations, such the size and industry, were gathered through company document analysis and through other news-sources. We aimed to interview one employee from the business and one person from the IT department per case. This was done so different perspectives on governance in the same organization could be examined. Business staff may have different ideas and views on governance than the IT department, and by interviewing we could also find a possible misfit between business and IT perspectives.

In total, seven interviews were conducted. Each interview was conducted at the company’s main office in The Netherlands and recorded using an audio device. Each interviewee was asked for their position in the organization, their On Demand knowledge level, and their role towards the use of On Demand. A summary of the interviewee’s characteristics can be seen in table 5 below. Case B did not have an IT representative that had any knowledge of CRM On Demand, so two interviewees from the business with CRM experience were chosen instead. Also, in case D there was not enough time available to plan an interview with a business representative.

Table 5. A summary of the characteristics of the interviewees per case. Legend: A = Administrator, C = Configurator, D =

Developer, O = Owner, and U = User.

Case A Case B Case C Case D

I1 I2 I3 I4 I5 I6 I7

Position IT Business Business Business Business IT IT

On Demand knowledge

High Medium High Low High Low Medium

Role in On Demand

A/C/D/U A/C/U A/C/U U A/O/U O A

5.6. Data Analysis 5.6.1. Coding

After the interviews were taken, the audio recordings were transcribed, following common transcription practices (McLellan, MacQueen, & Neidig, 2003). Then, we created a codebook mainly based on concepts from literature (DeCuir-Gunby, Marshall, & McCulloch, 2011), but also based on our findings. We coded each transcript by research variable, but also coded other unforeseen and potentially significant findings. Coding was done using qualitative data analysis software1_{. In the codebook, we also included a way of}

assessing the induced mode of governance. The codebook can be viewed in appendix C.

5.6.2. Rating the Variables

In this research, it was very important to rate the contingencies and other aspects accurately according to the answers provided by the interviewees, because these ratings were needed to answer our research questions. Also, each rating could potentially change the outcome of the study, so this is a crucial aspect to our research. For an overview of how each variable was rated and analyzed, refer to appendix B.

5.6.2.1. Absorptive Capacity

IT business knowledge and business IT knowledge were rated on a scale of low, medium, and high. A low rating would typically include a negative response by an interviewee, while a high rating would be a very positive answer. For the absorptive capacities, the answers of each interviewee were combined to form the general case rating (see appendix B). The combined ratings were on a five-point scale (low, low-medium,