Web Analytics

(1)

Strategy & Innovation

Web Analytics

Institutionalizing Analytical Maturity

Coordinator: Thijs Broekhuizen Year: 2008 / 2009

Supervisor: Rene van der Eijk Semester: 2.2

Company Supervisor: Remi van Beekum Date: 2 February 2010

2nd_{Company Supervisor: Janco Klijnstra} _Version: _1.01

Module: Master Thesis Student nr. s1747894

(2)

Web Analytics: Institutionalizing Analytical Maturity ₂

Abstract

This thesis investigates what the critical success factors are for institutionalizing a new governance process within an organization. This thesis is specifically focused on the increasingly popular activity of analyzing internet data with the goal of optimizing a firms online channel. This management process is called web analytics and in this thesis its developments and the aspects influencing its success will be discussed. Nine variables influencing the social embeddedness on web analytics, and thus its success, were identified using a literature study and interviews with internal web analytics specialists. Using a data sample, comprising of 186 different web analytics data users, multiple regression analyses are performed. The statistical evidence of this research suggests, that propagating the relative advantages of web analytics to employees in organizations is the most important segment for web analytics policy makers to focus on. This statement is backed up with the survey findings that web analytics users still perceive that their organization are stuck in using traditional offline marketing methods on the online channel.

Mark van Kasteren

(3)

Web Analytics: Institutionalizing Analytical Maturity ₃

Preface

This thesis forms the conclusion of my MscBa Strategy and Innovation at the University of Groningen. The last six months, I have studied the rapid innovative developments in the field of web analytics. For all this time one slide of the master thesis instructions presentation echoed in the back of my mind. In this last slide of the presentation it showed the following text:

“Writing your master-thesis should be fun...”

Surely, while writing this preface I am indeed having fun. Of course this is because I am currently under the impression that I am close in receiving my diploma (hurray!). Yet, I cannot neglect the times when I was reading a book and felt guilty it was not about web analytics. Or all the hours I spend behind a monitor for my thesis but did no wrote a single word. All and all, I could not have completed this thesis without the help of a number of people who kept me motivated for the realization of it. First of all, I would like to thank everyone at Traffic4u for the great six months I had during my internship. From the start of my internship, everyone made me feel completely at ease at this company and enthusiastically provided me with useful insights and perspectives from their practical experience. My special thanks go out to my company supervisors Remi van Beekum and Janco Klijnstra who helped me during the process of writing, by reading my analyses and providing feedback and practical insights that helped me improve the quality of my work. Moreover, I would like to thank my supervisor at the University Rene van der Eijk, who helped me focus on the structure of my research and provided me with many useful comments on my written chapters as well. Also, I would like to thank all of my interviewees for making time for the face to face interviews on this topic. It has been great to use your personal experiences on the growth of web analytics in your organizations in my thesis. I would also I would like to thank the weblogs who were kind enough to publish my article and survey and are thus responsible for the gross of its respondents. In addition I would like to thank all of the survey respondents, especially those who left all those motivating comments on the website (which my thesis evaluators can find on the last page!).

(4)

Web Analytics: Institutionalizing Analytical Maturity ₄

1. INTRODUCTION

“If there are stormy waters ahead, what are you going to do? Batten down the hatches and hope for the best? Or invest in the right navigation equipment, learn how to use it, and plot

the smoothest possible course to keep ahead of the pack?”

(Recession Looming: Analytics to the Rescue, ClickZ, 2008)1

For most companies the web has really “grown up” as a business channel. And suddenly there is a deep demand for the web channel to be held just as accountable as any other business channel. Since the boom and bust on the web, there has been ever-increasing scrutiny and companies are demanding that the online channel justifies the investments being poured into the channel. For these companies web analytics enters the story. Web analytics refers to the collection, analysis and reporting of website usage by visitors and customers of a website. This information is used by those responsible for the success of the website to better understand the effectiveness of online initiatives and other changes to the website. This is done in an objective scientific way through experimentation, testing, and measurement. Understanding and knowledge from the data can be used wisely to optimize the website, so that it is more effectively accomplishing the goals of the business. The optimization process can occur in any number of areas such as site content and media offerings, product and merchandising, site navigation, creative design, internal search and the checkout process. 1.1 The Web Analytics Practice

As the brief preceding brief introduction indicates there are many facets of the web analytics practice, which makes defining it very challenging. The first official definition of web analytics was not defined until 2006 (Kaushik, 2006). Which is rather odd considering that the dawn of web analytics already occurred in the early 90’s (p.2)? The definition was created by the newly born web analytics association. Officially the Web Analytics Association defines the activity as;

“Web analytics is the measurement, collection, analysis and reporting of Internet data for the purposes of understanding and optimizing web usage”

1

(7)

Web Analytics: Institutionalizing Analytical Maturity ₇

Several web analytics experts have argued on the integrality of this definition, since the term ‘web usage’ due to its vagueness, is asking for a definition itself. That’s why this research will follow the simple and relatively broad definition formulated by Dave Chaffey (2004);

"Web analytics is the customer-centered evaluation of the effectiveness of Internet-based marketing in order to improve the business contribution of online channels to an

organization." 1.2 Problem Statement

According to (Kaushik, 2009, p.1) for far too long online efforts have accurately been classified as faith-based initiatives. The cause of this is that when organizations moved online they duplicated there offline decision practices to their online efforts. But on the world of the web, organizations don’t have to rely on faith, because the web allows organizations to collect a wide variety of data (quantitative, qualitative and competitive). This data can be used by firms to determine how to market effectively, how to truly connect with their audiences, how to improve the customer experience on their website, how to invest meager recourses and how to improve their return on investment.

Yet, remarkably, many organizations have mixed feelings about the use of web analytics. On the one hand organizations members understand the need of gathering online data. On the other hand many organization members value the actual added value of the data as insufficient2. This is shown for example by research done by Clickvalue on the use of web statistics. (Markus, 2008, p.19). Acquiring the data is one thing, but finding the insights followed by the appropriate actions is the challenge. Research of Davenport (2006), and expert opinions like the one of Kaushik (2006), show that in many organizations where the online channel does play a crucial role in day to day business, they are still far from calling themselves analytical mature. According to Kaushik (2009, p.2) the paradox of all that web data is that there are fundamental barriers in making intelligent decisions. For example, the results of the recent ‘Online Measurement & Strategy’ report of eConsultancy (2009). Show that “insufficient management support”, “lack of strategy”, “isolated in the organization” and “HIPPO-opinion (Highest paid person’s opinion)” are the most popular obstacles facing web

(8)

Web Analytics: Institutionalizing Analytical Maturity ₈

analytics today. Supporting the argument that online decision making can still be classified as being too much faith-based.

The interesting developments in web analytics and the role of the Web form the principal motives to further explore this topic. Again, remarkably, little to none scientific research has been done in finding the solution of this problem. Objective of this thesis is to provide additional insights into the development of (online) analytical success. Furthermore, it aims to explore whether, and to what extent, the social system surrounding web analytics is related to its success. The following research question is formulated.

What are the critical success factors for institutionalizing analytical success within an organization?

In order to answer this main question, the following sub questions are formulated:

1) How has web analytics and its role within organization been developing in the last ten years?

2) How can the web analytics activity be classified best according to existing literature? 3) What potential influences on web analytics success can be derived from existing literature

and practice specialists?

4) To what extent does a significant relationship between the potential influences derived from the literature/practice and the level of analytical success exist?

1.3 Research Approach

The first three questions are answered by performing a literary study and interviews with internal web analytics specialists. The literature study consisted in the search and consulting of already existing literature (books, reports, journal articles) on the subject. The advantage is that it is an easy way to gather background information and learning’s from other studies. The disadvantage is that the data most of time was collected for other purposes meanings (Schreuder Peters, 2005, p.53). Because there is not much scientific literature on web analytics this study will only use can only use a literature study as an exploratory tool and try to compare web analytics with literature on equal like of activities (like business intelligence).

(9)

Web Analytics: Institutionalizing Analytical Maturity ₉

method of open oral interviews and was used as part of the explorative study of this research. The base of this type of interviews will be a list of subjects, brief explanations and open questions. This method holds several advantages and disadvantages. Besides the labor intensity, the method does not suggest specific aspects or answers. This allows the researcher to probe for further questions, but also makes the data analysis harder for the researcher since the answers can have different meanings (Schreuder Peters, 2005, p.44). In this research use was made of five internal web analytics Specialists from different companies (Wehkamp.nl, Bol.com, SNS Bank, DSB Bank, Centraal Beheer Achmea) to identify the problem area and developing the conceptual model.

The empirical research that follows intends to answer sub question four. This empirical part serves as a reflection on the established expectations in designed conceptual model. This is done by a written questionnaire that is unaccompanied by the researcher. Advantages of this method are that the respondent’s anonymousness can be guaranteed and it is ideal way to test a theory. Disadvantages of this method are that it offers little possibilities for further in depth questioning, there is no control and it is often accompanied by a low respondent response (p. 143).

1.4 Contribution

This research will not only contribute to the (still far too little) scientific research on data-based-based initiatives but will also contribute to the already existing theories on institutionalizing innovations within a social environment. The research will focus in specific on those factors that an actor within an organization can have influence on, thus in general the non-technical factors in regard to web analytics. The research results will only be relevant for companies that have a clearly formulated (or are aware of) an evaluation process on the effectiveness of their online (marketing) channel(s).

1.5 Reading Guide

The formulated research questions also give an overview of the structure of the paper. Chapter 2 gives a clear explanation of web analytics development is given. And also with which other practices it is equivalent with.

(10)

Web Analytics: Institutionalizing Analytical Maturity ₁₀

Chapter 4 describes in detail how the variables for the empirical part of the thesis are constructed.

Chapter 5 consists out of an explanation of the research methodologie that will be used in the empirical part of this study

Chapter 6 will give an overview of the results of the empirical analyses. Chapter 7 will provide recommendations for web analytics practitioners and recommendations for further research.

(11)

Web Analytics: Institutionalizing Analytical Maturity ₁₁

2. THE DEVELOPMENT OF WEB ANALYTICS

In order to explain the role that web analytics can play in organizations, this chapter pays attention to the development. Web analytics has experienced, and still in fact is experiencing, important institutional changes. These changes have had a substantial influence on the entry thresholds for companies to start with web analytics. This chapter gives attention to all of the development web analytics had undertook in the last decade. First web analytics developments and there influences will be described. Then, in order to compare web analytics with other activities, the general classification of the process will discussed. Lastly the general web analytics maturity model will be outlined.

2.1 Web Analytics Development

(12)

Web Analytics: Institutionalizing Analytical Maturity ₁₂

Google had a major effect on the web analytics landscape in 2006 when it released its free tool, called Google Analytics. With the entry of Google Analytics, the market had simply exploded, because no anyone who wanted to have access to data from their website can have it for free (assuming that companies don’t have problems that the data isn’t stored in their own management). The pace of innovation in the web analytics world continues with newer and easier ways to visualize complex sets of data from site interaction. Innovations like click density heat maps, intelligence alerts and newer ways to segment data continue to be released into the industry. Although now most people think that web analytics is just clickstream data. In reality, clickstream data is just a proportion of web data available. Although clickstream can generate a lot of reports, it won’t tell companies what they should do (no insights).

Nowadays there is more data than ever available. Take for example A/B or Multivariate testing, which is a statistical significant way to determine which elements on a page are helping the performance of a web page, and which are not. Also qualitative data (usability, surveys, direct observation) gives you information consumer experience. More info on these practices can be found in Appendix Q. This expansion of web analytics means that companies have a significantly enhanced ability to listen to their website customers. According to Kaushik (2008, p.6) web analytics is metaphorically speaking a toddler. In the last five years the web analytics toddler had seen a rapid growth spurt as a result of growing importance of the online channel. The growth portrayed itself with the start of the professionalization of the web analytics specialist’s field and new (free) tools and functionalities. Yet, there is still a lot of growth and changes in front of web analytics before it can call himself a full-grown man (or woman).

2.2 Classifying Web Analytics

2.2.1 The Web Analytics Innovation

(13)

Web Analytics: Institutionalizing Analytical Maturity ₁₃

experiences or the introduction of a new style of organization. In contrast, technical innovations lead to new technical appliances (p. 21).” Although web analytics is dependable on its technical software and its developments, in its core it is a non-technical innovation. The technical web analytics software, like Google Analytics, is only used as a tool to measure and analyze what is going on with the online channel.

Jacobs (2006) also makes other distinguishes in the field of innovation. He distinguishes between product/service innovations, process innovations and transaction innovations. At first it may seem that web analytics is balancing between being classified as a process innovation or a service innovation. So it is interesting to zoom in both definitions. According to Jacobs (2006) process innovations are changes in the production processes of products (including services) which in principle should lead to more efficient production, not only within factories and service organizations but also between organizations by organizing procedures efficient and effectively. Like product innovation, process innovation has for a long time been seen as mainly technical: primarily involving forms of automation which lead to higher levels of productivity. However, more often than not the overemphasizing of technology has led to missed opportunities because new process technologies were not aligned with the organizational processes of firms (Jacobs, p.43). This problem is exactly in line with the research problem of this paper. It can be confusing to classify web analytics as a process innovation, since a sub goal of web analytics is to change the design of existing services and thus is arguably a service innovation. Nonetheless these improved services are only based upon a web analytic process, and thus a process innovation.

2.2.2 The Web Analytics Process

(14)

Web Analytics: Institutionalizing Analytical Maturity ₁₄

processes provide people and means like information, facilities, (housing) and finance. Web analytics can be classified as a governance process which is line with Eric Petersons call to see web analytics as an continuous improvement process;

“Fundamental to the continuous online improvement process is the notion that no changes are made on the website without a reason for making the changes and expectations about what effect the changes should produce.” (Peterson, 2004)

The practice causes a constant stream of information requests; thus the governance process is never ‘done’. All steps that were used into classifying web analytics are illustrated in figure 1.

Figure 1. Classifying Web Analytics

(15)

Web Analytics: Institutionalizing Analytical Maturity ₁₅

performance of the organization is called Business Performance Management (BPM). That’s why this study will categorize web analytics as a kind of BPM process.

2.3 Web Analytics Maturity Model

Maturity models have in recent years expanded enormously. The most famous one is the Software Capability Maturity Model (CMM) which focuses specifically on the software development. The purpose of a maturity model is to give the organization handling points in order to be spoken of a controlled process (Hermarij, 2007, p.1).

Many experts on the area of web analytics assign the use of the analytics activity also into a number of maturity phases. Figure 2 shows a web analytics maturity model designed by Bill Gasman (2008) who is an industry analyst at Gartner. In this model phase 0 “Ad Hoc” reflects a situation were just one or two employees are experimenting with web analytics within an organization. The goal of phase 1 “Control” is measuring for the sake of measuring in order to keep control on the online channel. In this phase web analytics is only used to measure metrics like for instance: number of page views/visits, top 10 pages visited and the demographic characteristics of visitors.

In phase 2 “Optimizing” web analytics is already being used in optimizing several conversion processes. This can be done with the help of tools that analyze the path consumers take on a website. For instance with the help of transaction funnels (standard e-commerce paths) that can analyze where consumers step in a transaction process or leave the transaction process. Other methods that can be used are A/B testing or Multi-variant testing. These methods test several variants of website content online to see which one scores better. The goal of this phase is to find out what consumers find to be the general thresholds on a website.

(16)

Web Analytics: Institutionalizing Analytical Maturity ₁₆

Figure 2. Web Analytics maturity levels

Of course not every organization has the goal to obtain the highest web analytics level, simply because it is not profitable for an organization since the costs way higher than the anticipated benefits. Mostly this is the case in smaller organizations or for organizations where strategic goals are not depended on the performance of the online channel. However, research of Davenport (2006), Clickvalue (2008) and expert opinions like Kaushik (2006), show that many businesses in which the online channel does play a crucial role are still far from an optimal integration of web analytics. This notion is illustrated by the blue dotted line in figure 2.

2.4 Common Web Analytics Obstacles

Like mentioned in the previous paragraph most organizations are in a situation where the use of web analytics is still far from it’s most effective and efficient usage. How is it possible then that many organizations are still stuck in the first two phases of web analytics? The following web analytics obstacle list was formulated with a brainstorm session with Traffic4u web analytics consultants (2009), and the results of the recent “Online Measurement & Strategy“ report of eConsultancy (2009).

The most common web analytics obstacles are; 1. Stuck in traditional thinking

2. Lack of web analytics strategy 3. Lack of recourses

(17)

Web Analytics: Institutionalizing Analytical Maturity ₁₇

The stuck in traditional thinking blockade can be viewed from two perspectives. One of them is that for a lot of companies the online channel has grown the last years from a communication channel to a full grown important business channel in a relative short time. In companies, like Wehkamp and Achmea, traditionally the product guide was the most important business channel. However, these days the website has grown to be their most valuable business channel. The change in the relative business channel importance also requires a different mindset of organization members and allocation of recourses.

Lack of strategy and vision is a popular problem with IT related projects. Most of time the

answer is already fought to be found into an IT solution while the problem is not clearly defined yet. To cope with a problem people use a tool, but when there are not any qualified people to actually use the tool, the problem stays uncured. At the foundation of the web analytics process there is often no clear vision for it, any growth occurs ad hoc. Without a goal there is no sense of direction and challenges of the process. Traffic4u web analytics consultant André Scholten (2009) believes that in order for the web Analytic process to grow within organizations it is important to have identified the current web analytics state and have clear vision of the stadium where the web Analytic process should be within for instance a year. There is a need, if you will, for a clear roadmap.

Kaushik (2009) believes that the lack of recourses can be blamed for the web analytics process having too little legitimacy within the organization. If this is the case the web analytics practitioner within an organization should earn his right for asking budget. This can be done for instance by using free tools like Yahoo Analytics and Google Analytics (for website statistics) and Google Website Optimizer (for multi-variant testing). 3

Lot internal web analytics practitioners feel they are still somewhat isolated in the organization because of the lack of cooperation with other departments. They envision a customer approach where information from all the business channels is getting combined together in order to learn lessons from the paths that consumers take in becoming a customer. Instead the web analytic process is most of the time added to the existing practices of

3

(18)

Web Analytics: Institutionalizing Analytical Maturity ₁₈

someone responsible of the website. Then, providing the web analytics practitioner is successful, a formal website improvement process is created. However, the process still stays fairly isolated and web analyst practitioners find it extremely difficult to grow into a more multidisciplinary approach.

It is safe to say that most of the employees (that take decisions on the firm’s online channel) are unknown with the technical definitions of web analytics terms like bounce rate and session. Even worse is the case when relevant decision takers within an organization are overwhelmed with for them a big amount of data. Kaushik (2009) believes that the cause of this may lay in hands of the web analytics practitioners who get ‘metric-happy’ and thus create reports with an overflow of data. Thus not making the reports of the online channel

actionable enough. 4

According to Kaushik it is not unnatural that organizations have problems in finding

experienced staff. Web analytics experts with 10 years experience are diamonds in the rough

for an industry that really has grown since the last 5 years. So Kaushik advices to find experts in similar fields like Business Intelligence.

4

(19)

Web Analytics: Institutionalizing Analytical Maturity ₁₉

3. THEORETICAL FRAMEWORK

In this chapter, the existing literature on institutionalizing analytical success (or an innovation) will be discussed. Because existing literature on this specific subject is limited, especially in the case of web analytics, other sources of information were used in composing this literature framework. Such as expert interviews with internal web analytics specialists and literature on corresponding web analytics process as indentified in §3.2.2.

3.1 Institutionalizing and Adoption of Innovation: Prevailing Theories

This section discusses the prevailing theories and researched determinants that influence the adoption rate of a new innovation.

3.1.1 Social Embeddedness theory

A successful web analytics practice requires a wide user acceptance trough out (relevant) departments within an organization (Kaushik, 2009, p.25). How some practices are adopted widely, while others, equally plausible alternatives, languish is a question that has troubled researchers for some time (Tushman and Anderson 1986; Utterback 1994). Traditional explanations on the adoption of innovations are according to Lawrence and Philips (2004) too much focused on functionality and price. However these variables neglect what researchers call the ‘social embeddedness’. In simpler terms, this means that in interpreting an innovation actors largely accept them by a set of terms in understanding available to them. In his book “Adding Values” Jacobs argues that the success of innovations is more about the non-technical aspect (norms, values and expectations) then the technical aspect (2007, p.13). Several researchers like Jacobs have compared this phenomenon with the Darwinian perspective ‘survival of the fittest’; where fitness means fitting into a certain environment, a certain ‘ecosystem’ (p.77). So when we applying this framework to the web analytics innovation in order to enhance ‘web analytics success’, we have to address the question of what ‘fitness’ means in the organizational environment.

3.1.2 Rate of Adoption theory

(20)

Web Analytics: Institutionalizing Analytical Maturity ₂₀

These are (1) relative advantage, (2) compatibility, (3) complexity, (4) trialability and (5) observability (Rodgers, 2005, p.221). In addition to these five attributes, there are other variables that affect an innovations rate of adoption. The type of innovation decision, the nature of communication channels diffusing the innovation, the nature of the social system in which the innovation is diffusing, and the extent of change agents’ promotion efforts in diffusing the innovation. This theory is visualized in figure 3.

Figure 3. Variables determining the Rate of Adoption Innovations (Rodgers, 2005, p. 222)

(21)

Web Analytics: Institutionalizing Analytical Maturity ₂₁

an innovation’s rate of adoption is also affected by the extent of change agent’s promotion efforts. This key segment is in line with the institutional entrepreneur theory that will be discussed in the §4.2.1.

Criticism on the Adoption Theory

Although the theory as part as Rodgers ‘Diffusion Theory’ has received much praise in the past, there are some criticism to be found. Criticisms on theories are necessary after all “The prejudice of training is always a certain ‘trained incapacity’: The more we know about how to do something, the harder it is to learn how to do it differently”. (Kaplan 1964, p.31). There are two criticism that even Rodgers in his book “Diffusion of Innovations” recognizes and warns researchers of his theory for (2005, p.105); These criticisms are (1) the pro-innovation bias, which is the implication that an innovation should be adopted (rapidly) by all members of a social system, and that the innovation should be neither re-invented nor rejected (p.106). The (2) individual blame bias is a tendency for research on innovation to side with the change agencies that promote innovations, rather than with the individuals who are the potential adopters (p.118). The (3) recall problem is the problem that respondents can’t accurate remember the time they officially adopted a new practice. All and all several pitfalls researchers on this top should keep in the back of their mind.

3.1.3 Institutional Entrepreneur theory

The adoption theory explained above, recognized five qualities that are an important explanation of the rate of adoption of an innovation. One of them was the extent of change agents’ promotion efforts in diffusing the innovation. The institutional entrepreneur theory can be considered as a complement of this determent of Rodgers. Since it’s trying to explain how actors can try to alter the social ‘understandings’ or ‘scripts’ of an innovation to their advantage. Al and al of course in order to enhance the chance of a new practice or technology being adopted. Actors that try to alter these ‘scripts’ (an individual actor, a project team or higher management) are called institutional entrepreneurs (Barley and Tolbert, 1997).

According to Dejan et al. (2004: 743) the precise definition on institutional entrepreneurs is:

(22)

Web Analytics: Institutionalizing Analytical Maturity ₂₂

The parent thought behind this theory is that the more the innovation is accepted by its users, the more successful it becomes. But how do intuitional entrepreneurs change the ‘social embeddedness’ in practice? A famous case study on institutional entrepreneurship is the study by Munir and Philips (2005) on the Kodak camera. As a initial failure, Kodak successfully managed to transform photography from a highly specialized activity to one integrated in our daily lives. They did this by changing the social embeddedness with around the photography activity in four separate ways.

First Kodak tried to embed the technology in existing practices, since the camera was mostly used by artists in the late 19th century (p.9). By marketing the camera as a fun technology to bring along with vacations. This strategy increasing the legitimacy of the new technology. Another discursive strategy was creating a new role. In 1889 Kodak intervened in the gender divide in photography, making it a legitimate and even ‘required’ activity for women who had previously been excluded from this practice (p.11). The third discursive strategy that appears in Kodak’s efforts at institutional entrepreneurship is the creation of

new institutions at the field level. Through the creation of the ‘photo album’(p.15). The

fourth discursive strategy used by Kodak involved changing the existing institutions at the field they were constituted, by changing the discourses that constituted them. The camera was first a ‘fun’ device, and later became a essential one (p.17). Table 1 gives an overview of the discursive strategies that Kodak used.

Strategy Discursive process Institutional outcome

Embed technology in existing practices

Include interdiscursive references to existing institutions

Increasing the legitimacy of new technology (e.g. redefining ‘vacation’ around the new technology)

Create new roles Constitute new subject positions Make it legitimate for new users to adopt existing technology (i.e. woman should carry cameras) Create new institutions

within the field

Accrete a new discourse that constitutes new objects and concepts

New technologies become institutionalized (i.e. the photo album becomes a part of every home)

Modify existing institutions within the field

Influence existing discourses Existing technologies become understood differently (the camera was first a ‘fun’ device, and later became a essential one)

(23)

Web Analytics: Institutionalizing Analytical Maturity ₂₃

The above discursive strategies demonstrate how an actor, in this case the Kodak company, was able to change the social embeddedness surrounding the portable photo camera.

3.2 Theoretical Framework Determents of Web Analytics Success

Like mentioned earlier in §3.1.3 it stands to reason that in order to enhance the chance of web analytics being successful, promoters of a new practice/technology would try to alter the ‘understandings’ on web analytics to their advantage. The factors that influence the web analytics case are identified with the help of two methods; (1) a literature study on comparable activities like web analytics and (2) with the help of five expert interviews with several internal web analytics specialists from several Dutch companies (Wehkamp, SNS Bank, Centraal Beheer Achmea, DSB Bank, Bol.com). These factors are divided into Compatibility, Observability, Relative Advantage, Trialability, Complexity (all of Rodgers’ perceived attributes) and web analytics Change Agent (another Rodgers key segment, §3.1.2). When taking these different elements into account, the following conceptual model is proposed (see figure 2).

(24)

Web Analytics: Institutionalizing Analytical Maturity ₂₄

3.2.1 Critical Success Factors Web Analytics

The idea of identifying critical success factors (CFS) as a basis for determining the recourses needs was first proposed by Daniel (1961) and further popularized by Rockart (1979). The idea is simple: in any organization certain factors will be critical to the success of that organization, in the sense that, if certain requirements are not acquired, the organization will fail.

3.2.1.1 Critical Success Factors Literature

In paragraph 2.2 web analytics was defined as a governing & improving process. Governing processes are considered to be strategic assets of an organization since the help to understand, manage, and improve other processes to deliver value added products and services to the clients. These processes go hand in hand with several holistic management approaches that promote business effectiveness and efficiency while striving for innovation, flexibility, and integration with technology. Examples of such managerial approaches are; Total Quality Management (TQM), Continuous Improvement Process (CI), Six Sigma, Business Process Management and the Balanced Scorecard. These philosophies all fall under the wider category of Business Intelligence or Business Performance Management (Chaffey & Wood, p.14, 2005), just like web analytics. Thus, it is plausible that web analytics will share the same set of critical success factors. It is important to note however that web analytics does differ both in cost and focus with Business Intelligence or Business Performance Management. Because when talking about the costs of implementing a simple Business Intelligence project one talks about the tons, while implementing web analytics can be achieved with only spending a couple of thousands. Secondly Business Intelligence focuses on analyzing data of the business environment and all of the consumer channels. While web analytics is primarily focused on the online channel. Thus, while doing the literary study, web analytics specific requirements were kept in mind.

Regarding the success of a performance management systems several authors have introduced a variety of critical success factors. This study will focus on five of these studies, each recognized by international standards.

Huff and Barki field study in examining the effects of certain factors which influence support system implementation success (1990).

(25)

Web Analytics: Institutionalizing Analytical Maturity ₂₅

Stelzer and Mellis results from a survey regarding key success factors for implementing a software process improvement (2002).

Remus (2007) his literary overview of the critical success factors for implementing enterprise portals.

And Finney and Corbett (2007) article with a compilation and analysis of critical success factors that are related to BPM approaches from 25 other articles.

Table 2 gives an overview of the critical success factors mentioned in the articles5. An ‘empty’ cell indicates that the factor was not studied by the researchers. As the table shows many articles share the same critical success factors.

Critical Success Factor

R em u s (2 0 0 7 ) F in n ey (2 0 0 7 ) F ro li ck (2 0 0 8 ) S te lz er (2 0 0 2 ) H u ff ( 1 9 9 0 ) Management Support/Commitment • • • • • Staff Involvement • • •

Change Agents and Opinion Leaders • • • •

(Encourage) Communication • • •

Accessibility • •

User Skills • • • •

Clear Strategy (link with business strategy) • •

Accountability • •

Process Redesign • • •

Table 2. Critical success factors Business Performance Management

3.2.1.2 Critical Success Factors Interviews

To compliment the critical success factors found in the literature, five expert interviews were held with internal web analytics specialists. The five interviewed experts were;

1. Frederieke van Perlo – web analytics specialist at Wehkamp (Post order retailer) 2. Reinout Wolftert – web analytics specialist at SNS Bank (Bank)

3. Jorien Brangert – web analytics specialist at Bol.com (Online retailer) 4. Mark Hoebe - web analytics specialist at DSB Bank (Bank)

5. Tjaard Prins - web analytics specialist at Centraal Beheer Achmea (Insurer)

5

It is important to note that naming of each critical success factor may differ in each article, however the

(26)

Web Analytics: Institutionalizing Analytical Maturity ₂₆

The addition to add the expert interviews is based on two arguments. First of course because the used papers from the literature review were initially intended for other (yet comparable) practices then web analytics, so it is logical to add a more web analytics minded opinion. A second reason is that in the studies, such as used in the previous paragraph, the main dependent variable of the studies is successful implementation (that is putting the innovation to use) rather than the adoption of the users (managing the innovation).

When putting the critical success factors from the interviews and the literature together the following overview was created (see table 3).

Critical Success Factor In

t e r v ie w s B o l. co m S N S B a n k A ch m e a W e h k a m p D S B B a n k L it e r a tu r e R e m u s (2 0 0 7 ) F in n e y ( 2 0 0 7 ) F ro li ck ( 2 0 0 8 ) S te lz e r (2 0 0 2 ) H u ff ( 1 9 9 0 ) Management Support/Commitment

• • •

• • • • •

Staff Involvement

•

• • •

Change Agents and Opinion Leaders

• • •

• • • •

(Encourage) Communication

Propagation Web Analytics Results

•

• • •

Accessibility

• •

User Skills

• •

• • • •

•

Clear 'Online' Strategy

• •

Accountability

Clear Web Analytics Ownership

• • • •

•

Process Redesign

• •

•

Data-driven Decision Culture

• • •

Table 3. Critical success factors interviews and literature

The interviews with web analytics added several changes to the success factors that will be tested in the empirical part of this study. The most important change is the addition of a new critical success factor that was not discovered in the literature research, ‘data-driven decision culture’. This success factor was added because several specialists (Perlo, 2009; Prins, 2009) indicated that in order for web analytics to become a success it is necessary that the data is seen as legit. And thus people are not reluctant in using data to make decisions.

(27)

Web Analytics: Institutionalizing Analytical Maturity ₂₇

not clear on first sight, it has been changed into the propagation of web analytics results. Another name change was that of accountability into clear web analytics ownership. This change was done for a more clearer clarification that this success factor is focused upon who owns web analytics and who it reports to from a more organization perspective (Kaushik, 2009 II). The literature success factor process redesign was dropped because all of the interviewees indicated that the growth of web analytics within an organization is a slow incremental process instead of radical change process.

3.2.2 Compatibility attributes

Compatibility is the degree to which an innovation is perceived as being consistent with the existing values, past experiences, and needs of potential adopters (Rodgers, 2005, p.240). An idea that is incompatible with the values and norms of a social system will not be adopted as rapidly as an innovation that is compatible. The adoption of an incompatible innovation often requires the prior adoption of a new value system, which is a relatively slow process. Critical success factors that comply with this key segments description are Accountability and Data-Driven Decision Culture.

Decision Culture

In the expert interviews it was often mentioned that users had to make a cultural shift from a gut-feeling decision making process to a more data-drive decision making process. Several experts (Centraal Beheer Achmea, DSB Bank) saw that often this was a growth process. In specific they had to sell the legitimacy of the data they reported. That’s why the second hypothesis suggests that if web analytics users have a more data-driven background, a more pro-active use of web analytics (and thus an increase in its success) would occur.

H1: The level of web analytics success is positively related with a Data-Driven Decision Culture.

Web Analytics Ownership (Accountability)

(28)

Web Analytics: Institutionalizing Analytical Maturity ₂₈

practice. This can for example lower the complexity of users since they have a internal contact person within the organization that they can go to if they have questions.

Concluding, having a clear and structured web analytics ownership established will lead to a higher level of web analytics success.

H2: The level of web analytics success is positively related with a clear perceived web analytics ownership.

3.2.3 Observability attributes

This is the degree to which the result of an innovation is visible to others (Rodgers, 2005, p.248). The easier it is for individuals to see the results of an innovation, the more likely they are to adopt it. Such visibility stimulates peer discussion of a new idea, as friends and neighbors of an adopter often request innovation-evaluation information about it. Critical Success factors that comply with this key segments description are the Propagation of Web Analytics Results and Management Support.

Propagation Web Analytics results

Intensive communication is not only necessary to rectify rumors or to preclude misunderstandings and thus overcoming resistance. It is also necessary to successfully improve the process. Close cooperation of business units will provide natural feedback loops, enhances staff members understanding and knowledge, encourages people to exploit synergy and consequently improves productivity and quality (Stelzer, 1999). For example facilitating communication between business and IT functions leads to mutual understanding of the organizations strategic direction an goals. That way IT is better able to understand the usage of the business metrics so they try to get better in capturing them (Remus, 2007).

Concluding, having more visible web analytics process results will lead to a higher level of web analytics success.

H3: The level of web analytics success is positively related with a perceived web analytics result visibility.

Management Support

(29)

Web Analytics: Institutionalizing Analytical Maturity ₂₉

help sustain the funding of recourses for the entirety of the practice as well as help in the creation and communication of critical metrics of interest to assess performance management (Frolick, Ariyanchandra, 2008).

It is important to note that senior management can also create a pitfall for the web analytics practitioner when agreeing to improvement initiatives without completely realizing the investment required for the effort. Because in some organizations top managers might assume that the initiative will occur without modification of other commitments. As a consequence, the practitioner often finds himself caught in a trap. On the one hand they have to modify the web analytic process, while on the other hand they have to accomplish the ‘project’ without affecting deadlines, budget restrictions, and functional requirements (Wohlwend, Rosenbaum, 1994). Therefore the fourth hypothesis tested is:

H4: The level of web analytics success is positively related with the perceived Management Support.

3.2.4 Relative Advantage Attributes

This is the degree to which an innovation is perceived as better than the idea it supersedes (Rodgers, 2005, p.229). The degree of relative advantage may be measured in economic terms, but social prestige, convenience, and satisfaction are also important factors. It does not matter so much if an innovation has a great deal of objective advantage. What does matter is whether an individual perceives the innovation as advantageous. The greater the perceived relative advantage of an innovation, the more rapid its rate of adoption will be. Critical Success factors that comply with this key segments description are Staff Involvement and Clear Strategy.

Staff Involvement/participation

(30)

Web Analytics: Institutionalizing Analytical Maturity ₃₀

representation by others, (3) formal participation using formal groups/ teams/ mechanisms) and by (4) informal participation through informal relationships, discussions and tasks.

Both participation and involvement are essential in performance management activities because employees must adopt process innovations in their day-to-day activities. This is challenging because process improvement initiatives are often accompanied by rumors, fears and resistance from staff members (Stelzer, 2007;16). It is also necessary, as Oestreich and Webb (1995) point out, to insure that staff members separate accordingly the implementation of a new tool with the whole performance management process. If this is not achieved staff members might see the new tool as just a ‘cool’ gadget and do not commit themselves to the entire performance management process. Concluding, staff involvement/participation is necessary for the success of web analytics in an organization.

H5: The level of web analytics success is positively related with staff involvement with the web analytics process.

Clear Strategy

A common problem with IT projects is that in the IT solution the answer is thought to be found without having a clear idea of what the problem is. A clear perspective of what should be measured and optimized is necessary. Schiff (2004) also advices to stick with the original project goals, since sometimes partial failure can be worse than a total project failure. Because when a system produces enough benefits to keep itself alive a company will learn to life with it far from the ideal ‘picture’ that could be achieved. That said, objectives set should also be realistic. Therefore the sixed hypothesis is set as:

H6: The level of web analytics success is positively related with a clear perceived web analytics strategy.

3.2.5 Trialability

(31)

Web Analytics: Institutionalizing Analytical Maturity ₃₁

Accessibility

In the case of web analytics Accessibility can mean two things. The ability for web analytics users to openly and directly access web analytics data and experiment with the web analytics tools. In order for users to get acquainted with web analytics terms (bouncerate, CTC etcetera) get a better idea of its possibilities. And second generating performance metrics that are tied to strategic value drivers can be challenging. Several experts like Becher (2004) state that metrics should not reflect an organization’s activity, but the outcomes it is trying to achieve. For instance it may be tempting to create a activity metric like percentage of consumer calls returned (Easy data to require from a CRM system). However this metric does not measure the preferred outcome of the organization, namely satisfied customers. This could for instance be achieved by a consumer satisfaction survey. In this case ‘accessibility’ means creating clear and actionable reports. However in this research focus will be laid on the first meaning since it is not possible to test the respondents in a survey if all there metrics are actionable. Therefore the hypothesis set is:

H7: The level of web analytics success is positively related with web analytics tool accessibility.

3.2.6 Complexity Attributes

This is the degree to which an innovation is perceived as difficult to understand and use (Rodgers, 2005, p.257). Some innovations are readily understood by most members of a social system; others are more complicated and will be adopted more slowly. New ideas that are simpler to understand are adopted more rapidly than innovations that require the adopter to develop new skills and understandings.

Since, web analytics prove a completely new user interface together with changed or new processes, it is crucial to train potential users on how the web analytics application works and how its functionality relates to the web analytic process. Often, in complex implementation projects, consultants are involved. In this context, it is important to ensure that knowledge is transferred from the consultants to the internal employees (Remus, p.544). Therefore the 8th hypothesis is set as:

(32)

Web Analytics: Institutionalizing Analytical Maturity ₃₂

3.2.7 Web Analytics Change Agent

This critical success factor emphasizes on the presence of a committed, energetic ‘project champion’. The project champion actively supports and promotes the BPM project often providing information, material resources, and political support. They do this by encourage or initiate improvement projects, request recourses and establish interfaces and communication channels between various groups. Thus the champion often evangelizes the BPM solution selling its value to the entire organization (Frolick, 2008). Concluding, it’s a critical success factor that tries to gain user motivation, management support and helps in creating awareness for web analytics.

H9: The level of web analytics success is positively related with an web analytics change agent.

An overview of the hypothesized influences of all independent variables of the web analytics success level is shown below.

Independent Variable Expected Sign

Compatibility

H1. (Data-driven) Decision Culture +

H2. Web Analytics Ownership +

Observability

H3. Communication +

H4. Management Support +

Relative Advantage

H5. Clear Online Strategy +

H6. Staff Involvement +

Trialability

H7. Accessibility +

Complexity

H8. User Skills +

(33)

Web Analytics: Institutionalizing Analytical Maturity ₃₃

4. VARIABLE CONSTRUCTION

This chapter discusses the variables used in the empirical analysis. The operationalisation of the variables is clarified and justified using existing research on web analytics – and the Diffusion Theory of Rodgers. The dependent variable ‘web analytics success’ is discussed in paragraph 4.1. Then in paragraph 4.2 the independent variables are described.

4.1 Dependent Variable: Web Analytics Success

Like mentioned earlier (§2.3) the purpose of a maturity model is to give the organization handling points in order to be spoken of a controlled process (Hermarij, 2007, p.1). But how do you make the quality of a controlled process measurable? The first phase in implementing a maturity model is to focus on consumer needs (p.7). In the web analytics case the consumers are the web analytics users6. But setting ‘consumer need’ metrics for web analysis is challenging since it is ultimately about doing (i.e. performance management, §3.2) not about creating hard cash. Thus the result of web analytics is largely intangible. However, Judah Philips (2007) has developed a ‘web analytics output’ ratio that is measured with the following metrics:

1. Added value with optimizing (targeted) online advertising (increase revenue).

2. Added value with optimizing landing pages websites and conversion tunnels (improving operations).

3. Added value with understanding the online user experience with the information architecture (reducing costs).

4. Added value with understanding of site traffic, visitor activity, conversions, and online-value generation (improving operations).

5. Added value by understanding of organic and paid search (reducing costs).

The metrics match the three main goals of web analytics which are; (1) to increase revenue with web analytics, (2) decrease costs with web analytics (since it helps companies to spend less money), and by (3) improving operations (web analytics helps you work smarter and more efficiently).

(34)

Web Analytics: Institutionalizing Analytical Maturity ₃₄

4.1 Independent Variables

This section will give an overview on how the independent variables will be measured. The first variable that will be specified is web analytics user skills. The web analytics user skills variable is inspired by the one Daniel Markus used in his 2008 study on the Dutch web analytics market. This web analytics user skills ratio consists out of training credentials, years of experience, perceived usability of the tools and perceived clearness of the web analytics reports.

All other variables falling under Rodgers his perceived attributes will be measured on how they are ‘perceived’ by the web analytics user. After all Rodger himself states that, although economic terms are possible, social prestige factors, convenience and satisfaction are equal important variables (Rodgers, p.15). This is because it does not matter so much whether an innovation has a great deal of “objective” advantage. What does matter is whether an individual perceives the innovation as advantageous. As mentioned earlier because this is a study on a specific case (the web analytics case), several critical success factors from the literature study are therefore operationalized in with that in mind. Accountability for instance was already operationalized in how clear the ownership of web analytics is within an organization. And communication was changed in the way the user perceives the propagation of web analytics results within an organization. Management support, (Clearness) Online Strategy and Staff Involvement (with web analytics) are also measured in the way they are perceived by the web analytics users.

(35)

Web Analytics: Institutionalizing Analytical Maturity ₃₅

5. METHODOLOGY

In this chapter, the data collection method in the research is discussed and the composition of the data sample is clarified.

5.1 Method of Data Collection

In this section the data sample for the empirical part of this research is explained. But first the population and the data collection methods of the research is described.

5.1.1 Population definition

The target group of this research is the users of web analytics. Web analytics users are organizational employees that are responsible for the online marketing and/or the firm’s website. Because this populations experiences the new activity of web analytics, the population is the ideal target group to find out what for them are the ideal circumstances to fully accept and web analytics. Most likely job descriptions of people in this target group are digital marketers, internal web analytics specialists and online content specialists. This population however comes with a problem, since at the time of the this study there is no hard data available of the size of this population in the Netherlands. External web analysts have been deliberately left out the population in order not to let the results be polluted.

5.1.2 Sampling Method

The method of data collection is a questionnaire that is send out to the population. The questionnaire was distributed on four popular websites within the population namely

www.marketingfacts.nl, www.frankwatching.com, www.contentgirls.nl and

www.webanlisten.nl. In general this means the research will use a an simple random sample (SRS) of a given size. Since all the respondents are given an equal response probability. This should minimize bias and simplify the analysis of results. However, it is important to note that people who visits these websites do so in order to keep up to date on all of the latest trends in online marketing. Although it is hard to determine how this will affect the study results, potential respondents from 30 listed AEX businesses will be contacted directly before putting the survey on these websites.

5.2 Data sample

(36)

Web Analytics: Institutionalizing Analytical Maturity ₃₆

5.2.1 Descriptive statistics

First, the total descriptive statistics of all of the variables will be examined. This is to get insight in how each of the variables is divided on the 5-point ordinal scale. The descriptive statistics are based on a number of 183 respondents.

5.2.2 Multiple regression analysis

Based on the constructed variables in the previous chapter, the multiple regression analysis will be performed using the following equation:

WASucces = β0 + β1(StaffInvolvementi) + β2(UserSkillsi) + β3(ManagementSupporti) + β4(Ownershipi) + β5(OnlineStrategyi) + β6(CommunicationResultsi) +

β7(Accountabilityi) + β8(DecisionCulturei) + β9(ChangeAgenti) + εi

Here, β0 is the unknown intercept (i.e. constant), β++ denotes the unknown parameter for the specific independent variable, εi is a standard error term and ‘i’ refers to one of the respondents taken into account. In order to examine the relative explanatory power of the perceived attributes of an innovation, these five categories will run a multiple regression analysis separately. This allows us to examine the relative influence of the different key constructs.

5.2.2.1 Model specifications

When performing a multiple regression analysis, a number of assumptions are made that have to be satisfied in order to adequately perform this analysis.

Correlation

First there needs to be a check for multicollinearity. Multicollinearity exists when two or more of the independent variables used in the analyses are highly correlated. The problem with a high correlation is that it becomes difficult to determine the influence of the independent variables of interest on the dependent variable. Following Pallant (2007), a correlation coefficient of 0.7 (regardless being negative or positive) is usually considered to be too excessive and leads to exclusion of one of the two correlating variables. In appendix F the correlation matrix of the independent variables is shown7. As can be concluded from the matrix, the two independent variables highly correlating are the variables staff involvement and change agent, The correlation coefficient is measured at 0.552. This high correlation seems quite reasonable. After all the goal of the web analytics change agent is to get the staff

7

Since all of the independent variables are of ordinal scale use was made of the Spearman correlation

Web Analytics

Web Analytics

Institutionalizing Analytical Maturity

Abstract

Preface

TABLE OF CONTENTS

1. INTRODUCTION

2. THE DEVELOPMENT OF WEB ANALYTICS

3. THEORETICAL FRAMEWORK

•

• •

• • • • •

•

•

•

• •

•

• •

•

• • •

•

•

•

• •

• •

• •

•

• • •

•

• •

• •

• • • •

•

•

• •

•

• • •

4. VARIABLE CONSTRUCTION

5. METHODOLOGY