Assessing Digital Marketing Performance: How to use online metrics to evaluate digital marketing initiatives

(1)

University of Amsterdam

Faculty of Sciences

Assessing Digital Marketing Performance:

How to use online metrics to evaluate digital

marketing initiatives

Pedro Henrique Resende de Lacerda

11127783

Supervisor

Mieke Kleppe

Submitted in fulfilment of the requirements for the degree of

Master of Science in Business Information Systems

of the University of Amsterdam, 08 of July of 2016

(2)

(3)

Abstract

Currently, marketers can reach and engage with actual and potential consumers in a large scale using personalised messages through many digital platforms as social media and websites. These platforms provide them a huge amount of metrics that can be used to assess all aspects of a digital marketing campaign. However, it also makes the task of evaluating the perfor-mance of a campaign rather complicated and inaccurate due to the complexity of the metrics available. Three studies were conducted in order to understand digital marketing metrics that can depict a company‘s performance. A literature review on the metrics being used across industries. Interviews with digital marketers, to understand how they assess their campaigns performance and which metric they use. And a survey, to evaluate whether the practises and metrics highlighted on the literature review and the interviews were, in fact, being applied. Results from literature review and interviews indicated that there is no ”one size fits all” for metrics. The industry, geographical location, brand maturity, and category of product impact the selection and evaluation of digital marketing metrics. Also, these metrics need to be aligned with the business KPIs, enabling marketers to assess whether their campaigns are performing well according to the business requirements. Therefore, digital marketing professionals should define a measurement framework aligned with the campaign’s goals to select and monitor better metrics, have accurate and meaningful information, and be able to make better decisions on their communication strategies.

(4)

(5)

Acknowledgements

First of all, I would like to thank my supervisor, Mieke Kleppe, for her insightful guidance and constant support throughout this process. She is a big responsible for this achievement and taking this project to a higher level. I would like to thank all HEINEKEN team for giving me the opportunity to work in such a high performance company with some of the best minds in the world. Thank you all for the awesome drinks we had together! Special thanks to my supervisor, Matthijs van Elk, who gave not only the tools to work, but also the best insights and advices I could have. Thanks also to the best friends this journey could give me, the lovely Brazilians who took over my house as it was theirs. You are the best. Last, and most important, I would like to thank my family, that is my foundation and inspiration. I could feel all your love crossing the ocean and embracing me here. This is for you four!

(6)

(7)

List of Tables

1.1 Problems with online metrics . . . 3

2.1 Classification of Social Media . . . 12

3.1 Metrics categories . . . 23

A.1 Category of marketing metrics . . . 49

A.2 Digital and social media metrics . . . 51

A.3 Online metrics . . . 56

B.1 Interview Structure . . . 63

(12)

(13)

List of Figures

2.1 Which of these channels is your marketing team likely to invest? . . . 10

2.2 Total media spending worldwide . . . 11

2.3 Customer life cycle . . . 17

2.4 Marketing metrics continuum . . . 18

3.1 Digital marketing metrics dimensions . . . 25

5.1 Frequency of checking and reporting metrics . . . 35

A.1 Social media metrics framework . . . 57

A.2 Social media metrics 1 . . . 58

A.3 Social media metrics 2 . . . 59

(14)

(15)

Chapter 1 Introduction

1.1 Motivation

It is estimated that by the end of 2016, 46% of the global population will have access to Internet, accounting for a total of, approximately, 3.5 billion users (InternetLiveStats, 2016). This huge amount of people are not restricted anymore to their locality to find goods and services. They now can browse nationally and internationally to look for products and services that better satisfy their needs. This setting is shifting the power from business to customers, as the latter now can decide from whom to buy based on their own criteria instead of geographical availability (Phippen et al., 2004). Customer empowerment has increased the competition among companies making it difficult and expensive to attract customers but rather easy to lose them (Phippen et al., 2004, p.285). A company that is not able to develop a sustainable and effective digital marketing strategy tend to loose market share due to its inability to reach its target audience or to motivate them to buy its products. As stated by Wind and Mahajan (2002, p.44), ”no business, regardless of its geographical location, is thus immune from the impact of the web revolution (...)”. On the other hand, this scenario is also creating opportunities for brands to reach customers that were out of their targets before. Therefore, digital marketing is becoming an increasingly relevant source of competitive advantage that companies want to master to reach more consumers, deliver their messages and increase their market share

(16)

2 Chapter 1. Introduction

(Leeflang et al., 2014).

One way in which companies can benefit from the Internet is by collecting data from their con-sumers. The growth of users generating content and interacting with brands online provides a whole new world of data that could be unravelled and provide new insights and intelligence to business (Leeflang et al., 2014). However, companies struggle at adjusting their organisational design to provide better interactions with customers and gather insights from online communi-cation channels. Consequently, they face several problems in implementing actionable metrics to evaluate their brands performance in the digital world (Leeflang et al., 2014).

Digital marketers need to have tools and procedures to be able to analyse performance of their digital strategy. The customer interaction with an online platform generates a huge amount of data about users’ profiles, behaviours, spends, and preferences. As stated by Spiller and Tuten (2015, p. 114) ”Organisations have gone from being starved for data to being overwhelmed by it”. Understanding the customer journey is a vital for companies to get acquainted to their costumers1 and create more effective messages (Leeflang et al., 2014). However, processing and analysing this high amount of data is still a challenge for many organisations. According to Leeflang et al. (2014, p. 5), the main challenges include ”capture, curation, storage, search, sharing, transfer, analysis, and visualisation”. One emerging technology which is gaining ground on this matter is Big data analytics. It has been embraced as a disruptive technology that will transform the way business define strategies based on accurate and real-time data. Its capability to process huge amounts of data, and derive patterns and trends is replacing the old fashion marketing intelligence that relied on market surveys to understand customer behaviour (Zhao et al., 2014). Thus, big data analytics is the technology that will enable companies to use this high volume of data to measure digital marketing performance and understand how customers interact with their brands online. Although most digital marketers understand this importance, most of them are not able to measure it in an effective way (Jrvinen and Karjaluoto, 2015; Leeflang et al., 2014). Spiller and Tuten (2015, p.115) cites five main reasons given by marketers to fail in assessing measurement: 1) Companies perceive it as too expensive., 2) They dont know how to use and measure them, 3) They believe their programs are so different

(17)

1.2. Scientific and Managerial Rationale 3

so they cannot be measured, 4) Companies feel that public relations is volatile hard to draw business results, and 5) They feel that there are no standard methodologies. The high amount of online metrics available through all web analytics tools makes the task of analysing digital marketing performance rather complicated as can be seen in Table 1.1.

Issue Percent of

re-spondents Not able to quantify the financial impact on the business 31%

Difficult to understand what these metrics measure 24% Not directly comparable with the traditional metrics 23% Don’t help to identify the relevant non-financial, behavioral

pre-dictors in my business (e.g. repurchase rate)

23%

Not actionable 20%

Too many metrics, difficult to tell which ones matter most 18% Too ’fluffy’ and not grounded in tangible data 18% Not detailed enough 11% Too different from traditional metrics 9%

Table 1.1: Problems with online metrics. The table is a result of a survey conduced by Leeflang et al. (2014) with 777 marketers. They were asked the main challenges they face when analysing online metrics.

1.2 Scientific and Managerial Rationale

One of the main advantages of digital platforms is that they are able to track users’ behaviours and measure conversion from clicks to actual purchases. Traditional basic metrics (i.e., Face-book ”Likes” and page views) only tell part of the story. Companies with online presence should focus on metrics that track end-to-end user’s behaviour to be able to link digital mar-keting campaigns to actual sales (Hanna et al., 2011). For example, questions like: Do ’likes’ of brands on social media have an impact on business performance?, are still unanswered. An important issue is whether it will be possible to derive a few single online metrics that function as a kind of overarching metric that reflects the reality of the business performance (Leeflang et al., 2014).

Leeflang et al. (2014), and Spiller and Tuten (2015) also emphasised the issue of the increasing availability of new metrics, tools, and methodologies resultant from improvements on tech-nologies available. Although most companies recognise they need to be active online, they

(18)

do not truly understand how to do it effectively, what performance indicators they should be measuring, and how they should have being measuring them (Hanna et al., 2011). Market-ing researchers will have to put significant efforts in understandMarket-ing the importance of digital marketing metrics and how they translate the business performance. Currently, there is a widespread perception that online measures are not easily translated into financial impact, and that online metrics are not readily comparable to traditional metrics (Leeflang et al., 2014; Spiller and Tuten, 2015).

One specific challenge with digital marketing is the attribution of specific marketing actions to business results. Customers are attracted with multiple online and offline media and channels to online and offline stores. Companies often wonder what was the relative contribution of a specific measure on sale in its respective channels (Leeflang et al., 2014). Thus, digital marketers have a rather complex task of ”measuring, optimising, and proving the business value of their enagement strategies, programs, and campaigns” (Forrester Research Inc, 2016, p.1). Integration of survey analytics and big data analytics to increase the validity of market studies will require new techniques in business intelligence and marketing sciences. Zhao et al. (2014) stated that analysts will have opportunities to develop new methods and applications to fill this gap and take advantage of digital platforms data.

Currently, there is a lack of research on why companies select some metrics and ignore others. Also, little is known about how companies define a comprehensive and manageable set of metrics to drive their business strategies. Another gap is regarding to whether quantitative online metrics can replace qualitative, therefore subjective, marketing metrics (Jrvinen and Karjaluoto, 2015). The results of these few researches revealed a non-promising landscape of the use of online metrics for assessing performance of digital marketing initiatives. These studies revealed that web analytics are used in an ad-hoc manner and the data gather from it are not used for strategic purposes (Jrvinen and Karjaluoto, 2015).

One of the goals of this research is to provide better information to support digital mar-keters’ decision making. As stated by the Media Richness Theory (MRT) (Daft et al., 1987), organisations process information to reduce both uncertainty and equivocally. According to

(19)

1.2. Scientific and Managerial Rationale 5

Daft and Lengel (1986), different media should be used for different communication objectives. Thus, managers need to select appropriate media to communicate and optimise the information transfer. MRT classifies the following learning media from least effective to most effective: 1) unaddressed documents (bulk mail, posters); 2) written, addressed documents (letters, e-mail); 3) 2-way radio; 4) telephone; 5) video conferencing; 6) face-to-face. The six main learning media are continually extended as new technologies appear. Brunelle (2009) demonstrated that MRT can be used to address not only intra-organisational communication but also communication be-tween companies and its consumers explaining how they relate to each other. He demonstrated how analysing consumers’ behaviour under the perspective of MRT can provide insights about which aspects affects an consumer’s intention to use an online store, for instance. Therefore, the application of MRT in this research is two folded. First, well defined and comprehensive metrics can be the source of information required to reduce uncertainty in the decision making process. Selecting the right medium, i.e. visualisation, to display these metrics can increase its potential to transfer information in an accurate manner. Second, understanding how a po-tential consumer uses non-sensory information provided by digital platforms to decide upon a purchase can provide insights on how to improve digital marketing campaigns.

To understand the fundamentals that guide the decision making cognitive process and what types of information individuals use to make decisions requires the evaluation of decision making theories, i.e. Behavioural Decision Theory (BDT) (Todd and Benbasat, 1994). BDT has two interrelated facets, normative and descriptive. The first is regarding to prescribe courses of action that are aligned with the decision maker’s beliefs and values. The second is concerned about describing these beliefs and values and how they are incorporated into an individual’s decisions (Slovic et al., 1977). Underlying the descriptive theory, there are several theories that are used to describe behaviour. Probabilistic judgement is the theory that aims to evaluate how people perceive, process and evaluate the probabilities (intuitive statistics) of a certain event. Choice theory tries to explain how choices are weighted and how people process information to choose between options. Models of risky choice theory, on the other hand, evaluate specific aspects of making decisions under conditions of risk (e.g. gambling, investing). Finally, dynamic decision making theory studies tasks in which decisions are made sequentially in time and tasks

(20)

specifications may change overtime independently or as a result of previous decisions (Slovic et al., 1977; Edwards, 1961). Making sense of these concepts and the information in which they are build upon will guide the information structure of the measures and the way they’re displayed.

1.3 Objectives

This research aims to provide guidance on which online metrics can be used to depict business performance of digital marketing campaigns. It will shed light on what KPIs need to be monitored, what data needs to be analysed and how data needs to be combined and structured to provide the right information to support decision making. It also evaluates the relation between online metrics and digital marketing performance.

In order to achieve the above mentioned objectives, this research depicted how digital marketers perceive a good digital marketing metric and which are the main characteristics underlying these metrics. When describing these metrics, it is also analysed how these can be collected and processed. Furthermore, an extensive list of the main metrics used to evaluate marketing campaigns is analysed in order to define which ones reflect business performance and whether they can be extracted from digital platforms. This research will take into account mainly platforms used in the western world (e.g. Facebook, Youtube, Twitter, Instagram, Google+). In order to evaluate these aspects, the following section depicts the research questions and the sub-questions derived out of it.

1.4 Research Questions

1.4.1 Main question

In order to address the issue of misalignment between digital marketing and business perfor-mance, it is necessary to get acquainted on the metrics that can be used to create a link between

(21)

1.4. Research Questions 7

them. Therefore, a deep understanding on the metrics available and all possible analysis that can be done combining multiple sources of data is required. Based on these assumptions, the main research question is as follows:

Which digital marketing metrics can be used to monitor the effectiveness of digital marketing campaigns on the business performance perspective?

1.4.2 Research Sub-question 1

To answer the main question it is necessary to be acquainted of the current digital marketing metrics being used in other industries. Benchmarking metrics are necessary to define criteria of how good online metrics look like in different industries. Therefore, the first research question is as follows:

Which digital marketing metrics are successfully applied across different industries?

1.4.3 Research Sub-question 2

To answer the main question, it is also necessary to understand digital marketers perceive the effectiveness of digital marketing initiatives using online metrics. In addition, it is important to understand the criteria they use to make decisions and evaluate an initiatives’ performance. Thus, the second sub-question is as follows:

How does a good digital marketing metric look like in the perspective of a digital marketer?

1.4.4 Research Sub-question 3

To create a link between the effectiveness of digital marketing initiatives and business perfor-mance, it is required to acknowledge the different metrics that digital platforms provide. In addition, a link between these metrics and business performance needs to be established. Thus, the third sub-question is as follows:

(22)

Which digital marketing metrics can be used to provide an accurate representation of the busi-ness performance?

Combining the results of the three sub-questions will enable to answer the main question as they provide information about how a good metric is perceived, what gap of information these metrics intent to fill and which data need to be extracted and combined from each platform.

The following chapters covers the literature review describing the main concepts used in this research and how they are related to each other (i.e. conceptual framework). The third sec-tion contains the research methodology describing how the data was collected and analysed. Subsequently, an analysis section describes the results from the data collection. Finally, the last section draws the conclusions from the data collection and sheds light on implications and further developments of this research.

(23)

Chapter 2 Literature Review

2.1 Digital Marketing

In order to understand Digital Marketing, it is necessary to conceptualise Digital Content and Marketing. Digital content can be defined as bit-based objects distributed via electronic chan-nels (Koiso-Kanttila, 2004). Marketing can be described as all sorts of communication between a company and its (potential) customers (Gordon, 2011). Therefore, bringing together these two main concepts, we can establish that Digital Marketing is the communication between companies and its potential consumers via electronic channels. Following this reasoning, Smith (2011) defines Digital Marketing as the practice of promoting products and services using digital distribution channels.

Currently, digital marketing is driving a shift of marketing strategies from traditional media (e.g. TV, radio, newspapers, magazines, billboards) to online media (e.g. social media, website, in-video ads). As stated by Mulhern (2009), the main difference between traditional and online advertising is the idea of communications being now about a world of networks, algorithms and automated systems for managing connections between information and people instead of focusing only on the media channel. Digital technologies are, therefore, changing the way po-tential consumers and brands interact with each other, and are creating a mutual exchange of information instead of an old-fashion one way communication stream (Edelman, 2010).

(24)

10 Chapter 2. Literature Review

tional advertising has focused on delivering messages to large audiences aggregated by a simple set of characteristics, e.g. announcing in a newspaper that has middle age business men as main readers or during a cartoon TV show to address children. While this strategy is still re-ceiving considerable attention from marketers, novel ways of working focused on search, social networks, cloud computing and algorithms that filter and serve information are gaining more attention (Mulhern, 2009). Figure 2.1 describes the channels that are being focus of marketers’ attention.

Figure 2.1: It is a result of a survey conduced with 153 marketers of companies and agencies based in the United Kingdom (ALF Business Development, 2014). They were asked Which of these channels is your marketing team likely to invest?

In order to address new media requirements, companies are investing every time more in digital marketing and paid media. It is expected that companies will spend around $542.5 billion dollars in paid media ads globally, as can be seen in Figure 2.2.

Digital Marketing can make use of several digital platforms to reach customers, one of the most currently used is Social Media due to its capability to target specific consumers based on behaviour and preferences.

(25)

2.2. Social Media 11

Figure 2.2: Total media spending worldwide

2.2 Social Media

To understand social media, it is relevant to first understand its bases. Social media platforms are built on top of the so-called Web 2.0. Web 2.0 uses the World Wide Web as a platform whereby content and applications are no longer created and published by individuals, but instead are continuous modified by all users in a participatory and collaborative fashion (Kaplan and Haenlein, 2010).

Social Media is a group of Internet-based applications that build on the ideological and tech-nological foundations of Web 2.0, and that allow the creation and exchange of User Generated Content (UGC)1 _{(Kaplan and Haenlein, 2010). Another definition of Social Media is provided}

by Peters et al. (2013, p.282) as ”communication systems that allow their social actors to com-municate along dyadic ties”. They derived this meaning from two distinct areas of research, communication science and sociology. In other words, it means a medium to transfer or store information being exchanged between two or more actors.

Social media, in a traditional sense, enables companies to talk directly to their customers, while

1_{UGC is the sum of all ways in which people make use of Social Media. It is usually applied to describe the} various forms of media content created and provided by users (Kaplan and Haenlein, 2010)

(26)

in a nontraditional sense it also enables customers to talk directly to one another creating communities around brands (Mangold and Faulds, 2009). In this sense, Peters et al. (2013) describes the egalitarian nature of social media. For them, social media eliminated the one-way hierarchical relation between brands and potential consumers, where people were ”enforced”, for instance, to watch a commercial on TV. As social media is based by a network of individuals, brands are now just another actor (a node) with the same power as the others. Thus, they need to rethink the way they interact with their customers to fit this new setting (Peters et al., 2013). This interaction creates user communities generating useful information that can be used to improve companies’ products and services. Kaplan and Haenlein (2010) define six groups of social media platforms classifying them by social presence/media richness and self-presentation/self-disclosure, as can be seen in Table 2.1.

Self-presen

tation/

Self-disclosure

Social Presence/Media richness Low Medium High High Blogs Social networking

sites (e.g., Face-book)

Virtual social worlds (e.g., Sec-ond Life) Low Collaborative projects (e.g., Wikipedia) Content com-munities (e.g., YouTube) Virtual game worlds (e.g., World of Warcraft)

Table 2.1: Classification of Social Media by social presence/media richness and self-presentation/self-disclosure (Kaplan and Haenlein, 2010)

Social media platforms are useful tools for digital marketing as they log users’ behaviours and provide data about their preferences. Precision marketing strategies make use of these data to provide specific content to targeted users and optimise campaigns.

2.3 Precision Marketing

Currently, there is an abundance of data about online users’ behaviour that can be used to provide companies a better understanding about what consumers need and want. Using users’ online behaviour data as input, Precision Marketing is the technique that aims to deliver the right message to the right person at the right time (Cook, 2013). Therefore, Precision Marketing

(27)

2.4. Data Analytics 13

offers personalised customer service and is used to help companies increase their profits by means of high-efficiency marketing (You et al., 2015). Underlying precision marketing, there are Programmatic Buying Algorithms processing these data to provide the most efficient match between advertisement and user, increasing the effectiveness of campaigns.

However, marketers still focusing on branding and creative work rather than data, metrics, quantitative models and digital technology (Mulhern, 2009). Although precision marketing is a technique designed to overcome the lack of data-driven decision marketers make, it needs to rely on a large scale data set that provides accurate and real-time results. Data Analytics techniques are useful to analyse data generated on digital platforms and have a better understanding of target customers. These analysis can be the input for precision marketing strategies as well as the inputs for business strategies.

2.4 Data Analytics

The topic of data analytics, also known by the variation of big data analytics, is gaining atten-tion nowadays due to its value to generate business intelligence and the increasing processing capacity provided by cloud computing (Kambatla et al., 2014).

Big sets of data need to be adjusted in order to become manipulable. The most common operations are pre-processing, filtering, aggregation, and transformation (Assuno et al., 2015). The analysis can be categorised into three main categories:

1. Descriptive: model past behaviour;

2. Predictive: forecasts based on available data;

3. Prescriptive: assess actions, assist decision making (Assuno et al., 2015);

To give a web-focused perspective on data analytics, Web Analytics Association (2008) defines the term web analytics as the measurement, collection, analysis and reporting of Internet data for the purposes of understanding and optimising Web usage.

(28)

Web analytics techniques enable companies to collect data from online users’ behaviour. The data is collected from user interactions with a web page or a social media, which results in a large set of unbiased data. Once users are not required to actively provide information, the data is more reliable than traditional measurement methods, e.g. customer surveys and interviews, as it depicts users’ actual behaviour instead of the their perceptions or beliefs (Jrvinen and Karjaluoto, 2015).

Gathering reliable data is a challenging task. However, it can be standardised and automated, making it an incentive to perform data analytics, instead of an obstacle (Jrvinen and Karjaluoto, 2015). The main issue with processing and analysing large data sets is that they need to be translated to make them interpretable. Information visualisation techniques aim to display large sets of data in such a way that people can easily read and understand the information provided.

2.5 Information Visualisation

Information visualisation has several definitions according to different authors. It can be either a lean definition as ”a mapping between discrete data and a visual representation” (Manovich, 2010), or a more complex one as ”Information visualisation the technique used to discover the structure of a (typically large) data set. This structure is not known a priori; a visualisation is successful if it reveals this structure” (Manovich, 2010).

For this study, a user-oriented definition will be used as the concept of information visualisation, as data analytics is already taking into account a more technical point of view. Therefore, ”Visualisation of data makes it possible for researchers, analysts, engineers, and the lay audience to obtain insight into these data in an efficient and effective way thanks to the unique capabilities of the human visual system, which enables us to detect interesting features and patterns in a short period of time” (van Wijk, 2006).

Information visualisation is, therefore, an important capability organisations need to master in order to be able to analyse information in a effective way. However, most companies are

(29)

2.6. Online Metrics 15

still struggling in finding the best way to represent information visually and create reports that depict information in the most suitable way for executives. Data gathering is useless without proper analysis and interpretation (Jrvinen and Karjaluoto, 2015). One of the main causes is the lack of an analytics plan that defines the right metrics and how they are related to the organisation’s goals. Thus, understanding the metrics available online and how they can be organised and aggregated to support the decision making process, is vital for companies relying in digital platforms.

2.6 Online Metrics

Peters et al. (2013, p.283) and Farris et al. (2006) define metrics as ”a measurement system that quantifies static (i.e., state) or dynamic (i.e., processes) characteristics”. Metrics are widely used in business and research to define and monitor goals and, therefore, support decision making (Peters et al., 2013).

The increasing availability of digital and social media formats that can be used by marketers to communicate with their potential customers brings a whole world of metrics that revolutionised marketing measurement (Spiller and Tuten, 2015). Measuring and analysing these metrics helps companies to produce significant strategic insight. Marketers now have access to logs of site activity data and behavioural patterns associated with web activity. Social media also collect a huge amount of behavioural data and UGC that can be used for analysis (Spiller and Tuten, 2015). Marketing analysts can now combine all that data to support their marketing strategies and their decision making. In order to define the best metrics to monitor, it is important to get acquainted with the plethora of metrics available. Therefore, understanding how different authors categorise them is a key part of the selecting the most suitable metrics for each digital marketing initiative. Spiller and Tuten (2015) organised general marketing metrics into two broad categories, i.e. financial and non-financial, to address the needs of marketers and managers (see Appendix A.1). Specifically for online metrics, they provided a categorisation into three different types:

(30)

• Activity: are used to measure how active a brand is online. Use the inputs the brand makes online to develop digital presence. It is input focused.

• Interaction: are used to measure the engagement of the target audience with the brand. Tracks several mediums of interaction with the brand and the influence beyond the initial target group. According to Spiller and Tuten (2015) ”it is the most critical assessment component for digital and social campaigns”. It is process focused.

• Return metrics: are used to measure, directly or indirectly, the success of the brand. They aim to give a financial representation of the value of online actions. It is outcome focused.

These three categories are relevant when comes to defining which metrics to monitor according to the business goals defined. A detailed list with examples of metrics for each category can be seen in Appendix A.2.

Cutler and Sterne (2000) and Rappaport (2014) stated that metrics need to provide meaning and be attached to a customer journey, i.e., the customer life cycle. By doing so, they will be able to better tell the brand story and understand how customers interact with it, providing insights on how to create the best communication strategy towards the defined target group. Each stage of the customer journey can be described by a different set of metrics. Generally, a customer life cycle can be defined by the following five stages (Cutler and Sterne, 2000):

1. Claim someone’s attention

2. Bring them into your sphere of influence

3. Turn them into a registered and/or paying customer 4. Keep them as a customer

5. Turn them into a company advocate

The costumer life cycle can be interrupted in each of its phases, affecting negatively the brand as it results in reduction of the customer base, market share, and, ultimately, leads to financial

(31)

2.6. Online Metrics 17

losses. Hence, tracking the interruption of the customer journey is also an important aspect to take into account when defining a set of online metrics to monitor. The complete customer life cycle is described in Figure 2.3 and a complete list of metrics is available in Appendix A.3

Figure 2.3: Customer life cycle (Cutler and Sterne, 2000)

Marketing metrics can also be categorised according to the company’s maturity level. Pat-terson (2007) described a metrics continuum that is organised in five levels: 1)Activity-based, 2)Operational, 3)Outcome-based, 4)Leading-indicators, and 5)Predictive. Each of these stages require a distinct set of measures that better evaluates the company’s performance. The repre-sentation can be seen in Figure 2.4. Understanding and defining the level of maturity of digital marketing initiatives in a company can be used as a guideline to define the most suitable set of metrics in a given time and define improvement targets for future initiatives.

An important issue that needs to be taken into account when defining the measurement plan, is the time frame of analysis. Metrics can be collected in real-time, hourly, daily, weekly, monthly, quarterly, and annually. Defining the right frequency of collection for each metrics is required to have accurate comparisons and to define a historical basis for future benchmarking (Cutler and Sterne, 2000).

Online measurement enables marketers and managers to take actions on their digital and com-munication strategy. Based on the data gathered, they can perform five core tasks, i.e. modify site design, change ads, alter promotion strategy, change product mix, and rethink partner

(32)

Figure 2.4: Marketing metrics continuum (Patterson, 2007)

strategy (Cutler and Sterne, 2000). However, strictly quantitative metrics do not completely address the monitoring of qualitative goals such as enhancing brand image, increasing customer penetration, or consideration (Jrvinen and Karjaluoto, 2015). In order to perform these tasks accordingly, marketers need to have access to qualitative data as user behaviour and content analysis. Quantitative metrics can be also defined as ”mind-set metrics” (e.g, awareness, lik-ing, consideration. For examples, see Appendix A.4). Unfortunately, they cannot be collected automatically online, requiring traditional methods as surveys and focus groups (Srinivasan et al., 2009). Mixing quantitative data analysis with qualitative research methodologies can complement each other and provide meaning for behaviours identified through the analysis of online metrics. Therefore, the ideal landscape for evaluating digital metrics would be merg-ing quantitative data provided by digital and social media with qualitative data provided by traditional research methods Spiller and Tuten (2015).

Zahay and Griffin (2010) state the importance of having metrics that link marketing and busi-ness performance. These metrics were defined as ”customer-based performance measures”, being a mix of financial and behavioural metrics. According to them, analysing them together, as a cohesive group, provides a big picture of the company’s business performance and the impact marketing initiatives had on it. As stated by Phippen et al. (2004), over time,

(33)

busi-2.6. Online Metrics 19

nesses have begun to find the use of basic metrics such as hits and pages views to be woefully inadequate for assessing the success of websites, due to the fact that their simplistic and ambigu-ous nature can induce misleading conclusions(...). Therefore, companies need more elaborated metrics that provide understanding of the relationship between (potential)customers and the brand. However, basic metrics are not useless, they are part of advanced metrics formulae as can be seen in Appendix A.3. Examples of customer-based performance metrics can be seen in Appendix A.5.

An organisation’s efforts to use marketing metrics systems and their resulting outcomes cannot be understood without considering the reasoning behind the chosen metrics, the processing of metrics data, and the organisational context surrounding the use of the system (Jrvinen and Karjaluoto, 2015). Therefore, companies need to design a manageable metrics system that demonstrates the progress towards marketing objectives. Also, establishing a process that fosters the effective use of metrics data within the organisation, and ensures that the organisational context supports the use of the metrics system are key requirements to succeed in a measurement initiative (Jrvinen and Karjaluoto, 2015; Rappaport, 2014). Metrics should be selected following a pre-defined framework that makes clear the reasoning behind each measurement and how each of them contributes to tell the story underlying the brand’s story. They also need to be focused on business objectives rather than platform driven, and be flexible to adjust according to the evolution of the campaign/initiative (Rappaport, 2014).

(34)

Chapter 3 Study 1: Literature Review

3.1 Method

A literature review was undertaken to get acquainted with the current knowledge about the main concepts concerning digital marketing metrics. The main concepts were summarised in Chapter 2. The literature review was used to answer Research Sub-question 1 (1.4.2), surveying the most used and studied metrics, and the main characteristics of each. Furthermore, it was used to answer Research Sub-question 3 (1.4.4), as previous researches have studied several metrics and how to link marketing metrics to business performance. The metrics were described according to many categories defined by different authors. The literature review was also used as input to structure the questionnaire and the questions of the interview, and to define the set of hypotheses.

3.2 Results

The increasing number of analytics platforms makes the task of defining a set of existing metrics rather complex (Bughin et al., 2008). New technologies boost the appearance of metrics that aim to monitor specific user behaviours and sometimes are quite exclusive of a certain platform

(35)

3.2. Results 21

(e.g. Facebook reactions, Vine video loops). In order to manage new and specific metrics, authors often opt for using generic aliases or categories to describe them, reflecting broad common behaviours and characteristics among metrics. Spiller and Tuten (2015) created a categorisation over three different levels of social media participation that span over any type of online platforms. For each category, it is possible to define qualitative and quantitative metrics. Therefore, the categories are: 1) Activity, 2) Interaction, and 3) Return. Details and examples of metrics in each categories can be seen in Appendix A.2 and Section 2.6.

Cutler and Sterne (2000) defined an extensive set of online metrics (named by them as e-metrics), though without a clear reasoning supporting the categorisation. Roughly, their categories embrace tracking customer behaviour and customer life cycle. Also important to highlight, they have defined Primary metrics, being the foundation of all other metrics. There-fore, Cutler and Sterne (2000)’s metrics categories are: 1) Primary, 2) Customer life cycle, 3) Customer life cycle interrupted, 4) Best customer, 5) Promotion calculations, 6) Successful life cycle, 7) Customer behaviour, 8) Personalisation index, life time value, loyalty value, freshness factor. Although these categories lack foundation, Cutler and Sterne (2000)’s work is the most extensive in listing metrics. They have detailed these metrics in such way that they can be applied for any digital platform (not only social media or e-commerce). In addition, the authors have provided formulae for most of them, making it clear about how to extract and analyse them. For the complete list of metrics, see Appendix A.3.

Digital marketing metrics can be also be categorised to match distinct organisational stages of development. Companies with different maturity levels have different measurement require-ments. To address this need, Patterson (2007) developed a metrics continuum that categorise metrics into five levels of maturity. The continuum provides clarity for marketers about what the next steps to better assess digital marketing performance are and what metrics fits better each stage. It is important to emphasise that moving along the metrics continuum, does not mean that metrics from previous stages will be unnecessary. In fact, metrics from early phases are more important to members of the functional team while metrics high up on the continuum are more relevant to the executive team (Patterson, 2007). The continuum lists metrics from simply measuring activities to predictive metrics. The five levels are: 1) Activity-based, 2)

(36)

Op-22 Chapter 3. Study 1: Literature Review

erational, 3) Outcome-based, 4) Leading-indicators, and 5) Predictive. The metrics continuum can be seen in Figure 2.4.

Given the increasing relevance of social media and the online interaction between users, a social media focused setting of metrics is proposed by Peters et al. (2013). However, the scheme and metrics conceived by them can also apply for other types of online platform (e.g., e-commerce platforms). As a social media based categorisation, it tries to capture specific properties as network characteristics, actors dynamics, contingency aspects of information exchanged, and the specifics of the medium being used. The authors then define that digital marketing metrics need to translate (and therefore be categorised into) four main aspects of social media: 1) Motives, 2) Content, 3) Network structure, and 4) Social roles and interactions. The framework depicted can be seen in Appendix A.6 and the list of metrics in Appendix A.7. Table 3.1 provides a summary of the findings.

Author Categorisation criteria Categories Spiller and Tuten (2015)

Smit and Neijens (2011) Brown (2010)

Social media participation

1. Activity 2. Interaction 3. Return

Cutler and Sterne (2000) Ad-hoc

1. Primary

2. Customer life cycle

3. Customer life cycle interrupted 4. Best customer

5. Promotion calculations 6. Successful life cycle 7. Customer behaviour 8. Personalisation index, life time value, loyalty value, freshness factor

(37)

3.3. Discussion 23 Patterson (2007) Feinberg et al. (2012) Level of maturity 1. Activity-based 2. Operational 3. Outcome-based 4. Leading-indicators 5. Predictive Peters et al. (2013)

Wasserman and Faust (1994) Trusov et al. (2010)

Ransbotham et al. (2012)

Social media properties

1. Motives 2. Content

3. Network structure

4. Social roles and interactions Table 3.1: Metrics categories

3.3 Discussion

The literature review provided an extensive list of metrics that fit distinct purposes. Although many categorisations were proposed, they helped to understand main characteristics that un-derlie all metrics.

During the research, a common sense that there is no ”one size fits all” when it comes to online metrics was clear. The point of view of each author (highlighted in Table 3.1) helps to define which metric to use according to the company’s landscape. For instance, if a company has its digital marketing initiatives heavily based on engaging with its potential consumers on social media, the definitions proposed Spiller and Tuten (2015); Smit and Neijens (2011); Brown (2010) can suit best its needs. If an organisation has an e-commerce based digital marketing operation, Cutler and Sterne (2000) metrics will provide more structured definitions to explore the behaviour of its consumers. Lastly, if the company develops its digital marketing strategy by the co-creation with a community of consumers/users, Peters et al. (2013); Wasserman and Faust (1994); Trusov et al. (2010) categories provide a clear idea on how the actors interact and influence each other. Therefore, defining the most suitable metrics does not follow clear

(38)

24 Chapter 3. Study 1: Literature Review

rules or have specific formulae. It has to take into account business goals and clear KPIs as starting point, and define metrics that will support the business to achieve these goals.

The set of digital marketing metrics that best fit a company’s needs is totally dependent on the organisation’s activities, brand positioning, and geo-cultural aspects (Jrvinen and Karjaluoto, 2015; Spiller and Tuten, 2015; Patterson, 2007). Thus, defining a set of metrics that fits all companies and all brands regardless of their business activities and brand positioning is not useful for marketers. However, it is possible to define grounding rules that can provide guidance on the selection of such metrics. As stated by Jrvinen and Karjaluoto (2015) the first step of implementing the measurement of digital marketing performance, should be the definition of simple and manageable set of metrics that is linked to the company’s performance and will be used to calculate the brand equity1 afterwards. Having clear business goals makes easier to select metrics that are really useful and provide information to support decision making (Zahay and Griffin, 2010).

In order to put all digital marketing metrics concepts, categories and goals together, it is required to understand the many dimensions metrics can have. Therefore, when creating a measurement plan, one can take these dimensions as reference in order to define which metrics to use, how to collect data, and how to report them.

1. Customer journey: The phase of the customer journey the metric is representing ac-cording to the company’s maturity level.

2. Medium: The platform(s) where the metric will be collected (e.g., social media, website) 3. Time-frame: Definition of the periodicity the metric needs to be checked and reported. 4. Target group: The stakeholders that will use the metric to make decisions.

5. Financial perspective (ROI): The financial perspective it represents. 6. Goal of the campaign: The campaign/initiative it represents.

1_{Brand equity, in short, is the value of brand.} _{It can be calculated taking into account, for instance,} changing market share, profit margins, consumer recognition of logos and other visual elements, brand language associations made by consumers, consumers’ perceptions of quality and other relevant brand values (Grannell, 2009). However, there is no formal definition of how brand equity is calculated.

(39)

3.3. Discussion 25

7. Goal of the metric: What information the metric provides to guide towards the business goals.

(40)

Chapter 4 Study 2: Interviews

4.1 Method

Semi-structured interviews were performed to understand how digital marketers use online metrics to make decisions (decision making process), how they perceive a good digital met-ric, and what are the main pitfalls with the current metrics used by them. They aimed to answer Research Sub-question 2 (1.4.3) and provide input to answer Research Sub-question 3 (1.4.4). The interviews were conduced with 9 people working directly with digital marketing. The interviewees are HEINEKEN International’s employees working for the brands Strongbow, Desperados, Cidrerie Stassen, Old Mout, and Heineken. From those, three have a technical pro-file, called Functional Consultants (FC ), responsible to design and develop digital marketing tools; five are brand managers (BM ), responsible to develop brands, and define their commu-nication strategies; and the last one is a media buying agent (MA), responsible to define and execute the ad buying strategy.

The interview was structured under twelve questions split into 1)Introduction: warm-up phase to get acquainted with the interviewee and her/his background; 2)Content questions: questions that address specific topics about the use of metrics and the rationale underlying them; and 3)Closure: a space for the interviewee talk freely about any topics that can complement the interview and were not covered by the questions (the structure of the interview and the objective

(41)

4.2. Results and Discussion 27

of each question can bee seen in Appendix B.1).

Interviews were performed until the answers started to repeat with the answers collected before, what happened by the seventh one. These interviews were recorded and transcribed in order to link similar quotes and answers. Moreover, similar responses were grouped to have a clear view of the main characteristics of a good digital marketing metric according to marketers’, i.e. Research Sub-question 2.

4.2 Results and Discussion

To validate whether the metrics categories and characteristics found in the literature review reflect the perceptions of digital marketers, they were asked during interviews to describe char-acteristics of main metrics they use, how they are reported and evaluated.

When explaining how they assess the performance of their digital marketing initiatives, and which metrics they use to evaluate it, different groups gave slightly distinct responses. All three FCs focused their answers on specific metrics for each platform, highlighting quantitative and platform driven metrics, for instance: ”The metrics we monitor depend on the campaign and the platform being used to activate.”, ”Currently, we use visitor and visits as metrics to define success.”. When asked specifically about which metrics they use to assess a digital marketing initiative’s performance, a FC answered: ”For Facebook and Twitter it’d be reach. Audience metrics like spread and shares. For our website, it’d be page views. If there is a code redemption campaign, amount of conversions, amount of prices, etc.”. On the other hand, when the same question was asked to BMs and the MA, although they have also provided a platform focused point of view (”We look at [ad] impressions (...) and [ad] views, but more importantly, we really try looking at engagement rates. We look at site specific metrics as dwelling time, unique visitors and peak visitors.”), a broader perspective was given. They tend to see digital marketing as one component of the marketing portfolio instead of looking at specific online platforms: ”In our evaluation we monitor how many people are aware of our brand. How many people would consider to consume our brand. How many people did buy that brand at least once

(42)

28 Chapter 4. Study 2: Interviews

a year. How many actually buy it regularly.”. Brand equity is a metric often mentioned by BMs when talking about marketing performance: ”In a global sense, our marketing campaigns are basically evaluated looking at the Brand Equity KPIs. Its variation through the years give us the idea of how we are performing against our competitors”.

Three interviewees emphasised the relevance of having spends figures represented on metrics. For them, efficiency is strongly linked with the ability to reach more and better potential consumers with less money: ”If we know what is being spent in each channel, and how we attract visitors and how relevant they are for our brand. We can focus in specific customers and channels. This is where precision marketing plays a role. (...)Good metrics could also help us shape the way we communicate with our consumers. Nowadays we only use gut feelings to make these decisions, but the interaction with them online could provide us way more insights than we have right now”.

When asked about the how they decided upon which metrics to monitor, 6 out of 9 respondents pointed out the difficulty to select, monitor and make sense of them due to the huge amount of data available (”Companies are measuring so many things that they get ’data blindness’. People just see tenths of numbers and they dont really understand what that means.”). All respondents answered that although they recognise the importance of having a measurement plan prior to the implementation of a digital marketing initiative, it is not done because the company does not have the required people and culture to do so: ”The problem that we have is that it seems to have a lot of data, that has this barrier to accessibility as the amount of data scares people. We do not promote the use of data, we dont have the discipline to access it regularly. Too much data that makes it a hurdle to look at these figures and people that are not educated and have discipline to look at it frequently.”.

Another common point seen in the answers is the issue and importance of tracking multi-platform metrics and understanding how consumers interact with a brand through all commu-nication channels used (”Our challenge now is to integrate data from all our digital platforms (...). We are communicating through Facebook, Instagram, Website, YouTube. We need to un-derstand how these platforms are related to each other and how are our customers interacting

(43)

4.2. Results and Discussion 29

with them. The first step is to see how many people we’ve reached. Are we reaching the same people across all the platforms or are they different people? How do they interact with each platform? Where they stick more?”). Four interviewees also highlighted the importance of tracking the whole customer journey and monitor different metrics in each stage: ”(...) under-standing the gap between each stage gives us an idea about where our communication is failing. If a person is aware of our brand, why isnt he or she consuming it? People do want to buy my brand but they are not finding it in stores, then I have a distribution problem, not a market-ing problem”. Four interviewees also stated that metrics need to combine data from multiple sources, providing accurate information. Although some technical and legal problems may ap-ply (e.g., restrict use of cookies across platforms, incomplete user’s information), mainstream platforms as Google Analytics and Facebook Insights already provide solutions to monitor users interaction with the brand through many platforms: ”Because of this integrated digital world, it’s pretty hard to understand how many people you’ve, in fact, reached. Its difficult to know whether I’ve reached thirty people or three people ten times. The same person can be accessing the website, Facebook, and Instagram, using desktop or mobile. Therefore, combining data from three sources, social, ’.com’, and financial, we could have much better insights than before.”. In order to understand how they perceive good digital marketing metrics, interviewees were asked about the main strengths and weaknesses of the reports they create and receive. The main problems reported was having clear KPIs to monitor and reporting it in a simple and effective way: ”[In my previous position], we used click-through rates, and simple Google Analytics data. It was simple, it was straightforward, we had just a few and very clear KPIs, following industry KPIs. Here we seem to have a lot of data but we are not using them at all because it just seems too complicated to access and analyse it. (...) First you need to define your goals and what youre going to do with the data. Otherwise we just get lost in the middle of these many data. (...) We should simply our operation. Define top six or five KPIs per brand that we should keep track”.

For digital marketers, it is important to assess whether their initiatives are, in fact, impacting business performance. Creating this link is, however, a difficult task that faces technological and organisational problems. Firstly, as the organisation do not make consistent use of online

(44)

30 Chapter 4. Study 2: Interviews

selling points, it is difficult to see whether a person that have been exposed to an add or interacted in any way with the brand has actually purchased a product: ”Its very hard to link online advertisement with sales. Mostly, because we sell offline. We know that if we dont advertise, the sales will drop. But in which extent they are related, we dont know. We cant say exactly how much each euro spent in online marketing converted in sales”. Secondly, the company does not have a data driven culture for some products. Therefore, marketers still using old-fashion tools to evaluate their performance: ”We do not promote the use of data, we dont have the discipline to access it regularly. Too much data that makes it a hurdle to look at these figures and people that are not educated and have discipline to look at it frequently.”.

(45)

Chapter 5 Study 3: Survey

5.1 Method

A survey was used to have quantitative data to corroborate the qualitative findings from litera-ture review and interviews. The survey’s main goal was to validate assumptions from literalitera-ture review and interviews, and analyse whether these assumptions apply to the respondents. The survey was designed to answer a set of hypotheses drawn based on the literature review (the structure of the survey can be seen in Appendix B.2). Therefore, the results were used to answer Research Sub-question 2 1.4.3 in a quantitative manner, providing information on how professionals with distinct experiences perceive and use digital marketing metrics. The partic-ipants were gathered by spreading the questionnaire within HEINEKEN International B.V.’s Marketing e-mail list, covering digital marketers from 34 local branches, each branch localised in a different country. The survey was also made available in two LinkedIn groups (i.e., Digital Marketing, Social Media Marketing) and using the researcher’s personal network. The survey was pre-tested with a group of 5 people to gather feedback on the clearness of words and ex-pressions used, and to avoid misinterpretation of the questions. In order to filter undesired respondents, a filtering question was added and responses from people with no digital market-ing experience were removed. In addition, responses with a duration time less than 3 minutes, with the same options checked for all, and not filled until the end were discarded as it qualifies

(46)

32 Chapter 5. Study 3: Survey

a poor respondent. The amount of responses discarded were 5, 1, and 9, respectively. A total of thirty valid responses were used for analysis (from what 18 were women). The respondents aged from 24 to 46 years old (M = 31.1, SD = 6.04 ), and worked in countries localised in Africa, Asia, European Union, North America, Oceania, and South America.

For each hypothesis, dependent and independent variables were defined and a specific statistical test was assigned. Likert-scales were used in multiple choice questions with ordinal weights. The survey measured the relation among different levels of seniority and which metrics they use more often, and which decisions they make (Q12. Following it is shown five main actions you can take using digital marketing metrics. Rate them according to their importance for you. and Q13. How often do you check the following metrics? ). Another important assessment was the relation between the most used metrics according to different stages of the customer life cycle (Q11. Following it is shown five main goals of a digital marketing initiative. Rate them according to how frequent they are assigned as a goal of your initiatives). Hypothesis 1 assessed if there is a relation between the marketer’s seniority and the decisions they make using online metrics. Hypothesis 2 evaluated whether basic (standard) metrics are reported more often than advanced (tailored, and complex to collect) metrics. Hypothesis 3 measured whether more mature companies do have an automated process of measurement. Hypothesis 4 assessed if more mature companies measure more steps of the customer life cycle. Finally, hypotheses 5 to 9 evaluated if marketers select their metrics taking the different stages of the customer life cycle into account. All hypotheses were tested against p<.05.

5.2 Results

Hypothesis 1

High positions (managers, directors, and CEOs) will use digital metrics to take different actions compared to low and intermediate positions (Analysts and Interns) (Cutler and Sterne, 2000; Patterson, 2007)

(47)

5.2. Results 33

• Variables:

– Independent Variable: Position levels

– Dependent Variable: Level of importance of actions

• Test: 5 t-Tests

– The test was performed for each of the five categories (i.e., possible actions taken based on digital metrics): 1. Modify site design, 2. Change ads, 3. Alter promotion strategy, 4. Change product mix, and 5. Rethink partner strategy. Also, for each test, the measurements were split into two groups: Higher positions (i.e., Group 1) and Lower positions (i.e., Group 2). The t-Tests performed for each category defined whether there was a difference in the means reported by each group.

No significant difference was found in the means of the scores reported by high and low positions for any of the categories.

• Action 1 - Modify site design: t (28) = 0.826, ns (Group 1: M = 3.5, SD = 0.941; Group 2: M = 3.19, SD = 1.109)

• Action 2 - Change ads: t (28) = 0.414, ns (Group 1: M = 4.07, SD = 1.072; Group 2: M = 3.94, SD = 0.680)

• Action 3 - Alter promotion strategy: t (28) = 0.267, ns (Group 1: M = 4.07, SD = 0.917; Group 2: M = 4, SD = 0.516)

• Action 4 - Change product mix: t (28) = 0.530, ns (Group 1: M = 3.64, SD = 0.497; Group 2: M = 3.5, SD = 0.894)

• Action 5 - Rethink partner strategy: t (28) = 1.286, ns (Group 1: M = 3.57, SD = 0.938; Group 2: M = 3.13, SD = 0.957)

(48)

Hypothesis 2

Basic metrics (1 to 6) are reported and checked more often than advanced metrics (7 to 29) (Cutler and Sterne, 2000; Spiller and Tuten, 2015; Zahay and Griffin, 2010).

• Variables:

– Group 1: Metrics checked

1. Frequency basic metrics are checked 2. Frequency advanced metrics are checked – Group 2: Metrics reported

1. Frequency basic metrics are reported 2. Frequency advanced metrics are reported

• Test: Paired t-Test

– The mean frequency of checking basic metrics was compared with the mean frequency of checking advanced metrics. The result of the test indicated whether the one mean is statistically significantly different from the other. The same procedure was executed for the frequency of reporting basic and advanced metrics.

The results indicated that basic metrics were reported more often than advanced metrics.

• Pair Basic metrics X Advanced metrics (Checked): t (29) = 7.899 (Basic: M = 4.21, SD = 1.414; Advanced: M = 3.31, SD = 1.788)

• Pair Basic metrics X Advanced metrics (Reported): t (29) = 3.458 (Basic: M = 3.22, SD = 1.360; Advanced: M = 2.60, SD = 1.548)

The bar chart presented in Figure 5.1 depicts the frequency averages metrics are checked and reported.

(49)

5.2. Results 35

Figure 5.1: Frequency of checking and reporting metrics

Hypothesis 3

Companies with a higher level of expertise (Advanced and Expert) have a higher proportion of automated measurement process than companies with a lower level of expertise (None, Basic, and Intermediate) (Patterson, 2007).

• Variables:

– Independent Variable: Level of expertise – Dependent Variable: Has automated process

• Test: Fisher’s exact test

– The responses were split into two groups: companies with high level of expertise, and low level of expertise. Fisher’s exact test defined whether there is significance in the means reported by each group.

No significant difference was found in the means of the scores reported by the different groups, χ2(1) = 0.677, ns.

(50)

Hypothesis 4

Companies with a higher level of expertise (Advanced and Expert) have different campaign’s goals than companies with a lower level of expertise (None, Basic, and Intermediate) (Cutler and Sterne, 2000; Rappaport, 2014).

• Variables:

1. Dependent Variable: Level of expertise.

2. Independent Variable: Frequency the goal is assigned to a campaign • Test: 5 t-Tests

– The test was performed for each of the five categories (i.e., possible actions taken based on digital metrics): 1. Claim the attention of a potential customer, 2. Bring a potential consumer into your sphere of influence, 3. Turn a user into a registered and/or paying consumer, 4. Retain registered and/or paying customer, and 5. Turn a consumer into a company advocate. Also, for each test, the measurements were split into two groups: Companies with low level of expertise (i.e., Group 1) and companies with high level of expertise (i.e., Group 2). The t-Tests performed for each category defined whether there was significance in the means reported by each group.

No significant difference was found in the means of the scores reported by high and low positions for any of the categories.

• Goal 1 - Claim the attention of a potential customer: t (28) = 1.363, ns (Group 1: M = 4.19, SD = 0.801; Group 2: M = 3.5, SD = 1.732)

• Goal 2 - Bring a potential consumer into your sphere of influence: t (28) = 1.104, ns (Group 1: M = 3.81, SD = 0.801; Group 2: M = 3.25, SD = 1.708)

• Goal 3 - Turn a user into a registered and/or paying consumer: t (28) = 0.441, ns (Group 1: M = 3.23, SD = 0.863; Group 2: M = 3, SD = 1.633)

(51)

5.2. Results 37

• Goal 4 - Retain registered and/or paying customer: t (28) = 0.912, ns (Group 1: M = 3.08, SD = 1.055; Group 2: M = 3.19, SD = 1.915)

• Goal 5 - Turn a consumer into a company advocate: t (28) = 0.938, ns (Group 1: M = 3.31, SD = 1.011; Group 2: M = 2.75, SD = 1.708)

Hypothesis 5

There is a correlation between companies that often have ”Claim the attention of a potential customer” as a goal and the frequency they check and report the metric ”Reach” (Cutler and Sterne, 2000).

• Variables:

1. Frequency the goal ”Claim the attention of a potential customer” is assigned to a campaign

2. Frequency of checking the metric ”Reach” 3. Frequency of reporting the metric ”Reach”

• Test: Correlation

– The correlation with the goal will be calculated separately for the frequency the metric is reported and checked.

The results found a marginally significant correlation between the variables ”Claim attention” and ”Reach (check)”. For the variables ”Claim attention” and ”Reach (report)”, no significant correlation was found.

• Claim attention X Reach (check): Pearson’s R = 0.340

(52)

Hypothesis 6

There is a correlation between companies that often have ”Bring a potential customer into your sphere of influence” as a goal and the frequency they check and report the metrics ”Acquisi-tion/engagement” and ”Abandonment” (Cutler and Sterne, 2000).

• Variables:

1. Frequency the goal ”Bring a potential customer into your sphere of influence” is assigned to a campaign

2. Frequency of checking the metric ”Acquisition/engagement” 3. Frequency of reporting the metric ”Acquisition/engagement” 4. Frequency of checking the metric ”Abandonment”

5. Frequency of reporting the metric ”Abandonment”

• Test: Correlation

– The correlation with the goal will be calculated separately for the frequency each metric is reported and checked.

No significant difference was found in the correlation between the variables.

• Bring consumer into your influence X Acquisition / engagement (check): Pearson’s R = 0.170, ns

• Bring consumer into your influence X Acquisition / engagement (report): Pearson’s R = 0.131, ns

• Bring consumer into your influence X Abandonment (check): Pearson’s R = 0.268, ns

Assessing Digital Marketing Performance: How to use online metrics to evaluate digital marketing initiatives

University of Amsterdam

Faculty of Sciences