Clicking paid results : if we know paid results are relevant, are we willing to use them?

(1)

Clicking paid results

If we know paid results are relevant, are we willing to use them?

Master Thesis - January 28, 2016 University of Amsterdam (UvA)

MSc. in Business Administration – Marketing Track Wierd Schamhart #10002071

(2)

Statement of Originality

This document is written by student Wierd Schamhart who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

Abstract

Over 70% of internet users start their online quest for information on a search engine page. This makes the search engine result page (SERP) an interesting place for advertisers to find an

audience for their products and services through advertisements in the form of paid results. How consumers react to these paid results has not yet been fully researched. Especially the user’s clicking behavior on paid results in relation to organic results and what influences this relationship is not part of current academic knowledge. This paper constructs and tests a framework to understand the influence of knowledge of ranking mechanisms, ability to detect ads and overall attitude towards ads on the ads clicking behavior. Over 335 respondents have completed the empirical study. During the survey three individual scenarios with transactional intention were tested. Results show that although users have a neutral attitude versus paid results they are avoiding their usage in their actual clicking behavior.

(4)

Contents

1 Introduction ... 4

2 Theoretical Framework ... 5

2.1 Search Engine Result Page (SERP) ... 5

2.2 Information search behavior ... 7

2.3 Information search intentions ... 8

2.4 Structure of an online search engine result page ... 9

2.5 An organic result... 11

2.6 A paid result ... 12

2.7 Knowledge off paid results and attitudes related to paid results ... 12

2.8 Hypotheses and Framework ... 15

3 Methodology ... 16 3.1 Survey design... 16 3.2 Sample ... 18 4 Results ... 19 4.1 Preliminary steps ... 19 4.2 Reliability ... 19 4.3 Hypotheses testing ... 21 5 Discussion ... 24 6 Conclusion ... 27 7 References ... 28 8 Appendix – Questionnaire ... 31

(5)

1 Introduction

Over the past decade, the internet has become a vibrant marketing environment. After developing as a grass root information network in the nineties, the zeros showed numerous new options for businesses to use internet as a marketing and communication tool (MacManus, 2004). With the rise of Web 2.0, new participation driven communication options arose for both consumers and marketers (O’Reilly & Battelle, 2009; Rose & Levinson, 2004). Although the new participation driven platforms such as Facebook and YouTube gave consumers numerous new options to use the internet, the initial role as information network remained and continued to gain in importance. Each quest for information, from the weather to new shoes starts at a search engines. Therefore Search Engine Advertisement and Search Engine Optimization have become mainstream marketing tools (Truong & Simmons, 2010) with corresponding marketing budgets.

This study was selected because there is little research about the factors that influence the use of paid results (advertisements) by consumers versus the use of non-advertisement search results (subject to optimization). It combines the necessary theoretical framework to review behavior with an empirical study to expand the current understanding of users capabilities and preferences. The research question:

“How does a user’s knowledge about relevance based ranking mechanisms for advertisements influence a user’s attitude towards advertisements and his clicking behavior on a search engine result page after

he performs a transactional search?”

This research paper is built as follows. Chapter 2 introduces relevant literature and thereby creates the theoretical framework and ends with the theoretical model and hypotheses. Chapter 3 shows the research methodology used to answer the main research question and test the hypotheses. In chapter 4, the results of the data are shown and hypotheses are accepted or rejected. Chapter 5 discusses these findings with the theory as a backdrop and tries to find explanations for the different effects found. And finally, chapter 6 provides conclusions from the research.

(6)

2 Theoretical Framework

To answer the main research question, this chapter examines previous literature on the surrounding subject. It starts by introducing the Search Engine Results Page (SERP), followed by online search behavior in general. Then it looks into the different kinds of online search intentions. Next the detail related to the structure of an online search engine results page is discussed and the definitions of organic and the paid search engine results. In the end,

knowledge of ranking mechanisms and attitudes towards paid results versus organic results are discussed and the hypotheses are stated. Finally, a framework is created to show how the hypotheses relate to each other.

2.1 Search Engine Result Page (SERP)

In the beginning of the internet, search engines were non-existing. Instead listing platforms provided a catalogue of pages as found by their employees. After the fast growth of the internet during the nineties (Albert, Jeong, & Barabási, 1999), the indexing platforms such as Yahoo Directory needed to reinvent themselves to cope with the new demand. The platform, Yahoo Directory, shifted to Yahoo Search Engine using an automated indexing method. It started using the words on the page, known as keywords, to index the page (Brin & Page, 2012). A search query now triggered results from pages with the same words on it. A few years later in 1998, Google developed the algorithm Page Rank which started to take incoming links in to account to determine the importance of a page (Page, Brin, Motwani, & Winograd, 1999). The underlying assumption was that a higher number of incoming links would reflect a higher popularity and therefore higher importance. The PageRank algorithm made searching by words and phrases possible. With the development of semantic search, this became even better. These algorithms are the foundation of the search engine experience we now take for granted.

From a user’s point of view, the most important thing for a search engine is to provide relevant results to the input of the user’s search query. Search engines employ a large amount of factors to decide the relevance and rank of the results shown on the result page. The average user has little idea what leads to the ranking of the different results shown (Höchstötter & Lewandowski, 2009). The only ranking mechanism that is highlighted on the result page is the paid vs. organic results. Paid results are the results generated after the search engine received

(7)

payment for their placement. On the contrary, the organic results are generated by the ranking mechanism of the search engine without receiving payment for their placement. But even with ads highlighted, users are unaware of ranking mechanisms including paid results (Marable, 2003). This is not a big issue, since research shows that both organic and paid results are equally relevant to transactional search intent (Jansen & Spink, 2008). A transactional search is a search with the intent to find a website with the purpose to execute a service in order to obtain a

product (Broder, 2002). At the same time, research also shows this is of a major concern since paid results do not weigh in the reliability of a vendor (Ma, Pant, & Sheng, 2007).

Research by Jansen and Spink shows no difference in click rates for paid results if they are mixed into the organic result. From their results, they conclude that users are well equipped to judge the result for its relevance to their search query and base their clicking behavior on that relevance and thus, are indifferent to organic versus paid results (Jansen & Spink, 2008). However, they do not look at the ability of their subject to distinguish between organic and paid results. So the research only shows that the general clicking behavior is based on the relevance of the link and not the result and also, not by the result being paid or organic. This still does not mean the searcher perceives paid and organic results in the same way. There are several studies that show a strong preference for organic results and a negative attitude towards paid search results (Jansen & Resnick, 2005). This means that if a result is labeled as a paid result, a consumer rates its relevance to his search query lower than when the same result is not labeled as a paid result. At the same time it means that when two results are perceived as equally relevant and one is labeled as paid result and the other is not, the searcher prefers the non-paid result (Jansen & Resnick, 2005).

Several studies researched the concept of attitude in the internet context using a hierarchy of effect models (Chen and Wells 1999; Stevenson et al. 2000; Lee et al. 2004). Attitude and hierarchy of effect models have been shown to add to the understanding and explanation of consumer online behavior (Gauzente, 2009). A hierarchy of effect model can be used to explain how attitude drives behavior (Gauzente, 2009). The core hierarchy of the model is that attitude drives intention and intention drives behavior. The construct attitude is formed in different ways. It can be seen as an outcome of only affect, but also as a combination of the dimensions cognition and affect or even an outcome of three dimensions affects, cognitive and behavioral (Yoo, Kim, & Stout, 2004).

(8)

2.2 Information search behavior

Studies of users search behavior stems from long before there were any online possibilities. Information and library science looked at why users search and what strategies they employ to do this effectively. When the online context was introduced, researches started to apply these questions to the new online context. Take Bates for example. In the seventies, he looked at ways people searched and their search tactics (Bates, 1979). Later on in the late eighties, he looked at information browsing techniques and how these techniques were employed in an online search context (Bates, 1989).

After online search engines became popular, research on this topic followed quickly. One of the early studies shows how users predominantly use short search queries for their online search questions (Silverstein, Marais, Henzinger, & Moricz, 1999). A round up of these early studies and their findings was made by Jansen and Pooch (2001).

The research by Jansen and Spink studied how online search behavior developed over time between 1997 and 2001 (Jansen & Spink, 2006). By looking at query logs during this period, they found that although some minor changes, for instance, the declining willingness to look at more results, the overall search strategies are fairly constant (Jansen & Spink, 2006).

Online searchers tend to quickly evaluate the search results before clicking at one or two results (Spink & Jansen, 2006). They do not want to scroll down, over half of them only looks at page one of the search results and about half the searchers only tries one query per search

session (Hotchkiss, Garrison, & Jensen, 2005). Furthermore, half of the search queries only employs one word and on average a query only contains 1.6 to 3.3 terms depending on search languages (Hochstotter & Koch, 2009; Jansen & Spink, 2006).

Before the invention of the World Wide Web, the general search goal was informational, meaning the searcher was looking to ‘find out’ about his preferred topic (Rose & Levinson, 2004). Both due to the users with access to search engines (researchers, students, etc.) and the kind of databases and their content, this was a fair assumption (Rose & Levinson, 2004).

In contrast to that, in a web based search environment the use is far greater than just research. Just a quick look at a query log of a large online search engine (such as Yahoo, Google) shows that there are more goals in play then just finding out.

(9)

All these studies together paint a picture of what users are searching for and how they are doing that, but the next question is why are they searching online. What is their intention behind searching purposes?

2.3 Information search intentions

Broder finds three different intentions of webs searchers to answer the question of why users search online (Broder, 2002). For this research, he looks at the search query and tries to mark the kind of information retrieval the user is looking for. He calls this the ‘need behind the query’. He finds three needs.

He classifies the first need as the Navigational intention. The user needs to find an address to navigate to. His intention is to reach a predetermined website. This can be due to a prior visit to the site or the belief that such a site exists. This kind of searches usually has only one desired outcome that is the page the user already has in mind (Broder, 2002). Since there is only one desired outcome, the results are easily judged on relevance and only the desired result is clicked if available.

The second need Broder (2002) classifies is the Informational intention. The user intents to find some information he assumes to be present somewhere on the web. This intent is the web equivalent to the former ‘find out’ goal. The user wants the information in a clear form and wants to consume this information, but has no further intention to interact with the information. In this type of search the user looks for an overview of the information rather than one particular page. The user hereby creates an overview of the available information rather than the

information assembled by one site (Broder, 2002). Therefore, there are more than one desired outcome in a search result and the user is more likely to click several results creating an overview of information he is looking for.

The third need classified by Broder (2002) is the Transactional intention. When a user has this intention, he is looking to perform some sort of web mediated activity. When the user finds the site with the intended content, he is likely to further interact with the content of the site to fulfill the desired transaction. This desired transaction defines his transactional intention. These transactions can involve shopping, downloading, gaming, or any other web mediated transaction (Broder, 2002). The results to the query are hard to judge for relevance since factors like pricing, service and quality play an important part. These are the factors that are often not

(10)

present on the result page itself. Since this study looks for the user’s behavior on a search page with regards to a product buying intention, this study focuses on this transactional intention. 2.4 Structure of an online search engine result page

What a search engine result page (SERP) shows differs between different search engines. Looking at the search engine usage in The Netherlands shows a remarkable market dominance of Google (see figure 1). Well over ninety percent (94.07%) of the online searches are done via Google, with Bing a far second with only two and a half percent (2.56%)1. With a far majority of users in The Netherlands using Google, this study into user’s behavior on search engine result pages uses Google as main search engine example for the remainder of this study.

Figure 1

A typical Google result page for the search query ‘Theater’ might look like the print screen in figure 2. This ‘might’ be the case because search engines use personalized results. This

1_{Source StatCounter}

http://gs.statcounter.com/#desktop+mobile+tablet-search_engine-NL-monthly-201401-201501-bar (retrieved on 23-12-15) 0,92 1,17 1,28 2,56 94,07 0,14 0,05 1,58 0,3 97,93 0,29 0,54 0,82 0,56 97,79 1,26 1,6 1,3 3,57 92,27 0 20 40 60 80 100 OVERIG STARTPAGINA YAHOO! BING GOOGLE

Search Engine market share

in The Netherlands for 2014

(11)

is a factor that cannot be ignored since it severely influences the results and at the same time, it cannot be turned off.

Figure 2

2 First of all the page is built up as a list. The list has the ‘best answer’ at the top which is the highest possible position on the result page.

There are several areas that can be defined on a SERP. Firstly, there is an area visible at first sight and an area that can only be seen after scrolling. These two areas are referred to as

2_{Source Google}_{https://www.google.nl/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=theater} (retrieved 23-12-15)

(12)

above and below the fold. Since different users use different screen resolutions, the fold is not a set limit. On average, at least six results will be shown in the left column.

Second distinction is that between organic results and paid results. These two sorts of results are discussed later on. Typically the SERP is made up out of organic results and

surrounded by paid results. Paid results are marked and can be placed in the top three results as seen on the example above, but can also be placed on the bottom of the page (not shown in example since this is a print screen. The bottom results are below the fold). There is also a column on the right. These are also paid results. Google uses the query to determine whether or not it displays paid results and whether those will be shown above, below or to the right of the organic results. All paid results are labeled by a yellow AD label. Notice that the results in the main column (left) are individually labeled and the paid results on the right are marked as a total column. This column on the right will never show organic results.

Two schematic markups by Google (see figure 3) show the organic results oulined in the figure on the left and the paid results outlined in the figure on the right.

Figure 3

3 2.5 An organic result

Online search engines typically offer two kinds of results, organic reults and paid results. The organic result is the result returned by the search engines algorithm. The organic result is

3_{Source Google:}_{https://support.google.com/adwords/answer/1722080?hl=nl&ref_topic=3121771}_{(retrieved on} 23-12-15)

(13)

also referred to as the natural result, the non-paid results and the unsponsored result. The organic results were not paid for by the referred site to the search engine.

The search engine ranking mechanism (Google uses an algorithm known as PageRank) returns these results combining a couple of factors. The relevance of the page content to the search query and the popularity of the referred page determine the PageRank of the page. This PageRank determines where the result is placed on the result page. A higher PageRank leads to a higher placement (Brin & Page, 2012).

2.6 A paid result

Next to organic results, search engines often provide paid results. Paid results are also referred to as sponsored results and advertisements. Paid results are, as the name suggests, result placements paid for by an advertiser.

The paid results returned by the search engine are determined by another search engine ranking mechanism than the organic results. Google calls this algorithm the AdRank4. This algorithm also entails the relevance of the content of the page and ad to the search query and combines this with a cost per click bid that the advertiser can set.

2.7 Knowledge off paid results and attitudes related to paid results

Although organic and paid results are both on the same page and both are listed because their relevance is judged as high by the search engine, searchers have different attitudes towards and beliefs about relevance of paid results versus organic results. While a study in 2012 among 2253 adults found users to be “more satisfied than ever with the quality of search results” (Purcell, Brenner, & Rainie, 2012), other studies show that there is an overall negative attitude against paid results (Fain & Pedersen, 2006; Gauzente, 2009; Jansen & Pooch, 2001). One empirical study by Jansen and Resnick (2006) tested two groups. One group had the paid results labeled as such and the other had the organic results labeled as paid and the paid as not labeled (so perceived as organic). With the two groups combined more than 80% of the first clicks were on a not labeled so ‘organic’ result. Paid results received just 6% of the first clicks (Jansen & Resnick, 2006).

(14)

Another survey in 2007 with nearly 6000 respondents finds users complaining about paid results and vague ranking mechanisms (Höchstötter & Lewandowski, 2009).

In 2004, Hotchkiss found that novice users were having trouble identifying paid results and that the paid results received lower quality rating than the organic results (Hotchkiss, 2004). The preference for organic results was also supported by the study of Greenspan in 2004. Greenspan (2004) also found that paid results have a higher click rate if the search engine does not clearly label them.

This study therefore tests:

H1. The probability of clicking an AD is significantly and negatively associated with the labeling of the ad as such.

In an online survey by Georgia Tech University (Xing & Lin, 2006), results show that the attitude for paid results is negative since they are believed to be less objective and more biased than organic results. Over 70% of the respondents said to prefer clicking on an organic listing over clicking on a paid result (Xing & Lin, 2006). This concurs with the empirical study of Jansen and Resnick (2006). An empirical study conducted a few years earlier also found more clicks for organic results. A survey of SEMPRO showed that over 70% of first clicks were on organic results (Xing & Lin, 2006).

In 2005 participants said to click on results from trusted source with unbiased

information. Over 77% of these participants said to favor organic over paid results (Hotchkiss et al., 2005). This preference even looks to hold up if the search query has a transactional

intention.

This study therefore tests:

H2. The probability of clicking an AD is significantly and positively associated with the level of the positive attitude towards Ads in a search with transactional intention.

The likelihood of a result being clicked is a curvilinear function of the ranking that is the placement on the page (Brooks, 2004). Study shows this is true for both organic and paid results. The ranking seems to be an intrinsic relevance measure. In 2005, participants seemed to trust search engines but they reported to be unclear about how the results rank and how they are presented. Only 38% of the respondents said that they are aware of the distinction between organic and paid results. Fewer than 17% of the respondents said they can always identify the difference between paid and organic results (Fallows, 2005). Since the paid results are placed

(15)

above the organic results and the likelihood of being clicked is a curvilinear function of the placement, the paid result should be more likely to be clicked if the users see no difference between paid and organic results, or in other words is unable to detect the paid result. This study therefore tests:

H3. The probability of clicking an AD is significantly and negatively associated with the ability to detect an AD in a search with transactional intention.

A few studies show that in attitude and clicking behavior, searchers tend to lean towards organic links. Users are negative about paid ads because they do not trust them to be objective and unbiased. Additionally, they expect them to be of lower quality. At the same time, studies find searchers to be unable to identify ads and searchers are unclear about the ranking

mechanisms used by search engines. This study therefore tests:

H4. Knowledge about the weight of relevance in online ranking mechanisms is significantly and positively associated with (a) the attitude towards ADs, (b) the probability of clicking Ads in a search with transactional intention.

(16)

2.8 Hypotheses and Framework

Main research question: How does a user’s knowledge about relevance based ranking

mechanisms for advertisements influence a user’s attitude towards advertisements and his clicking behavior on a search engine result page after he performs a transactional search? H1: The probability of clicking an AD is significantly and negatively associated with the

labeling of the ad as such.

H2: The probability of clicking an AD is significantly and positively associated with the level of the positive attitude towards Ads in a search with transactional intention.

H3: The probability of clicking an AD is significantly and negatively associated with the ability to detect an AD in a search with transactional intention.

H4: Knowledge about the weight of relevance in online ranking mechanisms is significantly and positively associated with (a) the attitude towards ADs, (b) the probability of clicking Ads in a search with transactional intention.

Attitude towards ADs

Ability to detect ADs

Knowledge about relevance weight Probability of clicking AD Labeling of the AD H1 (-) H2 (+) H3 (-) H4b (+) H4a (+)

(17)

3 Methodology

This section shows the methods used in this study. An online survey was used to obtain the data for this research. Most likely the majority of participants would be Dutch natives. Therefor only a Dutch version of the survey was distributed. The questionnaire was opened on the 24th of October 2015. Via email, social media (Facebook and LinkedIn) and word of mouth, friends, family, colleagues and acquaintances were asked to participate in the survey and

forward it to their friends and relatives. Just four weeks later on the 20th of November 2015 the survey was closed. Objective of the survey was to obtain data about people’s attitude towards paid results, their knowledge of ranking mechanisms, their ability to detect paid placements and the paid versus organic placement clicking behavior. The next paragraphs explain the survey design and sample.

3.1 Survey design

The questionnaire was created to research knowledge about ranking mechanisms, attitude towards paid results, ability to detect paid results and clicking behavior on paid versus non paid results. At the start respondents were asked for some demographic control variables (gender, educational level) and their frequency of search engine usages. Next they were randomly assigned to two independent groups to check for the general effect of labelling an paid result on an SERP. Finally the respondents were asked questions to determine their attitude towards paid results and knowledge of ranking mechanisms. The full questionnaire is printed in the appendix.

Independent variables. The attitude towards paid results (4 items) and the ability to determine whether the respondent is able to detect ads (3 items) were measured using a 5 point likert scale (1 totally disagree to 5 totally agree). The items for attitude were derived from previous research by Gauzente (2009) and the items for ability to detect from Fallows (2005). The independent items were subjected to a principal components analysis. The analysis showed a one factor scale with loadings between .69 and .87 and explaining a variance of 68% for attitude and for ability to detect a one factor scale with loadings between .70 and .87 and

explaining a variance of 63%. Scale reliability was high with Cronbach’s Alpha respectively .83 and .79. Knowledge of weight of relevance in paid results ranking was measured through a single item.

(18)

Dependent variable. Respondents were given three different search queries. They were asked to select the result they would click if they had been in the real life situation. These clicks were recorded and coded as a click on organic results or a click on paid result. An alternative query was used to the one used for transactional searching used by Jansen, Brown and Resnick (2007). Keeping the main ideas for the query in place, respondents are supposedly ready to buy a specific product; therefore they are looking for a seller of this particular product (Broder, 2002). The SERPs used are all created from real SERPs with the small distinction that any vendor information that could influence the respondent trough prior experience was taken out of the result and replaced by a generic nonexistent vendor name. Second difference was that only the top six results were shown as would be the above the fold part. The six results consisted of the first three paid results and the first three organic results generated by the original search engine (google.nl). The layout of the results was consistent with that used at the original SERP. After a short textual introduction of the search context the respondents were asked to select the result that, according to them, they were the most likely to select in a real search task.

(19)

3.2 Sample

After 28 days, 395 people replied to the survey request, with 335 completing the entire questionnaire, a completion rate of 84.8%. Gender division was 150 female (45%) to 185 male (55%). Highest level of education ranges from high school (7.5%) to Doctoral (3%), but the majority has a university degree (37%). To the question how often respondents use search engines only one respondent answered a few times per month, while 167 respondents (50%) replied with multiple times a day. The sample details are listed in table 1.

Table 1: Sample Characteristics (n = 335)

Gender Male 185 (55%)

Female 150 (45%)

Educational level High School 25 (7.5%) Intermediate Vocational Education 63 (18.8%)

Bachelors 113 (33.7%)

Masters 124 (37%)

Doctoral 10 (3%)

SE use freqency Few times a month 1 (.3%)

Few times a week 8 (2.4%)

Multiple times a week 45 (13.4%)

Daily 114 (34%)

Multiple times a day 167 (49.9%)

This section reported on the methods used to acquire the data. The next section reports on the results of the survey.

(20)

4 Results

This part shows the results of the study. It start with explaining the preliminary steps needed to clean the data obtained from the questionnaire. Then it shows the reliability analyses of the used scales and finally the hypotheses are tested.

4.1 Preliminary steps

The Qualtrics survey was filled out by 395 subjects. 335 subjects completed the entire survey. The uncompleted surveys were not used for analysis. After striking the uncompleted surveys from the record, no missing values were left in the dataset. The subjects were randomly divided into two groups. The main group (n=167) was shown a version of the search engine result page with labeled paid search results. The control group (n=168) was shown the same search engine result page but without the label for paid results.

4.2 Reliability

The reliability for the attitude toward paid results scale (AttAD) is high with Chronbach’s Alpha = .834. All the items have sufficient correlation with the total scale

outcome, indicated by corrected item-total correlations of above .30 (for all items). None of the items would substantially affect the scales reliability if it was deleted.

The reliability for the ability to detect a paid result scale (DetectAD) is sufficient with Chronbach’s Alpha = .788. All the items have sufficient correlation with the total scale

outcome, indicated by corrected item-total correlations of above .30 (for all items). None of the items would substantially affect the scales reliability if it was deleted.

The scales were analyzed with a principal axis factoring analyses (PAF).The sampling adequacy was verified with the Kaiser-Meyer-Olkin measure, KMO= .744. Bartlett’s test of sphericity x2(28) = 526,834, p<.000. This indicates that the correlation between the items was large enough for PAF. A first analysis was done to see the eigenvalues (EV) of the components in the data. Two components returned EV of over 1, the Kaisers criterion. Together they

explained 65.9 % of the variance. Also the scree plot showed a leveling off after the second factor. Therefore, two factors were used and rotated with an Obliminal with Kaiser

(21)

1 suggest that this factor represents attitude toward paid results and the items clustered in factor 2 suggest this factor represents the ability to detect a paid result.

Table 2: Principal axis factoring analyses Item

Rotated Factor Loadings AttAD DetectAD Ik vind betaalde zoekresultaten op een SERP handig. .861 -.146 Het is goed dat er betaalde zoekresultaten op een SERP staan. .783 .008 Ik ben blij met betaalde zoekresultaten op een SERP. .777 -.016 Ik vind het normaal dat er betaalde zoekresultaten staan op een SERP. .582 .295 Tijdens dit onderzoek waren de betaalde zoekresultaten duidelijk gemarkeerd op

de getoonde SERPs. .032 .919

Als ik op een SERP op een betaald zoekresultaat klik, ben ik me daar altijd

bewust van. .003 .698

Tijdens dit onderzoek heb ik betaalde zoekresultaten gezien op de

getoonde SERPs. -.080 .616

Op een SERP is het verschil tussen een organisch en betaald zoekresultaat

duidelijk zichtbaar. .110 .573

Eigenvalues 2.76 2.52

% of variance 34.47 31.44

Note: factor loadings > .40 appear bold

After the scales were computed, a correlation analysis was done to quantify the meaning and intensity of the relationship between the control and scale variables. Results are shown in table 3. The results show that a few relationships have a significant tendency to a positive relation. For instance, there is a significant tendency towards a negative relation between the ability to detect ads and the clicking on ads on SERPs 1,2&3.

Table 3: Main group: Means, Standard Deviations, Correlations

MAIN (N=167) M SD 1 2 3 4 5 6 7 8 9 10 1 Gender 0.59 0.49 - 2 Education 4.02 0.96 .00 - 3 SEuse 5.27 0.81 .11 .23** - 4 RankingOR 3.53 1.03 .21** .06 .18* 5 RankingAD 2.81 1.13 .08 .02 -.05 .20* -

6 Attitude towards Ads 2.99 0.78 .15 -.03 -.04 .17* .24** (.83)

7 Ability to identify Ads 3.75 0.83 .13 14 .24** .39** .04 .05 (.79) 8 SE Result Page 1 0.35 0.48 -.04 -.09 -.23** .16* -.00 .12 -.35** -

9 SE Result Page 2 0.23 0.42 .118 -.07 -.01 _-.04 .12 .19* -.18* .27** -

10 SE Result Page 3 0.22 0.42 .10 -.09 -.12 _-.13 .09 .07 -.17* .30** .46** - **. Correlation is significant at the 0.01 level (2-tailed).

(22)

4.3 Hypotheses testing

H1. The probability of clicking an AD is significantly and negatively associated with the labeling of the ad as such.

Both the predictor (control group/ main group) and the outcome (AD/ no AD) variables are categorical variables. To test the H1 a Chi-square test is used (see table 4).

Table 4: Chi-square test for H1

SERP Not labeled (Control) Labeled (Main) X2 Phi 1 Organic 56 (-2.9) 108 (2.9) 32.912 .313 1 Ad 112 (2.8) 59 (-2.8) 2 Organic 108 (-1.0) 128 (1.0) 6.147 .135 2 Ad 60 (1.5) 39 (-1.5) 3 Organic 108 (-1.0) 130 (1.0) 6.673 .142 3 Ad 58 (1.5) 37 (-1.5)

Note. **= p < .01 *= p < .05. Adjusted standardized residuals appear in parentheses below group frequencies.

A statistically significant association between the labeling of ads and clicking on Ads is shown on all three Search Engine Result Pages (SERPs) with X2 (1) = 32.912, p<.001 / X2 (2) = 6.147, p<.05 / X2 (3)=6.673, p<.05. Based on the Phi statistic the effects are of medium (1: 0.313) and small (2: 0.135 & 3: 0.142) size. Interpreting the frequency distribution, this seems to represent that the probability of clicking an AD is lower when it’s labeled than when it’s not labeled.

H2. The probability of clicking an AD is significantly and positively associated with the level of the positive attitude towards Ads in a search with transactional intention.

H3. The probability of clicking an AD is significantly and negatively associated with the ability to detect an AD in a search with transactional intention.

The outcome variable, whether a paid or organic result is clicked, is a categorical variable. The predictor variables, attitude towards the ad and the ability to detect an ad, are continuous scales. Therefore, a logistic regression is used to test the relationships on all tree outcomes (see table 5).

(23)

Table 5: Logistic regression on SERP1,2&3

SERP1 95 % CI for odds ratio Sig.

B (SE) Lower Odss Ratio Upper Included

Constant 1.703 (1.012)

AttAD .448 (.236) .987 1.565 2.484 .057

AdDetect -.999 (.230) .235 .368 .578 .000

Note: R2= .014 (Cox), 0.19 (Nagelkerke). Chi-square= 24.499 p=0.000

Constant -1.179 (1.104)

AttAD .694 (.265) 1.191 2.002 3.367 .009

AdDetect -.591 (.234) .350 .554 .876 .012

Note: R2= .08 (Cox), 0.11 (Nagelkerke). Chi-square= 12.953 p=0.002

Constant -.236 (1.086)

AttAD .259 (.251) .792 1.295 2.118 .303

AdDetect -.493 (.229) .390 .611 .956 .031

Note: R2= .014 (Cox), 0.19 (Nagelkerke). Model * p=0.128. Chi-square= 5.542 p=0.063

Three independent logistic regression analyses were conducted to predict clicking on a labeled AD on three different SERPs using the attitude towards ads and the ability to detect an ad as predictors. The Chi-square of the first two models (SERP1&2) are both significant

indicating the predictors have a significant effect. For the third model (SERP3) p= 0.063 > 0.05 so the predictors do not significantly add to the model with only constants.

Since the results are mixed, the outcome is not conclusive. But there is a difference between the two factors. AttAD is not significant in the last SERP taking the whole model down. At the same time AdDetect is significant in all three models. Therefore, although H2 and H3 cannot be accepted since not all tests show a significant result, AdDetect looks to be a factor for further research.

H4. Knowledge about the weight of relevance in online ranking mechanisms is

significantly and positively associated with (a) the attitude towards ADs, (b) the probability of clicking Ads in a search with transactional intention.

H4a) The outcome variable, AttAD, is a continuous scale. The predictor variable, knowledge about ranking mechanisms, is a categorical variable. Therefor an one way ANOVA

(24)

is used. The ANOVA shows a statistically significant difference between groups (F(4.162)=3.867, p=0.005).

A Tukey post-hoc test revealed that the attitude towards paid results is statistically significantly lower when people disagree (p = .020) or strongly disagree (p = 0.006) with relevance being a strong factor in paid results ranking mechanisms related to people who agree with this. It stands out that no significant difference is measured with people who strongly disagree. This could be due to the rather small part of the sample that’s in this group (N=11)

H4b) Both the predictor (knowledge about ranking mechanisms) and the outcome (AD/ no AD) variables are categorical variables. To test the H4b a Chi-square test is used. All three SERPs outcomes were tested. None had a significant value. Therefore H4b is rejected.

In sum H1 was accepted, H2 & H3 were accepted on 2 out of 3 SERPs, H4a was accepted and H4b was rejected. The findings are further discussed in the discussion part.

(25)

5 Discussion

This study researched whether a user’s knowledge about relevance based ranking mechanisms for advertisements influence a user’s attitude towards advertisements and his clicking behavior on a search engine result page after he performs a transactional search. The previous chapter presented the results of the survey. The next part discusses the meaning of these results.

The paid results represent an easy way for businesses to find a new audience for their website. To see what factors influence the attitude towards and clicking behavior on paid results this study created and tested a new framework.

Results show promising aspects of the model and create several opportunities for future research. The only hypothesis that was not (partly) confirmed was the direct effect of knowledge of ad ranking mechanisms on clicking ad probability. A potential explanation is that the effect is too small to be significant in a small sample survey like this study. Another explanation is that since the found effect on attitude is small, it could be that variables that weren’t in this

framework like perceived risk associated with clicks on paid results also influence the clicking probability. This study however builds a framework that shows that attitude towards paid results and the ability to detect ads explain parts of a user’s ad clicking probability.

From a theoretical standpoint this study creates a conceptual view of possible

mechanisms at play when a user is shown a result page. This conceptual view was not available in previous studies. In particular the placement of knowledge and ability into the framework was lacking in earlier research. Also this study proposes a scale for ability to detect ads and proves it’s validity. The framework and developed scale are a first step for future research to build on. To better understand whether knowledge of ad ranking has an effect on clicking behavior a solid measurement needs to be developed.

Another outcome of this research is the neutral attitude users have towards paid results. While previous research found users to be negative towards paid results (Xing & Lin, 2006) this study finds users to be neutral towards paid results. This could be explained by the more

familiar users have gotten with paid results over the last years. Early 2000 studies suggested that users were unfamiliar with paid results and therefore are unconcerned with them (Marable, 2003). A few years later Jansen and Resnick found that if users click on a paid result they are

(26)

more likely to click on paid results in the future (2006). A small decade later users have gotten far more familiar with online search in general and paid results in particular. Over half of the respondents uses search engines a couple of times a day and overall users say to be able to detect ads on the result pages. Yet this is the discrepancy found in this research between what users say and think and what they do. While on the one hand they are neutral towards paid results, their actions suggest otherwise. The results showed that results labeled as a paid result where less likely to be clicked than when they were not labeled as such. This means that users think to be neutral about paid placements but are reluctant to actually click on them. This clicking behavior was also found by Jansen and Resnick (2006) but they concluded from this that users have a negative attitude towards paid results. The results from our survey suggest this conclusion is missing nuance. However the results did show that the there is a link between attitude and behavior. Our tests confirmed that if the user has a more positive attitude towards paid results he is more likely to click on them. It’s just not an equal match since the attitude is neutral and the behavior shows an overall unwillingness to click paid results. An explanation for this difference is that there is another factor driving attitude not yet measured in the attitude scale. A variable like previous experience, trust or relevance could be a more determining factor and should be included in future research.

Results also show that on average users think that relevance is not an important ranking factor for paid results, while at the same time they do think relevance is an important ranking factor for organic results. This is in contrast with how users actually perceive the information linked to in an transactional search. Previous study showed that users experience the same relevance for content linked to by paid results as to content linked to by organic results in a search with transactional intention (Jansen & Spink, 2008).

Finally this study widened the transactional search query. Previous study used only one search engine page to show how users interact with the results after a transactional search. This study tried three different search pages. This made the outcomes less clear, while findings were supported on two out of three pages, at the same time this demonstrates that maybe there isn’t such an universal search category as is imposed by previous research. Using three different search pages with the same intent shows that within that intent there is still a wide variety of outcomes. The three SERPs used in this study where

(27)

From a managerial point of view this study shows that users hold no negative feelings against paid results that could in turn reflect on the business using them, but at the same time are far more likely to click on the organic results. So while paid results are a fast and easy way to get a top of the page placement, investing in top of the page placements for organic results (which is harder and takes more time) is likely to return larger sums of traffic.

Since the results show that people indicate to detect ads business should be aware of the implications of using ads for their business. Further research could see whether there are in fact spillovers from the attitude versus paid results to the attitude towards the brand using those marketing options.

There are a few methodological limits in the study. First of all there are some limits due to the data collection process. The sample population is more technologically sophisticated than the average population. This comes from the methods used to acquire respondents. Respondents were approached via various online media such as email and social media like Facebook and LinkedIn. By asking respondents to ask their relatives (snowball technique) this effect was reinforced. Therefor future studies may focus on respondents that are less technologically

sophisticated to enhance the generalizability of the findings. Second thing to reconsider in future studies is the scenario approach. The scenarios left no options for respondents to choose their own search focus, but rather limited the search option to the scenario of the study. A the same time the respondents were asked to respond to a SERP instead of actually doing their own search and finally they had to choose one preferred result and could not click none of the above or two best results. Despite these limitations the respondents did have a high completion rate (84%). Third limitation was the focus on just the first six results of the SERP. This is the average above the fold total of results generated, but a real SERP would generate more results that can be scrolled and generate extra results in a column on the right and extra snippet results like pictures and places. Further research could look in to the effect of those extra results and how these effect the clicking behavior.

Overall this study couldn’t find a direct link between a user’s knowledge of ranking mechanisms and their clicking behavior, but did show that it influences the attitude towards paid results. To develop understanding of how knowledge of ranking mechanisms drives consumer clicking behavior further research must focus on the measurement of the knowledge before this can be linked to clicking behavior.

(28)

6 Conclusion

In the last two decades internet has been adapted as part of consumers everyday lives. Within this context search engine marketing (SEM) has grown to be a forceful marketing tool. Search engine advertising, as a part of SEM, is the number one source of income for search engines and receives the majority of online advertising spend. Still there is little known of the factors that influences paid results usages by consumers.

Using hierarchy-of-effects models this paper finds three main contributions. Firstly a framework is constructed to create a conceptual view of the mechanisms at play in a consumers paid versus organic clicking behavior. Secondly a measurement scale for ability to detect an ad on a search engine result page is developed and tested on a substantial sample and the

relationship between detecting an ad and clicking an ad is validated. Finally the empirical results suggest that users attitudes versus paid results are neutral, while their ranking is still perceived as less dependent on relevance than organic results. This findings strengthens the idea that businesses need to be aware of the relevance of their ad placements. From an academic point of view this research topic is still in an early stage leaving lots of opportunities for further research.

(29)

7 References

Albert, R., Jeong, H., & Barabási, A. (1999). Internet: Diameter of the world-wide web. Nature, 401(6749), 130-131.

Bates, M. J. (1979). Information search tactics. Journal of the American Society for Information Science, 30(4), 205-214.

Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407-424.

Brin, S., & Page, L. (2012). Reprint of: The anatomy of a large-scale hypertextual web search engine. Computer Networks, 56(18), 3825-3833.

Broder, A. (2002). A taxonomy of web search. ACM Sigir Forum, , 36(2) 3-10.

Brooks, N. (2004). The atlas rank report: How search engine rank impacts traffic. Insights, Atlas Institute Digital Marketing,

Fain, D. C., & Pedersen, J. O. (2006). Sponsored search: A brief history. Bulletin of the American Society for Information Science and Technology, 32(2), 12-13.

Fallows, D. (2005). Search Engine Users: Internet Searchers are Confident, Satisfied and Trusting–but they are also Unaware and Naïve.Pew Internet & American Life Project, Gauzente, C. (2009). Information search and paid results—proposition and test of a

hierarchy-of-effect model. Electronic Markets, 19(2-3), 163-177.

Hochstotter, N., & Koch, M. (2009). Standard parameters for searching behaviour in search engines and their empirical evaluation. Journal of Information Science, 35(1), 45-65. Höchstötter, N., & Lewandowski, D. (2009). What users see–Structures in search engine results

(30)

Hotchkiss, G. (2004). Inside the mind of the searcher its jungle here. Retrieved from

http://searchenginewatch.com/sew/news/2065095/inside-searchers-mind-its-jungle-here Hotchkiss, G., Garrison, M., & Jensen, S. (2005). Search engine usage in north america. White

Paper, Enquiro, Kelowna, BC, Canada,

Jansen, B. J., Brown, A., & Resnick, M. (2007). Factors relating to the decision to click on a sponsored link. Decision Support Systems, 44(1), 46-59.

Jansen, B. J., & Pooch, U. (2001). A review of web searching studies and a framework for future research. Journal of the American Society for Information Science and Technology, 52(3), 235-246.

Jansen, B. J., & Resnick, M. (2005). Examining searcher perceptions of and interactions with sponsored results. Workshop on Sponsored Search Auctions,

Jansen, B. J., & Resnick, M. (2006). An examination of searcher's perceptions of nonsponsored and sponsored links during ecommerce web searching. Journal of the American Society for Information Science and Technology, 57(14), 1949-1961.

Jansen, B. J., & Spink, A. (2006). How are we searching the world wide web? A comparison of nine search engine transaction logs. Information Processing & Management, 42(1), 248-263.

Jansen, B. J., & Spink, A. (2008). Investigating customer click through behaviour with

integrated sponsored and nonsponsored results. International Journal of Internet Marketing and Advertising, 5(1-2), 74-94.

Ma, Z., Pant, G., & Sheng, O. R. L. (2007). The inorganic side of paid search. Proceedings of the 6th Workshop on E-Business, 434-440.

(31)

Marable, L. (2003). False oracles: Consumer reaction to learning the truth about how search engines work. Results of an Ethnographic Study,

O’Reilly, T., & Battelle, J. (2009). Web squared: Web 2.0 five years on. Web 2.0 Summit, 1-13. Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking:

Bringing order to the web.

Purcell, K., Brenner, J., & Rainie, L. (2012). Search engine use 2012.

Rose, D. E., & Levinson, D. (2004). Understanding user goals in web search. Proceedings of the 13th International Conference on World Wide Web, 13-19.

Silverstein, C., Marais, H., Henzinger, M., & Moricz, M. (1999). Analysis of a very large web search engine query log. ACm SIGIR Forum, , 33(1) 6-12.

Spink, A., & Jansen, B. J. (2006). Web search: Public searching of the web Springer Science & Business Media.

Truong, Y., & Simmons, G. (2010). Perceived intrusiveness in digital advertising: Strategic marketing implications. Journal of Strategic Marketing, 18(3), 239-256.

Xing, B., & Lin, Z. (2006). The impact of search engine optimization on online advertising market. Proceedings of the 8th International Conference on Electronic Commerce: The New E-Commerce: Innovations for Conquering Current Barriers, Obstacles and Limitations to Conducting Successful Business on the Internet, 519-529.

Yoo, C. Y., Kim, K., & Stout, P. A. (2004). Assessing the effects of animation in online banner advertising: Hierarchy of effects model. Journal of Interactive Advertising, 4(2), 49-60.

(32)

8 Appendix – Questionnaire

Introductie

Zoekmachines maken een steeds groter deel uit van ons dagelijks leven. Je zoekt op wat voor weer het wordt, hoe die acteur ook alweer heet en waar je dat ene paar schoenen het goedkoopst kunt krijgen. Allemaal vragen waar een zoekmachine, zoals Google of Yahoo, je doorgaans snel en eenvoudig een antwoord op kan geven. Maar je krijgt nooit één antwoord. Na je

zoekopdracht krijg je een zo genaamde resultatenpagina te zien. Een pagina met meerdere zoekresultaten. Aan jou de keuze welk resultaat, cq welk antwoord je kiest. Dit onderzoek richt zich op dat laatste. Welk resultaat kies jij?

Procedure

Straks krijg je drie praktijkvoorbeelden van een vraag waar je mee zou kunnen zitten. Aan jou de keus… welk resultaat kies je op de resultatenpagina van de zoekmachine? Het enige dat je hoeft te doen, is je voorstellen dat jij de zoekopdracht uitvoert. Welk resultaat kies je? Klik dat resultaat aan.

Naast de zoekvoorbeelden vraag ik je een paar demografische gegevens in te vullen. Dit onderzoek duurt circa 5 minuten.

Risico's

De risico's voor deelname aan dit onderzoek zijn minimaal.

Compensatie/ bijdrage

Er is geen (financiële) compensatie beschikbaar voor deelname aan dit onderzoek. Wel lever je een bijdrage aan de ontwikkeling van kennis over het gebruik van online zoekmachines.

Vertrouwelijk

Alle verkregen data van deelnemers wordt vertrouwelijk beheerd. Deze data zal enkel in samengestelde vorm worden gerapporteerd (individuele resultaten worden niet openbaar gemaakt). Enkel de hoofdonderzoeker (W. Schamhart) en zijn begeleider (A. Zerres) hebben toegang tot de originele data. Alle verkregen data wordt opgeslagen in de HIPPA-compliant, Qualtrics-secure database, totdat de hoofdonderzoeker deze verwijdert.

Deelname

Deelname aan dit onderzoek is volledig vrijwillig. Je hebt ten alle tijden het recht om te stoppen of je terug te trekken uit het onderzoek zonder dat dit verder gevolgen heeft. Om het onderzoek af te breken kun je de browser sluiten. Wil je aangeven waarom je bent gestopt? Dat kan via een email aan de hoofdonderzoeker.

Vragen over het onderzoek

Wil je meer weten of heb je vragen over het onderzoek? Neem dan via email contact op met de hoofdonderzoeker Wierd Schamhart (w.schamhart@student.uva.nl)

Dit onderzoek wordt uitgevoerd in het kader van de master Business Administration richting Marketing aan de Faculteit Economie en Bedrijfskunde (UvA).

(33)

Ik heb de bovenstaande informatie gelezen en begrepen en verklaar dat ik uit vrije wil aan dit onderzoek deelneem.

□ Ja □ Nee

Email adres: (niet verplicht)

………

Geslacht:

o Man o Vrouw

Hoogst genoten opleiding:

o Lagere school o Middelbare school o MBO o HBO o Universiteit o Doctoraat

Hoe vaak gebruik je gemiddeld een zoekmachine zoals Google.nl?

o Nooit

o Een enkele keer per maand o Een keer per week

o Meerdere keren per week o Dagelijks

(34)

SERP 1 main group (with advertisements shown)

Al jaren draag je Levis 501 broeken. Helaas is je huidige exemplaar aan vervanging

toe. Je wilt online een nieuwe kopen. Welk zoekresultaat kies je?

(klik het resultaat aan dat jij in deze situatie kiest)

Levi's® Original 501® - buyonline.com

Adv.www.buyonline.com/501

Bekijk de Nieuwste Collectie op de Officiële Levi's® Site. Koop nu! Gratis verzending bij 99€ · Gratis Retourneren · Shop de Nieuwe Producten

Levi's 501® - webshop.nl

Adv.www.webshop.nl/501

Check alle Jeans van Levi's. Score, de jeans store voor mannen!

Levi's® Original 501® - outfit.nu

Adv.www.outfit.nu/501

Levi's 501 - Kleding online kopen? | kledingbestellen.nl | Lage prijs

www.kledingbestellen.nl › Kleding

Levi's 501 Ct Customized Tapered jeans shordich 36/34. LEVI'S Boyfriendjeans 501.

Levi's® Jeans, Jackets & Clothing | Levi's® Modeshop.com

www.modeshop.com/NL/nl_NL/

Een nieuw seizoen. Chino's, corduroy en moleskin. Een nieuw seizoen. Nieuwe materialen. Shop nu▷. 501® Original. THE ICONIC STRAIGHT FIT

501® - Levi's | Netherlands

www.jeansgraag.com/NL/nl_NL/collections-home/501

... van de originele jeans. DE ORIGINELE 501® KOPEN ... Geen wonder dat de Levi's 501 -jeans de ultieme blauwe -jeans is. ACQUISTA I 501® ORIGINAL ...

(35)

SERP 2 main group (with advertisements shown)

Op kantoor moet je een defibrillator (AED) hebben. Je besluit hem online te

bestellen. Waar doe je dat?

AED goedkoopste - procardio.nl Adv.www.procardio.nl/Philips_actie

HeartStart HS-1 AED, bekijk nu onze aanbieding, direct leverbaar

AED Kopen? - Groot assortiment betrouwbare AED’s Adv.www.aedleverancier.nl/

Eenvoudig en snel online bestellen!

Alle merken & type AED's · Gratis bezorging! · Ook service & onderhoud

Online AED kopen? - Best betaalbare AED van NL Adv.www.preveco.nl/AED_Kopen

Servicepakket 5 jaar en pads incl !

Geleverd binnen 48uur · Gratis verzending · Gratis servicecontract

AED Kopen ← AEDcompany.nl aedcompany.nl › AED SHOP

U wilt een automatische externe defibrillator (AED) kopen? Bekijk en vergelijk onze AED's hieronder.

AED kopen? AED.nl https://www.aed.nl/

AED.nl heeft alle bekende AED merken, elektroden, accu's en accessoires op voorraad. ✓ Gratis Bezorging ✓ Achteraf betalen ✓ Demonstratie op locatie.

AED kopen? Online kieshulp, advies van de specialist ... https://www.aedwinkel.nl/

AED kopen? Kies voor de goede service van de grootste AED winkel van Europa. Ruime keuze

(36)

SERP 3 main group (with advertisements shown)

Je neefje is binnenkort jarig. Hij heeft houten speelgoed gevraagd. Dat kun je

natuurlijk eenvoudig online bestellen. Waar ga je kijken?

Leuk Houten Speelgoed - duketoys.nl Adv.www.duketoys.nl/houten-speelgoed

Voor Jongens, Meisjes & Baby's. Mooi afgewerkt - Direct leverbaar!

Houten Speelgoed Nodig? - Houtenspeelgoedplezier.com Adv.www.houtenspeelgoedplezier.com/Hout

Leuk & Exclusief Houten Speelgoed. Voor Ouders Door Ouders. Bestel Nu!

Bestellen Speelgoed - ILoveSpeelgoed.nl Adv.www.ilovespeelgoed.nl/

Dè site voor origineel speelgoed Voor 16u besteld, morgen in huis. Gratis bezorgd vanaf €50 · Gratis Inpakservice · Binnen 90 dagen ruilen

Houten Speelgoed | Lobbes.nl

www.lobbes.nl/speelgoed/houten-speelgoed

Bekijk het mooie aanbod Houten Speelgoed bij Lobbes. Houten poppenhuizen, keukentjes, treinen en veel meer, voor 17:00 besteld, volgende werkdag in huis!

Houtenspeelgoedplezier: Houten speelgoed online kopen www.houtenspeelgoedplezier.com/

Houten speelgoed online kopen. ... Rollebollen speelwereld. Rollebollen - Basisdoos - Spiraalbaan. Nieuw. Rollebollen - Basisdoos - Spiraalbaan.

Hout-doe duurzaam houten speelgoed Schijndel www.hout-doe.nl/

Hout-doe, duurzaam houten speelgoed in Schijndel en in Hartje ... Bestelling webwinkel en op voorraad, besteld op een werkdag voor 15.00 uur wordt dezelfde ...

(37)

SERP 1 control group (without advertisements shown)

Al jaren draag je Levis 501 broeken. Helaas is je huidige exemplaar aan vervanging

toe. Je wilt online een nieuwe kopen. Welk zoekresultaat kies je?

Levi's® Original 501® - buyonline.com

www.buyonline.com/501

Levi's 501® - webshop.nl

www.webshop.nl/501

Check alle Jeans van Levi's. Score, de jeans store voor mannen!

Levi's® Original 501® - outfit.nu

www.outfit.nu/501

Levi's 501 - Kleding online kopen? | kledingbestellen.nl | Lage prijs

www.kledingbestellen.nl › Kleding

Levi's 501 Ct Customized Tapered jeans shordich 36/34. LEVI'S Boyfriendjeans 501.

Levi's® Jeans, Jackets & Clothing | Levi's® Modeshop.com

www.modeshop.com/NL/nl_NL/

Een nieuw seizoen. Chino's, corduroy en moleskin. Een nieuw seizoen. Nieuwe materialen. Shop nu▷. 501® Original. THE ICONIC STRAIGHT FIT

501® - Levi's | Netherlands

www.jeansgraag.com/NL/nl_NL/collections-home/501

... van de originele jeans. DE ORIGINELE 501® KOPEN ... Geen wonder dat de Levi's 501 -jeans de ultieme blauwe -jeans is. ACQUISTA I 501® ORIGINAL ...

(38)

SERP 2 control group (without advertisements shown)

Op kantoor moet je een defibrillator (AED) hebben. Je besluit hem online te

bestellen. Waar doe je dat?

AED goedkoopste - procardio.nl www.procardio.nl/Philips_actie

HeartStart HS-1 AED, bekijk nu onze aanbieding, direct leverbaar

AED Kopen? - Groot assortiment betrouwbare AED’s www.aedleverancier.nl/

Eenvoudig en snel online bestellen!

Alle merken & type AED's · Gratis bezorging! · Ook service & onderhoud

Online AED kopen? - Best betaalbare AED van NL www.preveco.nl/AED_Kopen

Servicepakket 5 jaar en pads incl !

Geleverd binnen 48uur · Gratis verzending · Gratis servicecontract

AED Kopen ← AEDcompany.nl aedcompany.nl › AED SHOP

U wilt een automatische externe defibrillator (AED) kopen? Bekijk en vergelijk onze AED's hieronder.

AED kopen? AED.nl https://www.aed.nl/

AED.nl heeft alle bekende AED merken, elektroden, accu's en accessoires op voorraad. ✓ Gratis Bezorging ✓ Achteraf betalen ✓ Demonstratie op locatie.

AED kopen? Online kieshulp, advies van de specialist ... https://www.aedwinkel.nl/

AED kopen? Kies voor de goede service van de grootste AED winkel van Europa. Ruime keuze

(39)

SERP 3 control group (without advertisements shown)

Je neefje is binnenkort jarig. Hij heeft houten speelgoed gevraagd. Dat kun je

natuurlijk eenvoudig online bestellen. Waar ga je kijken?

Leuk Houten Speelgoed - duketoys.nl www.duketoys.nl/houten-speelgoed

Voor Jongens, Meisjes & Baby's. Mooi afgewerkt - Direct leverbaar!

Houten Speelgoed Nodig? - Houtenspeelgoedplezier.com www.houtenspeelgoedplezier.com/Hout

Leuk & Exclusief Houten Speelgoed. Voor Ouders Door Ouders. Bestel Nu!

Bestellen Speelgoed - ILoveSpeelgoed.nl www.ilovespeelgoed.nl/

Dè site voor origineel speelgoed Voor 16u besteld, morgen in huis. Gratis bezorgd vanaf €50 · Gratis Inpakservice · Binnen 90 dagen ruilen

Houten Speelgoed | Lobbes.nl

www.lobbes.nl/speelgoed/houten-speelgoed

Bekijk het mooie aanbod Houten Speelgoed bij Lobbes. Houten poppenhuizen, keukentjes, treinen en veel meer, voor 17:00 besteld, volgende werkdag in huis!

Houtenspeelgoedplezier: Houten speelgoed online kopen www.houtenspeelgoedplezier.com/

Houten speelgoed online kopen. ... Rollebollen speelwereld. Rollebollen - Basisdoos - Spiraalbaan. Nieuw. Rollebollen - Basisdoos - Spiraalbaan.

Hout-doe duurzaam houten speelgoed Schijndel www.hout-doe.nl/

Hout-doe, duurzaam houten speelgoed in Schijndel en in Hartje ... Bestelling webwinkel en op voorraad, besteld op een werkdag voor 15.00 uur wordt dezelfde ...

(40)

Tot slot nog enkele stellingen.

Note: een zoekmachineresultatenpagina (Search Engine Result Page (SERP)) is de pagina die bijv Google laat zien nadat de zoekopdracht is uitgevoerd

Note 2: een organisch zoekresultaat is een resultaat op de SERP waarvoor niet wordt betaald aan de zoekmachine.

Hoe hoog een organisch zoekresultaat staat op een SERP hangt sterk af van

de relevantie.

o Helemaal mee eens o Mee eens

o Niet eens / Niet oneens o Mee oneens

o Helemaal mee oneens

Hoe hoog een betaald zoekresultaat (advertentie) staat op een SERP hangt

sterk af van de relevantie.

Op een SERP is het verschil tussen een organisch en betaald zoekresultaat

duidelijk zichtbaar.

Het is goed dat er betaalde zoekresultaten op een SERP staan.

Ik ben blij met betaalde zoekresultaten op een SERP.

Ik vind betaalde zoekresultaten op een SERP handig.

(41)

Ik vind het normaal dat er betaalde zoekresultaten staan op een SERP.

Tijdens dit onderzoek heb ik betaalde zoekresultaten gezien op de

getoonde SERPs.

Tijdens dit onderzoek waren de betaalde zoekresultaten duidelijk

gemarkeerd op de getoonde SERPs.

Als ik op een SERP op een betaald zoekresultaat klik, ben ik me daar altijd

bewust van.

(42)

Het onderzoek is afgerond!

Bedankt voor je bijdrage. Als je je email adres hebt ingevuld, word je op de hoogte gehouden over de uitkomst van dit onderzoek. Nogmaals dank voor de moeite en een hele fijne dag gewenst.

Wierd Schamhart

Heb je vragen naar aanleiding van dit onderzoek? Neem dan contact met mij op via w.schamhart@student.uva.nl