Consumer Click Behavior at a Search Engine: The Role of Knowledge and Ability
If you know that paid results are relevant, do you use them?
Master Thesis - March 25, 2016 University of Amsterdam (UvA)
MSc. in Business Administration – Marketing Track Wierd Schamhart #10002071
Statement of Originality
This document is written by student Wierd Schamhart who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it. The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
Abstract
Over 70% of internet users start their online quest for information on a search engine page. This makes the search engine result page (SERP) an interesting place for advertisers to find an audience for their products and services through advertisements in the form of paid results. How consumers react to these paid results has not yet been fully researched.
Especially the user’s clicking behavior on paid results in relation to organic results and what influences this relationship is not part of current academic knowledge. This paper constructs and tests a framework to understand the influence of knowledge of ranking mechanisms, ability to recognize ads and overall attitude towards ads on the ads clicking behavior. Over 335 respondents have completed the empirical study. Results show that although users have a neutral attitude versus paid results they are avoiding their usage in their actual clicking behavior.
Contents
1 Introduction ... 5
2 Theoretical Framework ... 8
2.1 Search engine result page (SERP) ... 8
2.2 Information search behavior ... 9
2.3 Information search intentions ... 10
2.4 Structure of an online search engine result page ... 11
2.5 An organic result... 13
2.6 A paid result ... 14
2.7 Labelling, recognizing, clicking advertisements ... 14
2.8 Knowing ranking mechanisms, attitude, clicking advertisements... 16
2.9 Framework and hypotheses overview ... 19
3 Methodology ... 20 3.1 Sample ... 20 3.2 Survey design... 21 3.3 Individual measures ... 21 3.4 Analysis strategy ... 22 4 Results ... 23 4.1 Preliminary steps ... 23 4.2 Reliability ... 23 4.3 Correlations... 24 4.4 Hypotheses testing ... 26 5 Discussion ... 29 6 Conclusion ... 32 References ... 33 Appendix – Questionnaire ... 37
1 Introduction
Over the last two decades, online search has become an important channel for finding information on products and services. Search engines play a critical role for businesses by directing customers to their websites (Dou, Lim, Su, Zhou, & Cui, 2010). Many of Internet users reach websites through search engines, instead of direct links from other websites. Search engines hereby claim an important role in the online environment (Shih, Chen, & Chen, 2013).
When a query is entered on a search engine page a user gets a list of search results in return. These results are typically partly sponsored (paid results) and partly non-sponsored (organic results) (Jansen & Schuster, 2011). A large proportion of these search queries is only about finding information (over 80 %) (Jansen, Booth, & Spink, 2008). A much smaller share of search queries is performed to either navigate to a predetermined website (10%) or to perform a transaction (10%) (Jansen et al., 2008).
Identifying these search intentions opens possibilities for search engines to show better targeted results and sponsored results in particular. This is especially interesting in search queries with a transactional intention (Jansen et al., 2008). Assuming that advertisers typically look for the search queries that show a transactional intention. Transactional intention means that a user is about to perform, or in the middle of, a purchase, auction, selling, paid service, etc. (Dai et al., 2006). These transactional intention search queries are more likely to lead to a conversion and therefore return a positive return on investment for the advertiser (Jansen et al., 2008). At the same time these conversions are more likely to occur when an advertiser creates a well aligned conversion funnel, meaning a relevant path from query to transaction (IE a relevant ad to the query, a landing page in line with the ad and a good product offering) (Bagherjeiran, Hatch, & Ratnaparkhi, 2010).
For the three largest search engines (Google, Yahoo!, bing) sponsored results account for the majority of their income (Graepel, Candela, Borchert, & Herbrich, 2010). Using different forms of keyword bid auctions, as part of their ad ranking mechanism, search engines assign spaces to advertisers on their search engine result pages (Graepel et al., 2010). The ad ranking mechanisms decide for every query how many, which and in what order to show sponsored results. By taking into account the bid for the query and the likelihood it will be clicked (click through rate) they maximize their profit (Richardson, Dominowska, & Ragno, 2007). Search engines could increase their short term profits by
showing more sponsored results, but this might lower the overall search performance and over time disengage users (Hillard, Schroedl, Manavoglu, Raghavan, & Leggetter, 2010). To minimize the negative impact of sponsored results search engine needs to align the intention of the searcher and the sponsored results shown (Jansen et al., 2008). This way search engines must balance between business goals and user impact. And by doing that they select the most relevant ads for the user, since these are the most likely to be clicked and generate revenue for the search engine while at the same time keeping search performance high (Edelman, Ostrovsky, & Schwarz, 2005).
While businesses and search engines strive to display the perfect ad tailored to the query of the user and thereby enhance the user’s search experience, users hold a bias against sponsored results (Fallows, 2005; Greenspan, 2004; Hotchkiss, 2004). This can be explained by looking at the alignment of the intention of both the user and the advertiser. While only 10% of the search queries has a transactional intention (Jansen et al., 2008), only a small part of the search queries has a good alignment with sponsored results linking to mainly
commercial webpages (Jerath, Ma, & Park, 2014). Jansen and Resnick tried to find if this bias against sponsored results was any different if only searches with transactional intent were incorporated, but they still found a strong preference for non-sponsored results (Jansen & Resnick, 2006). The bias against sponsored results seems to come from being unaware of their actual relevance. The same study also finds that when users are introduced to relevant sponsored results the bias against sponsored results is overcome (Jansen & Resnick, 2006). For both search engines and businesses using sponsored results overcoming this bias against sponsored results is the task at hand. First for search engines since this is their main source of income, second for businesses to make sure their investment was not wasted.
Although Jansen and Resnick report that previous experience can overcome the bias (Jansen & Resnick, 2006), users still feel unfamiliar with this form of advertising (Fallows, 2005). Users indicate to be unclear about how paid results work and how they can be identified (Fallows, 2005). This study looks how a better understanding of the relevance of sponsored results in a transactional search and the user’s ability to recognize sponsored results influence the user’s click behavior on sponsored and non-sponsored results on a search engine result page.
“How does the user’s ability to recognize advertisements and the user’s knowledge about relevance based ranking mechanisms for advertisements, influence the user’s clicking behavior on a search engine
result page after he performs a transactional search?”
This research paper is built as follows. Chapter 2 introduces relevant literature and thereby creates the theoretical framework and ends with the theoretical model and
hypotheses. Chapter 3 shows the research methodology used to answer the main research question and test the hypotheses. In chapter 4, the results of the data are shown and hypotheses are accepted or rejected. Chapter 5 discusses these findings compared to the theoretical framework and provides explanation for the different effects found. And finally, chapter 6 shows the overall conclusions from the research.
2 Theoretical Framework
To answer the main research question, this chapter examines previous studies and related literature. It starts by introducing the Search Engine Results Page (SERP), followed by online search behavior in general. Then it looks into the different kinds of online search intentions. Next the structure of an online search engine results page is discussed and the specific definitions of an Organic and a Paid search engine result. The last three sections introduce:
the user’s ability to recognize advertisements and
knowledge about relevance based ranking mechanisms for advertisements. Finally an overview of the Hypotheses and Framework is provided.
2.1 Search engine result page (SERP)
In the beginning of the internet, search engines did not exist. Instead, listing platforms provided a catalogue of pages as found by their employees. After the fast growth of the internet during the nineties (Albert, Jeong, & Barabási, 1999), the indexing platforms such as Yahoo Directory needed to reinvent themselves to cope with the new demand. The platform, Yahoo Directory, shifted to Yahoo Search Engine using an automated indexing method. It started using the words on the page, known as keywords, to index the page (Brin & Page, 2012). A search query now triggered results from pages with the same words on it. A few years later in 1998, Google developed the algorithm Page Rank which started to take
incoming links into account to determine the importance of a page (Page, Brin, Motwani, & Winograd, 1999). The underlying assumption was that a higher number of incoming links would reflect a higher popularity and therefore higher importance. The PageRank algorithm made searching by words and phrases possible. With the development of semantic search, this became even better (Guha, McCool, & Miller, 2003). These algorithms are the
foundation of the search engine experience we now take for granted.
From a user’s point of view, the most important thing for a search engine is to provide relevant results to the input of the user’s search query. Search engines employ a large amount of factors to decide the relevance and to rank the results shown on the result page. The average user has little idea what leads to the ranking of the different results shown (Höchstötter & Lewandowski, 2009). The only ranking mechanism that is highlighted on the result page is the paid vs. organic results. Paid results are the results generated after the
search engine received payment for their placement. On the contrary, the organic results are generated by the ranking mechanism of the search engine without receiving payment for their placement. But even with ads highlighted, most users are unaware of ranking mechanisms including paid results (Marable, 2003).
2.2 Information search behavior
Studies of users search behavior stems from long before there were any online possibilities. Information and library science looked at why users search and what strategies they employ to do this effectively. When the online context was introduced, researchers started to apply these questions to the new online context. Take Bates for example. In the seventies, he looked at ways people searched and their search tactics (Bates, 1979). Later on in the late eighties, he looked at information browsing techniques and how these techniques were employed in an online search context (Bates, 1989). He suggested that search queries are no longer static, but actually evolve during the search process. Searchers are no longer looking for one piece of information that contains all, but are rather looking for smaller bites of information to compose a whole like ‘berrypicking’ (Bates, 1989).
After online search engines became popular, research on this topic followed quickly. One of the early studies shows how users predominantly use short search queries for their online search questions (Silverstein, Marais, Henzinger, & Moricz, 1999). A round up of these early studies and their findings was made by Jansen and Pooch (2001).
The research by Jansen and Spink studied how online search behavior developed over time between 1997 and 2001 (Jansen & Spink, 2006). By looking at query logs from this period, they found that although some minor changes, for instance, the declining willingness to look at more results, the overall search strategies are fairly constant (Jansen & Spink, 2006).
Online searchers tend to quickly evaluate the search results before clicking at one or two results (Spink & Jansen, 2006). They do not want to scroll down, over half of them only looks at page one of the search results and about half the searchers only tries one query per search session (Hotchkiss, Garrison, & Jensen, 2005).
Before the invention of the World Wide Web, the general search goal was
informational, meaning the searcher was looking to ‘find out’ about his preferred topic (Rose & Levinson, 2004). Both due to the users with access to search engines (researchers,
students, etc.) and the kind of databases and their content, this was a fair assumption (Rose & Levinson, 2004).
In contrast to that, in a web based search environment the goal of search is far broader than just informational research. Just a quick look at a query log of a large online search engine (such as Yahoo, Google) shows that there are more goals in play then just finding out.
All these studies together paint a picture of how users expand their search scope from information only to just about anything and how they are doing that, but the next question is why they are searching online. What is the intention behind their search queries?
2.3 Information search intentions
Broder finds three different intentions of webs searchers to answer the question of why users search online (Broder, 2002). For this research, he looks at the search query and tries to mark the kind of information retrieval the user is looking for. He calls this the ‘need behind the query’. He finds three needs.
He classifies the first need as the Navigational intention. The user needs to find an address to navigate to. His intention is to reach a predetermined website. This can be due to a prior visit to the site or the belief that such a site exists. This kind of searches usually has only one desired outcome that is the page the user already has in mind (Broder, 2002). Since there is only one desired outcome, the results are easily judged on relevance and only the desired result is clicked if available.
The second need Broder (2002) classifies is the Informational intention. The user intents to find some information he assumes to be present somewhere on the web. This intent is the web equivalent to the former ‘find out’ goal. The user wants the information in a clear form and wants to consume this information, but has no further intention to interact with the information. In this type of search the user looks for an overview of the information rather than one particular page. The user hereby creates an overview of the available information rather than the information assembled by one site (Broder, 2002). Therefore, there are more than one desired outcome in a search result and the user is more likely to click several results creating an overview of information he is looking for.
The third need classified by Broder (2002) is the Transactional intention. When a user has this intention, he is looking to perform some sort of web-mediated activity. When the user finds the site with the intended content, he is likely to further interact with the content of the site to fulfill the desired transaction. This desired transaction defines his transactional
intention. These transactions can involve shopping, downloading, gaming, or any other web mediated transaction (Broder, 2002). The results to the query are hard to judge for relevance since factors like pricing, service and quality play an important part. These are the factors that are often not present on the result page itself (Jansen et al., 2008). At the same time the transactional intention is presumably the search intention with the best alignment between searcher and advertiser (Jansen et al., 2008). Because it is hard to judge for relevance for this type of query it is expected that a user’s knowledge of relevance based ranking mechanisms will change the actual choice to click. Therefore this study focuses on this transactional intention and the impact of knowledge on ranking mechanisms.
2.4 Structure of an online search engine result page
What a search engine result page shows differs between different search engines. Looking at the search engine usage in The Netherlands shows a remarkable market dominance of Google (see figure 1). Well over ninety percent (94.07%) of the online searches are done via Google, with Bing a far second with only two and a half percent (2.56%)1. With a far majority of users in The Netherlands using Google, this study into user’s behavior on search engine result pages uses Google as main search engine example for the remainder of this study.
A typical Google result page for the search query ‘Theater’ might look like the print screen in figure 1. This ‘might’ be the case because search engines use personalized results. This is a factor that cannot be ignored since it severely influences the results and at the same time, it cannot be turned off.
1 Source StatCounter http://gs.statcounter.com/#desktop+mobile+tablet-search_engine-NL-monthly-201401-201501-bar (retrieved on 23-12-15)
Figure 1
2
First of all the page is built up as a list. The list has the ‘best answer’ at the top, which is the highest possible position on the result page.
There are several areas that can be defined on a search engine result page. Firstly, there is an area visible at first sight and an area that can only be seen after scrolling. These two areas are referred to as above and below the fold. Since different users use different screen resolutions, the fold is not a set limit. On average, at least six results will be visible before the fold in the left column.
2 Source Google https://www.google.nl/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=theater (retrieved 23-12-15)
Second distinction is that between organic results and paid results. These two sorts of results are discussed in section 2.5 and 2.6 respectively. Typically the search engine result page is made up out of organic results and surrounded by paid results. Paid results are marked and can be placed in the top three results as seen on the example above, but can also be placed on the bottom of the page (not shown in example since this is a print screen. The bottom results are below the fold). There is also a column on the right. These are also paid results. Google uses the query to determine whether or not it displays paid results and whether those will be shown above, below or to the right of the organic results. A yellow “AD” label marks all paid results. Notice that the results in the main column (left) are individually labeled and the paid results on the right are marked as a total column. This column on the right will never show organic results.
Figure 2
3
Two schematic markups by Google (see figure 2) show the organic results outlined in red in the figure on the left and the paid results outlined in red in the figure on the right.
2.5 An organic result
Online search engines typically offer two kinds of results, organic results and paid results. The organic result is the result returned by the search engines algorithm. The organic result is also referred to as the natural result, the non-paid results and the unsponsored result. The referred site did not pay the search engine for the organic results.
3 Source Google: https://support.google.com/adwords/answer/1722080?hl=nl&ref_topic=3121771 (retrieved on 23-12-15)
The search engine ranking mechanism (Google uses an algorithm known as PageRank) returns these results combining a couple of factors. The relevance of the page content to the search query and the popularity of the referred page determine the PageRank of the page. This PageRank determines where the result is placed on the result page. A higher PageRank leads to a higher placement (Brin & Page, 2012).
2.6 A paid result
Next to organic results, search engines often provide paid results. Paid results are also referred to as sponsored results and advertisements. Paid results are, as the name suggests, results paid for by an advertiser.
The paid results returned by the search engine are determined by another search engine ranking mechanism than the organic results. Google calls this algorithm the AdRank4.
This algorithm also uses the relevance of the content of advertisement to the search query and combines this with a cost per click bid that the advertiser can set.
Besides the yellow “AD” label there are several other features that separate the way paid results and organic results are presented on a search engine results page. A paid results on Google has a heading of only 25 characters5, where an organic result can have a title of
approximately 50-60 characters6 (this actually is a pixel based cutoff so using more W
lowers your maximum character count as opposed to using more I’s increasing the maximum character count). Also the number of characters in the description is different. An
advertisement on Google can have two description lines with both a maximum of 35
characters7. An organic result can have up to approximately 150-160 characters8 (again with the actual cutoff based on pixels).
2.7 Labelling, recognizing, clicking advertisements
To better understand the user’s click behavior on sponsored and non-sponsored results on a search engine result page, after a transactional search query, the question is: how is this behavior influenced by the user’s ability to recognize sponsored results? The next
4 Source Google https://support.google.com/adwords/answer/1752122?hl=en (retrieved 24-12-15) 5 Source Google https://support.google.com/adwords/answer/1704389?hl=en (retrieved 21-3-16) 6 Source MOZ https://moz.com/learn/seo/title-tag (retrieved 21-3-16)
7 Source Google https://support.google.com/adwords/answer/1704389?hl=en (retrieved 21-3-16) 8 Source MOZ https://moz.com/learn/seo/meta-description (retrieved on 21-3-16)
section refers to the existing research and builds hypotheses and a framework from that for this relationship.
The likelihood of a result being clicked is a curvilinear function of the ranking that is the placement on the page (Brooks, 2004). Study suggests this is true for both organic and paid results. The ranking seems to be an intrinsic relevance measure. In 2004 a study by Greenspan suggested that paid results have a higher click rate if the search engine does not clearly label them (Greenspan, 2004). This study therefor hypothesizes that:
H1 Labelling of advertisements is negatively related the probability of clicking
advertisements.
Besides labels there are several distinct features that separate paid results from organic results. Still Hotchkiss found that novice users have trouble identifying paid results and that the paid results receive lower quality rating than the organic results (Hotchkiss, 2004). The preference for organic results was also supported by the study of Greenspan (2004). In 2005 only 38% of the respondents said that they are aware of the distinction between organic and paid results. Fewer than 17% of the respondents said that they can always identify the difference between paid and organic results (Fallows, 2005).
Research by Jansen and Spink shows no difference in click rates for paid results if they are mixed into the organic result. From their results, they conclude that users are well equipped to judge the result for its relevance to their search query and base their clicking behavior on that relevance and thus, are indifferent to organic versus paid results (Jansen & Spink, 2008). However, they do not look at the ability of their subject to distinguish between organic and paid results. So the research only shows that the general clicking behavior is based on the relevance of the link and not by the result being paid or organic.
Since the paid results are placed above the organic results and the likelihood of being clicked is a curvilinear function of the placement, the paid result should be more likely to be clicked if the user see no difference between paid and organic results, or in other words is unable to recognize the paid result. This study therefor hypothesizes that:
H2 The ability to recognize advertisements is negatively related to the probability of
clicking advertisements.
As proposed in the previous paragraphs labelling of advertisements and the ability to recognize advertisements both impact the probability of clicking advertisements. With labelling being the most obvious way to distinct between the two, it can be expected that labelling advertisements also impacts the user’s ability to recognize advertisements. This
study therefor hypothesizes that:
H3 Labelling of advertisements is positively related to the ability to recognize
advertisements.
If labelling advertisements impacts the user’s ability to recognize advertisements, and that in return impacts the user’s probability of clicking advertisements, the ability to
recognize advertisements might play a mediating role in the relationship between the labelling advertisements and the probability of clicking advertisements. This study therefor hypothesizes that:
H4 The ability to recognize advertisements mediates the effect of labelling
advertisements has on the probability of clicking advertisements.
2.8 Knowing ranking mechanisms, attitude, clicking advertisements
Besides the ability to recognize sponsored results, knowing how relevant they are might also impact the probability of clicking sponsored results.
In 2005, participants seemed to trust search engines but they reported to be unclear about how the results rank and how they are presented (Fallows, 2005). Another survey in 2007 with nearly 6000 respondents finds users complaining about vague ranking
mechanisms for paid results (Höchstötter & Lewandowski, 2009). Research by Jansen and Spink suggested that users rate the relevance of organic and paid results as equal, if no distinction is made between the two (Jansen & Spink, 2008). Still an empirical study by Jansen and Resnick (2006) found paid results are less likely to be clicked. They tested two groups. One group had the paid results labeled as such and the other had the organic results labeled as paid and the paid as not labeled (so perceived as organic). With the two groups combined more than 80% of the first clicks were on a not labeled so ‘organic’ result. Paid results received just 6% of the first clicks (Jansen & Resnick, 2006). These findings
combined suggests that users avoid clicking on paid results due to their lack of knowledge of why they rank high and not due to their actual relevance. This study suggest that if users are aware of how important a factor relevance in advertisements ranking mechanisms they are more likely to click on them. This study therefor hypothesizes that:
H5 The user’s knowledge about relevance based ranking mechanisms for advertisements
is positively related to the probability of clicking advertisements
Although organic and paid results are both on the same page and both are listed because their relevance is judged as high by the search engine, searchers have different attitudes towards paid results versus organic results. While a study in 2012 among 2253 adults found users to be “more satisfied than ever with the quality of search results” (Purcell, Brenner, & Rainie, 2012), other studies show that there is an overall negative attitude against paid results (Fain & Pedersen, 2006; Gauzente, 2009; Jansen & Pooch, 2001).
Results from a survey by Georgia Tech University (Xing & Lin, 2006), suggest that the attitude for paid results is negative since they are believed to be less objective and more biased than organic results. Over 70% of the respondents said to prefer clicking on an organic listing over clicking on a paid result (Xing & Lin, 2006). This concurs with the empirical study of Jansen and Resnick (2006). An empirical study conducted a few years earlier also found more clicks for organic results. A survey of SEMPRO showed that over 70% of first clicks were on organic results (Xing & Lin, 2006). In 2005 participants said to click on results from trusted source with unbiased information. Over 77% of these
participants said to favor organic over paid results (Hotchkiss et al., 2005). This preference even looks to hold up if the search query has a transactional intention.
There are several studies that suggest a strong preference for organic results and a negative attitude towards paid search results (Jansen & Resnick, 2005). This means that if a result is labeled as a paid result, a consumer rates its relevance to his search query lower than when the same result is not labeled as a paid result. At the same time it means that when two results are perceived as equally relevant and one is labeled as paid result and the other is not, the searcher prefers the non-paid result (Jansen & Resnick, 2005). This suggests that the stronger the negative attitude towards sponsored results is, the less likely the user is to click a sponsored result. This study suggests the reverse is also true; the more positive the attitude towards sponsored results, the more likely a paid results is clicked. This study therefor hypothesizes that:
H6 The attitude towards advertisements is positively related to the probability of clicking
advertisements.
Several studies researched the concept of attitude in the internet context using a hierarchy of effect models (Chen and Wells 1999; Stevenson et al. 2000; Lee et al. 2004). Attitude and hierarchy of effect models have been shown to add to the understanding and explanation of consumer online behavior (Gauzente, 2009). A hierarchy of effect model can be used to explain how attitude drives behavior (Gauzente, 2009). The core hierarchy of the model is that attitude drives intention and intention drives behavior. The construct attitude is formed in different ways. It can be seen as an outcome of only affect, but also as a
combination of the dimensions cognition and affect or even an outcome of three dimensions affects, cognitive and behavioral (Yoo, Kim, & Stout, 2004). If knowledge is a dimension driving attitude than knowledge about relevance based ranking mechanisms should drive a user’s attitude towards paid results. This study therefor hypothesizes that:
H7 The user’s knowledge about relevance based ranking mechanisms for advertisements
is positively related to the attitude towards advertisements.
If the user’s knowledge about relevance based ranking mechanisms for
advertisements drives his attitude towards advertisements, and that in return impacts the user’s probability of clicking advertisements, the attitude towards advertisements might play a mediating role in the relationship between the user’s knowledge about relevance based ranking mechanisms for advertisements and the probability of clicking advertisements. This study therefor hypothesizes that:
H8 The attitude towards advertisements mediates the effect of the user’s knowledge
about relevance based ranking mechanisms for advertisements on the probability of clicking advertisements
2.9 Framework and hypotheses overview Main research question:
How does the user’s ability to recognize advertisements and the user’s knowledge about relevance based ranking mechanisms for advertisements influence the user’s clicking behavior on a search engine result page after he performs a transactional search?
H1 Labelling of advertisements is negatively related the probability of clicking advertisements.
H2 The ability to recognize advertisements is negatively related to the probability of clicking advertisements. H3 Labelling of advertisements is positively related to the ability to recognize advertisements.
H4 The ability to recognize advertisements mediates the effect of labelling advertisements has on the probability of clicking advertisements.
H5 The user’s knowledge about relevance based ranking mechanisms for advertisements is positively related to the probability of clicking advertisements
H6 The attitude towards advertisements is positively related to the probability of clicking advertisements. H7 The user’s knowledge about relevance based ranking mechanisms for advertisements is positively related
to the attitude towards advertisements.
H8 The attitude towards advertisements mediates the effect of the user’s knowledge about relevance based ranking mechanisms for advertisements on the probability of clicking advertisements
3 Methodology
This section provides an overview of the methodology used in this study. The next paragraphs give an overview of the sample, data collection and analyze strategy used to test the hypotheses.
3.1 Sample
An online survey was used to obtain the data for this research. Most likely the
majority of participants would be Dutch natives. Therefor only a Dutch version of the survey was distributed. The questionnaire was opened on the 24th of October 2015. Via email, social media (Facebook and LinkedIn) and word of mouth, friends, family, colleagues and acquaintances were asked to participate in the survey and forward it to their friends and relatives. Just four weeks later on the 20th of November 2015 the survey was closed.
Objective of the survey was to obtain data about people’s attitude towards paid results, their knowledge of ranking mechanisms, their ability to recognize paid results and the paid versus organic results clicking behavior.
After 28 days, 395 people replied to the survey request, with 335 completing the entire questionnaire, a completion rate of 84.8%. Gender division was 150 female (45%) to 185 male (55%). Highest level of education ranges from high school (7.5%) to Doctoral (3%), but the majority has a university degree (37%). To the question how often respondents use search engines only one respondent answered a few times per month, while 167
respondents (50%) replied with multiple times a day. The sample details are listed in table 1. Table 1: Sample Characteristics (n = 335)
Gender Male 185 (55%)
Female 150 (45%)
Educational level High School 25 (7.5%)
Intermediate Vocational Education 63 (18.8%)
Bachelors 113 (33.7%)
Masters 124 (37%)
Doctoral 10 (3%)
SE use freqency Few times a month 1 (.3%)
Few times a week 8 (2.4%)
Multiple times a week 45 (13.4%)
Daily 114 (34%)
3.2 Survey design
The questionnaire was created to research knowledge about ranking mechanisms, attitude towards paid results, ability to recognize paid results and clicking behavior on paid versus non paid results. At the start respondents were asked for some demographic control variables (gender, educational level) and their frequency of search engine usages. Next they were randomly assigned to two independent groups to check for the general effect of labeling a paid result on a search engine result page. Finally the respondents were asked questions to determine their attitude towards paid results and knowledge of ranking mechanisms. The full questionnaire is printed in the appendix.
3.3 Individual measures
Independent variables. The attitude towards paid results (4 items) and the ability to to recognize ads (3 items) were measured using a 5-point Likert scale (1 totally disagree to 5 totally agree). The items for attitude were derived from previous research by Gauzente (2009) and the items for ability to recognize from Fallows (2005). The independent items were subjected to a principal components analysis. The analysis showed a one factor scale with loadings between .69 and .87 and explaining a variance of 68% for attitude and for ability to recognize a one factor scale with loadings between .70 and .87 and explaining a variance of 63%. Scale reliability was high with Cronbach’s Alpha respectively .83 and .79. Knowledge of weight of relevance in paid results ranking was measured through a single item.
Dependent variable. Respondents were given a search engine result page. They were asked to select the result they would click if they had performed the given query leading up to the displayed result page. These clicks were recorded and coded as a click on organic result (0) or a click on paid result (1). An alternative query was used to the one used for transactional searching used by Jansen, Brown and Resnick (2007). Keeping the main ideas for the query in place, respondents are supposedly ready to buy a specific product; therefore they are looking for a seller of this particular product (Broder, 2002). The searcvh engine result page used is created from a real search engine result page with the small distinction that any vendor information that could influence the respondent trough prior experience was taken out of the result and replaced by a generic nonexistent vendor name. Second difference was that only the top six results were visible, as would be the above the fold part. The six results consisted of the first three paid results and the first three organic results generated by
the original search engine (google.nl). The layout of the results was consistent with that used at the original search engine result page. After a short textual introduction of the search context the respondents were asked to select the result that, according to them, they were the most likely to select in a real search task.
3.4 Analysis strategy
To test the hypotheses SPSS was used. Since the different natures of the independent (ordinal/ binary), dependent (binary) and proposed mediation variables (continuous) several analyses were used. A Chi square test was used for H1 and H5, logistic regression for H2 and
H6 and One way Anova for H3 and H7. To test H4 and H8, the mediation test procedure
Process macro model 4 was used (Preacher & Hayes, 2008). This macro tests several steps to test for mediation.
It starts testing the direct relation between the independent variable (X) and the
dependent variable (Y) (path c). Second it tests the relation between the independent variable and the proposed mediation variable (M) (path a). Third the relation between the proposed mediation variable and the dependent variable is tested (path b). Finally it tests the effect of the independent variable on the dependent variable controlling for the effect of the proposed mediation variable (path c’). If path a, b and c are significant, the difference between the c and the c’ path is tested. If this difference is significant mediation has occurred. This difference was analyzed using the confidence intervals of the bootstrapping.
This section reported on the methods used to acquire the data. The next section reports on the results of the survey.
4 Results
This chapter presents the results of the study, which have been analyzed using SPSS. Based on these analyses the suggested hypotheses are accepted or rejected. First the
preliminary steps needed to clean the data obtained from the questionnaire are explained. Second the reliability analyses are provided. Third correlations between the variables used in the study are covered. The fourth part covers the actual testing of the hypotheses
4.1 Preliminary steps
395 subjects filled out the Qualtrics survey. 335 subjects completed the entire survey. The uncompleted surveys were not used for analysis. After striking the uncompleted surveys from the record, no missing values were left in the dataset. The subjects were randomly divided into two groups. One group (n=167) was shown a version of the search engine result page with labeled paid search results. Another group (n=168) was shown the same search engine result page but without the label for paid results.
4.2 Reliability
The reliability for the attitude toward paid results scale (AdAttit) is high with Chronbach’s Alpha = .834. All the items have sufficient correlation with the total scale outcome, indicated by corrected item-total correlations of above .30 (for all items). None of the items would substantially affect the scales reliability if it was deleted.
The reliability for the ability to recognize a paid result scale (AdRecon) is sufficient with Chronbach’s Alpha = .788. All the items have sufficient correlation with the total scale outcome, indicated by corrected item-total correlations of above .30 (for all items). None of the items would substantially affect the scales reliability if they were deleted.
The scales were analyzed with a principal axis factoring analyses. The sampling adequacy was verified with the Kaiser-Meyer-Olkin measure, KMO= .744. Bartlett’s test of sphericity χ2 (28) = 526,834, p<.000. This indicates that the correlation between the items was large enough for principal axis factoring. A first analysis was done to see the
eigenvalues of the components in the data. Two components returned eigenvalues of over 1, the Kaisers criterion. Together they explained 65.9 % of the variance. Also the scree plot showed a leveling off after the second factor. Therefore, two factors were used and rotated with an Obliminal with Kaiser Normalization rotation. Table 2 shows the rotated factor loadings. The items clustered in factor one suggest that this factor represents attitude toward
paid results and the items clustered in factor two suggest this factor represents the ability to recognize a paid result.
Table 2: Principal axis factoring analyses Item
Rotated Factor Loadings AdAttitu AdRecon Ik vind betaalde zoekresultaten op een SERP handig. .861 -.146 Het is goed dat er betaalde zoekresultaten op een SERP staan. .783 .008 Ik ben blij met betaalde zoekresultaten op een SERP. .777 -.016 Ik vind het normaal dat er betaalde zoekresultaten staan op een SERP. .582 .295 Tijdens dit onderzoek waren de betaalde zoekresultaten duidelijk gemarkeerd op
de getoonde SERPs. .032 .919
Als ik op een SERP op een betaald zoekresultaat klik, ben ik me daar altijd
bewust van. .003 .698
Tijdens dit onderzoek heb ik betaalde zoekresultaten gezien op de
getoonde SERPs. -.080 .616
Op een SERP is het verschil tussen een organisch en betaald zoekresultaat
duidelijk zichtbaar. .110 .573
Eigenvalues 2.76 2.52
% of variance 34.47 31.44
Note: factor loadings > .40 appear bold 4.3 Correlations
After the scales were computed, a correlation analysis was done to quantify the meaning and intensity of the relationship between the control and scale variables. Results are shown in table 3. The results show that all but one of the dependent, independent and
mediator variable have a significant tendency to a correlation. For instance, there is a
significant tendency towards a positive relation between the labeling of ads and the ability to recognize ads (r= .48; p<.001). The control variables do not significantly correlate to the outcome variable (clicking on the ad) except from search engine use frequency (r=-.16; p<.01).
8 (.7 9 ) -.28 ** 7 (.8 3 ) .12 * .13 * 6 .05 .48 ** -.31 ** 5 .04 .30 ** .09 .05 4 .29 ** -.03 .25 ** .26 ** -0 .01 3 .07 -.04 -.05 -.05 .16 ** -.16 ** Tabl e 3 : M ean s, Stand ard D ev iati o n s, C o rrela tio n s 2 .16 ** .01 -.07 -.08 -.06 .00 -.09 * Co rr elati o n is sig n ifi ca n t at the 0 .0 5 lev el ( 2 -tailed ) ** C o rrela tio n is sign ifi ca n t at t h e 0 .0 1 lev el ( 2 -tailed ) 1 -.07 .11 * .12 * .04 .07 .05 .13* -.04 SD .5 0 .9 9 .8 1 1 .0 2 1 .1 5 .5 0 .8 2 .8 4 .5 0 M .5 5 4 .0 9 5 .3 1 3 .5 5 2 .7 6 .5 0 2 .9 5 3 .3 4 .5 1 (N= 3 3 5 ) G e n d e r (fe m a le =0 , m a le = 1 ) Hig h e s t e d u c a ti o n (1 =Pr im a ry Sc h o o l, 6 =D o c to ra l) Fre q u e n c y SE u s e (n e v e r=1 , m u lt ip le t im e s a d a y = 6 ) im p o rta n c e o f re le v a n c e i n o rg a n ic ra n k in g m e c h a n is m s ( s tro n g ly d is a g re e =1 , s tro n g ly a g re e =5 ) im p o rta n c e o f re le v a n c e i n a d ra n k in g m e c h a n is m s (s tro n g ly d is a g re e =1 , s tr o n g ly a g re e = 5 ) la b e lli n g o f t h e a d (n o t la b e lle d = 0 , la b e lle d =1 ) Att itu d e t o war d s a d s (n e g a ti v e = 1 , p o s iti v e =5 ) Ab ili ty t o re c o g n iz e a d s (b a d = 1 , g o o d =5 ) c lic k in g o n t h e ad (o rg a n ic =0 , a d =1 ) 1 2 3 4 5 6 7 8 9
4.4 Hypotheses testing
Hypothesis 1 proposes that labelling of advertisements is negatively related the probability of clicking advertisements by a user in a search with transactional intent. A Chi-square analysis between the independent variable labelling of the ad and the dependent variable clicking on the ad confirms this hypothesis (χ2 = 32.912, p=.000 ). Based on the Phi statistic (.313) the effects is of medium size. Interpreting the frequency distribution, this seems to represent that the probability of clicking an ad is lower when it’s labeled than when it’s not labeled.
Hypothesis 2 proposes that the ability to recognize advertisements is negatively related to the probability of clicking advertisements. A logistic regression analyses between the independent variable ability to recognize ads and the dependent variable clicking on the ad states this hypothesis is to be accepted (Wald= 25.05; p=.000).
Hypothesis 3 proposes that the labelling of advertisements is positively related to the ability to recognize advertisements. A one-way ANOVA analyses between the independent variable labelling of the ad and the dependent variable ability to recognize ads states there was a significant effect (F(1, 133) = 98.55, p= .000). Looking at the means scores of ability to recognize ads from the group with ads labeled (2.99) and of the group with ads not labeled (2.91) the positive relation in the hypothesis is to be accepted.
Until now all the hypothesis have been accepted. The next analyses aims to combine the variables in one model. This analyses was done using the PROCESS macro model 4 from Hayes (2015). Labelling of the ad was submitted into the model as the independent variable (X). Clicking on the ad was entered as the dependent variable (Y). Finally ability to recognize ads was set as mediator variables (M) in the model to test for the mediation effect. To start, in concurrence with hypothesis 1, the result indicates that the labelling of the ad negatively relate to probability an ad is clicked (c path; b=-1.298, Z=-5.64, p=.000). Second, in concurrence with hypothesis 2, the analyses showed that the ability to recognize
advertisements is also negatively related to the probability of clicking advertisements (b path; b=-.457, Z= -2.8709, p=.004). Third, again in concurrence with hypothesis 3, the analyses showed that that the labelling of advertisements is positively related to the ability to recognize advertisements (a path; b=.80, t(333)= 9.93, p=.000). Finally since the a, b and c path were significant, mediation was tested with the bootstrapping method with bias corrected confidence estimates. With 1000 bootstrap resamples, a 95% confidence interval was created (Preacher & Hayes, 2008). The outcome indicates that the ability to recognize
ads (b=-.367, CI=-.65 to -.11) has a significant mediation effect on the relation between the labelling of ands and the probability ads are clicked. Although the c’ path did not become insignificant by the effect (c’ path; b=-.964, Z=-3.7653, p=.000), the change was strong enough that partial mediation is accepted.
Hypothesis 5 proposes that the user’s knowledge about relevance based ranking mechanisms for advertisements is positively related to the probability of clicking advertisements. A Chi-square analysis between the independent variable importance of relevance in ad ranking mechanisms and the dependent variable clicking on the ad rejects this hypothesis (χ2 = 1.177, p=.882 ). This means there is no direct effect between the user’s
knowledge about relevance based ranking mechanisms for advertisements and the probability of clicking advertisements. Hypothesis 5 is rejected.
Hypothesis 6 proposes that the attitude towards advertisements is positively related to the probability of clicking advertisements. Using a logistic regression analyses between the independent variable attitude towards ads and the dependent variable clicking on the ad states this hypothesis is to be accepted with a significance of p<.05 (Wald= 5.35; p=.021).
Hypothesis 7 proposes that the user’s knowledge about relevance based ranking mechanisms for advertisements is positively related to the attitude towards advertisements. A one-way ANOVA analyses between the independent variable importance of relevance in ad ranking mechanisms and the dependent variable attitude towards ads states there was a significant effect (F(4, 130) = 8.48, p= .000). Post hoc comparisons using the Tukey test indicated that the mean score for attitude towards ads are significantly different between groups that either strongly disagree (M = 2.52, SD = .83) or disagree (M = 2.83, SD = .74) and the groups that either agree (M = 3.27, SD = .80) or strongly agree (M = 3.34, SD = 1.12) on the statement that relevance is important in ad ranking mechanisms. However, the
group that neither agree nor disagree (M = 2.91, SD = .71) did not significantly differ from any of the other groups.
Hypothesis 8 proposes that the attitude towards advertisements mediates the effect of the user’s knowledge about relevance based ranking mechanisms for advertisements on the probability of clicking advertisements. Since hypothesis 5 is rejected there is no direct effect the independent variable importance of relevance in ad ranking mechanisms and the
dependent variable clicking on the ad therefor there can be no mediating effect on this relationship, the hypothesis is rejected.
5 Discussion
This study researched how the user’s attitude towards advertisements and his clicking behavior on a search engine result page, after he performs a transactional search, is
influenced by the user’s knowledge about relevance based ranking mechanisms for
advertisements and his ability to recognize advertisements. The previous chapter presented the results of the survey. The next part discusses the meaning of these results.
The paid results represent an easy way for businesses to find a new audience for their website. To see what factors influence the attitude towards and clicking behavior on paid results this study created and tested a new framework. Results show promising aspects of the model and create several opportunities for future research. The only hypothesis that was not confirmed was the direct effect of knowledge of ad ranking mechanisms on clicking ad probability. A potential explanation is that the effect is too small to be significant in a small sample survey like this study. Another explanation is that since the found effect on attitude is small, it could be that variables that weren’t in this framework like perceived risk associated with clicks on paid results also influence the clicking probability. This study however builds a framework that shows that attitude towards paid results and the ability to recognize ads explain parts of a user’s ad clicking probability.
From a theoretical standpoint this study creates a conceptual view of possible mechanisms at play when a user is shown a result page. This conceptual view was not available in previous studies. In particular the placement of knowledge and ability into the framework was lacking in earlier research. Also this study proposes a scale for ability to recognize ads and proves its validity. The framework and developed scale are a first step for future research to build on. To better understand whether knowledge of ad ranking has an effect on clicking behavior a solid measurement for this factor needs to be developed.
Another outcome of this research is the neutral attitude users have towards paid results. While previous research found users to be negative towards paid results (Xing & Lin, 2006) this study finds users to be neutral towards paid results. This could be explained by the more familiar users have gotten with paid results over the last years. Early 2000 studies suggested that users were unfamiliar with paid results and therefore are unconcerned with them (Marable, 2003). A few years later Jansen and Resnick found that if users click on a paid result they are more likely to click on paid results in the future (2006). A small decade later users have gotten far more familiar with online search in general and paid results in
particular. Over half of the respondents’ uses search engines a couple of times a day and overall users say to be able to recognize ads on the result pages. Yet this is the discrepancy found in this research between what users say and think and what they do. While on the one hand they are neutral towards paid results, their actions suggest otherwise. The results showed that results labeled as a paid result where less likely to be clicked than when they were not labeled as such. This means that users think to be neutral about paid placements but are reluctant to actually click on them. This clicking behavior was also found by Jansen and Resnick (2006) but they concluded from this that users have a negative attitude towards paid results. The results from our survey suggest this conclusion is missing nuance. However the results did show that the there is a link between attitude and behavior. Our tests confirmed that if the user has a more positive attitude towards paid results he is more likely to click on them. It’s just not an equal match since the attitude is neutral and the behavior shows an overall unwillingness to click paid results. An explanation for this difference is that there is another factor driving attitude not yet measured in the attitude scale. A variable like previous experience, trust or relevance could be a more determining factor and should be included in future research.
Results also show that on average users think that relevance is not an important ranking factor for paid results, while at the same time they do think relevance is an important ranking factor for organic results. This is in contrast with how users actually perceive the information linked to in a transactional search. Previous study showed that users experience the same relevance for content linked to by paid results as to content linked to by organic results in a search with transactional intention (Jansen & Spink, 2008).
Finally this study finds a mediating role of the user’s ability to recognize ads on the relationship between labelling advertisements and if ads are clicking. It seems fairly logical that the labeling influences the recognizing, but this relationship also shows there is more to recognizing advertisements than just the label. Future study should pay attention to this. If the difference between organic and paid is examined just removing the label is not the same as presenting the same sort of results. Users are aware of differences between the
presentation of paid and organic results beyond just the label.
From a managerial point of view this study shows that users hold no negative feelings against paid results that could in turn reflect on the business using them, but at the same time are far more likely to click on the organic results. So while paid results are a fast and easy
way to get a top of the page placement, investing in top of the page placements for organic results (which is harder and takes more time) is likely to return larger sums of traffic.
Since the results show that people indicate to recognize ads business should be aware of the implications of using ads for their business. Further research could see whether there are in fact spillovers from the attitude versus paid results to the attitude towards the brand using those marketing options.
There are a few methodological limits in the study. First of all there are some limits due to the data collection process. The sample population is more technologically
sophisticated than the average population. This comes from the methods used to acquire respondents. Respondents were approached via various online media such as email and social media like Facebook and LinkedIn. By asking respondents to ask their relatives (snowball technique) this effect was reinforced. Therefor future studies may focus on respondents that are less technologically sophisticated to enhance the generalizability of the findings. Second limitation to reconsider in future studies is the scenario approach. The scenarios left no options for respondents to choose their own search focus, but rather limited the search option to the scenario of the study. A the same time the respondents were asked to respond to a search engine result page instead of actually doing their own search and finally they had to choose one preferred result and could not click none of the above or two best results. Despite these limitations the respondents did have a high completion rate (84%). Third limitation was the focus on just the first six results of the search engine result page. This is the average above the fold total of results generated, but a real search engine result page would generate more results that can be scrolled and generate extra results in a column on the right and extra snippet results like pictures and places. Further research could look into the effect of those extra results and how these effect the clicking behavior.
Overall this study couldn’t find a direct link between a user’s knowledge of ranking mechanisms and their clicking behavior, but did show that it influences the attitude towards paid results. At the same time this study did find a mediating role of recognizing ads on the relation between labelling ads and clicking them. To develop understanding of how
knowledge of ranking mechanisms drives consumer-clicking behavior further research must focus on the measurement of the knowledge before this can be linked to clicking behavior.
6 Conclusion
In the last two decades internet has been adapted as part of consumers everyday lives. Within this context search engine marketing (SEM) has grown to be a forceful marketing tool. Search engine advertising, as a part of SEM, is the number one source of income for search engines and receives the majority of online advertising spends. Still there is little known of the factors that influences paid results usages by consumers.
This paper makes three main contributions. Firstly a framework is constructed to create a conceptual view of the mechanisms at play in a consumers paid versus organic clicking behavior. Secondly a measurement scale for ability to recognize an ad on a search engine result page is developed and tested on a substantial sample and the relationship between labeling an ad, recognizing an ad and clicking an ad is validated. Finally the empirical results suggest that user attitudes versus paid results are neutral, while their ranking is still perceived as less dependent on relevance than organic results. These findings strengthens the idea that businesses need to be aware of the relevance of their ad placements. From an academic point of view this research topic is still in an early stage leaving lots of opportunities for further research.
References
Albert, R., Jeong, H., & Barabási, A. (1999). Internet: Diameter of the world-wide web. Nature, 401(6749), 130-131.
Bagherjeiran, A., Hatch, A. O., & Ratnaparkhi, A. (2010). Ranking for the conversion funnel. Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 146-153.
Bates, M. J. (1979). Information search tactics. Journal of the American Society for Information Science, 30(4), 205-214.
Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407-424.
Brin, S., & Page, L. (2012). Reprint of: The anatomy of a large-scale hypertextual web search engine. Computer Networks, 56(18), 3825-3833.
Broder, A. (2002). A taxonomy of web search. ACM Sigir Forum, , 36(2) 3-10.
Brooks, N. (2004). The atlas rank report: How search engine rank impacts traffic. Insights, Atlas Institute Digital Marketing,
Dai, H. K., Zhao, L., Nie, Z., Wen, J., Wang, L., & Li, Y. (2006). Detecting online commercial intention (OCI). Proceedings of the 15th International Conference on World Wide Web, 829-837.
Edelman, B., Ostrovsky, M., & Schwarz, M. (2005). Internet advertising and the generalized second price auction: Selling billions of dollars worth of keywords. American Econ. Rev, 97(7), 242-259.
Fallows, D. (2005). Search Engine Users: Internet Searchers are Confident, Satisfied and Trusting–but they are also Unaware and Naïve.Pew Internet & American Life Project,
Gauzente, C. (2009). Information search and paid results—proposition and test of a hierarchy-of-effect model. Electronic Markets, 19(2-3), 163-177.
Graepel, T., Candela, J. Q., Borchert, T., & Herbrich, R. (2010). Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. Proceedings of the 27th International Conference on Machine Learning (ICML-10), 13-20.
Greenspan, R. (2004). Searching for balance. ClickZ (April 30), Http://Www.Clickz.Com/3348071,
Guha, R., McCool, R., & Miller, E. (2003). Semantic search. Proceedings of the 12th International Conference on World Wide Web, 700-709.
Hayes, A. F. (2015). The PROCESS Macro for SPSS and SAS,
Hillard, D., Schroedl, S., Manavoglu, E., Raghavan, H., & Leggetter, C. (2010). Improving ad relevance in sponsored search. Proceedings of the Third ACM International
Conference on Web Search and Data Mining, 361-370.
Höchstötter, N., & Lewandowski, D. (2009). What users see–Structures in search engine results pages. Information Sciences, 179(12), 1796-1812.
Hotchkiss, G. (2004). Inside the mind of the searcher its jungle here. Retrieved from
http://searchenginewatch.com/sew/news/2065095/inside-searchers-mind-its-jungle-here Hotchkiss, G., Garrison, M., & Jensen, S. (2005). Search engine usage in north america.
White Paper, Enquiro, Kelowna, BC, Canada,
Jansen, B. J., Booth, D. L., & Spink, A. (2008). Determining the informational, navigational, and transactional intent of web queries. Information Processing & Management, 44(3), 1251-1266.
Jansen, B. J., Brown, A., & Resnick, M. (2007). Factors relating to the decision to click on a sponsored link. Decision Support Systems, 44(1), 46-59.
Jansen, B. J., & Pooch, U. (2001). A review of web searching studies and a framework for future research. Journal of the American Society for Information Science and
Technology, 52(3), 235-246.
Jansen, B. J., & Resnick, M. (2006). An examination of searcher's perceptions of
nonsponsored and sponsored links during ecommerce web searching. Journal of the American Society for Information Science and Technology, 57(14), 1949-1961.
Jansen, B. J., & Schuster, S. (2011). Bidding on the buying funnel for sponsored search and keyword advertising. Journal of Electronic Commerce Research, 12(1), 1.
Jansen, B. J., & Spink, A. (2006). How are we searching the world wide web? A comparison of nine search engine transaction logs. Information Processing & Management, 42(1), 248-263.
Jansen, B. J., & Spink, A. (2008). Investigating customer click through behaviour with integrated sponsored and nonsponsored results. International Journal of Internet Marketing and Advertising, 5(1-2), 74-94.
Jerath, K., Ma, L., & Park, Y. (2014). Consumer click behavior at a search engine: The role of keyword popularity. Journal of Marketing Research, 51(4), 480-486.
Marable, L. (2003). False oracles: Consumer reaction to learning the truth about how search engines work. Results of an Ethnographic Study,
Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: Bringing order to the web.
Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879-891.
Richardson, M., Dominowska, E., & Ragno, R. (2007). Predicting clicks: Estimating the click-through rate for new ads. Proceedings of the 16th International Conference on World Wide Web, 521-530.
Rose, D. E., & Levinson, D. (2004). Understanding user goals in web search. Proceedings of the 13th International Conference on World Wide Web, 13-19.
Silverstein, C., Marais, H., Henzinger, M., & Moricz, M. (1999). Analysis of a very large web search engine query log. ACm SIGIR Forum, , 33(1) 6-12.
Spink, A., & Jansen, B. J. (2006). Web search: Public searching of the web Springer Science & Business Media.
Xing, B., & Lin, Z. (2006). The impact of search engine optimization on online advertising market. Proceedings of the 8th International Conference on Electronic Commerce: The New E-Commerce: Innovations for Conquering Current Barriers, Obstacles and Limitations to Conducting Successful Business on the Internet, 519-529.
Appendix – Questionnaire
Introductie
Zoekmachines maken een steeds groter deel uit van ons dagelijks leven. Je zoekt op wat voor weer het wordt, hoe die acteur ook alweer heet en waar je dat ene paar schoenen het
goedkoopst kunt krijgen. Allemaal vragen waar een zoekmachine, zoals Google of Yahoo, je doorgaans snel en eenvoudig een antwoord op kan geven. Maar je krijgt nooit één antwoord. Na je zoekopdracht krijg je een zo genaamde resultatenpagina te zien. Een pagina met meerdere zoekresultaten. Aan jou de keuze welk resultaat, cq welk antwoord je kiest. Dit onderzoek richt zich op dat laatste. Welk resultaat kies jij?
Procedure
Straks krijg je drie praktijkvoorbeelden van een vraag waar je mee zou kunnen zitten. Aan jou de keus… welk resultaat kies je op de resultatenpagina van de zoekmachine? Het enige dat je hoeft te doen, is je voorstellen dat jij de zoekopdracht uitvoert. Welk resultaat kies je? Klik dat resultaat aan.
Naast de zoekvoorbeelden vraag ik je een paar demografische gegevens in te vullen. Dit onderzoek duurt circa 5 minuten.
Risico's
De risico's voor deelname aan dit onderzoek zijn minimaal.
Compensatie/ bijdrage
Er is geen (financiële) compensatie beschikbaar voor deelname aan dit onderzoek. Wel lever je een bijdrage aan de ontwikkeling van kennis over het gebruik van online zoekmachines.
Vertrouwelijk
Alle verkregen data van deelnemers wordt vertrouwelijk beheerd. Deze data zal enkel in samengestelde vorm worden gerapporteerd (individuele resultaten worden niet openbaar gemaakt). Enkel de hoofdonderzoeker (W. Schamhart) en zijn begeleider (A. Zerres) hebben toegang tot de originele data. Alle verkregen data wordt opgeslagen in de HIPPA-compliant, Qualtrics-secure database, totdat de hoofdonderzoeker deze verwijdert.
Deelname
Deelname aan dit onderzoek is volledig vrijwillig. Je hebt ten alle tijden het recht om te stoppen of je terug te trekken uit het onderzoek zonder dat dit verder gevolgen heeft. Om het onderzoek af te breken kun je de browser sluiten. Wil je aangeven waarom je bent gestopt? Dat kan via een email aan de hoofdonderzoeker.
Vragen over het onderzoek
Wil je meer weten of heb je vragen over het onderzoek? Neem dan via email contact op met de hoofdonderzoeker Wierd Schamhart (w.schamhart@student.uva.nl)
Dit onderzoek wordt uitgevoerd in het kader van de master Business Administration richting Marketing aan de Faculteit Economie en Bedrijfskunde (UvA).
Ik heb de bovenstaande informatie gelezen en begrepen en verklaar dat ik uit vrije wil aan dit onderzoek deelneem.
□ Ja □ Nee
Email adres: (niet verplicht)
………
Geslacht:
o Man o Vrouw
Hoogst genoten opleiding:
o Lagere school o Middelbare school o MBO o HBO o Universiteit o DoctoraatHoe vaak gebruik je gemiddeld een zoekmachine zoals Google.nl?
o Nooito Een enkele keer per maand o Een keer per week
o Meerdere keren per week o Dagelijks
search engine result page with advertisements labelled
Al jaren draag je Levis 501 broeken. Helaas is je huidige exemplaar aan
vervanging toe. Je wilt online een nieuwe kopen. Welk zoekresultaat kies je?
(klik het resultaat aan dat jij in deze situatie kiest)Levi's® Original 501® - buyonline.com
Adv.www.buyonline.com/501
Bekijk de Nieuwste Collectie op de Officiële Levi's® Site. Koop nu!
Gratis verzending bij 99€ · Gratis Retourneren · Shop de Nieuwe Producten
Levi's 501® - webshop.nl
Adv.www.webshop.nl/501
Check alle Jeans van Levi's. Score, de jeans store voor mannen!
Levi's® Original 501® - outfit.nu
Adv.www.outfit.nu/501
Bekijk de Nieuwste Collectie op de Officiële Levi's® Site. Koop nu!
Gratis verzending bij 99€ · Gratis Retourneren · Shop de Nieuwe Producten
Levi's 501 - Kleding online kopen? | kledingbestellen.nl | Lage prijs
www.kledingbestellen.nl › KledingLevi's 501 Ct Customized Tapered jeans shordich 36/34. LEVI'S Boyfriendjeans 501.
Levi's® Jeans, Jackets & Clothing | Levi's® Modeshop.com
www.modeshop.com/NL/nl_NL/
Een nieuw seizoen. Chino's, corduroy en moleskin. Een nieuw seizoen. Nieuwe materialen.
Shop nu▷ . 501® Original. THE ICONIC STRAIGHT FIT
501® - Levi's | Netherlands
www.jeansgraag.com/NL/nl_NL/collections-home/501
... van de originele jeans. DE ORIGINELE 501® KOPEN ... Geen wonder dat de Levi's
