Budget allocation of digital marketing instruments: performance under bias and inconsistencies Master Thesis

(1)

Budget allocation of digital marketing instruments:

performance under bias and inconsistencies

Master Thesis

W. Verboom

Econometrics, Operations Research and Actuarial Studies

Faculty of Economics and Business

Rijksuniversiteit Groningen

(2)

Master Thesis Econometrics

(3)

Abstract

(4)

1 Introduction

One of the main problems marketing managers face nowadays, is the allocation of their marketing budget. There is much traditional marketing literature that has its focus on the effect of aggregated advertisement spending on sales (e.g., Baghestani 1991; Dekimpe and Hanssens 1995; Nerlove and Arrow 1962). However, with the growth in individual data, a popular method that managers use is attribution for multimedia budget allocation. This method implies that marketing managers use the touch points that an individual customer has had with a certain marketing instrument to explain how much this instrument contributed to the purchase of this consumer. This is a reasonable thought if all touch points the consumer has made are available to the marketing manager. Therefore, there has been done quite some research in attribution methods in order to allocate multimedia budget (e.g., Danaher and Heerde 2018, Li and Kannan 2014).

With an attribution analysis, one uses more data as it considers individual level data. However, there are still some concerns with regard to attribution models, such as the one proposed by Danaher and Heerde (2018), where certain channels get too much or too little credit in the customer journey. One reason why an attribution would not give valid results for a certain firm is that the firm does not have available data about an individual’s impressions of certain media channels. An example of an impression of a media channel is seeing an online display banner on a website, but not clicking on it. Here it could be the case that the online display banner causes the individual to make a purchase, but that this individual searches for the firm’s banner on a search engine. In this case, the search engine would get all the credits for the purchase, instead of this online display banner.

More concerns regarding attribution methods on individual level are mentioned by Kannan, Reinartz, and Verhoef (2016). One very important concern they discuss is cookie deletion. The reason why this is of interest, is because individual data is measured by means of cookie detection. An individual visitor of the website is assigned a certain identification number, which is stored as a cookie. Whenever an individual visits the website, he or she is recognized by its cookie stored identification number. Now, with the upcoming trend in online privacy, internet browsers such as Safari already use automatic cookie blocking. With automatic cookie blocking, one cannot obtain the touch points that an individual had with digital marketing channels. In addition, another con-cern is the use of multiple devices, such as smartphones, tablets, and desktops (Kannan et al., 2016). When an individual uses multiple devices, it is in many cases not possible to determine all distinct touch points with the online marketing channels.

The aforementioned reasons provide uncertainty of the future existence of individual level data. Therefore, it is important to consider methods for budget allocation that are not based on individual level data. A possible method to still find the optimal allocation of marketing budget, is by means of aggregated advertisement effectiveness. When one knows the underlying function of the cost of advertisement, it is possible to find the maximum utility of a marketing manager corresponding to the allocation of budget.

There is quite some literature on the effect of marketing instruments on sales. One of the first methods used to measure the long run effect of advertisement effect is by using an Adstock trans-formation (e.g., Aurier and Broz-Giroux 2013; Broadbent and Fry 1995). This method originally stems from older economics literature (Almon, 1965), and is widely used in marketing literature. The underlying theory of Adstock implies that exposure to advertisement leads to an increase in the level of awareness of a brand, where the awareness decays in time (Joseph, 2006). The advantage of this model is that it is fairly simple to use when finding the effect of advertisement.

(6)

means of a vector autoregression (VAR) model (e.g. Clarke 1976; Dekimpe and Hanssens 1995; Pauwels 2004; Steenkamp, Nijs, Hanssens, and Dekimpe 2005). It could be the case that the marketing manager tracks its advertisement performance or its competitor reactions and then change its marketing strategy correspondingly. One main advantage of using a VAR model is that this accounts for these feedback effects (Horv´ath, Leeflang, Wieringa, and Wittink, 2005).

Finally, another common method to measure the long-term dynamics of advertisement and sales is a varying parameter model (e.g., Ataman, Heerde, and Mela 2010; Rubel, Naik, and Srinivasan 2010; Sriram, Balachander, and Kalwani 2007). Because of varying competition, strategy, and op-portunities, the parameters of advertisement effectiveness are also likely to vary over time. The use of a varying parameter model is therefore also very intuitive.

However, there are also some problems that come into play when the advertisement effect is esti-mated. There are possible biases, inconsistencies and violated assumptions that a marketing manager may face performing such an analysis. Even though there is quite some literature in the performance of the earlier mentioned methods regarding the effectiveness of advertisement on sales, to my knowl-edge there is no literature about the performance of models in terms of fixed budget allocation and how biases or inconsistencies influence them.

Strikingly, the right choice in budget allocation can be one of the main factors in the total profit that a firm faces. In their study Fischer, Albers, Wagner, and Frie (2011) even found a potential increase in profits of more then fifty percent worth of nearly 500 billion Euro, implementing their proposed model for budget allocation at one of the largest firms in the pharmaceutical and chemical business. Therefore, it is highly important to study the flaws of budget allocation in marketing instruments. That brings us to our main question: ”How do potential biases, inconsistencies and wrong assumptions influence fixed budget allocation?”.

In order to answer our main question, several cases are simulated to investigate the effect of common pitfalls on fixed budget allocation. First, the effect size of advertisement is investigated on the performance of budget allocation. Next, omitted variable bias will be studied. Moreover, fixed budget allocation of advertisement spent is investigated under data interval bias, endogeneity, time-varying parameters and violated distributional assumptions. Section 2 describes all the biases, inconsistencies and violated assumptions. Section 3.1 considers a utility function of a marketing manager. Sections 3.2-3.5 explain the models used and proposes methods to obtain the optimal fixed budget allocation. Section 4 describes the field data and the simulated data. Section 5 shows the results of fixed budget allocation under different situations. Section 6 discusses the results. Section 7 concludes the findings.

2 Biases, inconsistencies and violated assumptions

2.1 Effect size

One possible scenario that could lead to poor estimates of the advertisement effectiveness is that the sales caused by advertisement is too small compared to the total number of sales. When there is only a small effect, while the variance in the total sales is very large, it is very hard to distinguish the sales caused by advertisement. The result will then be fairly small and leads to insignificant estimates of the long run advertisement effect on sales.

(7)

differences in the effects can be relatively larger. A practical case where such a bias would play a role is whenever a marketing manager is willing to test a new marketing channel for advertisement. If the marketing manager does not spend enough on this new marketing instrument, the estimate of the advertisement effect could be fairly imprecise or inefficient. This could be due to a reason where the total sales caused by this marketing instrument might be too small compared to the total sales. Eventually, the manager will consider to allocate more or less budget to this instrument than the optimal value. Therefore, it is possible that we must have a certain minimum value for advertisement in order to create an unbiased fixed budget allocation.

In order to test how the effect size influences the fixed budget allocation, artificial marketing channels will be created. An Adstock, VAR, dynamic linear model (DLM), and distributional robust model will then be used to allocate a fixed budget for several effect sizes for these artificial channels. The expectation would be that for a larger effect size, the models perform relatively better in finding the optimal fixed budget allocation.

2.2 Omitted variable bias

Another scenario that could lead to inconsistent estimates, is omitted variables bias. Omitted vari-able bias is a specific type of endogeneity, which comes from an unobserved varivari-able that correlates with the regressors used in the model. As an example, we take an arbitrary model such that the true underlying data generated process is given by:

y = α + βx + γz + u.

Further, suppose that the variable z is not observed and therefore is omitted in our model, we would have the following model:

y = α + βx + ε,

such that ε = γz + u. It follows that our error term contains γz. If one would use Ordinary Least Squares (OLS) to estimate β, the expectation of this estimate would then converge to ˆβ1

p

→ β1+ ργ,

where ρ is the correlation between x and z. However, this phenomenon does not only occur for OLS estimations, but this is very troubling for most models. A solution to this problem would be to use an instrumental variable. However, finding a good instrumental variable is also not always possible. Nowadays, the whole system of sales and advertisement is relatively complex compared to a couple of decades ago. The reason for this is the growth in marketing instruments and strategies. When a marketing manager is willing to investigate the causal effect of advertisement spending, he or she has to account for all possibilities that could affect sales in order to make sure that the effect comes from advertisement. However, in practice this is often not possible because of the lack of data. Nowadays, data of online marketing instruments are usually well stored, which is not always the case for offline marketing instruments such as TV or radio. Now, suppose the marketing manager is not able to account for offline advertisement in its model, even though the offline advertisement is correlated with the spent of online advertisement due to a certain marketing campaign. It then follows that the estimated effect of the online advertisement spent will contain the effect of the offline advertisement. Therefore, we expect the estimates for advertisement effectiveness to be overestimated whenever there are omitted variables. Eventually, the marketing manager will allocate to much of its budget into this marketing channel due to omitted variable bias.

(8)

2.3 Data interval bias

Next, a bias in finding the effect of advertisement is data interval bias. Data interval bias is well discussed in literature (Bass and Leone 1983; Clarke 1976), where the choice in data interval can have impact on the estimated effect of advertisement. The choice of the level of aggregation of the data leads to smaller carry-over effect and a larger immediate effect (Leone, 1995). However, Clarke (1976) found that using different time intervals, resulted in varying results for the estimates of the effect of advertisement on sales and the carry-over effect. Since these estimates can be used to allocate a fixed budget, the allocation will also be influenced by choosing a different data interval.

Nonetheless, the fixed budget allocation is based on the relative advertisement effects that is estimated. When all effects are relatively equally overestimated or underestimated, the weights of the fixed budget allocation are not affected. However, the amount of observations available might be crucial in the performance of finding the optimal fixed budget allocation, when using an estimated advertisement response function. In the ideal case, where the estimates of our advertisement response function are relatively overestimated or underestimated the same, the amount of observations still have a large impact on the performance of the proposed allocation for a certain model. With less observations, the statistical power decreases. Therefore, the efficiency of the estimates for the advertisement response function decreases as well, which might lead to an inefficient proposition for the optimal budget allocation.

In order to distinguish whether the data interval choice has impact on the proposed fixed budget allocation, we will compare the proposed budget allocation using a daily data interval and a weekly data interval using an Adstock, VAR, DLM, and a distributional robust model. The expectations of using weekly data instead of daily is that these models would lose efficiency in their estimates, such that the proposed fixed budget allocation using these models will be less optimal than when we would use a daily data interval.

2.4 Simultaneity

A well discussed inconsistency for the estimates of advertisement effect in literature is endogeneity. The most common type of endogeneity, which is omitted variable bias, is already mentioned above. However, another type of endogeneity that can come across is simultaneity bias. Suppose we have a true underlying data generated process given by the system

yt= β1xt+ γ1zt+ ut,

zt= β2xt+ γ2yt+ vt.

Moreover, let x and v be uncorrelated with u, then it follows that

E (ztut) =

γ2

1 − γ1γ2

E (utut) 6= 0.

We can clearly see that the variable zt is correlated with the error term ut, which causes an

endo-geneity problem. This endoendo-geneity follows in inconsistent estimates of our advertisement response function.

A reason for this could be that there is unobserved demand for a certain product, which also influences the effect of advertisement that causes inconsistent estimates. Another reason for simul-taneity is self reflection of the marketing manager; whenever the marketing manager finds more or fewer sales by either increasing or decreasing the advertisement in a certain marketing channel, the manager will adjust its spent in this marketing channel.

(9)

that also accounts for unobserved effects such as demand shocks. In order to find how simultaneity influences the weights of the proposed budget allocation, a simultaneity parameter is added to the artificial marketing channels of interest, where the spent in advertisement depends on the lagged value of sales. An Adstock, VAR, DLM, and a distributional robust model will then be used for fixed budget allocation for different values of the simultaneity parameter.

2.5 Time varying parameters

Furthermore, time-varying parameters of the effect of advertisement on sales could lead to poor estimates. One reason of time-varying parameters in the effect of advertisement is competition (Gijsenberg and Nijs, 2019). An increase in competition can decrease the effect of advertisement on sales, because the cost of advertisement increases. Since competition is varying over time, the total effect of advertisement on sales could vary over time as well. However, using a model that assumes constant parameters is not able to allocate a fixed budget, such that it allocates a larger part in periods where the effect of advertisement in a certain marketing instrument is larger then other instruments. It follows that constant parameter assumptions result in inefficient fixed budget allocation.

Besides inefficient allocation, the assumption of constant parameters can result in under- or over-estimation of the long run effect of advertisement, which can lead to wrong allocation. If a constant parameter model is only able to capture the higher (lower) parameter estimates in certain periods, the advertisement response function will overestimate (underestimate) the total response of advertisement.

Another reason why constant parameter models could not perform well in estimating the adver-tisement response function is that the underlying varying parameters are captured in the statistical power of the estimates. When the true parameters vary a lot over time and the constant parameter model is able to find good estimates, then these estimates find a large variance such that it loses statistical power.

To investigate the results of not accounting for time varying parameters, an Adstock, VAR model, and a distributional robust model is used to obtain estimates for the optimal fixed budget allocation, where constant parameters are assumed. Furthermore, a dynamic linear model is used to propose an optimal allocation where time varying parameters are assumed.

2.6 Distributional assumptions

Finally, a reason for inconsistency are violated distributional assumptions. Most models that esti-mate advertisement effectiveness make assumptions about the error term, where these models assume the error term to be drawn from an underlying normal population (e.g., Vakratsas, Feinberg, Bass, and Kalyanaram 2004; Bruce 2008; Rubel et al. 2010). However, in practice the error term does not have to be drawn from a normal population.

Whenever there are unobserved demand shocks in the market, a firm can find more extreme values then assumed by a normal distribution. It follows that the estimated effects could be underestimated since these ignore extremes when a model with normal assumptions are used.

(10)

the optimal fixed budget allocation for different sizes of the extreme values added to the simulations.

3 Methodology

In this section the utility of a marketing manager considering the allocation of budget will be discussed. Furthermore, four different models will be explained to estimate the total effect of adspent and to allocate a fixed marketing budget.

3.1 Managers utility

In order to transform the estimates of certain models into a fixed budget allocation, a utility function of a marketing manager is introduced. It is not the goal of this study to propose a utility function of marketing managers in practice, but merely using one that is intuitive as an example to see how various models are able to obtain the maximum of the utility. Let wi,t be the weight of a certain

daily budget Bt allocated to channel i at day t = 1, . . . , T . Now let the utility function of the

manager be given by

uj(w1,t, . . . , wm,t) = τj(w1,t, . . . , wm,t, Bt) − p(w1,t, . . . , wm,t),

such that X

i∈M

wi,t= 1 and 0 ≤ wi,t ≤ 1 for all i ∈ M, (1)

where τj(·) is a function of the obtained sales according to model j. Furthermore the second term of equation (1) is the penalty function that penalizes the utility when the weights are close to zero, such that we have

p(w1,t, . . . , wm,t) = m

X

i=1

νi(ewi,t− e−1),

where νi ∈ R+ is a parameter about personal preference of the marketing manager. The reason

for including this penalty function is that most managers in practice are averse against giving a zero weight to a certain channels. Including this penalty, is also intuitive and we do not argue that this is an important component in the utility function of a marketing manager. However, to make the utility function more realistic and for illustrative purposes, the penalty function adds an extra dimension in finding a combination of weights that maximizes the utility. Throughout this study, the utility function is the key part in finding the optimal allocation using a fixed budget. Furthermore, we assume that the daily budget Bt is given, where the spending strategy of the

marketing manager is optimal when it maximizes its utility for a fixed budget given at time t. The reason for this assumption is that the process of optimal control of a marketing budget, given that the market response parameters are a stochastic process, is very complex and does not add value for the illustrative purposes of bias and inconsistencies on the allocation of a marketing budget. However, to find the optimal control of spent over time, one could use a strategy similar to exercising an American option, where spent is determined according to the expected value of the stochastic process.

3.2 Adstock

(11)

an Adstock model. One of the most popular methods is the Koyck transformation (Boschan and Koyck, 1956). However, this method uses a single carry-over parameter, which is not realistic in digital advertisement. An example is the difference between advertisement on a search engine and a social media platform. When customers are searching for a specific product, then their journey to a purchase is smaller than for a customer that gains awareness about a product through a social media platform. Another popular method to obtain the estimates for an Adstock model is that of Johnston (1984), who proposes to iterate over different values for the carry-over parameter, and use the OLS estimates for the carry-over parameter that minimizes the sum squared residuals. In this study a slight modified version of the method of Johnston (1984) is used. First of all, we will use maximum likelihood (ML) instead of OLS, since this yields similar or even better estimates (Vanhonacker, 1988). Furthermore, we will add multiple carry-over parameters, as different marketing instruments can have different carry-over effects.

Now, let xt,m be the total spent in channel m at time t, then the Adstock of channel m at time t is

given by

ASt,m= xt,m+ λmASt−1,m for t = 2, . . . , T, ∀m ∈ M, (2)

for some 0 ≤ λm< 1. Next, we will model yt, the sales in time t, as a function of Adstock. However,

online advertisement is based on bidding systems where a firm can bid a certain amount to display online advertisements to a consumer. There are only limited consumers to bid on, such that the prices of an online advertisement, and the frequency in which it is shown, increases. Therefore, in some cases the relation between sales and Adstock is not solely linear, for which we need to account for. Moreover, Dinner, Heerde, and Neslin (2013) found in their research evidence of cross-correlation effect between online channels. Therefore, we also need to account for possible cross-correlations. In order to do so, we will add quadratic and interaction effects of the Adstock variables, and model them in a linear relation with sales. The equation of sales is then given by

yt= α + X i∈M βiASt,i+ X j∈M X k∈M βj,kASt,jASt,k+ εt for t = 2, . . . , T, (3)

where M is the set of channels that are used in the model. Now, by using Newton-Raphson iterations, we find the parameters that maximize the log-likelihood function

l α, β, λ, σ2 = log T Y i=t p yt|ASt; α, β, λ, σ2 = T X i=1 log p yt|ASt; α, β, λ, σ2 = −n 2 log 2π − n log σ − 1 2σ2 T X t=1 yt− α − AS>tβ 2 ,

where AStis a vector containing all normal, quadratic, and interaction values of Adstock at time t,

β are the corresponding beta’s and λ a vector of all the carry-over effects from equation (2). What is important about this method is that we make somewhat strong distributional assumptions about the error term being normally distributed, εt∼ N (0, σ2). As stated before, this does not need to be

(12)

Now, let the Hessian of the log-likelihood function be given by

H (ψ, y) = ∂

2_l t(ψ, y)

∂ψ∂ψ0 ,

where ψ = α, β, λ, σ2_{. Then the standard errors are given by}

se( ˆψ) = SqrtDiag− ˆH−1.

After we obtain estimates for the effect of advertisement, we want to find the optimal fixed budget allocation. First of all, we want to find the estimated sales using a fixed budget τadstock_{, which is}

given by τadstock(w1,t, . . . , wm,t, Bt) = X i∈M ˆ βi 1 − λi wi,tBt+ X j∈M X k∈M ˆ βj,k (1 − λj)(1 − λk) wj,twk,tBt2. (4)

Because of the assumption of constant parameters, we see that this method proposes a relative constant allocation of the daily budget. The optimal allocation at time t given the fixed budget Bt

is then obtained by

{w∗

1,t, . . . , w∗m,t}adstock= arg max w1,t,...,wm,t

uadstock(w1,t, . . . , wm,t, Bt), (5)

where u(·) is the utility function given in equation (1).

3.3 Vector autoregression

Another method to find the effect of ad spent is by means of a VAR model. VAR models can be used to capture long run dynamic and cross-correlation effects. In the research of Dekimpe and Hanssens (1995), there are even suggestions to caption persistence effects with the use of VAR models. Now, let the (4M +M (M −1)+2₂ ) × 1 vector Ytcontain the sales and the variables regarding ad spent of all

channels of interest at time t. In order to account for quadratic and interaction effects, these are also incorporated in the vector Yt. Then, the VAR(1) model can be written as

Yt= α + Yt−1Φ + εt, for t = 2, . . . , T, (6)

where α is a constant vector, Φ the coefficient matrix and εt is the error term, such that εt ∼

N (0, Σ). There are various methods to obtain estimates for the above VAR(1) model. Note that we do not add exogenenous variables, since the goal of this study is to observe the performance of the VAR model when unobserved shocks are not accounted for. Furthermore, we do not account for the direct effect in this model. The aim is to find the optimal allocation of a fixed budget obtained from relative effects of advertisement of different channels. Therefore, we make the assumptions that the long-term effect is representative for the total effect of advertisement on sales. One of the most common methods is to write the VAR model as a seemingly unrelated regression model, and then use OLS to obtain the estimates (Davidson and MacKinnon, 2009). Now, let us define

Xt≡1 Yt and Π ≡α Φ .

(13)

the assumptions of multivariate normal errors (Davidson and MacKinnon, 2009). Furthermore, a consistent estimation of the covariance matrix is given by

ˆ Σ ≡ 1

T(Y − X ˆΠ)

>_{(Y − X ˆ}_Π),

which is the ML estimate of the covariance matrix.

The next step in finding the optimal allocation using a fixed budget is finding the cumulative sales caused by a unit shock in a certain channel. The reason not using a shock, based on the Cholesky decomposition, is because the standard deviation in ad spent is not fair to compare when the spent of a certain channel is relatively higher then other channels. Therefore we use

ˆ

Yj₀= e(j+1),

where ej is a zero vector with a one on the (j + 1)th place and j is the position of the corresponding

channel. Now, we iterate the equation

ˆ Yjt= ˆY

j t−1Φ,ˆ

until the difference between the current and lagged cumulative sales is neglectable. Let rj be the

cumulative sales caused by a one unit increase in channel j. It follows that the estimated sales using a fixed budget τVAR, is given by

τVAR(w1,t, . . . , wm,t, Bt) = X i∈M riwi,tBt+ X j∈M X j∈M rj,kwj,twk,tBt2. (7)

Again, we can see that the relative allocation of the daily budget is constant due to the assumption of constant parameters the the VAR model has. The optimal allocation at time t given the fixed budget Btis then obtained by

{w∗_1,t, . . . , w_m,t∗ }VAR= arg max w1,t,...,wm,t

uVAR(w1,t, . . . , wm,t, Bt), (8)

where uVAR_{(·) is the utility function given in equation (1).}

3.4 Varying parameter model

There are a lot of versions of varying parameter models to estimate the effect of advertisement on sales. The most common model in literature is a dynamic linear model (DLM). The advantage of using a DLM is that we do not make the assumption that the complex system of advertisement effectiveness has constant parameters. In order to find the optimal allocation of a fixed budget we consider the DLM proposed by West and Harrison (1999). Now, let our DLM be given by

yt= Ftθt+ vt, vt∼ Nm(0, Vt) , (9)

θt= Gtθt−1+ ωt, ωt∼ Np(0, Wt) , (10)

for t = 1, . . . , T . Here yt are the sales, Ft is a vector containing the intercept and the ad spent

variables of our channels and θt is vector with the corresponding parameters at time t. Moreover,

vtand ωt are the error terms. Further, the initial state is given by θ0∼ N (m0, C0). Equation (9)

(14)

entails the transition equation that describes the state process of our model given our transition matrix Gt.

In order to estimate the parameters of our DLM θt, we will use ML, where we assume that our

initial state and error terms are identically and independently normal distributed. Furthermore, we assume the error terms to be independent for simplicity. Now, let the innovation at time t be given by

t= yt− Ftθt, t = 1, . . . , T.

It follows that the likelihood, LY(Θ), can be written in the following form

− ln LY(Θ) = 1 2 T X t=1 log |Ct(Θ)| + 1 2 T X t=1 t(Θ)0Ct(Θ)−1t(Θ), (11)

where Θ is the set of yet to be determined parameters. Now, we can find the parameters that min-imize the equation (11). Just as with the Adstock model, where Maximum Likelihood Estimation (MLE) is used, the Newton-Raphson algorithm is used to iteratively update initial-stated parame-ters to find the optimal parameparame-ters that maximize the likelihood function. Unfortunately, we can not perform inference by means of a single test to validate the estimates, since the parameters are varying over time. If one would be interested in the significance of certain estimates in time, one could use the estimated variance to calculate a time varying test statistic and perform inference on the estimates of interest. However, since this is not of interest for this study, it will not be included.

Using the estimates of the time varying parameters of the advertisement effectiveness, our aim is to find the optimal fixed budget allocation. First, we want to find the estimated sales using a fixed budget τDLM_{, which is given by}

τDLM(w1,t, . . . , wm,t, Bt) = X i∈M ˆ θiwi,tBt+ X j∈M X k∈M ˆ θj,k,twj,twk,tBt2. (12)

Note that we do not use a discount factor to estimate the long run effect since this is relatively the same for each channel, such that the relative allocation does not change by excluding this term. Since we assume time varying parameters, we see that this method proposes a relative allocation of the daily budget that varies over time. The optimal allocation given the fixed budget Btis then

obtain by

{w1,t∗ , . . . , wm,t∗ }DLM= arg max w1,t,...,wm,t

uDLM(w1,t, . . . , wm,t, Bt), (13)

where u(·) is the utility function given in equation (1).

3.5 Distributional robust model

For the distributional robust model the method proposed by Rubel et al. (2010) is used to find the optimal fixed budget allocation. As mentioned earlier, most models have very strong distributional assumptions, which can lead to inconsistent estimates. The method proposed by Rubel et al. (2010) makes robust estimation of the effect of ad spent against distributional assumptions, which is there-fore interesting to review in term of performance on fixed budget allocation. First, we consider the DLM given by

(15)

where yt denotes the sales, αt the unobserved state variable, and vt the error term at time t.

Moreover, the state vector evolves dynamically over time such that we get

αt+1= φαt+ dt+ ωt,

where φ is the carry-over effect, dtis the drift term, and ωtthe error term at time t. Furthermore,

let the drift term be defined by

dt= X i∈M βixi,t+ X j∈M X k∈M βj,kxj,txk,t,

where xi,t is the ad spent of channel i at time t. The reason of letting the drift term be defined by

the advertisement input is because of the minimax principle this robust method uses. By choosing ˆ

αt, such that we maximize ωt and after that minimize vt, it follows that we do not have to make

any distributional assumptions. Now, let us define parameter θ = (β1, β1, φ, h, q), where h and q are

the variances of vt and ωt respectively. Let the P0 be the initial value of the uncertainty in ˆα, for

which its next iterations are given by

Pt+1= φPt I − Pt/γ + h−1Pt

−1 φ + q,

where γ is a positive constant that embodies the level of conservatism in designing the optimal ˆα. Let the objective function be given by

Sγ(θ) = − 1 2 N X t=1 [ln (det (Ft(θ))) + (yt− ˆαt(θ))0Ft(θ)−1(yt− ˆαt(θ)) ,

where Ft(θ) = Pt(θ) + h. Now, the robust parameter estimates are given by

ˆ

θ = arg max Sγ(θ).

It follows that the robust standard errors are given by the diagonal of the negative inverse of the Hessian of Sγ(θ). Hence, the standard errors are given by

se(ˆθ) = SqrtDiag− ˆG−1, G = ∂ˆ 2Sγ(θ)/∂θ∂θ0|θ= ˆθ.

Now, we have robust estimates without making any distributional assumptions.

To find the estimated sales using the robust model using a fixed budget we use the function τDLM_,

which is given by τrobust(w1,t, . . . , wm,t, Bt) = X i∈M ˆ βi 1 − φwi,tBt+ X j∈M X k∈M ˆ βj,k (1 − φ)2wj,twk,tB 2 t. (14)

The optimal allocation at time t given the fixed budget Btis then obtain by

{w_1,t∗ , . . . , w∗_m,t}robust= arg max w1,t,...,wm,t

urobust(w1,t, . . . , wm,t, Bt), (15)

(16)

4 Data

4.1 Combination of field and artificial data

To investigate how various models perform in finding an optimal fixed budget allocation, the data is partly simulated and combined with actual sales data. The true underlying system of sales and advertisement can be very complex. When solely simulation would be used to obtain advertisement effects, it could be the case that the models used perform much better than in practice, because the white noise in the simulation is not as complex as in practice. There would also be a conflict of interest in the choice of the white noise, since this could influence the estimates of the models such that it yields better outcomes. Since the aim of this study is to find bias and inconsistencies that a marketing manager faces in the real world, this study uses a combination of field and artificial data. The idea behind this combination of data is that if the underlying response function of advertisement is known, and we are able to retrieve unbiased and consistent estimates for this response function, then the marketing manager is also able to retrieve efficient estimates for the response function when his hypothesis about the underlying function is right. For the field data, we use the data from a webshop of a commercial men’s clothing retailer. The data contains daily sales and advertisement spent of various channels from 2016-06-01 until 2019-11-01. This field data is then used as white noise to cover our artificially created sales data.

4.2 Artificial data

For the simulation part of the data, two artificial channels are created. The first channel represents a new search engine where advertisement is offered in the form of higher ranking on a the search engine. Now, the spent in this first channel at time t, x_1,t, is simulated with the following system

x1,t= max{γ(ρ1xsearch,t+ µ1s1,t+ u1,t+ zyfieldt ), 0},

s1,t= 0.5s1,t−1+ v1,t,

u1,t∼ N (2µ1, σ12),

v1,t∼ N (0, 1).

Here γ is the amplify parameter, which will adjust the total spent of x1,t. Furthermore, ρ1 is the

correlation between the spent of our first simulated channel x1,tand the spent in an existing channel

xsearch,t, which is also representing spent in a search engine. The reason for including this term is

because we can investigate how omitted variable bias can affect our estimates of spent effectiveness when not all of the channels are included in the model. Moreover, we have that µ1s1,t is a random

walk, which is instinctive to add since the spent of yesterday is partly correlated with the spent of today. This random walk consists of a AR(1) process that is given by s1,tand a certain value µ1that

determines the size of the random walk. Next, we have parameter z that is a simultaneity parameter where the spent is determined by the size of the actual field sales yfield

t . This is also intuitive since

a marketing manager is more willing to advertise when his or her performance is well. Lastly, we have an error term that is normally distributed with mean 2µ1 and variance σ21.

(17)

system

x2,t= max{γ(ρ2xsocial,t+ µ2s2,t+ u2,t+ zytfield), 0},

s2,t= 0.8s2,t−1+ v2,t,

u2,t∼ N (2µ2, σ22),

v2,t∼ N (0, 1).

Again, γ is the amplify parameter, which will adjust the total spent of x2,t, and z is the simultaneity

parameter. Furthermore, ρ2 is the correlation between spent in x2,t and the spent in an existing

channel xsocial,t, which is spent in a social media platform. Also, we have a random walk part µ2s2,t,

to simulate the spending in x2,t. However, the AR(1) part s2,t is more correlated with its lagged

part than it is for the first channel. This is because banner based advertisement channels are more common to be used campaign based such that the spent of today is more correlated to the spent of yesterday than it is with the first channel.

Now that the spent of advertisement of two different channels are simulated, we want simulate the caused sales. In order to do so, an advertisement response function is proposed. Again, it is not the goal of this study to argue the validity of the advertisement response function, but solely to determine how one is able to find the optimal fixed budget allocation when the underlying advertisement response function is known. Let the advertisement response function given by

y_t∗=

∞

X

i=0

κ1,t(λi1η1x1,t−i+ λ2i1η1,sqx21,t−i) + κ2,t(λi2η2x2,t−i+ λ2i2η2,sqx22,t−i) + λ i 1λ i 2η1,2x1,t−ix1,t−i, (16) where λi

j denotes the time-decay effect such that 0 < λj < 1, ηj is the effect of spent, ηj,sq is the

total effect of squared spent for channel j = 1, 2. Moreover, η1,2 is the interaction effect of spent

in botch channels. Further, we have that κ1,t and κ2,t are time varying parameters that represents

the market conditions that are, for instance, influenced by competition. As a follow up, we simulate these market conditions as random walks given by the following system

κi,t= 1 + ai

¯

κi,t− E[¯κi,t]

pVar(¯κi,t)

,

¯

κi,t= 0.99¯κi,t−1+ εt+ 0.8εt−1,

εt∼ N (0, 1),

for i = 1, 2, where a ∈ R+ influences the amount of variation over time. For simplicity, we assume

that the interaction effects are not influenced by the market conditions, and that the two time vary-ing parameters for market conditions are independent.

Furthermore, we would like to simulate extra noise from an other distribution than the normal case. Since extreme events could give inconsistent estimates for the advertisement response function, where the normal assumption holds, it is interesting to examine how much these extreme events influence the proposed fixed budget allocation. In order to simulate these extreme events, a hundred arbi-trary days are chosen for which extra sales are simulated that are drawn from a Poisson distribution. Now, let the simulated sales due to extreme events be given by dt∼ P(λdist) for t ∈ D and dt= 0

(18)

If we combine our simulated sales y_t∗, the field sales yfield

t , and the extreme event shocks dt, we find

that the total simulated sales at time t is given by yt= ytfield+ y∗t + dt. Therefore, yt is the total

simulated sales at time t, which is generated by the field data white noise, artificial created sales, and extreme shocks.

Now that the underlying system of our artificial channels and sales is known, we will need the corresponding parameters to simulate the artificial channels. To compare bias and inconsistencies, a standard case of parameters is selected. This will then be the standard case of parameter selection, which is given in table 1. Again, the choice of the parameters are purely intuitive with the illustrative purpose of investigating bias and inconsistencies. To investigate the biases, inconsistencies and violated assumptions argued in section 2, where we will adjust some parameters and compare them to the standard case.

Table 1: Standard case parameter selection

channel 1 channel 2 effects other

parameters value parameters value parameters value parameters value

µ1 40 µ2 30 η1 0.05 a1 0.1 σ1 10 σ2 20 η2 0.03 a2 0.125 γ 1 γ 1 η1,sq 0 λ1 0.7 ρ1 0 ρ2 0 η2,sq −0.00001 λ2 0.9 z 0.1 z 0.1 η1,2 0.00005 λdist 0 v1 1 v2 1

5 Results

First, the utilities of the standard parameter selection case are calculated. The goal of the marketing manager is to maximize its utility over the whole period of interest. Therefore, we will compare the outcomes of the models by the sum of the utility given the allocation proposed by the model. Furthermore, the current allocation is the utility that was obtained using the simulated channels. Now, we denote the fixed daily budget as the sum of the spent of all channels such that Bt =

x1,t+ x2,t. This is not due to a restriction of the budget of the marketing manager, but merely

to simulate the fixed budget that is chosen by the marketing manager upfront. The goal of the marketing manager is then to allocate this fixed budget better than the current allocation given by x1,t and x2,t. In this section, we will calculate the optimal allocation that was possible given

the same fixed daily budget and use the insights of the proposed methods in section 3, to find the allocations using these methods.

(19)

Table 2: Standard Case Utility utility Current Allocation 47782.2815 (90.9%) Optimal Allocation 52565.4475 (100%) Adstock Allocation 51598.9620 (98.1%) VAR Allocation 18656.8550 (35.4%) DLM Allocation 48875.2914 (92.9%) Robust allocation 44980.7373 (85.6%)

5.1 Effect size

Now, we are interested in how the total effect size influences the fixed budget allocation. In order to test how the effect size influences the fixed budget allocation, we will compare the standard case of parameter selection to the cases where we change the amplify parameter to γ = 0.2, γ = 0.5, γ = 1 and γ = 3. Then, we will look at how the change in the amplify parameter, which scales the total advertisement spent, influences the fixed budget allocation. Since the amplify parameter is only a relative parameter to observe, we will also look at the ratio between the simulated caused sales and the sales from the field data to see how much the total effect size is. For the standard case where γ = 1, the ratio of the caused and field sales is 0.37. Furthermore, we find for the cases γ = 0.2, γ = 0.5 and γ = 3 a ratio of 0.07, 0.18 and 0.59 respectively. In table 3 we find the total obtained utility of the marketing manager corresponding to method of allocation. We can see the the Adstock method does a relative good job on finding the optimal allocation. There is only a drop in the performance at γ = 0.5, but as the amplify parameter gets larger the performance is getting better again. When we look at the VAR model we can see that its performance is overall very poor relative to the other methods. Interestingly, the performance of the DLM and the Robust model get notably better the larger the amplify parameter gets. This means that these models are most sensitive to the size of the total effect of the spent.

Table 3: Utility for different values of γ

γ = 0.2 γ = 0.5 γ = 1 γ = 3 Current Allocation 6130.7631 (82.4%) 21085.1878 (91.3%) 47782.2815 (90.9%) 177265.6027 (79.7%) Optimal Allocation 7438.2765 (100%) 23072.9401 (100%) 52565.4475 (100%) 221997.6269 (100%) Adstock Allocation 7120.7478 (95.7%) 15562.2513 (67.4%) 51598.9620 (98.2%) 221879.4207 (99.9%) VAR Allocation 3133.2085 (42.1%) 14972.4739 (64.8%) 18656.8550 (35.5%) -177251.0599 (-79.8%) DLM Allocation 5872.1597 (78.9%) 18733.1183 (81.2%) 48875.2914 (93.0%) 208918.0702 (94.1%) Robust allocation 3733.4964 (50.1%) 15395.0192 (66.7%) 44980.7373 (85.6%) 221293.8213 (99.7%)

5.2 Omitted variable bias

Now, we will start with the standard case of the parameter choice again. Since we do not know the true underlying data generated process from the field data used, it is not possible to show the omitted variable bias in our field data. To make the effect of omitted variable bias on fixed budget allocation visible, we will use a specific kind of omitted variable bias. The correlation parameters ρ1and ρ2will be adjusted but the variable to which these parameters correspond are not accounted

(20)

are overvalued due to omitted variable bias. Since our proposed methods will overestimate one or multiple channel(s), this could adjust the proposed fixed budget allocation. We will calculate the obtained utility using our proposed methods using the standard case, where ρ1= ρ2= 0, a another

case where ρ1 = 0.5 and ρ2 = 0, then a case where ρ1 = 0 and ρ2 = 0.5 and finally a case where

ρ1 = ρ2 = 0.5. In table 4 we can see the utility of the manager when our proposed methods are

used. We can immediately see that, again, the VAR model performs relatively poor, where in some cases the utility of the manager even becomes negative. Furthermore, we can see that, especially in the case where ρ1= 0 and ρ2= 0.5, all our models perform poorly with respect to the standard

case, where there is no correlation with unobserved channels.

Table 4: Utility for different values of ρ1 and ρ2

ρ1= ρ2= 0 ρ1= 0.5 & ρ2= 0 ρ1= 0 & ρ2= 0.5 ρ1= ρ2= 0.5 Current Allocation 47782.2815 (90.9%) 165693.9112 (77.0%) 51532.0794 (37.4%) 261219.7907 (76.6%) Optimal Allocation 52565.4475 (100%) 215245.9152 (100%) 137931.5340 (100%) 341194.3688 (100%) Adstock Allocation 51598.9620 (98.2%) 213853.6457 (99.4%) 80520.2358 (58.4%) 336497.5576 (98.6%) VAR Allocation 18656.8550 (35.5%) 99655.547 (46.3%) -94314.9100 (-68.4%) -432156.3766 (-126.7%) DLM Allocation 48875.2914 (93.0%) 195615.5151 (90.8 %) 36346.1831 (26.4%) 251337.3147 (73.7%) Robust allocation 44980.7373 (85.6%) 213638.6606 (99.3%) 63820.0913 (46.3%) 336073.3377 (98.5%)

5.3 Data interval Bias

An important choice that a marketing manager has to make regarding advertisement effectiveness is the choice of the data interval. It could be that daily interval is not available and the manager is obliged to use weekly data. However, the use of weekly data results in data interval bias, where effects are larger but the carry-over effect is smaller. Furthermore, the choice of weekly data versus daily has seven times less observations than daily data, which may cause wrong estimates for the advertisement effect. To determine whether the choice of the data interval can impact the allocation of a fixed budget, we will use the standard case where a daily data interval is used compared to the case where we use a weekly data interval.

(21)

Table 5: From daily to weekly data Daily Weekly Current Allocation 47782.2815 (90.9%) 47782.2815 (90.9%) Optimal Allocation 52565.4475 (100%) 52565.4475 (100%) Adstock Allocation 51598.9620 (98.2%) 30825.3904 (58.6%) VAR Allocation 18656.8549 (35.5%) 18656.7889 (35.5%) DLM Allocation 48875.2914 (93.0%) 22338.7471 (42.5%) Robust allocation 44980.7373 (85.6%) 28534.1230 (54.3%)

5.4 Simultaneity

In finding advertisement effectiveness, a marketing manager is very likely to find endogeneity in his or her model. Endogeneity can come in many different forms. Self reflection is, however, a common type of endogeneity when a marketing manager is finding the optimal fixed budget allocation. This type of endogeneity comes in the form of simultaneity of sales and advertisement spent. The idea behind the simultaneity is that the marketing manager self-reflects to know when the sales are higher, and feels more comfortable to increase its advertisement spending in times when the sales are high. In order to examine how the size of this simultaneity influences the fixed budget allocation, the simultaneity parameter z will be adjusted to compare the proposed methods to the standard case. In table 6, one can find the utilities of the proposed methods that are used to maximize the utility for different values of z. Again, the VAR model does not perform well relative to the other proposed methods. Furthermore, we can see that the Adstock and the DLM model perform less when the simultaneity increases from z = 0 to z = 0.1. However, when z = 1, we see that both models perform best in terms of maximizing the utility. Interestingly, for using the Robust method, we see a monotone increasing utility when z increases.

Table 6: Utility for different values of z

z = 0 z = 0.1 z = 0.5 z = 1 Current Allocation 40578.1559 (89.7%) 47782.2815 (90.9%) 80898.2657 (93.4%) 128758.8593 (81.9%) Optimal Allocation 45232.3815 (100%) 52565.4475 (100%) 86612.9573 (100%) 138024.5332 (100%) Adstock Allocation 44907.7508 (99.2%) 51598.9620 (98.2%) 85379.5888 (98.6%) 488834.4819 (99.5%) VAR Allocation 19234.9562 (42.5%) 18656.8550 (35.5%) 3712.5815 (4.3%) -45888.0218 (-33.3%) DLM Allocation 42753.7021 (94.5%) 48875.2914 (93.0%) 78322.4724 (90.4%) 132889.9870 (96.3%) Robust allocation 37658.7252 (83.2%) 44980.7373 (85.6%) 77443.7008 (89.4%) 134519.4587 (97.4%)

5.5 Time Varying parameters

Since there is competition, demand, and other market conditions influencing the effect of adver-tisement, the estimates proposed by models that do not account for this could be relatively poor to models that do not make the assumption of constant parameters. By changing a1 and a2, the

(22)

when the parameter vary more over time. Overall, the Adstock method performs relatively well to the other methods. Strikingly, the DLM method, which does not make the assumption of constant parameters, also performs less when the parameters vary more over time. The Robust allocation has mixed performance, where in the case of a1= 0.1 and a2= 0.5, the Robust allocation increases

in terms of performance relative to the standard case.

Table 7: Utility for different values of a1 and a2

a1= 0.1 & a2= 0.125 a1= 0.4 & a2= 0.125 a1= 0.1 & a2= 0.5 a1= 0.4 & a2= 0.5

Current Allocation 47782.2815 (90.9%) 47765.2445 (90.2%) 48150.7776 (90.4%) 48133.7406 (90.0%) Optimal Allocation 52565.4475 (100%) 52946.5178 (100%) 53255.2174 (100%) 53502.1476 (100%) Adstock Allocation 51598.9620 (98.2%) 51346.9597 (97.0%) 51725.6529 (97.1%) 51528.1052 (96.3%) VAR Allocation 18656.8550 (35.5%) 18656.8311 (35.2%) 18440.1609 (34.6%) 18440.1362 (34.5%) DLM Allocation 48875.2914 (93.0%) 45387.5294 (85.7 %) 48338.3463 (90.7%) 44500.3186 (83.2%) Robust allocation 44980.7373 (85.6%) 43320.1486 (81.8%) 45981.1118 (86.3%) 44619.6411 (83.4%)

5.6 Distributional assumptions

Since most methods that make estimations about the advertisement response function make a normal assumption about the error term, we want to investigate how extreme effects, which are not normal, influence the fixed budget allocation of our proposed models. In order to determine how the addition of extreme affect the allocation, we will use different values of λdistand compare them to the standard

case where λdist= 0. In table 8 we can see the utilities for each proposed method for the different

values of λdist. We can see that, except for the VAR model which does not perform well, the

performance of our models drop majorly when these extreme values are added to our simulated sales. The Robust model is the only model that does not make normal assumptions, however, we can see that it performs relatively poor to the Adstock and the DLM model, which do make normal assumptions.

Table 8: Utility for different values of λdist

λdist= 0 λdist= 100 λdist= 200

Current Allocation 47782.2815 (90.9%) 47782.2815 (90.9%) 47782.2815 (90.9%) Optimal Allocation 52565.4475 (100%) 52565.4475 (100%) 52565.4475 (100%) Adstock Allocation 51598.9620 (98.2%) 50773.5306 (96.6%) 45778.3134 (87.1%) VAR Allocation 18656.8550 (35.5%) 18656.8550 (35.5%) 18656.8550 (35.5%) DLM Allocation 48875.2914 (93.0%) 39129.7162 (74.4 %) 33472.3089 (63.7 %) Robust allocation 44980.7373 (85.6%) 35261.8632 (67.1%) 28866.7698 (54.9%)

6 Discussion

6.1 VAR performance

(23)

and insignificant. The allocation proposed by the VAR model allocates the complete budget to x2,t

for all t = 1, . . . , T . This is due to the fact that the VAR model estimates this as the best possible allocation. Therefore, the VAR model decides to choose to allocate the budget solely to one channel. Furthermore, the VAR model finds a negative interaction effect, such that it penalizes the use of both channels. We also find positive quadratic terms for the effects of ad spent in our simulated channels, where allocating the complete fixed budget is not punished when there is too much spent in a certain channel. Since we do not find significant values for our estimates of the effects of advertisement spent on sales, the model does not have enough statistical power to be confident about the proposed fixed budget allocation of the VAR model.

Besides the statistical significance of the VAR model, stationarity is also an important component in the inference of a VAR model. A reason for the poor estimates could be a wrong assumption of stationarity, although this might, in a first glance, not necessarily the case. In order to check the stationarity, we can look at the Eigenvalues of our estimated coefficient matrix ˆΦ. Again, if we look at the standard case, we have that Eigenvalues are given by r1 = 0.7284720, r2 = 0.6765656,

r3= 0.4377979, r4= 0.4156849, r5= 0.2477638 and r6= 0.2075031. Since all the Eigenvalues lie in

the unit circle, it follows that the roots of the characteristic equation are all out of the unit circle, such that the stationarity assumption holds.

Now, for the other cases, the same happens as in the standard case where the VAR model is unable to make good estimates, where the weights of the fixed budget allocation rest on a single channel. It follows that in all cases the utility is relatively poor compared to the other methods, and is therefore far from the optimal allocation. Since the VAR model performs the same in all cases, where all the weights are given to one channel, it is not interesting to further discuss the VAR model in the rest of this section. Therefore, in the discussion of the results from the simulated cases, the VAR model will not be addressed anymore.

6.2 Effect size

Now, we will discuss the results of using different values of γ in our simulations. We expected that the performance of finding the optimal fixed budget allocation, using the proposed methods, would increase when the effect size would increase. For the DLM and the robust methods we see that this is indeed the case, where we only observe a monotone increasing performance for larger sizes of values for γ (see table 3). Furthermore, we can see that the robust allocation is more affected by the total effect size, since the performance increases for larger choices of γ. When a marketing manager would have to make the choice between the models, the total effect size would then play a certain role. For the small values of γ, the DLM method outperforms the robust method.

(24)

have that the performance of the Adstock model in the case where γ = 0.2 is better than when γ = 0.5. We can therefore not conclude that the performance of the Adstock model is relatively good for small effect sizes, but merely that the importance of our linear effects are more important, as the performance of our Adstock model is better for the case where γ = 0.2 than the case where γ = 0.5. If we look at the case where γ = 1 we start to find a positive estimate for the interaction effect of the channels, such that we see a monotonic increase in the performance for the Adstock method when we increase the effect size. If quadratic and interaction effects play an important role in the optimal allocation, the effect size is also very important in finding the fixed budget allocation.

6.3 Omitted variable bias

For the different simulations in the cases where ρ1and ρ2are adjusted, we see mixed results. Except

for the case where ρ1= 0 and ρ2= 0.5, the performance is overall not bad compared to the standard

case. One of the reasons that in the case when ρ1= 0 and ρ2= 0.5 the performance is much worse

than for all other cases, is that the unobserved advertisement spent of the social network channel is responsible for the overestimation of channel 2. The true data generating process has a negative squared term, such that we do not want to many weight on this channel. This also explains that the case where ρ1 = 0.5 and ρ2 = 0 is not as much affected, since more weight in channel 1 is not

punished by a negative quadratic term by the true data generating process. Strikingly, in the case where ρ1= ρ2= 0.5, the performance of the models are relatively better than the case where ρ1= 0

and ρ2 = 0.5. Since the effect of the first channel is also overestimated, we see improving results.

Moreover, what is interesting is that, overall, the DLM model performs less relative to the other models. The importance of noticing omitted variable bias therefore grows with the use of a DLM model to allocate a fixed budget.

6.4 Data interval bias

The choice of the data interval can have a large impact on the proposed fixed budget allocation. If we look at table 5, we can see that, leaving VAR out of the discussion, all of our models perform less when we use a weekly data interval instead of a daily data interval. The reason of this performance is due to the drop in observations, such that our models lose statistical power. Because of the drop in observations, our proposed methods are unable to make right estimates and the proposed allocation almost halves the utility relatively to the daily data interval. It could be that using a weekly interval data captures the long run effects better than the daily interval data, but in the specific setting of this simulation study, we have that the use of weekly data is not recommended. Especially in the DLM model we found a large drop in using weekly data compared to the other methods. It is therefore recommended to the marketing manager to carefully make a trade-off between data interval and performance when finding the optimal fixed budget allocation.

6.5 Simultaneity

(25)

larger values of z. We could therefore argue that the robust model is also robust to simultaneity bias. Now, if we look at the fluctuations in the performance of the models under various biases, the simultaneity bias is not the largest problem that a marketing manager faces. Especially when spent is large enough, all the proposed methods perform relative well under simultaneity bias.

6.6 Time varying parameters

For the cases where we adjust the market conditions, we have some interesting results. On forehand, the expectations were that the DLM model would perform better, relatively to the other methods, since it is not restricted to constant parameters. However, if we look at table 7, we can clearly see that the overall performance, in the cases where there are more time varying market conditions, is less for the DLM compared to the Adstock model. However, the performance of the DLM model is overall better than the robust model, which is also not what was expected. The Adstock model also performs less in the cases where there are more time varying market conditions, however the loss in the utility is fairly small. A reason for this could be that the Adstock model finds the best fit regarding the time varying parameters, such that it is still able to find the underlying advertisement response function. Even under very volatile market conditions, it performs best relative to the other methods in finding the optimal fixed budget allocation.

For the robust method we find some mixed results. In the case where a1= 0.1 and a2= 0.5, we

have that the performance of the robust method has a better performance than our standard case. One explanation is that in the case where the second channel is more volatile in its effectiveness, the average of the response function yields a better predictor than in the case where the first channel is more volatile. The reason for this assumption is that the other methods also find a higher utility for the cases where only the second channel finds more volatility in its effectiveness. This means that we can not argue about the performance regarding certain channels, but merely look at the overall performance when there are more time varying market conditions.

6.7 Distributional assumptions

Since most of our distributions are based on normal assumptions, having extremes in our data could bias the proposed fixed budget allocation. If we look at table 8, we can see that not only the methods that make normal assumptions, but also the robust method performs relatively poor when extremes are added to our data. Interestingly, the Adstock model even performs best when we compare our methods for larger values of λdist. However, we still see a monotone decrease in the utility for

the proposed Adstock method to allocate the fixed budget when we increase λdist. The normal

(26)

7 Conclusion

Throughout this study, we aimed at finding the optimal fixed budget allocation under bias, inconsis-tencies, and wrong distributional assumptions. The allocation of a fixed budget is an important part of the long-term performance of a firm. Therefore, it is highly useful to know which flaws there are in finding the advertisement response function, and what impact it may have on finding the optimal fixed budget allocation. The factors that have been discussed are the effect size, omitted variable bias, data interval bias, simultaneity, time varying parameters, and distributional assumptions. By combining simulation and field data, all these factors are analysed as realistic as possible, such that the marketing managers knows the most important factors that influences the findings of the optimal fixed budget allocation.

After concluding that the VAR model performed very poor in all of our simulated cases, we left out this model in our discussion as we are not able to distinguish the difference between different cases. Furthermore, we used an Adstock model, DLM, and a distributional robust model, which all showed mixed performance in finding the optimal fixed budget allocation. Overall, we found that all of our factors had an impact on the allocation. However, the factors that had the largest impact on the allocation of the fixed budget were effect size, omitted variable bias, and data interval. The effect size has impact on the statistical power of the estimates, such that the larger the effect size, the more reliable the estimates are. It follows that a marketing manager will need to have a large enough effect size of advertisement on sales in order to capture the underlying function. Furthermore, in some of the cases where we added omitted variable bias in our simulation, we found relative poor performance where. We found that, for instance, the Adstock model dropped in its performance from finding an allocation of 98.2 percent from the optimal in the cases where no omitted variable occurred, to an allocation of 58.4 percent from the optimal allocation in the cases where there was omitted variable bias. It could therefore happen that a marketing manager can increase its market-ing performance with more than 30 percent by includmarket-ing all the important variables in its model. Moreover, we found that marketing managers have to be careful in the data interval choice to find the advertisement response function, and thereby finding the optimal fixed budget allocation. By making use of a weekly data interval instead of a daily data interval, we found for the DLM model a drop of more than 50 percent from the optimal allocation. Therefore, it is important for the man-ager to know the impact of choosing a different data interval. Another interesting finding was that the DLM allocation, which accounts for time varying parameters, did not out perform the Adstock model that assumed constant parameters. Therefore, it is not always advised to use a DLM model over a more simpler Adstock model. Finally, we found that extreme values in our data impacted all of our models in terms of finding the optimal fixed budget allocation. The distributional robust model even performed worst relative to the other methods that we discussed.

(27)

References

Almon, Shirley (1965). The distributed lag between capital appropriations and expenditures. Econo-metrica 33 (1), 178.

Ataman, M. Berk, Harald J. Van Heerde, and Carl F. Mela (2010). The long-term effect of marketing strategy on brand sales. Journal of Marketing Research 47 (5), 866–882.

Aurier, Philippe and Anne Broz-Giroux (2013). Modeling advertising impact at campaign level: Empirical generalizations relative to long-term advertising profit contribution and its antecedents. Marketing Letters 25 (2), 193–206.

Baghestani, Hamid (1991). Cointegration analysis of the advertising-sales relationship. The Journal of Industrial Economics 39 (6), 671.

Bass, Frank M. and Robert P. Leone (1983). Temporal aggregation, the data interval bias, and empirical estimation of bimonthly relations from annual data. Management Science 29 (1), 1–11.

Boschan, Paul and L. M. Koyck (1956). Distributed lags and investment analysis. Economet-rica 24 (1), 95.

Broadbent, Simon and Tim Fry (1995). Adstock modelling for the long term. Market Research Society. Journal. 37 (4), 1–18.

Bruce, Norris I. (2008). Pooling and dynamic forgetting effects in multitheme advertising: Tracking the advertising sales relationship with particle filters. Marketing Science 27 (4), 659–673.

Clarke, Darral G. (1976). Econometric measurement of the duration of advertising effect on sales. Journal of Marketing Research 13 (4), 345.

Cohen, Jacob (2009). Statistical power analysis for the behavioral sciences: Jacob Cohen. Psychology Press.

Danaher, Peter J. and Harald J. Van Heerde (2018). Delusion in attribution: Caveats in using attribution for multimedia budget allocation. Journal of Marketing Research 55 (5), 667–685.

Davidson, Russell and James G. MacKinnon (2009). Econometric theory and methods. Oxford University Press.

Dekimpe, Marnik G. and Dominique M. Hanssens (1995). The persistence of marketing effects on sales. Marketing Science 14 (1), 1–21.

Dinner, Isaac M., Harald J. Van Heerde, and Scott A. Neslin (2013). Driving online and offline sales: The cross-channel effects of traditional, online display, and paid search advertising. Journal of Marketing Research 50 (5), 527–545.

Ebbes, Peter, Dominik Papies, and Harald J. Van Heerde (2011). The sense and non-sense of holdout sample validation in the presence of endogeneity. Marketing Science 30 (6), 1115–1122.

(28)

Gijsenberg, Maarten J. and Vincent R. Nijs (2019). Advertising spending patterns and competitor impact. International Journal of Research in Marketing 36 (2), 232–250.

Horv´ath, Csilla, Peter S.h. Leeflang, Jaap E. Wieringa, and Dick R. Wittink (2005). Competitive reaction- and feedback effects based on varx models of pooled store data. International Journal of Research in Marketing 22 (4), 415–426.

Johnston, J. (1984). Econometric methods, 3rd edition. New York McGraw-Hill.

Joseph, Joy V. (2006). Understanding advertising adstock transformations. SSRN Electronic Jour-nal .

Kannan, P.K., Werner Reinartz, and Peter C. Verhoef (2016). The path to purchase and attribution modeling: Introduction to special section. International Journal of Research in Marketing 33 (3), 449–456.

Leone, Robert P. (1995). Generalizing what is known about temporal aggregation and advertising carryover. Marketing Science 14 (3).

Li, Hongshuang (Alice) and P.K. Kannan (2014). Attributing conversions in a multichannel on-line marketing environment: An empirical model and a field experiment. Journal of Marketing Research 51 (1), 40–56.

Nerlove, Marc and Kenneth J. Arrow (1962). Optimal advertising policy under dynamic conditions. Economica 29 (114), 129.

Pauwels, Koen (2004). How dynamic consumer response, competitor response, company support, and company inertia shape long-term marketing effectiveness. Marketing Science 23 (4), 596–610.

Rubel, Olivier, Prasad A. Naik, and Shuba Srinivasan (2010). Optimal advertising when envisioning a product-harm crisis. SSRN Electronic Journal .

Sriram, S, Subramanian Balachander, and Manohar U Kalwani (2007). Monitoring the dynamics of brand equity using store-level data. Journal of Marketing 71 (2), 61–78.

Steenkamp, Jan-Benedict E. M., Vincent R. Nijs, Dominique M. Hanssens, and Marnik G. Dekimpe (2005). Competitive reactions to advertising and promotion attacks. Marketing Science 24 (1), 35–54.

Vakratsas, Demetrios, Fred M. Feinberg, Frank M. Bass, and Gurumurthy Kalyanaram (2004). The shape of advertising response functions revisited: A model of dynamic probabilistic thresholds. Marketing Science 23 (1), 109–119.

Vanhonacker, Wilfried R. (1988). Estimating an autoregressive current effects model of sales response when observations are aggregated over time: Least squares versus maximum likelihood. Journal of Marketing Research 25 (3), 301.

(29)

Appendices

A. Standard Case Model Inference

Table 9: Standard Case Adstock Model

estimate se tvalue pvalue

α 78.0507 0.9413 82.9215 0.0000 β1 0.3060 0.2394 1.2783 0.2331 β1,1 0.0000 0.8080 0.0000 1.0000 β2 0.5599 0.5136 1.0901 0.3040 β2,2 -0.0015 0.0024 -0.6385 0.5390 β1,2 0.0004 0.0010 0.4219 0.6830 λ1 0.2151 0.7290 0.2951 0.7746 λ2 0.0463 0.6771 0.0684 0.9469 σ 47.2477 0.8080 58.4734 0.0000

Table 10: Standard Case Var Model

Dependent variables: yt x1,t x21,t x2,t x22,t x1,tx2,t yt−1 0.732∗∗∗ 0.013 2.408 0.006 0.964 0.031 (0.022) (0.024) (4.897) (0.022) (3.918) (2.935) x1,t−1 −0.061 0.425∗∗∗ 42.007∗∗ 0.007 6.124 14.144 (0.085) (0.091) (18.448) (0.082) (14.759) (11.055) x2 1,t−1 0.0001 0.00002 0.209∗∗ 0.00002 −0.009 0.010 (0.0004) (0.0004) (0.086) (0.0004) (0.068) (0.051) x2,t−1 −0.006 0.031 5.712 0.680∗∗∗ 45.242∗∗∗ 42.310∗∗∗ (0.077) (0.083) (16.871) (0.075) (13.497) (10.110) x2 2,t−1 0.0001 −0.0002 −0.033 0.0001 0.441∗∗∗ −0.010 (0.0004) (0.0004) (0.078) (0.0003) (0.062) (0.046) x1,t−1x2,t−1 −0.0002 0.00001 −0.007 −0.0003 −0.082 0.227∗∗∗ (0.001) (0.001) (0.109) (0.0005) (0.087) (0.065) const 45.431∗∗∗ _50.223∗∗∗ _4,063.352∗∗∗ _21.713∗∗∗ _918.640 _756.251 (5.632) (6.069) (1,226.943) (5.478) (981.581) (735.220) Observations 1,271 1,271 1,271 1,271 1,271 1,271 R2 _0.507 _0.190 _0.174 _0.458 _0.423 _0.328 Adjusted R2 0.505 0.186 0.170 0.455 0.420 0.325 Residual Std. Error (df = 1264) 37.919 40.866 8,261.163 36.882 6,609.107 4,950.328 F Statistic (df = 6; 1264) 216.864∗∗∗ _49.313∗∗∗ _44.382∗∗∗ _177.818∗∗∗ _154.149∗∗∗ _103.045∗∗∗ Note: ∗_p<0.1;∗∗_p<0.05;∗∗∗_p<0.01

Table 11: Standard Case Robust Model

estimate se tvalue pvalue

β1 0.1323 0.0469 2.8240 0.0199

β1,1 0.0001 0.0003 0.3260 0.7519

β2 0.2092 0.0409 5.1094 0.0006

β2,2 -0.0007 0.0002 -3.2673 0.0097

Budget allocation of digital marketing instruments: performance under bias and inconsistencies Master Thesis