Comparing forecast accuracy of the EUR to GBP exchange rate using rolling window regressions

(1)

Comparing forecast accuracy of the

EUR to GBP exchange rate using

rolling window regressions

Kiki van Rongen

10772820

University of Amsterdam

Bachelor Econometrics and Operations Research

December 22, 2017

(2)

Statement of Originality

This document is written by Student Kiki van Rongen who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document are orig-inal and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the su-pervision of completion of the work, not for the contents.

(3)

Chapter 1 Introduction

Parameter instability has primarily been identified as a widespread problem in forecast models by Stock and Watson (1996). This instability points to fluctuating coefficients over time and can severely bias forecast outcomes. Among others, Meese and Rogo↵ (1983a) observed this phenomenon in their research. The main aim of their article was to present a comparison based on the predictive ability of a structural model and a random walk model. Both models are applied to several exchange rate time series. It seems reasonable to think that currency variations could be captured best by a model based on economic fundamentals. Much to the contrary, Meese and Rogo↵ (1983a) have found evidence that current macroeconomic models predict equal, if not worse, than a random walk model. This finding was particularly surprising, as they expected the forecast performance to increase when implementing real values of future explanatory variables. Possible explanations that are given in the research include time variation of parameters. Meese and Rogo↵ justify that critical events in the past may cause parameter instability which often leads to misleading results. Moreover, Stock and Watson (1996) have confirmed this theory and stressed that forecasting results can be significantly improved when taking parameter instability into consideration.

The natural question arises as to how such coefficient variation is caused. First of all, the perception that economic agents act simultaneously and identically is often violated. Furthermore, critical events, such as elections or wars, can possibly cause a macroeconomic environment to change. A specific and also topical example of this is the Brexit. The British Pound (GBP) has su↵ered extremely under the consequences of the referendum in 2016, that ruled in favor of the United Kingdom (UK) leaving the European Union (EU). Because of the decline in price of goods and commodities, the position of the UK in the financial market has weakened. As a result, the GBP

(5)

decreased in value and is very unpredictable due to its unstable behaviour. Several other reasons can be found and are further explained in this research herein.

The outperformance of a random walk with regard to models based on economic fundamentals, as depicted in research by Meese and Rogo↵ (1983a), is further in-vestigated by Rossi (2006). This research documented the influence of time-varying parameters in current forecast models. Also, Rossi (2006) questions the capacity of parameter instability for explaining the poor performance of economic exchange rate models. An out-of-sample test is proposed by use of the Mean Squared Forecast Error (MSFE), to assess the value of economic explanatory variables in predicting currency fluctuations. The link between exchange rates and its lagged fundamentals is repre-sented in multiple regressions. This implementation of lagged explanatory variables in the model specification distinguishes the approach from other methods. Finally, the conclusion is drawn that there is no sign of a stable relationship between exchange rate fluctuation and its fundamentals (Rossi, 2006, p. 32).

Since macroeconomic approaches cannot seem to model the right dynamic in ex-change rate variations, improvement should be sought elsewhere. It is clear that forecast methods for exchange rate time series could be enhanced when the model accommodates for parameter instability. Much research has been done on developing new techniques for this aspect. For example, Stock and Watson (1996) established a framework, called the random walk coefficient time-varying parameter (TVP) model. This particular model allows parameters to fluctuate to a certain degree. By applying multiple regressions, Stock and Watson argue once again that a random walk almost always outperforms a fixed-coefficient model. Another method is the rolling regres-sion model, applied in many researches such as Inoue, Jin and Rossi (2017). This forecasting technique adjusts the estimation window continuously. Furthermore, a weighting function can be implemented for the observations in the sample.

This thesis focuses on forecasting the euro to British pound (EUR/GBP) exchange rate, by using the rolling regression method. Two distinct weighting functions are applied based on predictive ability and parameter estimation. First, the rolling re-gression model with equal weights is considered. Subsequently, the rolling window method with exponentially decaying weights is evaluated. After assessing both mod-els, the main question arises as to which forecast model for time-varying parameters has a better predictive ability for the EUR/GBP exchange rate.

The data of the EUR to GBP spot exchange rate are obtained from de Nederland-sche Bank and have been measured daily from the 1st of January 1999 until the 30th of October 2017. This database is a reliable source and encompasses many financial

(6)

time series, including macroeconomic fundamentals and exchange rate fluctuations. Therefore, it is suitable to meet the objectives of this particular research.

The next section presents a summary of important background literature regarding exchange rate volatility. This includes currently available frameworks for coefficient fluctuations in predictive modelling, along with methods of detecting them. Also, theorems on comparing forecast accuracy of di↵erent models are described. Chapter 3 points out the research method applicable in this study. The empirical results are presented and examined in Chapter 4. Chapter 5 concludes.

(7)

Chapter 2 Literature

In this section existing theorems on time-varying parameter models are discussed. This includes the causes and consequences of stochastic behaviour of coefficients. Moreover, the link between exchange rate data and parameter instability is pointed out. Some tests are proposed to detect structural breaks in the data and to choose the optimal forecast model.

2.1 Causes of parameter instability

Exchange rates tend to be notoriously difficult to predict over time. When plotting the time series data, long periods of alternating upward and downward drifts can be observed. Furthermore, the probability that a currency is in a certain state is generally large, except for geographically close situated countries (Engel, 1994, p.153). It is clear that there is some form of parameter instability present. Schinasi and Swamy (1989) presented several possible reasons for this fact. First, assuming that a model is correctly specified, all available information is in some way represented in the explanatory variables. Of course, this assumption is not realistic as models always contain some form of unmodeled dynamics or omitted variables (Giacomini & White, 2006, p. 1547). Nevertheless, it substantiates the perception that the way people use information varies among di↵erent time horizons and policy regimes. Consequently, parameter instability is inevitable.

Another reason for coefficient variation is the presence of many participants in the foreign exchange market. Individuals have di↵erent behavioural patterns and hence their reactions on macroeconomic shocks are not necessarily similar. The complex relationship between exchange rates and its fundamentals cannot directly be depicted in a fixed coefficient and should therefore be time dependent. In addition, assuming

(8)

that individuals act similarly to certain shocks, would also require identical reaction functions. This assumption is most likely not viable.

Third, economic models are often set aside due to their poor forecasting perfor-mance (Meese & Rogo↵, 1983a, p. 21), whereas in most cases these models are solely tested on linear and nonlinear relationships. In reality, the relationship between ex-change rates and economic fundamentals is known to be much more complicated. Rejecting forecasting techniques based on these grounds is not reasonable. The ex-tent to which economic theory can predict exchange rates is limited, but surely not zero. Choosing to exclude this information would require a di↵erent framework for coefficients. Intuitively, an economically supported model is preferred (Schinasi & Swamy, 1989, p. 378).

While these three reasons mentioned above are plausible for explaining parameter instability, the most obvious cause is the event of a macroeconomic shock. Conse-quently, most forecasting techniques for time-varying coefficient models implement the occurrence of structural breaks in their framework (see Pesaran, Pettenuzzo, & Timmermann (2006), Pesaran & Timmermann (2007), and Pesaran, Pick, & Pra-novich (2011)). The main obstacle with such models is further discussed in research of Pesaran, Pettenuzzo, & Timmermann (2006). They state that allowing breaks in a model, means every break segment needs a computed probability. If the number of breaks increases, so does the number of break locations, which complicates the situation considerably.

2.2 Detecting breaks in data

Because the impact of breaks in exchange rate data on forecast outcomes is severe, it is necessary to conduct a test to verify whether time variation is actually observed. Giacomini and Rossi (2009) have developed a framework that detects forecast break-downs in the past. In this context, a forecast breakdown is defined as “a situation in which the out-of-sample performance of a forecast model, judged by some loss func-tion, is significantly worse than its in-sample performance” (Giacomini & Rossi, 2009, p. 670). To measure the di↵erence in forecasting ability, Giacomini and Rossi intro-duce the surprise loss, which is defined as the out-of-sample loss minus the average in-sample loss. A test is proposed, by setting the average of the surprise losses (H0)

equal to zero. Under the null, the applied method is expected to forecast accurate both in-sample and out-of-sample. A positive surprise loss mean, however, indicates

(9)

that the forecast model is likely to break down in the future. The latter case is further explained in the end of this section.

A well-known test for structural breaks in data has been developed much earlier by Chow and is widely applied by many scientists. Chow (1960) questions the stability of a relationship between two periods or economic units. To test whether two sets of observations can follow the same regression form, two distinct models are considered. The first model contains all information before the possible break date and the second model includes only data after the particular break. Both are compared based on the estimated coefficients of explanatory variables, by performing an F-test. If these estimations of parameters are equal, the subsets can be seen as one and thus, fall under the same linear regression model.

2.3 Possibilities for time-varying parameter

models

By tests of Chow (1960) and Giacomini and Rossi (2009) the occurrence of a break can be ascertained. Taking this as a fact, there are several models that are applicable. A distinction is made between forecast models that focus on implementing time-varying parameters and models that account for the timing of the break.

First, time-varying parameter models are considered. The TVP model of Stock and Watson (1996) is a good representation of how forecasting can be improved when taking parameter instability into consideration. To start with, Stock and Watson specified 16 di↵erent models, eight of which contain fixed-coefficients. The other eight consist of varying parameters or rolling regressions. The out-of-sample one-month-ahead forecasts of all models are determined and compared based on their Mean Squared Error (MSE). The results show no dominant performing method, however, Stock and Watson conclude that adaptive models generally predict better than fixed-coefficient models (1996, p. 21), although it should be mentioned that the gain in forecasting accuracy is relatively small.

Another approach to parameter variation has been developed by Creal, Koopman and Lucas (2013). In this research, they introduced a mechanism that updates co-efficients automatically. The coco-efficients are allowed to behave stochastically, just as in the random walk TVP model of Stock and Watson. The model is known as the generalized autoregressive score (GAS) model. The foundation of the GAS model lies within the score function and has multiple benefits. The foremost advantage is that it minimizes the error of the models local fit, as the score function ensures

(10)

a steep ascent in the direction of the regression line. This natural property of the score allows parameters to fluctuate and therefore update automatically. Moreover, all moments of the observations are considered, which means the entire density is taken into account. This distinguishes the GAS model from other forecast methods with time-varying parameters.

Evidently, the range of forecasting models is endless when parameter instability is observed. Many scientists have therefore come to believe that the optimal choice is not always a single method, but rather a combination of forecasting techniques. Research of Pesaran and Timmermann (2007) points out that combined forecasts are presumably more accurate when parameter breaks are small. However, Pesaran and Timmermann emphasize that their method is not based on di↵erent techniques, but di↵erent estimation windows. For this reason the research is an application of rolling regressions, which will be further outlined below. Each forecast model is weighted according to their inverse out-of-sample loss. This ‘loss’ is represented in the MSFE value. If the MSFE is small, the corresponding predictive model is more reliable. Consequently, a higher weight is assigned. A more basic and also suitable strategy is using equal weights, known as pooled forecasting. Because models often contain some sort of defect, an important advantage of the weighted approach documented in Pesaran and Timmermann (2007) is risk diversification. In addition, parameters do not have to be estimated on specific breakpoints and thus, it avoids the calculation of the exact time and size of the break. As presented in Elliot (2005), this task can be highly difficult.

To optimize forecasting, weights do not necessarily have to be assigned to models with di↵erent window sizes. Alternatively, Elliot (2005) suggested a weighted average forecast model that distinguishes with regard to possible break dates. This method is particularly appropriate when breaks are relatively small and hence, not easy to detect. Elliot argues that it is precisely these breaks that tend to be relevant econom-ically (2005, p. 4). Note that in statistics, small breaks can still have a large impact economically. The research also describes the consequences of including an incorrect break date. When the calculated timing is later than or equal to the correct break date, the bias term becomes zero, but the variance increases. On the other hand, if the break date is estimated too early, the bias will increase. The alternating rise in bias and variance can only be balanced by averaging multiple forecasting results. Finally, Elliot illustrates that large parameter breaks can rightfully be employed in the model. Due to the consistency property, the estimated break date will converge asymptotically to the true break date.

(11)

The last approach, already discussed briefly in this paper, is the rolling regression method, also widely applied by many researchers (see Stock & Watson (1996) and Inoue, Jin, & Rossi (2017)). This method is based on excluding observations that are in the more distant past and hence, are likely to be less informative. The estimation window needs to be adjusted constantly for each h-step ahead forecast. There are two main obstacles that need to be overcome when applying rolling regressions. First of all, the number of observations included in the model, also known as the window size, should be determined. Inoue, Jin and Rossi (2017) proposed that the optimal window size is chosen in accordance with the forecast model that minimizes the conditional MSFE. A common thread accompanied with this objective is using only post-break data. However, Pesaran and Timmermann (2007) stress that including pre-break data can cause forecast errors to be unbiased and have lower variance. In addition, they have proven forecast performance to improve when taking the full data into account. Secondly, a function needs to be determined to assign weights to the observations in the sample. Looking at the estimation window it is justified that the first few ob-servations presumably contain less predictive ability for the one-step ahead forecasts than the last set of observations. Therefore, forecasting accuracy may increase when taking this into account. The most standard procedure for weighting observations is exponential smoothing. Assuming the parameters follow a random walk model, which is typical in exchange rate time series, the breaks in the data are likely to be continuous (Pesaran, Pick, & Pranovich, 2013, p. 134). This means that parameters are slightly changing with each period. The research argues that in this particular case, an exponential weighting function is optimal.

While rolling regressions might seem optimal, Elliot (2005) states a few disadvan-tages that should be considered before applying this technique. First of all, the rolling regression method does not allow any stable coefficients. Yet, not all parameters in a model are necessarily time-varying. Furthermore, adapting a certain window range means that every new forecast period an observation drops out. This observation was appointed an equal weight as the others in the data set for the previous forecast pe-riod. Excluding it entirely implies it suddenly lost value over one time period, which seems rather unrealistic.

2.4 Comparing forecast performance

Now that some forecasting methods, such as the GAS model, rolling regressions and random walk TVP model are outlined, the question arises as to which model has

(12)

the best predictive ability. Diebold and Mariano (2002) have developed a test for comparing prediction accuracy between two models based on the same data set. They introduce the loss function at time t to be an arbitrary function of the observed and estimated value yt. Rewriting this function leads to a formula depending only on the

forecast error. The null hypothesis of equal forecast accuracy is depicted by setting the mean of the loss di↵erentials equal to zero and can readily be assessed. Note that the test of Diebold and Mariano is an asymptotic test, however, some finite-sample tests are also discussed in the research. To illustrate, the researchers evaluated the sign test and Wilcoxon’s signed-rank test to comparatively analyse their method. The flaw in currently available predictive accuracy tests is that they do not allow non-Gaussian, serially correlated, nonzero mean or contemporaneously correlated forecast errors. The Diebold and Mariano test tolerates all of these potential difficulties and is therefore widely applicable. As this research focuses on examining two distinct models for the same exchange rate time series, it is stressed that contemporaneous correlation of forecast errors is inevitable.

Another framework for testing forecast performance is given in Giacomini and White (2006). This research provides tests for both conditional and unconditional predictive ability and points out that the latter case is more of an extension to the Diebold and Mariano test. They have developed an environment that encompasses di↵erent types of parameter estimation. Also, Giacomini and White focus on forecast-ing methods rather than merely the forecastforecast-ing model. This covers several complex model specifications. Moreover, the test comprises di↵erent types of forecasts such as point, interval, probability or density.

Interestingly, a model that forecasts well over one period does not necessarily do so over the subsequent period. Therefore, Giacomini and Rossi (2009) also described a theoretical approach for detecting future forecast breakdowns. This, of course, is related to the forecast breakdown test for past breaks discussed earlier. The research assesses the probability that a positive surprise loss mean in the past remains positive. By performing a linear regression of surprise losses on certain explanatory variables, the inconsistency of in-sample and out-of-sample model fit can be explained. Future forecast breakdowns are an important factor in generating accurate exchange rate predictions in the long term. However, assessing di↵erent types of forecasts, rather than prediction models, is not the main focus of this thesis.

(13)

2.5 Approach

This research focuses on rolling regressions and proposes a comparative analysis be-tween two models for forecasting exchange rate fluctuations: the rolling window method with equal weights and exponentially decaying weights. This study most resembles research of Pesaran, Pick and Pranovich (2013), although more attention is paid on comparison techniques for forecast accuracy. Rolling regression can be seen as a time-varying parameter model because the estimation window is constantly mov-ing. This causes coefficients to adapt equally for each period. The material disparity of both forecasting methods ensures a good framework for testing predictive ability.

Fortunately, the applied models do not necessarily require the determination of an exact break point to generate prediction values. It should, however, be stated that they are all based on some form of parameter instability. While most financial time series are essentially dealing with varying coefficients, it is best to confirm this assumption using the Chow test for structural breaks. If the Chow test detects no particular break point in the data set, a model that does not account for varying coefficients is preferred.

Finally, the Giacomini and White (2006) test is appropriate to asses the generated point forecasts of the rolling regression models. The test coincides partially with the Diebold and Mariano (2002) test and focuses on rolling regression models. Although their main emphasis is on comparing two competing forecast methods, it can also provide a comparative framework for prediction values. Hence, it is a general appli-cation for analyzing out-of-sample forecasts that fits well with the objective of this research. Furthermore, the study contains several advantages, such as the allowance for misspecification and other complexities accompanied by parameter estimation, with regard to other forecast performance techniques.

(14)

Chapter 3 Research method

Some theorems have been suggested for addressing the issue of parameter instability in forecasting models. This chapter provides an additional description of the selected models, namely the rolling regression method with equal weights and exponentially decaying weights. Both are outlined mathematically and the incurred assumptions are stressed. Also, the test for structural breaks of Chow (1960) is explained as well as the Giacomini and White (2006) test for comparing the di↵erence in predictive accuracy of the two considered models.

3.1 Test for structural breaks

To start with, some clarification is needed regarding the occurrence of breaks in the data set. The Chow (1960) test questions whether one regression model can cover the dynamic of two di↵erent sets of observations. In this research, the EUR/GBP exchange rate is considered and therefore some possible break dates regarding this particular time series are set. Looking at the British economic environment, a clear example of a possible structural break in the data is the Brexit on the 23rd of June 2016. As mentioned in the introduction, this event has caused many instabilities and insecurities in the economy and hence, the British currency is likely to be a↵ected.

Another option is the European debt crisis. Since the end of 2009 several members of the European Union, such as Portugal, Spain, Ireland, Greece and Cyprus, have had trouble repaying their government debt and asked for the support of other Eurozone countries. The EU has su↵ered greatly under this. Financial markets were dominated by panic and distress, which is represented in a decrease in value of the EUR/GBP exchange rate. Finally, the global financial crisis from 2007 to 2009 is an obvious choice of break date in the Chow test. It has shaped the entire world economy to what it is today and is likely to cause complications in the forecasting model. A quick

(15)

Figure 3.1: The daily EUR/GBP spot exchange rate

look at the data (Figure 3.1) justifies all of these possible break dates, however, to obtain empirical evidence the Chow test must be performed. Note that an ascent direction of the line in Figure 3.1, depicts a decrease in value of the GBP.

The model is given by  y1 y2 =  X1 0 0 X2  1 2 +  ✏1 ✏2 , or y1t= X1t 10 + 0· 2+ ✏1t, (3.1) y2t= 0· 1+ X2t 20 + ✏2t, (3.2)

where y1t, y2t, ✏1t and ✏2t are column vectors with n elements, X1t and X2t are

re-spectively nonsingular n1⇥ k, n2 ⇥ k matrices, 1 and 2 are k ⇥1 column vectors.

The dependent variable y is the EUR/GBP spot exchange rate and k is the number of regression coefficients, in this case 1. The errors ✏1t and ✏2t are independent and

normally distributed with mean zero and standard deviation . The subscripts 1 and 2 denote the division of the data into two sets of observations n1 and n2. Under the

null, the coefficients are equal (H0 : 1 = 2 = ) and the model can be described as

yt = Xt0 + ✏t. (3.3)

The alternative states unequal coefficients (Ha : 1 6= 2) and substantiates the

presumption of breaks in the data set. The Chow test statistic is then calculated by (SSRc (SSR1+ SSR2))/k

(16)

and is F-distributed with k and n1+ n2 2k degrees of freedom. SSR1, SSR2 and

SSRc are the sum of squared residuals of the model specified in Equation (3.1), (3.2)

and (3.3) respectively.

3.2 Rolling regression model

The following data generating process (DGP) is assumed:

yt+h = Xt h,t0 + t+h, (3.5)

where {(yt, Xt), t = 1, . . . , T} are observed, Xt is an exogenous k-dimensional

re-gressor, h,t is a k ⇥ 1 vector of unobserved time-varying parameters, t+h is an

unobservable disturbance, T denotes the full sample size and h indicates the fore-cast horizon. The errors are assumed to be temporally uncorrelated. Estimations of the parameters in Equation (3.5) can be obtained via rolling ordinary least squares (OLS). This method considers di↵erent weighting functions for the observations in the information set and is widely used by many studies. For comparison, various estimation windows are applied.

3.3 Exponential smoothing

Next, the weighting function is introduced. Note that the data is most likely to contain continuous breaks, as stated in Section 2.3. Taking this as a fact, an exponential smoothing function is preferred to weigh the observations, which is set up as follows:

!t= t, for t = 1, . . . , p, (3.6)

where !t is a p⇥ 1 vector of weights calculated for each time t, p is equal to the

window size and denotes the rapidity with which the weights decline. In this research, is selected in accordance with the window size. Further clarification on the determination of the smoothing function parameter is given in Chapter 4.

3.4 Test for comparing forecast accuracy

The test considered in this research is the Giacomini and White (2006) test. The main focus is the comparing point forecasts, rather than the predictive models. Although

(17)

Giacomini and White (2006) also provide a framework for the latter case. Each one-step ahead forecast i or j is assigned a score value denoted by

Si,t+1= log( ˆf (yt+1)), for t = p, . . . , T, (3.7)

where T denotes the full sample size, p is the window size and ˆf (_{·) is the normal} density function with mean and variance calculated as E(ˆyi) and V ar(ˆyi), evaluated

for each yt+1. By taking the di↵erence of the score values of both forecasts, a two-sided

hypothesis test is set up as

H0 : E(dt+1) = 0, Ha : E(dt+1)6= 0, (3.8)

with

dt+1= Si,t+1 Sj,t+1. (3.9)

Standardizing the di↵erences leads to a test statistic of the form p

T d¯ ˆd

d

! N(0, 1). (3.10)

If the long-run variance is used for estimation, this test statistic is similar to the Diebold and Mariano test statistic. Giacomini and White consider the entire density of the forecasts to calculate the score value. To obtain a standard normal distributed test statistic, it is assumed that the score di↵erences are normally distributed.

(18)

Chapter 4 Empirical results

This chapter points out the empirical results of the conducted tests as well as the forecasts of the two competing models. However, first some clarification is needed with regard to the stability of the series. It should be noted that most financial time series contain a form of exponential behaviour. To smooth out this trend, the data is transformed by taking the logarithm.

Furthermore, econometric models are often developed under the assumption of a stationary time series. As seen in Figure 3.1, the series does not fluctuate around zero, nor does it show a clear trend. Figure 4.1 illustrates that the autocorrelation function (ACF) of the log EUR/GBP exchange rate slowly decreases over time. In addition, Figure 4.2 shows no significant lags, implying that first di↵erencing is sufficient to obtain a stationary time series. To verify this assumption, the augmented Dickey-Fuller (ADF) test is conducted. With a significance level of 5%, the null hypothesis that a unit root is present is not rejected. Hence, the time series has a stochastic trend and can only be stabilized by taking first di↵erences. The altered time series is given in Figure 4.3.

After transforming the data, the Chow test for structural breaks is performed. Several possible break dates are applied and the outcome is assessed with a 5% sig-nificance level according to the corresponding p-value. The break points are set equal to the economic crisis, the European debt crisis and the Brexit with p-values of respec-tively 0.000, 0.122 and 0.001. Evidently, there are two significant structural breaks in the data set. Note that this research has specified only three possible break points, however, there could be many more. The main reason for performing the Chow test is to justify the choice of a time-varying parameter model. Structural breaks in finan-cial time series are common practice in forecasting, as previously discussed in Section 2.1. The European debt crisis does not provide a significant outcome. The rationale behind this could be that many EU parties, including the UK, had to support other

(19)

0.00

0.50

1.00

-0.50

acflog.indd 1 19-12-17 15:58

Figure 4.1: ACF Log EUR/GBP

0.00 0.05 0.10 -0.05 -0.10 Naamloos-1 1 19-12-17 15:40

Figure 4.2: ACF Di↵ Log EUR/GBP

countries financially and thus, it is not the EUR/GBP exchange rate that is severely a↵ected.

A detailed description of the variables is given in Appendix A. Figure 4.4 reports the one-step ahead point forecasts of the rolling regression model with equal weights and exponentially decaying weights for approximately three months of the sample. For convenience, the empirical results are restricted to a limited interval of the full forecast data set. The graph provides convincing evidence that the forecasts of the equally weighted rolling regression model are closer to the observed values. Moreover, the line chart of the equally weighted rolling regression forecasts coincides more with the behaviour of the observed time series, than those of model 2. When the exchange rate is decreasing, so does the forecasted exchange rate under model 1. Although, both prediction models appear to cause an enhanced e↵ect. Because relative changes

(20)

Figure 4.3: Di↵ Log EUR/GBP

per day are minor, deviations of the predictions seem very large in the graph.

The application of an exponentially decaying window was based on the assumption that observations from the more distant past contain less information for the one-step ahead forecast than the last set of observations in the window size. To the contrary, Figure 4.4 shows that forecast accuracy is increased when considering all observations equally. This is in line with Section 2.1, which states that exchange rates behave in long periods of alternating upward and downward trends. It is apparent that the first set of observations in the estimation window are extremely important in forecasting. To provide empirical and more general evidence, some tests are conducted.

A reasonable criteria for measuring predictive ability is the mean squared error, defined as

M SE = PT

t=1000(ˆyi,t yt)2

T 1000 , (4.1)

where T denotes the full sample size and ˆyi,t is the predicted value at time t of model i

for i is 1, 2 and ytis the observed value at time t. This measurement tool is frequently

used in many studies. Simply stated, the MSE is the squared di↵erence between the observed and forecasted value. From Table 4.1, it can be inferred that the equally weighted rolling regression method generally has a lower MSE. However, it should be noted that expanding the length of the estimation window will not increase accuracy in point estimations for the exponentially weighted regression model. The first set of observations are all assigned a weight almost equal to zero and thus it has little influence on the forecast outcome. A more logical comparison of forecasting models is therefore obtained by choosing a di↵erent window size for the rolling regression

(21)

Figure 4.4: Actual and predicted exchange rate changes. The long dashed line is the observed exchange rate change. The short dashed line is the predicted change from the rolling regression model with equal weights, and the solid line is the predicted change from the rolling regression model with exponentially decaying weights. methods considered. Table 4.1 shows a decrease in MSE when the estimation window of the models is extended. The window size of model 1 and 2 are hereinafter referred to as n1 and n2. When n1 is selected large, n2 must be chosen even larger. As

a consequence, the exponentially decaying rolling regression method will loose its predictive relevance.

Because exchange rate time series are very challenging to predict, it might be more optimal to include fewer observations. For example, the one-day ahead forecast is determined by analyzing only the exchange rates of the past couple of weeks. Table 4.1 provides convincing evidence to the contrary and substantiates the application of a larger window size. Hence, n1 is set equal to 2000 and n2 is 3000. Subsequently,

the weights function is adjusted by determining the surface beneath its graph for the first n2 n1 observations. If this area is negligible, the weighting function is sufficient

for application in the exponentially decaying rolling window model. This is the case for is equal to 0.9975, as shown in Figure 4.5.

Next, the forecasting measurement technique is analyzed. To obtain conclusive results, the hypothesis tests are set up as one-sided. This is underpinned by the presumption that using equal weights fits better with the observed exchange rates

(22)

Window size Model MSE n=50 1 0.0116 2 0.0116 n=100 1 0.0066 2 0.0066 n=500 1 0.0021 2 0.0023 n=1000 1 0.0011 2 0.0018 n=1500 1 0.0008 2 0.0019 n=2000 1 0.0007 2 0.0021 n=3000 1 0.0002 2 0.0007

Table 4.1: MSE values of model 1 and 2 for window size n

Figure 4.5: Weighting function, =0.9975

than exponentially decreasing weights. As research of Giacomini and White (2006) is partially based on the study of Diebold and Mariano (2002), this tool is extremely appropriate for exchange rate time series.

The Giacomini and White test starts with assigning a score to each point forecast that is generated in the sample. This score value is calculated by taking the density of the forecast distribution evaluated for a specific observation, assuming that the

(23)

forecasts are normally distributed. Consequently, a high score indicates that the forecast lies within an area with high predictive density. For comparison this research has also considered the density function of the Student-t distribution, but this does not lead to an increase of the score. Hence, Figure 4.2 reports the score values of normally distributed forecasts of both models, restricted to a three month interval. It is evident that the model with the highest average score is preferred. This can easily be measured by the following formula:

¯ Si,t+1 = 1 m T X ni log( ˆf (yt+1)), (4.2)

where m = T ni, ni is the window size of model i for i = 1, 2, T is the full sample

size and ˆf (yt+1) is the normal density function evaluated in yt+1. Indeed, the results

show a higher score average for model 1.

Next, the one-sided hypothesis test is conducted. Under the null and the alterna-tive it is assumed that

H0 : E(dt+1) = 0, Ha: E(dt+1) > 0 for all t, (4.3)

with

dt+1 = S1,t+1 S2,t+1, (4.4)

where S1,t+1 and S2,t+1 are the score of respectively model 1 and model 2 at time

t + 1. The foremost advantage of this method is that it considers the entire density. By di↵erencing the score of the two models, the test statistic can be computed. Assuming this statistic is normally distributed, the critical values can be found in any standard normal distribution table. The calculated test statistic is equal to -210.3, which is located in the far left tail of the distribution. It is clear that the null hypothesis of equal performance is rejected at the 1% significance level. There is sufficient evidence to conclude that, based on the Giacomini and White test, the point forecasts of model 1 are closer to the observed exchange rate than those of model 2.

(24)

Table 4.2: Rolling out-of-sample forecasts of the EUR/GBP exchange rate, score values, residuals and RMSE (September to November 2012).

(25)

Chapter 5 Conclusion

This research provides a method for assessing the di↵erence in predictive accuracy of two competing models. Several forecasting techniques have been proposed, with a particular emphasis on the EUR/GBP exchange rate. To detect the best among them, the Giacomini and White test is conducted. In addition, other forecast measurement tools, such as the MSE and average score, are also given.

Parameter instability has first been confirmed by the structural break test of Chow (1960). By means of this test, it can be deduced that macroeconomic shocks can cause a shift in economic behaviour of market participants. Furthermore, from the empirical results presented in Chapter 4, the conclusion is drawn that the rolling window model with equal weights provides more accurate forecasts than the regression model with exponentially decaying weights. The foremost reason for this lies within the behaviour of exchange rate time series. Due to the long periods of upward or downward trend, forecasting accuracy is increased when all observations are equally weighted in the model. This presumption is verified by the Giacomini and White test. The method in this research captures both the point forecasts and associated density and is therefore a sufficient, general framework for comparing predictive ability. By adjusting the window size, it is shown that smaller estimation window lengths are preferred. Also, several choices for the distribution of the forecasts are considered.

Finally, a concluding remark is made towards further investigation. This study provides convincing evidence that the trend in exchange rate time series is extremely difficult to forecast. As daily changes are so small, essentially a white noise is pre-dicted. It may be more convenient to focus on the volatility of the series, however, this area of research remains future work.

(26)

Appendix A

dlograte EUR/GBP exchange rate log

forecast forecasts of the rolling regression model with equal weights (model 1)

level forecast1 forecasts of the rolling regression model with

exponentially decaying weights (model 2)

level score score value for each observation of model 1 level score1 score value for each observation of model 2 level di↵score di↵erence in score value of model 1 and 2

res residual of the regression equation for model 1

level res1 residual of the regression equation for model

2

level rmse root mean squared error, measured as the

dif-ference between the two forecasts

level Table A.1: Data description.

(27)

References

Chow, C. (1960). Tests of equality between sets of coefficients in two linear regressions. Econometrica, 28 (3), 591-605.

Creal, D., Koopman, S. J., & Lucas, A. (2013). Generalized autoregressive score models with applications. Journal of Applied Econometrics, 28 (5), 777-795. Diebold, F. X., & Mariano, R. S. (2002). Comparing predictive accuracy. Journal of

Business and Economic Statistics, 20 (1), 134-144.

Elliot, G. (2005). Forecasting when there is a single break. Manuscript, University of California at San Diego.

Engel, C. (1994). Can the Markov switching model forecast exchange rates. Journal of International Economics, 36 (1–2), 151–165.

Giacomini, R., & Rossi, B. (2009). Detecting and predicting forecast breakdowns. The Review of Economic Studies, 76 (2), 669-705.

Giacomini, R., & White, H. (2006). Tests of conditional predictive ability. Econo-metrica, 74 (6), 1545–1578.

Inoue, A., Jin, L., & Rossi, B. (2017). Rolling window selection for out-of-sample forecasting with time-varying parameters. Journal of Econometrics, 196 (1), 55-67.

Meese, R. A., & Rogo↵, K. (1983a). Empricial exchange rate models of the seventies: Do they fit out of sample. Journal of International Economics, 14 (1–2), 3–24. Pesaran, M., Pettenuzzo, D., & Timmermann, A. (2006). Forecasting time series

subject to multiple structural breaks. The Review of Economic Studies, 73 (4), 1057–1084.

Pesaran, M., & Timmermann, A. (2007). Selection of estimation window in the presence of breaks. Journal of Econometrics, 137 (1), 134-161.

Pesaran, M. H., Pick, A., & Pranovich, M. (2013). Optimal forecasts in the presence of structural breaks. Journal of Econometrics, 177 (2), 134-152.

Rossi, B. (2006). Are exchange rates really random walks? Some evidence robust to parameter instability. Macroeconomic dynamics, 10 (1), 20–38.

Schinasi, G. J., & Swamy, P. A. V. B. (1989). The out-of-sample forecasting perfor-mance of exchange rate models when coefficients are allowed to change. Journal of International Money and Finance, 8 (3), 375–390.

Stock, J. H., & Watson, M. W. (1996). Evidence on structural instability in macroe-conomic time series relations. Journal of Business and Emacroe-conomic Statistics, 14 (1), 11–30.

(28)

variance in a time-varying parameter model. Journal of the American Statistical Association, 93 (441), 349–358.

Comparing forecast accuracy of the EUR to GBP exchange rate using rolling window regressions