Correlation Structure Estimation techniques and their forecasting performance

Correlation Structure

Estimation techniques and

their forecasting performance

Rutger Kamps

Correlation Structure

Estimation techniques and

their forecasting performance

Thesis MSc Finance

Faculty of Economics and Business University of Groningen

Rutger Kamps†

August 2014

Abstract

In this research ten techniques that estimate correlation structures are com-pared on their forecasting performance. The techniques comcom-pared are: the histori-cal model, four single-index models, the constant correlation model, two shrinkage models and two multivariate GARCH models. Industry sector benchmarks are used as security returns, and the MSCI World Index is used as the market index. With-out any adjustment by the mean correlation coefficient, the techniques Shrink-SF, EWMA, DCC, Historical and Shrink-CC generally perform best. After these tech-niques the Constant technique performs best. The worst performing techtech-niques are the single-index models. When doing a historical adjustment to the mean correla-tion coefficient, the best performing techniques remain the same. The single-index models perform relatively better with the historical adjustment, except for the Beta = 1 technique.

JEL classification: C30, C51, G11, G17.

Keywords: correlation matrix forecasting, model comparison, historical model, single-index mod-els, unadjusted beta, Vasicek’s adjustment, Blume’s adjustment, assumed beta, constant cor-relation, overall mean, shrinkage, multivariate GARCH models, exponentially weighted moving average, dynamic conditional correlation

†_{Student number: s1793128}

E-mail: r.kamps@student.rug.nl Supervisor

Contents

1 Introduction 1 2 Techniques 3 2.1 Returns in general . . . 3 2.2 Historical model . . . 3 2.3 Single-index models . . . 4 2.4 Multi-index models . . . 6 2.5 Mixed models . . . 7

2.6 Constant correlation model . . . 7

2.7 Shrinkage . . . 7

2.8 Multivariate GARCH models . . . 9

3 Methodology 12 3.1 Absolute forecast errors . . . 12

3.2 Differences in absolute forecast errors . . . 14

3.3 Distribution of differences . . . 14

3.4 Adjustment by the mean correlation coefficient . . . 15

4 Data 17 4.1 Security returns . . . 17 4.2 Market index . . . 17 4.3 Periods . . . 18 5 Results 23 5.1 Unadjusted forecasts . . . 23

5.2 Adjustment by the mean correlation coefficient . . . 35

6 Conclusion 38 6.1 Forecasting performance of techniques . . . 38

6.2 Further research . . . 39

References 40

Appendices 45

A Industry portfolios in subsamples . . . 45

B Means and standard deviations . . . 46

C Differences in absolute errors . . . 52

1. Introduction

Markowitz’ mean-variance optimization (Markowitz, 1952, 1959) has been a popular and widely used optimization technique for constructing security portfolios. Mean-variance optimization is dependent on the expected returns and variances of securities and the correlations among securities.

Goodman (2009) identifies five categories of criticism on the mean-variance optimiza-tion: 1. the mean and variance of securities are not the only things investors care about; 2. the means and covariance (and correlation) matrices of securities are hard to estimate; 3. the means and covariance matrices of securities are not a complete description of mar-ket returns; 4. returns are not linear functions of the investment weights on securities; and 5. investment strategies are more sophisticated than just calculating the investment weights on securities.

In terms of the second category of criticism on the mean-variance optimization, Elton and Gruber (1973) argue that analysts face two problems when estimating the correlations among securities: a. the large number of estimates; and b. the accuracy of techniques used for estimating the correlation structure.

Regarding the first problem analysts face when estimating correlations among secu-rities, computational power continues to increase as well (Moore, 1965). Therefore, the large number of estimates becomes less and less of a problem.

Regarding the second problem analysts face however, the accuracy of techniques used for estimating the correlation structure remains an issue. Small differences in correlation structures can have a huge impact on the optimal portfolio weights, calculated using Markowitz’ mean-variance optimization. In this research different techniques for estimat-ing the correlation among securities are compared.

Elton et al. (1978) already compared three techniques (the full historical model; single-index models, including types of adjustments; and the constant correlation model) over the estimating periods 1961 to 1965 and 1966 to 1970 and forecasting periods 1966 to 1970 and 1971 to 1975 and concluded that the overall mean method (the constant correlation model) is the preferred method for forecasting future correlation coefficients.

But the techniques that can be used for estimating the correlation structure are not limited to these three. Other techniques, such as multi-index models and mixed models, do exist (Elton et al., 2010).

Another alternative is the multivariate GARCH model. According to Zakamulin (2014), multivariate GARCH models provide superior covariance matrix forecasting performance compared to the full historical model. Drawback of these multivariate GARCH models is

that the correlations tend to exhibit ghost-features (Alexander, 1996), i.e. major market movements are reflected in the correlation for a period after they have occurred. These ghost-features depend on the length of the rolling-window used, which determines the smoothness of temporal movement in the data set.

This research tests the forecasting performance of different techniques used for esti-mating the correlation structure of securities. The techniques used in this research are: the historical model, four single-index models (unadjusted betas, Vasicek’s adjustment, Blume’s adjustment, assumed beta), the constant correlation model, two shrinkage models (using unadjusted betas as shrinkage target and using constant correlation as shrinkage target) and two multivariate GARCH models (the EWMA and the DCC). The techniques are discussed in chapter 2.

The techniques are compared in terms of differences in absolute forecast errors and in terms of the distribution of the differences in absolute forecast errors. An adjustment by the mean correlation is made as well, as described in chapter 3. The techniques are compared over five periods, as discussed in chapter 4.

2. Techniques

In this chapter various techniques are discussed, that can be used for forecasting the correlation structure between securities. In section 2.2 the historical model is discussed. In section 2.3 the single-index models are discussed, with several adjustments to it. Multi-index models are discussed in section 2.4 and mixed models are discussed in section 2.5. In section 2.6 and in section 2.7, the constant correlation model and shrinkage are discussed respectively. Finally, multivariate GARCH models are discussed in section 2.8.

But before all the techniques are being discussed, some general remarks regarding returns are made in section 2.1.

2.1

Returns in general

Wherever in this research is spoken about security returns, whether it be Rior Rm, excess returns are meant. The (excess) return of a security Rtat time t is calculated within the periods of table 4.1 using equation (2.1), in which rtis the actual return of a security and E(r) is the average return of a security over one of the periods of table 4.1.

Rt= rt− E(r) (2.1)

2.2

Historical model

The (full) historical model assumes that the past values of the correlation coefficients are the best estimates for the future values (Cohen and Pogue, 1968; Blume, 1970; Levy, 1971; Elton and Gruber, 1973). There’s no assumption regarding the co-movement be-tween securities. The forecast of the correlation structure is determined by calculating the correlations between securities over a historical period.

The covariance matrix of the historical model, ˆΣHistorical is calculated using equa-tion (2.2), with T being the sample size and Rt a 1 × N vector of returns of N securities at time t. ˆ ΣHistorical = 1 T − 1 T X t=1 RT_t · Rt (2.2)

The correlation coefficients can be derived from the covariance matrix of the historical model by applying equation (2.3). In this equation ρij is the correlation coefficient between the securities i and j, which can be calculated by dividing the covariance between the

returns of securities i and j (cov(Ri, Rj)) by the prodct of the standard deviations of securities i and j (σiσj). ρij = cov(Ri, Rj) σiσj (2.3)

2.3

Single-index models

Single-index models, in all their variants, have the assumption that there is a systematical co-movement between securities because of a single index (Elton et al., 1978). Betas describe this systematical co-movement with the market as the single index. The betas are estimated over a historical period and used to forecast the correlation structure.

The correlation coefficient between securities i and j (ρij) is obtained from the betas of securities i and j and standard deviations of securities i and j and of the market m, by applying equation (2.4). This is equal to the correlation coefficient between security i and the market m, times the correlation coefficient between security j and the market m.

ρij =

βiβjσm2 σiσj

= ρimρjm (2.4)

2.3.1

Unadjusted betas

The unadjusted betas variant of the single-index model uses equation (2.5). From this formula, ordinary least squares is used to estimate αi and βi (Draper and Smith, 1998; Brooks, 2008). ˆβi is estimated as shown in equation (2.6). No adjustments to the betas are being made, hence unadjusted betas (Sharpe, 1963).

ri = αi+ βirm+ εi (2.5) ˆ βi = P Ri,tRm,t P(Rm,t)2 (2.6)

2.3.2

Vasicek’s adjustment

Another variant of the single-index model is Vasicek’s adjustment to the betas (Vaˇs´ıˇcek, 1973). This procedure adjusts the betas of equation (2.5). The procedure of adjusting the betas is shown in equation (2.7).

In order to adjust the beta of security i, βi,2, this procedure takes a weighted average of the average of betas over the historical period ¯β1, as shown in equation (2.8) with N securities; and the beta of security i over the historical period βi,1, as estimated in equation (2.6). The weights depend on the variance of the beta of security i over the historical period, σ2

βi,1, which is the squared standard error of the estimated βi,1 as shown

in equation (2.9) with s being the estimate of the variance of εias shown in equation (2.10) with sample size T ; and the variance of the beta of the average of betas over the historical period, σ_β2¯₁, as shown in equation (2.11).

¯ β1 = 1 N N X i=1 βi,1 (2.8) σ_β2_i,1 = SE (βi,1)2 = s2 1 P Rm,t (2.9) s2 = P ε 2 i,t T − 2 (2.10) σ_β2¯₁ = var( ¯β1) (2.11)

2.3.3

Blume’s adjustment

Blume’s adjustment to the betas uses two historical periods (Blume, 1971, 1975). First the betas over the first historical period, βi,1, are estimated using equation (2.6), and then the betas over the second, later, historical period, βi,2, are estimated, again using equation (2.6). Then the estimates of betas over these two historical periods are regressed using equation (2.12) by applying the ordinary least squares method.

ˆ

βi,2 = ˆβ0+ ˆγ ˆβi,1 (2.12)

This regression will yield an outcome, e.g. as shown in equation (2.13). Now the betas estimated over the second historical period are substituted into this regression as ˆβi,1 in order to get the betas for the forecast period.

ˆ

βi,2 = 0.343 + 0.677 ˆβi,1 (2.13)

2.3.4

Assumed beta

Fisher and Kamin (1971) motivated the use of all betas equal to one, i.e. for all N securities β = 1, because betas have the tendency to regress to one (Blume, 1975; Levy, 1971). In this way no regression has to be made in order to get the betas.

2.4

Multi-index models

Contrary to single-index models, which assume that securities systematically move to-gether only because of a single index, multi-index models, or factor models, assume that securities systematically move together because of multiple indices (Elton et al., 2010). An example of a multi-index model is shown in equation (2.14), which has L indices.

ri = αi+ βi,1I1+ βi,2I2+ . . . + βi,LIL+ εi (2.14)

According to Cohen and Pogue (1968); Elton and Gruber (1971); Elton et al. (2010) multi-index models generally do a better job of explaining historical correlation matrices, but they do a poorer job of predicting future correlation matrices when adding additional indexes to the single-index model.

Farell built a multi-index model based on Kings’ model (which includes the indices market, industry and company) (King, 1966), in which he classified securities into four pseudo-industries: 1. growth stocks; 2. cyclical stocks; 3. stable stocks; and 4. oil stocks (Farrell, 1974, 1976). According to Farrell, his multi-index model performed better in predicting future correlation matrices than the single-index model. According to Fertuck (1975) however, traditional industry grouping performed better than the pseudo-industry grouping of Farrell in most cases.

Other multi-index models, use (some) factors that are incompatible with the data used for security returns (see section 4.1). E.g. Fama and French (1993) use the model shown in equation (2.15), in which rm − rf is the market risk premium, SMB stands for “Small market capitalization Minus Big” which measures the historic excess returns of small caps over big caps, and HML stands for “High book-to-market ratio Minus Low” which measures the excess returns of value stocks over growth stocks. Using this model would imply rearranging the portfolios used (as described in section 4.1) to other portfolios, making the outcomes of predicting performance of the two types of portfolios less comparable.

ri = rf + β1(rm− rf) + β2SMB + β3HML + α (2.15)

one of the seven explanatory variables of the Salomon Brothers’ model; the other being economic growth, long-term interest rates, short-term interest rates, inflation, U.S. dollar rates, and the uncorrelated part of the market index). The portfolios used as security returns might consist of different actual securities that responds differently to the index business cycle.

Because of the nature of the data used for security returns (see section 4.1) and the overall outcomes of research that multi-index models do a poorer job of predicting future correlation matrices, multi-index models are not used in this research.

2.5

Mixed models

Mixed models use the single-index model of section 2.3 as the basis and use a second model to predict the extramarket (components of) covariance, i.e. the covariances among securities beyond those attributable to an overall market factor (Rosenberg, 1974; Elton et al., 2010). The model of Rosenberg (1974) for example uses 114 variables (including e.g. traditional industry classification, debt-equity ratios and dividend payout measures) to explain the extramarket covariance.

These types of models are not used in this research for the same reason that multi-index models are not used in this research (see section 2.4); firstly, the nature of the data used in this research might not always be compatible with the nature of the data used in mixed models, and secondly, the overall outcomes of research indicate that mixed models do a poorer job of predicting future correlation matrices.

2.6

Constant correlation model

The constant correlation model (or overall mean correlation model or average correlation model), assumes that historical data only contains information on the mean correlation coefficients (Elton et al., 1978). It assumes that pair-wise differences from the average are random. This technique therefore sets every correlation coefficient equal to the average of all pair-wise correlation coefficients.

2.7

Shrinkage

Ledoit and Wolf (2004) proposed the concept of shrinkage: shrinking a sample covariance matrix toward a structured estimator, as shown in equation (2.16).

ˆ

ΣShrink = δF + (1 − δ)S (2.16)

In this equation ˆΣShrink is the shrunken covariance matrix; δ is the shrinkage constant, which must have a value between 0 and 1; F is the highly structured estimator or shrinkage target; and S is the sample covariance matrix.

While Ledoit and Wolf (2003) suggest using the single-factor matrix of Sharpe (1963) of section 2.3.1 as the shrinkage target, Ledoit and Wolf (2004) suggest using the constant correlation model of section 2.6.

As the shrinkage constant δ, Ledoit and Wolf (2004) propose using the optimal shrink-age constant δ∗, which is the shrinkage constant that minimizes the expected distance between the shrinkage estimator ΣShrink and the true covariance matrix Σ. The opera-tional model for the shrinkage estimator is shown in equation (2.17).

ˆ

ΣShrink = δ∗F + (1 − δ∗)S (2.17)

Ledoit and Wolf (2004) propose to use the estimated shrinkage of equation (2.18). In this equation T equals the number of sample observations on each of N security returns (Frost and Savarino, 1986) and the estimator ˆκ is defined as ˆκ = ˆπ− ˆ_γˆρ. The estimators ˆπ,

ˆ

ρ and ˆγ are described below.

ˆ δ∗ = max  0, min ˆκ T, 1  (2.18)

The estimator ˆπ is the estimator for π, which denotes the sum of asymptotic variances of the entries of the sample covariance matrix scaled by√T . The estimator ˆπ is calculated as denoted in equation (2.19), with ˆπij as shown in equation (2.20), in which yit is the return on security i in period t and ¯yi is the average return on security i and sij is a typical entry of S. ˆ π = N X i=1 N X j=1 ˆ πij (2.19) ˆ πij = 1 T T X t=1 {(yit− ¯yi)(yjt− ¯yj) − sij}2 (2.20)

The estimator ˆρ is the estimator for ρ, which denotes the sum of asymptotic covari-ances of the entries of the shrinkage target with the entries of the sample covariance matrix scaled by √T . The estimator ˆρ is calculated as denoted in equation (2.21), with

ˆ

ϑii,ij as shown in equation (2.22) and ˆϑjj,ij as shown in equation (2.23). In these formulas ¯

r is denoted in equation (2.24), with rij = √_ssij

ˆ ρ = N X i=1 ˆ πii+ N X i=1 N X j=1,j6=i ¯ r 2 r sjj sii ˆ ϑii,ij+ r_s ii sjj ˆ ϑjj,ij  (2.21) ˆ ϑii,ij = 1 T T X t=1

(yit− ¯yi)2− sii × {(yit− ¯yi)(yjt− ¯yj) − sij} (2.22)

ˆ ϑjj,ij = 1 T T X t=1

(yjt− ¯yj)2− sjj × {(yit− ¯yi)(yjt− ¯yj) − sij} (2.23)

¯ r = 2 (N − 1)N N −1 X i=1 N X j=i+1 rij (2.24)

The estimator ˆγ is the estimator for γ, which measures the misspecification of the population shrinkage target. The estimator ˆγ is calculated as denoted in equation (2.25), with fij being a typical entry of F.

ˆ γ = N X i=1 N X j=1 (fij − sij)2 (2.25)

Since this section constantly talks about the covariance matrix instead of the corre-lation matrix, the formula of retrieving the correcorre-lation matrix from a covariance matrix is shown in equation (2.4) of section 2.3.

2.8

Multivariate GARCH models

Research has shown that correlations are not constant over time (Longin and Solnik, 1995; Tse, 2000; Engle and Sheppard, 2001; Goetzmann et al., 2003; Wong and Vlaar, 2003). Longin and Solnik (2001); Ang and Chen (2002) conclude that the correlations of international equities are much higher during bear markets than during tranquil markets or during bull markets.

Because of the correlations not being constant over time, Autoregressive Conditional Heteroskedasticity (ARCH) models and Generalized Autoregressive Conditional Het-eroskedasticity (GARCH) models have been applied to estimate correlations between security returns (Engle, 1982).

Though a lot of variations of the (G)ARCH models exist (see e.g. Bollerslev (2009)), this research only uses the EWMA model (see section 2.8.1) and the DCC model (see section 2.8.2).

2.8.1

Exponentially Weighted Moving Average

The Exponentially Weighted Moving Average (EWMA) model of Longerstaey and Spencer (1996) uses recursion to estimate the covariance matrix ˆΣEWMA. The variances and co-variances are defined as IGARCH-type models (Engle and Bollerslev, 1986; Bauwens et al., 2006). The formula of the EWMA is shown in equation (2.26). In this formula, λ is the decay constant, with 0 < λ < 1, Rt−1 is a 1 × N vector of N security excess returns at time t − 1. RiskMetrics recommend using λ = 0.94 when using daily returns, and λ = 0.97 when using monthly returns (Longerstaey and Spencer, 1996; Sheppard, 2013; Zakamulin, 2014). It is assumed that ˆΣEWMA,t=0= 0N,N, i.e. at t = 0, ˆΣEWMA is a N × N null matrix.

ˆ

ΣEWMA,t = (1 − λ)RTt−1Rt−1+ λ ˆΣEWMA,t−1 (2.26)

2.8.2

Dynamic Conditional Correlation model

The Dynamic Conditional Correlation (DCC) model (Christodoulakis and Satchell, 2002; Engle, 2002; Tse and Tsui, 2002) is an extension of the Constant Conditional Correlation (CCC) model of Bollerslev (1990). The CCC model assumes that the conditional corre-lations are constant over time; the DCC model allows the conditional correcorre-lations to be time-varying.

The assumption of the CCC model that the conditional correlations are constant may seem unrealistic in many empirical applications (Bauwens et al., 2006). Therefore, the DCC model is used in this research.

The covariance matrix of the DCC model at time t, ˆΣDCC ,t, is calculated using equation (2.27). In this equation Dt is a diagonal matrix of standard deviations, i.e. Dt = diag(σ1t, σ2t, . . . , σN t), and Ωt is the correlation matrix at time t.

ˆ

ΣDCC ,t= DtΩtDt (2.27)

The standard deviations of matrix Dt are calculated using the GARCH(1,1) model, as shown in equation (2.28).

The correlation matrix Ωt is calculated using equation (2.29), with Qt as defined in equation (2.30) and Q∗_t being a diagonal matrix with the square root of the elements of Qt at time t, i.e. Q∗t =pdiag(Qt)

Ωt= Q∗−1t QtQ∗−1t (2.29)

Qt = (1 − a − b) ¯Q + aeTt−1et−1+ bQt−1 (2.30) ¯

Q of equation (2.30) is the unconditional covariance matrix of standardized errors, as shown in equation (2.31). The standardized errors et of equation (2.30) and equa-tion (2.31) are defined as et = D−1t εTt.

¯ Q = E(eT_tet) = 1 N N X i=1 (eT_tet) (2.31)

According to Engle (2002) a should be a small positive number and b functionally equals λ of the Exponentially Weighted Moving Average of section 2.8.1. The parameters a and b must satisfy a + b < 1 (Billio and Caporin, 2006). Therefore, in this research, b is set equal to λ of section 2.8.1, i.e. b = 0.97.

Tests have been conducted to determine the value of parameter a. These tests have been conducted in accordance with section 3.2, using the data described in chapter 4. The outcomes of these test are shown in table B.1 and table B.2.

Although the outcomes of the means and standard deviations of the correlations of the DCC and of the absolute forecast errors of the DCC do change per period, on average, a = 0.025 has the lowest mean and standard deviation in terms of absolute forecast errors (table B.2). Therefore, in this research, a = 0.025 will be used.

3. Methodology

Two types of empirical tests are applied to judge the superiority of the techniques of table 3.1 over each other in terms of their ability to forecast. These are the same tests as used by Elton et al. (1978). The first empirical test judges the differences in absolute forecast errors between each pair of techniques, as described in section 3.2. The second empirical test judges the distribution of the differences in absolute forecast errors, as described in section 3.3. The first empirical test with adjustments by the mean correlation coefficient is described in section 3.4.

Table 3.1: Techniques used

Model Abbreviation

Historical model Historical

Single-index model, with unadjusted betas Unadjusted Single-index model, with Vasicek’s adjustment Vasicek Single-index model, with Blume’s adjustment Blume Single-index model, with assumed beta Beta = 1

Constant correlation model Constant

Shrinkage, with unadjusted betas as shrinkage target Shrink-SF Shrinkage, with constant correlation as shrinkage target Shrink-CC Exponentially Weighted Moving Average model EWMA

Dynamic Conditional Correlatioin model DCC

Recall from chapter 2 that all techniques require one historical period and one forecast period (except for Blume’s adjustment to the single-index model of section 2.3.3, which requires two historical periods and one forecast period). Like Elton et al. (1978), this research uses 5-year, non-overlapping, consecutive periods as historical periods and as forecast period. The historical periods and the forecast periods used in this research, with their period name, are shown in table 4.1.

3.1

Absolute forecast errors

standard deviations of the correlation matrices, for all five periods, are shown in table B.3. Note that the diagonal of the correlation matrix is excluded when calculating the mean and standard deviation.

Ω =          ρ11 ρ12 ρ13 · · · ρ1n ρ21 ρ22 ρ23 · · · ρ2n ρ31 ρ32 ρ33 · · · ρ3n .. . ... ... . .. ... ρn1 ρn2 ρn3 · · · ρnn          =          1 ρ12 ρ13 · · · ρ1n ρ21 1 ρ23 · · · ρ2n ρ31 ρ32 1 · · · ρ3n .. . ... ... . .. ... ρn1 ρn2 ρn3 · · · 1          (3.1)

For these calculated correlation matrices (both the forecast correlation matrix and the realized correlation matrix for the same period), the absolute forecast error matrix Ωε is calculated for each technique, as shown in equation (3.2). The means and standard deviations of the absolute forecast errors, for all five periods, are shown in table B.4. Note that the diagonal of the correlation matrix is excluded when calculating the mean and standard deviation.

Ωε = |Ωrealized− Ωf orecast| (3.2)

For the absolute forecast error matrices Ωε, the distribution can be calculated. These distributions are calculated using the central limit theorem (Montgomery and Runger, 2007). In the central limit theorem, the sampling distribution of sample means of the absolute forecast errors and of the differences in absolute forecast errors will be approxi-mately normally distributed with mean µ and standard deviation √σ

n, if the sample size n is large. By applying the central limit theorem to the correlation matrix, normal statis-tics can be applied, even though the correlation matrices themselves do not have to be normally distributed.

Yamane (1967); Israel (2009) provides a simplified formula (compared to the formula of Cochran (1963)) to calculate the sample size n needed, which is shown in equation (3.3). In this formula n is the sample size, N is the population size, and e is the level of precision.

n = N

1 + N e2 (3.3)

Using 49 industry portfolios as security returns (see section 4.1) and using a signifi-cance level of 5%, the sample size in this case should be at least 343 according to Yamane’s formula, as shown in equation (3.4).

n = 49

2

1 + 492_{· 0.05}2 ≈ 343 (3.4)

In this research the sample size will be rounded to 400. The sample size is the size of the randomly picked samples from the correlation matrix, which contains 492 _{= 2401} el-ements. The amount of samples, i.e. the amount of times the aforementioned sample size will be picked from the correlation matrix, will be set to 400 as well. All in all, a sample size of 400 randomly picked correlation coefficients will be picked from the correlation matrix for 400 times.

The analysis has been applied using both the means and standard deviations of the absolute forecast without the central limit theorem applied and using the means and standard deviations with the central limit theorem applied. The results with and without the central limit theorem applied show hardly any differences. Thus the conclusions would also be identical. Therefore the means and standard deviations of the absolute forecast errors for all five periods, after having applied the central limit theorem according to the aforementioned settings, are not included in the appendices, but are available upon request.

3.2

Differences in absolute forecast errors

The first test examines the differences in absolute forecast errors between each pair of different techniques. This test is performed as follows.

Suppose there are two techniques: technique A and technique B. Technique A is said to dominate technique B if the difference in means, E(Ωε,A) − E(Ωε,B), is negative and statistically significantly different from zero at the 5% level based on a z-test.

Thus, the hypothesis for the first test can be written as:

H0 : E(Ωε,A) − E(Ωε,B) = 0 H1 : E(Ωε,A) − E(Ωε,B) < 0

(3.5)

3.3

Distribution of differences

The second test examines the differences in the distribution of absolute forecast errors between techniques to see if the probability of any size error was less with one technique than with a second. I.e. the first order stochastic dominance in terms of absolute forecast errors between techniques is examined.

Suppose again that there are two techniques: technique A and technique B, and that these techniques have probability density functions f (x) and g(x) and cumulative distribution functions F (x) and G(x), respectively.

Technique A is said to dominate technique B in the first order stochastic dominance if F %F SDG (Wolfstetter, 1999), i.e. F (z) has a lower cumulative frequency function than G(z). This equation can also be written as shown in equation (3.6).

F %F SDG = F (z) ≤ G(z) = Z z −∞ f (t)dt ≤ Z z −∞ g(t)dt (3.6)

Many methods exist for conducting statistical inference for the aforementioned tests, e.g. multiplier methods of Hansen (1996) and bootstrap methods of Maasoumi and Hesh-mati (2000); Abadie (2002). In this research tests based on multiple comparisons are used (Anderson, 1996; Davidson and Duclos, 2000). Although, according to Barrett and Donald (2003) the tests based on multiple comparisons introduces the possibility of inconsistency because of the fact that the comparisons are made at a fixed number of arbitrarily chosen points, these tests are used because of their simplicity.

Because, in this research, the technique that has a higher cumulative frequency func-tion (and thus lower error sizes) than another technique, the technique that is dominated in the first order stochastic dominance, outperforms this other technique.

The hypothesis that is being tested, with ∆1(zl) = G(z) − F (z), is:

H0 : ∆1(zl) ≤ 0 for all l ∈ P H1 : ∆1(zl) > 0 for some l ∈ P

(3.7)

In this hypothesis, P is defined as P = {0.20, 0.18, . . . 0.02}, i.e. 10 intervals of 0.02 against which the distributions of the absolute forecast errors are being tested.

3.4

Adjustment by the mean correlation coefficient

It is useful to examine the performance of the techniques with the effect of bias in their mean forecast removed, because all techniques are likely to produce estimates of the future correlations that tend to be above or below the average correlation of the historical period. When making adjustments by the mean correlation coefficient, the forecasting per-formance can be examined when the forecast of the mean correlation coefficient is done exogenously. Unlike Elton et al. (1978), who consider two types of adjustments (the historical adjustment and the future adjustment), in this research only the historical ad-justment is applied. The future adad-justment has no practical value, since it assumes perfect

knowledge of the correlation coefficients.

The historical adjustments on the other hand assumes that the average correlation coefficient follows a zero drift random walk, i.e. the unbiased forecast will be the last observed average correlation coefficient.

The correlation matrices with historical adjustment are calculated as shown in equa-tion (3.8). In this equaequa-tion Ωhistorical adjustment,tis a correlation matrix of a technique with historical adjustment at time t. It is calculated by subtracting the mean of the correla-tion matrix of that technique, E(Ωf orecast,t), from the forecast correlation matrix of that technique, Ωf orecast,t, and adding the new estimate of the mean, which is the mean of the correlation matrix of the historical technique at time t, E(Ωhistorical,t).

4. Data

In this chapter the data regarding the security returns Ri will be discussed first in sec-tion 4.1. Then, in secsec-tion 4.2, the market returns Rm will be discussed. And finally, some general descriptive statistics of the periods being used will be discussed in section 4.3.

4.1

Security returns

Industry sector benchmarks are used as security returns instead of individual securities. Therefore there will be spoken of sectors instead of securities hereafter. Reason for this is that performing the analysis on sector benchmarks, implies incurring less noise in the data set than performing the analysis on individual securities. When dealing with sector benchmarks, there’s no chance that one or more of the sectors will seize to exist, while there is a chance that individual securities will seize to exist, when dealing with individual securities.

Another benefit of using sector benchmarks as security returns instead of individual securities, is that different introduction dates of the securities do not have to be dealt with.

In this research the sector benchmarks of French (2014) are used. More specifically, the data set consisting of 49 industry portfolios based on average equal weighted returns is used. This data set contains the monthly continuously compounded returns on 49 in-dustry portfolios from July 1926 to December 2013. The inin-dustry portfolios are shown in table A.1.

4.2

Market index

The MSCI World Index1 _{is used as the market index, which includes securities from 23} countries and measures the value weighted performance of the global developed equity markets. The total returns of the MSCI World Index are fetched from Thomson Reuters’ Datastream2_{; it goes by the index mnemonic MSWRLD$ (Thomson Financial, 2005).}

The monthly total return index of the MSCI World Index is available on Datastream from December 1969 to present. Continuously compounded returns are used to calculate the returns rm,t, as shown in equation (4.1), in which pt is the return index at time t.

1

http://www.msci.com/

2

http://thomsonreuters.com/datastream-professional/

rm,t = 100% · ln  pm,t pm,t−1  (4.1)

4.3

Periods

Table 4.1 shows the five periods used in this research with their historical periods, their forecast period, and the means and standard deviations of the historical periods and the forecast period for both the market index and the securities. Note that the diagonal of the correlation matrix is excluded when calculating the mean and standard deviation. Figure 4.1 and figure 4.2 show the monthly continuously compounded returns of the market index and industry sectors respectively. Each of the five periods is described below.

4.3.1

First five years (forecast period 1985 – 1989)

The means of returns of the market index of the two historical periods of the first five years (1975 – 1979 and 1980 – 1984) are average compared to the other periods. In terms of the volatility, the historical periods of the market index belong to the least volatile periods of the market index compared to the other periods. The average returns of the historical periods of the industry sectors however, belong to highest average returns compared to the other periods. In terms of volatility, the historical periods of the securities are average compared to the other periods.

Interesting to see is that the historical periods of the sectors show some peaks and troughs which are hardly noticeable in the market index. This is the case with the peak between 1975 and 1977, the trough between 1977 and 1980 and the peak between 1982 and 1985. This could be explained by the fact that the industry sectors consist sheerly of U.S. companies, whereas the market index is the MSCI World Index, i.e. the peaks and troughs are only noticeable in the U.S. market. The troughs before and after January 1980 are noticeable in both the market index and the securities.

4.3.2

Second five years (forecast period 1990 – 1994)

The first historical period of the second five years (1980 – 1984) is the same as the second historical period of the first five years. The second historical period of the second five years (1985 – 1989) is the same as the forecast period of the first five years.

The returns of the forecast period of the second five years (1990 – 1994) have a low mean compared to the other periods in both the market index and in the sectors. In terms of volatility, the returns of the forecast period are average in the market index, but have the lowest volatility in the sectors.

In the forecast period there is a trough noticeable after January 1990 in both the market index and the sectors. There seems to be a peak noticeable just after the trough and before June 1992 in the sectors. These peaks are less noticeable in the market index. This could well be the early 1990s recession, which was a worldwide recession. This event will most likely result in higher correlations.

4.3.3

Third five years (forecast period 1995 – 1999)

The first historical period of the third five years (1985 – 1989) is the same as the second historical period of the second five years. The second historical period of the third five years (1990 – 1994) is the same as the forecast period of the second five years.

The forecast period of the third five years (1995 – 1999) has the second highest average return in the market index. The mean of the average returns of the sectors is average compared to other periods. In terms of volatility, the average returns of the sectors are the second lowest of all periods. Same is true for the volatility of the forecast period in the market index.

A trough is clearly noticeable in both the market index and the industry sectors between 1997 and 2000. This is the period of the Internet bubble burst, which had its climax on March 10, 2000. This event too might increase the correlation, in particular the IT-related industry sectors.

A peak, followed by a trough, seems to be noticeable between 1995 and 1997 in the sectors, while it is not the case in the market index. An explanation for this could again be that the sectors are composed of U.S. companies only.

4.3.4

Fourth five years (forecast period 2000 – 2004)

The first historical period of the fourth five years (1990 – 1994) is the same as the second historical period of the third five years. The second historical period of the fourth five years (1995 – 1999) is the same as the forecast period of the third five years.

The forecast period of the fourth five years (2000 – 2004) is the only period that has negative average return in the market index, and is therefore the period that the lowest

average return in the market index compared to the other periods. The average of the returns of the sectors performs better. It has the third highest average of all periods in the sectors.

Noticeable is, especially when looking at the returns of some particular sectors, that the returns are highly volatile. The returns of both the market index and the sectors are the second highest in terms of volatility compared to the other periods. This might be ex-plained by the early 2000s recession, which would most likely result in lower correlations, because of the volatility.

4.3.5

Fifth five years (forecast period 2005 – 2009)

The first historical period of the fifth five years (1995 – 1999) is the same as the second historical period of the fourth five years. The second historical period of the third five years (2000 – 2004) is the same as the forecast period of the fourth five years.

The forecast period of the fifth five years (2005 – 2009) is the most volatile in both the market index and in the sectors compared to the other periods. In terms of the average return, the forecast period performs as second lowest in the market index and as lowest in the securities.

Table 4.1: Periods used with means and standard deviations of monthly continuously compounded returns3

Period name Historical period Forecast period Market index Industry sectors

Historical 1 Historical 2 Forecast Historical 1 Historical 2 Forecast

First five years Jan 1980 – Dec 1984 Jan 1985 – Dec 1989 0.0123 0.0097 0.0206 0.0321 0.0179 0.0093

(Jan 1975 – Dec 1979) (0.0362) (0.0400) (0.0445) (0.0719) (0.0660) (0.0642)

Second five years Jan 1985 – Dec 1989 Jan 1990 – Dec 1994 0.0097 0.0206 0.0035 0.0179 0.0093 0.0098

(Jan 1980 – Dec 1984) (0.0400) (0.0455) (0.0428) (0.0660) (0.0642) (0.0579)

Third five years Jan 1990 – Dec 1994 Jan 1995 – Dec 1999 0.0206 0.0035 0.0154 0.0093 0.0098 0.0127

(Jan 1985 – Dec 1989) (0.0455) (0.0428) (0.0375) (0.0642) (0.0579) (0.0617)

Fourth five years Jan 1995 – Dec 1999 Jan 2000 – Dec 2004 0.0035 0.0154 -0.0017 0.0098 0.0127 0.0159

(Jan 1990 – Dec 1994) (0.0428) (0.0375) (0.0455) (0.0579) (0.0617) (0.0790)

Fifth five years Jan 2000 – Dec 2004 Jan 2005 – Dec 2009 0.0154 -0.0017 0.0021 0.0127 0.0159 0.0048

(Jan 1995 – Dec 1999) (0.0375) (0.0455) (0.0524) (0.0617) (0.0790) (0.0826)

3_{Historical periods between brackets denote the first historical period used for Blume’s adjustment of the single-index model. The second historical period used}

for Blume’s adjustment is equal to the historical period used for the other techniques. For the sectors, the unweighted average mean of all sectors and the average standard deviation of all sectors is shown.

Time t (mm/yyyy) M on th ly re tu rn s rm t 1/1975 6/1977 1/1980 6/1982 1/1985 6/1987 1/1990 6/1992 1/1995 6/1997 1/2000 6/2002 1/2005 6/2007 12/2009 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

Figure 4.1: Monthly continuously compounded market returns

Time t (mm/yyyy) M on th ly re tu rn s ri t 1/1975 6/1977 1/1980 6/1982 1/1985 6/1987 1/1990 6/1992 1/1995 6/1997 1/2000 6/2002 1/2005 6/2007 12/2009 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

Figure 4.2: Monthly continuously compounded sector returns

5. Results

In this chapter, the results of the unadjusted forecast are discussed first in section 5.1. Then, in section 5.2, the results of the adjustment by the mean correlation coefficient in accordance with the methodology of section 3.4 are discussed.

5.1

Unadjusted forecasts

The results of the unadjusted forecasts consist of the differences in absolute forecast errors in accordance with the methodology of section 3.2 (section 5.1.1) and the distribution of differences in accordance with the methodology of section 3.3 (section 5.1.2).

Table 5.1 shows the average correlation coefficient of the ten techniques and the av-erage realized correlation coefficient over the five periods. Note that the techniques His-torical, Constant and Shrink-CC give the same estimates, which are equal to the true mean of the previous period. The realized average correlation coefficient over the first five years (forecast period 1985 – 1989) and over the fifth five years (forecast period 2005 – 2009) are the highest, whereas the average correlation coefficient over the fourth five years (forecast period 2000 – 2004) is the lowest. These results seem to be in line with the periods of section 4.3. The most prominent crashes, i.e. Black Monday and the Global Financial Crisis, result in higher average correlations in the first and fifth five years re-spectively. The fourth five years (in which the early 2000s recession occurred) show a lower average correlation, as expected, because of the increased volatility. The impact of early 1990s recession and the Internet bubble burst (which occurred in the second and third five years) on the average correlation is low relative to the aforementioned more prominent crashes.

5.1.1

Differences in absolute forecast errors

Table 5.2 shows the average absolute forecast errors without adjustments for the five periods. There is not one technique that performs best during all five periods.

Interesting to see however, are the differences between the average absolute forecast errors between the techniques. As can be seen from the column “Combined” is that the differences between the five best performing techniques (Shrink-SF, EWMA, DCC, Historical and Shrink-CC) are relatively small. Also the differences between the four worst performing techniques (Beta = 1, Blume, Unadjusted and Vasicek) are relatively small, albeit that these differences are greater than the five best performing techniques.

Table 5.1: Forecast of average correlation coefficient

First five years Second five years Third five years

Forecast period 1985 – 1989 Forecast period 1990 – 1994 Forecast period 1995 – 1999

Historical 0.7637 Historical 0.7562 Beta = 1 0.6054

Constant 0.7637 Constant 0.7562 Historical 0.6053

Shrink-CC 0.7637 Shrink-CC 0.7562 Constant 0.6053

DCC 0.7544 DCC 0.7513 Shrink-CC 0.6053

EWMA 0.7456 EWMA 0.7509 DCC 0.5862

Shrink-SF 0.7394 Shrink-SF 0.6624 Shrink-SF 0.5755

Unadjusted 0.5049 Beta = 1 0.5152 EWMA 0.5637

Vasicek 0.4978 Vasicek 0.3883 Vasicek 0.1244

Blume 0.4862 Unadjusted 0.3872 Unadjusted 0.1236

Beta = 1 0.4367 Blume 0.3694 Blume 0.0731

Realization 0.7562 Realization 0.6053 Realization 0.6067

Fourth five years Fifth five years

Forecast period 2000 – 2004 Forecast period 2005 – 2009

Blume 0.6291 EWMA 0.5774 DCC 0.6103 DCC 0.5631 Historical 0.6067 Historical 0.5544 Constant 0.6067 Constant 0.5544 Shrink-CC 0.6067 Shrink-CC 0.5544 EWMA 0.5995 Blume 0.5451 Shrink-SF 0.5516 Shrink-SF 0.5125 Beta = 1 0.4293 Beta = 1 0.4382 Unadjusted 0.2558 Unadjusted 0.3441 Vasicek 0.2557 Vasicek 0.3421 Realization 0.5544 Realization 0.7209

The Constant technique seems to be somewhat in the middle of five best performing techniques and the four worst performing techniques.

Similar results, i.e. categorizing the used techniques into the five best performing techniques (Shrink-SF, EWMA, DCC, Historical and Shrink-CC) and the four worst performing techniques (Beta = 1, Blume, Unadjusted, Vasicek), can be derived when ex-amining the robustness of the relative performance (see table C.3, column “Combined”). The robustness is examined by applying the methodology of section 3.2, i.e. the differ-ences in absolute forecast errors, to two non-overlapping subsamples of industries. The subsamples used in this research are shown in table A.1.

Table 5.2: Average absolute forecast errors (no adjustment)4

First five years Second five years Third five years

Forecast period 1985 – 1989 Forecast period 1990 – 1994 Forecast period 1995 – 1999

1. 

Historical 0.0670 1. Shrink-SF 0.1026 1. Shrink-CC 0.0960

2. DCC 0.0685 2.  EWMA 0.1523 2.    Shrink-SF 0.1019 3. EWMA 0.0694 3. DCC 0.1540 3. Historical 0.1032 4. Shrink-SF 0.0746 4. Historical 0.1570 4. DCC 0.1047 5. Shrink-CC 0.0806 5. Shrink-CC 0.1710 5. EWMA 0.1090 6. Constant 0.1312 6. Constant 0.1910 6. Constant 0.1660 7. Unadjusted 0.2638 7. Beta = 1 0.2205 7. Beta = 1 0.2165 8. Vasicek 0.2667 8. Unadjusted 0.2479 8. Vasicek 0.4854 9. Blume 0.2923 9. Vasicek 0.2575 9. Unadjusted 0.4858

10. Beta = 1 0.3495 10. Blume 0.2852 10. Blume 0.5348

Fourth five years Fifth five years Combined

Forecast period 2000 – 2004 Forecast period 2005 – 2009

1. Shrink-SF 0.1021 1. EWMA 0.1550 1. Shrink-SF 0.1185 2.



Historical 0.1093 2. DCC 0.1675 2. EWMA 0.1196

3. EWMA 0.1121 3. Shrink-CC 0.1722 3. DCC 0.1219

4. DCC 0.1151 4. Historical 0.1773 4. Historical 0.1227 5. Shrink-CC 0.1224 5. Constant 0.2019 5. Shrink-CC 0.1285 6. Blume 0.1462 6. Shrink-SF 0.2115 6. Constant 0.1772

7. Constant 0.1961 7. Blume 0.2210 7. Beta = 1 0.2711

8. Beta = 1 0.2236 8. Beta = 1 0.3457 8. Blume 0.2959

9. Unadjusted 0.3046 9. Unadjusted 0.3771 9. Unadjusted 0.3358 10. Vasicek 0.3064 10. Vasicek 0.3791 10. Vasicek 0.3390

5.1.1.1 Four worst performing techniques

The four worst performing techniques consist of the techniques Beta = 1, Blume, Unad-justed and Vasicek. When looking at the four worst performing techniques, it is worth mentioning that these techniques are all variants of single-index models. The character-istic of single-index models, i.e. that they only take into account the correlation that is associated with the market index and ignore all other sources of correlation, might be the reason why the these techniques perform badly over all five periods.

In order to compare the techniques separately, the Unadjusted technique is taken as the reference, since it uses unadjusted betas.

4_{All differences between the average absolute forecast errors are statistically significant at the 5% level,}

unless grouped by bracket. The column “Combined” shows the unweighted average of the average absolute forecast errors. For the probabilities and the z-statistics of the differences in absolute forecast errors, table C.1.

As can be seen in table 5.2, the Unadjusted technique and the Vasicek technique are statistically insignificant from each other in 4 out of 5 periods (the errors of these techniques are statistically significant from each in the second five years). The errors of the Unadjusted technique and the Vasicek technique are statistically insignificant 7 out of 10 times, when looking at the subsamples of table C.3. Again, the errors of these techniques are statistically significant from each other in both subsamples in the second five years. One might say that Vasicek’s adjustment is not an improvement over using unadjusted betas.

With the Vasicek technique, there is an adjustment of all betas towards the mean, as can be seen from standard deviations of table 5.3. This decrease in standard deviation relative to the historical realization results in higher forecast correlation, i.e. reducing the difference between the estimated average correlation coefficient and the realized average correlation coefficient, when the mean of the betas would stay the same. But in practice the mean of the betas is most likely to change over time, and therefore the Vasicek adjustment does not necessarily improve the results.

Table 5.3: Means and standard deviations of betas5

Period name Historical betas Estimated betas Realized

Historical 1 Historical 2 Vasicek Blume

First five years 1.2753 1.1371 1.1105 1.0804 0.8809

Forecast: 1985 – 1989 (0.2435) (0.2626) (0.1811) (0.1078) (0.1265)

Second five years 1.1371 0.8809 0.8770 0.8485 0.4592

Forecast: 1990 – 1994 (0.2626) (0.1265) (0.0657) (0.0160) (0.1071)

Third five years 0.8809 0.4592 0.4570 0.3493 0.8106

Forecast: 1995 – 1999 (0.1265) (0.1071) (0.0311) (0.0279) (0.2364)

Fourth five years 0.4592 0.8106 0.7984 1.2597 0.9918

Forecast: 2000 – 2004 (0.1071) (0.2364) (0.1414) (0.3022) (0.4545)

Fifth five years 0.8106 0.9918 0.9745 1.2900 1.2163

Forecast: 2005 – 2009 (0.2364) (0.4545) (0.3461) (0.7482) (0.3287)

The Blume technique is the only technique that uses two historical periods in order to estimate the betas. There are two periods in which the Blume technique outperforms the Unadjusted technique: in the fourth five years and in the fifth five years.

There are two possible reason why the Blume technique performs better in these two periods. The first reason is the same why the Vasicek technique could produce higher estimates, i.e. the adjustment of all betas towards the mean. The second reason is the extrapolation of the rise in the betas over the two historical periods, making it produce

5_{Note that de estimated betas for the Unadjusted technique and the Beta = 1 technique are not explicitly}

higher estimates and reducing the difference between the estimated average correlation coefficient and the realized average correlation coefficient. As can be seen from table 5.3, there is indeed a rise in betas over the two historical and thus an extrapolation of this rise with the Blume technique.

The Beta = 1 technique does not outperform the Unadjusted technique only in the first five years. In all other period, the Beta = 1 technique outperforms the Unadjusted technique (and the Vasicek technique), and it is only outperformed in the fourth and fifth five years by the Blume technique. The only reason why the Beta = 1 technique could produce lower absolute average forecast errors is the same as is the case with the Vasicek technique: the adjustment of all towards a mean, which, in case of the Beta = 1 technique, equals 1.

In the first five years the techniques Unajusted, Vasicek and Blume all produce an estimated beta slightly higher than one, while the realized beta is smaller than one. The fact that the Beta = 1 technique has a standard deviation of zero, could be the reason that the Beta = 1 technique performs worse, even though its mean beta is closer to the realized mean beta.

The realized beta in the second five years is relatively low. All single-index models produce higher estimated betas, with relatively low standard deviations. Nevertheless, the Beta = 1 technique performs better, albeit marginally, than the other single-index models. Reason for this might be that the average correlation coefficient of the Beta = 1 technique is closer to the realized average correlation coefficient, as can be seen in table 5.1.

In the third five years, where the techniques Unadjusted, Vasicek and Blume pro-duce low estimated betas, the average realized beta equals 0.8106. In this period the Beta = 1 technique clearly performs better than the other single-index models, because of its assumed beta of 1.

It is in the fourth and fifth five years that the Blume technique produces an estimated beta higher than 1 with relative high standard deviations. The realized betas in these periods are slightly lower and higher than 1 respectively, while having relatively high standard deviations as well. This might explain why the Blume technique outperforms the Beta = 1 technique in these two periods.

5.1.1.2 Constant technique

The Constant technique is the sixth best technique during the first, the second and the third five years. During the fourth five years it’s the seventh best technique (the Blume technique performs better), and during the fifth five years it’s the fifth best technique (the Shrink-SF technique performs worse).

As to why, in general, the Constant technique performs better than the single-index models and worse than the five best performing techniques, there are two things to say.

Firstly, as can be seen from table 5.1, the Constant technique has the same average forecast correlation coefficient as the Historical technique (and the Shrink-CC technique). Only difference is that all correlation coefficients of the Constant technique are equal, whereas those of the Historical and Shrink-CC technique are not. This may be the reason why the Constant technique is not performing as well as the Historical technique.

Secondly, contrary to the single-index models, the Constant technique is not based on the correlation associated with the market index. Since the single-index models perform badly in the samples under study, and the other techniques perform better, it is not surprising that the Constant model performs better than the single-index models.

5.1.1.3 Five best performing techniques

The five best performing techniques consist of the techniques Shrink-SF, EWMA, DCC, Historical and Shrink-CC. When looking at the five best performing techniques, it can be seen in table 5.2 that the errors of the techniques EWMA, DCC and Historical are statistically insignificant from each other in 4 out of 5 periods. This is only the case 4 out of 10 times when looking at the subsamples of table C.3. Note however, that this does not include the case in which either the errors of DCC and Historical or of EWMA and DCC are insignificant, which occurs 3 out of 10 times.

In order to compare the techniques separately, the Historical technique is taken as the reference, since its correlation matrix is based on the average volatility of the security returns.

The EWMA technique, instead of being based on the average volatility, assigns more weight to the recent observations. The only period in which the EWMA technique statis-tically outperforms the Historical technique is in the fifth five years. The fifth five years (in which the Global Financial Crisis occurred) are characterised by a substantial decline in the mean return of the securities and an increase in the volatility of the returns of the securities over the forecast period compared to the second historical period.

In the second five years the opposite is the case; the volatility of the securities declines and the mean return increases. In this period the EWMA technique does have a lower average absolute forecast error than the Historical technique as well, but not statistically significant.

The DCC technique is similar to the EWMA technique, except that it includes a term which makes it persistent to a long-term average, i.e. the term for mean reversion. Like the EWMA technique, the DCC technique statistically outperforms the Historical technique in the fifth five years. It also has a lower average absolute forecast error than the Historical technique in the second five years, but not statistically significant.

the securities in the forecast period of the fifth five years, on which the persistence to a long-term average has a negative effect.

As can be seen from table 5.2, the performance of the Shrink-SF technique and the Shrink-CC technique seems to change a lot depending on the period. During the second and the fourth five years the Shrink-SF technique performs best. During the third five years the Shrink-CC technique performs best. As can be seen from table 5.1, the forecast of the average correlation coefficient of the Shrink-SF technique is closer to the realized average correlation coefficient (hence a smaller average absolute forecast error), than during the other periods.

But the forecast of the average correlation coefficient is not the only factor that influences the performance of the Shrink-SF and Shrink-CC techniques. The standard deviation of the correlations also is a factor that influences the average absolute forecast error. Recall that the Shrink-CC technique has equals means as, but lower standard deviations than the Historical technique over all five periods. It is in the third and fifth five years where the Shrink-CC technique statistically outperforms the Historical technique. As can be seen in table B.3, the standard deviation of the Shrink-CC technique is slightly lower than the realized standard deviation in the third five years (the standard deviation of the Historical technique is higher than the realized standard deviation), and in the fifth five years the standard deviation of the Shrink-CC technique is slightly higher than the realized standard deviation (the Historical technique has an even higher standard deviation than the Shrink-CC technique).

5.1.2

Distribution of differences

Table 5.4 shows the dominance of the techniques in terms of error size. The techniques are ranked from worst to best for every period based on the ranking of table 5.2. One would expect that a technique that has a (statistically) lower average absolute forecast error than another technique, would also have smaller errors. While this is true for most techniques, it is certainly not true for all techniques. The noteworthy results of table 5.4 are discussed below.

5.1.2.1 First five years (forecast period 1985 – 1989)

As can be seen from subtable 5.4a the Blume technique has smaller errors in all ten categories of error sizes than the Beta = 1 technique, whereas the techniques Vasicek and Unadjusted (which have a statistically lower average absolute forecast error than the Blume technique) only have smaller errors in 7 and 5 out of 10 categories respectively.

While the average absolute forecast errors of the techniques Historical, DCC and EWMA are statistically insignificant from each other, the Historical technique performs better than the techniques EWMA and DCC when comparing these techniques with the

Shrink-CC technique. When comparing these techniques with the Shrink-SF technique, the technique EWMA and DCC are performing better than the Historical technique.

In the first five years the average absolute forecast errors of the Unadjusted technique and the Vasicek technique are statistically insignificant. The Unadjusted technique has smaller errors than the Vasicek technique in 0 out of 10 categories. This implies that the differences in error sizes between these techniques are ≤ 0.02.

5.1.2.2 Second five years (forecast period 1990 – 1994)

As can be seen from subtable 5.4b, the Shrink-SF is also the best technique in terms of the error size over the second five years.

The Shrink-CC technique performs better than the techniques Historical, DCC and EWMA when comparing these techniques with the Constant technique, since the Shrink-CC has smaller error in all ten categories, while the techniques Historical, DShrink-CC and EWMA only have so in 6, 7, and 6 categories respectively.

5.1.2.3 Third five years (forecast period 1995 – 1999)

Even though the Shrink-CC technique has a statistically lower average absolute forecast error than the other techniques over the third five years, the Shrink-CC technique does not have smaller errors than the other techniques in all categories, as can be seen from subtable 5.4c.

While the average forecast errors of the techniques Unadjusted and Vasicek are sta-tistically insignificant from each other, the Vasicek technique has smaller errors in all ten categories than the Blume technique, whereas the Unadjusted technique only has smaller errors in 8 categories.

The techniques EWMA and DCC have smaller errors than the Beta = 1 technique in all categories, whereas the techniques Historical, Shrink-SF and Shrink-CC have smaller errors in 9 out of 10 categories. The average absolute forecast errors of the techniques Shrink-SF, Historical, DCC and EWMA are statistically insignificant.

5.1.2.4 Fourth five years (forecast period 2000 – 2004)

As can be seen from subtable 5.4d, the Shrink-SF technique, which has statistically the lowest average absolute forecast error over the fourth five years, does not have smaller errors than the other techniques in all categories. It has smaller errors than the techniques DCC and Historical in 8 out of 10 categories.

In this period the differences in size errors between the Unadjusted technique and the Vasicek technique are smaller than 0.02, as is the case in the first five years.

5.1.2.5 Fifth five years (forecast period 2005 – 2009)

As can be seen from subtable 5.4e, the DCC technique has smaller errors than the other techniques in all ten categories.

The Constant technique only has smaller errors than the Beta = 1 technique in 8 categories. The techniques Blume and Shrink-SF however, have smaller errors than the Beta = 1 technique in all categories, while both of these techniques have a statistically higher average absolute forecast error.

Where the techniques Shrink-CC and DCC have smaller errors than the Blume tech-nique in respectively 6 and 9 categories, the Historical techtech-nique has smaller errors than the Blume technique in all categories, despite having a statistically higher average abso-lute forecast error than the techniques Shrink-CC and DCC.

In this period the differences in size errors between the Unadjusted technique and the Vasicek technique are smaller than 0.02, as is the case in the first five years and the fourth five years.

Table 5.4: Dominance by cumulative frequency function (no adjustment)6

(a) First five years (forecast period 1985 – 1989)

Beta = 1 Blume Vasicek Unadjusted Constant Shrink-CC Shrink-SF EWMA DCC Historical

Beta = 1 – Blume 10 – Vasicek 7 5 – Unadjusted 5 3 0 – Constant 10 10 10 10 – Shrink-CC 10 10 10 10 10 – Shrink-SF 10 10 10 10 10 8 – EWMA 10 10 10 10 10 9 10 – DCC 10 10 10 10 10 9 10 8 – Historical 10 10 10 10 10 10 6 4 3 –

(b) Second five years (forecast period 1990 – 1994)

Blume Vasicek Unadjusted Beta = 1 Constant Shrink-CC Historical DCC EWMA Shrink-SF

(c) Third five years (forecast period 1995 – 1999)

Blume Unadjusted Vasicek Beta = 1 Constant EWMA DCC Historical Shrink-SF Shrink-CC

Blume – Unadjusted 8 – Vasicek 10 10 – Beta = 1 10 10 10 – Constant 10 10 10 6 – EWMA 10 10 10 10 10 – DCC 10 10 10 10 10 8 – Historical 10 10 10 9 10 7 6 – Shrink-SF 10 10 10 9 10 7 7 10 – Shrink-CC 10 10 10 9 10 8 8 10 9 –

(d) Fourth five years (forecast period 2000 – 2004)

Vasicek Unadjusted Beta = 1 Constant Blume Shrink-CC DCC EWMA Historical Shrink-SF

(e) Fifth five years (forecast period 2005 – 2009)

Vasicek Unadjusted Beta = 1 Blume Shrink-SF Constant Historical Shrink-CC DCC EWMA

Vasicek – Unadjusted 0 – Beta = 1 10 10 – Blume 10 10 10 – Shrink-SF 10 10 10 1 – Constant 10 10 8 2 3 – Historical 10 10 10 10 10 10 – Shrink-CC 10 10 10 6 10 10 3 – DCC 10 10 10 9 10 10 7 10 – EWMA 10 10 10 10 10 10 10 10 10 –

6_{The numbers in the table indicates the number of times the technique on the left had a larger cumulative frequency function (and hence smaller errors) than}

the technique on the top based on the interval of P of section 3.3. Note that the techniques are ranked based on their average absolute forecast error of table 5.2 for each period, with the worst performing technique on the top and the best performing technique on the bottom. The numbers are derived from table D.1.

5.2

Adjustment by the mean correlation coefficient

Table 5.5 shows the average absolute forecast errors when the historical adjustment is made, as described in section 3.4. Note that the average absolute forecast errors of the techniques Historical, Constant and Shrink-CC remain unchanged, since they all had the same forecast of the average correlation coefficient (see table 5.1). The first thing that is noteworthy is the increase in the amount of techniques of which the average absolute forecast errors are statistically insignificant. By forcing all techniques to have the same mean, the differences in the forecasts of the average correlation coefficients decrease, and thus the differences in the average absolute forecast errors decrease.

Because of the decrease in differences of the average absolute forecast errors of some, but not all, techniques with the historical adjustment, some changes in the performance of the techniques are noticeable. Most techniques perform better with the historical adjust-ment, but there are exceptions. The historical adjustment could be easily incorporated into the unadjusted forecasts, when it would result in a lower absolute average forecast error.

In the second five years the techniques Shrink-SF, EWMA and DCC have a higher average absolute forecast error with the historical adjustment than without. Same is true for the techniques Shrink-SF and EWMA in the fourth five years and the techniques EWMA and DCC in the fifth five years.

As can be seen in the column “Combined”, the differences in the average absolute forecast errors of the five best performing techniques is relatively small. These five best performing techniques with the historical adjustment are the same five best performing techniques as discussed in section 5.1.1.3. Of the rest of the techniques a distinction can be made between the Beta = 1, which is the worst performing technique, and the other techniques that are in between the five best performing techniques and the Beta = 1 technique.

The aforementioned difference between the five best performing techniques and the rest of the techniques is less well noticeable than the distinction made in section 5.1.1, because of the decrease in the differences in the average absolute forecast errors. In the first five years there can be a distinction made between the five best performing techniques, of which the first four (i.e. the techniques EWMA, Shrink-SF, Historical and DCC) are statistically insignificant from eachother, and the Beta = 1 technique, which performs worst, and the rest of the techniques.

This distinction is harder to make in the second five years. As is the case in the first five years, the first four techniques are statistically insignificant from each other. The Shrink-CC technique, which is part of the five best performing techniques based on the column “Combined”, is statistically insignificant from the Unadjusted technique. The worst performing technique, the Beta = 1 technique, has the highest average absolute

forecast error, but it is insignificant from the Constant technique.

In the third five years the five best performing techniques can be clearly distinguished from the rest. The first two of the best performing techniques (the techniques Shrink-CC and Shrink-SF) are statistically insignificant, as are the three techniques following these. The difference in the average absolute forecast errors between the Beta = 1 technique and the techniques that perform better than the Beta = 1 technique is high as well.

The same distinction between the five best performing techniques, the Beta = 1 technique and the rest of the techniques can be made in the fourth five years, albeit that the differences in the average absolute forecast errors between the techniques are smaller than in the third five years.

The Beta = 1 technique is clearly the the technique that performs worst in the fifth five years. The difference in average absolute forecast error between the Blume technique, which is ranked ninth, and the Beta = 1 technique is large. A distinction between the five best performing techniques and the rest of the techniques is harder to make, since the techniques ranked first to seventh are statistically insignificant from each other.

Table 5.5: Average absolute forecast errors (historical adjustment)7 First five years Second five years Third five years

Forecast period 1985 – 1989 Forecast period 1990 – 1994 Forecast period 1995 – 1999

1.    EWMA 0.0663 1.    Shrink-SF 0.1542 1. Shrink-CC 0.0960 2. Shrink-SF 0.0666 2. Historical 0.1570 2. Shrink-SF 0.0995

3. Historical 0.0670 3. EWMA 0.1571 3.  Historical 0.1032 4. DCC 0.0671 4. DCC 0.1582 4. DCC 0.1045 5. Shrink-CC 0.0806 5.  Shrink-CC 0.1710 5. EWMA 0.1065

6. Unadjusted 0.0999 6. Unadjusted 0.1719 6. Unadjusted 0.1560

7. Vasicek 0.1013 7. Vasicek 0.1793 7.  Vasicek 0.1623 8. Constant 0.1312 8.  Blume 0.1899 8. Blume 0.1637

9. Blume 0.1334 9. Constant 0.1910 9. Constant 0.1660

10. Beta = 1 0.1602 10. Beta = 1 0.2003 10. Beta = 1 0.2165

Fourth five years Fifth five years Combined

Forecast period 2000 – 2004 Forecast period 2005 – 2009

1.    Shrink-SF 0.1092 1.            Shrink-CC 0.1722 1. Shrink-SF 0.1205 2. Historical 0.1093 2. Shrink-SF 0.1731 2. Historical 0.1227

3. EWMA 0.1140 3. EWMA 0.1737 3. EWMA 0.1235

4. DCC 0.1141 4. DCC 0.1745 4. DCC 0.1237

5. Shrink-CC 0.1224 5. Unadjusted 0.1761 5. Shrink-CC 0.1285 6. Blume 0.1393 6. Historical 0.1773 6. Unadjusted 0.1517 7. Unadjusted 0.1544 7. Vasicek 0.1777 7. Vasicek 0.1571

8. Vasicek 0.1651 8. Constant 0.2019 8. Blume 0.1684

9. Constant 0.1961 9. Blume 0.2160 9. Constant 0.1772

10. Beta = 1 0.2047 10. Beta = 1 0.2703 10. Beta = 1 0.2104

7_{All differences between the average absolute errors are statistically significant at the 5% level, unless}

grouped by bracket. The column “Combined” shows the unweighted average of the average absolute forecast errors. For the probabilities and the z-statistics of the differences in absolute forecast errors, see table C.2.