Backtesting CoVaR

(1)

Master’s Thesis

Backtesting CoVaR

Menko ten Cate

Student number:

10057749

Date of final version:

August 15, 2017

Master’s programme:

Econometrics

Specialisation:

Financial Econometrics

Supervisor:

Dr. S. A. Broda

Second reader:

Prof. dr. H. P. Boswijk

Faculty of Economics and Business

Amsterdam School of Economics

Requirements thesis MSc in Econometrics.

1. The thesis should have the nature of a scientic paper. Consequently the thesis is divided up into a number of sections and contains references. An outline can be something like (this is an example for an empirical thesis, for a theoretical thesis have a look at a relevant paper from the literature):

(a) Front page (requirements see below)

(b) Statement of originality (compulsary, separate page) (c) Introduction (d) Theoretical background (e) Model (f) Data (g) Empirical Analysis (h) Conclusions

(i) References (compulsary)

If preferred you can change the number and order of the sections (but the order you use should be logical) and the heading of the sections. You have a free choice how to list your references but be consistent. References in the text should contain the names of the authors and the year of publication. E.g. Heckman and McFadden (2013). In the case of three or more authors: list all names and year of publication in case of the rst reference and use the rst name and et al and year of publication for the other references. Provide page numbers.

2. As a guideline, the thesis usually contains 25-40 pages using a normal page format. All that actually matters is that your supervisor agrees with your thesis.

3. The front page should contain:

(a) The logo of the UvA, a reference to the Amsterdam School of Economics and the Faculty as in the heading of this document. This combination is provided on Blackboard (in MSc Econometrics Theses & Presentations).

(b) The title of the thesis

(c) Your name and student number (d) Date of submission nal version

(e) MSc in Econometrics

(2)

Statement of Originality

This document is written by Menko ten Cate who declares to take full responsibility for the contents of this document. I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it. The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

1 Introduction

Since the financial crisis, there has been a lot of attention for systemic risk, the risk that one institution in financial distress can contaminate the entire financial system. Federal Reserve Governor Daniel Tarullo formulated systemic importance in a 2009 testimony before the Senate

Banking, Housing, and Urban Affairs Committee as follows:1

“Financial institutions are systemically important if the failure of the firm to meet its obli-gations to creditors and customers would have significant adverse consequences for the financial system and the broader economy.”

The most widely used risk measure in finance was (and still is) the Value at Risk (V aR), which is defined as the maximum return loss of a financial institution within a certain probability. The V aR focuses on one financial institution in isolation and does not capture its risk contribution to the financial system. Therefore, there has been a lot of interest in risk measures which take the interaction between financial institutions into account. Systemic risk measures aim to capture the amount of risk in the financial system that can be attributed to a financial institution.

Several systemic risk measures have been developed. Billio, Getmansky, Lo, & Pelizzon (2012) use the time variation in the number of principal components necessary to explain a certain fraction of the volatility of the return of the financial system, to determine the time variation in the level of interconnectedness between individual financial institutions. They analyse the directionality of shocks with Granger causality tests.

Huang, Zhou, & Zhu (2012) construct a systemic risk measure from a hypothetical in-surance premium against distress of the financial system based on the probability of default of individual institutions and their asset correlations.

Acharya, Pedersen, Philippon, & Richardson (2010) define the Systemic Expected Shortfall (SES) as the Expected Shortfall of an institution given that the financial system is in crisis. (The Expected Shortfall is defined as the expected loss of an institution given that its loss is above it’s V aR level.) Acharya, Engle, & Richardson (2012) extend this definition to the expected amount of capital that a firm needs in case of a financial crisis.

This thesis focuses on a systemic risk measure developed by Adrian & Brunnermeier (2016). It is called the Difference in Conditional Value at Risk or ∆CoV aR. The ∆CoV aR of an insti-tution j is defined as the V aR of the financial system conditional on j being in distress minus the

(5)

V aR of the financial system conditional on the return of j being at its median. This definition captures the systemic risk in line with the definition mentioned earlier, the amount of systemic risk that can be attributed to institution j.

1990 1995 2000 2005 2010 2015 0 0.05 0.1 0.15 Quantile regression

Quantile regression with beta correction AR quantile regression

AR quantile regresion with beta correction Multivariate GARCH

Realized ∆CoVaR Multivariate HEAVY

Figure 1: The ∆CoV aR (y-axis) of the financial system conditional on the return of Citigroup “the amount of systemic risk that can be attributed to Citigroup” (x-axis represents years).

Figure 1 illustrates the ∆CoV aR of Citigroup estimated using seven different estimation methods. The stock crash of 1987 and the financial crisis of 2008 are clearly visible. A detailed description of the estimation methods used to create Figure 1 is provided in Section 2.4 and the data is discussed in Section 5.1. The lines represent the quarterly averages of the daily ∆CoV aR estimations. The figure shows that the estimation methods give different results. Therefore, it would be useful to have a formal procedure to test which method gives the most accurate results.

A lot of further research on ∆CoV aR has been published after the first working paper of Adrian & Brunnermeier was published in 2008. Girardi & Erg¨un (2013), Mainik & Schaanning (2012), and Banulescu, Hurlin, Leymarie, & Scaillet (2016) provide backtesting methods to test whether an estimated (∆)CoV aR series is specified correctly.

The general idea of these methods is to use conventional V aR backtesting methods on the subset of CoV aR predictions in which the condition is satisfied (i.e. where institution j is in distress). A disadvantage of these methods is that testing only this subset (often only about 1%

(6)

or 5% of the predictions) could lead to limited test power in realistic sample sizes.

In this thesis, we propose a new CoV aR backtesting method. Instead of using a subset in which the condition is satisfied, we predict the CoV aR as a function of its condition (the return of j) and evaluate the predicted CoV aR at the realisation of this condition (the observed return of j). This allows us to perform tests on the full set of CoV aR predictions.

The remainder of this thesis is organised as follows. Section 2 formally defines the ∆CoV aR and discusses different estimation methods. Section 3 introduces a previous CoV aR backtesting method as well as our new CoV aR backtesting method.

In Section 4, we use Monte Carlo simulation to assess the finite sample properties of our test. We find that our test is useful in a realistic setting and that it has better small sample properties than previous tests. In Section 5 we discuss the data used in this thesis and in Section 6 we apply our test to real world data and compare the performance of three CoV aR estimation methods. Finally, Section 7 makes some concluding remarks and some suggestions for further research.

(7)

2 CoVaR

2.1 Definition

The one-day ahead Value at Risk V aRi

qt is defined as the q-th quantile of the return loss of institution or group of institutions i at time t, given the information available at time t − 1:

PXti≤ V aR i

qt|Ft−1 = q, (1)

where Xi

t is the return loss of i and Ft−1 is the information set available at time t − 1. Often t is measured in days and q is 0.95 or 0.99. Subsequently, the Conditional Value at Risk (CoV aR) is defined as the Value at Risk of institution or group of institutions i conditional on some event Etj on institution or group of institutions j at time t:

P h X_ti≤ CoV aRi|Etj qt Ft−1, E j t i = q, (2)

where Etj is often a realisation of the return loss of j.

In the literature, we find two different definitions of ∆CoVaR, which both attempt to capture the amount of systemic risk of i that can be attributed to j:

∆CoV aRi|j_qt = CoV aRi|X

j t=V aR j qt qt − CoV aR i|Xj t=V aR j 0.5,t qt , (3)

j t≥V aR j qt qt − CoV aR i|µj_t−σj t≤X j t≤µ j t+σ j t qt . (4)

Adrian & Brunnermeier (2016) define the ∆CoV aR according to (3), which is the V aR of i given that the return loss of j (Xtj) is at its q-th quantile (its V aR) minus the V aR of i given that the return loss of j is at its 0.5-quantile (its median).2 We will refer to this by Quantile ∆CoV aR.

Girardi & Erg¨un (2013) change this definition to (4), which is the V aR of i given that the return loss of j (X_tj) is larger than or equal to its q-th quantile minus the V aR of i given that the return loss of j is no more than one standard deviation from its mean.3 We will refer to this by T ail ∆CoV aR.

2_{The first working paper of Adrian & Brunnermeier was released in 2008.} 3_{The final ∆CoV aR definition used by Girardi & Erg¨}_{un (2013) is}

∆CoV aRi|jqt = 100 × CoV aRi|X j t≥V aRjqt qt − CoV aR i|bj_t qt CoV aRi|b j t qt

(8)

One of the reasons Girardi & Erg¨un (2013) prefer CoV aR based on an inequality con-dition, is monotonicity in the dependence parameter. Mainik & Schaanning (2012) argue that conditioning on an equality condition could violate this relationship. Under a joint normality

assumption for example, CoV aRi|X

j t=V aR

j qt

qt is not monotonically increasing in the correlation

pa-rameter. However, after subtracting CoV aRi|X

j t=V aR

j 0.5,t

qt , ∆CoV aR based on equality conditions

and a normality assumption is monotonically increasing in the correlation parameter.

The advantage of using quantiles in the return realisation condition (instead of a fixed level of return loss) is that we also take the likelihood of such an event into account, which enables us to compare the ∆CoV aRs over time.

Similar to Adrian & Brunnermeier (2016) and Girardi & Erg¨un (2013), and in line with the definition of systemic risk discussed in the introduction, we will focus on the ∆CoV aR of the financial system conditional on the return of a specific institution (∆CoV aRsystem|j_{). Adrian &} Brunnermeier (2016) argue that it is an advantage that the ∆CoV aR does not causally allocate the source of systemic risk (the correlation might originate from a common source) because it assigns a high risk value to small financial institutions that are of no significant importance on their own but are significant as part of a larger group of financial institutions with similar risks. In the following sections, we will use the following notation: CoV aR based on an equality condition (e.g. CoV aRi|X

j t=V aR

j qt

qt ) is referred to by Q-CoV aR (Quantile CoV aR) and CoV aR

based on an inequality condition (e.g. CoV aRi|X

j t≥V aR

j qt

qt ) is referred to by T -CoV aR (T ail

CoV aR).

2.2 Estimating Q-CoVaR

2.2.1 Quantile regression

Adrian & Brunnermeier (2016) use quantile regression to estimate ∆CoV aR. Quantile regression is similar to ordinary linear regression. However, instead of estimating the mean of the dependent variable, it estimates a quantile of the dependent variable.

Since the Q-CoV aR is defined as the q-th quantile of the return loss of i given the return loss of j, we can use quantile regression to estimate the Q-CoV aR. The advantage of the quantile regression method is that it does not need any distributional assumptions. The approach is similar to V aR estimation by quantile regression, but this time xj_t (the realisation of X_tj) is included as

(9)

an explanatory variable: xi_t= αq+ γq Mt−1+ βq x j t+ εqt (5) b F−1 Xi t|X j t (q) = \CoV aRi|X j t=x j t qt = ˆαq+ ˆγq Mt−1+ ˆβq x j t, (6) where F−1 Xi t|X j t

(·) is the inverse cumulative distribution function of Xi

t given X

j

t = x

j

t and Mt−1

contains lagged market state variables (e.g., the lagged risk-free rate or lagged market volatility). Following Koenker & Bassett (1978), the parameters are found by the minimizing the following expression over (αq, γq, βq):

X q x i t− αq− γqMt−1− βqxjt + X (1 − q) x i t− αq− γqMt−1− βqxjt . t∈{t|xi t≥αq+γqMt−1+βqxjt} t∈{t|xit<αq+γqMt−1+βqxjt} (7)

Note that the expression for Q-∆CoV aR simplifies to

∆ \CoV aRi|j_qt = ˆβq [ V aRj_qt− [V aRj_0.5,t . (8)

2.2.2 Asymmetric GARCH-DCC model with Gaussian innovations

A disadvantage of the quantile regression method is that the parameter βq, which measures the dependency of the CoV aR of i on the return loss of j, does not change over time, even when the correlation between Xi

t and X

j

t changes. Therefore, we will also discuss a second estimation method which is similar to the GARCH-DCC model used by Girardi & Erg¨un (2013).

We start by following the derivation of Adrian & Brunnermeier (2016) for an analytical expression of Q-CoV aR based on a joint normality assumption:

(10)

  Xi t X_tj  ∼ N     0 0  ,   (σi t)2 ρ ij t σtiσ j t ρij_tσi_tσj_t (σj_t)2     (9) =⇒ X_ti|X_tj = xj_t∼ N σ i tρ ij t σj_t x j t, (1 − (ρ ij t) 2_)(σi t) 2 ! (10) =⇒ CoV aRi|X j t=x j t qt = F_X−1i t|X j t (q) = σ i tρ ij t σ_tj x j t+ σit q 1 − (ρijt )2 Φ−1(q) . (11)

where Φ−1(·) is the inverse standard normal CDF. The expression for ∆ CoVaR simplifies to:

∆CoV aRi|jqt = Φ−1(q) ρ ij

tσit, (12)

as the right-hand side in (11) cancels from taking the difference, and because under the normality assumption V aRj_qt= F−1

Xj_t (q) = Φ

−1_{(q) σ}j

t and Φ−1(0.5) = 0.

In order to find CoV aR estimates using (11) we need a method to find the volatility and correlation series. For this purpose, we use the Asymmetric GARCH-DCC model. The first step is to find the volatility series by estimating two univariate Threshold Generalized Autoregressive Conditional Heteroscedasticity (TGARCH) models. In contrast to the standard GARCH model, the TGARCH accounts for the leverage effect in the volatility series. The leverage effect refers to the property that large negative returns often lead to more increased volatility than large positive returns. The TGARCH(1,1,1) model is:

rt= σtνt (13)

σ2_t _{= α + (β + γI (r}t−1< 0)) rt−12 + δσ 2

t−1, (14)

where {rt} is a univariate return series (rt= −xt), α > 0, β ≥ 0, δ ≥ 0 and νt∼ N (0, 1) i.i.d. A value of γ > 0 would represent the leverage effect. The parameter values are found by maximum likelihood.

After we have estimated the volatility series {ˆσi_t} and {ˆσj_t}, we obtain the correlation series {ˆρijt } using the Asymmetric Dynamic Conditional Correlation (DCC) model. First we obtain the

(11)

standardized return vector series ˆ t=   ˆ i t ˆ j_t  =   ri t/ˆσit rj_t/ˆσj_t  =   −xi t/ˆσti −xj_t/ˆσj_t  , (15)

then we estimate by maximum likelihood

Qt= (1 − θ1− θ3) ¯Q + θ1t−10t−1+ θ2(vt−1vt−10 − ¯N ) + θ3Qt−1, (16)

where θ1≥ 0, θ3≥ 0, vt= max(0, −t), ¯Q is the unconditional covariance matrix of tand ¯N is the unconditional covariance matrix of vt. The correlation series are obtained by

ˆ

ρij_t = qˆ12,t p ˆq11,tqˆ22,t

, (17)

where qkl,tis the kl’th element of Qt. The correlation matrix ˆρthas unit diagonal elements and off-diagonal elements ˆρij_t. The maximum likelihood specification is based on t|Ft−1∼ N (0, ρt).

Maximum likelihood is equivalent to maximising −1

2 T P t=2

(log| ˆρt| + ˆt0ρˆtˆt) over θ1, θ2, and θ3.

2.2.3 Asymmetric GARCH-DCC model with Student-t innovations

Instead of assuming a bivariate normal distribution for (X_ti, Xtj), we can also assume a standard-ized bivariate Student-t distribution for the standardstandard-ized returns. This might be more appropriate as it is often argued that stock return distributions have heavier tails than the normal distribution (e.g., Bradley & Taqqu, 2001).

Using quasi maximum likelihood, we find the same parameters (and therefore, the same volatility and correlation series) from the Asymmetric GARCH-DCC model estimated under a normality assumption. Once we have obtained the standardized return loss vector series { ˆt} and the correlation matrix series { ˆρt} (with unit diagonals), we can find the number of degrees of freedom ν by maximising the likelihood of the standardized return vector series assuming a standardized bivariate Student-t distribution. This bivariate distribution has the following pdf:

ft i_t, j_t= Γ ((ν + 2) /2) Γ (ν/2) (ν − 2) π |ρt| 1/2 1 + 1 ν − 2 0 tρ −1 t t −(ν+2)/2 . (18)

(12)

It follows that the pdf of i

tconditional on j

t can be expressed as (Kotz & Nadarajah, 2004):

ft it jt = Γ ((ν + 2) /2) p(ν − 2) πΓ ((ν + 1) /2) |ρt| 1/2× 1 + (1/ (ν − 2))jt 2(ν+1)/2 1 + (1/ (ν − 2)) 0 tρ −1 t t (ν+2)/2. (19)

Therefore, the conditional distribution of Xi

t given X j t = x j t= −σ j t j

t has the following pdf:

ft xi_txj_t= Γ ((ν + 2) /2) σi tp(ν − 2) πΓ ((ν + 1) /2) |ρt|1/2 × 1 + (1/ (ν − 2))xj_tσj t 2(ν+1)/2 1 + (1/ (ν − 2)) x0_tΣ−1_t xt (ν+2)/2 , (20)

where Σt is the covariance matrix of (Xti, X j

t) and xt= (xitx j

t)0. Finally, the Q-CoV aR can be found by solving \ CoV aRi|X j t=x j t qt Z b ft xit xjt dxit= q −∞ (21)

for \CoV aRi|X

j t=x j t qt , where bft xit xjt

is the estimated pdf of Xti conditional on X j t = x

j t.

2.3 Estimating T-CoVaR

In this section we will shortly explain how the T -CoV aR can be found. From the T -CoV aR definition we know that

P Xti≥ CoV aR i|X_tj≥V aRjqt qt X j t ≥ V aR j qt, Ft−1 = 1 − q (22) =⇒ P Xi t≥ CoV aR i|X_tj≥V aRj qt qt , X j t ≥ V aR j qt Ft−1 P h X_tj ≥ V aRj_qt Ft−1 i = 1 − q. (23)

Next, since the denominator on the left-hand side of (23) is equal to 1 − q, we have

P Xti≥ CoV aR i|X_tj≥V aRj qt qt , X j t ≥ V aR j qt Ft−1 = (1 − q)2. (24)

(13)

Finally, after one has found [V aRj_qtand the joint distribution of Xi

tand X

j

t using some estimation method, the T -CoV aR can be found by solving

∞ ∞ Z Z b ft(xit, x j t)dx j tdx i t= (1 − q) 2 \ CoV aRi|X j t≥V aRjqt qt V aR[ j qt (25)

for \CoV aRi|X

j t≥V aR j qt qt (where bft(xit, x j

t) is the estimated joint probability distribution function of (Xi

t, X j

t) conditional on Ft−1). For a distribution that is symmetric around zero (e.g. a Gaussian distribution with zero mean) the equation simplifies to:

b Ft − \CoV aRi|X j t≥V aR j qt qt , − [V aR j qt = (1 − q)2, (26)

where bFt(·, ·) is the estimated cumulative distribution function of (Xti, X j t).

2.4 Estimating Q-∆CoVaR

2.4.1 Quantile regression

Adrian & Brunnermeier (2016) use quantile regression to estimate the ∆CoV aR. Lagged state variables (denoted by Mt−1) are included in the quantile regression to capture the time-variation of the ∆CoV aR. We estimate the V aR and CoV aR using quantile regression (as explained in Section 2.2.1): [ V aRj_qt= bF−1 Xj_t (q) = ˆα j q+ ˆγ j qMt−1 (27) \ CoV aRi|X j t=x j t qt = bF_X−1i t|X j t

(q) = ˆαi|jq + ˆγqi|jMt−1+ ˆβqi|jx j

t, (28)

where F (·) is the cumulative distribution function and xj_t is the observed return loss of j. Next, the ∆CoV aR can be found as:

∆ \CoV aRi|jqt = ˆβqi|j [ V aRjqt− [V aR j 0.5,t , (29) where [V aRj_qt= ˆαj

q+ˆγqjMt−1. The parameters are found by the minimization procedure developed by Koenker & Bassett (1978).

(14)

2.4.2 Quantile regression with beta correction

The only time-variation captured in (29) originates from the time-variation in the V aR of j. It could easily be argued that the dependency of the V aR of i on the return loss of j (captured by

ˆ

βqi|j) is time-dependent. From the Capital Asset Pricing Model we know that

−Xi t = r f t + σi_tρi,system_t σ_tsystem −X_tsystem− rf_t, (30)

where the minus signs are present because Xti and X system

t represent return loss (Sharpe, 1964).

If we replace system by j we find the same coefficient (σi tρ

ij t /σ

j

t) as in the conditional expectation under the normality assumption in (10). We will use this coefficient as a correction factor for the coefficient in (28): \ CoV aRi|X j t=x j t qt = ˆα i|j q + ˆγ i|j q Mt−1+ ˆβqi|j ˆ σj_tρˆij_t ˆ σi t xj_t ! . (31)

This means we use (σ_tjρij_txj_t/σi

t) as the explanatory variable instead of x j

t. The correlations {ˆρij_t } and volatilities {ˆσi

t} and {ˆσ j

t} are obtained from the GARCH-DCC estimation discussed in

Section 2.4.5. Next, we change (29) to find the ∆CoV aR:

∆ \CoV aRi|j_qt = σˆ j tρˆ ij t ˆ σi t ˆ β_qi|j ! [ V aRj_qt− [V aRj_0.5,t (32)

One could argue that this method is less efficient due to the two-step estimation procedure. How-ever, this method could lead to more accurate estimates, since the quantile regression explained in the previous section could be misspecified because of the time-independency of ˆβi|jq and the GARCH-DCC model could be misspecified since it requires distributional assumptions.

2.4.3 Autoregressive quantile regression

Engle & Manganelli (2004) propose an autoregressive term as an extension to V aR estimation using quantile regression, which they call CAViaR (Conditional Autoregressive Value at Risk). The third estimation method includes this term in the V aR equation (27):

[ V aRj_qt = bF−1 X_tj (q) = ˆα j q+ ˆθ j qV aR[ j q,t−1+ ˆγ j qMt−1 (33)

(15)

2.4.4 Autoregressive quantile regression with beta correction

The fourth estimation method uses both the beta time-variation correction factor (32) (which we introduced in Section 2.4.2) and the autoregressive term for the estimation of the V aR of j (33) (discussed in Section 2.4.3).

2.4.5 GARCH-DCC model

The GARCH-DCC model is the symmetric equivalent of the model in Section 2.2.2. This means

that it is the same model where we assume γ = 0 in (14) and θ2 = 0 in (16). In the first

step, we estimate the volatility series by modelling the return series as two univariate Gaussian GARCH(1,1) processes. After we have estimated the volatility series {ˆσi

t} and {ˆσ j

t}, we obtain the correlation series { ˆρij_t } using the Dynamic Conditional Correlation (DCC) model. Finally we know from (12) the Q-∆CoV aR under a normality assumption:

∆ \CoV aRi|j_qt = Φ−1(q) ˆρij_tσˆi_t (34)

where Φ−1(·) is the inverse standard normal CDF.

2.4.6 Realised Q-∆CoVaR

The realised ∆CoV aR method uses high frequency data, which allows us to estimate the ∆CoV aR for each day separately. The first step is to estimate the realised variances and realised covariance of the return series. Under certain assumptions, in particular that the returns each follow a Brownian motion with drift, the following holds:

RV_tNi = N X n=1 (X_tni )2 −→ (σ_ti)2 (35) RV_tNj = N X n=1 (X_tnj )2 −→ (σ_tj)2 (36) RC_tNij = N X n=1 Xtni X j tn−→ ρ ij tσ i tσ j t, (37) where Xi tnand X j

tnare the intraday return losses at time n on day t. For proof of the statements above we refer to Barndorff-Nielsen & Shephard (2001) and Sheppard (2006).

(16)

The realised correlation can easily be found as RC_tNij q RVi tNRV j tN. Using (12), we find the realised ∆CoV aR under a normality assumption by:

∆ \CoV aRi|j_qt = Φ−1(q) RC ij tN q RV_tNj . (38)

The realised ∆CoV aR could be regarded as a reliable benchmark to which to compare other estimation methods. However, due to market microstructure noise resulting from bid-ask spreads and infrequent trading, the assumptions may be violated. Epps (1979) finds that using a more than hourly frequency when estimating realised covariance statistics will cause the realised cor-relation to be biased towards zero. The microstructure noise, in combination with the normality assumption, might cause the realised ∆CoV aR to be biased considerably itself.

2.4.7 Multivariate HEAVY model

The Multivariate High-Frequency-Based Volatility (HEAVY) model combines features of the GARCH-DCC model with realised statistics. Based on methods developed by Noureldin, Shep-hard, & Shephard (2012) we model the return losses as follows:

Xt=   Xti X_tj  , RCtN =   RV_tNi RC_tNij RC_tNij RV_tNj   (39) Xt= Σ 1/2 t t (40) Σt= ¯Π + α RCt−1,N+ β Σt−1, (41)

where ¯Π > 0, α ≥ 0, β ≥ 0 and t ∼ N (0, I2) i.i.d. The paramaters are found by maximum likelihood. If α + β > 1, we assume the process is first order integrated and we restrict the model to:

Σt= (1 − α) RCt−1,N+ α Σt−1. (42)

After finding the estimated correlation and volatility series we can estimate the ∆CoV aR under a normality assumption using (12).

(17)

2.5 Estimating T-∆CoVaR

From Section 2.1 we know that the T -∆CoV aR is defined as:

j t≥V aR j qt qt − CoV aR i|µj_t−σ_tj≤X_tj≤µj_t+σj_t qt . (43)

The estimation of CoV aRi|X

j t≥V aR

j qt

qt (the T ail CoV aR) is explained in Section 2.3. Similarly,

once we have estimated ˆµj_t, ˆσ_tj, and the distribution of (Xi t,X

j

t), we find the second part of (43) by solving ∞ µˆj_t+ˆσ_tj Z Z b ft(xit, x j t)dx j tdx i t= (1 − q)ˆp j t \ CoV aRi|bj_qt µˆj_t−ˆσj_t (44)

for \CoV aRi|b

j

qt , where bft(xit, x j

t) is the estimated joint pdf of (Xti, X j

t) conditional on Ft−1, bj is the base state of X_tj : µj_t− σ_tj ≤ X_tj ≤ µj_t+ σj_t, and pj_t is the probability that X_tj is in its base state: pj_t_{= P}hµj_t− σj_t ≤ X_tj≤ µj_t+ σ_tj

Ft−1 i

(18)

3 Backtesting

3.1 Backtesting T-CoVaR

A second reason why Girardi & Erg¨un (2013) prefer T ail CoV aR (besides the monotonicity

property of the T -CoV aR) is that it enables them to backtest their CoV aR predictions. In other papers about backtesting CoV aR, a similar aproach is used (Mainik & Schaanning, 2012; Banulescu et al., 2016). We will briefly explain the backtesting procedure used in these papers, which is based on conventional V aR backtesting methods. We know that for a correctly specified T -CoV aR: P Xti> CoV aR i|X_tj≥V aRjqt qt Ft−1, X j t ≥ V aR j qt = 1 − q. (45)

If we focus on the subset of T -CoV aR predictions for which the observed xj_t≥ [V aRj_qt (where xj_t is the realisation of X_tj), we can define a hit series similar to a V aR hit series:

I_ni _{= I} xi_n> \CoV aRi|X j n≥V aR j qn qn , n ∈ {1, . . . , N }, (46)

where N is the number of V aR violations of j, xi

nis the realisation of Xni, and I (·) is the indicator function which is equal to one if the condition is true and zero otherwise. This subset has an expected cardinality of E [N ] = (1 − q)T , provided that [V aRj_qt is correctly specified (where T is the cardinality of the full set of CoV aR predictions).

Under the null hypothesis of a correctly specified T -CoV aR, it follows from (45) that PIni = 1 = (1 − q) and P Ini = 1 Ii n−1= 1 = P Ini = 1 Ii

n−1= 0. The first equation corre-sponds to the unconditional coverage test, which tests whether the fraction of hits is not signifi-cantly different from (1 − q). The second equation corresponds to the independence test, which

tests whether the historic information set Fn−1 provides no additional information about the

occurrence of the hits. A third test is the conditional coverage test, which tests both hypotheses simultaneously (Christoffersen, 1998).

(19)

3.2 Backtesting Q-CoVaR

A disadvantage of the T -CoV aR backtesting method explained in the previous section, is that it only uses the subset of CoV aR predictions for which xj_t ≥ [V aRj_qt. This means that the expected number of predictions in the test sample (hit series) is E [N ] = (1 − q)T , where T is the cardinality of the full set of CoV aR predictions. In this test sample, the expected number of hits is (1 − q)E [N ] = (1 − q)2T . This means that for q = 0.95 we have one expected test sample hit in 400 predictions in the original set. For q = 0.99, this is only one in 10,000, which is approximately one every forty years. Therefore, these tests have little power for T -CoV aR prediction series of limited length.

In this thesis, we propose a backtesting procedure for Q-CoV aR which uses the full set of predictions. Instead of predicting a fixed value, we predict the Q-CoV aR of i as a function of the return loss of j (47). Next, we evaluate it at the observed return of j. We then use these “realised CoV aR predictions” to define the hit series (48):

\ CoV aRi|X j t=x j t qt = f i qt(X j t) (47) I_ti_{= I}xi_t> f_qti(xj_t), (48) where fi qt(x j

t) is the predicted CoV aR of i evaluated at the observed x j

t. Past information should have no explanatory power for the occurrence of the hits, as this would violate the conditioning on Ft−1 in the CoV aR definition. Moreover, the observed return of j at time t (xjt) should not have any additional explanatory power either, as the CoV aR should contain this information by definition. Therefore, under the null hypothesis of a correctly specified CoV aR

P h X_ti> CoV aRi|X j t=x j t qt Ft−1, X j t = x j t i = 1 − q =⇒ EhI_ti Ft−1, X j t = x j t i = 1 − q, (49)

which we can test in multiple ways. The information set Ft−1 includes all past information,

including xi t−1, x j t−1, V aRt−1i , V aR j t−1, CoV aR i|X_t−1j =xj_t−1

q,t−1 , It−1i , and all other historic variables and all further lags (t − 2, t − 3 . . .). For simplicity, we choose to include only Ii

t−1and x

j t in our tests. The first test is based on the V aR backtesting method of Engle & Manganelli (2004) and

(20)

uses a linear regression:

I_ti= α + β I_t−1i + γ xj_t+ εt. (50)

The first hypothesis we will test is H0 : γ = 0 and the second is H0 : α = (1 − q), β = γ = 0. Both hypotheses will be tested using a likelihood-ratio statistic (which are asymptotically χ2

(1) and χ2

(3) distributed). 4

Since a linear specification might not be the most appropriate for the dichotomous charac-ter of the hit series, our second test uses a logistic regression and is based on the V aR backtesting method of Dumitrescu, Hurlin, & Pham (2012):

PIti= 1 = F (α + β I i

t−1+ γ x

j

t), (51)

where F (·) is the cumulative distribution function of the logistic distribution. The hypotheses correspond to the linear regression test hypotheses, H0 : γ = 0 and H0 : α = F−1(1 − q), β = γ = 0. Similar likelihood-ratio statistics are used.

4_{To the best of my knowledge, this CoV aR backtesting procedure has not been proposed before. Especially,}

testing if the dependence on X_tj has been specified correctly by testing the explanatory power of xj_t on the hit series is (to the best of my knowledge) new.

(21)

4 Monte Carlo study

4.1 Distributional misspecification

In order to assess the finite sample properties of our test (the Q-CoV aR test discussed in Section 3.2), we implement a Monte Carlo simulation. In this section, we will investigate whether our test is able to detect a distributional misspecification. In Section 4.2, a dynamic misspecification will be investigated.

4.1.1 Size

The first step is to simulate random returns that have real world return properties. To this end, we simulate returns following the Asymmetric GARCH-DCC model, which is explained in Section 2.2.2. The parameters used in the random return simulation are obtained by estimating an Asymmetric GARCH-DCC model on real world daily log-returns of the Dow Jones US Financials

Index and JPMorgan Chase ranging from January 2001 to December 2014 (3500 observations).5

In each replication, a new bivariate return series with 3500 observations is simulated using this model. The first 2500 simulated observations are used to estimate new Asymmetric GARCH-DCC parameters. Based on these parameters, one-day ahead Q-CoV aR predictions will be made for the last 1000 observations. Using these 1000 out-of-sample CoV aR predictions we find the hit series and subsequently the likelihood-ratio test statistics. Since this process is quite time consuming, all the reported simulation results are based on 1000 replications (with 1000 daily CoV aR predictions each).

5_{The estimated univariate TGARCH parameters for the Dow Jones US Financials Index are}

α = 0.0000015, β = 0.0064825, γ = 0.1294960, and δ = 0.9217418, and for JPMorgan Chase

α = 0.0000026, β = 0.0222182, γ = 0.1038000, and δ = 0.9229578, where the parameters correspond to (14). The positive values for γ represent the leverage effect.

The unconditional covariance matrices of the standardized returns are estimated as:

¯ Q = V (t) = 1 0.847 0.847 1 , and ¯N = V (vt) = 0.525 0.444 0.444 0.502 .

Finally, the Asymmetric DCC parameters are estimated as follows:

θ1= 0.0089828, θ2= 0.0295914, and θ3= 0.9614814,

(22)

Using a bivariate Gaussian distribution to simulate the standardized returns (15) and using the analytical Q-CoV aR expression based on the joint normality assumption (11) we find the empirical size as the rejection frequency at the nominal 5% level.

In Table 1 the results of the empirical size simulation are shown. We find that for seven out of the eight test specifications, the size is significantly different from the nominal 5% level. To arrive at the asymptotic distribution, the number of observations in the out-of-sample hit series as well as the number of in-sample observations used for the Asymmetric GARCH-DCC parameter estimation should go to infinity.

To illustrate this, we repeat the size simulation, but we assume the parameters to be known. As can be seen from Table 2, when the parameters are assumed to be known, the empirical sizes are closer to their nominal 5% level (except for the bottom-right case), and the number of test specifications for which the size is significantly different from the nominal 5% level decreases from seven to two. Therefore, we can conclude that one of the reasons why the test over-rejects is the parameter estimation error.

(23)

95% CoV aR 99% CoV aR Linear test 0.062* (4.32) [3.84] 0.064** (4.26) [3.84] H0: γ = 0 Logistic test 0.062* (4.26) [3.84] 0.059 (4.14) [3.84] H0: γ = 0 Linear test 0.081*** (9.14) [7.81] 0.079*** (9.27) [7.81] H0: α = (1 − q) β = γ = 0 Logistic test 0.077*** (8.49) [7.81] 0.036** (7.06) [7.81] H0: α = F−1(1 − q) β = γ = 0

Table 1: Empirical sizes based on the rejection frequency in the Monte Carlo simulation with 1000 replications. The hypotheses are rejected if the test statistics exceed their 5% asymptotic critical value. The number of stars indicate whether the size is significantly different from 0.050 (*:=10% level, **:=5% level, ***:=1% level). The empirical 5% critical values are reported between parentheses with the corresponding asymptotic 5% critical values between brackets.

95% CoV aR 99% CoV aR Linear test 0.048 (0.062*) 0.053 (0.064**) H0: γ = 0 Logistic test 0.045 (0.062*) 0.050 (0.059) H0: γ = 0 Linear test 0.052 (0.081***) 0.069*** (0.079***) H0: α = (1 − q) β = γ = 0 Logistic test 0.056 (0.077***) 0.031*** (0.036**) H0: α = F−1(1 − q) β = γ = 0

Table 2: Empirical sizes based on the rejection frequency in the Monte Carlo simulation with known parameters (1000 replications). The hypotheses are rejected if the test statistics exceed their 5% asymptotic critical value. The number of stars indicate whether the size is significantly different from 0.050 (*:=10% level, **:=5% level, ***:=1% level). The corresponding sizes from the simulation with parameter estimation are reported between parentheses.

(24)

4.1.2 Power

After assessing the finite sample properties of the test when the Q-CoV aR is specified correctly, we change the simulation design to see how well our test performs when the CoV aR is misspecified. One option would be to estimate unconditional V aRs and use them as misspecified CoV aRs to investigate the power of the test. However, it is obvious that xj_t will have explanatory power on the V aR hit series (provided that X_ti and X_tj are correlated). Therefore, it is more interesting to change the dependency structure between Xtiand X

j

t in the return simulation and check whether our test is able to detect the distributional misspecification of our Q-CoV aR when we still use the the analytical CoV aR expression based on the joint normality assumption (11).

In the size simulation we used a bivariate Gaussian distribution to simulate the standard-ized returns. Two random variables which are jointly normally distributed can be described as two variables with Gaussian marginal distributions and a Gaussian dependency structure. In our power simulation, we would like to change the dependency structure to a Student-t ’s while maintaining the Gaussian marginals.

This is implemented as follows. If we draw a random vector (T1, T2) from a

bivari-ate Student-t distribution with ν degrees of freedom, correlation parameter ρ (the ρt from

the Asymmetric GARCH-DCC model) and marginal CDFs F ( · ; ν), then C1 = F (T1; ν) and

C2 = F (T2; ν) are both uniformly distributed on [0, 1]. The distribution of (C1, C2) is called

a “Student-t copula”. If we transform these random variables as follows, Z1 = Φ−1(C1) and

Z2 = Φ−1(C2), then (Z1, Z2) is a bivariate random variable with Gaussian marginals and a Student-t dependency structure. Finally, if we multiply them by the volatilities from the Asym-metric GARCH-DCC model, we have the simulated returns.

We assume the parameters to be unknown in the power simulation (so we estimate new parameters in each replication) just as in the first size simulation. Therefore, we use the em-pirical critical values from the first size simulation to find the size-corrected power. Since the predicted CoV aRs using (11) are misspecified in this case (due to the Student-t copula used in the simulation) we will find whether the test is able to detect this misspecification.

(25)

2 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: γ=0 Linear test Logistic test 2 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: γ=0 Linear test Logistic test 2 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test 2 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test

Figure 2: For each test specification, the graph shows the size-corrected power (y-axis) as a func-tion of the number of degrees of freedom of the Student-t copula used in the simulafunc-tion (x-axis). The results are based on 1000 replications for each point in the graphs. The horizontal grey line represents the nominal 5% level.

In Figure 2 the size-corrected power is shown as a function of the number of degrees of freedom of the Student-t copula used to simulate the returns. Note that as the number of degrees of freedom approaches infinity, the CoV aR is correctly specified. When the number of degrees of freedom becomes small, the dependency structure becomes more different from the Gaussian dependency structure (which is assumed in the CoV aR prediction), and our test detects the misspecification more often. In line with Dumitrescu et al. (2012), we find that the performance of the two tests (linear and logistic) is similar in most cases, although in some situations the logistic test outperforms the linear test (bottom-right).

(26)

200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: γ=0 Linear test Logistic test 200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: γ=0 Linear test Logistic test 200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test 200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test

Figure 3: For each test specification, the graph shows the size-corrected power (y-axis) as a func-tion of the number of out-of-sample CoV aR predicfunc-tions (x-axis). The Student-t copula used in the simulation has one degree of freedom. The results are based on 1000 replications for each point in the graphs. The horizontal grey line represents the nominal 5% level.

Figure 3 shows the size-corrected power as a function of the number of out-of-sample CoV aR predictions (the length of the hit series). The number of in-sample observations is still fixed at 2500 and the Student-t copula used in the simulation has one degree of freedom. For each power simulation, we use the empirical critical value from the corresponding size simulation. As expected, the size-corrected power increases when the length of the hit series increases.

The zero size-corrected power for small out-of-sample sizes on the bottom-right is caused by a perfect fit in the linear test regression in the size simulation when there are zero hits, which leads to infinite empirical critical values.

(27)

95% CoV aR 99% CoV aR Linear test 0.97 (0.06) 0.99 (0.06) H0: γ = 0 Logistic test 0.98 (0.06) 0.98 (0.06) H0: γ = 0 Linear test 0.99 (0.08) 0.94 (0.08) H0: α = (1 − q) β = γ = 0 Logistic test 0.99 (0.08) 0.97 (0.04) H0: α = F−1(1 − q) β = γ = 0

Table 3: Size-corrected power at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. The Student-t copula used in the simulation has one degree of freedom. The corresponding sizes are reported between parentheses.

In Table 3 the size-corrected power is shown with the corresponding sizes between parentheses. The number of out-of-sample predictions is 1000 and the standardized returns are simulated using a Student-t copula with one degree of freedom. Since all test specifications have a size-corrected power above 0.9, we can conclude that our test detects the misspecification very well in this setting.

The power simulation shows us that our test is able to detect a distributional misspeci-fication in a realistic sample size. Combined with the size simulation results from the previous section, we can conclude that our test as proposed in section 3.2 is useful in a realistic setting when we want to test whether a CoV aR specification describes the underlying distribution correctly.

(28)

4.1.3 Alternative backtesting methods

Now we have shown that our test has the desired finite sample properties and that it is able to detect a distributional misspecification, we want to assess the performance of other tests in this setting. The first alternative test has the same Q-CoV aR test specification as in the previous section, with the only difference that we exclude xj_t as explanatory variable in the test regression equations. Therefore the regression equations and hypotheses in this test are as follows:

Linear test: Ii

t= α + β It−1i + εt H0: α = (1 − q), β = 0

Logistic test: _PIi

t= 1 = F (α + β It−1i ) H0: α = F−1(1 − q), β = 0

So essentially we are using V aR backtesting methods on the Q-CoV aR evaluated at the observed return loss xjt. We will refer to this test by “Test 1”. The second alternative test (“Test 2”) is the T -CoV aR backtesting method discussed in Section 3.1, which only tests the subset where the inequality condition is satisfied.

The results of the simulation for Test 1 are shown in Table 4 (this is the power simula-tion using the Student-t copula with one degree of freedom, 1000 out-of-sample predicsimula-tions, and including parameter estimation). We find that by excluding xjt from the test regression, we lose some of the power, but not all of it.

95% Q-CoV aR 99% Q-CoV aR Linear test H0: α = (1 − q) 0.68 (0.09) 0.27 (0.09) Linear test H0: β = 0 0.07 (0.04) 0.07 (0.05) Linear test 0.67 (0.07) 0.12 (0.07) H0: α = (1 − q) β = 0 Logistic test 0.60 (0.07) 0.49 (0.03) H0: α = F−1_{(1 − q)} β = 0

Table 4: Size-corrected power for Test 1 at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. The true underlying dependency structure is a Student-t copula with one degree of freedom. The corresponding sizes are reported between parentheses.

(29)

In Table 5 the simulation results for Test 2 are shown. The top three rows show the results of the simulation where 1,000 out of sample T -CoV aR predictions are tested. We find that this test has almost no power. Since only a small subset is tested here (so about 50 predictions for the 95% CoV aR and about 10 for the 99% CoV aR), we increase the number of out-of-sample predictions to 10,000. The results are shown in the bottom three rows. It shows that the test still has almost no power.

1,000 predictions 95% T -CoV aR 99% T -CoV aR

Unconditional coverage test 0.04 (0.11) 0.06 (0.02)

Independence test 0.05 (0.08) 0.07 (0.01)

Conditional coverage test 0.06 (0.08) 0.06 (0.03)

10,000 predictions

Table 5: Size-corrected power for Test 2 at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. The true underlying dependency structure is a Student-t copula with one degree of freedom. The corresponding sizes are reported between parentheses.

It might be unexpected that Test 1 is able to detect the misspecification based on the hit frequency most of the times (for the 95% Q-CoV aR, the hypotheses α = 1 − q has size-corrected power 0.68) while Test 2 is not, even with 10,000 out-of-sample predictions (for the 95% T -CoV aR, the unconditional coverage test has size-corrected power 0.07). Table 6 summarises the average hit series statistics for the tests. We see that for Test 1, the average fraction of hits changes from 4.94% in the size simulation to 3.33% in the power simulation, while for Test 2 this changes from 5.17% to 5.67% (in the simulation with 10,000 predictions).

To further illustrate this, the binomial LR statistic for p = 3.33% against p0= 5.00% with 1,000 observations is 6.60, while for p = 5.67% against p0= 5.00% with 500 observations the LR statistic is only 0.43. So Test 1 rejects the null for the average number of hits, while Test 2 does

(30)

not. This may illustrate why Test 1 is able to detect the misspecification most of the time, while Test 2 is not.

Size simulation Power simulation Difference

Test 1: Average no. of hits 49.41 (4.94%) 33.32 (3.33%) -16.09 (-1.61%)

Test 2: Avg. hit series length 50.01 (5.00%) 50.38 (5.04%) 0.37 (0.04%)

1,000 predictions

Test 2: Avg. hits in hit series 2.54 (5.08%) 2.84 (5.63%) 0.30 (0.55%)

1,000 predictions

Test 2: Avg. hit series length 500.23 (5.00%) 498.82 (4.99%) -1.40 (-0.01%)

10,000 predictions

Test 2: Avg. hits in hit series 25.85 (5.17%) 28.26 (5.67%) 2.41 (0.50%)

10,000 predictions

(31)

4.2 Dynamic misspecification

In this section we will investigate whether our test is also able to detect a dynamic misspecification. We will use a Gaussian distribution both in the return simulation and in the CoV aR prediction. The TGARCH specification for both volatility series will remain the same as in the previous section. However, in the CoV aR prediction, we will assume a constant correlation over time. The goal of the Monte Carlo study is to find whether our test is able to detect this misspecification.

4.2.1 Size

The design of the size simulation is exactly the same as in the previous section. The only

difference is that a constant correlation is assumed in the Q-CoV aR prediction and used in the return simulation. In Table 7 the results of the size simulation are shown. Similar to the previous section, we find that the test over-rejects for most test specifications. In Table 8 the results of the size simulation with known parameters are shown. In line with our previous findings, the number of test specifications for which the empirical size is significantly different from the nominal 5% level decreases and most of them become closer to the nominal 5% level.

(32)

95% CoV aR 99% CoV aR Linear test 0.055 (4.13) [3.84] 0.070*** (4.59) [3.84] H0: γ = 0 Logistic test 0.056 (4.10) [3.84] 0.062* (4.27) [3.84] H0: γ = 0 Linear test 0.068*** (8.86) [7.81] 0.089*** (9.69) [7.81] H0: α = (1 − q) β = γ = 0 Logistic test 0.065** (8.58) [7.81] 0.042 (7.24) [7.81] H0: α = F−1(1 − q) β = γ = 0

Table 7: Empirical sizes based on the rejection frequency in the Monte Carlo simulation with 1000 replications. The hypotheses are rejected if the test statistics exceed their 5% asymptotic critical value. The number of stars indicate whether the size is significantly different from 0.050 (*:=10% level, **:=5% level, ***:=1% level). The empirical 5% critical values are reported between parentheses with the corresponding asymptotic 5% critical values between brackets.

95% CoV aR 99% CoV aR Linear test 0.045 (0.055) 0.068*** (0.070***) H0: γ = 0 Logistic test 0.045 (0.056) 0.063* (0.062*) H0: γ = 0 Linear test 0.051 (0.068***) 0.072*** (0.089***) H0: α = (1 − q) β = γ = 0 Logistic test 0.054 (0.065**) 0.030*** (0.042) H0: α = F−1(1 − q) β = γ = 0

Table 8: Empirical sizes based on the rejection frequency in the Monte Carlo simulation with known parameters (1000 replications). The hypotheses are rejected if the test statistics exceed their 5% asymptotic critical value. The number of stars indicate whether the size is significantly different from 0.050 (*:=10% level, **:=5% level, ***:=1% level). The corresponding sizes from the simulation with parameter estimation are reported between parentheses.

(33)

4.2.2 Power

In the power simulation we continue to use the Gaussian distribution and TGARCH volatilities both in the return simulation and in the Q-CoV aR prediction. However, the DCC equation (16) used in the return simulation will be specified as follows:

Qt= 1 − θ1− 1 2(1 − δ)θ2− θ3 ¯ Q + δ θ1t−10t−1+ δ θ2(vt−1v0t−1− ¯N ) + θ3+ (1 − δ) θ1+ 1 2θ2 Qt−1, (52)

where θ1, θ2, and θ3 are the empirically estimated values reported in Section 4.1.1. Note that the stationarity condition is satisfied for all values of δ, and that the unconditional expectation E [Qt] = ¯Q for every δ. Also note that for δ = 0, the correlation is constant, so for δ = 0 the model is correctly specified (the size simulation). For δ = 1, the real world coefficients are used, and for δ > 1 the correlation will be more fluctuating than in the real world situation. Since all coefficients should be larger than or equal to zero the following condition should hold: 0 ≤ δ ≤ δmax≈ 41.

The simulation design will be the same as in the previous sections (with parameter esti-mation), and a constant correlation will be assumed in the Q-CoV aR prediction.

(34)

0 5 10 15 20 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: γ=0 Linear test Logistic test 0 5 10 15 20 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: γ=0 Linear test Logistic test 0 5 10 15 20 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: α=(1-q), β=γ=0 Linear Logistic 0 5 10 15 20 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test

Figure 4: For each test specification, the graph shows the size-corrected power (y-axis) as a func-tion of δ (x-axis) used in the DCC equafunc-tion underlying the simulafunc-tion (52). A constant correlafunc-tion is assumed in the CoV aR prediction, and the results are based on 1000 replications for each point in the graphs. The horizontal grey line represents the nominal 5% level, and the vertical grey line represents the real world situation (δ = 1).

In Figure 4 the size-corrected power is shown as a function of δ, for different test specifications. The empirical critical values from the size simulation are used. We find that in the real world situation (δ = 1), where the correlation is relatively stable, our test is not performing very well. However, in more extreme cases, the test detects the dynamic misspecification most of the times.

(35)

200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: γ=0 Linear test Logistic test 200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: γ=0 Linear test Logistic test 200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 95% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test 200 400 600 800 1000 0 0.2 0.4 0.6 0.8 1 99% CoVaR, H0: α=(1-q), β=γ=0 Linear test Logistic test

Figure 5: For each test specification, the graph shows the size-corrected power (y-axis) as a func-tion of the number of out-of-sample CoV aR predicfunc-tions (x-axis). A constant correlafunc-tion is as-sumed in the CoV aR prediction, while the true underlying dynamics are specified by (52) and δ = 10. The results are based on 1000 replications for each point in the graphs. The horizontal grey line represents the nominal 5% level.

In Figure 5 the size-corrected power is shown as a function of the number of out-of-sample CoV aR predictions. The number of in-sample observations is fixed at 2500 and δ is fixed at 10. For each power simulation, we use the empirical critical values from the corresponding size simulation. As expected, the size-corrected power increases when the number of out-of-sample CoV aR predictions increases.

(36)

In Table 9 the size-corrected power is shown for δ = 1 (i.e. the empirically estimated coefficients are used) using 1000 out-of-sample predictions. We find that our test does not have a lot of power in this situation. Table 10 reports the size-corrected power for δ = 10. We will use this table to which to compare the performance of alternative tests in the next section.

Table 9: Size-corrected power at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. A constant correlation is assumed in the CoV aR predic-tion, while the true underlying dynamics are specified by (52) and δ = 1 (the real world situation). The corresponding sizes are reported between parentheses.

Table 10: Size-corrected power at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. A constant correlation is assumed in the CoV aR pre-diction, while the true underlying dynamics are specified by (52) and δ = 10. The corresponding sizes are reported between parentheses.

(37)

4.2.3 Alternative backtesting methods

In this section, we will assess whether the alternative tests from Section 4.1.3 (Test 1 and Test 2) are able to detect the dynamic misspecification. The same DCC equation, simulation design, and CoV aR prediction method as in the previous section are used. Table 11 shows the size-corrected power for Test 1, using 1000 out-of-sample CoV aR predictions, where δ = 10. Similar to the distributional misspecification, we find that excluding xj_t from the test equation leads to a loss in power (compare the tests with three restrictions in Table 10 to the tests with two restrictions in Table 11).

Note that although the loss in power resulting from excluding xj_tfrom the test equations is relatively small in this situation, in other situations it can result in losing much more power (e.g. Section 4.1) or even losing all power. For example, if we would use the unconditional V aR of i as misspecified CoV aR, while the correlation between Xti and X

j

t is positive, Test 1 would have no power at all while our test would be able to detect the misspecification based on the significance of the γ coefficient in the test regressions.

95% Q-CoV aR 99% Q-CoV aR Linear test H0: α = (1 − q) 0.41 (0.08) 0.37 (0.08) Linear test H0: β = 0 0.68 (0.04) 0.41 (0.07) Linear test 0.79 (0.07) 0.54 (0.08) H0: α = (1 − q) β = 0 Logistic test 0.74 (0.07) 0.65 (0.03) H0: α = F−1_{(1 − q)} β = 0

Table 11: Size-corrected power for Test 1 at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. A constant correlation is assumed in the CoV aR prediction, while the true underlying dynamics are specified by (52) and δ = 10. The corresponding sizes are reported between parentheses.

In Table 12 the size-corrected power for Test 2 (the T -CoV aR test) is shown, where δ = 10. Similar to the distributional misspecification, we do not find any observable power, even for 10,000 out-of-sample predictions.

(38)

1,000 predictions 95% T -CoV aR 99% T -CoV aR

10,000 predictions

Table 12: Size-corrected power for Test 2 at nominal 5% level based on the rejection frequency in the Monte Carlo simulation with 1000 replications. A constant correlation is assumed in the CoV aR prediction, while the true underlying dynamics are specified by (52) and δ = 10. The corresponding sizes are reported between parentheses.

(39)

5 Data

5.1 Illustration data

In this section we discuss the data used to create Figure 1. The analysis is based on the stock returns of 11 large American banks. The return loss of j (X_tj) is the one-day return loss of Citigroup. The one-day financial system return loss, (Xti) is constructed as the average of the return losses of the 10 other banks, weighted by their market capitalisation.

The daily returns are obtained from the Center for Research in Security Prices (CRSP) database and the high-frequency data is obtained from the NYSE Trade and Quote (TAQ) database. Table 13 summarises the 11 banks and their market capitalisation as of January 23, 2017. The reported rank is their rank in all US banks ranked by market capitalisation obtained from Relbanks (2017). As can be seen, the dataset contains all seven largest US banks.

Bank MCap ($bn) Rank Data available HF data available

JPMorgan Chase & Co 299 1 1973 - 2016 1999 - 2013

Wells Fargo & Co 273 2 1973 - 2016 1999 - 2013

Bank of America Corp 228 3 1973 - 2016 1999 - 2013

Citigroup Inc 160 4 1986 - 2016 1999 - 2013

Goldman Sachs Group Inc 97 5 1999 - 2016 1999 - 2013

U.S. Bancorp 87 6 1973 - 2016 1999 - 2013

Morgan Stanley 79 7 1986 - 2016 2006 - 2013

Bank of New York Mellon Corp 47 11 1973 - 2016 1999 - 2013

Capital One Financial Corp 42 12 1994 - 2016 1999 - 2013

Regions Financial Corp 17 21 1973 - 2016 2002 - 2013

Lehman Brothers Holdings Inc - - 1994 - 2008 1994 - 2008

Table 13: Banks used in Figure 1 ranked by their market capitalisation.

We use a 10 minute interval to match and calculate the high-frequency realised statistics. At this frequency, market microstructure noise should have a limited impact. It might be an improvement to use realised kernels such as the one developed by Barndorff-Nielsen, Hansen, Lunde, & Shep-hard (2011). However, because of computational power limitations and because the constructed market return would not be sufficiently frequent, we choose to compute the realised statistics at

(40)

a 10 minute interval.

Following Adrian & Brunnermeier (2016), we include six lagged daily market state variables (Mt−1) in the quantile regressions:

• The change in the three-month Treasury bill rate (change in risk free rate)

• The change in the spread between the ten-year Treasury constant maturity rate and the three-month Treasury bill rate (change in the slope of the yield curve)

• The difference between the three-month LIBOR rate and the three-month Treasury bill rate in the secondary market (TED spread)

• The change in the spread between Moody’s Seasoned Baa Corporate Bond Yield and the ten-year Treasury constant maturity rate (change in the credit spread)

• The daily S&P 500 return (market return)

• The 22-day rolling standard deviation of the daily S&P 500 return (market volatility)

The market state data is obtained from the Federal Reserve Economic Data (FRED) database. The main statistics are reported in Table 14.

Variable Mean St.dev Min Max

∆ 3-month T-Bill rate 0.00 0.10 -1.27 1.34

∆ Yield curve slope 0.00 0.09 -1.04 0.86

TED spread 0.59 0.44 0.09 4.58

∆ Credit spread 0.00 0.04 -0.27 0.58

Market return 0.00 0.01 -0.22 0.11

Market volatility 0.01 0.01 0.00 0.06

(41)

5.2 Empirical application data

In this section we will discuss the data used in the empirical application of our test in Section 6. As a proxy for the financial system, we use the Dow Jones US Financials Index. We will make out-of-sample CoV aR predictions for the five largest US banks by market capitalisation according to Relbanks (2017), which are also the five largest components of the Dow Jones US Financials Index (Wikinvest, 2008). These banks are:

• JPMorgan Chase • Bank of America • Wells Fargo • Citigroup • Goldman Sachs

We obtain daily log-return series ranging from January 2, 2001 to December 1, 2014, which contain 3500 observations each. The Dow Jones US Financials Index return series is obtained from Yahoo Finance and the return series of the banks are obtained from the Center for Research in Security Prices (CRSP) database.

Similar to Adrian & Brunnermeier (2016), we will use the following market state variables in the quantile regression:

• The change in the three-month Treasury bill rate (change in risk free rate)

• The change in the spread between the ten-year Treasury constant maturity rate and the three-month Treasury bill rate (change in the slope of the yield curve)

• The difference between the three-month LIBOR rate and the three-month Treasury bill rate in the secondary market (TED spread)

• The change in the spread between Moody’s Seasoned Baa Corporate Bond Yield and the ten-year Treasury constant maturity rate (change in the credit spread)

• The daily Dow Jones US Financials Index log-return (financial system return)

• The 22-day rolling standard deviation of the daily Dow Jones US Financials Index log-return (financial system volatility)

(42)

The daily three-month Treasury bill rate series, the daily ten-year Treasury constant maturity rate series, the daily TED spread series, and the daily Moody’s Seasoned Baa Corporate Bond Yield series are obtained from the Federal Reserve Economic Data (FRED) database.

Table 15 provides summary statistics for the data used in Section 6.

Daily log-returns Mean St.dev Min Max

Dow Jones US Financials Index 0.00 0.02 -0.18 0.15

JPMorgan Chase 0.00 0.03 -0.23 0.22

Bank of America 0.00 0.03 -0.34 0.30

Wells Fargo 0.00 0.03 -0.27 0.28

Citigroup 0.00 0.03 -0.49 0.46

Goldman Sachs 0.00 0.02 -0.21 0.23

Daily market state variables Mean St.dev Min Max

∆ 3-month T-Bill rate 0.00 0.05 -0.81 0.74

∆ Yield curve slope 0.00 0.07 -0.51 0.74

TED spread 0.44 0.47 0.09 4.58

∆ Credit spread 0.00 0.04 -1.20 1.15

Financial system return 0.00 0.02 -0.18 0.15

Financial system volatility 0.01 0.01 0.00 0.08

(43)

6 Empirical application

In this section we will discuss an empirical application of our test (the Q-CoV aR test with xj_t included in the test regression equations as explained in Section 3.2). We will use our test to compare the performance of the three estimation methods discussed in Section 2.2. As a proxy

for the financial system, we use the Dow Jones US Financials Index. We will make

out-of-sample Q-CoV aR predictions for the five largest US banks by market capitalisation according to Relbanks (2017), which are also the five largest components of the Dow Jones US Financials Index (Wikinvest, 2008).

For each bank, we use the first 2500 observations to estimate the quantile regression and Asymmetric GARCH-DCC parameters. Based on these parameters, we make one-day ahead out-of-sample predictions for the next 10 days and predict the 95% and 99% Q-CoV aR. Then we re-estimate the parameters based on observation 11 - 2510 and predict the CoV aR for day 2511 - 2520. This procedure is repeated until we have 1000 out-of-sample CoV aR predictions for day 2501 - 3500.

Once we have obtained the 95% and 99% out-of-sample CoV aR prediction series for the three estimation methods (the quantile regression, the Asymmetric GARCH-DCC model with Gaussian innovations, and the Asymmetric GARCH-DCC model with Student-t innovations) for each bank, we evaluate the predicted CoV aRs at the observed xjt, we find the hit series, and we apply our tests. The results for each bank are shown in Table 16−20.

(44)

JPMorgan Chase

Quantile regression Asymmetric GARCH-DCC

95% CoV aR Gaussian innovations Student-t innovations

Linear test 3.96 (0.05) ** 1.29 (0.26) 0.26 (0.61) H0: γ = 0 Logistic test 3.91 (0.05) ** 1.27 (0.26) 0.26 (0.61) H0: γ = 0 Linear test 4.73 (0.19) 1.50 (0.68) 0.62 (0.89) H0: α = (1 − q) β = γ = 0 Logistic test 4.65 (0.20) 1.47 (0.69) 0.60 (0.90) H0: α = F−1(1 − q) β = γ = 0 99% CoV aR Linear test 8.73 (0.00) *** 0.17 (0.68) 3.68 (0.06) * H0: γ = 0 Logistic test 7.33 (0.01) *** 0.17 (0.68) 3.33 (0.07) * H0: γ = 0 Linear test 9.61 (0.02) ** 1.53 (0.67) 6.38 (0.09) * H0: α = (1 − q) β = γ = 0 Logistic test 8.51 (0.04) ** 2.01 (0.57) 5.28 (0.15) H0: α = F−1(1 − q) β = γ = 0

Table 16: LR test statistics for the Q-CoV aR of JPMorgan Chase for each test specification with corresponding p-values between parentheses. The number of stars corresponds to the significance level (*:=10% level, **:=5% level, ***:=1% level).

(45)

Bank of America

Quantile regression Asymmetric GARCH-DCC

95% CoV aR Gaussian innovations Student-t innovations

Linear test 35.57 (0.00) *** 8.10 (0.00) *** 2.93 (0.09) * H0: γ = 0 Logistic test 34.11 (0.00) *** 8.04 (0.00) *** 2.95 (0.09) * H0: γ = 0 Linear test 43.15 (0.00) *** 8.87 (0.03) ** 6.49 (0.09) * H0: α = (1 − q) β = γ = 0 Logistic test 41.64 (0.00) *** 8.74 (0.03) ** 5.71 (0.13) H0: α = F−1(1 − q) β = γ = 0 99% CoV aR Linear test 21.31 (0.00) *** 15.57 (0.00) *** 2.55 (0.11) H0: γ = 0 Logistic test 17.01 (0.00) *** 13.57 (0.00) *** 2.38 (0.12) H0: γ = 0 Linear test 23.87 (0.00) *** 23.63 (0.00) *** 2.66 (0.45) H0: α = (1 − q) β = γ = 0 Logistic test 20.62 (0.00) *** 27.17 (0.00) *** 2.59 (0.46) H0: α = F−1(1 − q) β = γ = 0

Table 17: LR test statistics for the Q-CoV aR of Bank of America for each test specification with corresponding p-values between parentheses. The number of stars corresponds to the significance level (*:=10% level, **:=5% level, ***:=1% level).

Backtesting CoVaR

Master’s Thesis