The accuracy of value-at-risk methods

(1)

The Accuracy of Value-at-Risk Methods

Abstract

The aim of this thesis is to evaluate the accuracy of Value-at-Risk (VaR) methods. Three widely used VaR methods namely Historical Simulation, Parametric Approach and Monte Carlo Simulation were used to estimate the daily 95%-VaR and the 99%-VaR of the S&P 500, both in a tranquil financial period and in the recent financial crisis. In order to evaluate the accuracy of the VaR calculation methods, the predictions were backtested using Kupiec’s unconditional coverage test and Christoffersen’s conditional coverage test. In general, VaR methods performed better at the 95% confidence level of VaR than at the 99% confidence level. The different VaR methods led to considerably different VaR estimates. VaR methods generally performed well during the tranquil financial period. However, the performance of almost all VaR methods decreased substantially during the crisis period. Measuring volatility through the GARCH and EGARCH models generally improved the accuracy of VaR estimates. For the S&P 500, the best performing VaR method is Monte Carlo simulation at the 95% confidence level of VaR and student-t approximation at 99% confidence level of VaR, both estimated using conditional volatility from the EGARCH(1,1) model.

Friso Haitsma 10657835

Semester 2, 2015/2016 28/06/2016

Bachelor thesis Economics and Finance Supervised by Rob Sperna Weiland Faculty of Economics and Business (FEB)

(2)

Statement of Originality

This document is written by student Friso Haitsma who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document are original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

1 Introduction

The recent financial crisis has led to a new wave of criticism on risk models used in financial markets. Most risk models predict risk accurately in stable financial periods. However, during the recent financial crisis, the deficiencies of risk models were revealed as none of the risk models were able to predict the recent financial crisis or at least prevent some of its effects (Jorion, 2009). One of the most widely used risk models in financial markets is Value-at-Risk (VaR). Berkowitz and O’Brien (2002) state that VaR is a standard measure of market risk for commercial banks and that it is increasingly used by other financial and even nonfinancial firms. Loosely, VaR summarizes the worst loss over a target horizon that will not be exceeded given a certain confidence level (Jorion, 2007). During the recent financial crisis, VaR has also been subject to criticism. O’Brien and Szerszen (2014) argue that the criticism on VaR is mainly due to the fact that VaR estimates adjust inadequately if market conditions change rapidly, such as in a financial crisis. Therefore, VaR did not predict the high level and frequency of realized losses during the recent financial crisis.

The practical application of VaR began in the early 1990s. In this period, a number of large banks began to measure the market risk of their trading portfolios using VaR (O’Brien & Szerszen, 2014). The use and importance of VaR increased rapidly when The Basel Committee on Banking Supervision (BCBS) implemented new rules for capital requirements in 1998. This rule stated that the capital charges for commercial banks would be based on their VaR. The BCBS allows banks to use their own internal risk measurement models to determine their VaR. According to the BCBS standard, the VaR calculated by banks needs to have a horizon of 10 trading day and a confidence level of 99%. In addition, the 10-day 99%-VaR should be based on an observation period of at least one year of historical data. The capital requirements are then determined by multiplying the VaR of a bank with a factor k of at least 3. A penalty component is added to the capital requirements if backtesting reveals that the bank’s internal model forecasts risk incorrectly (Jorion, 2007). This emphasizes the importance of a correctly computed VaR.

As stated above, critics argue that VaR estimates are inaccurate in periods of financial distress. If VaR only performs well in tranquil financial periods, it is not a very useful risk measurement tool. Therefore, in this thesis, VaR estimates before and during the recent financial crisis will be compared in terms of accuracy. In addition, there are numerous methods to estimate VaR. This thesis will also deal with the difference in accuracy of VaR estimates between these calculation methods.

The VaR calculation methods used in this thesis are Historical Simulation, Parametric Approach and Monte Carlo Simulation. These methods will be used to calculate the daily VaR of the S&P

(5)

500, both in a tranquil financial period and in a period of financial distress. Accurate calculation of VaR requires accurate volatility estimations. In addition to using sample volatility from historical data, the GARCH(1,1), GARCH(2,2) and EGARCH(1,1) models are used to estimate (conditional) volatility. To test the accuracy of VaR estimates of the applied methods in the two periods, the VaR estimates are backtested. Backtesting consists of comparing the VaR for day t with the actual return on day t. Subsequently, the number of VaR exceedances is counted. Kupiec’s unconditional coverage test is performed to test if the number of VaR exceedances is in line with the selected confidence interval of VaR. Christoffersen’s serial independence test is performed to test if VaR exceedances occur in clusters over time. Finally, these tests are combined to yield the test of conditional coverage. Conclusively, the best performing VaR methods can be appointed based on the outcomes of these tests. The results of this thesis confirm that VaR methods generally result in inaccurate VaR estimates during periods of financial distress. In general, VaR methods performed better at the 95% confidence level of VaR than at the 99% confidence level. Measuring volatility through the GARCH and EGARCH models improved the accuracy of VaR estimates considerably. For the S&P 500, the best performing VaR method is Monte Carlo simulation at the 95% confidence level of VaR and student-t approximation at 99% confidence level of VaR, both estimated using conditional volatility from the EGARCH(1,1) model.

The outline of this thesis is as follows. First, based on existing literature, a theoretical background and a summary of previous research on VaR will be given in chapter 2. Thereafter, chapter 3 provides the methodology of the applied methods to estimate VaR, together with a description of the used data. Chapter 4 discusses the results of this study. Finally, chapter 5 draws a conclusion based on these results and provides some suggestions for further studies.

(6)

2 Literature Review

This chapter will first provide the definition of VaR as well as its advantages and disadvantages. Thereafter, the previous research on VaR will be discussed. Finally, a theoretical background on VaR estimation methods, volatility models and backtesting techniques will be provided.

2.1 Definition of VaR

VaR is a method for measuring and controlling exposure to market risk, which is the risk of unexpected changes in prices (Duffie & Pan, 1997). Jorion (2007) defines VaR as the worst loss over a target horizon that will not be exceeded with a given level of confidence. This loss can either be expressed as a currency value or as a return in percentages.

For example, if the 10-day 95%-VaR of a commercial bank is -2%, this means that the loss of this bank over the next ten days will not exceed -2%, with 95% confidence. This example is graphically displayed in the figure below, where the VaR is located at the 5% (= α) lower tail of the return distribution.

Figure 1: VaR

The computation of VaR consists of three steps, determining the target horizon, setting the confidence level and creating a probability distribution of returns. The last step is the most crucial part in computing VaR and will be explained in more detail later in this chapter.

2.2 Advantages and Disadvantages of VaR

Measuring market risk using VaR has its advantages as well as its disadvantages. Engle and Manganelli (2001) describe that the popularity of VaR is mainly due to its conceptual simplicity. They state that VaR is simple and easy to interpret because it reduces the total market risk for any stock or portfolio to just one number. Jorion (2007) states that VaR is applicable for many financial instruments, such as stocks, bonds, currencies and derivatives. Because VaR is expressed in just one number, it is a good measure to compare risk between these financial

(7)

instruments. On the other hand, VaR has to deal with criticism. Jorion (2007) states that in order for VaR to be a good risk measure, it should rely on good assumptions. If, for example, it is assumed that returns are normally distributed, while this is not the case, VaR estimates will be inaccurate. Basak and Shapiro (2001) state that a shortcoming of VaR as a risk management tool is that it only specifies the probability of a loss exceeding a certain level, while it does not say anything about the magnitude of this loss.

Artzner et al. (1999) describe four desirable properties that should hold for any risk measure that is to be used to effectively regulate or manage risk. These properties are: Monotonicity, Translation invariance, Homogeneity and Subadditivity. A risk measure that satisfies all these properties is called a coherent risk measure. Artzner et al. (1999) show that VaR fails to satisfy the Subadditivity property and that VaR is therefore not a coherent risk measure. They describe Subadditivity as follows: 𝜌(𝑊1+ 𝑊2) ≤ 𝜌(𝑊1) + 𝜌(𝑊2), meaning that a merger cannot increase

risk. However, Jorion (2007) argues that VaR does satisfy this property if returns have an elliptical distribution, such as the normal distribution or the student-t distribution. Generally, aggregate bank portfolios look symmetric and close to a normal distribution due to the central limit theorem. This theorem states that, as the number of observations increases, the sum of independent random variables converges to a normal distribution. Jorion (2007) highlights this as the main reason why the banking industry keeps using VaR as the standard measure for market risk.

2.3 Previous Research on the Performance of VaR Methods

There are different methods to estimate VaR, which are all extensively discussed in literature. The empirical research on how well these different methods perform will be discussed below. Beder (1995) was one of the first to empirically evaluate the performance of VaR methods. She created three portfolios and applied historical simulation and Monte Carlo simulation to obtain the VaR of these portfolios with holding periods of one day and two weeks. For historical simulation she selected different sample sizes, while for Monte Carlo Simulation she used different correlation assumptions. Her main finding is that the VaR estimates are largely dependent upon the applied VaR method as well as the selected sample size and the assumptions behind the specific calculation. She concludes that different VaR methods with different assumptions lead to significantly different VaR estimates for the same portfolio. She therefore argues that, although VaR is necessary for an effective risk management program, it is not sufficient to control risk.

One year later, Hendricks (1996) found more consistent VaR estimates than Beder. Hendricks applied VaR methods to 1.000 randomly chosen foreign index portfolios. He estimated VaR using

(8)

historical simulation and a parametric approach, where he assumed that the index returns are normally distributed. He concludes that the parametric approach underestimates VaR because extreme outcomes occur more often than predicted by a normal distribution. The parametric approach particularly underestimated VaR at high confidence levels. Similar, the VaR estimated by historical simulation only performed well at confidence levels that are not higher than 95%. Furthermore, he concludes that the size of market movements is not constant over time. Therefore, he modeled volatility to account for these changes in market risk over time (conditional volatility). He concludes that methods using conditional volatility resulted in more accurate VaR estimates than methods using unconditional volatility.

Bao et al. (2006) calculated the VaR for five Asian Composite Price Indexes using various VaR methods. Their research differs from the research of Beder and Hendricks because Bao et al. compared VaR methods’ performance before, during and after a financial crisis. Amongst others, they compared historical simulation and Monte Carlo simulation. Additionally, they modeled volatility using the GARCH(1,1) model to account for the problem described by Hendricks. They conclude that both historical simulation and Monte Carlo simulation performed well before the crisis. During the crisis their performance decreased, although Monte Carlo Simulation performed slightly better than historical simulation. In the period after the crisis, Monte Carlo simulation again outperformed historical simulation, although historical simulation still had satisfactory estimates in this period. Using the conditional volatility from the GARCH(1,1) model appeared to be useful most of the time for Monte Carlo simulation. Their main conclusion is that most VaR models perform rather poor during the crisis.

Lin and Shen (2006) investigated how well VaR forecasts performed for four stock market indices using the student-t distribution. To test the performance they compared student-t VaR forecasts with a benchmark, namely the normal distribution. They find that using the student-t distribution can improve the VaR estimation, especially with high confidence level of VaR, e.g. 98,5% or higher. This is the case because the student-t distribution has fatter tails than the normal distribution. Therefore, student-t approximation incorporates more extreme outcomes than the normal approximation, which can particularly improve VaR estimates in times of financial distress.

The above described empirically evaluated VaR methods are all theoretical VaR methods. Berkowitz and O’Brien (2002) were the first to analyze the performance of VaR models actually in use. They performed backtests on the internal VaR models of six large U.S. banks. Banks adjust the theoretical VaR methods for their own needs such that it measures their risk accurately. They compared the performance of the banks’ models with the performance of a standard

(9)

GARCH(1,1) model. Because there is more detailed information employed in bank VaR models, it should deliver superior forecasts to the GARCH(1,1) model. However, they conclude that bank VaR models are not better than the GARCH(1,1) model as the latter is better at predicting changes in volatility.

The conclusion that can be drawn from the previous research on VaR is that there is no ideal method for VaR calculations. The most appropriate method depends on the financial instrument for which VaR is needed, the confidence level of VaR, the market conditions, sample size and the underlying assumptions of the method. However, modeling volatility such that it accounts for volatility clustering seems to improve the accuracy of VaR estimates most of the time.

2.4 VaR for Backtesting

As stated in the introduction, the BCBS requires banks to compute the 10-day 99%-VaR. The BCBS finds this VaR the most appropriate for capital charges. However, Jorion (2007) argues that changing the holding period and the confidence level of VaR can increase the power of a backtest on VaR, which is of particular importance when evaluating the accuracy of VaR methods. He states that a VaR with a shorter holding period and a lower confidence level is more appropriate for backtesting. A shorter holding period increases the number of observations, while a lower confidence level increases the expected number of observations in the tail, both increasing the power of a backtest. Hence, the computed VaR in this thesis is the daily VaR with a confidence level of 95%. Previous research on the accuracy of VaR methods showed that the confidence level of VaR influences the performance of some VaR methods, especially normal and student-t approximation. Therefore, computation of the daily VaR with a confidence level of 99% is also included in this thesis. Another component of VaR calculations which influences VaR estimates is the length of the observation period. As mentioned in the introduction, the BCBS requires that VaR calculations are based on an observation period of at least one year of historical data (approximately 250 observations). Jorion (2007) states there is a tradeoff between choosing a short sample and choosing a long sample. There is a substantial estimation error of the sample quantiles, means and variances with a short sample. The problem with a long sample is that this may involve observations that are no longer relevant. According to Jorion most banks use periods between 250 and 750 days. The selected sample length in this thesis is 250 observations, which form a rolling window through the dataset, abandoning the oldest observation and adopting a new observation for every new computation of VaR. This results in a new VaR estimate for each day t in the investigated period, based on the 250 returns prior to day t. The process of the rolling window is showed in figure 2.

(10)

Figure 2: Rolling Window

2.5 VaR Methods

This section will specify the three VaR calculation methods used in this thesis. The theoretical background on the applied methods will be discussed as well as advantages and disadvantages of these methods.

2.5.1 Historical Simulation

From the methods used in this thesis, Historical Simulation is the most simple to implement. Historical Simulation is a nonparametric method because it makes no assumption about the distribution of returns. Instead, Historical Simulation follows the empirical distribution of historical returns. The VaR at day t+1 is the return corresponding to the α-quantile of the empirical return distribution of returns r in the following way (Jorion, 2007):

𝑉𝑎𝑅𝑡+1(1 − 𝛼) = 𝑉𝑎𝑅𝛼(𝑟𝑡, 𝑟𝑡−1, … , 𝑟1)

Historical Simulation has its advantages as well as its disadvantages. Jorion (2007) states that the two main advantages of Historical Simulation are that it accounts for fat tails, if the fat tails are present in the historical data and that there is no need to make assumptions about the distribution of the returns. Furthermore, Historical Simulation is a relatively simple method to implement. On the other hand, Jorion (2007) argues that the main disadvantage of Historical Simulation is that the VaR estimates perform well, only if the past represent the immediate future fairly.

(11)

2.5.2 Parametric Approach

This approach is parametric because it assumes that stock returns form a specific parametric distribution. The parametric distributions used in this thesis are the normal distribution and the student-t distribution. If the stock returns are parametrically distributed, the VaR can be directly derived from the mean and the standard deviation of the stock returns, together with an adjustment factor that is directly related to the selected distribution and the selected confidence level (Jorion, 2007).

2.5.2.1 Normal Distribution

Calculation of VaR assuming a normal distribution of returns is done in the following way (Jorion, 2007):

𝑉𝑎𝑅𝑡+1(1 − 𝛼) = µ𝑡+ 𝜎𝑡∗ 𝑞𝛼

Where µ𝑡 and 𝜎𝑡 are the mean and standard deviation of the returns. 𝑞𝛼 corresponds to the left

quantile of the standard normal distribution at α%.

Jorion (2007) states that this method is easy to implement and computationally fast. The assumption that stock returns are normally distributed is criticized by Duffie and Pan (1997). They state that, for many markets, the actual distribution of stock returns has fatter tails than the normal distribution. Therefore, computing VaR assuming normally distributed returns could underestimate VaR. This underestimation increases with higher confidence levels of VaR (Jorion, 2007).

2.5.2.2 Student-t Distribution

Calculation of VaR assuming a student-t distribution of returns is done in the following way (Jorion, 2007):

𝑉𝑎𝑅𝑡+1(1 − 𝛼) = µ𝑡+ 𝜎𝑡∗ 𝑞𝛼∗ √

𝑣 − 2 𝑣

Where µ𝑡 and 𝜎𝑡 are the mean and standard deviation of the returns. 𝑞𝛼 corresponds to the left

quantile of the student-t distribution with v degrees of freedom at α%. The optimal degrees of freedom v is based on the kurtosis k of the returns in the following way: 𝑣 =4𝑘−6

𝑘−3. If v →∞, the

student-t distribution becomes a normal distribution (Jorion, 2007).

The student-t distribution has fatter tails than the normal distribution, especially with low degrees of freedom. Therefore, if return distributions are fat-tailed, the student-t distribution is a better approximation of the actual return distribution than the normal distribution. In general,

(12)

the student-t distribution will generate higher VaR estimates than the normal distribution. The difference between VaR estimates of the two methods increases with higher confidence levels of VaR (Jorion, 2007).

2.5.3 Monte Carlo Simulation

Monte Carlo Simulation is a method which simulates random future paths for stock returns and thus recreates the returns distribution. This is done by repeatedly simulating random shocks to the stock returns. These shocks are drawn from a prespecified probability distribution (Jorion, 2007). In this thesis, the stock returns are simulated using the Geometric Brownian Motion (GBM), described by Bao et al. (2006) as follows:

𝑑𝑆𝑡

𝑆𝑡

= µ𝑡∗ 𝑑𝑡 + 𝜎𝑡∗ 𝑧𝑡

Where S is the stock price, 𝑧𝑡 is simulated from a standard normal distribution, µ𝑡 and 𝜎𝑡, are the

drift and volatility parameters, respectively. Boa et al. (2004) show that, for the one-step ahead VaR, the GBM can be rewritten to yield the equation for the return r of a stock as follows:

𝑟𝑡= 𝑙𝑛 ( 𝑆𝑡 𝑆𝑡−1 ) = µ𝑡− 𝜎𝑡2 2 + 𝜎𝑡∗ 𝑧𝑡

The repeatedly simulated stock returns will form a normal distribution. Jorion (2007) shows that this process generates a mean of µ𝑡−

𝜎_𝑡2

2 with a standard deviation of 𝜎𝑡. The VaR can be

obtained by calculating the α-quantile of the simulated normal distribution.

Jorion (2007) states that Monte Carlo Simulation is the most powerful method to compute VaR because of its flexibility. It is flexible because of the different simulation methods that can be used to create return paths, which can incorporate, for example, fat tails and extreme scenarios. Therefore, the method can be adjusted for the different needs of different users of VaR. On the contrary, Jorion specifies that the main disadvantage is the computational time, which makes it an expensive method to implement.

2.6 Modeling Volatility

Apart from historical simulation, all VaR calculation methods require volatility as an input variable. These methods only produce accurate VaR estimates if volatility is measured accurately as well. A widely employed method to model volatility is the moving average approach (sample volatility), where volatility is measured from a rolling window of T observations in the following way (Duffie & Pan, 1997):

(13)

𝜎𝑡,𝑇2 = 1 𝑇 − 𝑡 ∑ (𝑟𝑠− µ𝑡,𝑇) 2 𝑇 𝑠=𝑡+1 Where µ𝑡,𝑇= 𝑟_𝑡+1+⋯+𝑟_𝑇

𝑇−𝑡 . Jorion (2007) states that this method has some serious drawbacks. He

argues that all observations receive the same weight, while recent information may be more relevant than older observations. Another problem of the moving average approach is that this method does not account for one of the stylized facts frequently observed in financial time series, namely the occurrence of stochastic volatility. Duffie and Pan (1997) define stochastic volatility as the random change of volatility over time, usually with persistence. By persistence they mean that high recent volatility implies relatively high volatility predictions in the near future and vice versa. This is also known as volatility clustering. To deal with this problem, Bollerslev (1986) introduced the Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model. In this model, volatility is not only based on the squared difference between returns and means, but also on previous volatility. GARCH(p,q) is modelled in the following way, as described by Bollerslev (1986): 𝑟𝑡= 𝜎𝑡∗ 𝜀𝑡 𝜎𝑡2= 𝛼𝑖+ ∑ 𝛽𝑖𝑟𝑡−𝑖2 𝑞 𝑖=1 + ∑ 𝛾𝑖𝜎𝑡−𝑖2 𝑝 𝑖=1

Where 𝜎𝑡2 is the variance on day t, conditional on the lagged return prior to day t: 𝑟𝑡−𝑖2 , the lagged

conditional variance prior to day t: 𝜎_𝑡−𝑖2 and the parameters α, β and γ. Of the parameters, γ is the key one, namely the persistence parameter. A high γ implies a high dependence of future volatility on past volatility, which indicates volatility clustering, and vice versa (Duffie & Pan, 1997). p and q are the number of lags included in the model.

The most widely used GARCH model is the GARCH(1,1) model, where the conditional volatility for day t is based on one lagged return and one lagged conditional variance. The model is stationary if the sum of β and γ is less than 1. If the model is stationary, the conditional volatility will mean revert to its unconditional (long-term) volatility: 𝜎̅ which is given by (Duffie & Pan, 1997):

𝜎̅ = √ 𝛼 1 − 𝛽 − 𝛾

Although the GARCH(p,q) model accounts for the occurrence of volatility clustering, it does not deal with another stylized fact frequently observed in financial time series, namely the leverage

(14)

effect. Nelson (1991) states that the GARCH(p,q) model rules out the negative correlation between current returns and future returns volatility. He argues that losses usually increase future volatility more than gains would do. To simultaneously account for volatility clustering and the leverage effect, Nelson (1991) proposed the Exponential GARCH (EGARCH) model. The EGARCH model differs from the standard GARCH model because EGARCH allows the sign and the magnitude of return r to have a different effect on the conditional volatility 𝜎𝑡. Conditional

volatility from the EGARCH(1,1) model is given by (Duffie & Pan, 1997):

𝑙𝑛(𝜎𝑡2) = 𝛼 + β1(| 𝑟𝑡−1 𝜎𝑡−1 | − √2 𝜋) + β2 𝑟𝑡−1 𝜎𝑡−1 + 𝛾ln (𝜎𝑡−12 )

The main issue of estimating 𝜎𝑡 through the GARCH and EGARCH models is estimation of the

parameters α, β and γ. The parameters of the model can be found by maximizing the log-likelihood function. The exact form of the log-log-likelihood function depends on the distribution of 𝜀𝑡. If 𝜀𝑡 is normally distributed, the log-likelihood function takes the following form (Jorion,

2007): ∑ (𝑙𝑛 ( 1 √2𝜋𝜎_𝑡2) − 𝑟𝑡2 2𝜎_𝑡2) 𝑇 𝑡=1

If 𝜀𝑡 has a student-t distribution with v > 2, the following log-likelihood function needs to be

maximized, as described by Alberg et al. (2008):

𝑇 (𝑙𝑛Γ(𝑣 + 1 2 )− 𝑙𝑛( 𝑣 2)− 𝑙𝑛(𝜋(𝑣 − 2)) 2 ) − 1 2∑ (𝑙𝑛(𝜎𝑡 2_{) + (1 + 𝑣)𝑙𝑛 (1 +} 𝑟𝑡 2 𝜎_𝑡2_{(𝑣 − 2)})) 𝑇 𝑡=1

Nelson (1991) describes the following log-likelihood function, if 𝜀𝑡 has a Generalized Error

Distribution (GED): ∑ (𝑙𝑛 (𝑣 𝜆) − 1 2| 𝑟𝑡 𝜎𝑡𝜆 | 𝑣 − (1 + 𝑣−1) ln(2) − 𝑙𝑛 (Γ(1 𝑣)) − 1 2𝑙𝑛(𝜎𝑡 2₎₎ 𝑇 𝑡=1

Where Γ(. ) Is the gamma function and 𝜆 = (2−2 𝑣⁄ Γ(1 𝑣⁄ )

Γ(3 𝑣⁄ ) )

1 2

. Nelson (1991) states that the log-likelihood function with 𝜀𝑡 from a GED needs to be maximized when using the EGARCH model.

He argues that some distributions of 𝜀𝑡, such as Student-t with finite degrees of freedom,

(15)

2.7 Backtesting VaR

VaR models are only useful if they estimate risk well. To test how well VaR models perform, various backtesting techniques have been developed. Backtesting consists of comparing VaR forecasts for day t with the actual return on day t. A VaR exceedance occurs when the actual loss is higher than the VaR forecast (Jorion, 2007). This chapter will specify three backtesting techniques. First the unconditional coverage test will be explained, which tests if the number of VaR exceedances is in line with the selected confidence level of VaR. Second, the test of serial independency of VaR exceedances will be specified. Finally, the conditional coverage test, which is a combination of the aforementioned tests, will be explained.

The choice of the confidence level for the decision to reject a VaR method is not related to the selected confidence level of the VaR. In this thesis, the confidence level for the decision rule is 95%, meaning that the decision to reject a VaR method, for both the 95%-VaR and 99%-VaR, is made with 95% confidence.

2.7.1 Unconditional Coverage

To evaluate if the number of VaR exceedances is in line with the selected confidence level of VaR, Kupiec (1995) developed the unconditional coverage test. For example, if a 99%-VaR is computed, it is expected that a VaR exceedance occurs in 1% of the observations. In this example, the unconditional coverage test can be used to test if the proportion of actual VaR exceedances is significantly higher or lower than 1%. The test statistic is given by the following log-likelihood ratio, as described by Kupiec (1995):

𝐿𝑅𝑢𝑐 = −2 ∗ 𝑙𝑛((1 − 𝛼)𝑇−𝑥∗ 𝛼𝑥) + 2 ∗ 𝑙𝑛 ((1 − 𝑥 𝑇) 𝑇−𝑥 ∗ (𝑥 𝑇) 𝑥 )

Where T is the number of observations, x is the number of VaR exceedances and α is one minus the confidence level of VaR. Under the null hypothesis that α is the true probability of a VaR exceedance, 𝐿𝑅𝑢𝑐 is asymptotically chi-squared distributed with one degree of freedom. With a

confidence level of 95%, the null hypothesis is rejected for 𝐿𝑅𝑢𝑐 > 3,841.

2.7.2 Serial Independence

The unconditional coverage test does not account for the clustering of exceedances while this could also invalidate the model. It could mean, for instance, that the market is experiencing increased volatility, which is not captured by the VaR model (Jorion, 2007). That is why Christoffersen (1998) extended the 𝐿𝑅𝑢𝑐 statistic with the 𝐿𝑅𝑖𝑛𝑑 statistic to test if VaR

(16)

influences the probability of exceeding VaR on day t+1. The test statistic is given by (Christoffersen, 1998):

𝐿𝑅𝑖𝑛𝑑= −2 ∗ ln ((1 − 𝜋)(𝑇00+𝑇10)∗ 𝜋(𝑇01+𝑇11)) + 2 ∗ 𝑙𝑛((1 − 𝜋0)𝑇00∗ 𝜋0𝑇01∗ (1 − 𝜋1)𝑇10∗ 𝜋1𝑇11)

Where π is the total number of exceedances divided by the total number of observations. 𝜋𝑖 is

the proportion of observed VaR exceedances, conditional on state i the previous day. Where i = 0 if there was no exceedance on the previous day and i = 1 if there was an exceedance on the previous day. 𝑇𝑖𝑗 is the number of days in which state j occurred in one day conditional on state i

the previous day. Where i and j take the value 0 if there was no exceedance and i and j take the value 1 if there was an exceedance. So, for example, 𝜋0 is the proportion of observed VaR

exceedances, conditional on that VaR was not exceeded on the previous day and 𝑇01 is the

number of days that VaR is exceeded, conditional on that VaR was not exceeded on the previous day. Similar to 𝐿𝑅𝑢𝑐, 𝐿𝑅𝑖𝑛𝑑 is asymptotically chi-squared distributed with one degree of freedom.

With a confidence level of 95%, the null hypothesis of serial independence is rejected for 𝐿𝑅𝑖𝑛𝑑 > 3,841.

2.7.3 Conditional Coverage

Combining the unconditional coverage statistic with the serial independence statistic results in the conditional coverage statistic in the following way (Christoffersen, 1998):

𝐿𝑅𝑐𝑐= 𝐿𝑅𝑢𝑐+𝐿𝑅𝑖𝑛𝑑

𝐿𝑅𝑐𝑐 is asymptotically chi-squared distributed with two degrees of freedom. With a confidence

level of 95%, the joint null hypothesis of unconditional coverage and serial independence is rejected for 𝐿𝑅𝑐𝑐 > 5,991.

(17)

3 Methodology

This chapter will focus on the research method. First the selection process of the data will be described and descriptive statistics of the data will be given. Second, the different computation methods of VaR will be specified. Backtesting procedures will be described at the end of this chapter.

3.1 Data

The VaR calculations in this thesis are based on the returns of the daily stock prices of the S&P 500. While VaR is mostly used by commercial banks, it can also be used for providing information about potential losses of stock portfolios (Jorion, 2007). The S&P 500 is used instead of commercial banks because there is more data available on historic day-to-day stock prices than there is on day-to-day profits and losses of commercial banks.

Approximately seven years of daily stock prices were retrieved, from the 8th_{of August 2003 until}

the 6th_{of August 2010. The seven years of daily stock prices are divided into three periods, a}

sample period, a tranquil period and a crisis period. As described chapter 2, the VaR calculation methods require one year of past observations to calculate VaR. Therefore, the sample period is necessary to be able to calculate the VaR of the first day in the investigated period. The observations from the 8th_{of August 2003 until the 6}th_{of August 2004 constitute the first sample}

period. The period from the 7th_{of August 2004 until the 6}th_{of August 2007 reflects the tranquil}

period. The period from the 7th_{of August 2007 until the 6}th_{of August 2010 is selected as the}

crisis period. The 7th_{of August 2007 is selected as the starting date of the recent financial crisis}

because of the sudden rise of the TED spread that day. The TED spread is the difference between the risky LIBOR rate and the risk-free US Treasury bill rate. In times of uncertainty, the LIBOR rate rises because banks charge higher interest for unsecured loans, while the Treasury bond rate falls because Treasury bonds become safer relative to other investments. Because of both reasons the TED spread rises in times of a financial crisis (Brunnermeier, 2009). The sharp rise of the TED spread around the 7th_{of August 2007 is shown in figure 1.}

(18)

Figure 3: TED spread

3.1.1 Summary Statistics

Figure 4 shows the graph of the daily returns of the S&P 500 over the total period. From the graph, it can easily be seen that the volatility increased with persistence during the financial crisis, justifying the use of the GARCH volatility.

Figure 4: Daily returns of the S&P 500

Table 1 provides the summary statistics of the daily returns of the S&P 500 over the total period as well as over the two sub periods.

(19)

Table 1: Summary statistics Tranquil Period 07/08/2004 -06/08/2007 Crisis Period 07/08/2007 -06/08/2010 Total 08/08/2003 -06/08/2010 Observations 754 757 1.761 Minimum -3,5345% -9,4695% -9,4695% Maximum 2,3864% 10,9572% 10,9572% Mean 0,0427% -0,0355% 0,0078% Median 0,0838% 0,0802% 0,0835% Standard Deviation 0,6810% 1,9352% 1,3729% Skewness -0,3335 -0,1484 -0,2683 Kurtosis 4,7037 8,3786 14,3263

The normal distribution and the student-t distribution have a skewness of 0. In both periods, the return distribution of the S&P 500 has non-zero skewness, meaning that the distribution is not symmetric around its mean. Instead, the distributions are negatively skewed, which indicates long left tails and hence generates large negative values. The kurtosis of a normal distribution is 3, while the kurtosis of the student-t distribution is conditional on its degrees of freedom. In both periods the kurtosis is greater than 3, leading to a greater likelihood of large values than estimated by the normal distribution. Therefore, the student-t distribution might better represent the distribution of the S&P 500 returns because it generates fatter tails than the normal distribution for low degrees of freedom. Figure 5 and 6 display histograms of the returns in the tranquil period and the crisis period, respectively. A normal distribution curve is added to both figures. Indeed, the normal distribution does not seem to fit the return distribution well. The return distributions have higher peaks in both periods. The occurrence of more extreme returns than predicted by the normal distribution is particularly visible in the crisis period.

(20)

3.2 VaR – Historical Simulation

From the methods used in this thesis, Historical Simulation is the easiest to implement. Historical Simulation makes no assumption about the distribution of stock returns. Instead, Historical Simulation follows the empirical distribution from historical data. The VaR is calculated for each day using Excels PERCENTILE function, which returns the kth_{percentile from}

a sample. As described in the chapter 2, the sample is a rolling window consisting of 250 returns. For the 95%-VaR, k = 5%. The 95%-VaR for day t is therefore the 5th_{percentile from the 250}

returns prior day t. Because the 5th_{percentile of 250 observations corresponds to the 12,5}th

lowest return, Excel interpolates between the 12th_{and 13}th_{lowest returns. For the 99%-VaR, k =}

1%. This percentile corresponds to the 2,5th_{lowest return, so the 99%-VaR is also calculated by}

interpolation. Subsequently, the computed VaR’s are backtested by comparing the VaR for day t with the actual return on day t. Finally, the number of VaR exceedances is counted.

3.3 VaR – Normal Distribution

This method assumes that stock returns are normally distributed. In order to calculate VaR assuming a normal distribution, the distribution parameters, mean µ𝑡 and standard deviation 𝜎𝑡,

are needed. With µ𝑡, 𝜎𝑡 and the quantile value of the normal distribution 𝑞𝛼, VaR can be

calculated using the equation described in chapter 2. For the 95%-VaR, 𝑞5% = -1,645. For the

99%-VaR, 𝑞1% = -2,326. µ𝑡 is the sample mean from the rolling window of the past 250

observations. 𝜎𝑡 is estimated four times. First as the sample standard deviation from the rolling

window of the past 250 observations. Second, 𝜎𝑡 is estimated by the GARCH(1,1), GARCH(2,2)

and EGARCH(1,1) models. This means that this method results in four VaR estimates for every day, one based on sample volatility and three based on conditional volatility. Then, similar to Historical Simulation, the computed VaR’s are backtested and the number of VaR exceedances is counted.

3.4 VaR – Student-t Distribution

This method assumes that stock returns follow a student-t distribution. In order to calculate VaR assuming a student-t distribution, the distribution parameters, mean µ𝑡 and standard deviation

𝜎𝑡 are needed, as well as de degrees of freedom of the distribution 𝑣𝑡. With µ𝑡, 𝜎𝑡, 𝑣𝑡 and the

quantile value of the student-t distribution 𝑞𝛼, VaR can be calculated using the equation

described in chapter 2. Similar to the normal distribution, µ𝑡 is estimated from the rolling

window and 𝜎𝑡 is estimated from the rolling window as well as through the GARCH and EGARCH

models. 𝑞𝛼 represents the value from the student-t distribution with 𝑣𝑡 degrees of freedom at

the α-quantile. The optimal degrees of freedom is based on the kurtosis of the return distribution, following the equation described in chapter 2. Because the kurtosis of the return

(21)

distribution changes over time, so does the optimal degrees of freedom. Therefore, the degrees of freedom for every day is based on the sample kurtosis from the rolling window of the past 250 observations. Meaning that for every day, the VaR could be based on a student-t distribution with different degrees of freedom. Note that, with high degrees of freedom, the student-t distribution becomes a normal distribution. Therefore, some VaR estimates will be the same as under the assumption of a normal distribution. Again, the computed VaR’s are backtested and the number of VaR exceedances is counted.

3.5 VaR – Monte Carlo Simulation

Monte Carlo Simulation recreates the distribution of stock returns. The returns are simulated using the Geometric Brownian Motion equation, as described in the chapter 2. µ𝑡 is the sample

mean from the rolling window of the past 250 observations. 𝜎𝑡 is estimated from the rolling

window as well as through the GARCH and EGARCH models. The random shock 𝜀𝑡 is computed

by taking the standard normal value of a random number between 0 and 1. This random number is generated by Excel’s RAND function. The return is simulated a 1.000 times for every day in the investigated period. The VaR is then calculated for each day, similar to Historical Simulation, using Excels PERCENTILE function. For the 95%-VaR this function will return the 50th_smallest

return from the 1.000 simulated returns. For the 99%-VaR this will be the 10th_{smallest return.}

Again, the results are backtested and the number of VaR exceedances is counted.

3.6 GARCH and EGARCH

As described above, the volatility used to estimate VaR assuming a parametric distribution and using Monte Carlo simulation is also estimated by the GARCH and EGARCH models. The models used in this thesis are GARCH(1,1), GARCH(2,2) and EGARCH(1,1). As described in chapter 2, the first step of calculating conditional volatility through these models is estimation of the parameters α, β and γ. The parameters are found by maximizing the log-likelihood function, which is dependent on the distribution of 𝜀𝑡. As stated before, the kurtosis of a normal

distribution is 3. The observed kurtosis of the returns, both in the tranquil period and the crisis period, is higher than 3. Therefore, the log-likelihood function with student-t distributed 𝜀𝑡 is

maximized. Following the comments of Nelson (1991) described in chapter 2, the log-likelihood function with GED 𝜀𝑡 is maximized for the EGARCH(1,1) model. The degrees of freedom v is

based on the sample kurtosis over the total period. The maximization of the log-likelihood functions is executed in STATA. With the resulting parameters, the conditional volatility can be calculated for every day in the investigated period, following the equations described in chapter 2. In this thesis, the parameters are assumed to be constant over time and are therefore estimated once.

(22)

3.7 Backtesting VaR

In order to evaluate which VaR calculation method is the most accurate, the methods are backtested. As stated before, backtesting consists of comparing the VaR for day t, calculated using the above described methods, to the actual return on day t. A VaR exceedance occurs when the loss on day t is higher than the estimated VaR for day t. First, the VaR exceedances are counted which results in the number and proportion of VaR exceedances. The test statistic for the unconditional coverage test is subsequently calculated using the number and proportion of VaR exceedances, following the equation described in chapter 2. Thereafter, the sequence of the VaR exceedances is determined. With the sequence of VaR exceedances, the test statistic for the serial independence test is calculated, using the equation described in chapter 2. Finally, the test statistics of the unconditional coverage test and the serial independence test are combined which yields the test statistic for the conditional coverage test. The above described procedure is performed for all VaR methods in both periods and at both the 95% and 99% confidence level of VaR. Based on these test statistics, the decision is made to reject a VaR method or not. The values of the test statistics can be interpreted as follows: the higher the test statistic, the worse is the corresponding VaR method. Therefore, the VaR method with a test statistic closest to zero has the best performance in the corresponding tests.

(23)

4 Results

This chapter will provide the results of the used VaR methods. First, the number and proportion of VaR exceedances will be given. Second, the results of the unconditional coverage and serial independence tests will be discussed. Finally, a discussion on the outcomes of the conditional coverage test will be provided. Graphs including the returns of the S&P 500 and the daily VaR’s of all the used methods and for both confidence levels are shown in the appendix.

4.1 Number and Proportion of VaR Exceedances

Table 2 provides the number of VaR exceedances of the 95%-VaR and the 99%-VaR, respectively. The number of VaR exceedances as a percentage of the observations per period is shown between parentheses. The target number of VaR exceedances is given by α * number of observations per period.

Table 2: Number and proportion of VaR exceedances

95%-VaR 99%-VaR

VaR Method Tranquil Crisis Tranquil Crisis

Target 37,70 (5%) 37,85 (5%) 7,54 (1%) 7,57 (1%) Historical Simulation 38 (5,04%) 50 (6,61%) 12 (1,59%) 15 (1,98%) Normal 42 (5,57%) 58 (7,66%) 15 (1,99%) 34 (4,49%) Normal – GARCH(1,1) 33 (4,38%) 52 (6,87%) 9 (1,19%) 15 (1,98%) Normal – GARCH(2,2) 35 (4,64%) 49 (6,47%) 5 (0,66%) 22 (2,91%) Normal – EGARCH(1,1) 36 (4,77%) 48 (6,34%) 10 (1,33%) 17 (2,25%) Student-t 43 (5,70%) 61 (8,06%) 13 (1,72%) 27 (3,57%) Student-t – GARCH(1,1) 34 (4,51%) 53 (7,00%) 4 (0,53%) 13 (1,72%) Student-t – GARCH(2,2) 35 (4,64%) 52 (6,87%) 3 (0,40%) 16 (2,11%) Student-t – EGARCH(1,1) 38 (5,04%) 50 (6,61%) 7 (0,93%) 13 (1,72%) Monte Carlo 39 (5,17%) 58 (7,66%) 14 (1,86%) 33 (4,36%)

Monte Carlo – GARCH(1,1) 32 (4,24%) 49 (6,47%) 7 (0,93%) 15 (1,98%) Monte Carlo – GARCH(2,2) 34 (4,51%) 50 (6,61%) 5 (0,66%) 21 (2,77%) Monte Carlo – EGARCH(1,1) 34 (4,51%) 48 (6,34%) 10 (1,33%) 19 (2,51%)

For the 95%-VaR, all methods seem to give satisfactory results in the tranquil period, as all proportions of exceedances have values around the α of 5%. Deviation from the α is higher for the 99%-VaR during this period. The proportion of exceedances increases for all methods during the crisis period. On the other hand, using conditional volatility from the GARCH and EGARCH models decreases the number of VaR exceedances for the applicable methods in both periods.

(24)

These decreases appear to improve VaR estimates in the crisis period. This can easily be seen in the graphs in the appendix. VaR estimates of methods using sample volatility adjust too slowly during the high volatility period at the end of 2008, leading to many VaR exceedances during this period. The lines representing the VaR estimates with GARCH and EGARCH volatility respond more quickly to the increased volatility in this period. However, during the tranquil period, the conditional volatility models lead to an overestimation of VaR. As a result, most proportions of exceedances of conditional volatility methods have values below the applicable α’s in the tranquil period. Furthermore, normal approximation seems to outperform student-t approximation for the 95%-VaR, as six out of the eight proportions of exceedances are closest to the α of 5%. On the contrary, student-t approximation appears to outperform normal approximation for the 99%-VaR, with also six out of the eight proportions of exceedances closest to the α of 1%. This is in line with the comments of Jorion (2007) on both methods, described in chapter 2. He stated that, with higher confidence levels of VaR, the underestimation of normal approximation increases as well as the difference between normal and student-t approximation. Surprisingly, despite the relatively easy implementation of the method, historical simulation seems to perform well for both confidence levels of VaR. Monte Carlo simulation with sample volatility appears to result in poor VaR estimates. However, conditional volatility from the GARCH and EGARCH models increases the performance of Monte Carlo simulation substantially. Only cautious conclusions can be drawn from table 2. The number and proportion of VaR exceedances close to the target level may indicate a well performing model. However, table 2 does not indicate anything about the significance of these numbers and proportions. Furthermore, table 2 shows no information about the possible clustering of VaR exceedances, which could also violate the VaR method. More meaningful conclusions can be drawn from the statistical tests presented next.

(25)

4.2 Unconditional Coverage

Table 3 provides the results of the unconditional coverage test for the 95%-VaR and the 99%-VaR, respectively. The values in bold are below 3,841 and therefore pass the test on unconditional coverage.

Table 3: Unconditional coverage test

95%-VaR 99%-VaR

Historical Simulation 0,0025 3,7456 2,2591 5,7296 Normal 0,4987 9,7801 5,7896 50,2306 Normal – GARCH(1,1) 0,6427 5,0120 0,2689 5,7296 Normal – GARCH(2,2) 0,2083 3,1763 0,9808 18,3610 Normal – EGARCH(1,1) 0,0819 2,6507 0,7354 8,7658 Student-t 0,7518 12,6770 3,2829 30,3169 Student-t – GARCH(1,1) 0,3947 5,7075 2,0253 3,2391 Student-t – GARCH(2,2) 0,2083 5,0120 3,5779 7,1838 Student-t – EGARCH(1,1) 0,0025 3,7456 0,0400 3,2391 Monte Carlo 0,0467 9,7801 4,4635 47,1856

Monte Carlo – GARCH(1,1) 0,9541 3,1763 0,0400 5,7296

Monte Carlo – GARCH(2,2) 0,3947 3,7456 0,9808 16,2360

Monte Carlo – EGARCH(1,1) 0,3947 2,6507 0,7354 12,2846

Table 3 confirms the cautious conclusions drawn from the proportions of VaR exceedances in table 2. Indeed, VaR methods perform well during the tranquil period, as only the normally approximated and the Monte Carlo simulated 99%-VaR with sample volatility do not pass the test on unconditional coverage. The decreased performance of all the VaR methods in the crisis period is also shown in table 3, as all test statistics increased during this period. Table 3 also shows that normal approximation performs better at the 95% confidence level while student-t approximation performs better at the 99% confidence level. Furthermore, almost all test statistics decrease when using GARCH of EGARCH volatility instead of sample volatility, meaning that using these conditional volatility models improves VaR estimates for almost all applicable methods. The difference between the GARCH and EGARCH models is less clear and seems to depend upon the selected VaR method. From the GARCH and EGARCH models, using GARCH(1,1) and EGARCH(1,1) volatility generally results in the most accurate VaR estimates.

(26)

4.3 Serial Independence

Table 4 provides the results of the serial independence test for the 95%-VaR and the 99%-VaR, respectively. The values in bold are below 3,841 and therefore pass the test on serial independence. The value of the test statistic is zero if there are no consecutive VaR exceedances in the entire period.

Table 4: Serial independence test

95%-VaR 99%-VaR

Historical Simulation 1,9744 0,8819 0 0 Normal 1,1149 0,0535 1,0841 0,1475 Normal – GARCH(1,1) 0,2108 2,8763 0 0 Normal – GARCH(2,2) 0,0894 2,2447 0 0 Normal – EGARCH(1,1) 0,0484 2,0504 0 0 Student-t 0,9421 0,2132 0 0,0015 Student-t – GARCH(1,1) 0,1434 3,1030 0 0 Student-t – GARCH(2,2) 0,0894 2,8763 0 0 Student-t – EGARCH(1,1) 0,0041 2,4471 0 0 Monte Carlo 0,3613 0,0535 1,2964 0,2157

Monte Carlo – GARCH(1,1) 0 2,2447 0 0

Monte Carlo – GARCH(2,2) 0,1434 2,4471 0 0

Monte Carlo – EGARCH(1,1) 0,2295 0,5604 0 0

As shown in table 4, clustering of VaR exceedance is not problematic for the applied VaR methods, as all methods pass the test on serial independence. Similar to the unconditional coverage test, using GARCH and EGARCH volatility improves the performance in the serial independence test for almost all methods. Only exceptions are the methods used to calculate the 95%-VaR in the crisis period. For these methods, all the values of the test statistic increased when using GARCH or EGARCH volatility instead of sample volatility, indicating more clustered VaR exceedances. Monte Carlo simulation with EGARCH(1,1) volatility appears to be the best VaR calculation method to avoid the clustering of VaR exceedances.

(27)

4.4 Conditional Coverage

Table 5 provides the results of the conditional coverage test for the 95%-VaR and the 99%-VaR, respectively. The values in bold are below 5,991 and therefore pass the test on unconditional coverage.

Table 5: Conditional coverage test

95%-VaR 99%-VaR

Historical Simulation 1,9769 4,6275 2,2591 5,7296 Normal 1,6136 9,8337 6,8738 50,3781 Normal – GARCH(1,1) 0,8535 7,8883 0,2689 5,7296 Normal – GARCH(2,2) 0,2977 5,4210 0,9808 18,3610 Normal – EGARCH(1,1) 0,1303 4,7011 0,7354 8,7658 Student-t 1,6938 12,8902 3,2829 30,3184 Student-t – GARCH(1,1) 0,5381 8,8105 2,0253 3,2391 Student-t – GARCH(2,2) 0,2977 7,8883 3,5779 7,1838 Student-t – EGARCH(1,1) 0,0066 6,1928 0,0400 3,2391 Monte Carlo 0,4080 9,8337 5,7598 47,4013

Monte Carlo – GARCH(1,1) 0,9541 5,4210 0,0400 5,7296

Monte Carlo – GARCH(2,2) 0,5381 6,1928 0,9808 16,2360

Monte Carlo – EGARCH(1,1) 0,6242 3,2111 0,7354 12,2846

Table 5 again confirms the strong performance of VaR methods in the tranquil period. In this period, the VaR methods perform particularly well at the 95% confidence level, as all test statistics have values far below the critical value of 5,991. Using a confidence level of 99% leads to decreased performance of most VaR methods in the tranquil period. The only method that is rejected in the tranquil period is the normal approximation with sample volatility at the 99% confidence level of VaR. The number of VaR methods that is rejected in the crisis period is substantially higher. This supports the research of Bao et al. (2006). They concluded that most VaR methods perform rather poor during a financial crisis while VaR methods result in accurate VaR estimates during stable financial periods. Table 5 also confirms that normal approximation performs better at the 95% confidence level while student-t approximation performs better at the 99% confidence level. This is in accordance with the earlier findings of Lin and Shen (2006) who concluded that student-t approximation particularly outperformed normal approximation for confidence levels higher than 98,5%. The conditional coverage test shows that using conditional volatility from the GARCH and EGARCH models improves VaR estimates

(28)

substantially for almost all methods. Surprisingly, historical simulation passes the test on conditional coverage in both periods and for both confidence levels of VaR. The only other method with the same result is Monte Carlo simulation with GARCH(1,1) volatility. It must be noted that both methods resulted in too many VaR exceedances to pass the test on unconditional coverage in the crisis period at the 99% confidence level. Both methods pass the test on conditional coverage mainly because of the zero consecutive VaR exceedances in this period. Most financial institutions that use VaR choose only one confidence level for their VaR. It might therefore be more important to separately look at the best performing VaR method for each confidence level of VaR. For the 95%-VaR, Monte Carlo simulation with EGARCH(1,1) volatility results in the most accurate VaR estimations. Numerous other methods resulted in lower test statistics in the tranquil period. However, Monte Carlo simulation with EGARCH(1,1) volatility still has a relatively low test statistic in this period. More important, this method resulted in the lowest test statistic in the crisis period. The student-t approximation, also with EGARCH(1,1) volatility, is the best performing method at the 99% confidence level of VaR. This method resulted in the lowest test statistic in both periods. These best performing methods for the 95% and the 99% VaR’s also separately pass the tests on unconditional coverage and serial independence.

(29)

5 Conclusion

In this chapter, a conclusion based on the findings in the previous chapter will be provided. This chapter will also discuss the shortcomings of this thesis as well as possible suggestions for future research.

In this thesis, the performance of some of the most widely used VaR methods was evaluated. The used VaR methods are historical simulation, normal approximation, student-t approximation and Monte Carlo simulation. Furthermore, besides using sample volatility, the GARCH(1,1), GARCH(2,2) and EGARCH(1,1) models were used to measure conditional volatility. The above described VaR and volatility models were used to calculate the daily VaR of the S&P 500 with confidence levels of 95% and 99% over the period of August 2004 – August 2010. Besides the difference in accuracy between these methods, the methods were also compared before and during the recent financial crisis. To evaluate the accuracy of the applied VaR methods, tests on unconditional coverage, serial independence and conditional coverage were performed.

Previous research on the performance of VaR methods showed that VaR estimates are dependent upon the selected confidence interval of VaR, the method used to calculate VaR and the market conditions. This thesis confirms these findings. In general, VaR methods performed better at the 95% confidence level of VaR than at the 99% confidence level. Furthermore, different VaR methods led to considerably different VaR estimates. VaR methods generally performed well during the tranquil financial period. However, the performance of almost all VaR methods decreased substantially during the crisis period. Previous studies also found that modeling volatility such that it accounts for some of the stylized facts of financial time series can improve VaR estimates. This is also confirmed by the findings in this thesis. Based on this thesis, no ideal method can be appointed which produces the most accurate VaR estimates under all circumstances. For the S&P 500, the best performing VaR method is Monte Carlo simulation at the 95% confidence level of VaR and student-t approximation at 99% confidence level of VaR, both estimated using conditional volatility from the EGARCH(1,1) model. The best performing conditional volatility models for the returns of the S&P 500 are GARCH(1,1) and EGARCH(1,1). The purpose of this thesis was to find the most accurate VaR method. As stated in chapter 2, the most accurate VaR method is dependent upon the financial instrument for which VaR is needed. Therefore, the above described best performing VaR methods for the S&P 500 might not be the most accurate VaR methods for other financial instruments such as currencies, options, other stocks or bank portfolios. Another shortcoming of this study is that the parameters of the GARCH and EGARCH models were estimated ex ante and were assumed to be constant. This led to biased results of the parameters which influenced the accuracy of the VaR methods. Ex ante

(30)

estimation of the parameters possibly resulted in more accurate VaR estimates. This is due to the extra information about returns of the S&P 500 included in maximizing the log-likelihood functions, which would have been absent in the case of ex post parameter estimation. The assumption of constant parameters possibly led to less accurate VaR estimates. Because of this assumption, the GARCH and EGARCH models were incapable of accommodating systematic changes in the persistence parameter and in the unconditional volatility, which both could have improved the accuracy of the volatility estimates. As the effect of ex ante parameter estimation on the accuracy of volatility estimates might be negative, while the assumption of constant parameters might have positively affected the accuracy of volatility estimates, the total effect is ambiguous. The main shortcoming of the applied backtests lies in Christoffersen’s serial independence test. According to this test, a VaR method is only violated when first-order VaR exceedances occur. Some of the investigated VaR method resulted in periods with frequent VaR exceedances. This can easily be seen in the figures in the appendix. Especially the VaR methods using sample volatility resulted in many VaR exceedance during the end of the year 2008. All VaR methods still passed the test on serial independence in this period because VaR exceedances were separated by one day without a VaR exceedance most of the time. This second-order dependency of VaR exceedances is not recognized by the serial independence test. Improvements of the above described shortcomings indicate the possible suggestions for future research. The applied VaR methods could be used to obtain the VaR for different financial instruments to test whether the most accurate VaR methods indeed are dependent on the financial instrument. To get more reliable results, the GARCH and EGARCH parameters could be recalculated every day. Furthermore, there are numerous other volatility models such as TGARCH or GJR-GARCH which could be used. Also the number of lags included in those volatility models could be changed to see if this improves the accuracy of VaR methods. Finally, more backtests could be performed. To deal with the second-order dependency of VaR exceedances, Christoffersen introduced another test. This duration test focusses on the on the time period between VaR exceedances. Because second-order dependency of VaR exceedances could also violate a VaR method, the duration test might give more reliable results than the serial independence test.

(31)

References

Artzner, P., Dalbaen, F., Eber, J.M. & Heath, D. (1999): Coherent measure of risk. Mathematical Finance, 9(3), 203-228.

Alberg, D., Shalit, H. & Yosef, R. (2008): Estimating stock market volatility using asymmetric GARCH models. Applied Financial Economics, 18, 1201-1208.

Bao, Y., Lee, T.H. & Saltoglu, B. (2006): Evaluating predictive performance of Value-at-Risk in emerging markets: a reality check. Journal of Forecasting, 25(2), 101-128.

Basak, S. & Shapiro, A. (2001): Value-at-Risk based risk management: optimal policies and asset prices. The Review of Financial Studies, 14(2), 371-405.

Beder, T.S. (1995): VaR: seductive but dangerous. Financial Analysts Journal, 51(5), 12-24.

Berkowitz, J. & O'Brien, J. (2002): How accurate are Value-at-Risk models at commercial banks? The Journal of Finance, 57(3), 1093-1111.

Bollerslev, T. (1986): Generalized Autoregressive Conditional Heteroskedasticity. Journal of Econometrics, 31, 307-327

Brunnermeier, M.K. (2008): Deciphering the liquidity and credit crunch 2007-08 (NBER Working Paper Series No. 14612). Cambridge: National Bureau of Economic Research.

Christoffersen, P. F. (1998): Evaluating interval forecasts. International Economic Review, 39(4), 841-862.

Duffie, D., & Pan, J. (1997): An overview of Value at Risk. The Journal of Derivatives, 4(3), 7-49. Engle, R.F., & Manganelli, S. (2001): Value at Risk models in finance (European Central Bank

Working Paper Series No. 75). Frankfurt am Main: European Central Bank.

Hendricks, D. (1996): Evaluation of Value-at-Risk models using historical data. Economic Policy Review, 2(1), 39-67.

Jorion, P. (2007): Value at Risk: the new benchmark for managing financial risk (Third edition). New York: McGraw-Hill.

Jorion, P. (2009): Risk management lessons from the credit crisis. European Financial Management, 15(5), 923-933.

Kupiec, P. (1995): Techniques for verifying the accuracy of risk measurement models. Journal of derivatives, 6, 7-24.

Lin, C.H. & Shen, S.S. (2006): Can the student-t distribution provide accurate Value at Risk? The Journal of Risk Finance, 7(3), 292-300.

Nelson, D.B. (1991): Conditional Heteroskedasticity in asset returns: a new approach. Econometrica, 59(2), 347-370.

O’Brien J. & Szerszen, P.J. (2014): An evaluation of bank VaR measures for market risk during and before the financial crisis (Finance and Economics Discussion Series No. 21). Washington, D.C.: Federal Reserve Board.

(32)

Appendix

Figure 7: Historical Simulation: 95%-VaR

(33)

Figure 9: Normal approximation: 95%-VaR

(34)

Figure 11: Student-t approximation: 95%-VaR

(35)

Figure 13: Monte Carlo simulation: 95%-VaR

The accuracy of value-at-risk methods