_______________
¹E-mail: j.d.van.der.meulen.2@student.rug.nl
Banking structure and stress testing:
Assessing financial stability in a stressed environment
Jasper van der Meulen (s2130092)¹
Supervisor: A.A. Tsvetkov
Study program: MSc. Finance
University of Groningen – Faculty of Economics and Business
26 June 2015
Abstract
In recent years, stress testing has gained popularity among regulators in assessing the financial stability on individual bank level and the banking system as a whole. As the 2014 stress test consists of an extensive and complicated framework, this paper suggests other indicators of assessing financial stability of banks under adverse economic conditions. Hazard rates, leverage, the current core tier 1 capital ratio, profitability and credit ratings are found useful in assessing financial stability of banks, especially when used in conjunction. Credit default swap spreads and credit ratings are an imperfect measure of financial stability. Banks, stakeholders and supervisors can use these implications on a more continuous base together with the stress tests as an indication of the financial stability of banks in a stressed environment.
Word count: 15.000
Page 2 of 55
1. Introduction
At the end of October 2014, the European Central Bank (ECB) published the results of their most recent stress test. This test, generally regarded as the toughest so far, included 123 European banks from which 24 banks failed the test. Especially the banks failing the test received considerable attention from the financial markets. Large fluctuations in stock prices of the banks resulted, possibly because failed banks are required to attract more capital. Stress tests consist of a complicated framework, where abundant data is necessary as input, requiring banks to devote resources for providing this data to the ECB. This paper looks for simpler indicators to assess financial stability of a bank under adverse conditions.
Stress tests are carried out by the European Banking Authority (EBA). The aim of stress tests is twofold. First, stress tests assess the adaptability of financial institutions to adverse market conditions, such as economic crises. Furthermore, they provide stakeholders with an overview of the systematic risk in the European financial system. Apart from regulators at the EBA/ECB and each country’s central bank, stakeholders involve governments, the banks subject to the stress tests, investor and creditors of the banks as stress tests provide an assessment of the financial stability of the banks. The fact that stock prices of banks who failed the stress test plunged in reaction in October 2014, shows that investors pay attention to the outcomes of the stress test and these test provide information to the market about a banks’ financial position that apparently investors did not had previously.
The only determent for banks to pass or fail a ECB stress test is the capital ratio of a bank. This is usually defined as the amount of capital (tier 1 and tier 2 capital) over assets, where risks are assigned to different asset classes. Furthermore, policy makers have imposed tougher capital requirements on banks in recent years, mainly because of the ongoing financial crisis starting in 2008. Stability of the Eurozone’s financial sector is at the core of European policy. Therefore, the recent development towards higher capital requirements from regulators is unsurprising. In the context of an ongoing crisis, higher capital requirements serve a number of functions (Crouhy and Galai, 1986). Firstly, it is said to protect creditors against insolvency, reducing the probability of financial distress for the bank. Furthermore, more capital absorbs unanticipated losses, thus increasing the perceived stability of the bank. Next, higher capital ratios provide both fixed assets financing and working capital. Lastly, asset expansion beyond the objectives of the bank is restrained. In short, more capital enhances stability of the bank and the financial system as a whole and reduces idiosyncratic risk associated with banking activities.
Page 3 of 55
obligation to achieve regulatory compliance. 95% of the respondents have hardly ever changed business plans as reaction to the stress test results. One explanation is that the stress tests before 2014 were not as extensive as the consecutive test. Apart from application of the insights, most banks agree that stress testing provides interesting understanding in especially the field of risk appetite and de-risking. Furthermore, stress testing can add value in internal decision-making processes, for example by serving as a tool in risk management.
This paper makes an effort in developing transparent, easy-to-compute and understandable indicators of financial stability of banks in a stressed macroeconomic environment using the 2014 stress test. The main research question is specified as follows:
What are important indicators in assessing financial stability of banks in a stressed macroeconomic environment?
Besides developing models for assessing financial stability, these indicators can enhance the understanding of why banks pass or fail the stress test and what banks will have to focus on to pass the next stress test. When calibrated to the next stress test correctly, the indicators might be deployed to predict which banks will fail. The main difference with the models developed in this paper and the ECB stress test is that the latter uses a complicated framework in the stress testing process, is resource intensive and requires senior management involvement. Besides, the indicators derived here can be applied on a more continuous base. First, standalone measures of financial stability are applied on the 2014 stress test: Hazard rates, credit ratings and the S-score. These measures, together with other variables present in academic literature, will then be exploited in deriving two extended models of financial instability based on hazard rates and credit ratings. Banks, stakeholders and supervisors can use the implications of this research to better assess the impact of adverse macroeconomic conditions on financial stability.
Page 4 of 55
2. Literature review
In this section, the ECB stress test is outlined together with criticism about the stress testing framework and underlying assumptions. Furthermore, a description of existing models in estimating financial stability is provided, including the Bankometer model, external credit ratings, and CDS spreads based probabilities of default. The last subsection discusses empirical literature on early warning systems and indicators, key characteristics of failed banks in the past and theory on the relation between various banking variables and bank failure.
2.1 ECB stress tests
The latest ECB stress test dates from October 2014, in which 123 banks from 22 countries are assessed in terms of their Common Equity Tier 1 capital ratio (hereafter CET1 ratio) using scenario analysis. Descriptive statistics about the 123 banks can be found under sample description in Section 4. In the first subsection, an overview of the four historical stress tests by the ECB is provided. Next, the underlying stress testing framework is addressed, consisting of four pillars. The third subsection summarizes the stress test results and the final section discusses the informational value of stress tests.
2.1.1 Overview of stress tests
There is a continuous development in stress testing, for example in making the base scenarios as realistic and the adverse scenarios as plausible as possible. Note that these scenarios are developed to identify vulnerabilities of banks and are not a prediction of the future. So far, there have been four stress tests and more are to come. Table 1 lists the most important features of these four tests.
Table 1: Historical overview of stress tests
2009 2010 2011 2014
Number of banks subject to test 22 91 90 123
% of EU banking assets covered 60% 65% Not disclosed 70%
Period of scenarios 2009 – 2010 2010 - 2011 2010 - 2012 2014 - 2016
Number of banks failing the test 0 7 (8%) 20 (22%) 24 (20%)
Hurdle rate for passing the test 6% 6% 5% 5.5%
CET1 ratio in adverse scenario Above 8% 9.2% 7.7% 8.5%
Shortfall CET funds in adverse scenario Not disclosed €3.5 bln €27 bln €24 bln
Page 5 of 55
Stress tests have increased over the years in both number of banks that were subject to the test and the length of the period the test covered. The increasing number of banks that fail the test, accompanied by larger shortfalls can be seen as a signal of a worsening banking system over the years, but it is more likely that stress tests have become more severe including worse adverse market developments and tougher capital requirements causing more banks to fail. The 2011 stress test is ambiguous in its ‘toughness’, since relative more banks failed despite the lower hurdle rate compared to the other tests. Section 5.4 elaborates on this ambiguousness where the 2011 stress test is used for out-of-sample testing.
2.1.2 Outline of the ECB stress test
The full outline of the 2014 test together with the full results can be found on the website of the European Banking Authority (eba.europe.eu). Together with an ‘Asset Quality Review’ (AQR), this stress test constitute the ‘Comprehensive Assessment’ with the aim of strengthen the balance sheets of the major European banks, enhancing transparency and building confidence as carried out by the ECB in its supervision role of the Single Supervisory Mechanism (SSM).
The stress test part of this Comprehensive Assessment consists of two hypothetical scenarios, a baseline scenario and an adverse scenario for the years 2014, 2015 and 2016. Under the baseline scenario, the market develops as expected in a realistic scenario. For example, real GDP growth in Europe is -0.1% in 2013 and 1.4% in 2014, as stated in the European Commission Spring 2013 Forecast. On the other hand, an economic shock is imposed under the hypothetical adverse scenario. The objective is to identify banks that fail in this scenario and therefore need to raise additional capital to improve their capital position. The ECB calculates the effect of several factors, including credit risk, risk exposure and profitability on the CET1 ratio. When a bank has a CET1 ratio below a predefined CET1 ratio, called the hurdle rate, the bank fails the stress test. For the 2014 stress test, this hurdle rate was set at 8% and 5.5% for the baseline and adverse scenario, respectively. An important assumption in the stress testing framework is the static balance sheet assumption. Over time and under both scenarios, the bank’s balance sheet is expected to remain constant, meaning that assets and liabilities which mature within the stated time frame are replaced with similar financial instruments. Although this assumption is likely not to hold in practice, it allows the ECB to calculate the effect of both scenarios on the bank’s balance sheet in a simpler and more transparent way. However, a dynamic balance sheet assumption would have been more appropriate, as banks’ assets seldom remain constant over time. The impact of both scenarios on, eventually, the CET1 ratio takes place within a four pillar framework as outlined in Fig. 1.
Page 6 of 55
Figure 1: Stress testing framework
The figure describes the stress testing framework used in the 2014 stress test, consisting of four pillars.
Source: European Central Bank
In the second pillar, top-down satellite models translate the macroeconomic model into bank-specific credit risk, market risk and interest rate risk parameters. Thus, the stress tests only tests these types of risks and disregards other important types of risk, such as operational, liquidity and model risk. Important to note is that markets are assumed to be illiquid, therefore it is impossible for management to take actions to reduce trading exposure when there is the need to. Also, the relation between the macroeconomic variables and the risk variables is often a complex and sensitive one. Many of the risk parameters are calculated by the banks themselves using internal models and a bottom-up approach. The ECB relies thereby on the self-declaration ability of the banks for the final assessment of whether a bank passes the stress test. This introduces an informational asymmetry problem as well, as banks have more information about their financial position than the ECB (Hull, 2012). Banks might declare more positive information then is actually the case, leading them to pass the stress test.
Page 7 of 55
𝑅𝑖𝑠𝑘 𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑 𝐴𝑠𝑠𝑒𝑡𝑠 = 𝐾 ∗ 12.5 ∗ 𝐸𝑥𝑝𝑜𝑠𝑢𝑟𝑒 𝐴𝑡 𝐷𝑒𝑓𝑎𝑢𝑙𝑡 (1) where
𝐾 = [ LGD ∗ N ( √(1 − 𝑅)1 ∗ G(PD) + √(1 − 𝑅)𝑅 ∗ G(0.999)) − (LGD ∗ PD)] ∗1 + (𝑚 − 2.5)𝑏 (1 − 1.5𝑏)
PD is the probability of default, estimated by banks under the IRB approach. Changes in PD result in modified risk weights. External credit ratings are not used, as these are only available for a subset of banks. LGD represents the loss given default and is held constant. Maturity adjustment is represented by the last term. The final solvency calculation is presented in Fig. 2.The ratio on the right corresponds with the CET1 ratio used throughout this paper and is calculated by dividing existing capital and net operating profit by the risk weighted assets. Existing capital is also named ‘Tier 1 capital’, usually comprised of equity capital and retained earnings. Each asset class is assigned a weight to base on its risk, constituting the risk-weighted capital. For example, cash has zero risk.
Figure 2: Final solvency calculation
Source: European Central Bank
Page 8 of 55 2.1.3 Final results
Under the adverse scenario, the average decline in CET1 ratio across banks amounts to 2.6% (from 11.1% in 2013 to 8.5% in 2016). This includes a joint effect with the AQR of 300 basis points. The ECB reported three factors to have the most impact on the capital ratio: credit risk losses, risk exposure amount and operating profit. These factors impacted the capital ratio with -4.4%, -1.1% and +3.2%, respectively. As noted before, 24 banks failed the test as their CET1 ratio fell below the hurdle rate under the adverse scenario. Their total maximum capital shortfall amounts to 24.6 billion euros. As already 15.1 billion euros has been raised during 2014 (not included in the stress test), a shortfall of 9.5 billion euros remains. The ECB requires banks with a shortfall to propose how to cover this shortfall within the coming nine months. See the website of the European Banking Authority (eba.europe.eu) for a complete overview of the ECB 2014 stress test results and its implications.
2.1.4 Value of stress testing
Many papers have discussed the informational value of stress testing. Do stress tests signal valuable information to the financial markets? If so, stock prices and CSD spreads should react to the publication of stress test results. Using event study methodology, Alves et al. (2015) measures the impact of disclosing the 2010 and 2011 stress test results by the ECB on the stock market and CDS market. They concluded that stress test results signal new information to the markets, although the CDS market seemed to partially have anticipated the results. The stock markets reacted most strongly to the stress test outcomes, which was particularly the case for the more risky financial institutions. In another paper, Petrella and Resti (2013) argue that the disclosure of stress test results reduces bank opaqueness, thereby contributing to restoring confidence of investors in the European financial sector. Using the same event study methodology on the 2011 stress test, they find that disclosing the outcomes is considered relevant by investors as it provides them with indicators that play an important role during an economic downturn (i.e. the adverse scenario).
Page 9 of 55
to be nationalized by the U.S. government based solely on the stress test results. However, using event study methodology, they find stock markets to foreseen the banks which would have capital shortfalls, but information on the size of the shortfalls was not foreseen. Especially banks with large shortfalls experienced large negative abnormal returns. No banks were nationalized in response to the outcomes.
2.2 Existing models for assessing financial stability
There are different models present in academic literature focusing on the financial stability of companies and banks. Financial stability is defined as the ability of banks to sustain adverse economic shocks. As the stress test provides an assessment of virtual defaults of banks under adverse market conditions, a common indicator of financial stability is the probability of default. External credit ratings and CDS spreads implied probability of defaults are frequent measures in this context. Furthermore, the Bankometer model from Shar et al. (2010) is useful in assessing financial stability of a bank. These models are described below.
A market-based indication of the probability of default is a Credit Default Swap (CDS). This is an instrument providing insurance against the reference entity, in this case a bank, defaulting on its debt. Once this entity defaults, the holder of the CDS contract has the right to sell bonds of the counterparty at face value. The protection seller receives regular payments from the protection buyer until the contract matures or the reference entity defaults. These payments are a percentage of the bonds’ face value, named the CDS spread. Contracts are usually traded over-the-counter. The use of CDS spreads as a market-based proxy for credit risk has increased in popularity. CDS spreads are said to reflect default risk much more accurate than credit ratings (Bijlsma et al. 2014). A shortcoming of CDS spreads relates to asymmetric information. Hull (2012) argues that some market participants might have more information and can therefore better assess the probability of default than others, fuelling speculative trading in CDS’s.
Page 10 of 55
is further investigated by Bolton et al. (2012). According to them, these conflicts cause two distortions. First, competition between agencies facilitates ‘ratings shopping’, thereby reducing efficiency. Second, credit ratings are likely to be amplified during economic expansion and more entrusting issuers. Furthermore, with limited disclosure on the rating process of these agencies, there is an opportunity to departure from quantitative models, thereby adding a subjective component to the rating process (Jollineau et al., 2014). Although credit ratings are supposed to provide information about the quality of securities, the aforementioned arguments seems to point at the opposite.
The Bankometer model is developed by Shar et al. (2010) following recommendations from the International Monetary Fund (IMF) on ‘Macro-prudential indicators of financial system soundness’ (occasional IMF paper 192, 2000). The methodology is similar to the discrimant analysis used in Altman’s well known z-score for assessing the default probability of manufacturing companies. The Bankometer model uses six bank-specific variables with different weights attached to each of them, resulting in an ‘S-score’. Shar et al. (2010) derives the weights using the CAMEL framework and parameters used in the Credit Leona’s Securities Asia (CLSA) stress test. The Bankometer model has gained much attention in Asia, but has not been applied to European banks so far. Bankometer has been applied to Sri Lanka (see Nimalathasan et al. 2012); Bangladesh (see Qamruzzaman 2014); Indonesia (see Erari 2014) and Pakistan (see Shar et al. 2010). The last paper most clearly shows a similar ability of the Bankometer model in assessing bank performance compared to the CAMEL approach and the CLSA stress test. Shar associated S-scores above 70 with financially sound banks and below 50 with insolvent banks.
2.3 Early warning systems and bank failures
As mentioned before, many studies have been conducted on bank failures during the recent banking crisis, starting in the U.S. in 2008 with the collapse of Lehman Brothers. An interesting field in academic literature focuses on early warning systems of the financial sector within an economy or, on individual bank level, early warning indicators of bank failure. This is particularly useful for this paper, as it provides banking variables which show a relation with financial stability.
Page 11 of 55
Although aimed at corporations, Altman et al. (1977) proposes a new model (‘Zeta’) to identify bankruptcy risk. When compared to other bankruptcy classification models, they find a linear structure to outperform a similar model with a quadratic structure with respect to the validity of the model. The model could identify 90% of the corporations one year before they went bankrupt. Earnings stability, measured by the volatility of the return on assets, together with retained earnings over total assets was found to contribute significantly in identifying bankrupt firms. Note that the model was tested on a sample of 111 corporations (manufacturers and retailers) during the period 1969 – 1975.
An interesting paper on identifying determinants of bank profitability is that of Athanasoglou et al. (2008). Taking an unbalanced panel of Greek banks during 1985 to 2001, they found many bank-specific determinants (with the exception of size) to be important in explaining variation of profitability as measured by the return on assets using general methods of moments estimation. An important determinant is leverage, measured by the equity over total assets ratio. Macroeconomic factors, such as inflation and cyclical output had an impact on the banks’ performance, whereas the business cycle only impacts profitability positively when cyclical output is above the long term trend. Furthermore, they reported evidence that increased exposure to credit risk lowers profits. Although the 2014 stress test was also carried out on five Greek banks, it remains to be seen how representative this research is for (Northern) European banks as well.
Continuing on macro-prudential early warning indicators as determinants of banking system stability is the research of Jahn and Kick (2012) from the Deutsche Bundesbank. Using panel regression techniques on a sample of 3,330 German financial institutions during 1995-2010, they developed a banking system stability indicator. This indicator consists of three components: a standardized probability of default, estimated using Moody’s credit rating for large institutions and a hazard rate model from the Deutsche Bundesbank for smaller ones, a credit spread reflecting a bank risk premium and a bank-based stock index. Using this indicator, they found indicators on asset prices (as measured by the national real estate index), business cycle (IFO-expectations index and gross fixed investments) and money market (three month LIBOR-rate) to be reliable early warning indicators.
Page 12 of 55
ratios of the 130 U.S. banks that filed for bankruptcy in 2009 (Santoni et al., 2010). They identified characteristics of these banks associated with a macroeconomic downturn, resulting in a worsening loan portfolio. These characteristics are captured in the following accounting ratios: non-performing assets over total assets (very high for failed banks), loans over total assets (significantly higher than the systems’ average), loans over deposits (very high for failed banks) and loans over customer deposits (very high for failed banks). Other ratios that were investigated as early warning indicators for bank failure are the log of total assets (lower for failed banks, though on average 2009 bank failures were of smaller size compared to bank failures in the forty years before), return on assets (on average negative for failed banks), state of liquidity (not an early warning indicator), tier 1 capital ratio and total capital ratio (both were markedly below the systems’ average).
Summarized it can be stated that several macroeconomic, market-based and accounting variables might help in explaining bank failure and therefore the financial stability of a bank. Based on the results of the aforementioned papers, this paper uses 13 accounting ratios based on the banks’ balance sheet and profit and loss statement and three additional external variables (credit rating, CDS spreads and volatility of CDS spreads). When these variables are found to be relevant in explaining banking failure, act as an early warning indicator or are part of an early warning system, the variables will be investigated in the remainder of this paper to examine whether they contribute to assessing financial stability under adverse conditions. All variables can be found under the data section.
Since this paper and the ECB stress tests rely so extensively on the CET1 ratio, this ratio is probably a strong determinant of whether a bank fails the stress test. Failing the test is namely determined by the adverse CET1 ratio. Therefore, inclusion of the current CET1 ratio seems reasonable in explaining variation in the CET1 ratio under the adverse scenario. The CET1 ratio under the base scenario is not feasible, since this is an output of the stress test. Therefore, the last available CET1 ratio for the 2014 stress test, the 2013 CET1 ratio, will be included in the regression. This is hypothesized as follows:
Hypothesis 1: Actual CET1 ratio has a strong positive relation with the adverse CET1 ratio
Page 13 of 55
3. Methodology
In the first part of this section, the methodology of the models described under Section 2.2 is outlined. The relation with existing literature is discussed and how the models will be applied to the data used in this paper. The second part concentrates on extending the hazard rate and credit rating model with additional variables discussed in the literature section.
3.1 Existing models
Each of the three models uses probability of default as dependent variable. This probability is a wide-used measure of financial stability in the banking industry. The objective is to study whether the model is sufficient in capturing variation of the CET1 ratio under the adverse scenario. After all, it is this CET1 ratio that determines whether a bank passes the stress test.
The first model uses CDS spread implied probabilities of default, called hazard rates, as possible determinant of passing or failing the ECB stress tests. The sample of 123 European banks subject to the 2014 stress test is used to estimate a possible causation between this hazard rate and the CET1 ratio under the adverse scenario. Following Hull (2012), the hazard rate is estimated using an approximate calculation, assuming the credit spread to be an average loss rate. The hazard rate is estimated using:
𝜆 = S(T) 10000∗(1+𝑅𝑓)
1−RR (2)
where 𝜆 is the average hazard rate between time zero and the maturity of the CDS, S(T) the continuously compounded CDS spread with T it’s time to maturity expressed in basis points, 𝑅𝑓 the risk-free rate and RR the recovery rate. Within a stressing environment, CDS spreads can be very volatile and quickly spike, thereby increasing a banks’ funding costs and contributing to a deteriorating financial stability. Under adverse economic circumstances, CDS spreads are likely to increase, as was the case during the Greek banking crisis in 2011. This resulted in higher funding costs for the affected banks, leading to lower capital ratios. The relation between adverse CET1 ratio and hazard rate is hypothesized as follows:
Hypothesis 2: When hazard rates increase, the CET1 ratios under adverse market conditions decrease
Hazard rates, serving as explanatory variable in a simple univariate regression model, is calculated for all banks for which information on the CDS spreads is available for 2013. The model which tests hypothesis 2 is estimated using the Ordinary Least Squares (OLS) estimation method. The linear regression looks as follows:
Page 14 of 55
This model to estimate hazard rates is relatively simple, as recovery rates are usually assumed a fixed percentage of 40% and risk-free rates are rather constant over time such that the hazard rate is impacted almost solely by the CDS spread. Nevertheless, the model can provide information on how sensitive the adverse CET1 ratio is to changes in CDS spreads. Additionally, size, ownership, and country are included as control variables in a multivariate setting. All calculations are done using Microsoft Excel and statistical software package EViews.
The second model tests for a possible relation between external credit ratings and the CET1 ratio under the adverse scenario. Notwithstanding the critique following the financial crisis as outlined under Section 2.2, the relation between credit ratings and CET1 ratios under adverse conditions is hypothesized as being positive but small. With higher credit ratings, a bank has a better financial stability as perceived by the CRA, resulting in higher capital ratios under adverse conditions. A small relation is expected, as CRA’s do not rate credit as if the bank were under adverse economic conditions. They rate ‘through the cycle’, meaning the rating process focuses on a ‘normal’ economic environment instead of a stressed one. The hypothesis looks as follows:
Hypothesis 3: Higher credit ratings are associated with slightly higher adverse CET1 ratios
Moody’s rates different types of debt, the most profound type being the long term domestic bank deposits. This credit rating is widely available and therefore used in this paper. Moody’s assigns credit ratings to rating classes, classified as either investment grade or speculative grade. To use these ratings in a quantitative framework, dummy variables are assigned to each rating class, as included in Appendix A. The dummy variables are linearly assigned to the rating classes, implying a linear relation between the dummy and the rating class. Stated differently, a downgrade from Aa2 to Aa3 is as ‘bad’ as going from Baa3 to Ba1, while in the latter case the bank moves from an investment grade bank to speculative grade. In practice, the bank experiencing a downgrade from Baa3 to Ba1 might find it more difficult to attract capital, and perhaps experience higher interest rates, then a downgrade from Aa2 to Aa3. This paper disregards this effect as it is difficult to model. The next step is to specify the linear regression, estimated using OLS:
𝑎𝑑𝑣𝑒𝑟𝑠𝑒 𝐶𝐸𝑇1 = 𝛼𝑖+ 𝛽1∗ 𝑐𝑟𝑒𝑑𝑖𝑡_𝑟𝑎𝑡𝑖𝑛𝑔 + 𝜀𝑖 (4) From this model, it is possible to infer whether banks with an adverse CET1 ratio below the hurdle rate, and so fail the stress test, have on average a worse credit rating than financially sound banks. As an extension, the same control variables as under the hazard rate model are included.
Page 15 of 55
banks, but has never been applied to a sample of European banks. Similar to the Altman z-score, the S-score uses accounting ratios and results in the following linear function:
𝑆 = 1.5 ∗ 𝐶𝐴 + 1.2 ∗ 𝐸𝐴 + 3.5 ∗ 𝐶𝐴𝑅 + 0.6 ∗ 𝑁𝑃𝐿 + 0.3 ∗ 𝐶𝐼 + 0.4 ∗ 𝐿𝐴 (5) where
CA = Capital to Assets (capital over total assets) EA = Equity to Assets (equity over total assets)
CAR = Capital Adequacy Ratio (capital over risk weighted assets)
NPL = Non-Performing Loans to loans (non-performing loans over total loans) CI = Cost to Income (total costs over total income)
LA = Loans to Assets (total loans over total assets)
The remainder of this paper uses the abbreviations on the left side to describe the variables. The weights are derived from accounting ratios used in the CAMEL(S) classification framework and an Asian stress test. Following Shar et al. (2010), higher S-scores are positively associated with more sound banks. This means that banks with a higher S-score should typically have a better capital position under adverse circumstances compared to banks with a low S-score. The hypothesis looks as follows:
Hypothesis 4: Banks with a higher S-score have higher CET1 ratios under adverse conditions
This is tested in a simple univariate setting, where a linear regression using the S-score as explanatory variable will use the standard weights in Eq. (5) and the adverse CET1 ratio as dependent variable, consistent with the former two models. The following regression is estimated using OLS:
𝑎𝑑𝑣𝑒𝑟𝑠𝑒 𝐶𝐸𝑇1 = 𝛼𝑖+ 𝛽1∗ 𝑆𝑠𝑐𝑜𝑟𝑒+ 𝜀𝑖 (6)
where the S-score is calculated using Eq. (5) and the individual variables are end-of-year accounting numbers as reported in Bankscope. This paper also considers that these weights might be altered to apply on European banks. All variables will therefore be tested separately for to determine the optimal weights for applying the model on European banks. It is beyond the scope of this paper to establish hypotheses on each individual relation between the components of the S-score and the adverse CET1 ratio. The regression is estimated using OLS and looks as follows:
Page 16 of 55
3.2 Extended models
Combining variables with hazard rates and credit ratings to construct a more comprehensive indicator serves two purposes. On the one hand, taking a micro-perspective, individual banks have an indication of whether the bank will pass or fail the next stress test. This is possible on a continuous base, to the extent that the relevant variables necessary as input for the model are available on a continuous base. On the other hand, regulators on both a national (governments, national central banks) and international (EU, ECB) scale have a model for assessing banks more frequent, being able to identify financially instable banks that are a possible threat to the stability of the banking system as a whole. In this case, the model can be used on a macro-prudential base. In fact, this model provides regulators with a ‘quick and dirty’ stress testing model. On a continuous base, regulators can now execute their supervision tasks without the need to go through the complex stress testing framework, which takes months to complete. This model can therefore serve as a simplified indicator for financial stability, in addition to the extensive major stress tests, taking place only once per year.
The first step in the development process is to specify which variables / indicators have had explanatory power in predicting whether banks failed the 2014 stress test. This is done by regressing several variables on the adverse CET1 ratio. This ratio determines whether a bank fails the stress test. As CET1 ratio is a continuous variable, it is not only possible to identify banks that fail the stress test on a yes/no basis, but also by how much the adverse CET1 ratio deviates from the hurdle rate. This is done by comparing the predicted CET1 ratios under the adverse scenario with the actual adverse CET1 ratios as calculated by the ECB. The predicted ratios are based on the model developed in this paper; the actual ratios are the result of the 2014 stress test. The model, estimated using OLS, is specified as follows:
𝐶𝐸𝑇1𝑎𝑑𝑣𝑒𝑟𝑠𝑒= 𝛼𝑖+ ∑(𝛽𝑖∗ 𝑋𝑖) + 𝛽2∗ 𝐶𝐸𝑇12013+ 𝜀𝑖 (8)
where 𝛼𝑖 is a constant included to ensure that the errors have zero mean, 𝑋𝑖 is a set of explanatory
variables, 𝐶𝐸𝑇12013 is the CET1 ratio for the last available year (2013) and 𝜀𝑖 is the error term of the
Page 17 of 55
Moreover, type I and type II errors are important in assessing the goodness-of-fit of the regression. Type I errors occur when the model predicts an adverse CET1 ratio below the hurdle rate, while the actual adverse CET1 ratio as determined by the ECB stress test is above the hurdle rate. Restated, the model predicts a bank to fail the stress test while in fact it passes. Even worse are type II errors. This appears when the model predicts an adverse CET1 ratio above the hurdle rate, thus passing the stress test, while the actual ratio is below the hurdle rate, thereby failing the stress test. The objective is to minimize both types of errors, with a focus on type II errors.
One of the assumptions that this paper makes is that the models exclude spill-over effects or ‘financial contagion’ such as funding liquidity drying up. This assumption is in line with pillar four of the ECB framework, where only a qualitative description of financial contagion is administered, but not explicitly tested for in the actual stress test.
3.2.1 Testing the model
The model is tested in two ways. In-sample testing is used first, although no serious conclusion can be attributed to this form of testing the goodness-of-fit of the model. Generally you will find a high R-squared using an ex post comparison with the model which variables have been found relevant using the same data that you test for. The second manner to test the model is to use an earlier stress test. In terms of sample size and ‘toughness’, the 2011 stress test comes closest to the 2014 one, although the model needs to be calibrated to take into account this ‘toughness’. Only 65 banks were subject to both the 2011 and 2014 stress test, of which nine banks failed the 2011 stress test. Earlier stress tests have an insufficient number of banks tested and it becomes very difficult to calibrate these ‘easy-to-pass’ tests. Out-of-sample testing by means of splitting the sample in two subsamples (failed versus non-failed banks) is impossible, given the small sample size of failed banks (24). No interferences can be drawn for this type of out-of-sample testing, making the 2011 and 2014 stress tests most suitable for back testing.
Page 18 of 55
4. Data
The sample of European banks is set forth in the first subsection. From each of the models, necessary variables are taken to infer which data is necessary and from which sources to select this data. Summary statistics, data and data sources are outlined under the second subsection.
4.1 Sample description
As mentioned before, the sample consists of 123 European financial institutions, the same that were subject to the 2014 stress test from the ECB. The only criterion of the ECB to include an institution in the sample is that the institution should have a national coverage of at least 50% of the national banking sector. It is reasonable to include these institutions in the sample, as the 2014 stress test is used to estimate the variables that predict which institutions passed or failed this test. The population includes all banks in Europe, which are potentially subject to the next stress test.
A complete list of all 123 banks subject to the 2014 stress test can be found in Appendix B. From these 123 banks, 24 banks failed the stress test. Fig. 3 classifies each financial institution to ownership type. Unsurprisingly, most institutions are banks, followed by government-owned institutions and funds/trustees. Ten institutions are missing, as data on their ownership is unavailable in Bankscope. From the 113 remaining banks, 62% is classified as bank. Three banks have an unknown ownership type. Throughout this paper, ‘banks’ and ‘financial institutions’ are used interchangeably.
Figure 3: Ownership type of financial institutions subject to the 2014 stress test
Source: Bureau van Dijk Bankscope
62% 15% 6% 5% 4% 3% 1% 1% 3% Bank Public/State/Government
Mutual & Pension Fund/Trustee
Page 19 of 55
Fig. 4 lists each financial institution according to the country of their head office. It also shows the number of banks failing the stress test under the adverse scenario. Especially banks in the southern European countries failed the stress test, although Spain stands out as zero banks failed. Notice that not only countries part of the European Union were subject to the stress test, but also Scandinavian countries. Northern European banks show a better financial stability. Most banks were located in Germany.
Figure 4: Country of financial institutions subject to 2014 stress test
Blue bars show the total number of banks from that country, the red bars indicate the number of failed banks.
Source: European Central Bank
4.2 Data
This part provides an overview of all necessary variables, how they are measured, calculated and the source they originated from. The variables are listed per model. Table 2, at the end of this subsection, provides a list of all variables including their data source.
4.2.1 ECB variables
Throughout the paper, the 2013 CET1 ratio and CET1 ratio under the adverse scenario are used. These three variables can be found – per bank – on eba.europe.eu. Since this research will be placed within the context of the ECB stress test, the definition of CET1 ratio according to the ECB is used. Moreover, this ratio is a widely used metric within the banking sector to communicate their capital position to outside investors and other stakeholders. Refer to Fig. 2 for the definition and calculation of the CET1 ratio. Following Fig. 5, the impact of the stress test is, on average, positive for banks under the baseline scenario and negative (up to -2.6% in 2016) under the adverse scenario. Impacts range from almost zero to more than 5%.
Page 20 of 55
Figure 5: Impact on CET1 ratio of the 2014 stress test
The figure shows the development of the CET1 ratio over the years under the baseline (blue bars) and adverse scenario (red bars).
Source: European Banking Authority
4.2.2 Hazard rates
The model defined in Eq. (2) uses three variables as input. The explanatory variable is the hazard rate. The CDS spread is the most important input variable and obtained via Markit. The Markit survey database has CDS data on about 2,600 entities (markit.com). Markit is well-known for the Purchasing Manager Index (PMI), based on a survey among selected companies, providing an indication of future economic conditions. Their CDS data is based on live quotes, market makers’ books of record and clearing submissions (markit.com). The resulting CDS spread is the average consensus spread of the best estimates of participating institutions. The type of CDS contract used in this paper is denoted in euro and takes a mid-quote spread reflecting the average of the bid and ask prices on senior bonds using a maturity of five year. This is the most liquid type of CDS and act as benchmark in the CDS market (CPB discussion paper). CDS spreads are computed on a yearly base by taking the average of all existing daily spreads in that particular year. For this model only the 2013 CDS spreads of all available banks are used (N=61, of which nine failed) as this is the most recent year available and therefore most likely to explain variation of the adverse CET1 ratio. Taking an average over the last, say, six years would be an alternative, but Fig. 6 below shows the 2013 hazard rate to be close to the average spread over the period shown (4.44% for 2013 versus 4.02% over a six-year period). Other inputs are the recovery rate and risk-free rate. Recovery rate is assumed to be 40%, which equals the historical average. The risk-risk-free rate is the German ten year government bond yield. This is a well-established proxy for the risk-free rate in finance. The yields are obtained via Thomson Reuters Datastream.
Page 21 of 55
The main caveat in using this model is that the CDS spreads are available for 61 banks out of the 123 only. These are likely to be large banks, causing them to pass the stress test ceteris paribus with a higher probability compared to smaller banks. For example, the average bank with CDS’s has total assets of 422 million euro, compared to only 244 million euro for the population of 123 banks. The average CDS spreads for these 61 banks together with their hazard rates are presented annually in Fig. 6. The rates peaked in 2011/2012, at the time of Greece defaulting on its debt.
Figure 6: CDS spreads and hazard rates
Hazard rates are calculated using equation 2. A recovery rate of 40% is assumed. The blue and red line show CDS spreads and hazard rates respectively, over time.
Source: Markit for spread, Datastream for risk-free rates
With respect to the control variables, size is measured as the logarithm of total assets. Ownership and country are assigned dummy variables to. The ownership types of Bankscope are used, see also Fig. 3. Appendix C lists the assigning of dummy variables to country and ownership type and summary statistics can be found in Appendix D.
4.2.3 Credit ratings
To calculate probabilities of default for this model, only credit ratings are necessary as input variable. As the author has subscribed to moodys.com, all banks were searched manually for on their website to identify the rating Moody’s Investors Service assigned to the long term domestic bank deposits. Of the 123 banks that were searched for, 99 were assigned a rating to. Of these banks, 18 banks failed the 2014 stress test. 71 had an investment grade rating versus 28 banks with a speculative grade rating. The average rating is Baa2 and no bank was found to have the lowest rating (i.e. a ‘C’ rating). Figure 7 lists the number of ratings per rating class. It shows 15 banks to be close to a speculative grade (Baa3). The next step is to assign a dummy variable to each rating class and run the regression as specified in Eq. (4).
Page 22 of 55
Figure 7: Credit ratings per rating class
Horizontal axis lists the rating classes from lowest to highest. Numbers show the total of banks in that rating class.
Source: Moody’s Investors Service
4.2.4 Bankometer model
The model uses six bank specific ratios, all accounting based. This means that these variables are made publicly available only quarterly or annually, depending on the frequency of earning announcements. This makes it difficult to apply this model on a continuous base. However, as the ratios are computed annually, the resulting S-score will provide the bank and its stakeholders with an overview of the financial stability in that particular year. For this paper, the variables of 2013 are used, such that the resulting S-score represents the financial stability of 2013.
All variables are available within Bankscope using the Legal Identifier (LEI) code to search the banks. Bankscope has missing data on 11 of the 123 banks, reducing the final sample to 101 banks. Following Eq. (5), the following variables serve as input for the S-score: CA, EA, CAR, NPL, CI and LA. A description on the variables can be found under Eq. (5) and summary statistics are presented in Appendix D.
4.2.5 Extended models
Page 23 of 55
Table 2: Input variables extended models
The table presents variables showing a relation with financial stability. Table lists the variables, how they are calculated, hypothesized relation with financial stability, data source to obtain these variables and literature source
where the variables have been found. The underlined variables are also part of the Bankometer model.
Variable Calculation Hypothesized relation
Data source Literature source 2013 CET1 ratio CET1 capital / RWA Positive BvD Bankscope Santoni et al. (2010) Loans - assets Loans / assets Negative BvD Bankscope Santoni et al. (2010) Loans – deposits Loans / deposits Positive BvD Bankscope Santoni et al. (2010) Loans – customer
deposits
Loans / customer deposits
Positive BvD Bankscope Santoni et al. (2010)
Size Log (assets) Positive BvD Bankscope Santoni et al. (2010) Operating risk or
cost-to-income
Operating expense / operating income
Negative BvD Bankscope Sarkar and Sriram (2001)
Return on Assets Net income / assets Positive BvD Bankscope Santoni et al. (2010) Leverage Equity / assets Positive BvD Bankscope Athanasoglou et al. (2008) Credit rating External assessment Positive Moody’s Jahn and Nick (2012)
Hazard rate Refer to Eq. (2) Negative Jahn and Nick (2012)
» CDS spread Consensus pricing Markit Survey
» Recovery rate Assumption Markit Survey
» Risk-free rate 10 year German bond Datastream Volatility CDS Standard error daily
CDS spreads 2013
Negative Markit Survey ECB stress test framework (market risk parameter)
Page 24 of 55
5. Results
Following the methodology employed in Section 3 and the data described in Section 4, results are discussed below. First, the three existing models are outlined, followed by the extended models. These models are then calibrated and tested, both in-sample and by using the 2011 stress test results. The section concludes with a sensitivity analysis.
5.1 Hazard rates
Refer to Table 3 below for the results. Although the R-squared is very low, the hazard rate is statistically significant on a 5% level and has the expected sign. A 1% increase in hazard rate decreases the adverse CET1 ratio with 0.50%. The results suggest a weak predictive power from CDS spreads in determining the CET1 ratio under adverse conditions.
Table 3: Univariate hazard rate model
Classical standard errors in parentheses. ** Significant at 5%. *** Significant at 1%. Variable Coefficient Hazard rate -0.501665** (0.244946) Intercept 0.111829*** (0.014734) Observations 61 Adjusted R2 0.050551
When including control variables (size, ownership and country), only ownership type is significant. Although insignificant, the coefficient on size is negative indicating larger banks to have lower adverse CET1 ratios. The complete results can be found in Appendix F. On average, banks with lower CDS spreads appear to have higher adverse CET1 ratios, making CDS spreads as a standalone measure of default under adverse conditions insufficient. However, the small sample size, poor fit of the data to the model, and not normally distributed hazard rates makes it impossible to draw inferences about the population and to confirm hypothesis 2.
5.2 Credit ratings
Page 25 of 55
hurdle rate is not harmful, since the focus is on banks that fail the stress test, that is, have a CET1 ratio below the hurdle rate.
Table 4: Univariate credit rating model, different samples
Classical standard errors in parentheses. *** Significant at 1%.
It follows that in both cases credit rating is significant on a 1% significance level with the expected positive sign. This indicates that an upgrade of one rating class by Moody’s results in a small increase of just 0.005% in the CET1 ratio under the adverse scenario. Whereas the average bank in the sample has a Baa2 rating, the rating in a subsample of all failed banks (N=18) is on average between Ba2 and Ba3, four rating classes below the rating of the complete sample and even five classes below the average rating of a non-failed bank. Thus, banks which suffer more from adverse market conditions (i.e. have a lower adverse CET1 ratio) are associated with a lower credit rating. Apparently credit ratings have some predictive power in identifying banks with a low financial stability.
Controlling for country and size, the adjusted R-squared improves to 21% with three of the four variables significant at the 5% level. Credit rating is significant at the 1% level with about the same coefficient as in the univariate case. Noteworthy is the negative coefficient on size, indicating larger banks to have on average a smaller credit rating. The complete results are included in Appendix H. Removing the three largest outliers makes the sample data normally distributed, such that hypothesis 3 can be confirmed. Banks with a low ability to repay their debt as reflected by the credit rating appear to perform worse under adverse market conditions as well. Credit ratings as a standalone measure of financial stability under stressed conditions is inadequate as it lefts much variation in adverse CET1 ratios unexplained.
5.3 Bankometer model
Following the results in Table 5, the average S-score is significant on a 5% level with a positive coefficient of 0.0699. S-scores are computed using standard weights according to Eq. (6). The positive coefficient is in line with hypothesis 4, but the small coefficient contradicts Shar et al. (2010) who found a larger explanatory power of the S-score. When the sample is split into a group of failed and a group of
Variable Coefficient Variable Coefficient
Page 26 of 55
non-failed banks, the failed subsample has an average S-score of 109.226, while the non-failed subsample has an S-score of 115.038 and the total sample of 113.83. The S-score values are not in line with Shar et al. (2010) classification. According to them, banks are insolvent with an score below 50. The lowest S-score for a European bank is 77.03, meaning all European banks are financially sound according to the Shar et al. (2010) classification. The high S-scores for European banks are probably due to different weights. The results are robust to taking an average S-score of the last 11 years instead of the S-score of 2013.
Table 5: Univariate Bankometer model using Shar weights
White standard errors in parentheses. ** Significant at 5%.
Variable Coefficient S-score 2013 0.0699** (0.0273) Intercept 0.0012 (0.0305) Observations 101 Adjusted R2 0.181744
To determine whether altering the weights improves the S-score, Table 6 presents the multivariate case in which individual components are tested for significance. Most components are significant at a 1% level, except for cost-to-income and loans-to-assets. The weights contradict Shar et al. (2010). Remarkable are the opposite signs for capital over assets and capital over risk weighted assets, since they only differ in the risk assigned to asset classes. Following hypothesis 1, a positive sign is expected. The negative sign for capital over risk weighted assets could be due to multicollinearity, since this variable is almost identical to capital over assets and the dependent variable (adverse CET1 ratio). The correlation between CA and CAR is 0.924.
Table 6: Multivariate Bankometer model
Classical standard errors in parentheses. *** Significant at 1%.
Variable Coefficient Variable Coefficient
CA 0.8159*** (0.0016) NPL -0.2485*** (0.0003) CAR -0.3683*** (0.0014) CI 0.0071 (0.0001) EA 0.5379*** (0.0011) LA 0.0067 (0.0002) Observations 101 Intercept 0.0264 Adjusted R2 0.618255 (0.0159)
Page 27 of 55
model with three variables, all significant at the 1% level. The adjusted R-squared has improved slightly from the model presented above.
Table 7: Revised Bankometer model
Classical standard errors in parentheses. *** Significant at 1%.
Variable Coefficient Variable Coefficient
CA 0.4206*** (0.0006) NPL -0.2461*** (0.0003) EA 0.5699*** (0.0010) Intercept 0.0156 (0.0108) Observations 102 Adjusted R2 0.632481
To test the revised model, the sample is again split in two subsamples. After applying a transformation on the S-score (multiplying by 1000), the average S-score for the subsamples of failed and non-failed banks is 48.5 and 89.3, respectively. These numbers are in line with the classification by Shar et al. (2010), where the average bank that failed under the ECB stress test has a S-score below 50 (insolvent) and the average bank that passed has a S-score above 70 (financially sound). Taking an average S-score over the last 11 years instead of only 2013 makes the scores 51.9 and 79.6 for the average failed and non-failed bank, respectively. Since the normality assumption holds, inferences can be drawn about the population, but an in-sample-test has limited predictive power. Summarized, the positive relation between the S-score and CET1 ratio under adverse conditions in all models presented above confirms hypothesis 4.
The revised Bankometer model is suggested to assess the financial stability of European banks instead of the traditional Bankometer model by Shar et al. (2010). Presumably, European banks have different characteristics than Asian banks in terms of accounting variables. Complete results of the Bankometer model can be found in Appendix H.
5.4 Extended models
Based on the aforementioned models and variables found in literature (refer to Table 2), a combination of three variables are reported significant in explaining variance of the adverse CET1 ratio. The significance of individual variables and standard error and adjusted R-squared of the model as a whole were decisive for the inclusion of variables. Two models contain this ‘optimal mix’ of variables. The models are named ‘extended hazard rate model’ where hazard rate is included as variable and ‘extended credit rating model’, where credit rating is included. This section explains both models, the next subsections test the models. Refer to Appendix I for the complete results of both models.
Page 28 of 55
assets as a measure of leverage and the 2013 CET1 ratio are included. The model is estimated using White standard errors, as the White test detects heteroscedasticity in the residuals. No autocorrelation is present given that the value of the Durbin-Watson statistic is close to two. The high Jarque-Bera statistic indicates the normality assumption not to hold, mainly because of a high kurtosis. Removing two outliers makes the adverse CET1 ratio to follow a normal distribution, but the sample reduces to 58.
Table 8: Extended hazard rate model
White standard errors in parentheses. *** Significant at 1%.
Variable Coefficient Variable Coefficient
2013 CET1 ratio 0.956880*** (0.070157) Volatility CDS spread -0.049577*** (0.015269) Hazard rate -0.856992*** (0.145515)
Equity over assets 0.514505*** (0.100632) Intercept -0.042182*** (0.009117) Observations Adjusted R2 60 0.858612
In a linear context, the extended hazard rate model looks as follows:
𝑎𝑑𝑣𝑒𝑟𝑠𝑒 𝐶𝐸𝑇1 = −.04 + 0.96 ∗ 𝐶𝐸𝑇12013− 0.86 ∗ ℎ𝑎𝑧𝑎𝑟𝑑 𝑟𝑎𝑡𝑒 + 0.51 ∗ 𝑙𝑒𝑣𝑒𝑟𝑎𝑔𝑒
−0.05 ∗ 𝑣𝑜𝑙𝑎𝑡𝑖𝑙𝑖𝑡𝑦 𝐶𝐷𝑆 𝑠𝑝𝑟𝑒𝑎𝑑 (9)
The current CET1 ratio contributes most to explaining variation in the adverse CET1 ratio as the beta is close to one. This is confirmed by employing a Wald test, which does not reject the null hypothesis of the coefficient on 2013 CET1 ratio being equal to one (p-value is 0.5653 using the F-statistic). It follows that current capital adequacy is a strong predictor of the capital adequacy under adverse conditions; a 1% increase in current CET1 ratio is associated with a 0.96% increase in CET1 ratio under adverse conditions. Hazard rate shows a strong negative relation with adverse CET1 ratios. This is in line with the former model on hazard rates and confirms hypothesis 2. A sudden increase of the hazard rate with a full percentage point might be an indication of a deteriorating capital adequacy when the bank finds itself in a stressed economic environment. A drawback of this model is that the inclusion of hazard rates limits the sample size dramatically, pointing at diminishing applicability of this model. Only banks having CDS’s outstanding can use this model. Besides, especially large banks have CDS’s outstanding, introducing a size bias in the model.
Page 29 of 55
environment, this multicollinearity is ignored. Lastly, equity over assets as a measure of leverage has a positive coefficient, meaning a 1% increase in equity relative to assets is associated with a 0.51 increase in CET1 capital over risk weighted assets. Note the similarity between the two variables, the correlation between equity over assets and adverse CET1 ratio is 0.30. Nevertheless, equity over assets measures leverage, whereas CET1 ratio is a more sophisticated measure of capital adequacy. About 14% of variation in adverse CET1 ratio is left unexplained.
Table 9 presents the extended credit rating model. Although the adjusted R-squared is slightly less than in the former model, the sample size is larger (97 instead of 60) and no multicollinearity is detected.
Table 9: Extended credit rating model
Classical standard errors in parentheses. *** Significant at 1%.
Variable Coefficient Variable Coefficient
2013 CET1 ratio 0.798962*** (0.056537) Return on Assets 0.444734*** (0.111543) Credit rating 0.002215*** (0.000663)
Equity over assets 0.287965*** (0.082444) Intercept -0.058771*** (0.010327) Observations Adjusted R2 97 0.817023
In a linear context, the extended credit rating model looks as follows:
Page 30 of 55 5.4.1 In-sample testing
When applying the extended hazard rate model on the 2014 stress test result (in-sample testing), the model classifies 93% of the banks correctly, or 56 of the 60 banks. These results are on a yes/no basis meaning that the model predicts correctly whether banks pass the test when the predicted adverse CET1 ratio is above the hurdle rate (i.e. 5%). The four failed banks are two type I and two type II errors. Of the failed banks, 78% is classified correctly. Fig. 8 presents the relation between actual (estimated by the ECB) and predicted (estimated by the model) ratios in a scatterplot. Note the clustering of banks with a CET1 ratio between 5% and 10%. Just over half of the banks (32 banks or 53%) are within this interval, making it hard to establish a linear relation. As these banks have a CET1 ratio close to the hurdle rate, it seems that especially these banks are on the edge of failing the stress test. After all, their adverse CET1 ratio only needs a small impact downward for failing the test.
Figure 8: Extended hazard rate model: in-sample testing
Relation between actual (horizontal axis) and predicted CET1 ratios (vertical axis).
The relation is also presented in Fig. 9. The average adverse CET1 ratio predicted by the model is 8.35% versus 8.76% for the actual ratios. The figure is sorted on actual CET1 ratios to focus on the banks that fail the test (left side of the graph). Although quite volatile, the predicted ratios move in line with the actual ratios as determined by the ECB stress test. Note the course of the actual ratios. The first part, until the 11th observation, shows an inverse exponential shape. As these banks are on the edge of failing, this part has the key focus. This is followed by a linear shape with a slowly increasing slope, roughly from observation 11 until 45. After this, the slope increases sharper.
Page 31 of 55
Figure 9: Extended hazard rate model: in-sample testing
On the horizontal axis are banks, represented by numbers. Figure is shown full size in Appendix I.
Another way of assessing the prediction power of the model is to look at the number of over- and underestimations of the model by constructing bands of one standard deviation around the predicted ratios. The standard deviation of the regression is 0.018738, meaning that the band width is 0.037476 as depicted in Fig. 9. Overall, the model predicts a higher ratio than is actual the case (i.e. overestimation) for ten banks and a lower ratio than actual (i.e. underestimation) for eight banks. Overestimation is more present with lower capitalized banks, which is a serious weakness of the model. The model estimates adverse CET1 ratios higher than they actual are, leading banks, stakeholders and supervisors to think banks are better capitalized in a stressed environment than they actually are. Overall, 70% of the actual ratios are within one standard deviation of the predicted ratios. This corresponds roughly to the ’68-95-99.7’ rule, stating 68% of the observations to lie within one standard deviation from the mean. This rule assumes the data to be normally distributed, while the Jarque-Bera statistic shows this is only the case when the banks with the smallest and largest predicted ratio are omitted, resulting in a sample of 58 banks.
The same calculations have been run for the extended credit rating model. Overall, this model classifies 94% of all banks correctly (91 out of 97) and 88% of all failed banks (15 out of 17), which is slightly better than the extended hazard rate model. The model classified a bank as failed while it actually passed the test four times (i.e. type I error) and classified a bank as passing the test while it actually failed two times (i.e. type II error). Fig. 10 shows a scatterplot of actual versus predicted ratios. Especially at higher values of the adverse CET1 ratio, the data points become more widely dispersed. Since the focus is on the lower values of adverse ratios, this is less of a problem.
-0.1 -0.05 0 0.05 0.1 0.15 0.2 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59
Page 32 of 55
Figure 10: Extended credit rating model: in-sample testing
Relation between actual (horizontal axis) and predicted CET1 ratios (vertical axis).
Figure 11: Extended credit rating model: in-sample testing
On the horizontal axis are banks, represented by numbers. Two banks with the largest adverse CET1 ratio are excluded as outlier. Figure is shown full size in Appendix I.
Constructing a one standard deviation band around the predicted ratios constitutes Fig. 11. Although difficult to observe, volatility of the predicted ratios increase with the predicted ratios. In total, 66% of the predicted ratios are within one standard deviation of the actual ratios. This corresponds to 17 overestimations, especially for banks with low adverse CET1 ratios and 16 underestimations. Under the extended credit rating model, the average bank has an adverse CET1 ratio of 8.67%. Eliminating two banks with the largest predicted ratios (N=95) makes the Jarque-Bera statistic insignificant, such that the null hypothesis of normality cannot be rejected.
-0.05 0 0.05 0.1 0.15 0.2 -0.1 -0.05 0 0.05 0.1 0.15 0.2 -0.1 -0.05 0 0.05 0.1 0.15 0.2 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94
Page 33 of 55 5.4.2 Out-of-sample testing
While out-of-sample testing using the 2014 stress test is impossible given the small sample size of failed banks and in-sample testing has a low predictive power, the 2011 stress test is used to test the models. However, the model cannot simply be applied on the 2011 stress test without considering the ‘toughness’ of this test. Table 10 presents the number of banks that failed the test in both years under the hurdle rate used in the 2011 stress test (5%) and the 2014 stress test (5.5%). For example, when the hurdle rate would have been 5.5% in the 2011 stress test, 27 banks (or 30% of all banks) would have failed the test. From this, it can be concluded that more banks failed the 2011 stress test relative to 2014.
Table 10: Number of failed banks under both stress tests and different hurdle rates Hurdle rate 2011 stress test 2014 stress test
5.0% 20 (22%) 21 (17%)
5.5% 27 (30%) 24 (20%)
Source: European Banking Authority
One explanation is that banks were undercapitalized compared to the situation of the 2014 test. For example, the average adverse CET1 ratio for banks under the 2011 test was just 7.7% (versus 8.5% for 2014), suggesting at least some undercapitalization of banks in 2011. In this case, more banks would fail the test, even when the same stress testing framework is used. From this point of view, failing the test is either a matter of ‘toughness’ of the stress test or capitalization of the banks. By taking into account macroeconomic variables that constitute the shock under the adverse scenario in both stress tests, this paper attempts to model the ‘toughness’ of the 2011 test. For this purpose, country-specific GDP growth, inflation, government bond yields and stock prices are obtained from both stress testing frameworks and added as calibration parameters. The numbers are taken as deviation from the baseline level. Bond yields are taken as spread over the German ten year bond yield. Since data on these four variables is unavailable for Norway, the average of the variables for Finland, Denmark and Sweden is taken.
Table 11: Calibration parameters
Averages for each parameter used under the 2012 and 2016 adverse scenario in the 2011 and 2014 stress tests, respectively.
2012 2016
GDP -2.3% -2.3%
Inflation -1.2% -0.5%
Stock prices -13.7% -20.2%
Bond yields 0.8% 1.6%
