University of Amsterdam
Faculty of Economics and Business
MASTER’S THESIS
Comparing Early Warning Systems for
Financial Stress Events
Author: Jana Urbankova
Supervisor: Dr. Marco J. van der Leij Academic Year: 2013/2014
Declaration of Authorship
I hereby declare that I have compiled this thesis independently, using only the listed resources and literature. The author also declares that she has not used this thesis to acquire another academic degree.
Acknowledgments
I would like to express gratitude to my thesis supervisor, Dr. Marco van der Leij, for inspiring me to write about measuring financial stress and for guiding me through the process of creating this thesis. Furthermore, I am grateful to Ph.D. Borek Vasicek from the Czech National Bank and Stephan Danninger from IMF for granting me assistance and data.
Bibliographic entry
URBANKOVA, J. (2014): ”Comparing early warning systems for financial stress events” (Unpublished master’s thesis). University of Amsterdam. Super-visor: Dr. Marco van der Leij
Length
Abstract
JEL Classification E44, E61, G01
Keywords financial stress, early warning systems, sig-nalling model
Author’s e-mail urbankovapineal@gmail.com Supervisor’s e-mail M.J.vanderLeij@uva.nl
Contents
List of Tables vii
List of Figures viii
Acronyms ix
1 Introduction 1
2 Theoretical concepts 3
2.1 Financial stress . . . 3
2.2 Early warning systems for financial stress events . . . 5
2.2.1 Signalling approach . . . 5
2.2.2 Limited dependent variable methods . . . 6
2.2.3 Regime-switching model and alternative methods . . . . 7
3 Data description 9 3.1 Crisis occurrence versus crisis incidence . . . 9
3.2 Independent variables . . . 10
4 Model presentation 13 4.1 Signalling model . . . 13
4.2 Panel logit model . . . 17
5 Results 20 5.1 Finding optimal threshold values in signalling models . . . 21
5.2 Performance of individual economic indicators in signalling and logit models . . . 24
5.3 Early warning model comparison . . . 26
5.4 Improving panel logit models . . . 28
Contents vi
5.4.2 The post-crisis bias and the multinomial logit approach . 29
6 Conclusion 34
Bibliography 39
List of Tables
3.1 Data description . . . 10
4.1 Signal scenarios . . . 14
5.1 Crisis ans post-crisis means of key indicators for the United States . . . 21
5.2 Comparison of thresholds given by different methods . . . 22
5.3 Performance of economic and financial variables . . . 25
5.4 Estimated coefficients by the fixed effect and the random effect model . . . 26
5.5 In-sample results . . . 28
5.6 Out-of-sample results . . . 28
5.7 Results for improved panel logit models . . . 30
5.8 Results for the multinomial logit model . . . 32 A.1 Data overview . . . I A.2 Signal scenarios of individual economic and financial indicators
for in-sample data . . . II A.3 Signal scenarios of individual economic and financial indicators
for out-of-sample data . . . II A.4 Signal scenarios for in-sample data . . . III A.5 Signal scenarios for out-of-sample data . . . III A.6 Main panel logit regression results . . . IV A.7 Signal scenarios for multinomial logit model . . . V A.8 Results for multinomial logit model . . . VI A.9 AUC statistics and ROC(0.175) statistics . . . VII A.10 Results of pairwise tests on equality of AUC values, p-values . . VIII
List of Figures
2.1 Financial system fragility . . . 3
2.2 The IMF financial stress index for the United States . . . 5
2.3 Regime-switching model for financial stress . . . 7
4.1 A ROC curve example . . . 15
4.2 Fully informative and uninformative ROC curves . . . 16
5.1 ROC curves for the composite indicator I, II, III and panel logit models . . . 23
5.2 Comparison of ROC curves . . . 27
5.3 Second composite indicator’s forecast for the United States . . 29
Acronyms
AUC Area under the ROC curve
ESRB European systemic risk board
EWS Early warning system
FE Fixed effect
FSI Financial stress index
GDP Gross domestic product
IMF International monetary fund
MU Multinomial
NTSR Noise-to-signal ratio
RE Random effect
Chapter 1
Introduction
Recent years have revealed that pursuing price stability in accordance with monetary policy and sound financial institutions in accordance with micro-prudential policy is not sufficient to prevent the financial crises. High economic and social costs of these crises force policy-makers and central bankers to take actions to mitigate their harmful impact on economies.
Financial stability framework is a keystone of any financial stability policy. It is necessary to clearly define several points, such as what the terms financial crisis and systemic risk mean, how they can be measured and monitored, which prudential instruments are capable of mitigating or eliminating these crises and who is responsible for pursuing financial stability in accordance with macro-prudential policy.
There is a generally accepted definition stating that ”systemic risks to finan-cial stability are risks of disruption to the finanfinan-cial system with the potential to have serious consequences for the real economy” (ESRB 2014). It is also understood that financial stability can be threatened by broader sources of risk than banking stability can, for example by insurance sector, pension funds, financial infrastructure or shadow banking. Possible risks to financial stability are measured by statistics reported in the financial stability reports compiled by central banks. Newly the statistics on financial cycles have been added to these reports mostly measured by credit and property prices. Basel III in-troduced the Net Stable Funding ratio and the Liquidity Coverage Ratio to control banks’ reliance on short-term wholesale funding (Northern Rock and Bear Stearns in 2007). Even though this new set of indicators should capture better the current threats to financial stability, it does not predict the future states and the incoming financial crises.
1. Introduction 2
My thesis aims to propose and compare models for predicting financial stress, which is closely related to financial (in)stability. The financial stress is measured by the single valued financial stress index constructed by the Inter-national Monetary Fund for 20 developed countries. This thesis shows that the signalling models give relatively better out-of-sample predictions compared to the logit models. It also reveals that the loss function is a better tool for finding optimal threshold values compared to the noise-to-signal ratio. On the other side, the noise-to-signal ratio has the self-correcting power when used as a weighting variable for constructing a composite indicator.
This thesis is organized as follows: Chapter 2 presents theoretical concept of financial stress and early warning systems used to predict financial stress. Chapter 3 outlines the data set used for empirical analysis. Chapter 4 describes signalling and panel logit models. It also includes description of models’ as-sessment methods. In Chapter 5 the results given by the different models are compared. Chapter 5 also includes additional robustness checks and improve-ments of models. Chapter 6 provides an conclusion.
Chapter 2
Theoretical concepts
2.1
Financial stress
Even though there is no generally accepted definition of financial stress, it can be characterized as an increased uncertainty about fundamental values of assets, market’s impaired ability to intermediate, increased asymmetry of in-formation, decreased willingness to hold risky assets, increased volatility, and common movement of asset prices (Hakkio & Keeton 2009; Carlson et al. 2012). Illing & Liu (2006) propose the below listed scheme, which describes how ex-ogenous shocks to economy and financial markets lead to financial stress.
Figure 2.1: Financial system fragility
Source: Illing & Liu (2006)
Davig & Hakkio (2010) mention two theoretical concepts of how an ex-treme financial stress influences economic activity. The theory of a real option highlights the uncertainty as a driving element of investment decisions. When there is a low uncertainty, investors do not postpone their decisions and fund
2. Theoretical concepts 4
even relatively risky investments. On the other hand, if the level of uncer-tainty on the markets is high, there is a higher probability of extreme events and, therefore, investors might find it more profitable to wait until this un-certainty is resolved. Financial stress brings more unun-certainty into economic activity (higher volatility, increased asymmetry of information) and leads to an investment reduction.
Second theory highlights the importance of a smooth transfer of funds from savers to borrowers followed by their effective allocation. When there is a high financial stress on markets, borrowers have to pay a higher risk premium which in turn leads to a declining economic growth. Banks, expecting this economic downturn, in response increase the risk premium required for money borrowing. In good times, firms have stronger balance sheets, appear to be less risky and therefore pay lower risk premium. As a result, they are encouraged to borrow more funds. This boosts an economic activity and further encour-ages economic growth. Banks are optimistic about the economic outlook and therefore decrease a required risk premium which in turn leads to a steeper eco-nomic growth. This theory is referred to as the financial accelerator framework (Bernanke et al. 1998).
Consistent measuring of financial stress in the economy is essential for the market participants. There are several methods such as observation of set of financial and macroeconomic variables or construction of a proxy variable. Financial stress indexes are single valued indicators that aim to measure the current level of financial stress on the markets. Indexes are usually constructed as a weighted average of several market variables or by a method of principal components (Elekdag et al. 2009; Balakrishnan et al. 2011; Hakkio & Keeton 2009; Kliesen & Smith 2010).
Financial stress index proposed by the IMF is constructed as a sum of seven standardized variables (banking-sector beta, TED spread, inverted terms spreads, corporate debt spreads, stock markets returns, stock market volatility, exchange market volatility). Zero values represent neutral markets conditions on average across the variables. The value of one implies a one standard devi-ation from the average conditions across these seven variables.1
2. Theoretical concepts 5
Figure 2.2: The IMF financial stress index for the United States
Bear Stearns collapse March 2008
Lehman Brothers failure, AIG rescue September 2008 Political debate over TARP
October 2008 Government assistance to Bank of AmericaJanuary 2009
Operation twist announced by FED September 2011 Quantitative easing announced by FED
March 2009
The Dodd−Frank act signed by president Obama July 2010
Financial regulation plan announced June 2009
U.S. housing downturn February 2007
The first in a series of interest rate cuts September 2007 −2 0 2 4 6
Number of standard deviations from a historical mean
2007m1 2008m1 2009m1 2010m1 2011m1 2012m1 Time
Source: The author
2.2
Early warning systems for financial stress events
Early warning system (EWS) is a macroeconomic tool which aims to predict fu-ture non-standard economic conditions, such as currency crises, banking crises, sovereign debt crises, private sector debt crises, equity market crises or finan-cial crises. There are two fundamental assumptions behind the EWS. The first assumption is that factors exist that cause crises. According to the second as-sumption, crisis-driving factors can be identified ex ante (Gramlich et al. 2010). Early warning systems can be divided into four main categories according a se-lected regression specification.
2.2.1
Signalling approach
Signalling approach belongs to a class of non-parametric models. The intuition behind this approach is that each indicator is transformed into a binary vari-able: if it crosses a given threshold, a signal is issued. More signal is issued by individual variables, the crisis is more likely to occur. The thresholds for
2. Theoretical concepts 6
particular variables are chosen to minimise the noise-to-signal ratio or the loss function.
This method was used for early warning systems of currency crisis by Kaminsky (1998) and followed by Lowe & Borio (2002); Borio & Drehmann (2009); Alessi & Detken (2011). Its advantage is simple application and a clear 0/1 definition of crisis occurrence. On the other hand, this simplification into the binary variable brings also significant loss of information compared with the limited dependent variable methods. Also there is an issue how to choose a set of the most relevant indicators to be used in the EWS.
2.2.2
Limited dependent variable methods
This section refers mostly to the logit and probit models which link a binary dependent variable of crisis occurrence with a set of independent variables and estimate a probability of experiencing a crisis. An advantage of these para-metric models is that they take into account interdependencies of explanatory variables. Unlike to signalling approach, these models do not transform ex-planatory variables into the binary variable and therefore mitigate information loss. Davis & Karim (2008) compare signalling approach and logit model for banking crisis and conclude that the logit models better reveal global banking stress while the signal models are more suitable for predicting country-specific crisis.
The limited dependent variable models are also characterized by their non-linearity: the marginal effects are not constant but depend on a precise state of the independent variables.
The multivariate logit approach was first popularized as the early warning system for a banking crisis by Demirg¨u¸c-Kunt & Detragiache (1998). Frankel & Saravelos (2012a) used the probit model for explaining the probability of a currency crisis. Bussiere & Fratzscher (2006) proposed a model for a financial crisis with three-state dependent variable to deal with a post-crisis bias.
Logit and probit models pose a restrictive assumption that the distribu-tional form of data is known. To relax this assumption Christensen & Li (2013) use a semiparametric early warning model which does not require these distributional assumptions.
2. Theoretical concepts 7
2.2.3
Regime-switching model and alternative methods
Regime-switching models belong to a parametric class of models. They assume that there is a latent variable which is not directly observable and its value determines a certain state or a regime. Literature on the early warning systems usually presents models which include two states - tranquil and crisis period (Abiad 2003; Davig & Hakkio 2010).
Dependent variable is a continuous variable and is directly influenced by the unobserved latent factor. The mean and variance of the dependent variable is different in each state. The latent variable can shift from one state to another and this likelihood is given by a set of probabilities (see Figure 2.3).
Figure 2.3: Regime-switching model for financial stress
Source: Davig & Hakkio (2010)
Unlike the previous two classes of models, regime switching models do not require specifically stated past crisis periods by 0/1 variable but take into ac-count continuous index. This feature avoids loss of information and allows for measuring crisis intensity on a continuous scale (Abiad 2003).
Switching models have been used in macroeconomic modelling since the 1970s of the last century. Initially switching models with constant transition probabilities were applied to interest rates (Hamilton 1988), the behaviour of GNP(Hamilton 1989), stock returns (Cecchetti et al. 1990), and floating
2. Theoretical concepts 8
exchange rates (Engel & Hamilton 1990). Recent works use models with time-varying probabilities and apply them on currency crises (Abiad 2003; Arias & Erlandsson 2004; Mouratidis et al. 2013) and financial crises (Davig & Hakkio 2010).
Frankel & Saravelos (2012b) use several bivariate and multivariate regres-sions to investigate whether leading indicators can help explain cross-country incidence of the 2008-2009 financial crisis. Some authors use qualitative and quantitative analysis of macroeconomic and financial variables around crisis periods and compare countries which suffered from a crisis and countries un-touched by any crisis (Edwards & Santaella 1992).
The most recent studies use alternative and innovative methods such as ar-tificial neutral networks (Sevim et al. 2014), regression trees (Ghosh & Ghosh 2003) and the Bayesian model averaging which incorporates model uncertainty (Babeck´y et al. 2012; Eicher et al. 2012; Zigraiova & Jakubik 2014). The Bayesian model averaging gives predictions and estimates by taking into ac-count probabilities of all possible models. This approach partly solves the problem of incorrectly chosen model specification.
Chapter 3
Data description
New quarterly dataset was constructed covering crisis episodes in 20 developed countries over the period of 1981-2010. This dataset was compiled based on data provided by the Czech National Bank and the International Monetary Fund. The data are split into the following two categories: 1) in-sample subset covering period of 1981-2007 and 2) out-of-sample subset including data for 2008-2010 (12 quarters).
The list of countries and their data time range can be found in Table A.1 in Appendix A.
3.1
Crisis occurrence versus crisis incidence
As already mentioned in Chapter 2, the models for early warning systems use both discrete and continuous measures to define a crisis. Discrete measure specifically states the periods of crisis occurrence. The continuous variables overcome the problem of defining a specific threshold and measure rather crisis incidence.
The financial stress index developed by the IMF (Elekdag et al. 2009) is used as a proxy for measuring financial stress. This index is a continuous monthly variable measuring financial stress intensity. The monthly data for each country j were transformed into a quarterly index, F SIjt, taking the highest value in
a given quarter.
Occurrence of a high financial stress event, HF Sjt, is defined as follows,
HF Stj =
(
1 if F SItj > µj+ 1.5σj
3. Data description 10
where µj is a mean and σj is a standard deviation of the quarterly F SIjt.
Authors of the IMF FSI conclude that a value of one standard deviation above country’s mean has in the past been associated with a crisis (Elekdag et al. 2009). This thesis uses 1.5 threshold value in accordance with Christensen & Li (2013) explaining that it reveals high financial stress events which very closely imitate financial market distress. Moreover 1.5 standard deviations provide enough observations for estimating the probability of high financial stress occurrence (Christensen & Li 2013).
All countries in our data set experienced high financial stress events at least five times and at maximum seventeen times during the time period. Taking into account that our data are unbalanced panel, the ratio of the high financial stress events to the number of observations for each country varies from 0.08 to 0.21 implying that heterogeneity in our data set in rather low.
3.2
Independent variables
This thesis aims to explain high financial stress events by 10 macroeconomic and financial variables (see Table 3.1). These variables are a subset of data originally used in Babeck´y et al. (2012) analysing the banking, debt and currency crises in 40 countries over the period of 1970-2010.
Table 3.1: Data description
No. Variable Description Transf. Ex. Sign
1 baaspread BAA corporate bond spread none + 2 curaccount Current account (% GDP) none -3 indshare Industry share (%GDP) none + 4 mmrate Money market interest rate none + 5 shareprice Stock market index % qoq -6 taxburden Total tax burden (% GDP) none +
7 winf Global inflation none
+/-8 wrgdp Global GDP % qoq
-9 wtrade Global trade (constant prices) % qoq -10 yieldcurve Long term bond yield - none
-money market interest rate
Table 3.1 includes descriptions and transformations of the independent vari-ables. It also includes an expected impact of each variable on the probability of high financial stress. The positive sign means that a higher value of a given variable leads to higher probability of high financial stress and the negative
3. Data description 11
sign means that the lower value of variables leads to higher probability of high financial stress.
Large spreads on BAA corporate bonds are a sign of higher uncertainty on markets which might increase financial stress. A rapid decline in a current account is expected to have a negative influence on the financial stability, as its drop may signify worsening of macroeconomic conditions. Generally, industrial share as a percentage of GDP has declined in western European countries compared to the previous century. Developed countries have a lower industrial share. Therefore an increase in the industrial share can be viewed as a relative decrease in the financial sector.
A significant increase of money market interest rates can be caused by higher uncertainty on markets and in turn leads to a lower lending and to an increase in financial stress. The share price index is expected to decline in bad times and grow in good times. Tax burden is a way how any government can lower its deficit or finance economic rescue attempts. An increase in tax burden is expected to signifies worsening economic conditions.
This thesis uses global variables on inflation, GDP and trade instead of country specific variables. It corresponds to the findings in Babeck´y et al. (2012) where the global variables are determined as the superior indicators to the domestic variables for banking crises. Babeck´y et al. (2012) conclude that the developed countries which are represented in my sample are more integrated into the global markets and therefore the global variables are better warning indicators. Even thought this thesis develops the early warning system for financial stress events, the global variables are used and they are expected to reflect general macroeconomic conditions.
The high global inflation rate means that currencies are losing their pur-chasing power and prices are rising, leading to higher global financial stress. On the other side, the global deflation is also very harmful and it could lead to the deflationary spiral. Declining GDP growth is a signal of economic recession and inevitably leads to worsening stability of a global financial sector. The decline in a global growth rate of trade may signify global economic recession. The last economic variable used in models is the difference between the long-term and short-long-term yield. Generally there are two main arguments for the high term spread. The first argument is that the short-term spread is low. That can usually happen when the national authorities want to boost economic activity during recession. In this sense the higher spread is associated with a current economic slowdown and possible economic instability. The second reason for
3. Data description 12
high term spread could be the high long-term interest rate which can be driven by high inflation expectations and potential future gdp growth. Moreover, the empirical fact is that most recessions are preceded by a sharp decline in the term spread (Wheelock & Wohar 2009).
Chapter 4
Model presentation
As explained in Chapter 3, the high financial stress event in a country j at time t, HF Stj, is defined as a situation when the financial stress index of a country
j is 1.5 standard deviations above its mean. This variable is transformed into a forward-looking variable Ytj, which is defined as
Ytj =
(
1 if ∃k = 1, ..., 4 s.t. HF St+k,j = 1
0 otherwise.
Ytj is crisis signalling variable which equals one if there is at least one period
of high financial stress in a following four quarters.
The signalling horizon is four quarters so this forward looking variable equals one if there is at least one high financial stress event within consec-utive four quarters1
4.1
Signalling model
The logic behind the signalling approach is explained in Chapter 2. The set of ten independent variables used to forecast periods of high financial stress is described in Chapter 3.
Each macroeconomic and financial variable, Xtj, is standardized and
trans-formed into a binary variable, Stj,
1I am aware of a trade-off between policy-makers demand for a long-term forecasts and
a credibility of this forecasts. Some authors use six quarters as a forecasting window but mostly for banking or currency crises. (Alessi & Detken 2011). In this thesis I use four quarters due to higher volatility of the IMF financial stress index.
4. Model presentation 14
Stj =
(
1 |Xtj| > |Xj∗|
0 otherwise
Sjt gives a signal when the corresponding variable crosses its threshold.
The trade-off problem of evaluating signalling models
The key issue of any signalling model is to define the threshold. This issue is similarly important in the logit models and it will be discussed later. If the threshold is too low, the signalling model gives too many signals and therefore it looses its informative value. If the threshold is too high, the model does not warn against incoming crisis events.
Generally there are four possible outcomes of signalling model as presented in Table 4.1.
Table 4.1: Signal scenarios
Crisis Y (Within 4 quarters) No crisis Y (Within 4 quarters) Signal S issued A B No signal S issued C D
Source: The author
Let’s define at as a correctly predicted high financial event at time t, or
in another words, a situation when signal, Stj, is issued and at least one high
financial stress event occurs within following four quarters. Then A is a sum of all signals at issued by indicators across the whole time interval, A =
PT
t=1at.
If the signal is issued and there is no high financial event in the given period then the outcome is denoted by bt. B is a sum of all signals bt. If no signal is
emitted by the indicators but there was a crisis, the sum of these outcomes is defined as C. If there is no crisis and no signal is issued, the state dt occurred
and the sum of them is denoted D. Therefore, a hypothetical perfect threshold would be such that the indicator only gives correct predictions, that is, only outcomes A and D, and never outcomes C and B.
Three methods how to choose an appropriate threshold value for each model are discussed: 1) the noise-to-signal ratio (NTSR), 2) receiver operating char-acteristic (ROC) curve and 3) loss function.
1) Threshold value Xj∗ can be chosen to minimize the N T SR which is defined as follows,
4. Model presentation 15 N T SR = B B+D A A+C = 1 − specificity sensitivity ,
where specificity, B+DD , is a percentage of the correctly predicted non-crisis episodes and sensitivity, A+CA , is a percentage of the correctly predicted crisis episodes. An indicator with the optimal threshold has zero NTSR because it gives non zero A and D outcomes and zero B and C outcomes.
2) The trade-off between sensitivity and specificity is depicted by ROC curve which plots sensitivity on y-axis and 1-specificity on x-axis for different threshold values (see Figure 4.1).
Figure 4.1: A ROC curve example
0 .0 0 0 .2 5 0 .5 0 0 .7 5 1 .0 0 Se n si ti vi ty 0.00 0.25 0.50 0.75 1.00 1 - Specificity
Area under ROC curve = 0.7377
Source: The Author
The area under the ROC curve of any indicator I, AUC(I), is a convenient summary statistic of its signalling quality (Drehmann & Juselius 2014),
AU C(I) = Z 1
0
ROC(1 − spec(I))d(1 − spec(I)).
The fully informative indicator has the AUC statistic equal to one and its ROC curve is on the left side of Figure 4.2. On the right side of the same figure, the fully uninformative ROC curve almost coincides with 45◦ degrees line and
4. Model presentation 16
its AUC statistic is approaching 0.5. The AUC(I) statistic is larger than 0.5 if the indicator I is growing in crisis periods and lower in calm periods. The AUC statistic serves as an evaluation of models relative performance without specifying thresholds.
Figure 4.2: Fully informative and uninformative ROC curves
0 .0 0 0 .2 5 0 .5 0 0 .7 5 1 .0 0 Se n si ti vi ty 0.00 0.25 0.50 0.75 1.00 1 - Specificity
Area under ROC curve = 1.0000
0 .0 0 0 .2 5 0 .5 0 0 .7 5 1 .0 0 Se n si ti vi ty 0.00 0.25 0.50 0.75 1.00 1 - Specificity
Area under ROC curve = 0.5052
3) The last method minimizes the loss function, L, which is a weighted sum of the Type I error, C
A+C, and the Type II error, B B+D, L = θ C A + C + (1 − θ) B B + D.
The parameter θ reveals the policy maker’s relative risk aversion between Type I and Type II errors. Alessi & Detken (2011) suggest that θ should be less than 0.5 and argue that policy-makers are less concerned about missing a signal, as a cost of an action relying on a false alarm is higher (public pressure and tight monetary policy) than a very accommodative monetary policy. On the other hand Bussiere & Fratzscher (2006) argue that Type II errors are less important, as Type II errors tend to be less costly than Type I errors and Type II errors do not have to be caused by imprecisions of our model. They argue that Type II errors might arise when the model gives a warning signal and consequently the appropriate policy measures are taken and they lead to crisis elimination. Based on the above stated I will use delta equal to 0.4.
Overall assessment
Each of 10 explanatory variables give a 0/1 signal at time t. Therefore at max-imum ten positive signals can be received in one period. Composite indicator, ComIt, is defined as a sum of all signals atand btat time t given by all variables
4. Model presentation 17 i, ComIt= 10 X i=1 ati+ bti, where ComIt∈0, 1, ..., 10 .
The second composite indicator, ComIIt, takes into account different
fore-casting accuracy of each variable and this indicator was proposed by Chris-tensen & Li (2013). It is defined as a weighted sum of all signals at and bt in
time t given by all variables i,
ComIIt= 10 X i=1 ati+ bti N T SRi ,
where the NTSR of each indicator is used as a weighting variable. Therefore indicators with the low NTSR have larger influence on the final composite indicator.
I propose my own composite indicator which is a weighted average of all signals at and bt in time t given by all variables i,
ComIIIt= 10 X i=1 ati+ bti Li ,
where the loss function Lt of each indicator serves as a weighting variable.
Again variables with low value of the loss function Lt have larger influence on
the final composite indicator.
4.2
Panel logit model
Description of the binary panel logit models is written based on Cameron & Trivedi (2005). In the previous section, the dependent variable, Yjt, is a
forward-looking binary variable which equals one if there is at least one high financial stress event within the next four quarters in a country j.
The probability of at least one high financial stress event in the consecutive four quarters is defined as pjt,
ptj = Prob(ytj = 1|xtj, β, αj) = F (αj + x 0 tjβ).
The F () is a logistic cumulative distribution function and xtj is a K ×
4. Model presentation 18
The ytj follows the Bernoulli distribution assumed that the sample (ytj, xtj) is
independent over j and t,
f (yj|xtj, β, αj) = T
Y
t=1
(ptj)ytj(1 − ptj)1−ytj. (4.1)
The term αj captures time-invariant unobserved country specific
hetero-geneity. The panel logit models can be distinguished based on an assumption about αj.
It would be ideal to compare predicted probabilities of high financial stess events obtained from these logit models with the actual probabilities. How-ever actual probabilities can not be observed. Therefore, the threshold for the predicted probabilities has to be chosen in order to predict outcomes, in the same fashion as in the signalling model. These thresholds are chosen based on techniques explained in Section 4.1.
Random effects logit model
The random effect logit model treats αj as an unobserved random variable that
is distributed independently from the explanatory variables. Individual effects are assumed to be normally distributed with αj ∼ N (0, σ2α). The random
effects (RE) maximum likelihood estimator of coefficients β and σ2α maximizes the log-likelihood PN j=1lnf (yj|xj, β, σ 2 α), f (yj|xj, β, σ2α) = Z f (ytj|xtj, β, αj) 1 p2πσ2 α exp −αj 2σ2 α 2 dαj,
where f (ytj|xtj, β, αj) is given in Equation 4.1.
The predicted probabilities are defined as a probability of a positive outcome assuming that the random effect is zero.
This estimator is convenient if countries included in the data set were ran-domly sampled from a large population of countries. However this assumption is questionable as the IMF financial stress index was constructed for twenty developed countries. Moreover the random effects model assumes that the country effects do not correlate with the independent variables, which seems to be unrealistic. For example if the political situation or other variables not included in the regression has an influence on financial stress, it is expected to have influence on the other independent variables, particularly government bal-ances or stock market indexes. For this reason conditional fixed effects model
4. Model presentation 19
seems to be an adequate substitution. Nevertheless, results of the RE logit model are included as the comparison to the other models.
Conditional fixed effects logit model
Conditional fixed effect logit model treats αj as an unobserved random
vari-able that is potentially correlated with the observed explanatory varivari-ables xtj.
Chamberlain (1980) proposed a consistent estimator which eliminates αj and
allows a consistent estimation of coefficients.
For each country j, there is a sequence of 0/1 signals, yjt, and their sum
P
tytj is a number of outcomes 1 in the period T. The Bc = [dj|P dtj =
P
tytj = c] is a set of all possible sequences of 0s and 1s for which the sum of T
binary outcomes is equal to c. By conditioning onP
tytj = c we can eliminate αj and f (yj| X t ytj = c, xj, β) = exp((P tytjx 0 tj)β) P d∈Bcexp(( P tdtjx 0 tj)β) .
An advantage of this estimator is that it is consistent even if the explanatory variables are correlated with the individual effect αj.
One drawback is that there are many sets Bc and sequences within these
sets and it is not possible to condition on P
tytj = 0 and P tytj = T . These restrictions, P tytj = 0 and P
tytj = T , mean that all countries that
did not experience a crisis in the past and these that got permanently stuck in a crisis have to be excluded from the dataset. However this drawback is not restrictive for our dataset as each country has experienced high and low financial stress periods.
The predicted probabilities are defined as a probability of a positive out-come conditional on one positive outout-come within group. It means that the probabilities are generated as a probability of one high financial stress event within next one year conditional on that there is one high financial stress event in each country.
Chapter 5
Results
As mentioned in Chapter 3, the quarterly dataset is divided into the in-sample subset covering the period of 1981Q1-2007Q4 and the out-of-sample subset covering the period of 2008Q1-2010Q41 (12 quarters). The in-sample data set is cleaned from the post-crisis bias and ‘hidden crisis’ events.
The post-crisis bias arises when results given by a model are influenced and partly explained by behaviour of independent variables after a crisis (Bussiere & Fratzscher 2006). To control for this bias, post-crisis observations were excluded from the in-sample data set used for models’ estimation.
To illustrate existence of the post-crisis bias, the variables’ subsample means for the Unites States are listed in Table 5.1 which reveals that means in tranquil periods differ from means in crises and post-crisis periods. Mean of the tax burden and inflation are discussed in Section 5.2.
The ‘hidden crisis’ events were also eliminated from the in-sample data set. These are the situations when a high financial stress event in the first period followed by a non-crisis quarter in the second period. After that the high financial stress event occurs again. The situation in the second period can be view as the ”hidden” crisis because the financial stress index did not reach its crisis threshold however the crisis is eminent in the surrounding periods.
Results of the in-sample and the out-of-sample estimation are evaluated based on several criteria. The probability of correct estimation of at least one high financial stress event within following four quarters, sensitivity, is defined as A+CA . The probability of a false alarm given that there is no high financial stress event within four quarters, 1- specificity, is B+DB . The probability of high financial stress event given that the alarm is issued, P[crisis|signal], is
1The IMF financial stress index for the year 2011 was taken into account when creating
5. Results 21
Table 5.1: Crisis ans post-crisis means of key indicators for the United States
Indicator Mean, Mean, Mean, Mean,
all periods tranquil crisis post-crisis
baaspread 1.83 1.74 2.19 2.53 mmrate 5.75 5.23 8.11 8.26 shareprice 3.15 3.19 4.18 -2.53 taxburden 33.03 33.01 33.25 32.55 winf 12.12 12.19 10.76 16.96 wrgdp 4.56 5.71 -3.30 9.92 wtrade 1.83 1.84 1.87 1.62
Source: The author
defined as A+BA . The percentage of all correctly estimated events is computed as A+B+C+DA+D .
5.1
Finding optimal threshold values in signalling
models
Threshold values for each individual economic variables were computed based on minimizing the noise-to-signal ratio and the loss function (δ = 0.4) and also restricted by risk-tail boundaries from 75th to 99th percentiles.2. The results for both methods are enclosed in Appendix A The threshold values computed based on minimizing the NTSR are higher and give a small number of signals. This is caused by the definition of the NTSR as for value A very close to zero the NTSR is approaching zero as well. High number of signals given by the loss function is preferable since all signals are grouped into the composite indicators and the threshold values for these composite indicators can be chosen from the broad spectrum of values.
Thresholds for the composite indicators ComI, ComII, ComIII and the logit models were also determined based on minimizing the NTSR ratio and the loss function (δ = 0.4) and bounded by the risk-tail boundaries from 75th to 99th percentile. The resutls are similar to that for the individual economic
2Each threshold value is obtained by a grid search and its value is restricted to lie between
75th and 99th percentile of a corresponding variable distribution. The boundaries are used to assure that the threshold value is an extreme value. I follow the approach proposed by Christensen & Li (2013) who used even narrower boundaries.
5. Results 22
variables. Table 5.2 presents a comparison of the individual thresholds found by different methods.
Table 5.2: Comparison of thresholds given by different methods
Indicator NTSR Loss function 1-specificity = 0.175
Composite ind. I 6 3 3
Composite ind. II 11.42 4.92 4.93
Composite ind. III 6.04 6.04 6.04
RE model 0.941 0.359 0.325
FE model 0.078 0.011 0.011
Source: The author
The thresholds given by minimizing the NTSR are for almost all indicators too high and give a small number of signals (see the red point with NTSR label in Figure 5.1). Vertical red lines corresponding to each red point represent the 95% confidence interval computed based on bootstrap standard errors with 1000 replications. The long-dash red line shows the level of specificity on which the indicators are compared.
Low sensitivity and high specificity for thresholds minimizing the NTSR ratio are caused by the structure of this ratio. The minimal values are reached when the statistic A is high, or in another words a high number of correct signals, and a low statistic B, a low number of false signals are received. In case that the B is equal to zero and, at the same time, the statistic A is nonzero, the NTSR is equal to zero. Restrictions on the thresholds posed by the risk tail boundaries can partly mitigate a tendency to give low number of signals (see Figure 5.1(c)) however the boundaries should be narrower to give reasonable threshold for all composite indicators. On the other hand, the loss function gives reuslts on a reasonable level of specificity even with wide boundaries.
The first composite indicator, ComI, can reach only 11 integers, from 0 to 10. It means that the threshold value can be set in 12 different positions. The blue points in Figure 5.1(a) depict these 12 different threshold positions. The second and the third composite indicator, ComII and ComIII, can reach more values than the first one as it is weighted sum different weights for each indicators signal (see Figure 5.1(b) and 5.1(c)).
5. Results 23
Figure 5.1: ROC curves for the composite indicator I, II, III and panel logit models NTSR Loss 0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 1-Specificity Se nsitiv ity
(a) Com. indicator I
Se nsitiv ity Loss NTSR 0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 1-Specificity Se nsitiv ity (b) Com. indicator II NTSR=Loss 0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 1-Specificity Se nsitiv ity
(c) Com. indicator III
NTSR Loss 0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 1-Specificity Se nsitiv ity (d) RE model Loss NTSR 0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 False-positive rate Se nsitiv ity (e) FE model
5. Results 24
The ROC curves in Figure 5.1 use a stair-step line connection style instead of a trapezoidal approximation. The AUC statistic computed based on the stair-step lines is lower for indicators with very few threshold values compared to the AUC statistics computed based on trapezoidal approximation. The trapezoidal approximation is more convenient in case of continuous indicators which is not the case of the first composite indicator. The AUC statistics computed by both methods are very close for indicators with many threshold positions, for e.g. indicators computed based on random effect and fixed effect panel logit models (in Figure 5.1(d) and Figure 5.1(e) the blue bold line comprises all positions of threshold values). The downward bias of the AUC statistic of an indicator with a few threshold positions is larger with stair-step connection style but even with the trapezoidal approximation the AUC statistic is biased. On the other side we can also understand this bias as a punishment for a limited number possible thresholds.
As there is a limited number of possible threshold positions for the first composite indicator, ComI, the benchmark value of 1-specificity, 0.175, was chosen in a way that it is given by the threshold found based on minimizing the loss function with δ = 0, 4 for ComI (see the red long-dashed line in Figure 5.1(a)). This benchmark value is chosen to make all models comparable. This benchmark value, 0.175, is also depicted in Figure 5.1(b) - Figure 5.1(e).
5.2
Performance of individual economic indicators
in signalling and logit models
The performance of individual economic and financial indicators in signalling models is presented in Table 5.3. The signalling scenarios are for in-sample and out-of-sample data are enclosed in the Appendix (Table A.2 and Table A.3).
Based on comparison of AUC statistics, the money market interest rate, yield curve, and BAA corporate bond spread perform relatively better than total tax burden or global inflation.
Sensitivity and 1-specificity is also very low for tax burden and inflation because its thresholds are too high even though the risk-tail boundaries were used for a grid search.3 The NTSR for inflation is approaching infinity because
3Each threshold value is obtained by a grid search and its value is restricted to lie between
75th and 99th percentile of a corresponding variable distribution. The boundaries are used to assure that the threshold value is an extreme value. I follow the approach proposed by Christensen & Li (2013) who used even narrower boundaries.
5. Results 25
Table 5.3: Performance of economic and financial variables
Stat baaspread curaccount indshare mmrate shareprice
AUC 0.627 0.523 0.619 0.665 0.563
NTSR 0.528 0.551 0.538 0.411 0.46
L 0.377 0.387 0.375 0.333 0.373
Sensitivity 0.276 0.192 0.318 0.437 0.2107
1-specificity 0.146 0.106 0.171 0.180 0.096
Stat taxbrdn winf wrgdp wtrade yield
AUC 0.526 0.572 0.626 0.599 0.673
NTSR 0.741 inf 0.496 0.576 0.402
L 0.403 0.407 0.359 0.391 0.329
Sensitivity 0.061 0 0.406 0.168 0.444
1- specificity 0.045 0.013 0.201 0.097 0.178
Threshold values found by minimizing the loss function with δ = 0.4 and restristed by the risk-tail boundaries from 75th to 99th percentile.
Source: The author
the A statistic is zero.
The low predictive power of the tax burden can be expected because it reflects long term economic conditions and does not vary in time (see Table 5.1 with means in the tranquil and crisis periods). The mean of the rate of inflation is surprisingly low during crisis periods and very high during the post crisis periods and it reveals that both high inflation and deflation are threats to economy.
When we consider the regression results of the random-effects and fixed-effects logit models, we observe that the coefficient of the rate of inflation in the random effect and fixed effect model have a negative sight so it means that the deflation increases the probability of high financial event in a following four quarters more than the high rate of inflation (see Table 5.4). Other coefficients of individual variables have expected sign except the rate of inflation and the world trade. The BAA corporate bond spread and the world trade are in-significant in both models. The tax burden is in-significant and have an expected positive sign.
5. Results 26
Table 5.4: Estimated coefficients by the fixed effect and the random effect model
Variable Ex. Sign RE sign FE sign
baaspread + 0.255 0.271 curaccount - −0.165∗∗∗ −0.242∗∗∗ indshare + 0.198∗∗∗ 0.237∗∗∗ mmrate + 0.266∗∗∗ 0.280∗∗∗ shareprice - −0.020∗∗ −0.021∗∗ taxburden + 0.079∗∗ 0.167∗∗∗ winf + −0.095∗∗∗ −0.111∗∗∗ wrgdp - −0.014∗∗ −0.012∗ wtrade - 0.042 0.367 yieldcurve + −0.204∗∗ −0.196∗ Panel robust errors, see Table A.8 for detailed results. * p < .1, ** p < .05, *** p < .01
5.3
Early warning model comparison
This subsection compares early warning indicators constructed by signalling models and panel logit models.
The second composite indicator, ComII, is a weighted sum of signals where the weighting variable is the NTSR. The ComII can eliminate low predictive variables and give larger weight to variables with low NTSR. Therefore it has self-correcting power and robustness to a misspecification. It follows that infla-tion divided by its NTSR is eliminated from the ComII (see Table 5.3). The impact of the tax burden on the ComII is also relatively lower due to its high NTSR.
The third composite indicator, ComIII, is a weighted sum of signals where the weighting variable is the loss function. The loss function is consistent with the NTSR and gives the lowest weights to the tax burden and the rate of inflation. However the value of the loss function for the rate of inflation exceeds the value for the yield curve only by 0.078. It shows that the ComIII eliminates the unpredictive economic variables in very low scale compared to ComII. It rather treats all variables equally as the ComI.
The ROC curves for all composite indicators and panel models are depicted in Figure 5.2. The AUC statistic measures overall performance regardless of any specific thresholds. The ComI indicator has the lowest AUC statistic. The fixed effect logit model has the highest AUC statistic and it outperforms the random effect logit model by 0.4 AUC point. However for any specific value of
5. Results 27
Figure 5.2: Comparison of ROC curves
0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 1-Specificity ComI AUC: 0.645 ComII AUC: 0.708 ComIII AUC: 0.705 RE AUC: 0.714 FE AUC: 0.755
Sensitiv
ity
specificity the fixed effect model might be outperformed by another model on the percentage of correctly predicted events. The AUC statistics of the ComII, the ComIII and the random effect logit model are very close. The p-values of pairwise Wald tests on equality of the AUC statistics are enclosed in Table A.10. The null hypothesis that two AUC statistics are the same is tested for all combinations, in total 10 tests.
The null hypotheses testing the pairs including the fixed effect model were rejected on 10.5% significance level for all combinations. The Wald test could not reject the AUC statistic equality for the random effect logit model and the first two composite indicators, ComII and ComIII, on 10% significance level. Other tests rejected the null hypothesis on 10% significance level.
In-sample performance of all models is relatively similar. The results in Table A.4 and in Table 5.5 show that the fixed effect model has the highest percentage of correctly predicted events and the highest probability of crisis occurrence given that the signal is issued at 0.825 level of specificity.4 These
results go together with the AUC statistics comparison.
5. Results 28
Table 5.5: In-sample results
Indicator AUC Sensit. 1-specif. NTSR P[Y=1|S=1] Corr(%)
ComI 0.645 0.471 0.175 0.372 0.426 74.8
ComII 0.708 0.471 0.171 0.363 0.432 72.9
ComIII 0.705 0.471 0.175 0.372 0.426 74.8
RE logit 0.714 0.479 0.174 0.363 0.431 75.1
FE logit 0.755 0.532 0.173 0.325 0.459 76.3
Source: The author
Table A.5 and Table 5.6 show out-of-sample signal scenarios and compar-ative statistics. The composite indicators I, II and III have identical out-of-sample forecast and outperform panel models in all observed indicators.
Table 5.6: Out-of-sample results
Indicator Sensit. 1-specif. NTSR P[Y=1|S=1] Correct(%)
ComI 0.667 0.154 0.230 0.821 75.3
ComII 0.667 0.154 0.230 0.821 75.3
ComIII 0.668 0.154 0.230 0.821 75.3
RE logit 0.444 0.236 0.532 0.667 59.9
FE logit 0.504 0.209 0.415 0.720 64.3
Source: The author
The forecast given by the second composite indicator for the United States is in Figure 5.3. The shaded regions cover periods with the high probability of a high financial stress event. The blue bars represent individual signals which warn against high financial stress events in following four quarters. The graph includes all major financial stress episodes identified in literature and mentioned in Elekdag et al. (2009).
5.4
Improving panel logit models
5.4.1
Self-explanatory power
The Financial Stress Index can be suspected of having some time series per-sistence. Therefore the index was included as an explanatory variable in panel logit models. In-sample results of improved models are slightly better compared
5. Results 29
Figure 5.3: Second composite indicator’s forecast for the United States
Source: The author
to basic models.5 However the percentage of correctly predicted out-of-sample
events increased by 8.8% and 6.6% for random effect and fixed effect models, respectively. Also the probability of a crisis given that the signal was issued increased (see Table 5.7 for detailed results).
5.4.2
The post-crisis bias and the multinomial logit approach
At the beginning of Chapter 5, the post-crisis bias was explained to be caused by including post-crisis observations in the data set used for estimating the coefficients. One way how to address this issue is to remove all the post-crisis observations from the data set used for estimating. This approach was used for models up to this point. However, removing any observations from the data
5Models are compared on 0.825 level of specificity. Thresholds are 0.320 and 0.007 for the
5. Results 30
Table 5.7: Results for improved panel logit models
Indicator AUC Sensit. 1-specif. NTSR P[Y=1|S=1] Correct(%)
In-sample RE* logit 0.734 0.502 0.175 0.349 0.441 75.5 FE* logit 0.761 0.536 0.174 0.325 0.459 76.3 Out-of-sample RE* logit 0.624 0.245 0.393 0.730 68.7 FE* logit 0.641 0.218 0.340 0.758 70.9
Source: The author
set leads to a loss of information on the period when independent variables are recovering.
The mutlinomial logit model is the second option how to address the post-crisis bias and was proposed by Bussiere & Fratzscher (2006). Its advantage is that it uses all information from the dataset. The in-sample forward-looking variable YtM is redefined as a three state dummy variable in the following way,
YtM = { Yt if biast= 0 2 if biast= 1,
where Yt is a two-state variable defined in Chapter 4 and biast is a dummy
variable equal one if the observation was removed due to the post-crisis bias or the ‘hidden’ crisis problem in the models with the two state dependent variable.
The state YM
t = 0 refers to a tranquil period, when the financial stress is
below the threshold. The state YM
t = 1 describes a period of high financial
stress and the YM
t = 2 depicts the post-crisis periods and the ’hidden’ crisis
periods.
The predicted probabilities of crisis occurrence were estimated by multi-nomial logit model where the regressors do not vary over alternatives. The probability of outcome i = 0, 1, 2 is defined as
pi,t = ex 0 tβi P2 i=0ex 0 tβi .
Because the sum of probabilities over all alternatives, P2
i=0pit, is one, a
5. Results 31
The thresholds found by minimizing the NTSR6 and the loss function (δ =
0.4) are equal to 0.706 and 0.269, respectively (see Figure 5.4). The threshold corresponding to 0.825 specificity level is equal to 0.254.
Figure 5.4: Multinomial logit’s ROC curve
NTSR
Loss
0 .2 5 .5 .7 5 1 0 .25 .5 .75 1 1-SpecificitySe
nsitiv
ity
The three state variable is used for in-sample signal scenarios because we can define the third state only ex post. However, for out-of-sample forecasting, the ex post redefinition is not possible as the future states are unknown in advance. For this reason, the original two state dummy variable is used for out-of-sample forecasting (see Table A.7).
Table 5.8 presents results for the multinomial logit models. In* and Out* refer to the in-sample and out-of-sample results for models including the IMF financial stress index as an explanatory variable.
The percentage of correctly predicted events given by both multinomial models is very close to that of panel logit models with two-state dependent variables. The probability of a crisis given that the signal is issued is also comparable to the panel logit models. The AUC statistic is equal to 0.730
6For computation of the thresholds the original two state Y
tvariable defined in Chapter
4 was used. The state YM
5. Results 32
which is very close to the AUC statistic of the improved random logit model (for testing equality of AUC statistic see Table A.10).
Table 5.8: Results for the multinomial logit model
Sample Thold Sensit. 1-specif. NTSR P[Y=1|S=1] Correct(%)
In 0.254 0.486 0.175 0.360 0.465 74.4
In* 0.259 0.476 0.174 0.366 0.461 75.3
Out-of 0.254 0.521 0.300 0.576 0.649 60.8
Out-of* 0.259 0.308 0.273 0.886 0.545 51.1
* refers to the models including the FSI as an explanatory variable.
Source: The author
The percentage of correctly predicted out-of-sample events is lower for the multinomial model where the financial stress index was included as an explana-tory variable. This is caused by pooling data together without considering panel data structure.
The signalling model can be viewed as a panel data model because each macroeconomic and financial variable Xj,t is standardized with respect to the
corresponding country mean and the standard deviation. The fixed effect and random effect logit models also take into account the panel data structure even thought with different assumptions.
Only the multinomial logit model pools data without taking into account panel structure. It is therefore reasonable to check poolability of data. It is analysed by estimating the same model for a different subset of countries and different time periods.
Two cross-section subsamples were used instead of - European countries and non European countries7. For the European countries the sign of coefficient has
changed for 3 variables at the state YM = 1 and for 3 variables at state YM = 2
compared to the panel models. The signs of the other variables remain same. For the non European countries, the sign of three variables also changed. The significant levels are different for many variables. So it seems better not to pool all countries without controlling for country specifications when creating the early warning system for the financial stress events.
The multinomial logit model even if estimated with financial stress index as an explanatory variable does not increase the percentage of correctly predicted
5. Results 33
events. Also pooling data without considering panel data structure is not convenient tool for correcting post-crisis bias.
Chapter 6
Conclusion
The financial crisis that burst out in 2007 has revealed that pursuing price sta-bility in accordance with monetary policy and existence of sound financial in-stitutions in accordance with micro-prudential policy were not sufficient means to maintain financial stability. It is also commonly understood that financial imbalances can lead to severe financial crises and recessions. As such this thesis aims to analyse and compare the use of the logit and signalling model as early warning systems for financial stress events. The IMF financial stress index is used as a proxy for measuring financial stress and imbalances in 20 developed countries over the period of 1981-2010. This thesis follows an approach of Christensen & Li (2013) and contributes by broadening the analysed data set and considering new techniques for predicting high financial stress events.
The results show that the use of the loss function for finding the thresholds, which differentiate extreme events from normal states, gives results with rea-sonable level of type I and type II errors and can be easily modified according to policy-makers’ needs. This thesis also reveals that the noise-to-signal ratio if used as a weighting variable, can eliminate variables with low predictive power from the signalling models and gives high impact to variables with high pre-dictive power. Moreove this thesis improves the logit models by including the financial stress index as an explanatory variable, further increasing their pre-dictive accuracy. It reveals that prepre-dictive accuracy of improved logit models is still lower than that of signalling models. The thesis also tried to address the post-crisis bias by multinomial logit model as suggested by Bussiere & Fratzscher (2006). Even though pooling data set over countries does not bring any improvement on predictive accuracy.
6. Conclusion 35
similarly well fit the in-sample data, however, the signalling models outperform the pane logit models on out-of sample forecasting accuracy. It is in accordance with results of Davis & Karim (2008) that logit models can better fit data for a large set of countries and they are more appropriate as a global early warning system. On the other hand, the signalling model serves better as an early warning system when considering specific countries or a set of common countries. The second option is a case of this thesis.
Further research should examine the suitability of switching models and their comparison to the signalling and logit models. It would be also beneficial to take into account a broader set of explanatory variables, such as data on house prices or private consumption expenditures, and resolve the problem of model uncertainty by Bayesian model averaging.
Bibliography
Abiad, A. (2003): “Early warning systems: A survey and a regime-switching approach.” IMF Working Papers 03/32, International Monetary Fund. Alessi, L. & C. Detken (2011): “Quasi real time early warning indicators for
costly asset price boom/bust cycles: A role for global liquidity.” European Journal of Political Economy 27(3): pp. 520–533.
Arias, G. & U. Erlandsson (2004): “Regime switching as an alternative early warning system of currency crises - an application to South-East Asia.” Working Papers 2004:11, Lund University, Department of Economics. Babeck´y, J., T. Havr´anek, J. Matˇej˚u, M. Rusn´ak, K. ˇSm´ıdkov´a, &
B. Vaˇs´ıˇcek (2012): “Banking, debt and currency crises: early warning in-dicators for developed countries.” Working Paper Series 1485, European Central Bank.
Balakrishnan, R., S. Danninger, S. Elekdag, & I. Tytell (2011): “The Transmission of Financial Stress from Advanced to Emerging Economies.” Emerging Markets Finance and Trade 47(0): pp. 40–68.
Bernanke, B., M. Gertler, & S. Gilchrist (1998): “The Financial Accel-erator in a Quantitative Business Cycle Framework.” Working Papers 98-03, C.V. Starr Center for Applied Economics, New York University.
Borio, C. & C. M. Drehmann (2009): “Towards an operational framework for financial stability: ’Fuzzy’ measurement and its consequences.” BIS Working Papers 284, Bank for International Settlements.
Bussiere, M. & M. Fratzscher (2006): “Towards a new early warning sys-tem of financial crises.” Journal of International Money and Finance 25(6): pp. 953–973.
Bibliography 37
Cameron, A. C. & P. K. Trivedi (2005): Microeconometrics. Number 9780521848053 in Cambridge Books. Cambridge University Press.
Carlson, M. A., K. F. Lewis, & W. R. Nelson (2012): “Using policy in-tervention to identify financial stress.” Finance and Economics Discussion Series 2012-02, Board of Governors of the Federal Reserve System (U.S.). Cecchetti, S. G., P.-s. Lam, & N. C. Mark (1990): “Mean Reversion in
Equilibrium Asset Prices.” American Economic Review 80(3): pp. 398–418. Chamberlain, G. (1980): “Analysis of Covariance with Qualitative Data.”
Review of Economic Studies 47(1): pp. 225–38.
Christensen, I. & F. Li (2013): “A Semiparametric Early Warning Model of Financial Stress Events.” Working Papers 13-13, Bank of Canada.
Davig, T. & C. Hakkio (2010): “What is the effect of financial stress on economic activity.” Economic Review (Q II): pp. 35–62.
Davis, E. P. & D. Karim (2008): “Comparing early warning systems for banking crises.” Journal of Financial Stability 4(2): pp. 89–120.
Demirg¨uc¸-Kunt, A. & E. Detragiache (1998): “The determinants of banking crises in developing and developed countries.” Staff Papers-International Monetary Fund pp. 81–109.
Drehmann, M. & M. Juselius (2014): “Evaluating early warning indicators of banking crises: Satisfying policy requirements.” International Journal of Forecasting 30(3): pp. 759–780.
Edwards, S. & J. A. Santaella (1992): “Devaluation Controversies in the Developing Countries: Lessons From the Bretton Woods Era.” NBER Work-ing Papers 4047, National Bureau of Economic Research, Inc.
Eicher, T. S., C. Christofides, & C. Papageorgiou (2012): “Did es-tablished early warning signals predict the 2008 crises?” Working Papers UWEC-2012-05, University of Washington, Department of Economics. Elekdag, S., R. Cardarelli, & S. Lall (2009): “Financial stress,
down-turns, and recoveries.” IMF Working Papers 09/100, International Monetary Fund.
Bibliography 38
Engel, C. & J. D. Hamilton (1990): “Long Swings in the Dollar: Are They in the Data and Do Markets Know It?” American Economic Review 80(4): pp. 689–713.
ESRB (2014): “Flagship report on macro-prudential policy in the banking sector.” Technical report, European Systemic Risk Board.
Frankel, J. & G. Saravelos (2012a): “Can leading indicators assess country vulnerability? Evidence from the 2008–09 global financial crisis.” Journal of International Economics 87(2): pp. 216–231.
Frankel, J. & G. Saravelos (2012b): “Can leading indicators assess country vulnerability? evidence from the 2008–09 global financial crisis.” Journal of International Economics 87(2): pp. 216–231.
Ghosh, S. R. & A. R. Ghosh (2003): “Structural Vulnerabilities and Currency Crises.” IMF Staff Papers 50(3): p. 7.
Gramlich, D., G. Miller, M. Oet, & S. Ong (2010): “Early warning sys-tems for systemic banking risk: Critical review and modeling implications.” Banks and Bank Systems 5(2): pp. 199–211.
Hakkio, C. S. & W. R. Keeton (2009): “Financial stress: What is it, how can it be measured, and why does it matter?” Economic Review (Q II): pp. 5–50.
Hamilton, J. D. (1988): “Rational-expectations econometric analysis of changes in regime : An investigation of the term structure of interest rates.” Journal of Economic Dynamics and Control 12(2-3): pp. 385–423.
Hamilton, J. D. (1989): “A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle.” Econometrica 57(2): pp. 357–84.
Illing, M. & Y. Liu (2006): “Measuring financial stress in a developed coun-try: An application to Canada.” Journal of Financial Stability 2(3): pp. 243–265.
Kaminsky, G. L. (1998): “Currency and banking crises: The early warnings of distress.” International Finance Discussion Papers 629, Board of Governors of the Federal Reserve System (U.S.).
Bibliography 39
Kliesen, K. L. & D. C. Smith (2010): “Measuring financial market stress.” Economic Synopses, Federal Reserve Bank of St. Louis (2).
Lowe, P. & C. Borio (2002): “Asset prices, financial and monetary stability: Exploring the nexus.” BIS Working Papers 114, Bank for International Settlements.
Mouratidis, K., D. Kenourgios, A. Samitas, & D. Vougas (2013): “Eval-uating Currency Crises: A Multivariate Markov Regime Switching Ap-proach.” Manchester School 81(1): pp. 33–57.
Sevim, C., A. Oztekin, O. Bali, S. Gumus, & E. Guresen (2014): “Devel-oping an early warning system to predict currency crises.” European Journal of Operational Research 237(3): pp. 1095–1104.
Wheelock, D. C. & M. E. Wohar (2009): “Can the term spread predict output growth and recessions? a survey of the literature.” Review (Sep): pp. 419–440.
Zigraiova, D. & P. Jakubik (2014): “Systemic Event Prediction by Early Warning System.” Working Papers IES 2014/01, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies.
Appendix A
Additional tables and figures
Table A.1: Data overview
Country Time range Missing values
Australia 1983q3-2010q4 X Austria 1996q1-2010q4 07q4-08q3 Belgium 1999q1-2010q4 07q3 - 08 q2 Canada 1981q1-2010q4 82q1-82q4 Denmark 1995q1-2010q4 01q1, 07q3-08q2 Finland 1995q4-2010q4 X France 1983q1-2010q4 93q1-93q4,07q1-07q4 Germany 1993q1-2010q4 07q1-07q4 Greece 2000q1-2010q4 07q1-07q4 Ireland 1999q1-2010q4 07q1-07q4, 09q3-10q2 Italy 1999q1-2010q4 X Japan 1981q1-2010q4 96q3-97q2, 99q4-00q3 Netherlands 1994q1-2010q4 07q1-07q4 Norway 1990q1-2010q4 X Portugal 1999q1-2010q4 X Spain 1999q1-2010q4 X Sweden 1996q1-2010q4 07q3 - 08 q2 Switzerland 1991q1-2010q4 06q3-07q2 United Kingdom 1986q4-2010q4 90q1-90q4, 06q1-06q4 United States 1982q1-2010q4 06q1-06q4
A. Additional tables and figures II
Table A.2: Signal scenarios of individual economic and financial indi-cators for in-sample data
Indicator Y=1 Y=0 Indicator Y=1 Y=0
baaspread S=1 72 138 curaccount S=1 50 100 S=0 189 809 S=0 211 847 indshare S=1 83 162 mmrate S=1 114 170 S=0 178 785 S=0 147 777 shareprice S=1 55 91 taxburden S=1 16 43 S=0 206 856 S=0 245 904 winf S=1 0 12 wrgdp S=1 106 191 S=0 261 935 S=0 155 756 wtrade S=1 44 92 yieldcurve S=1 116 169 S=0 217 855 S=0 145 778
Source: The author
Table A.3: Signal scenarios of individual economic and financial indi-cators for out-of-sample data
Indicator Y=1 Y=0 Indicator Y=1 Y=0
baaspread S=1 86 45 curaccount S=1 47 31 S=0 31 65 S=0 70 79 indshare S=1 4 1 mmrate S=1 27 0 S=0 113 109 S=0 90 110 shareprice S=1 63 13 taxburden S=1 13 8 S=0 54 97 S=0 104 102 winf S=1 12 0 wrgdp S=1 95 54 S=0 105 110 S=0 22 56 wtrade S=1 47 17 yieldcurve S=1 43 0 S=0 70 93 S=0 74 110
A. Additional tables and figures III
Table A.4: Signal scenarios for in-sample data
Indicator Y=1 Y=0 Indicator Y=1 Y=0
Comp. ind. S=1 123 166 Comp. ind. II S=1 123 162
S=0 138 781 S=0 138 785
Comp. ind. III S=1 123 166
S=0 138 781
RE logit S=1 125 165 FE logit S=1 139 164
S=0 136 782 S=0 122 783
Source: The author
Table A.5: Signal scenarios for out-of-sample data
Indicator Y=1 Y=0 Indicator Y=1 Y=0
Comp. ind. I S=1 78 17 Comp. ind. II S=1 78 17
S=0 39 93 S=0 39 93
Comp. ind. III S=1 78 17
S=0 39 93
RE logit S=1 51 26 FE logit S=1 57 23
S=0 66 84 S=0 60 87
