In the following section I subject my findings to three robustness checks. First, I alter the financial crises sample for the AUC method. Second, I expand on the underlying HP filter’s use for the BCG by altering the length of the data sample and the smoothing parameter. Third, I see whether the results for the main VEC specifications for the Netherlands also hold up for five other developed economies. While this is less of a robustness check, and more of an extension of the research, it still serves the purpose of internationally contextualising the results.
Altering the AUC Sample
I follow Lo Duca et al. (2017) in characterising three macroprudential relevant financial crises during the sample for the Netherlands. First is the crisis of the early 80s, when the effects of the crisis were exacerbated by an overheated housing market. Second is the crisis of the early 2000s which occurred due to a combination of a preceding period of excessive credit growth, the Dotcom crash, and the 2001 terrorist attacks. And third, the GFC and the SDC at the end of the 2000s and early 2010s. I prefer taking into account the crisis of the early 2000s for two reasons. Firstly, it followed a period of excessive credit growth and a significant inflation of asset prices, two components that the literature has indicated as being crucial in increasing financial systemic risk (e.g., Bordalo et al., 2020; Greenwood et al., 2020). Secondly, it allows for a larger number of observations for the AUC test, and as such a more statistically robust result for the AUC-values.
However, for the crisis of the early 2000s, it can be argued that due to it being instigated by an external shock (the terrorist attacks of 11-09-2011), and that the financial system was not in true distress, it holds less value for macroprudential policymakers.
Furthermore, due to this event being partially triggered by an equity prices bubble, the previous AUC results can be biased in favour of the composite asset price indicator. As such, I also run the AUC method where I take this second crisis out of the test sample. Due to the lack of observations, however, it is not possible to run the same 5 to 16 quarters ahead test that I ran before. I therefore run the AUC test using all available observations.
9 Appendix I reports the times the model specification converged.
Figure 11. AUC values for the sample excluding the crisis of the early 2000s. Source: author’s own calculations.
Figure 11 plots the results for the altered AUC test. Evidently, the AUC values have not changed by much. The AUC values suggest that for the main specification including the composite indicator for real asset prices and the real interest rate (model 1) continues to perform the best out of all other specifications and the BCG. So, throughout all the three different AUC tests performed, the main model specification has consistently had the highest AUC-values. This affirms the conclusion that not only house prices, but its combination with equity prices is important in explaining and measuring financial vulnerabilities.
Altering the Smoothing Parameter and Starting Points of the HP filter
As mentioned in the literature review, very few recent studies question the choice of the smoothing parameter in the BCG. The only two exceptions are Galan (2019) and Jokipii et al.
(2020). For the case of the Netherlands, questioning whether the smoothing parameter of 400,000 of the BCG is the best choice, is very relevant. The reason is the formula usually used for determining the smoothing parameter of a HP-filter. Most of the studies follow Ravn and Uhlig (2002) to set this parameter. Determining that for quarterly data on the business cycle, a smoothing parameter of 1,600 is optimal, they find that the choice of the smoothing parameter for the HP filter should be defined by the following formula:
1
0,8 0,82 0,84 0,86 0,88 0,9 0,92 0,94 0,96 0,98 1
180 185 190 195 200 205 210
AUC-value
Times the Model Converged
AUC-values (Excluding Crisis Early 2000s) and the Number of Times the Model Convergenced
Alpha stands for the adjustment factor relative to the initial smoothing parameter (which is 1600 for quarterly data on the business cycle, according to Ravn and Uhlig (2002)). Drehmann et al. (2012) assume a relative length of the credit cycle with respect to the business cycle to be four. As such, the calculated smoothing parameter would become 409,000, which is then set to 400,000. However, empirically, the assumption that the credit (or financial) cycle is four times as long as the business cycle for the Netherlands, does not hold up. For instance, Schüler et al. (2015) find that the financial cycle for the Netherlands is at most twice as long as the business cycle. This would then indicate that a significantly lower smoothing parameter of 25,000 is more apt.
Changing the smoothing parameter downwards will also alleviate the downwards bias present in the BCG. The reason is that it takes less time for past observations to be filtered out of the trend (Galan, 2019; Jokipii et al., 2020). Galan (2019) finds that for Spain it actually also improves on the predictive power of the BCG, and even performs as well as or better than several multivariate models. Considering this finding, and that the large majority of (recent) studies do not include it, it is relevant that it is done here.
I will also alter the sample size used for the HP-filter estimations. The sample size runs from 1970-Q1 to 2020-Q4, and to maintain proper comparability with the VECM specifications, which is also the sample size that is used for the HP-filter estimates. However, the BIS Stats Warehouse has credit-to-GDP data running as far back as 1961-Q1. Considering the start-point critiques highlighted in the literature review, this can have important implications for the credit gap estimates, especially at the beginning of the sample. As such, I will also run the HP-filters using this extended sample to see whether it improves the results of the BCG estimates, and whether it alters the conclusions made previously. Changing the sample starting point also allows for the testing and adding to the literature on the influence of the starting point in the HP filter estimates. Jokipii et al. (2020), for instance, recently found for Switzerland that it takes around 30 years for the series to converge again. Meanwhile, the BCBS recommends only a minimum period of 10 years before the use of the BCG as a macroprudential indicator.
Time Period Data 1961Q1 - 2020Q4 1970Q1 - 2020Q4 Smoothing
Parameter 400.000 125.000 25.000 800.000 400.000 125.000 25.000 VECM Asset Prices AUC 0,7936 0,7936 0,7558 0,7151 0,7209 0,7326 0,7151 0,9709**
AUC Contemporary 0,6526 0,6392 0,6083 0,5987 0,6027 0,6049 0,6033 0,9445***
AUC Contemporary excluding Dotcom Crash
0,7161 0,6956 0,6511 0,6388 0,6360 0,6449 0,6429 0,9505***
**: p<0,05; ***: p<0,01 of Wald Test where H0 is that the AUC-values of the model does not differ with respect to the HP filter credit gap estimates with the highest AUC (a smoothing parameter of 400.000 and a sample starting from 1961).
Table 7. AUC-values for different types of AUC tests and for HP filters with a different smoothing parameter and sample size. Model 1 of the VECM specifications is also listed.
Source: author’s own calculations.
Figure 12. HP filter estimates with a varying smoothing parameter. The starting point is 1961.
Source: Underlying data is from BIS (2021) and the estimates are the author’s own calculations.
Figure 13. HP filter estimates of the credit-to-GDP gap with a smoothing parameter of 400.000 and starting points from 1961 and 1970. The series converge after around 25 years. Source:
Underlying data is from BIS (2021) and the estimates are the author’s own calculations.
-40 -30 -20 -10 0 10 20 30
1970q 1 1972q
1 1974q
1 1976q
1 1978q
1 1980q
1 1982q
1 1984q
1 1986q
1 1988q
1 1990q
1 1992q
1 1994q
1 1996q
1 1998q
1 2000q
1 2002q
1 2004q
1 2006q
1 2008q
1 2010q
1 2012q
1 2014q
1 2016q
1 2018q
1 2020q
1
HP Filter Estimates from Sample Starting at 1961Q1
Financial Crises 5 to 16 quarters Ahead 400.000 25.000 125.000
-40 -30 -20 -10 0 10 20 30
1970q 1 1972q
1 1974q
1 1976q
1 1978q
1 1980q
1 1982q
1 1984q
1 1986q
1 1988q
1 1990q
1 1992q
1 1994q
1 1996q
1 1998q
1 2000q
1 2002q
1 2004q
1 2006q
1 2008q
1 2010q
1 2012q
1 2014q
1 2016q
1 2018q
1 2020q
1
HP Filter Estimates with a 400,000 Smoothing Parameter with Different Starting Points
Financial Crises 5 to 16 quarters Ahead 1961 1970
Table 7 reports the AUC-values of the different types of AUC tests performed, and for the combination of a different sample size and smoothing parameters. What is clear is that using the entirety of the data sample the BIS has available, improves the predictive properties of the HP-filter. The reason why is evident from figure 12, as the HP filtered credit gap with the larger data sample is higher before the crisis of the early 1980s, turns less negative as the crisis unfolds, and signals slightly more building vulnerabilities in the 1990s. This translates into a better fit for the AUC test. However, the improved test scores are still significantly less than the main VECM specification, thus further strengthening the findings that a VEC model with asset prices improves on the credit gap estimation compared to the BCG. What is more, altering the smoothing parameter downwards does not improve the BCG’s predictive properties. It actually seems to worsen it. This is the opposite of what Galan (2019) finds for Spain, but in line with what Drehmann et al. (2012) find internationally. However, what is clear from figure 13, is that the downwards bias of the BCG after the boom, caused by the excessive credit growth in the lead-up to the GFC and the low economic growth during the SDC, is less with a smoothing parameter of 25,000. Thus, there is a possible trade-off between diminishing the downwards bias of the BCG and decreasing its predictive properties.
VEC Model with Asset Prices for Multiple Countries
Figure 14. Credit gap estimates of the statistically best performing models and the BCG for the United States, the United Kingdom, Sweden, Germany, France, and the Netherlands.
Source: author’s own calculations.
In light of the results suggesting that for the Netherlands a VEC model with asset prices does indeed improve on the credit gap estimation compared to the BCG, it is interesting to see whether several model specifications would also be a good fit for other countries. I run the first twelve specifications of Appendix I, and the specifications with the different trend specifications for France, Germany, Sweden, the United States, and the United Kingdom.
Then, I run the AUC method using the full sample. I then select the statistically optimal combination of the number of times the models converged and their AUC values. The resulting model specifications are presented in figure 14 and graphed together with the BCG estimations. What is important to note here is that this analysis does not seek to be
comprehensive, nor to be conclusive about its findings. It mostly serves to put the earlier finding of the Netherlands in a more international context. As such, the analysis is less detailed and in depth than the analysis has been for the Netherlands. This is especially true for the qualitative analysis.
AUC Values Best Fitting
VECM Specification
BCG Best Fitting VECM
Specification Best HP Filter
France 0,6350 0,8258*** 0,6350 0,8258***
Germany 0,8838** 0,6178 0,8838 0,7064
Sweden 0,6175 0,5960 0,6175 0,5960
United Kingdom,
the 0,7642* 0,6884 0,7642 0,7422
United States,
the 0,8657 0,8280 0,8657 0,8280
Netherlands, the 0,9445*** 0,6027 0,9445*** 0,6027
*: p < 0,1; **: p < 0,05; ***: p < 0,01 of the indicator’s AUC-values being significantly different from the other indicator. The indicators in the first two columns on the left are compared to one another, and the indicators in the two columns on the right are
compared to one another.
Table 8. Reports the AUC values for the statistically best performing models for the listed countries, and whether they are statistically significant from one another.
The model with the asset price composite indicator, while performing very strongly in the Netherlands, performs poorly in most of the other countries. Only for Sweden and France it is the best performing model. In the other countries, the specification does not converge.
Rather, for France, the United States, and the United Kingdom, models with house prices perform the best. For Germany, the model with equity prices fares the best out of all specifications.
While the main specification with the composite asset price indicator for the Netherlands seems to fit periods of excessive credit growth better than the BCG, both statistically and intuitively, this is not the case for the majority of the other countries. I perform the same statistical comparison using the AUC method, and I also run the tests for other smoothing parameters for the HP-filter. For the majority of these countries, there are not enough observations to be able to generate AUC-values for 5 to 16 quarters before relevant macroprudential events. As such, the AUC for the full sample and the times the model converged are used as an indication whether the models are a good fit for the data.
Table 8 plots the AUC-values for the best fitting VEC model specification, the BCG (with a smoothing parameter of 400.000), and the best performing HP filter. What this shows is that all models have a much lower AUC-value than the main specification for the Netherlands.
Only Germany and the UK have a statistically significant difference in their AUC-value
compared to the Basel Credit Gap. However, when taking the best performing HP-filter specification’s AUC value, these differences are no longer statistically significant.
When looking at the graph of the credit gap estimates of the model for the United Kingdom, the VEC specification seems to be a reasonable fit for the domestic credit cycle.
Importantly, it signals substantial positive imbalances in the lead-up to the GFC, more so than the BCG. This is also true for France, where the model with house prices indicates a positive gap for most of the late 1990s and the period before the GFC. Sweden’s house price VEC specification is more volatile than the Netherlands’ specification until the early 90s. After that, it signals more strongly building financial imbalances in the lead-up to the GFC than the BCG.
For Germany, however, while the AUC-values would indicate otherwise, the VEC model with equity prices seems a poor fit. It does signal building imbalances correctly for the 90s, but indicates positive imbalances during the SDC, which seems highly unlikely. The HP-filter does the opposite. For the United States, the model with house prices seems to move quite puzzlingly. Importantly, it fails to signal building imbalances for the financial crisis in the 80s and the GFC. While it did drop as a result of the deflation of the house price bubble in 2008, it does not signal that positive financial vulnerabilities were prevalent before.
Thus, while the structural VEC model with asset prices performs strongly for the Netherlands, these results do not hold up as strongly in an international context. This has three implications. First, it underlines the importance of a detailed analysis focused on one nation at a time, as has been done in this study. As mentioned, I used the specifications tailored for the Netherlands, and ran them for these countries. Perhaps with more fine-tuning of the number of lags, cointegration relations, and specific variables, the results for a structural VEC model would improve for these other countries. Second, these results highlight the heterogeneity of domestic financial cycles (Aldasoro et al., 2020). If the domestic financial cycles and the drivers behind them would be similar to each other, then the results for the Netherland would also be reflected in the results for the other countries. Third, and interrelated to the second point, the results underline the importance of national discretion in macroprudential policy-making. Considering the substantial differences in both the optimal VEC specification (some with the composite indicator, others with house prices) and the time periods that the credit gaps turn positive, macroprudential policy should be decided by national policymakers who have the firmest grasp on what the situation regarding financial stability is in their country.