MIF Master Thesis:
"Probability of default for small and medium sized Dutch firms:
The role of behavioural characteristics"
Submitted by: Hasan Isik – 11081848
Supervisor: Razvan Vlahu
Amsterdam Business School | University of Amsterdam
Abstract
This paper analyses the importance of behavioural risk drivers on the probability of default (PD) for small and medium enterprises (SMEs) in The Netherlands. PD models determine the likelihood that a healthy SME customer defaults within 12 months and are based on such explanatory variables (risk drivers) that can be categorized either as `financial`, `categorical` or `behavioural` depending on which aspect of the client they explain.
A set of statistical models have been developed for a large Dutch SME portfolio of a global bank covering a period from January 2007 until June 2015. These models contain all 3types of risk drivers and we can deduce which of the risk drivers have the strongest influence on the model predicted PD by comparing their statistics (i.e. Wald Chi-square values). The hypothesis for the Dutch SME portfolios was that behavioural risk drivers are the most influential in determining final PDs.
20 different models (that are acceptable) were generated. It was seen that in 16 of the models (80%), the risk driver that had the highest Wald Chi-square value was a behavioural one. The risk driver ‘number of arrears past 12 months’ came out 13 times (60%) as the leading explanatory variable. Hence, to avoid bias on possibly strong variables, 3 of the models were tested excluding this variable while 2 models were constructed entirely on non-arrear clients. Still, the leading risk driver was largely a behavioural one and therefore the outcome suggests that indeed behavioural risk drivers are the most influential in determining probability of default for Dutch SMEs.
Contents
1 Introduction and rationale ... 4
2 Probability of default (PD) ... 6
3 Regulatory framework ... 7
4 Literature review ... 8
5 Hypothesis and methodology ... 11
Data ... 14
Sampling ... 18
I. Quarterly sampling... 19
II. Development sample and validation sample ... 21
Univariate analysis ... 22
I. Transformations and potential risk drivers ... 24
Multivariate analysis ... 26
6 Model comparison & conclusion ... 28
7 Bibliography ... 30
8 Appendix ... 33
I. Long list of risk drivers ... 33
II. Restructuring and recovery... 34
I. Risk driver transformations ... 35
II. Model testing example ... 37
1 Introduction and rationale
Hull (2015) explains credit risk simply as the “possibility that borrowers, bond issuers and counterparties may default”. One can interpret this as the risk that a borrower does not make a contractual payment on a timely basis or not at all. A good management of credit risk is crucial for the performance of financial institutions individually and for the functioning of the financial market as a whole. 3 main drivers of credit risk are probability of default (PD), exposure at default (EAD) and loss given default (LGD). Financial institutions assess these drivers via models constructed internally or externally. Statistical and financial econometric models of risk management have become very popular in the empirical literature, as stressed by Crook and Bellotti (2010). Credit risk models are part of a technical framework which provides information for risk managers and decision makers regarding to the implementation of credit policies defined by financial institutions. These models seek to evaluate the probability that some financial operation will not yield the expected return. PD models predict the chance a default occurs within the next 12 months. The EAD model predicts the level of the exposure at the moment of default related to the current exposure. The last component, LGD, expresses the percentage of loss to be expected from that EAD in case of a default. The relationship and timing of these 3 events is depicted in Figure 1.
Figure 1: Basic model types for credit risk
The combination of the three models gives the Expected Loss (EL) where EL = PD * EAD * LGD Banks routinely use these models for a broad range of activities, including underwriting credits; valuing exposures, instruments, and positions; measuring risk; managing and safeguarding client assets; determining capital and reserve adequacy; and many other activities. In their ‘Supervisory Guidance on Model Risk Management’ (2011) Board of Governors of the Federal Reserve System points out that banks in recent years have applied models to more complex products and with more ambitious scope, such as enterprise-wide risk measurement, while the markets in which they are used have also broadened and changed.
For financial institutions, statistical Probability of Default (PD) models are primarily constructed to be used as inputs for the credit risk part of the firm wide capital requirements, using the Advanced Internal Risk Based (AIRB) approach to calculate their capital requirements. These models – linear regression functions- make use of several variables (risk drivers) that can be categorized as either `financial`, `categorical` or `behavioural` depending on which aspect of the client they explain to the bank.
Financial risk drivers are mostly retrieved from annual financial statements of the client firm
and explain Capital, Growth, Liquidity, Activity, Profitability and Debt ratios to the bank.
Categorical risk drivers are usually bank internal classifications such as groupings per Basel
exposure class, geographical location, age group etc.
Behavioural risk drivers are constructed to explain the ongoing relationship with the client firm
and are usually translated into examples such as `the number of arrears in the last 6 months` or `overdraft utilisation in last 12 months`.
Financial, categorical and behavioural risk drivers are all client firm specific variables. In this data study no macro-economic external variable has been used whereas the research around them has been mentioned in the literature review section.
This paper focuses on the PD modelling aspects specifically for SMEs residing in The Netherlands. Small and medium-sized enterprises (SMEs) are a focal point in shaping enterprise policy in the European Union (EU). The European Commission considers SMEs and entrepreneurship as key to ensuring economic growth, innovation, job creation, and social integration in the EU.
In The Netherlands SMEs determine for a large part the economic landscape. According to the European Commission`s fact sheets, in The Netherlands in 2012 SME created 66.7% of all jobs, while 99.8% of all businesses are SME. Furthermore, the Dutch SME employees generate 62.9% of the value added by the private sector, which is relatively high compared to the EU zone average. Retrieved from http://ec.europa.eu/
Being aware of the importance of SME sector in The Netherlands, it is important for banks to know which attributes of an SME client are crucial to signal for a near future default event. If behavioural risk drivers are indeed found to be most influential in PD modelling, then this knowledge can be used in product pricing, client acceptance and data priority decisions geared towards Dutch SME portfolios of the banks.
This paper describes the key influence of behavioural risk drivers on PD modelling and consists of the following sections: Section II gives an overview of Probability of Default (PD) definition and modelling. Section III sets the regulatory framework. Section IV describes the literature review. Section V explains the hypothesis and methodology followed. Section VI concludes.
2 Probability of default (PD)
Probability of default is a quantitative measure which gives the likelihood that a client or borrower defaults within a given time period. As mentioned, the forecasting horizon of the probability of default estimates is one year as required by Basel II regulations and should be a long-run average of one-year estimates. Probability of default (PD) estimates are client based – or as stated in Basel II “oriented to the risk of borrower default”, and are the direct input for a number of applications such as pricing and acceptance decisions. Thus, PD is mostly measured at the customer level not at the facility level. For modelling purposes, an ever bad definition to default is assumed, i.e. a client that defaults at least once within the 12 month horizon is considered as a default.
The outcome of a PD model is a PD-rating. There are two types of PD rating models: models that predict a rating and use the rating to derive the PD and models that directly predict the PD which is then mapped to a rating. In general, institutions may use different estimation methods (and different data sources) to estimate PDs for borrower rating grades (or pools), including mapping internal rating grades to the scale used by an External Credit Assessment Institution (ECAI), in order to attribute the default rates observed by this ECAI to internal rating grades, statistical default prediction models or other estimation methods or combinations of methods. As a result the model not only gives an overall indication of the probability (a part of) a portfolio will default but also gives a granular distinction in the portfolio between the different customers.
A PD model outcome is expected to be accurate and granular enough. Accuracy can be described as the predictive power of the model, its ability to easily recognise bad customers. This is measured by c-statistic of the model and an acceptable statistical model is expected to have a c-statistic above 70%. Granularity on the other hand can be described as the ability of the model being able to sufficiently distinguish customers: no undue concentration is present amongst rating classes (<30%).
There are 2 flavours of PD depending on the period of time taken into account during modelling. Point in time (PIT) PD is defined as the unconditional probability of default over the 12 months performance period and is important when assessing collective provisions. A Through-the Cycle (TTC) PD on the other hand is representative for a longer period of time and should theoretically capture an entire macro-economic cycle. TTC PD is the one used for assessing the capital requirements. The fundamental PD modelling assumption is that the inclusion of a minimum of five years history ensures that the model will be calibrated to a long-term level and that it will sufficiently reflect the long-term effects of the economic/credit cycle. Therefore, it is regarded as an adequate long-term calibration level as a direct indication that the PD models’ estimates sufficiently reflect the effects of the TTC approach.
As this section relates to the Probability of Default (PD) models, some of the underlying PD methodology needs to be briefly explained to set the context. We choose a single point in time as the observation moment and select the non-defaulted clients in that snapshot. We then set the 12 months performance period for a Basel PD model after each observation moment. This is the period in which we identify whether or not a default has taken place. To determine the one-year default rate based on these definitions, it is needed to aggregate the results. The standard
method for PD is number-weighting which is preferred by the regulators. As such, the number of observed into default cases divided by the number of non-defaulted clients at the observation moment results in a one-year default rate for a single observation moment. Apart from setting the performance period and necessary aggregations it is also crucial to determine the correct averaging approach. The standard method for averaging PD through time is the time-weighted approach, in which equal importance is given to each observation moment by taking the simple average over all observed default rates. Time-weighted average of number-weighted default rates is the standard approach for PD modelling.
Having set the context for PD here, the detailed steps taken to build the model will be made clear in the methodology section which is first preceded by the regulatory framework and the existing literature around the subject.
3 Regulatory framework
Following the Basel Committee on Banking Supervision (BASELIII), Bank for International Settlements (BIS) published its Minimum Capital Requirements standards in 2016 and explained that “Banks that have received supervisory approval to use the IRB approach may rely on their own internal estimates of risk components in determining the capital requirement for a given exposure.” (BIS, 2016, p. 41). These estimates are subject to certain minimum conditions and disclosure requirements and in the end it will be up to the banks to characterize the riskiness of their counterparts and loans in their portfolios by means of risk categories or rating classes. A special feature of the regulation is that loans to SMEs will receive a different treatment than corporate loans. The main reason for this differential treatment is that small business loans and retail credit are generally found to be less sensitive to systematic risk. Their risk of default is thought to be largely of an idiosyncratic nature and, as a result, default probabilities are assumed to be more weakly correlated when compared with corporate loans.
PD modelling is a regulatory requirement for financial institutions and has inherent regulatory constraints that needs to be paid attention to. I will mention below 4 main considerations one needs to consider as part of the actual modelling exercise: PD floor of 3 basis points, country cap, minimum number and concentration of final rating grades and legal constraints aimed to prevent redlining.
From a regulatory point of view, PD has a floor of 0.03%; no lower PD is allowed to be used. The
0.03% PD floor does not apply to all exposures (e.g. sovereigns, central governments and central
banks). Several economic studies that reviewed the adoption of an International Ratings Based (IRB) approach to estimating risk weights by banks find that the lower reported risk weights using the IRB methodology to some extent reflect downward risk manipulation. Huizinga (2016) claims that input floors such as this one can play a useful role alongside an aggregate output floor, if they are targeted to address the problem of potential mis-measurement of risk. These floors are set by the regulator to prevent wholesale bank-level downward risk weight manipulation which gives rise to effective bank undercapitalization and a heightened probability of bank failure. Models have at the end of the rating process the so-called GC-rating cap. This cap prevents a rating for a company in a certain country being better than the rating (GC-rating) of the country itself. This means if the model rating (e.g. standalone, influenced, rating after adjustment) is
worse than the GC rating of the country of residence then the final rating is the model rating. On the other hand if the model rating (e.g. standalone, influenced, rating after adjustment) is better than GC rating of the country of residence then the final rating is the GC rating. Since this exercise focuses on the SME population in The Netherlands which is a fairly default risk free country, this cap will not be effective. The best internal rating class for the SME portfolio at hand is currently 10 (a rating class higher than 19 is considered default) while the probability of sovereign default for the country of The Netherlands is 1 which is also confirmed by Moody`s Aaa external rating.
The Basel Committee on Banking Supervision believes that banks adopting the IRB approach should have risk rating systems that effectively distinguish the level of credit risk across the entire spectrum; from borrowers that are virtually risk-free to those in default. Basel II minimum required number of rating classes is 6 for performing borrower grades and 2 for non-performing borrower grades. This floor on the number of model rating grades cannot be modified. The range for the minimum number of grades is due to the variation of banks’ lending activities and the use of specialised rating schemes for different types of borrowers, products or market segments. “Risk rating systems that have overly broad grade definitions, which result in borrowers of significantly different risk characteristics being assigned the same grade, are not acceptable.” (BIS, 2001, p. 43). Likewise, risk rating systems that materially assign borrowers of comparable risk to different grades are also not acceptable by the regulator.
The Basel committee also proposes the 30% threshold on gross exposures to avoid any possible excessive concentration in any particular rating grade. As an additional measure for undue
concentration among all rating grades banks largely use the Herfindahl index approach. Model
output with a Herfindahl index < 0.2 is generally acceptable.
Lastly, it is logical to expect legal constraints on the use of risk drivers to apply. For example the use of zip code as risk driver is not allowed in all jurisdictions as it is associated with the practice of red-lining. The use of zip code should carry positive advice from the legal department. All risk drivers must be in line with the bank`s internal legal policy. For example risk drivers referring to race or gender are generally not permitted.
4 Literature review
The advent of the recent financial international crises, such as the U.S. subprime crisis and the Euro Zone debt crisis, have all contributed to the empirical studies on the subject of credit risk models. In this scenario, econometric models have emerged to deal with the probability of default by both individuals and firms. In this literature, one can mention works by Lane, Looney, and Wansley (1986), Banasik, Crook, and Thomas (1999), Stepanova and Thomas (2001, 2002), Balzarotti, Falkenheim, and Powell (2002), Andreeva (2006), Bellotti and Crook (2009), Jimenez, Ongena, Peydro, and Saurina (2008), and Ioannidou, Ongena, and Peydro (2008). Since the focal point of this study is the behavioural aspect of risk drivers, following literature reviews have been conducted with the financial-categorical-behavioural distinction in mind.
Serov (2011) discusses the usage of financial risk drivers in SME PD modelling. One has to be cautious of the fact that same ratios can be of different importance and predictive power when
considered for SMEs as compared to large corporate. Good examples are leverage and liquidity ratios. While being one of the most important predictors for large corporates, leverage ratio ranks among one of the least important for smaller firms. Liquidity is an important factor in many credit decisions since it is a contemporaneous measure of default: if a firm is in default, its current ratio must be very low. While short-term to long-term debt appears of little use in forecasting PD for SMEs, cash to assets proves to be the most important single variable for small firms as compared to larger firms (Falkenstein, Boral, & Carty, 2000). Financial information are widely used in PD modelling and are the first type of explanatory variables that comes to mind in PD modelling. A vast number of studies were performed focusing only on firm balance sheets and external bureau financial inputs. Existing methodologies differ on the available information and data used for assessing the PD. Valle et al (2016) classifies them broadly in models based on market data (see e.g. Ji, 2010, Laajimi, 2012 or Sundaresan, 2013) and on accounting data (see e.g. Beaver, 1968 or Altman, 1968).
On the other hand, recent literature (e.g. Lehmann (2003) and Grunet et al. (2004)) concludes that quantitative variables are not sufficient to predict SME default. Altman and Sabato (2007) argue that including qualitative variables (such as the number of employees, the legal form of the business, the region where the main business is carried out, the industry type, etc.) improves the models’ prediction power. These variables mentioned here can best be labelled as
categorical. Another investigation focusing on categorical aspects has been Chen and Wu`s
(2014) paper published in the Journal of Banking & Finance where their findings suggest that there are important channels of default correlations at the sectorial level and ignoring them will lead to biased estimation of default correlation and tail losses. Firms in different businesses typically face different levels of competition and product cycles. As such, the likelihood of default can differ significantly for firms in different sectors even though they have similar balance sheets. “Historically, default correlation often concentrates on firms in a similar business; for example, in 2002 the telecom sector accounted for 56 percent of all corporate bankruptcies” (Chen & Wu, 2014). Jorion and Zhang (2007) and Hertzel and Officer (2012) document convincing evidence of an intra-industry effect in default clustering as well.
In addition Fantazzini et al (2009) highlight the presence of qualitative idiosyncrasies, like quality of management and business characteristics as important factors in explaining why one company defaults and another continues to serve its debt, while exhibiting similar financial fundamentals and debt structures. Once again, business characteristics mentioned here are on the line of the categorical variables.
Divino and Rocha (2013) have shown that inclusion of time-dependent macroeconomic
variables improved the overall discriminatory power of the estimated models. But they have
done this on a mortgage portfolio which essentially has different characteristics than SME portfolios. Qi, Zhang and Zhao (2014) have also conducted their analysis on the role played by the so called unobserved systemic risk factor in default predictions. They observed that this latent factor outweighs the observed systemic risk factors and can substantially improve the in-sample predictive accuracy at the firm. Leow and Crook (2016) claim it is the effects of changes in macroeconomic conditions that are most difficult to unravel and suggest that different macroeconomic conditions affect different people at different times differently. Yet, when they compare models developed on downturn data and a normal period data, they found a large decrease in estimated hazards. All these imply that even though they have taken into account
macroeconomic conditions and possible interactions between macroeconomic and application variables, the models are still unable to adequately model the various effects coming from the type of borrower, the time during which borrowing takes place, and how macroeconomic conditions would affect different individuals differently which are all categorical considerations. Bonfin (2007) on the other hand argues that firms are affected minimal in PD systemic, macroeconomic factors. Moreover Veurink (2014) shows that Dutch SMEs are less dependent on the macroeconomic environment than corporates. This suggests that client attributes play larger role in determining PDs in The Netherlands. According to Bonfin (2007), “Even though macroeconomic and financial conditions may offer a valuable contribution to explain credit risk at an aggregate level, it is the firm’s specific financial situation that will ultimately determine whether it will default on its liabilities”. It is worth to mention that as part of this study macroeconomic factors have not been present in the dataset.
Despite the extensive progress in default risk modelling, conventional models are unable to fully explain the observed clustering of default and consistently underestimate the probability of extreme default losses (see Das et al., 2007, Koopman et al., 2012). In particular, conventional models fail to generate sufficient dependencies across obligors to capture the observed default cluster and tail loss. A potential problem with these models is their exclusive reliance on observed firm-specific or aggregate factors.
Differences between corporate loans and both SME and retail credit have been the subject of a range of studies. A large part of the literature has focused on the special character of small business lending and the importance of relationship banking for solving information asymmetries. Cole (1998), for example, finds empirical support for the theory that banking relationships generate valuable private information about borrower quality. This indeed backs the purpose of this study up as well since the term banking relationships can be thought as a behavioural aspect.
One interesting recent paper by Perko (2017) focuses on the effect of behavioural analytics on short-term default predictions. Even though his investigation was invoice level rather than obligor level, as evidenced in the research results, the evaluation of debtor alignment with certain behavioural patterns, or even payment related strategies, has a positive effect on invoice PD accuracy. “The probability of default depends not only on payment capability but also on payment preparedness” (Perko, 2017). One can deduce that payment preparedness hints at a
behavioural and/or categorical aspect rather than a financial one.
It is worthwhile to note that available literature on the matter so far has been mainly focusing on 1) the usage of specific financial risk drivers in SME modelling 2) the importance of explanatory variables that are categorical 3) the fact that firm specific information is more important than systemic factors in PD modelling for SMEs 4) the need for quantitative variables being complemented by qualitative information for a stronger model performance and 5) the limited recent research on the effect of behavioural information.
This paper will aim to extend the points mentioned above and will focus on the remaining explanatory variables that are behavioural. In our dataset, the definition usually translates into client specific behaviours quantified such as `the number of arrears in the last 6 months` or `overdraft utilisation in last 12 months`.
5 Hypothesis and methodology
Logistic regression is the preferred modelling technique for statistical PD models. It is the industry standard and is most commonly used. Alternative techniques applied in the past have led to instable and counter intuitive models. As a result they are no longer preferred. This is especially true for decision tree models.
The hypothesis is that behavioural risk drivers are the foremost influential types of variables when assessing probability of default for Dutch SMEs. In the table below, an example is given of a statistically strong (c-statistic > 70%) PD model for a Dutch SME portfolio. Eight variables made it to the final model after the application of univariate and multivariate analysis steps explained below. They are being used as the input to a stepwise logistic regression model. All of them have significant contribution to the model. The ‘Wald Chi-Square’ value indicates the relative contribution of individual risk drivers to the model. The larger the value is, the earlier it enters the model during regression. For example, the variable ‘number of arrears past 12 months’ has the largest value (685.60) and is a behavioural risk driver.
RISK DRIVERS RISK WEIGHT STANDARD ERROR WALD CHI-SQUARE PR>CHI SQ
Intercept -5.60 0.10 3,364.30 <.0001
Current liabilities to total assets (logarithmic
transformation) 7.21 2.00 13.00 0.0003
EBITDA to payable expenses (logarithmic
transformation) 5.82 1.76 10.97 0.0009
Liquidity ratio (logarithmic transformation) 6.98 2.25 9.64 0.0019 Net profit margin (logarithmic
transformation) 14.78 1.60 84.92 <.0001
Net worth to current liabilities (logarithmic
transformation) 6.41 2.21 8.41 0.0037
Current arrear 1.02 0.06 294.99 <.0001
Overdraft utilization past 12 months (cubic
transformation) 12.79 0.90 201.21 <.0001
Number of arrears past 12 months 0.26 0.01 685.60 <.0001 Table 1: Example of a statistically strong Dutch SME PD model
The score is derived from the linear sum of the parameter times risk driver value (transformed) plus the intercept. In the example above;
Score = Intercept
+ 7.21 * Log of (current liabilities / assets)
+ 5.82 * Log of (EBITDA / (current interest bearing borrowings + interest expense)) + 6.98 * Log of (cash and cash equivalents / current liabilities)
+ 14.78 * Log of (100 * net profit / revenues)
+ 6.41 * Log of (equity and reserves / current liabilities) + 1.02 * Current arrear (Yes = 1, No = 0)
+ 12.79 * Cubic transformation of (overdraft outstanding average for the past 12 months / overdraft limit average for the past 12 months)
+ 0.26 * Number of current arrear statuses during the past 12 months (from 0 to 12)
Then PD is calculated as follows: PD = 1 / (1 + exp (-Score))
Special attention is given to the plausibility and business intuition at this stage. As mentioned in the literature review section, some financial ratios are meaningful for SME PD modelling such as liquidity or cash to assets while some are empirically not so meaningful for SMEs such as the leverage ratio. In the example above, 5 financial ratios made it to the final model addressing firm specific financials ranging from debt ratios (current liabilities to total assets, EBITDA to payable expenses, net worth to current liabilities) & liquidity (liquidity ratio) to profitability ratios (net profit margin). 3 behavioural risk drivers made it to the final model. ‘Overdraft utilisation past 12 months’ gives the annual overdraft product utilisation of the client firm. The degree of how much overdraft has been utilized in average by the client firm has been advocated by the business experts as being a good indicator of default probability for SMEs. This is also evidenced by the large positive risk weight of this driver in the given example above. 1% increase in the average annual overdraft utilisation would therefore increase the score by 12.8% everything else kept the same. Remaining 2 behavioural risk drivers (‘current arrear’& ‘number of arrears past 12 months’) are both constructed to explain delinquency related information. Behavioural relationship with a client firm is best and most commonly explained by arrear information. Another point of interest would be the sign of the risk drivers. All 8 risk drivers in this example have positive risk weights indicating positive relationship with the score (and therefore with the PD). This is at first glance an unintuitive result for such variables like liquidity ratio, net profit margin, and net worth to current liabilities and EBITDA to payable expenses. One would expect an increase in such ratios to decrease the final probability of default. Yet in reality the log transformation, as part of the univariate analysis, has already shaped these ratios so that an increase in the underlying value corresponds also to an increase in the final PD. The log transformation for the net profit margin is given below as an example:
Transformations for 100*NetProfit/revenues
C stat: Lin=60.4% Qua=60.3% Log=60.4% Exp=60.4% Qub=60.4% Figure 2: NetProfitMarg Transformation
The aim is to apply a long list of risk drivers to run numbers of multivariate analysis on a Dutch SME dataset of a Global Bank that spans from January 2007 to June 2015 effectively including downturn years as well, capturing systemic factors. This long list of risk drivers will include all three types of explanatory variables1. As mentioned earlier, regulatory standards require at least
5 years of data for statistical internal risk based approaches for PD estimations. Thus we have sufficient data history.
Comparing Wald-Chi Square values of each end model will then be a test for whether or not behavioural risk drivers are the most influential of the risk driver types as it is the case in the example above where top 3 Wald-Chi Square values are assigned to them (excluding the Intercept).
The main components for a PD model are the risk drivers. The whole model development process is one of subsequent selections to get from a gross list of all possible risk drivers into around 10 to 15 risk drivers for the final model. Describing these selections at a high level gives a first introduction in the process of a PD model development.
Figure 3: The risk driver funnel
After the initial data collection, a vast number of attributes are available. At the request of business experts, these already available attributes can even be enriched with additional data from other sources during data collection. This list of initial risk drivers is then expanded further by the model builders by the creation of derived variables like historic averages and ratios. This subsequent gross list is the input to the univariate analysis. All risk drivers which satisfy both statistical and business criteria enter the next phase of modelling as a potential risk driver. From this group one or more models are created and the final model is selected.
In the subsequent sections, these steps will be further explained in the following order: Data will start with portfolio description and follow up on collection and analysis of available data and analysis on availability, quantity, quality and representativeness. Sampling will explain the development vs. validation sample distinctions. Univariate analysis will analyse individual risk drivers and their relationship to the default event while multivariate analysis combines different predictive risk drivers into a combined model. This will be the stage where series of statistically different models will be populated. Model testing in the end will touch upon such model tests like performance, stability and intuitiveness. PD modelling for a bank consists of one more final step where rating buckets are created and mapped to a Masterscale (internal or external).2
Data
Each modelling exercise starts with the data. Considerable amount of time and effort is spent to get a representative and rich dataset that can accommodate for statistical analysis. The starting point is monthly snapshots of the portfolio from January 2007 until June 2015 which results in a dataset of 8.5 years. There are 15,021 unique Dutch SME customers with 499,762 records
2 Masterscale mapping is essentially a standardization of final PDs across several PD models. This step will not be
part of this research since the focus is on the risk driver composition of final models rather than the eventual PDs and their ranking.
Raw dataset
Gross list risk drivers including ratios, historic averages
Potential risk drivers after univariate analysis
Data preparation
Univariate analysis
Multivariate analysis
across this period. The Dutch SME PD model covers Enterprises residing in the Netherlands. They are identified as:
- Annual sales above EUR 50 million but below EUR 100 million or,
- Annual sales below EUR 50 million but Legal One Obligor (LOO) limit above EUR 1 million. The total dataset that is constructed during data collection needs to be a good representation of the actual portfolio. That is why before any modelling itself is started it is essential that the dataset is accepted by business experts. For this purpose key statistics of the collected data (number of customers, total limit and outstanding through time, segmentation of product types, number of defaults over time) are presented to the business. The acceptance of the dataset as “suitable for model development” is a formal milestone of the modelling project. Below figures aim to explain the current trends and segmentation in the portfolio:
It is clear that the portfolio size and associated exposure is decreasing over the past years. As of June 2015, there are 10,358 clients in the portfolio with a total of EUR 12.6 billion outstanding (and EUR 10.4 billion in RWA). Albeit it is decreasing, by its sheer size this portfolio is considered a substantial one and therefore models attached to it are frequently questioned by the regulator.
Figure 1: Portfolio trend
0 2000 4000 6000 8000 10000 12000 14000 16000 18000 0 5000 10000 15000 20000 25000 2007-01-01 2007-06-01 2007-11-01 2008-04-01 2008-09-01 2009 -02 -01 2009-07-01 2009-12-01 2010-05-01 2010-10-01 2011-03-01 2011-08-01 2012-01-01 2012-06-01 2012-11-01 2013-04-01 2013-09-01 2014 -02 -01 2014-07-01 2014-12-01 2015-05-01 TOTA L O /S M IL LIO N S N UM BE R O F C US TOM ER S
Portfolio Overview
os nrCurrent portfolio segmented on the customer type information yields the expected distribution of clients where majority of the customers are regular and holding enterprises. Owing to the identification of SMEs based on annual sales, it is also visible to see small companies who made it to the model scope. Their LOO limits are higher than EUR 1 million.
Table 2: Portfolio overview - Customer types
The industry distribution shows diversity across the industries.
The leading ones are Food, Beverage & Personal Care, Services, Real Estate, and General Industries. Industry type is usually a likely candidate for a meaningful categorical risk driver since its distribution is not concentrated in a few classes for SMEs.
Table 3: Portfolio overview – Industry type
The rating distribution shows that the majority of the portfolio is rated in the rating grades ranging from 10 to 13. This constitutes 69% of the organizations based on December end 2016. 11% of the organizations are in default by the end of December 2016. Rating grades 20, 21 and 22 are reserved for default ratings. 20% of the organizations are within grades 18-22 based on December 2016 end. The customers within grades 18-19 are at restructuring/recovery phase and under close monitoring.3
Table 4: Portfolio overview - rating distribution
3 Appendix II: Restructuring and recovery process
Cat Code Category Description Nr. of Ult. Par. 31 Dec 2016 Nr. of Org. 31 Dec 2016 Max Limit 31 Dec 2016 Outstanding 31 Dec 2016 Probability of Default 31 Dec 2016 CORP Corporate(s) 43 48 112,133,499 93,294,179 2.312726% KENT Enterprises 4,036 4,452 8,931,348,895 7,406,568,858 15.433185%
FFFM Fund and Fund Manager(s) 1 1 - -
-INDX Individual(s) - - - -
-FNBF Non-Bank Financial Institution(s) 10 10 18,971,338 16,815,082 4.488338%
REAL Real Estate 20 20 40,223,396 36,075,875 15.866436%
SCOM Small Companies 167 175 113,481,759 91,993,170 10.360509%
Overall Total 4,260 4,705 9,216,158,888 7,644,747,164 15.230162%
Cat Code Category Description Nr. of Ult. Par. 31 Dec 2016 Nr. of Org. 31 Dec 2016 Max Limit 31 Dec 2016 Outstanding 31 Dec 2016 Probability of Default 31 Dec 2016 1 Automotive 170 182 291,139,944 258,679,052 23.0%
21 Builders & Contractors 454 512 834,310,803 612,861,525 17.0%
3 Chemicals, Health & Pharmaceuticals 148 157 382,388,890 307,767,209 13.3%
4 Civic, Religious & Social Organizations 20 20 38,624,938 32,749,706 0.7%
7 Food, Beverages & Personal Care 782 834 1,802,739,107 1,545,655,727 17.4%
2 General Industries 482 518 968,037,589 708,992,731 7.6%
24 Lower Public Administration 2 2 2,209,187 1,688,954 0.3%
10 Media 107 112 209,336,264 168,167,222 17.3%
11 Natural Resources 91 100 220,145,890 170,687,916 7.1%
26 Non-Bank Financial Institutions 187 189 271,663,152 234,139,397 4.3%
12 Private Individuals - - - - -22 Real Estate 495 512 817,729,856 787,990,965 16.1% 14 Retail 299 323 538,366,057 410,842,795 16.4% 15 Services 587 618 1,423,887,237 1,168,260,341 10.2% 16 Technology 80 84 129,715,685 87,680,822 5.5% 17 Telecom 15 15 55,799,208 45,843,208 3.8%
18 Transportation & Logistics 476 523 1,193,441,627 1,071,896,729 24.7%
19 Unknown 1 1 5,010,000 134 0.4% 20 Utilities 5 5 31,613,454 30,842,731 4.2% Overall Total 4,260 4,705 9,216,158,888 7,644,747,164 15.2% Ca te gory De scription Nr. of Ult. Pa r. 31 De c 2016 Nr. of Org. 31 De c 2016 Ma x Limit 31 De c 2016 Outsta nding 31 De c 2016 10 723 736 1,361,773,695 1,041,847,419 11 827 846 1,881,402,773 1,436,226,260 12 852 867 2,002,584,541 1,541,011,268 13 554 564 1,320,402,369 1,059,204,397 14 342 348 686,890,955 574,082,116 15 189 190 367,958,149 322,705,218 16 66 69 118,060,241 99,102,856 17 22 23 20,743,980 18,454,302 18 141 177 314,912,363 277,107,409 19 238 300 372,465,169 330,471,401 20 232 286 585,662,772 582,284,470 21 46 59 20,899,748 29,402,559 22 198 254 162,402,134 332,847,489 Ove ra ll Tota l 4,260 4,705 9,216,158,888 7,644,747,164
As expected, the majority of product types (in terms of number) used by SME clients are the term loans and overdrafts (70%).
Product segmentation as a risk driver candidate makes the most sense for asset based businesses such as Lease and Factoring. In our dataset, there are examples of behavioural risk drivers constructed specific to the overdraft product. This effectively means that if a customer does not have an overdraft product, the value for these variables will be missing.
Table 5: Portfolio overview - product types
Definition of default is probably the most crucial point of interest in PD modelling since that is the information we are trying to calibrate our model against. Based on the Basel-II compliant standards, the default is recognized if at least one of the following default events (impairment triggers) has taken place4:
1. Bankruptcy or Financial Reorganization: The borrower has sought or has been placed (or is likely to seek or be placed) in bankruptcy or similar protection, where this would avoid or delay repayment of the financial asset;
2. Arrears5: The borrower has failed in the payment principal or interest/fees, when due or by acceleration and such payment default has remained unsolved/un-waived for the period indicated below:
i. Corporate borrowers (excluding Financial Institutions): 90 calendar days or more due to financial reasons;
ii. Financial Institutions: payment failure, due to non-operational reasons unsolved for more than 14 working days. and/or the Changed Forbearance parties with 30 days arrears
3. Unlikely to pay: The Borrower has evidenced significant financial difficulty, to the extent that such a financial difficulty will have a negative impact (that can be assessed with more than 50% confidence) on the future cash flows of the financial asset. Also as part of unlikely to
4 Materiality threshold of 250 EUR applies. Note that the new Basel definition of default –which has an updated
materiality threshold of EUR 500- had not been finalized by EBA by the time this modelling exercise was performed. From this perspective, it should be noted that final PDs in this study are on the conservative side.
5 Arrears only exist at the current account, since in this portfolio interest payments and contractual repayments of term loans are always booked from a current account.
Cat Code Category Description Nr. of Ult. Par. Nr. of Org. Lending Max Limit
Lending O/S
WSCD Cards 885 922 18,089,830 2,244,849
WMOC Commercial Property Finance 94 95 91,884,355 89,716,963
ISCS Common Stock 22 22 -
-WSCR Credit Replacement 14 14 12,436,508 10,612,299 WSLC Documentary LCs 33 33 37,969,046 21,996,397 FMFX FX Derivative 71 71 - -WSFA Factoring 121 129 246,653,672 192,635,252 WSFL Financial Leases 357 368 261,813,062 267,622,568 WSUG Guarantee 476 492 247,560,718 197,937,675
FMIR Interest Rate Derivatives 337 337 -
-WSL Operational Leases 11 11 6,056,278 6,077,202
NRING Other Risk - Internal Guarantee (3) 12 12 15,384,376 14,457,643
WSOV Overdrafts 2,498 2,748 1,994,761,247 1,000,738,170
WSPB Performance Bonds 130 134 188,021,810 115,880,130
FMPS Pre-Settlement (3) 114 115 -
-IPST Preferred Stock 5 5 -
-WSUR Revolvers 35 45 37,540,587 17,729,041
WSSL Standby LCs 9 9 26,759,922 20,822,483
WSSD Subordinated Debt 16 16 50,146,025 50,146,025
WSTM Term Loans (3) 3,684 3,883 5,630,585,311 5,494,289,641
pay, if the bank has any other reason why the borrower is deemed impaired, then he is also considered a default.
It is imperative to have a default rating to each case that satisfies these criteria but in reality such clients not always get a 20, 21, and 22 rating. This is mostly due to timing issues in counting the arrears and assigning the ratings in the system. Hence, as part of default determination in this exercise, those cases who have not yet received a default rating but are already 90 days past due have also been flagged as defaulted (arrear triggered defaults).
Based on the default definition provided, into default rate of the portfolio is analysed. The figure below shows into default rate (i.e. whether or not a healthy customer has defaulted in the following 12 months) trend across monthly observation moments during 01/2007 – 06/2014.
Figure 5: Into default rate over time for the portfolio
From figure 5 above, one can see that the difference between the overall into default rate (blue line) and rating triggered defaults (red line) is shrinking which further indicates that the 90 days past due cases are better captured in the ratings in recent periods. This is one aspect of the rating systems that the regulator keeps closer attention to in the recent years.
Sampling
There are three data samples created for this model development: the development sample, the validation sample, and the out of time sample. The purpose is explained below:
Development sample - used for all the steps in building this model: univariate analysis,
multivariate analysis & model construction.
0,00% 1,00% 2,00% 3,00% 4,00% 5,00% 6,00% 7,00% 200701 200704 200707 200710 200801 200804 200807 200810 200901 200904 200907 200910 201001 201004 201007 201010 201101 201104 201107 201110 201201 201204 201207 201210 201301 201304 201307 201310 201401 201404
Into-default rate over time
Validation sample - used for back testing the model: to test predictive strength of the
model and to test risk driver parameters.
Out of time (OOT) sample - used for testing the model on the most recent available data.
The development sample and validation sample both include clients observed between January 2008 and June 2014. The out of time sample includes clients observed between July 2014 and June 2015. The behavioural data between Jan 2007 and December 2007 is used for the creation of some behavioural risk drivers (e.g. number of arrears in past 12 months) for clients observed in 2008. The default information between June 2015 and June 2016 is used to flag the default/non default for clients in the out of time sample, i.e. those observed between July 2014 and June 2015.
The following figure illustrates the timeline of the three samples.
Figure 6: Creation of development sample, validation sample, and out of time sample
I.
Quarterly sampling
Before the separation of the development sample and the validation sample, there is a quarterly frequency sampling on the data. The reason and the details are explained below:
There are three kinds of risk drivers in this model: categorical, financial and behavioural. The financial risk drivers are sourced from the financial statements which are updated yearly. Behavioural risk drivers are updated monthly. If monthly data is used for model development, the same value will be populated to all 12 months for those financial risk drivers. This will result in too much correlation between records, which conflicts with the assumptions in logistic regression regarding the independency between records. If yearly data is used for model development, the changes within 12 months in those behavioural risk drivers will be lost. Therefore, a decision is needed on the frequency that the monthly data should be sampled with. The frequency should be within 1 and 12 months.
The following figure shows the historical outstanding value changes for one SME customer as an example. Most of the times there is no change in outstanding value for three months.
Figure 7: Historical outstanding values of a SME client
To arrive at a conclusion for a sampling frequency for all the data, the number of consecutive months without change has been analysed for the potential behavioural risk drivers. Take the risk driver ‘current arrear’ for example; the following table shows the distribution of all the periods without change for all the clients. Note that the data being used in this analysis includes observations from January 2008 to June 2014.
Maximum number of months without change in current
arrear or not Count %
1 9156 30.1% 2 3949 12.9% 3 1782 5.86% 4 1187 3.9% 5 948 3.12% 6 791 2.6% 7 664 2.18% 8 613 2.02% 9 761 2.5% 10 461 1.52% 11 500 1.64% 12 437 1.44% >=13 … … Total 30417 100%
Table 6: Distribution of number of months without change in behavioural risk driver: current arrear The percentage of periods without change for maximum 2 months is 12.9%, and for maximum 3 months is 5.86%. The periods without change for longer than 3 months are less in percentage. Note that the periods without change for longer than 12 months are not shown in this table because the sampling frequency should be within 12 months, as explained above. Based on this analysis, 3 months is chosen as the interval for sampling. The same analysis has been repeated for other behavioural risk drivers.
0 1000000 2000000 3000000 4000000 5000000 6000000 2008-01-01 2008-03-01 2008-05-01 2008-07-01 2008-09-01 2008-11-01 2009-01-01 2009-03-01 2009-05-01 2009-07-01 2009-09-01 2009-11-01 2010-01-01 2010-03-01 2010-05-01 2010-07-01 2010-09-01 2010-11-01 2011-01-01 2011 -03 -01 2011-05-01 2011-07-01 2011-09-01 2011-11-01 2012-01-01 2012-03-01 2012-05-01 2012-07-01 2012-09-01 2012-11-01 2013-01-01
os_total
The following figure shows the number of clients after using different number of months to sample the data. The more months being used to sample, the more clients get removed due to their short history. The drop in number of clients is because some clients only have one or two month’s observations. If the sampling is per quarter, there is minimum level of loss on clients.
Figure 8: Number of clients using different sampling frequency
Based on these analyses, the frequency of 3 months is chosen to sample the data, so that the changes in behavioural risk drivers are largely preserved, and the correlation between records is reduced.
II.
Development sample and validation sample
This section explains the creation of the development sample and the validation sample. For the model development, 70% of the total dataset has been randomly chosen as the development sample. Since one client may exist in multiple snapshots, the selection is on client level in order to ensure the independency between development and validation sample. From the total client list of 14,610 clients, 70% of them (10,227 clients) were randomly selected. After that all the records in the total dataset that belong to those clients were selected for the construction of the development sample, which contains 117,001 records. The remaining data is for the validation sample, which contains 51,130 records. After that, only the observations from the period between January 2008 and June 2014 are selected for both those two datasets. That resulted in the final development sample and the final validation sample, which contain 92,669 and 40,619 records respectively. The following figure illustrates the selection procedure:
Figure 9: Development and validation sample creation
11000 11500 12000 12500 13000 13500 1 2 3 4 5 6 7 8 9 10 11 12
In order to ensure the representativeness of the validation sample of the development sample, Portfolio Stability Index (PSI) has been calculated for the major risk drivers as well as the ratings. The results can be found in the table below:
Tested Variable PSI result
EBITDA 0.002 QUICK RATIO 0.003 TOTAL ASSET 0.002 ICR 0.001 CurLibTA 0.002 EBITDAtoPayEp_B 0.002 LIQUID_B 0.002 NetProfitMarg 0.003 NetWorCurLib 0.004
Utilization overdraft in past 12 months 0.046
Number of arrears in past 12 months 0.0008
Currently in arrear or not 0.00006
Industry group 0.001
Customer type 0.01
Rating 0.01
Default indicator 0.00001
Table 7: Test the representativeness of the development sample
Since all the PSI results are well below 0.1, the sample is acknowledged to be representative of the population.
Univariate analysis
The purpose of the univariate analysis is to select a preliminary list of potential risk drivers which needs to satisfy a set of criteria to be eligible for modelling.
First and foremost criteria is the statistical performance of the variable. The C-statistic represents the statistical power. In univariate analysis this measure indicates the ability of a single variable to make a distinction between defaulted and non-defaulted obligors/companies. The c-value varies between 0.5 (random model) and 1 (perfect model). The higher the c-value, the better. It is hard to give a clear prescription of when the performance of a variable is good or bad. This depends on the data and the number of defaults. The best way is to make an assessment based upon the performance of the other variables (relative assessment). Selection of a variable based on the univariate c-value is not recommended since it ignores the possible combined discriminative power of variables. A statistical relationship by itself does not necessarily imply a “logical” relationship. The variable is expected to be explanatory. Therefore, the observed relation between the variable and the number of events must be in line with, for example macro-economic theory or experience. Moreover, the risk driver is questioned if it is
available, objective, recognizable and transparent enough. There are a lot of ways for
6 In both the development sample and the validation sample, there are 40% of the observations with missing
dressing in the modelling hence the chosen risk drivers need also be hard to influence, so
non-manipulative, as well as abiding legal and internal constraints. In order to achieve the goal of
satisfying all these criteria, all the 47 variables in the long list have been scrutinized.
The first check is on the qualitative aspects: transparency, feasibility to implement, etc. The first check eliminates 14 variables from further model development steps. The reasons are usually due to unavailability, un-implement ability or high correlation of some candidate risk drivers and can be found in the table below. For example the risk driver ‘leverage’ is found to be 100% correlated with the risk driver ‘net debt to EBITDA ratio’ and therefore removed from the list.
Risk driver Description Reason to remove
Arrear days Maximum of arrear days delivered from 2
external sources Too complex construction from multiple sources, impossible to implement
Current arrear 2 Arrear information manipulated with its source
(full definition available in the appendix) Definition too complicated to be implemented
Industry type code A code to specify riskier industries Code has been reflected in industry_group
Overdraft outstanding
average past 12 months Average outstanding amount for overdraft facilities for the past 12 months Only overdraft could have meaningful utilization
Arrear days in Vortex Maximum of arrear days delivered from one
single source Part of the arrear information, cannot be a risk driver
EBITDA margin EBITDA/revenues 100% correlated with EBITSal
EBITDA to total assets EBITDA/AssetsTotal 90% correlated with ROA
Interest coverage ratio EBIT/InterestExpense 99% correlated with EBITDAtoPayEp_B Leverage sum(sum(LiabilitiesCurrentTotal, LiabilitiesNonCurrentTotal) -OverdraftCredit) /EBITDA 100% correlated with NetDebtEbitda_B
Net debt to EBITDA sum(sum(LiabilitiesCurrentTotal, LiabilitiesNonCurrentTotal) , -CashandCashEquivalents)/EBITDA
99.7% correlated with LIQUID_B
Quick ratio Quick Ratio 100% with LIQUID_B
Customer type code Indication of size of the company Too much concentration in a single customer type
Overdraft outstanding average past 3 months
Average outstanding amount for overdraft facilities for the past 3 months
95% correlated with os_util_avg_12m
Overdraft utilisation 3
months ago Average utilisation for overdraft facilities 3 months ago Since business prefer stable rating outcome, 12m is chosen
Table 8: First round of risk driver eliminations (qualitative)
The second check is on the quantitative aspect: the discriminatory power of the variable. The second check eliminates 5 additional variables from further model development steps, because their discriminatory power (c-stat) is below 55%. The list of the 2nd round of eliminations can be
seen in the following table. The threshold value 55% c-stat is determined through exploring different values, from 60% to 55%. If 60% or 59% was used, there would be too few variables left for further model development. Using 55%, there are still 28 variables left for the following steps.
Risk driver Description C-stat Accounts receivable turnover
ratio ReceivablesCurrent / revenues 54%
Borrowed funds to total
liabilities sum(IntBearingBorrowingsNonCurrent, IntBearingBorrowingsCurrent) / sum(EquityAndLiabilitiesTotal, - EquityAndReserves) 51%
EBITDA to net worth EBITDA/ TangibleNetWorth 53%
Net sales Net sales 52%
Number of defaults past 12 months
Number of defaults last 12 months 52%
Table 9: Second round of risk driver eliminations (quantitative)
As mentioned in the beginning of this section, risk driver eliminations based solely on single risk driver c-stats, ignores possible combined discriminative power of variables. Therefore it is not that recommended. This is the reason why the 2nd round of eliminations are limited. For example
the stand alone c-stat for ‘industry group’ variable is 50% yet it was left in the dataset going forward knowing that possible industry groupings may be helpful in determining industry specific explanatory variables. For example in the conclusion section we will be able to see for industries in real estate, builders & contractors ‘net worth to debt’ ratio is a strong default indicator.
I.
Transformations and potential risk drivers
To capture the correct relationship with default and thus improve the model performance, variables can be transformed. Transformations applied by SAS are indicated at the end of the risk driver name. For example liquidity risk driver has been applied a logarithmic transformation and has been named LIQUID_B_Log. The relationship to default is modelled so that it is captured in a smooth manner and caps extreme values. This “capping” not only eliminates the impact of outliers in the estimation of the parameters of the final model, but also ensures that the final ratings assigned to a firm are not distorted by the impact of a small number of observations. Note that the variable ‘net profit margin’ contains extremely large values, which might bring uncertainty to following modelling steps. Therefore, before univariate analysis, a capping (and flooring as well) was applied for ‘net profit margin’ using the 5 and 95 percentile values. The capped ‘net profit margin’ is used in the c-stat calculation and transformation, as well as all the following modelling steps.
In data transformation, the data are transformed or consolidated into forms appropriate for the modelling. Data transformation can involve normalization, where the data are scaled so as to fall within a small specified range. One of the benefits of normalization is that it enables an easy comparison of the relative weight of the variables in the model.
Several different smoothing methods can be used such as linear, quadratic, cubic, exponential and logistical. Since it is hard to say up front which transformation will be best, different transformations (different bucketing techniques and different smoothing functions) for each variable have been used. As long as the transformation is intuitive and explainable, it can be used. In the model specification and estimation process, the best transformation is selected based on a fit (R-square) and business relevance.
The final choice of transformation depends on two aspects: whether it fits the data, and also whether its relation to PD is as expected. The transformation that gives the best fit to the data is selected. And the c-stat corresponding to that transformation is used for the decision on elimination. A number of functions are estimated, using the average value of the risk driver per cluster and the observed default percentage:
- Normalisation = (value–mean)/standard deviation - Linear = (a + b * average)
- Square = (a + b * average + c * average^2 ) - Logistic = (a + b / (1 + exp(- ( c + d * average)) - Exponential = (a + exp(-(b + c * average)
- Cubic = (a + b * average + c * average^2 + d * average^3)
For every function the R-square of the transformation is determined. For the continuous variables, a standard macro has been used for the calculation of discriminatory power (c-stat). Transformation outputs and their corresponding R-squares are populated by this macro.7 Below
figure is given as an example for the ‘liquidity’ variable.
Figure 10: LIQUID_B risk driver transformation
In the figure above 6 different transformations are plotted. For each of the transformations the R-square is listed (numbers in the legend). The dots represent the clustered PD-percentages. In this example the log-function has the best fit same with the exponential function. The choice for transformation in this situation depends on expert knowledge. Both functions stay horizontal with higher values. It depends on the experts which pattern best fits the business expectations for this risk driver. In this case logarithmic transformation was chosen.
For categorical variables, e.g. industry group, there is no transformation. The cross-table is created between the variable and default. The stat is derived from Somer’s d (i.e. GINI):
c-stat=50+Somer’s d/2.
Multivariate analysis
Regression modelling is based on the statistical technique of regression analysis. Since the occurrence of default is a 0/1 event the (regression) technique is a logistic regression. After the univariate analysis the number of potential risk drivers from the gross list is reduced into risk drivers that each fulfil the requirements. Next step is the actual construction of the model. Since the model predicts defaults the regression analysis should be focused on predicting the ‘bad’s. The regression technique is focused on risk drivers with a strong predictive power (high C-stat). However the resulting model must not only have a strong predictive power but also a good distinction between the goods to avoid undue concentration in the lower rating grades. When very strong risk drivers are part of the regression analysis the risk exists that a model is developed with high predictive power that lacks granularity. The most important group of risk drivers that show this behaviour are the arrears risk drivers. One solution to avoid such situations is to define a sub-model for customers in arrears, either as separate regression model or as dedicated pool using a fixed (high!) PD percentage. In our case, the variable for ‘the number of arrears past 12 months’ has indeed proved to be a strong single default indicator. To avoid bias on model outputs, models where this variable has been kicked out or models only for non-arrear clients were also constructed. Below tables show 2 models generated specifically for clients with arrear and for those without arrear respectively.
RISK DRIVERS FOR CLIENTS WITH ARREAR RISK WEIGHT STANDARD ERROR WALD CHI-SQUARE PR>CHI SQ Intercept -3.9098 0.1216 1034.55 <.0001
Liquidity ratio (logarithmic transformation)
6.0067 2.9298 4.2034 0.0403
Net profit margin (logarithmic transformation)
12.6186 1.8631 45.8711 <.0001
Current arrear 0.8419 0.0576 213.733 <.0001
Overdraft utilization past 12 months (cubic transformation)
4.1985 1.112 14.2541 0.0002
Number of arrears past 12 months 0.1956 0.0114 294.309 <.0001 Table 10: Model for clients with arrears only
RISK DRIVERS FOR CLIENTS WITHOUT ARREAR RISK WEIGHT STANDARD ERROR WALD CHI-SQUARE PR>CHI SQ Intercept -6.5253 0.1278 2608.9 <.0001
Current liabilities to total assets (logarithmic transformation)
14.9573 2.8473 27.5951 <.0001 EBITDA to payable expenses (logarithmic
transformation)
8.6744 2.5112 11.9316 0.0006
Net profit margin (logarithmic transformation)
18.071 2.4214 55.699 <.0001
Net worth to current liabilities (logarithmic transformation)
12.8174 3.1786 16.26 <.0001
Overdraft utilization past 12 months (cubic transformation)
21.9012 1.3795 252.058 <.0001 Table 11: Model for clients without arrears only
Both these models were constructed over the entire development sample period, satisfying the minimum required number of explanatory variables and both are coincidentally led by behavioural risk drivers in terms of their Wald Chi-Square values. Especially in the case of a model output completely stripped of arrear information bias in Table 11, the model was still led by the behavioural risk driver `overdraft utilization past 12 months` with a relatively high Wald Chi-Square (252.058).
In both models, as it was the case for the 1st model showcased in Table1 (page 11), individual
risk weights except the intercept are all positive and this is due to the transformation of these variables. Intuitively, one would expect ‘net profit margin’ to be negatively correlated with the final PD. Positive risk weights for those variables with expectant negative relationship with PD are due to the transformation of these variables explained in the previous section.
In Table 10 for clients with arrear, apart from the intercept, the end model consists of 2 financial and 3 behavioural risk drivers. This was expected from the arrear clients, since it is logical to assume their default behaviour is influenced by behavioural aspects along with key financials. On the other hand, the model for non-arrear clients is consistent of 4 financials vs. a single behavioural risk driver. Yet, as it can be seen in Table 11, this single risk driver is the strongest variable in terms of its Wald Chi-square value. What is also visible from the output is that non-arrear clients are logically less influenced by non-arrear type explanatory variables. ‘Net profit margin’ and ‘overdraft utilization past 12 months’ are present in both models. These are also such explanatory variables that are in line with expectations of the business to be of importance for SMEs.
6 Model comparison & conclusion
Once a model is constructed, the next step is for it to be tested for its strength against the history.8 Since the aim of this paper is to compare final model compositions rather than their
relative strength, the next step will be to compare and explain the different models created during this investigation.
There are 20 models created following the steps explained throughout this paper for the same Dutch SME portfolio.9 In order to minimize strong risk driver influences, arrear influences,
downturn period influences and to pinpoint default predictors for distinctive industries several combinations of data has been subjected to these modelling steps.
Table 12: Models created
Table 12 summarizes the different models created. Models 1 to 9 use the entire development sample horizon while models from 10 to 18 exclude the downturn year of 2008. Models 19 & 20 are focusing more on the recent years by excluding 2010 still satisfying the regulatory 5 year horizon.
12 out of 20 models (60%) are led by the ‘number of arrears past 12 months’ behavioural risk driver in terms of their Wald Chi-square value. In order to relieve the model from a probable strong variable bias, 3 models (9, 18 and 20) were constructed with exclusion of this variable. In all cases this variable`s lead was replaced by yet another behavioural risk driver.
8 Appendix IV: Model testing example for the model in Chapter 5 9 Appendix V: Models created
Model ID Model Description Strongest risk driver Type Wald chi-square
1 2008-2014 All clients Number of arrears past 12 months Behavioural 685,6
2 2008-2014 Arrear clients Number of arrears past 12 months Behavioural 294,309
3 2008-2014 Non-Arrear clients Overdraft utilisation past 12
months (cubic transformation) Behavioural 252,058 4 2008-2014 All clients in industry = Food, Beverages & Personal Care & Technology Number of arrears past 12 months Behavioural 18,065 5 2008-2014 All clients in industry = Services Number of arrears past 12 months Behavioural 25,9351 6 2008-2014 All clients in industry = Real Estate & Builders & Contractors Number of arrears past 12 months Behavioural 20,072 7 2008-2014 All clients in industry = Automotive & Transportation & Logistics Total days in arrear past 12 months Behavioural 28,1257 8 2008-2014 All clients in industry = General Industries Number of arrears past 12 months Behavioural 16,4858 9 2008-2014 All clients excluding 'Number of arrears past 12 months' Number of arrears past 6 months Behavioural 16,4172
10 2009-2014 All clients Number of arrears past 12 months Behavioural 29,4112
11 2009-2014 Arrear clients Number of arrears past 12 months Behavioural 36,3485
12 2009-2014 Non-Arrear clients Debt capacity (logarithmic
transformation) Financial 19,4813
13 2009-2014 All clients in industry = Food, Beverages & Personal Care & Technology Number of arrears past 12 months Behavioural 33,0565
14 2009-2014 All clients in industry = Services Gearing ratio Financial 20,2244
15 2009-2014 All clients in industry = Real Estate & Builders & Contractors Net worth to debt Financial 26,3653 16 2009-2014 All clients in industry = Automotive & Transportation & Logistics Overdraft utilisation past 3 months Behavioural 12,2423 17 2009-2014 All clients in industry = General Industries Age of relationship Categorical 18,5402 18 2009-2014 All clients excluding 'Number of arrears past 12 months' Total days in arrear past 12 months Behavioural 20,7891
19 2010-2014 All clients Number of arrears past 12 months Behavioural 25,6546