Scoring retail clients under Basel II’s internal ratings-based approach

(1)

Scoring retail clients under Basel II’s Internal

Ratings-Based Approach

Author: Elisabeth van der Linde (10668675)

Supervisor: Patrick Tuijp

Master thesis University of Amsterdam

MSc Business Economics, Finance Track

July 2014

Abstract

The purpose of this research is to develop a scorecard for a portfolio of retail clients under the internal ratings-based approach of the Basel II framework. The focus lies on the strength of behavioral scoring, as this is a relatively unexplored field of research. Using the logistic regression model as a forecasting method, I find that a combination of a financial institution’s internal and external counterparty information leads to a more effective and efficient risk management process. My results provide an incentive for further research into behavioral scoring on the one hand and the relation between credit scoring models and the labor costs associated with risk management on the other hand.

(2)

1 Acknowledgements

I would like to thank ABN AMRO Lease N.V. for providing me with the data I needed for my research. Further, I want to thank Remco Hoving for providing me with new insights whenever I needed them, and Patrick Tuijp for being a really helpful supervisor.

(3)

2

1. Introduction

The purpose of this paper is to find the optimal scorecard for the retail portfolio of ABN AMRO Lease N.V., a financial institution that is subject to the Basel requirements for banks, based on the data available to this financial institution. The challenging economic conditions for small- and medium enterprises in the aftermath of the financial crisis combined with the increasingly stringent requirements for financial institutions provide the rationale for this research. Focusing on the application of scorecard building at the bank level adds to this strand of literature as this field of research is usually characterized by a top-level approach.

The main advantage of the Basel II framework, which came into force in 2008, compared to the first Basel Accords of 1988 is believed to be the promotion of stronger risk management practices by the banking industry. Under the Internal Ratings-Based Approach (IRB) of Basel II, banks are in the position to internally determine the rating system that is the core of the approach for determining regulatory capital requirements (Basel Committee on Banking Supervision [BCBS], 2006a). Under the Basel III framework, that is currently planned to be implemented in 2018, the IRB-approach will remain effective (BCBS, 2010).

The financial crisis of 2008 revealed weaknesses in the risk management and capital reserves of many financial institutions. At the same time, rating agencies were blamed for aggravating the situation by giving unjustified high ratings. In February 2014 the U.S. Department of Justice sued Standard & Poor’s, one of the main rating agencies, for consciously issuing overgenerous ratings (The Economist, 2014).

The flaws with respect to risk management that were revealed by the financial crisis have led to increased regulatory pressure, which emphasizes the need for advanced scorecard models for the prediction of the probability of default (Protiviti, 2013). However, there is no uniform perception of how these scorecards should be assembled, so financial institutions may struggle with developing and implementing them.

(5)

4

The literature relating to the credit risk models of financial institutions particularly focuses on the scorecards for business clients with large exposures, while the research on scorecards for retail clients with small exposures is rather confined (Allen, DeLong and Saunders, 2004). Basel allows financial institutions to define a pool of retail clients and rate all clients in this pool in a comparable fashion, provided that each individual exposure has a low value, and that these exposures are treated in a similar manner by the bank’s internal risk management (BCBS, 2006a). The notion of retail clients can either refer to loans to individuals or to small business loans. The focus of this paper will be on leases to small- and medium enterprises, which are part of the latter category.

Working as a junior portfolio manager at ABN AMRO Lease N.V.1 (hereafter: “AAL”), this company has offered me an opportunity to use the data on their portfolio of retail clients to conduct my research. With respect to its risk management, AAL differentiates between retail clients and non-retail clients. Retail clients, which are defined as all small- and medium enterprises with an exposure below €1 million, are added to a pool that is excluded from individual reviews. This group of exposures is characterized by the fact that they all have a very low value and the initial acceptation or denial of these loans is determined by an automated decision engine. When a counterparty is accepted and becomes part of this pool, a rating will be attached to this counterparty on a monthly basis.

For this rating, AAL relies on external information; it is based solely on the Dun & Bradstreet Failure score (ABN AMRO Lease, 2013). This score ranges from 1 to 100 and indicates the relative chance of a company failing within 12 months. For example, a D&B Failure score of 10 implies that 90% of the companies in the D&B database have a lower probability of failure compared to this specific company. Failure is described by D&B as the situation where a company obtains legal relief from creditors or ceases operations. The Failure Score is an automated decision engine based on several financial indicators and events signaling the onset of failure, such as a creditors meeting or the appointment of an administrator (Dun & Bradstreet, 2014). This system is called an early warning system, as it aims to signal future default at a time when the loss resulting from default can be limited.

(6)

5

Since the probability of default depends on more factors than just bankruptcy, AAL’s scorecard for retail exposures is probably not complete. For example, a business partner can experience a severe liquidity crunch or claims from other creditors, making him unable to satisfy his financial obligations. Even though a company may not be bankrupt at this point, a bank may enjoy a large advantage from receiving this information if it helps the bank to accurately map the risks on its portfolio. Another reason for AAL to reconsider its scorecard for retail clients is that relying solely on the information of a foreign, external rating agency seems paradoxical for an institution that has an abundance of information about its own clients in-house. Therefore, it is of interest for this financial institution from both a risk management and an efficiency point of view to assess whether adding other independent variables to the scorecard for retail clients will produce a more accurate prediction of the probability of default.

At ABN AMRO Bank (hereafter: “AAB”), data concerning the behavioral characteristics of companies is becoming an increasingly important factor with respect to the scoring of clients. A client’s behavior on his financial accounts seems to be a good predictor of his behavior in the future, and his probability of going into default at a certain point in time. AAB has therefore developed the CRG score, which is solely based on a client’s behavior on his accounts and includes a company’s liquidity, the overdrawing of the company’s account, satisfying payments before the expiring date and the volatility of the current account balance. The predictability of this score is proven to be very high.

The economic research on the development and implementation of scorecards mainly focuses on financial factors functioning as predictors of default, while not much research has been done on behavioral scorecards. Therefore, looking into the predictive power of such scorecards can add to the existing literature. From a broader perspective, a research into the relevance of several financial and non-financial indicators with respect to the rating of retail clients can add to and broaden the scope of the literature about this specific group of clients. The extensive dataset consisting of the retail portfolio of ABN AMRO Lease provides an opportunity to compose a scorecard based on this dataset and determine its predictive value compared to the old scorecard, which only includes external information.

(7)

6

Considering the large sample including Dutch companies with similar characteristics, my findings can have implications for the rating of this client group in a broader sense as well. However, that does not imply that the model I find is designed to be implemented by other banks. From the perspective of AAL, sharing the new credit scoring model would mean giving up on the competitive advantage this model could provide. And more important, the implementation of the same model across different banks can lead to inter-bank defaults which may cause systemic banking risk (Barnhill and Schumacher, 2011). Looking at the consequences of correlated banking failure during the recent financial crisis, some differentiation in the credit scoring models of banks seems to be preferred.

From a more general perspective however, the independent variables that are significantly important for one bank’s scorecard for a defined group of clients are probably also important for another bank while scoring a similar client group. This justifies zooming in on a dataset of one financial institution when researching how a scorecard can be improved in order to provide a stronger risk management. Also, since most credit rating agencies dispose of the same information about companies, this research can lead to a general advice on whether to use internal-, external-, or information based on both sources in the scorecard building and implementing process.

Based on these considerations, I have composed the following research question: “Does the addition of internal counterparty information to the scorecard for retail clients lead to a better prediction of the probability of default at financial institutions, and can the implementation of this new scorecard lead to a more effective and efficient early warning system?”

This question can be divided in two sub-questions, representing the two stages at which my research will take place. The first sub-question is: “Does the addition of internal counterparty information to the scorecard of the retail portfolio lead to a significantly better prediction of default?” My aim here is to add several independent variables based on internal data to the scorecard for retail clients to see whether this leads to a better prediction of the default risk that AAL has on these clients. These variables will include financial, non-financial and, most important, behavioral information about companies. I will investigate the predictive power of

(8)

7

each of these variables with a logistic regression method. The development stage will be followed by a validation stage, where the new scorecard will be compared with the old one.

The second sub-question is: “Does the implementation of the new scorecard create a more effective and efficient early warning system in the sense that it saves costs in terms of the loss at default and employment costs?” The effectiveness of the scorecard is determined by the functioning of the early warning process; namely how often the scorecard signals the onset of a failure at the right moment. An improved functioning of the early warning system can save costs by lowering the loss at default. The efficiency of risk management can be assessed by looking at the labor intensity of this process. Improved predictive accuracy of the scorecard can decrease the amount of labor involved in the process and thereby the associated labor costs. I will conclude by looking into the implications of my findings from a broader, high-level perspective.

With respect to the first sub-question, I find that the addition of internal information to the scorecard for the retail portfolio of AAL significantly increases the predictive power of this scorecard. My findings show that the variables related to behavioral characteristics of companies are especially important in predicting the probability of default for this portfolio. Furthermore, I find evidence for the new scorecard leading to a more effective risk management through decreasing the costs of default. Moreover, the scorecard also leads to a more efficient risk management by decreasing the labor costs involved in this process. My recommendation to AAL is to further develop the scoring model that I find. From a broader perspective, this research provides a strong cause for further research into the strength and implications of behavioral scoring models.

The article will be set out as follows. Section 2 provides a description of the academic framework that this research further builds upon. In section 3, a description is given of the data, followed by a detailed explanation of the methodology that will be used for this research in section 4. Results and answers to the subquestions will be given in section 5. Conclusions and policy implications of this research are summed up in section 6.

(9)

8

2. Theoretical framework

There is a broad scope of literature related to the implications of Basel II and the Internal Ratings-Based Approach. Under the IRB approach, banks may internally determine the risk parameters which, in combination with the risk-weight functions supplied by the Basel Committee, lead to the calculation of the risk-weighted assets that should be in place to account for credit risk (BCBS, 2006a). The main part of this literature comprises the impact of the IRB approach on a bank’s cost function in terms of regulatory capital, the implications that may result from proper working credit rating models and differences in these implications on financial institutions of various magnitudes.

Jankowitsch, Pichler and Schwaiger (2007) emphasize the importance of improving credit rating models, by showing that, in general, more fine-tuned scorecards will lead to lower capital requirements. This result is driven by the fact that rating models with a higher statistical power decrease the effects of adverse selection, and this can lead to a decrease of capital requirements when several qualitative standards are met. This paper provides a rationale for the improvement of credit rating systems by showing the potential economic value of beneficial enhancements of scorecards within the Basel II framework. It underscores the importance of improving scorecards by highlighting two different advantages; lower capital requirements on the one hand and a decrease of adverse selection costs on the other hand.

Adverse selection refers to the case where a bank and its clients have asymmetric information, which leads to a situation where the bank sets one price for all of its customers and is thereby more likely to select the bad (in this case, bad means that a company is less creditworthy) clients. This situation is caused by the fact that good clients are not willing to pay the higher price because this price captures part of the risk that banks face with respect to bad clients. This is in accordance with the findings of Broecker (1990) that without efficient credit scoring models, adverse selection may eventually lead to a state with only one bank or one rating system in a static general equilibrium framework. The bank that has implemented the most reliable or less costly rating model will stay in the market.

(10)

9

Stein (2005) provides another rationale for improving credit scoring models, by showing that strong models cost less to operate than weaker models, assuming a fixed cost function. This enables financial institutions with more powerful models to exercise more profitable lending compared to organizations with weaker models, even when the stronger model implies that less loans are granted.

Hakenes and Schnabel (2011) find that the ability of banks to choose between the standardized and IRB approaches gives a competitive advantage to larger banks due to the high fixed costs of implementation of credit scoring models. They argue that these costs may lead to small banks preferring riskier investments in order to obtain a higher deposit rate. One important characteristic of credit scoring models is that they should be reviewed and adapted often because of the constantly changing economic environment, the alterations in regulation and the difficulties associated with predicting future events. This implies that the literature regarding these models is also subject to variable conditions and is not demarcated in every aspect, so there is room for further research in various topics associated with credit scoring.

The risk parameters in credit ratings can be either determined internally, or by external credit rating agencies. There are substantial differences between rating systems created by agencies and by banks, as described by Carey and Treacy (2000). The key difference lies within the fact that agencies do not make an investment while banks do. At banks, the revenues that are made with credit products must cover the costs of modeling risk. Since the developing of scorecards can be very costly and time-consuming, banks might be better off by relying on external information even if that means relying on less up-to-date and reliable information. Furthermore, rating agencies are believed to handle a through-the-cycle perspective, while in contrast banks use point-in-time credit quality measures (Altman and Rijken, 2004). In other words, rating agencies often focus on long-term default rates instead of one-year investment horizons in order to avoid reacting to temporary shocks and to maintain a certain rating stability. Since for financial institutions rating stability is not an objective, they examine both the permanent and temporary parts of changes in credit quality. From the literature, it becomes clear that rating agencies and banks have a different approach towards the scoring of companies, but there is no unambiguous view on whether one of the two is more capable to produce accurate ratings. The main reason for

(11)

10

banks to outsource the activity of rating companies is a shortage of capacity in terms of time and people.

The IRB approach has led to large differences in the implemented internal risk rating systems of banks, which results in inconsistent estimates of portfolio credit losses (Jacobsen, Lindé and Roszbach, 2006). A main part of the literature focuses on evaluating and comparing the main methodologies for credit scoring at the portfolio level (see for example; Crouhy, Galai and Mark, 2000; Gordy, 2000; Lopez and Saidenberg, 2000). These articles are all Basel II and pre-financial crisis however. Jackson and Perraudin (2000) find that the main credit scoring models at that time were probably not reliable enough to be the sole determinant of regulatory capital requirements. Such an evaluation of scorecards has not yet been performed based on more recent information.

Carey and Hrycay (2001) evaluate methods of rating quantification by means of the simulation of internal ratings, since they do not have access to data on financial institutions’ internal ratings and the associated company characteristics. This paper provides an empirical evaluation of the major methods in financial institutions’ estimations of the probability of default. The authors conclude that accurate one-year default estimates by the internal rating-based approaches of financial institutions are achievable. Only a limited amount of research has been done on the implementation of the IRB approach by banks and the functioning of credit scoring models during and after the financial crisis.

As in accordance with the evidence found by Allen et al. (2004), the literature regarding risk management focuses to a large extent on the models used to rate wholesale clients, while with respect to retail clients the literature is quite limited. The articles of Lucas, Klaassen, Spreij and Straetmans (2001) and Niemann, Neukirchen and Schmidt (2008) are examples of evaluations of rating prediction models at the corporate level. Allen et al. (2004) focus on credit risk at the retail level, and find that this group of companies cannot simply be rated by means of downsizing a model for wholesale loans.

(12)

11

The literature that comes closest to this research is the article of Dietsch and Petey (2002), which proposes a Value-at-Risk model to derive the allocation of capital and loans pricing schemes. The methodology is applied on a large sample of companies from the French SME sector. However, this research does not conclude anything about the relevant factors in determining the probability of default, nor does it provide a recommendation on whether banks should base their rating on internally or externally collected information. Furthermore, this article, like most part of the literature on credit scoring models, bases its conclusions on pre-crisis information.

Clearly, the sector that represents all small- and medium firms constitutes an important part of the economy and in particular, an important source in the creation of new jobs. Evidence was found that SMEs have more difficulties in accessing formal sources of external financing (Beck and Dermirguc-Kunt, 2006; Berkowitz and White, 2004), and that the more stringent regulations following the financial crisis have made this even more difficult. Based on a dataset of the retail clients of a large German bank, Pun, Rocholl and Steffen (2011) found that the financial crisis led to a severe contraction of the supply of loans to retail clients. At the same time, the demand for these loans also decreased, but this effect was far less significant than the effect on the supply side.

Based on these findings, a more accurate prediction of the probability of default of retail clients might not only be an advantage for banks, but also for these clients, since this will lower the cases where companies are falsely predicted to go into default, and thereby are wrongfully denied credit. At the same time it will lower the cases where a company unwarranted receives a good rating, and thereby it will decrease the probability of events were a company that is not creditworthy receives credit and goes into default.

The Z-score analysis of Altman (1968; 2002) lies at the heart of most default prediction models, and this model has been frequently built upon in further research. It provides a multivariate discriminant analysis of financial ratios, and is a leading example of how to build scorecards for banks’ business clients. The firms examined in this research are all large, publicly held corporations, which is the case for most of the research on default prediction.

(13)

12

Another important part of the literature on scorecard building is the article of Press and Wilson (1978), which compares the linear discriminant model of Altman with a logistic regression model. They find evidence for the logistic regression model outperforming the multivariate model in different cases. The authors conclude that since Altman’s model requires multivariate normality, a condition that is often not satisfied in scoring applications, the logistic regression is often preferable. Especially in situations where non-normality is the case, for example when at least one of the variables is qualitative, a logistic regression is the best option. I assume this evidence as a given and therefore I use a logistic regression model to test my hypotheses. The independent variables that are examined in this research are mostly qualitative as opposed to quantitative, so performing a multivariate discriminant analysis might lead to biased results. Another advantage of the logistic regression model as opposed to the multivariate model is that it is relatively robust; many different underlying assumptions will lead to the same logistic formulation.

Even though extensive research has been devoted to corporate bankruptcy prediction based on financial factors, it is not clear how well these models perform out-of-sample (Grunert, Norden and Weber, 2005). With my research I expect to contribute to this part of the literature because I have a micro- rather than a macroeconomic view on the topic and because I have access to extensive company data from one financial institution. Furthermore, my sample includes data from during and after the recent financial crisis, which is also an addition to the existing literature.

Most of the literature on scorecards is about the developing, evaluating and comparing of these scorecards, and not so much on the independent variables that can be of interest when predicting the probability of default of a client or a portfolio. Traditionally, the rating of companies was mainly based on financial indicators and ratios such as profit, solvency, cash flow and working capital. The financial crisis of 2008 shed light on alternative approaches to forecasting the probability of default. In the years after the crisis it became evident that not only financial, but also non-financial factors can play an important role in the prediction of future default events. Already in the years pre-crisis, Grunert et al. (2005) show that adding ‘soft’ factors such as

(14)

13

management quality and market position to the scorecard leads to a more accurate prediction of default.

As already mentioned, the assessment of the effects of past payment behavior on the probability of default can be an addition to the existing literature, which still has a particular focus on financial ratios such as leverage and profitability. Payment behavior is a factor that gains in importance since banks increasingly focus on the liquidity of their clients (Thomson Reuters, 2013). The literature with respect to the predictable power of behavioral characteristics in credit scoring is rather limited. Behavioral characteristics include the actions of a company on its bank accounts and its overall ability to satisfy invoices within the required term. Since this is information that rating agencies often do not have unlimited access to, while banks do, it is not surprising that little research has been devoted to this subject so far. I expect companies that have shown a constant, good payment pattern in the past to have a low probability of going into default, since this implies that a company was able to satisfy payment obligations during periods characterized by different economic conditions. On the other hand this would mean that companies with a varying and inconsistent payment pattern have a higher probability of going into default.

Correlations of default within portfolios may be a significant aspect to consider. As Jorion and Zhang (2009) and Das, Duffie, Kapadia and Saita (2007) show, common financial shocks can lead to default correlation, and this in turn may increase probabilities of large losses on the portfolio. Related to this, the effect of macro-economic factors on the total portfolio can also be of relevance since this may influence the total risk profile of the portfolio. Several studies (Bonfim, 2009; Carling, Jacobson, Lindé and Roszbach, 2007) have aimed to incorporate the effects of macroeconomic conditions on the probability of default into credit risk models. Both articles find evidence for a strong effect of macroeconomic conditions on the probability of default. Since all companies in my sample are located in the Netherlands, I will investigate the effect of macroeconomic conditions on the portfolio that my research is based on.

A broad range of literature has been devoted to the development, implementation and evaluation of scorecards. The main part of the literature with respect to scoring business clients focuses on

(15)

14

models for large (holding-)companies. Traditional methods for credit scoring such as the one created by Altman have been adopted and further developed in later literature. With respect to the parties developing scorecards, it is clear that the difference between rating agencies and financial institutions is that agencies have a through-the-cycle perspective as opposed to the point-in-time perspective of banks. The underlying reason is that rating agencies aim for rating stability while banks do not. There is no unambiguous evidence on which of the two is more capable of providing accurate ratings.

The IRB approach of Basel II gave financial institutions the ability to develop and implement credit scoring models internally, and at the same time it has sharpened the focus on these models. The credit scoring literature is prone to constant changes and improvements, as a sequence of the internal nature of this field of study. Credit scoring models are based on historical samples and predictions about the future, but since the future always remains unpredictable, the models should be adapted regularly.

A large part of the existing literature consists of high-level research adopting a macro- rather than a micro point of view, while research on the internal implementation of models by financial institutions is rather restricted. This research aims to fill this gap by modelling a large sample of retail clients from one financial institution. This dataset, including available internal information about companies’ behavior on its bank accounts, gives an opportunity to contribute to the literature by researching the importance of behavioral scoring and the effects of the liquidity of companies on their probability of going into default. Furthermore, the availability of information provided by an external rating agency allows for a comparison between the predictability of internal and external information for a bank rating its clients.

3. Data and descriptive statistics

3.1 Data

In order to develop a scorecard for a specified portfolio of clients, I have collected historical data including general information about these clients and information about the factors that I expect to have a predictive value in forecasting the probability of default. I also have data about the

(16)

15

events of default during the sample period. This data is provided to me by ABN AMRO Lease, ABN AMRO Bank and Dun & Bradstreet (D&B) and will be described in detail below. The links between these three entities are shown in figure 3.1. AAL is a fully autonomous subsidiary of AAB, and is a financier of business assets. Dun & Bradstreet is an independent credit rating agency functioning as a supplier of counterparty information to AAL.

Figure 3.1 Data sources

ABN AMRO Lease N.V. is a 100% subsidiary of ABN AMRO Bank, and hires Dun & Bradstreet to collect counterparty information.

ABN AMRO N.V.

100%

ABN AMRO Lease

N.V. Dun & Bradstreet

The main dataset I intend to use for this research includes company information regarding the retail portfolio of ABN AMRO Lease. This information is available on a monthly basis from November 2009 until March 2014. Not all the companies were included in the sample during the whole sample period, since some companies first became a client during this period or because their lease closed during this period. The average amount of companies included in the sample per month lies around 2,000. The total amount of companies included in the dataset during the whole sample period is 3,375.

The dataset contains general company information including the company’s business name, location, primary and secondary Standard Industrial Classifications (SIC Code), starting date of the company and the number of employees. Additionally, the dataset contains several scores that are generated by Dun & Bradstreet. As mentioned before, there is the D&B Failure score, which indicates a company’s relative probability of going bankrupt. This score is based on several financial and non-financial indicators and events that can be a signal of an increased probability of failure. Additionally another financial score created by D&B is included, which is called the D&B Paydex score. This score refers to the payment behavior of a company based on trade experiences of its suppliers. Similar to the Failure Score, it ranges from 1 to 100, where a higher

(17)

16

score indicates that a company will pay its bills more promptly compared to a company with a lower score. Similar to the CRG Score of AAB, this score represents the payment performance of a company. The difference between both is that the CRG score refers to the payment of debts while the D&B Paydex score refers to the payment of bills.

From the database of ABN AMRO Lease, I collect data on the payment behavior of retail clients for the period starting November 2009 to March 2014. Three variables are created with respect to payment delays. DPD_1 indicates a payment delay of 1 to 30 days, DPD_2 indicates a payment delay of 30 to 60 days, and DPD_3 indicates a payment delay of at most 90 days. This data is available for the whole sample period. A company can have more than one delay of payment at a certain moment since companies can have multiple contracts with AAL. Furthermore, I have data about the event of a so-called Storno, which is the automatic retraction of a payment, and may imply that a company is not sufficiently liquid to suffice its payment obligations. From this database I also collect data with respect to the event of default in the respective period. The definition of default at ABN AMRO is the event where a company did not meet its payment obligation within 90 days after the expiration date of an invoice. This is the same definition of default as is managed by the Basel II committee (BCBS, 2006a).

As already mentioned above, another factor of interest that I want to test is the CRG Score, which is a behavioral scorecard composed by ABN AMRO Bank. This score is based on the behavior of the customer on all accounts at AAB. The CRG is proven to have large discriminatory power, since it has a Gini coefficient2 of 92. This score can take a value of 1 to 5 or 7, where 1 indicates very good and consistent behavior of a customer, 5 indicates bad behavior3, and 7 indicates default. I can collect this score for approximately 86% of my dataset, since that is the share of the retail portfolio of AAL that also has a credit facility at AAB. This variable is relevant and unique because ABN AMRO does not share information about its customers with rating agencies such as D&B.

2_{The Gini coefficient can be used to measure the discriminatory power of credit scoring models, where a high} coefficient indicates high discriminatory power and vice versa.

3_{The definitions of good and bad behavior are, among other factors, based on a client’s income, transactions, delay}

(18)

17

3.2 Descriptive statistics

The first main sample I start with is an unbalanced panel consisting of 109,998 observations, consisting of all companies appearing in AAL’s retail portfolio in the period from November 2009 to March 2014. The retail portfolio can be divided in two groups based on the products that AAL offers these clients. The first product is called “AAL Standaard” and covers all leases with a maximum exposure of €250.000 that are arranged through offices of AAB. The second product, “Bedrijfslease”, covers all leases with a maximum exposure of €1.000.000 that are arranged though the AAL office (AAL, 2013). In the dataset a distinction between these products can be made as the “Standaard” product is assigned a lessor code 320 in all the systems. In the total dataset, 66,036 observations have a lessor code 320, which implies that the largest part of this dataset concerns very small business loans. All companies in the sample are uniquely identified by a Business Partner number (BP number). I drop all duplicate BP numbers within the same month, and 93,769 observations remain. For the data with respect to payment delays and default I replace all observations that have a negative value by zero. These negative values imply that a company is in credit instead of debit at AAL, an event that may be caused by for example prepayments. Since I am only concerned with payment delays, these negative values are not relevant.

The variable default is given in the amount of euro’s that a company has in default, and since I am not concerned with the amount of euro’s a company has in default but rather with the event of default, I create a binary variable default_bin that is 1 when a company is in default at a certain moment and 0 otherwise. I also transform the variables with respect to payment delays

DPD_1, DPD_2 and DPD_3 into binary variables. Because I want to predict the probability of

default as a future event, I generate the dependent variable default_f_halfyear for default happening 6 months forward.

For the variable EMPLYS I have a total of 65,687 observations, and the mean of 10.69 indicates that the average company in this dataset is very small. For the variable FAIL_score, which represents the Failure Score as created by D&B I have 2834 missing observations. The reason for these missing observations is that the system used to match the retail clients of AAL with the scores generated by D&B sometimes fails to match scores to counterparties. This variable is

(19)

18

important since in the initial state, it is the only predictor of default, and therefore I fill the missing observations by taking an exponentially weighted moving average (EWMA) of the Failure score in the previous four months, and if there are not sufficient observations to calculate the EWMA I simply copy last month’s Failure score. Since the Failure Score often does not change very sudden, I consider this an acceptable method. With respect to the variable

PAYDEX_score I repeat this procedure, which leads to the replacement of 28,466 observations. I

create a binary variable Storno for the event of an automatic payment retraction in one month, since I am not concerned with the amount of storno’s but rather with the event of one happening.

With regard to the CRG score, I have data about a detailed CRG score, which is divided into more specific categories compared to the general CRG score and from which I am able to determine the general CRG score. As described before, the CRG score can take on a value from 1 to 5 or 7. With respect to CRG I have 78,416 observations, which is approximately 84% of my dataset. Since I expect the predictive value of the CRG score to be high, I split my dataset in one sample with companies for which a CRG score is available and one with companies without a CRG score, and develop a different model for both samples. Table 3.1 below provides descriptive statistics of the separate samples. Table A1 in the appendix displays the descriptive statistics of the complete dataset (both CRG and non-CRG). Since a CRG score of 7 indicates default, I create a new variable out of CRG. CRG_bin is a binary variable, which is one if CRG is equal to 7, indicating default, and 0 otherwise. For CRG, I replace all values of 7 by 0. In this way, the effect of a company scoring relatively bad at AAB can be separated from the effect of a company being in default at AAB. One thing that stands out from table 3.1 is that the average payment delays (either 1, 2 or 3 months) are higher for companies without a CRG score. The same holds for default.

Based on the data about the foundation year of a company I create a variable years_incorp that indicates the years a company has been in business. For the dataset of companies without a CRG score I have replaced the missing observations of years_incorp by 23, which is the mean of this variable. The reason for this is that this variable has a significant predictive power for this sample, which becomes more evident if the missing values are replaced.

(20)

19 Table 3.1 Summary statistics samples with and without CRG score

The CRG score is available for 84% of the complete dataset, and therefore the first sample is much larger than the second one. Companies that have a CRG score are also a client of ABN AMRO Bank, while companies without this score are not. Below the descriptive statistics of the dependent and independent variables of both samples are shown.

1_{= binary variable}

In order to test the model that I find by means of an in-sample validation, I randomly divide my complete dataset in a development set consisting of 75% of the total sample and a validation set consisting of 25% of the sample. I do this for both the sample with and without CRG score. With respect to the development group of clients with a CRG score, I have 58,812 observations for in-sample estimation. With respect to the development group of clients without a CRG score, I have 11,515 observations for in-sample estimation.

4. Methodology

As explained before, currently the only variable that indicates whether reviewing a retail client and its probability of default is necessary is the D&B Failure score. I expect that adding additional variables based on internal information of ABN AMRO Lease and ABN AMRO Bank to the scorecard will lead to a substantially higher discriminatory power for the predictor of the

Sample 1: With CRG Score Sample 2: Without CRG Score

Observations Mean S.D. Observations Mean S.D.

default_bin1 78,416 0.015 0.123 15,353 0.021 0.142 default_f_halfyear1 78,416 0.034 0.182 15,353 0.049 0.216 DPD_11 78,416 0.108 0.310 15,353 0.126 0.332 DPD_21 78,416 0.036 0.187 15,353 0.052 0.221 DPD_31 78,416 0.018 0.132 15,353 0.027 0.162 EMPLYS 55,296 10.119 26.485 10,391 13.696 44.768 FAIL_score 78,416 44.732 26.307 15,353 43.193 26.111 PAYDEX_score 78,416 70.520 11.629 15,353 69.287 12.945 Storno1 78,416 0.141 0.348 15,353 0.133 0.340 years_incorp 69,593 20.983 20.259 15,353 21.396 18.244 CRG 78,416 2.431 1.426 - - - CRG_bin1 78,416 0.096 0.294 - - -

(21)

20

probability of default related to the retail portfolio. The internal information that I use for my research is mostly related to the behavior of companies on their bank accounts and liquidity, since it is the potency of behavioral scoring that I am interested in. The hypotheses are described below, where the first hypothesis relates to my first sub-question, and the other two hypotheses relate to my second sub-question.

H1: With respect to the retail portfolio, a scorecard including both internal data of ABN AMRO (Lease) and external data of a rating agency can better predict the probability of default compared to a scorecard solely based on external counterparty information.

H2A: The implementation of the new scorecard will lead to a more effective risk management as the loss at default will be decreased by better signaling through the early warning system.

H2B: The implementation of the new scorecard will lead to a more efficient risk management as the employment costs will be decreased through a less labor intensive account management process.

To test the first hypothesis, the dataset consisting of retail clients will be divided in two randomly assigned samples; a development group and a validation group. The relevance for this approach to test the probability of default is derived from the research of Löffler and Maurer (2011). They first conduct an in-sample regression to find a dynamic model for the effect of leverage on default probabilities, and then they assess this model with an out-of-sample validation. My development group consists of 75% of the total dataset and will be used to test the effect of the explanatory variables on the event of default for the sample period. The findings of these regressions will be used to compose a new scorecard model. I will then perform an out-of-sample validation using the other 25% of the dataset, by means of plotting ROC curves and the Somers’ D statistic. An elaboration on the development and validation stage will be provided below.

(22)

21

4.1 Macro-economic background

Since all the companies included in the dataset are small- and medium enterprises that have their (main) office located in the Netherlands, a description of the macro-economic conditions in the Netherlands should be provided to put my findings in perspective. Figure 4.1 below shows the growth of the Gross Domestic Product (GDP) and the amount of bankruptcies for the period 2005-2013 in one graph. What stands out is the effect of the financial crisis of 2007-2008, which resulted in a drop in GDP growth starting in the last quarter of 2007. The amount of bankruptcies started growing at a later point, and was at its highest in the first quarter of 2013. It can be seen that there is a certain trend in the relation between GDP growth and the amount of bankruptcies; an increase in GDP growth is usually followed by a decrease in the amount of bankruptcies and vice versa. It seems that the longer the contraction of the economy lasted, the more rapid these events followed upon each other.

Figure 4.1 GDP growth and bankruptcies in the Netherlands

This figure shows the level of GDP growth and the amount of bankruptcies per quarter for the period 2005-2013. The percentage scale for GDP growth is shown on the right-hand side of the graph, and the scale for the amount of bankruptcies on the left-hand side. Source: Centraal Bureau Statistiek (CBS), 2014.

For the interpretation of my results, it is important to consider that my sample period includes a period of global economic distress, which has probably increased the financial distress of the companies in my dataset. In order to control my findings for the effect of the decreasing economic growth during the sample period, I will include time fixed effects in my model.

(23)

22

4.2 Development stage

In the development stage of my research, the independent variables that were described before will be added to the scorecard to find out their effect on the prediction of the probability of default. I expect the variables DPD_1, DPD_2, DPD_3, Storno, CRG_bin and CRG to have a positive effect on the probability of default. The variables years_incorp, EMPLYS, FAIL_score and PAYDEX_score I expect to have a negative effect on the probability of default.

To control for the effect of the challenging economic conditions during my sample period and the financial distress this could have caused for companies at certain periods, I include monthly dummies in my model. I decide to use monthly dummies since during the crisis, economic conditions changed at a very rapid pace. The coefficients for these time fixed effects are not relevant for interpretation.

I will perform logistic regressions of the dependent, binary variable default_f_halfyear on these independent variables, using my development group, to estimate the effect of the parameters. I am interested in finding out the effect on the probability of default in the following six months. Using panel data, the model I will research is the following:

( ) ( ) ( )

Based on the estimation given above I will decide which parameters have a significant effect on default and should therefore be added to the scorecard. If we notate the total estimation found with the equation above as PD*, then the probability of default is calculated with the logistic function:

(∑ )

(∑ ) ( _∑ ) ( )

This is a forecasting method that uses a logistic regression model, where i denotes individual companies, and t denotes the time in months. The dummy variables dt are included to capture the time fixed effects of the economic situation in the Netherlands. With this model I estimate the

(24)

23

effect of the independent variables on the probability of default during the 6 months after their occurrence.

One problem that occurs when predicting a future value at a certain point in time is that this leads to overlapping blocks of data with respect to the dependent variable. Therefore, as a robustness check of my findings I will correct for the serial correlation in the dependent variable. I will do this by estimating the logistic model with serial dependence in the dependent variable using the generalized estimating equations approach of Liang and Zeger (1986). The serial correlation is taken to be exchangeable, where the correlation structure is defined as:

{

The findings of these regressions will be added as an appendix in my research.

4.2.1 Likelihood ratio chi-square test

When I have decided which independent variables should be added to the scorecard based on Wald tests, I can check the overall goodness of fit of this model with the likelihood ratio (LR) chi-square test. This statistic is determined by comparing the computed model with a restricted model (Crainiceanu and Ruppert, 2004). I will compare my new scorecard to the old scorecard, since the old scorecard only includes the independent variable FAIL_score, and is therefore a restricted version of the new scorecard. The null hypothesis of the LR test assumes the new model is no different from the old model with all other coefficients set to zero:

Where are all independent variables except for FAIL_score. If the null hypothesis of this test can be rejected in favor of the alternative hypothesis, the full model has a significant higher predictive power compared to the reduced model that does not include the other independent variables. The statistic of the likelihood ratio chi-square test is:

(25)

24

( ) ( | )

( | ) ( )

Where ( | ) is the likelihood function pertaining to the model given in equation (1). 4.2.2 ROC curves

One more assessment of the overall fit of the model will be performed by means of ROC curves. In a receiver operating characteristic (ROC) curve, the sensitivity and the specificity of a model are plotted. The sensitivity of a model shows what proportion of true positives are classified as positive by the model4. The specificity shows what proportion of true negatives are classified as negative by the model. When the ROC curve is plotted, the goodness of fit of a model can be interpreted from the proportion that represents the area under the ROC curve. This area is equal to the probability that when there are two companies; one that has defaulted and one that has not defaulted, and they are selected at random, the model predicts a higher probability of default for the defaulted than for the non-defaulted company (Satchell and Xia, 2006).

4.3 Validation stage

In the development stage, a model is created that fits the data of this development group. The accuracy of this model needs to be tested more thoroughly by means of an out-of-sample test on a validation sample. Especially in forecasting, it is important to assess the results of the development stage on out-of-sample data since focusing on the results from one sample might lead to an over-fitted model.

The Basel Committee suggests several ways to cross-validate credit scoring models such as the ROC curve, the Accuracy Ratio (AR) and Somers’ D statistic. These methods are all very much connected to each other and are expected to similar findings (Orth, 2012). First I will plot the ROC curves again for the validation group, in order to test whether the model fits the data as well as for the development group. This test is needed because the model could be over-fitted with respected to the sample it is based on.

4_{In this case; positive means default and negative means no default. So a true positive is a default that is correctly} classified as default by the model.

(26)

25

For the cross-validation of my model, I will use Somers’ D measure of predictive accuracy (Somers, 1962). The Somers’ D statistic is an asymmetric measure of the relation between predictor X and outcome variable Y. The statistic is defined as follows:

( )

Where is the difference between corresponding conditional probabilities, namely the probability that the larger of two X-values is associated with the larger of two Y-values and the probability that the larger of two X-values is associated with the smaller of two Y-values (Newson, 2002). The statistic has a range from 0 to and determines the strength of a relation, where 1 indicates a strong relationship. I will compare the Somers’ D of my new model to that of the old scorecard in order to compare the predictive accuracy of both models based on an out-of-sample test.

4.4 Effectiveness analysis

To find out whether the implementation of a new scorecard will lead to an improved risk management for a financial institution, the first step I take is to look at whether the risk management has become more effective. The main assumption beforehand is that the new scorecard, which includes internal information, will have a higher predictive accuracy than the old one.

The effectiveness of the scorecard can be determined by looking at the costs in terms of losses at default that are saved through a better functioning of the early warning system, compared to the old scorecard. The approach here is to map the percentage of defaults that are correctly classified as bad by the old and new scorecard, and also at the defaults that were falsely classified as good. This can be done by looking at the sensitivity of the models as described in the subsection about ROC curves. The next step is to find out what amount of costs can be saved by the early signaling of a default. The formula that will be used to compare the effectiveness of the new and the old scorecard will then be:

(27)

26

With this approach, the potential effectiveness of risk management is tested by looking at whether the early warning system has improved by means of more accurately signaling the true defaults as defaults. The potential costs that can be saved through improved effectiveness are then calculated by multiplying this with the amount of defaults in the last six months of my dataset and the average amount in euros of defaults in the retail portfolio. I have chosen to use the last six months of my dataset since this is the most recent data I have with respect to default. Looking at one month only would not be sufficient since defaults in this portfolio are very rare and therefore the amount of defaults can vary largely between months.

The true positives and false negatives will be calculated separately for the scorecards with and without the CRG score. The total calculation will be based on the defaults of the complete portfolio, in order to find the total improvement of effectiveness compared to the old scorecard.

4.5 Efficiency analysis

The second phase of researching whether risk management has improved as a result of the new scorecard, is to look at the efficiency of this process. The efficiency of the risk management process can be evaluated by looking at the employment costs of this process. There are two sides to the employment costs associated with the development and management of a new scorecard.

On the one hand, the development of a scorecard will imply extra costs for a financial institution since it involves labor intensity to develop, test and implement the scorecard. The costs of developing the scorecard can be estimated by looking at past experiences with these processes. What makes this process often very costly is the time that is involved in the improvement of the scorecard at several levels of government within and outside the financial institution.

On the other hand, when the new scorecard is in place, it can save costs because less false predictions occur. The rationale behind the costs of false predictions is that when a probability of default is signaled that is considered too risky, a professional is required to compose a detailed review of this company in order to decide whether further steps should be taken. To map these costs, the amount of false positives should be looked at, which can be done by assessing the specificity of the model and compare this with the old scorecard. The potential decrease of false

(28)

27

positives should then be multiplied with the employment costs involved in reviews to find the efficiency of the new scorecard compared to the old scorecard:

[( { } ) ] ( )

Similar to the calculation of effectiveness, the false positives will be derived from the scorecards with and without CRG score separately, and then the total improvement of effectiveness will be calculated for the complete portfolio.

5. Results

5.1 Developing the model 5.1.1 Scorecard including CRG

First I run a logistic regression of all my independent variables on the dependent variable

default_f_halfyear. The left column of panel A in table 5.1 below shows the findings of the

logistic regression for the development group of clients with a CRG score. The monthly dummies that were added to this model are not shown in the figure since these are not relevant for interpretation. All standard errors are robust.

The estimated coefficients of the variables are now tested individually with the Wald Chi-square statistic at a 5% significance level. The null hypothesis for this test is that the coefficients are equal to zero and do not add to the predictive power of the scorecard. I find that for the variables

years_incorp and EMPLYS the null hypotheses cannot be rejected. I will not include these

variables in my scorecard as they do not have significant predictive power on the probability of default. Table A.2a of the appendix shows the Wald test statistics for all independent variables.

The right column of panel A in table 5.1 below shows the estimations of the coefficients for the new scorecard without the insignificant variables. Again, all standard errors are robust. It shows that the positive coefficients with respect to payment delays have a high magnitude in predicting the probability of default.

(29)

28 Table 5.1 Developing the scorecards

The predictive value of the independent variables on default for the development groups with and without CRG score are estimated by logistic regressions. Monthly dummies are included in all regressions but not shown in the table. All standard errors are robust.

Panel A: with CRG score Panel B: without CRG score

default_f_halfyear Full model Reduced model Full model Reduced model

DPD_3 2.263*** (0.143) 2.264*** (0.110) 2.810*** (0.403) 2.778*** (0.327) DPD_2 1.482*** (0.119) 1.573*** (0.094) 1.336*** (0.312) 1.221*** (0.280) DPD_1 1.031*** (0.101) 1.027*** (0.082) 1.110*** (0.262) 1.394*** (0.230) Storno 0.564*** (0.092) 0.525*** (0.074) 0.924*** (0.252) 0.983*** (0.228) CRG 0.146*** (0.033) 0.145*** (0.027) CRG_bin 1.589*** (0.130) 1.533*** (0.105) FAIL_score -0.489*** (0.151) -0.483*** (0.120) -0.350 (0.364) PAYDEX_score -1.397*** (0.257) -1.376*** (0.217) -1.443** (0.568) -1.148** (0.444) Years_incorp -0.004* (0.002) -0.034*** (0.009) -0.043*** (0.012) EMPLYS -0.002 (0.002) -0.003 (0.003) _cons -3.552*** (0.301) -3.707*** (0.238) -3.491*** (0.575) -4.564 (0.535) Pseudo R2 0.295 0.309 0.319 0.350 N 38,896 58,812 6,275 11,515

(30)

29

The binary CRG variable, which indicates that a company is in default at ABN AMRO Bank, also has a high magnitude. This is logical, since being in default at one creditor probably means a company is neither liquid enough to satisfy its payment obligations at another creditor. The variables Storno and CRG also have a positive effect, in accordance with my expectations. The magnitude of these variables is somewhat lower.

As expected, the variables FAIL_score and PAYDEX_score have a negative effect on the probability of default. The magnitude of the latter variable is higher however, implying that the effect of payment experience of a company’s creditors is higher than the effect of a company’s relative probability of going bankrupt.

The magnitude of the coefficient CRG is low, while I expected it to be high, which was the reason I divided by dataset in samples with and without a CRG score. However, since the coefficient for the binary CRG variable does have a high coefficient, I choose to stick to this division of the portfolio, and I base my further results on the two separate samples.

The new scorecard model is compared with the old model by means of a likelihood-ratio test, since the old scorecard is nested in the new model. In other words, the new scorecard is constructed by adding additional variables to the old scorecard, which only included the variable

FAIL_score as a predictive measure. This test follows a distribution with 56 degrees of

freedom. The likelihood-ratio test researches the null hypothesis which states that the added independent variables are equal to zero. I find that the p-value of this test is equal to zero and I reject the null hypothesis, concluding that the new scorecard has a significant higher predictive power compared to the old model. Table A.3a of the appendix shows the likelihood-ratio statistic for this model.

My findings result in the following scorecard for the client portfolio including a CRG score:

̂

[ {

} ]

(31)

30

The ROC curve for the new scorecard including a CRG score is plotted in figure 5.1 below. Ideally, the ROC curve would touch the upper left corner, which would make the area below the curve equal to 1. For the scorecard for clients with a CRG score, the area below the curve equals 0.8645, implying that the model is quite accurate.

Figure 5.1 ROC curve for the scorecard with CRG score

The specificity of the model is plotted on the x-as and the sensitivity on the y-as. The area under the ROC curve (the green line) is 0.8645, which implies that the model is quite accurate with respect to estimating the PD.

5.1.2 Scorecard excluding CRG

The left column of panel B in table 5.1 above shows the results of the logistic regression for the group of clients that do not have a CRG score. Again, monthly dummies were included in the regression but are not shown in the figure. All standard errors are robust.

I perform individual Wald tests on the independent variables at a 5% significance level. Table A.2b of the appendix shows the Wald test statistics for all independent variables. The null hypothesis stating that the coefficient is equal to zero cannot be rejected for the variable

0 .0 0 0 .2 5 0 .5 0 0 .7 5 1 .0 0 Se n si ti vi ty 0.00 0.25 0.50 0.75 1.00 1 - Specificity

(32)

31 FAIL_score, so I do not include this variable in my scorecard for the client portfolio without a

CRG score.

The right column of panel B in table 5.1 above shows the new model including all independent variables that have a significant predictive power on default. Again, all standard errors are robust. The results are very similar to the results for the scorecard including the CRG score. The effect of the variables with respect to payment delays is positive and has the highest magnitude. For this scorecard, the effect of Storno is higher than for the other scorecard. The negative relation between the years of incorporation of a company and its probability of default is small, but significant.

The likelihood-ratio test, following a distribution with 54 degrees of freedom, finds a p-value of zero for this model, so the null hypothesis that the coefficients of the independent variables do not have predictive power can be rejected.

Figure 5.2 ROC curve for the scorecard without CRG score

The specificity of the model is plotted on the x-as and the sensitivity on the y-as. The area under the ROC curve is 0.8617, which implies that the model is quite accurate with respect to predicting the PD.

0 .0 0 0 .2 5 0 .5 0 0 .7 5 1 .0 0 Se n si tivi ty 0.00 0.25 0.50 0.75 1.00 1 - Specificity

(33)

32

The ROC curve for the scorecard without a CRG score is shown in figure 5.2 above. The proportion of the area below the curve is equal to 0.8617, implying that the model fits the data quite well.

My findings result in the following scorecard for the portfolio excluding a CRG score:

̂

[ {

} ]

( )

5.2 Validating the model 5.2.1 Scorecard including CRG

The scorecard found in the previous section can now be validated by means of comparing the Somers’ D statistic of the old and new scorecard using my validation sample. My validation group for the portfolio with a CRG score contains of 19,370 observations. First I plot the ROC curve for this model, to see whether the model fits the data in my validation sample as well. I find that the proportion of the area under the ROC curve is 0.8803, which is even higher than in my development group. This implies that the model was not over-fitted with respect to my development group, and that it is considerably accurate in predicting default for my validation group.

The Somers’ D statistic I find for the new scorecard has a coefficient of 0.761 under a 95% confidence interval, indicating a strong relationship between the independent parameters of my model and the independent variable probability of default. The Somers’ D statistic for the old scorecard is equal to 0.212 under a 95% confidence interval. I conclude that compared to the restricted model, the new scorecard finds a strong relationship between the independent parameters and probability of default.

5.2.2 Scorecard excluding CRG

My validation group for the portfolio without a CRG score contains 12,145 observations. Plotting the ROC curve for the model that I found in the development phase shows that the area under the ROC curve has a proportion of 0.9114. This leads to the conclusion that the model is strong in predicting default with regard to the data in my validation group.

Scoring retail clients under Basel II’s internal ratings-based approach