Comparing Classification Techniques and Variable Selection Methods for Bankruptcy Prediction of US listed companies.

Comparing Classification Techniques and Variable Selection Methods for

Bankruptcy Prediction of US listed companies.

E. Kloosterman1 Master Thesis Finance

Supervisor: Prof. Dr. L.J.R. Scholtens

Abstract

This paper compares the performance of Multivariate Discriminant Analysis (MDA), Logistic regression (logit) and Classification And Regression Tree (CART) in predicting bankruptcies between 2010 and 2012 of US listed firms for one to five years prior to bankruptcy. In addition, stepwise selection, factor analysis, decision tree and a hybrid method are used to select variable sets for the classification techniques. Stepwise-logit (one year prior), factor analysis-MDA (two and three years prior), hybrid-MDA (four years prior) and factor analysis-CART (five years prior) yield the highest mean predictive accuracy. None of the classification techniques and variable selection methods yielded significantly superior prediction accuracy for all five years prior.

Keywords: Bankruptcy prediction, model comparison, MDA, logit, CART. JEL-Codes: C38, C45, G33.

Number of words (excluding appendices): 10,913.

Introduction

The current global economic situation and its subsequent vast amount of bankruptcy filings has reemphasized the importance of bankruptcy prediction models. In the United States alone, the number of corporate bankruptcy filings during the past five years has fluctuated between 40 and 60 thousand yearly2. Commonly, the inability of companies to pay their liabilities to their debt holders triggers the bankruptcy process. The losses debt holders incur vary greatly between individual cases. To help evaluate the bankruptcy risk of current and new credit requests a good bankruptcy prediction model is a vital requirement to determine the probability of default. Brezigar-Masten & Masten (2012) argue that a good bankruptcy prediction model should contain the following characteristics; (i) reliability and robustness; (ii) ease of implementation; (iii) high degree of prediction accuracy and (iv) clear interpretability and transparency of decision making process.

Traditional bankruptcy prediction research is aimed at developing the most accurate prediction model. The last decades numerous new artificial intelligence bankruptcy prediction models have challenged the classical bankruptcy prediction models (i.e. Multivariate Discriminant Analysis (MDA) and Logistic Regression (logit)). These newly proposed models frequently outperform the classical bankruptcy prediction models in terms of predictive accuracy. However, the major disadvantages of most of these models is their unclear interpretability and obscureness of the decision making process. For example, no clear link between dependant and independent variables or using constructed independent variables which are impossible to interpret. In addition, the difficulty of implementation in combination with the “black box” characteristics of the model can lead to unreliable results. Cestari, Risaliti & Pierotti (2013) argue that from the viewpoint of the user even though a large number of these artificial intelligence prediction models are rather accurate, it is equally true that some of these are extremely complicated to use and are expensive to either acquire or develop and use. An exception of this is Classification And Regression Tree (CART) which offers easy to interpret if-then rules to distinguish between bankrupt and healthy companies (Li, Sun & Wu, 2010).

MDA, logit and CART provide practitioners clear interpretability and transparency of the decision making process, making the results easier to explain and communicate. In addition, the ease of implementation is relatively straightforward for the professionals and consequently, reducing the risk of unreliable results. Therefore, this study aims to focus on MDA, logit and CART as classification techniques. Furthermore, four different variable selection methods are used to test their effectiveness in selecting variables suitable for use in each of these three classification techniques. In other words, to

analyse the bankruptcy prediction accuracy of each variable selection method-classification technique combination. The four variable selection methods used in this study are: First, stepwise selection which is the baseline selection method used in the majority of bankruptcy prediction studies. Second, employing factor analysis to reduce the number of variables. Third, constructing dummy variables based on the decision trees (such as CART) selected split values. Finally, a hybrid selection method which combines the variables selected by stepwise and dummy variables constructed based on CART. The latter two variable selection methods are recently introduced by Brezigar-Masten & Masten (2012) (based on the study by Cho, Hong & Ha, 2010) and show promising results for the logit model.

The contribution of this study is that it is the first study to apply the two variable selection methods recently introduced by Brezigar-Masten & Masten (2012) for MDA. In addition, testing the robustness of the results of Brezigar-Masten & Masten (2012) by using a dataset containing US listed bankrupt companies as opposed to the Slovenian based company data used in their study. The dataset uses the most recent set of bankruptcies dating between 2010 and 2012, allowing evaluation of the models in the post/mid-financial crisis environment. Furthermore, each model is estimated for five different prediction horizons (i.e. one to five years prior to the bankruptcy), most bankruptcy studies limits itself to prediction of only one year prior to bankruptcy.

This paper is organized as follows. Section 2 gives an overview of the history of bankruptcy prediction in the literature and presents some main results of studies using similar methodologies. Section 3 presents the methodology of each variable selection method, classification technique and comparison methods. Section 4 gives insights in the dataset used in this research. Section 5 presents the results of each variable selection method and classification techniques for each year prior. Finally, the conclusion and limitations of this research as well as suggestions for further research are given in section 7.

2. Literature review

Given the abundance of bankruptcy prediction models, this section will focus on the most relevant to this study. Section 2.1 will discuss the most popular prediction methods to date and give insight in their practical and theoretical relevance. Section 2.2 focusses on some popular variable selection methods and explores new promising methods introduced in recent literature. Section 2.3 gives an overview of features and results of previous similar research.

2.1 Prediction methods

(1968) introduced the Linear Multiple Discriminant Analysis (MDA) for bankruptcy prediction. In this study he developed the easy-to-use Altman Z-score. The Z-score is determined by five accounting ratios3 and classifies the company based on its score and the cut-off point either in the safety or bankrupt zone. This framework was able to predict with 72% accuracy the bankruptcy two years prior to the event. The next evolution in firm bankruptcy prediction models is introduced by Ohlson (1980); he developed a logit regression which does not require the threshold assumptions of the MDA models. He identified four significant factors (i.e. company size, financial structure, performance and liquidity) affecting the probability of default. In addition, he used market data (which is forward looking compared to backward looking accounting data) to increase the predictive power of the model. MDA and logit are regularly used as benchmark models to compare the accuracy of newly proposed models.

Multivariate Discriminant Analysis (MDA) and logistic regression (Logit) are the most popular classical bankruptcy prediction models (Balcaen & Ooghe, 2006). The main advantages of these models are the ease of implementation and clear interpretability of the results. MDA and logit are often criticized for the relative low predictive performance compared to some newly developed methods (Li & Sun, 2011). Balcaen & Ooghe (2006) discuss the drawbacks of MDA which lie in its underlying assumptions; these are (i) multivariate normally distributed independent variables; (ii) equal variance-covariance matrices across the healthy and failing group; (iii) known prior probability of failure and misclassification costs; and (iv) the absence of multicollinearity. Violation of the first two assumptions result in biased significance tests and error rates. Neglecting the third assumption in the estimation of the cut-off point may result in a misleading estimate of model accuracy. A severe violation of the final assumption can lead to difficult-to-explain parameter estimates and misleading accuracy.

The logit model uses a logistic function to estimate the firm’s financial health. The logit model produces a score between 0 and 1, this score is particularly attractive because it is calibrated as probability of default. The advantage of this model compared to linear MDA is that no assumptions are made regarding variance-covariance matrices and prior probabilities of default. Furthermore, it does not assume multivariate normally distributed independent variables. However, the shortcomings of logit are the assumption of variation homogeneity of data (Lee et al., 2006) and the sensitivity to multicollinearity that could lead to unstable and difficult-to-explain parameter estimates (Doumpos & Zopoudinis, 1999).

Decision tree models such as Classification And Regression Tree (CART) are non-parametric sequential (binary) classification techniques. Altman, Frydman & Koa (1985) introduced decision trees

for bankruptcy prediction, they use the so-called Recursive Partitioning Algorithm which, in their study, outperforms MDA in bankruptcy prediction accuracy. However, the out-of-sample performance is lower than the out-of-sample performance of MDA. Li, Sun & Wu (2010) point out the following advantages of CART: (i) ease of interpretation of the predictive results, (ii) capability of generating if-then rules, (iii) invariance of monotonic transformations of the explanatory features used for bankruptcy prediction, (iv) capability of modelling complex relationship between independent and dependent features in the task without strong model assumptions (non-parametric), and (v) ease and robustness to be constructed without a long training or testing process. Li, Sun & Wu (2010) showed that CART alone significantly outperformed MDA and logit at the 5 and 1% significance level respectively in terms of prediction accuracy.

The continuous substantial improvements in computing power supported the introduction of numerous artificial intelligence models in the bankruptcy prediction area. Neural Networks (NN) is the most used artificial intelligence models (Jackson & Wood, 2013). NN uses an algorithm which models complex relationships between inputs and outputs to find patterns in data using a hidden layer. This algorithm offers two advantages compared to MDA and logit. First, NN is a non-parametric model and therefore does not require assumptions regarding the distribution of predictors and properties of data. In addition, NN does not rely on linear approaches, this allows for detection of complex relationships between predictors and the dependent variable. In general NN outperforms MDA and logit in terms of predictive accuracy (see for example, Jo, Han & Lee, 1997; Lee et al., 2006; and Paliwal & Kumar, 2009). However, NN is often criticized for the lack of interpretability of the weights obtained during the model building process (Paliwal & Kumar, 2009). In addition, Shin, Lee & Kim (2005) elaborate on the practical difficulties of implementing NN: (i) the challenge to find the appropriate NN model, this can reflect problem characteristics due to the large number of controlling parameters and processing elements in the layer; (ii) the gradient descent search process to compute the synaptic weights may converge to a local minimum solution that is a good fit for the training sample and therefore lack good generalization performance for out of sample tests.

Support Vector Machine (SVM) is a supervised learning model which finds hyperplanes in the possible space for maximizing the distance from the hyperplane to the data points. SVM has two interesting features: (i) it offers extended model possibilities and flexibility in finding suitable or undiscovered variables in predicting bankruptcy as it takes linear non-separable situations into account and (ii) it is capable of extracting the optimal solution with the small training set size since it captures geometric characteristics of feature space without deriving weights of networks from the training data (Shin, Lee & Kim, 2005). The applicability of SVM for practical users remains open to discussion as SVM is a black box which offers little explanation on variables contributing to bankruptcy (Kaya, Gurgen & Okay, 2008).

2.2 Variable selection methods

The classification techniques of MDA and logit used as benchmark models in comparison studies often apply forward selection stepwise method to select variables. This method starts without variables and includes new variables based on their discriminating capacity until new variables do not improve the model. The disadvantage of the stepwise method is that it does not lead to a unique outcome, since sequencing and initial order affect it. In addition, it can lead to counterintuitive coefficients or variables which cannot be economically explained.

Factor analysis (FA) reduces the number of variables and can identify latent dimensions or constructs that stepwise analysis cannot. West (1985) combined logit with factor analysis, this proved an efficient method to give each bank a probability of being financially distressed. Li & Sun (2011) compared the stepwise selection method and FA for MDA and logit, the stepwise method significantly outperformed factor analysis for both MDA and logit.

2.3 Previous research using similar models

Table 1 provides an overview of the main features and results of a selection of bankruptcy literature applying similar variable selection methods and classification techniques to this study (see section 3 for a more detailed discussion of each method). Note, decision tree-MDA and hybrid-MDA has not been researched so far and are therefore missing in this overview. The table illustrates the history of the prediction models used in the literature. Initially, most bankruptcy studies focussed on MDA and logit utilizing stepwise selection methods, while in more recent years other selection methods such as factor analysis and decision trees became more popular. However, stepwise-MDA and stepwise-logit are still commonly used as a benchmark model for the newer models. The accuracy results show that there is no method consistently outperforming all other listed methods. In addition, most research has limited the years examined prior to bankruptcy to one year. In order to answer the research question of whether the various variable selection techniques (i.e. stepwise, factor analysis, decision tree and hybrid) lead to significant different prediction accuracies for MDA, logit and CART the following four hypotheses and fifteen sub-hypotheses are presented. These hypotheses will each be tested for one to five year prior to the bankruptcy.

H1: There is no significant difference in predictive performance between stepwise-MDA, factor analysis-MDA, decision tree-MDA and hybrid-MDA.

 H1a: Predictive performance of stepwise-MDA and factor analysis-MDA is not significantly different.

 H1b: Predictive performance of stepwise-MDA and decision tree-MDA is not significantly different.

 H1c: Predictive performance of stepwise-MDA and hybrid-MDA is not significantly different.

 H1d: Predictive performance of factor analysis-MDA and decision tree-MDA is not significantly different.

 H1e: Predictive performance of factor analysis-MDA and hybrid-MDA is not significantly different.

 H1f: Predictive performance of decision tree-MDA and hybrid-MDA is not significantly different.

H2: There is no significant difference in predictive performance between stepwise-logit, factor analysis-logit, decision tree-logit and hybrid-logit.

Table 1.

Main features and accuracy results of a selection of previous research which use techniques similar to this study.

Features Accuracy results (in %)

Researcher(s) (year of publication) Sample size B/H YPTB used

VSM Stepwise Factor Analysis Decision Tree Hybrid

Market Matching by Period CT MDA Logit MDA Logit CART Logit CART Logit

Dambolena & Khoury (1980) _{US Industry 1969-1975 46/46 1 94.4}

3 79.7

5 70.3

Altman, Frydman & Koa (1985) US Industry 1971-1981 58/142 1 76.0

91.0*

Casey & Bartczak (1985) _{US Industry 1971-1982 60/230 1 86.0 88.0}

2 84.0 84.0 3 84.0 84.0 4 72.0 82.0 5 61.0 80.0 Keasey, McGuinness & Short

(1990) UK Industry, Net Assets 1976-1984 40/40 1 63.0

2 74.5 3 64.5 4 65.0 5 41.0 Tennyson, Ingram & Dugan (1990) US Industry, Asset size 1978-1980 23/23 2

59.0

Wilson & Sharda (1994) US Industry 1975-1982 65/64 1 82.7

Jo, Han & Lee (1997) Korea Industry 1991-1993 271/271 1 82.2

West (2000) Australian sample*** Australia No match NA 307/383 1 86.0 87.3

84.4

West (2000) German sample*** Germany No match NA 300/700 1 72.6 76.3

69.6

Lee et al. (2006)*** Taiwan No match NA 1000/1000 1 69.0 70.9

78.0

Lacerda & Mordo (2008) Portugal No match 2005 2900/81606 1 58.2 58.4

Cho, Hong & Ha (2010) Korea Asset size 2000-2002 500/500 1

72.2

69.7** 67.8

Li et al. (2010) China Not specified NA 135/135 1 88.0 86.9

90.3

Chen (2011) China Not specified 2000-2007 50/50 1

89.2 82.3 90.8 92.2

2

91.7 81.7 85.9 85.9

Li & Sun (2011) China Not specified NA 135/135 1 89.0 88.4 88.7 87.0

Brezigar-Masten & Masten (2012) Slovenia Industry, Asset size 1995-2001 592/592 1

82.8

 H2a: Predictive performance of stepwise-logit and factor analysis-logit is not significantly different.

 H2b: Predictive performance of stepwise-logit and decision tree-logit is not significantly different.

 H2c: Predictive performance of stepwise-logit and hybrid-logit is not significantly different.

 H2d: Predictive performance of factor analysis-logit and decision tree-logit is not significantly different.

 H2e: Predictive performance of factor analysis-logit and hybrid-logit is not significantly different.

 H2f: Predictive performance of decision tree-logit and hybrid-logit is not significantly different.

H3: Predictive performance of factor analysis-CART and decision tree-CART is not significantly different.

H4: There is no significant difference in predictive performance between MDA, logit and CART using the most effective variable selection method.

 H4a: Predictive performance of MDA and logit using the most effective variable selection method is not significantly different.

 H4b: Predictive performance of MDA and CART using the most effective variable selection method is not significantly different.

 H4c: Predictive performance of logit and CART using the most effective variable selection method is not significantly different.

3. Methodology

Fig. 1. Research design

3.1 Data preparation

Construction and testing of bankruptcy prediction models require both healthy and bankrupt companies. There is no standard ratio between healthy and bankrupt companies in the literature. However, the majority of the bankruptcy prediction literature uses either a ratio between healthy and bankrupt companies of 50-50 (e.g. Shin, Lee & Kim, 2005 and Cho, Hong & Ha, 2010), 70-30 (e.g. Altman, Frydman & Koa, 1985 and West, 2000) or the unadjusted ratio of all available companies, which is usually around 98-2 (e.g. Lacerda & Mordo, 2008). The latter ratio is most similar to the ratio encountered by practitioners. However, because the classification techniques try to maximize the number of correctly classified companies and are indifferent between categories4 (i.e. correctly classifying 50 healthy and 50 bankrupt is equally desirable to correctly classifying 90 healthy and 10 bankrupt). Therefore, using such a ratio would lead to an overestimation of healthy companies. For example, when using a 98-2 ratio, classifying each company as healthy would lead to a correct classification rate of 98%, without any bankrupt company correctly classified. A 70-30 ratio, to a lesser extent, gives the classification techniques a similar tendency to focus on predicting healthy companies instead of bankrupt companies. The focus of bankruptcy prediction is to predict bankrupt companies. Therefore, this study

uses a 50-50 ratio between healthy and bankrupt companies to allow the models to focus more on correctly classify bankrupt companies.

Consistent with the majority of the bankruptcy prediction literature, the bankrupt companies are manually matched based on industry and total assets. Each bankrupt company is individually matched with a healthy company within the same industry (according to the Standard Industrial Classification code) with the most similar amount of total assets. The main reason for using a matched sample is to isolate key factors, which distinguish bankrupt from healthy companies (Charitou, Neophytou & Charalambous, 2004).

Each model’s predictive power is tested for one to five years prior to bankruptcy. This requires data for bankrupt companies up to five years prior to the bankruptcy and data for the matched healthy companies for the same years. To test the models for each year prior separately, the data is split per year prior.

In practice, bankruptcy models are trained using a sample of which the outcome is known a priori, followed by the application of the estimated models on the unknown sample. It is pointless to estimate a highly accurate model for the available data if its accuracy is considerably lower for out of sample prediction. Therefore, this study uses the split-sample validation technique applying a training-testing split ratio of 70:30, as suggested by Chen (2011) and Li, Sun & Wu (2011). The split-sample validation technique splits the sample in a training sample and a testing sample. The training sample consist of 70% of the observations and is used to estimate the parameters (or cut-off points for CART) of the prediction model. The remaining 30% is used as the testing sample in which the companies are classified using the prediction model with the estimated parameters based on the training sample. Testing the predictive performance of each model only once does not give very robust results. Furthermore, it can deliver very misleading results of which the statistical significance cannot be verified. Therefore, this study uses five5 random training-testing sample splits (consistent with Cho, Hong & Ha, 2010). The five random 70-30 splits are created using Monte-Carlo simulation and are equal for each model.

3.2 Variable selection techniques

The stepwise selection method selects variables based on their discriminating capacity measured by Wilks’ Lamda. Wilks' Lambda is a direct measure of the proportion of variance in the combination of dependent variables that is unaccounted for by the independent variable. Wilks’ Lambda’s statistic can be

transformed to a statistic which has approximately an F distribution. Using the F distribution, variables with a probability lower than 0.05 are added one at a time until each unselected variable has a probability greater than 0.05. In addition, variables are removed from the selected set of variables when the probability is no longer statistically significant (i.e. exceeds 0.1).

Factor analysis is a method used for the dimension reduction (i.e. reducing the number of explanatory variables for one explained variable) by the means of lower number of unobserved variables called factors. The basic idea is to represent a set of variables by a smaller number of factors. These factors can be thought of as underlying constructs that cannot be measured by a single variable. The information obtained on the variability of the predictors and interdependencies (i.e. correlation) among them is used to reduce the number of variables. Factor analysis assumes that each pair of variables have a bivariate normal distribution.

The most common method for determining the factor loadings is Principle component analysis (PCA). PCA tries to identify values of the factor loadings that estimate the sum of squared factor loadings for a predictor (i.e. communality) as close as possible to the total observed variance. In this particular case the covariances between factors are ignored. The selection of the predictors is based on the Kaiser’s criteria that require selecting common factors with eigenvalues6 greater than one and the communality of an explanatory variable greater than 0.8 in order to obtain suitable predictors. If the first factor solution does not provide the hypothesised structure of loadings, Varimax orthogonal rotation is applied to find additional set of loadings. This rotation method maximizes the sum of the variances of the squared correlations between variables and factors, which discourages selection of the factors that are related to all variables and encourages the variables that are related to few variables.

The decision tree method selects the variables based on the CART algorithm and converts them into dummies for use in MDA and logit. This method is introduced by Brezigar-Masten & Masten (2012) for logit and is called BM1 in their research. The methodology uses each split rule estimated by CART for constructing dummies. CART has the ability of exploiting nonlinear relationships between variables and probability of default. To capture this information each split (tree node) of the decision tree is manually converted to a dummy variable based on the split criterion. Values lower than or equal to the split criterion take value zero and values greater are converted to one. This creates a set of dummy variables for each of the 25 combinations of year prior and sample split (i.e. for one year prior-split one until five years prior-split five).

The hybrid method, denoted as BM2 in Brezigar-Masten & Masten (2012) combines the variables selected by the stepwise selection method with the dummy variables of the decision tree method. By entering these variables in the prediction models the hybrid method attempts to exploit both linear and nonlinear relationships between bankruptcy probability and variables.

3.3 Prediction models

Multivariate Discriminant Analysis (MDA) is a linear classification technique, which uses selected variables to draw a boundary that best separates the healthy and bankrupt group. Based on the variables it creates a score for each company and determines the optimal cut-off score to distinguish healthy from bankrupt companies. The score is calculated according to eq. (1).

∑ (1)

Where z is the score for the i-th company using j variables. The coefficients yj for variable xj are determined to create z-scores for the optimal discrimination between healthy and bankrupt companies. A company with a z-score higher (lower) than the cut-off point will be classified as healthy (bankrupt).

As MDA, logistic regression (logit) uses a selection of independent variables to estimate the probability of default. However in the logit model the linear relationship is adjusted through the logistic transformation. Eq. (2) shows the logit model.

∑ (2)

Where yi is a score bounded between 0 and 1. This score is determined by a constant and j variables (x) with coefficient βj. A company with a score lower (higher) than 0.5 will classified as healthy (bankrupt).

The CART algorithm is a nonparametric artificial intelligence learning method that produces binary classification trees. It is used to classify objects (i.e. companies) to certain groups (i.e. active or bankrupt). CART is one of the major algorithms of decision tree method (Chen, 2011) and it uses a sequential procedure when analysing the objects.

Fig.2. Splitting algorithm of CART, where tP, tL, tR are parent, left and right nodes respectively. xj is variable j, and xj R is the best splitting value of variable xj to assign companies in either the left or the right node.

3.4 Comparison of the models

To be able to compare the different models a measure of performance which can be calculated for each model is necessary. The goal of each model is to maximize the number of correctly classify companies (i.e. classification accuracy). In other words each model seeks to minimize Type I and Type II errors, therefore classification accuracy is defined as:

(3)

Where, Type I error is defined as classifying a healthy company as a bankrupt company and Type II error as classifying a bankrupt company as a healthy company. Every model is estimated using the five training-testing sample splits for each year prior. Having five observations of each model allows me to present comparable summary statistics of each model. As in Li, Sun & Wu (2010) and Li & Sun (2011), the mean, median, minimum, maximum and standard deviation of the five observations of classification accuracy are compared. In addition, the five observations (i.e. classification accuracies) of each model give the opportunity to test whether the differences in mean accuracy between each model are significant by employing a two-tail student t-test.

4. Data

Having a well-structured reliable dataset is key in bankruptcy prediction. This section will discuss the data used for this study. Section 4.1 discusses the bankrupt sample. Section 4.2 discusses the financial ratios used in this study and presents the summary statistics. In addition, the robustness to the data specific assumptions for the models discussed in section 2.1 are checked.

4.1 Bankrupt sample

The bankrupt part of the dataset consist of 125 US listed bankrupt companies. A company is defined bankrupt when it is filed for bankruptcy (chapter 7 or 11) between 2010 and 2012. The financial industry is excluded from the sample as the financial structure of such firms is significantly different from the non-financial industry and would require industry specific variables such as capital adequacy structure, assets liability structure and interest rate premium (Ohlson, 1980). The original list consisted of 317 bankruptcies, however, due to lack of sufficient available data and the exclusion of the financial industry only 125 remain. Table 2 gives an overview of the bankrupt sample of US listed firms from which at least one of the years prior to the bankruptcy had sufficient data. The table emphasizes the inconsistent available data, as each year prior misses sufficient data for more than 15 of the 125 unique bankruptcies. Only 59 companies that filed for bankruptcy had sufficient available data for all five years prior to the bankruptcy filing. This led to a trade-off between sample size and bankrupt sample consistency throughout the years prior (i.e. one through five). I believe that sample size is more crucial for a good comparison between the models as opposed to higher consistency of the bankrupt sample as the models are compared per year prior (i.e. within the same sample) and the conditions for predictive power throughout the years are equal for each model. In addition, using the 70-30 split sample validation technique mentioned in Section 3.1, results in a validation sample consisting of only 18 companies.

Table 2

Bankrupt sample per year(s) prior.

Year(s) prior 1 2 3 4 5

Capital intensive 49 70 72 71 74

Labor intensive 20 30 35 38 36

Total 69 100 107 109 110

Altogether, the matched sample consist of 990 company-year observations. The financial data necessary to create the financial ratios is collected from Datastream. To avoid excessive influence of variables with a higher dispersion the outliers of each financial ratio are capped (Lacerda & Moro, 2008). The outliers for each non-dummy variable are capped according to Hoaglin & Iglewicz (1987), who suggested the following adjustment:

(4)

test whether the missing data is missing completely at random, the Little’s MCAR test is performed. The test is significant at the one percent level, indicating that the missing values are not missing completely at random and cannot be ignored. List wise deletion of the missing variables would lead to an extremely limited sample size and replacing missing values by the mean would limit the information carried by each variable. Therefore, missing values are imputed using the Expectation-Maximization algorithm. This algorithm, introduced by Dempster, Laird & Rubin (1977), repeatedly performs two steps to estimate the unknown data. The first step, known as the E-step estimates variances, covariances and means between variables and uses these estimates to develop a regression equation to predict the missing data. Next, the second step known as the M-step uses the equations to calculate the variances, covariances and means with the predicted missing data (without imputing the missing data). The E-step then uses these new variance, covariances and means to develop a new regressing to predict the missing data, followed by another M-step. This process is repeated until the system stabilizes.

4.2 Descriptive statistics

Table 3 shows the descriptive statistics for the bankrupt and healthy subsamples for all five years (descriptive statistics for the per year prior subsamples are presented in Appendix A, table 7 through 11). Due to the fact that this study uses various variable selection techniques, the number of variables of which these techniques can choose should be as complete and widespread as possible. The financial ratios (variables) used in this study are chosen based on their popularity in previous empirical research (such as, Altman, 1968; Altman, Frydman & Koa, 1985; Jo, Han & Lee, 1997; Charitou, Neophytou & Charalambous., 2004; Lacerda & Moro, 2008; Cho, Hong & Ha, 2010; and Li & Sun, 2011) and is limited due to data availability. The 42 ratios cover different aspects of a company’s business such as (i) activity; (ii) financial structure; (iii) liquidity; (iv) profitability; and (v) non-financial ratios. The complete list of ratios and their descriptions can be found in appendix B. The tests for equality give an indication which variables have discriminating power. For most ratios the mean and/or variance is significantly different between the healthy and bankrupt group, therefore the distribution is de facto different for both groups. It should be noted that all the estimations are done using the per year prior subsamples.

Table 3

Descriptive statistics of the healthy and bankrupt subsample for all five years prior. Both the healthy and bankrupt subsample consist of 495 company-year observations. Tests for equality for both the mean and variance between the healthy and bankrupt subsample for each variable

Healthy subsample Bankrupt subsample Test for Equality

Mean Median Standard Deviation

Percentiles Mean Median Standard Deviation

Percentiles Variance Mean

Variable 5 95 5 95 Sig. Sig.

Activity Ratios

Accounts Payable to Sales 0.108 0.080 0.079 0.020 0.250 0.126 0.100 0.088 0.020 0.250 .000 .000

Asset turnover 0.976 0.830 0.706 0.096 2.302 1.047 0.760 0.974 0.000 3.170 .000 .191

CAPEX to Sales 0.057 0.030 0.063 0.000 0.210 0.070 0.030 0.077 0.000 0.210 .000 .003

COGS to Sales 0.665 0.660 0.256 0.280 1.430 0.777 0.770 0.320 0.280 1.430 .000 .000

Equity Turnover 1.833 1.410 2.315 -1.166 7.110 1.267 0.510 3.568 -4.420 7.110 .000 .003

Fixed Asset Turnover 9.681 5.310 10.440 0.250 30.670 9.275 3.370 11.228 0.000 30.670 .057 .556 Inventory turnover 17.708 10.190 16.108 1.878 48.370 19.155 14.520 15.997 0.298 48.370 .884 .156

R&D to Sales 0.103 0.038 0.144 0.000 0.458 0.143 0.066 0.176 -0.041 0.458 .000 .000

Working Capital Turnover 2.479 1.990 6.769 -12.640 15.682 1.963 0.190 8.183 -12.640 17.580 .000 .280 Financial Structure Ratios

Book Value per Share 5.861 2.670 8.480 -0.394 30.214 1.407 0.230 7.228 -9.994 12.408 .000 .000

Current Debt Ratio 0.294 0.240 0.203 0.068 0.760 0.393 0.330 0.260 0.050 0.760 .000 .000

Current to total Assets 0.484 0.460 0.270 0.080 0.940 0.470 0.430 0.287 0.070 0.960 .151 .430

Debt Ratio 0.192 0.120 0.217 0.000 0.790 0.260 0.170 0.273 0.000 0.790 .000 .000

Debt to Equity 0.809 0.690 1.617 -2.508 4.280 0.661 0.590 2.552 -2.850 4.280 .000 .278

Equity to Fixed Assets 3.254 1.900 6.633 -11.590 15.330 0.989 0.450 7.169 -11.590 15.330 .943 .000

Fixed to Total Assets 0.268 0.170 0.256 0.010 0.834 0.286 0.220 0.262 0.010 0.800 .101 .263

Interest Coverage Ratio 1.127 0.440 11.140 -19.480 21.900 -0.517 -0.150 9.502 -19.480 21.900 .000 .013 Short-term to total debt 2.356 1.420 2.134 0.180 6.590 1.935 1.240 1.938 0.090 6.590 .001 .001 Liquidity Ratios

Acid Test Ratio 1.571 1.170 1.335 0.048 4.660 1.183 0.700 1.291 0.038 4.660 .039 .000

Cash Coverage Ratio 2.134 0.790 13.627 -23.820 28.110 0.141 0.340 11.392 -23.820 28.110 .000 .013

Cash to Total Assets 0.147 0.090 0.160 0.000 0.540 0.137 0.070 0.164 0.000 0.540 .753 .320

Cash flow from Operations to Sales -0.022 0.030 0.197 -0.320 0.252 -0.083 -0.030 0.199 -0.320 0.232 .291 .000 Cash flow to Debt Ratio 0.074 0.090 0.928 -1.370 1.760 -0.303 -0.100 0.676 -1.370 0.642 .000 .000 Cash flow to Long-term Debt Ratio 0.079 0.120 1.123 -1.440 1.930 -0.292 -0.120 0.910 -1.556 1.773 .000 .000 Cash flow to Short-term Debt Ratio 1.652 0.550 12.454 -21.760 25.070 -1.581 -0.290 10.246 -21.760 23.496 .000 .000

Cost of Debt 0.080 0.061 0.181 -0.244 0.376 0.133 0.103 0.169 -0.244 0.376 .736 .000

Current Ratio 2.225 1.810 1.711 0.160 6.300 1.665 1.120 1.650 0.070 6.300 .086 .000

Operating Cash flow Ratio 0.043 0.120 0.870 -1.610 1.532 -0.228 -0.070 0.805 -1.610 1.122 .180 .000

Quick Ratio 1.753 1.330 1.406 0.078 5.000 1.341 0.830 1.370 0.060 5.000 .089 .000

Working Capital to Total Assets 0.158 0.190 0.360 -0.730 0.684 -0.012 0.030 0.413 -0.730 0.700 .006 .000

Table 3 continued

Healthy subsample Bankrupt subsample Test for Equality

Mean Median Standard Deviation

Percentiles Mean Median Standard Deviation

Percentiles Variance Mean

Variable 5 95 5 95 Sig. Sig.

Profitability Ratios

Dividend Payout Ratio 0.021 0.000 0.086 -0.145 0.195 -0.004 0.000 0.070 -0.195 0.194 .000 .000 Earnings per Share 0.288 -0.010 1.271 -1.582 2.980 -0.726 -0.310 1.434 -3.320 1.564 .008 .000

Gross Profit Margin 0.335 0.341 0.257 -0.432 0.718 0.222 0.235 0.320 -0.432 0.721 .000 .000

Gross Return on Assets 0.316 0.283 0.301 -0.075 0.881 0.251 0.196 0.399 -0.543 1.084 .000 .004 Net Profit Margin -0.103 -0.006 0.231 -0.434 0.182 -0.210 -0.206 0.217 -0.434 0.112 .861 .000 Pretax Profit Margin -0.116 -0.025 0.271 -0.509 0.235 -0.244 -0.222 0.252 -0.509 0.126 .873 .000

assumption. The assumption that independent variables should not be highly correlated is assessed using a correlating matrix, see appendix C, table 13. The table shows that 60 variable pairings have a significant absolute correlation between 0.5 and 0.8 (i.e. between 0.5 and 0.8 or between -0.5 and -0.8) and 13 variable pairing have a significant absolute correlation between 0.8 and 1.

5. Results

This section presents the results of all the selection methods and classification techniques. First, Section 5.1 will give an overview of the variables selected by the variable selection methods. Then, Section 5.2 presents and discusses the performance of each selection method-classification techniques combination. In addition, the results of the hypotheses stated at the end of Section 2.3 are discussed.

5.1 Variable selections

All the variable selection methods select a set of variables from the original list of 42 variables using each of the five training samples for each year prior separately. This results in five sets of variables for each selection method for each year prior and 25 sets for all the years prior together. Appendix D, table 15 shows the selected variables for each method for all five training samples together. Table 4 gives a summary of all the selected variables by grouping them together in their respective ratio class. Table 4 does not include hybrid-MDA and hybrid-logit since this is the sum of stepwise-MDA and decision tree or stepwise-logit and decision tree respectively, for completeness they can be found in Appendix D, table 14.

Stepwise selection includes variables from each ratio class7 at least once for each year prior, therefore the stepwise method considers that each ratio class has added value in the ability to classify the companies. This is consistent with the idea that each of the ratio classes cover unique aspects of a company. Contrarily, the 25 factor analysis variable sets and the one and four year prior variable sets for the decision tree method do not include one or more of the non-financial ratios.

Consistent with the research of O’Gorman & Woolson (1991) stepwise-MDA and stepwise-logit select fairly similar sets of variables. For both stepwise-MDA and stepwise-logit profitability ratios are the most selected for predicting bankruptcy one year prior. After further examination each stepwise variable set includes Earnings per Share (EPS). The calculation of EPS include net income in the

Table 4

Variables selected per group of variables with in parentheses the percentage that each group represents of the total for that year prior. Note, each year prior consists five observations (i.e. variables are selected five times for each of the five random

training sets). Panel A, B, C and D show the number of variables selected out of each group for MDA, Stepwise-logit, Factor Analysis and Decision Tree respectively.

Years prior 1 2 3 4 5 All

Panel A: Stepwise-MDA

Activity Ratios 1 (8.3%) 3 (15.0%) 2 (14.3%) 3 (12.5%) 3 (18.8%) 12 (14.0%) Financial Structure Ratios 1 (8.3%) 3 (15.0%) 2 (14.3%) 4 (16.7%) 1 (6.3%) 11 (12.8%) Liquidity Ratios 2 (16.7%) 7 (35.0%) 3 (21.4%) 7 (29.2%) 3 (18.8%) 22 (25.6%) Profitability Ratios 5 (41.7%) 4 (20.0%) 5 (35.7%) 6 (25.0%) 5 (37.5%) 26 (30.2%) Non-financial Ratios 3 (25.0%) 3 (15.0%) 2 (14.3%) 4 (16.7%) 3 (18.8%) 15 (17.4%) Panel B: Stepwise-logit

Activity Ratios 1 (7.7%) 1 (4.5%) 5 (31.3%) 4 (14.3%) 6 (23.1%) 17 (16.2%) Financial Structure Ratios 1 (7.7%) 3 (13.6%) 1 (6.3%) 5 (17.9%) 4 (15.4%) 14 (13.3%) Liquidity Ratios 3 (23.1%) 8 (36.4%) 4 (25.0%) 8 (28.6%) 5 (19.2%) 28 (26.7%) Profitability Ratios 5 (38.5%) 4 (18.2%) 4 (25.0%) 6 (21.4%) 7 (26.9%) 26 (24.8%) Non-financial Ratios 3 (23.1%) 6 (27.3%) 2 (12.5%) 5 (17.9%) 4 (15.4%) 20 (19.0%) Panel C: Factor Analysis

Activity Ratios 3 (10.3%) 8 (26.7%) 3 (12.5%) 2 (6.9%) 5 (19.2%) 21 (15.2%) Financial Structure Ratios 8 (27.6%) 7 (23.3%) 2 (8.3%) 9 (31.0%) 6 (23.1%) 32 (23.2%) Liquidity Ratios 7 (24.1%) 6 (20.0%) 9 (37.5%) 9 (31.0%) 5 (19.2%) 36 (26.2%) Profitability Ratios 11 (37.9%) 9 (30.0%) 10 (41.7%) 9 (31.0%) 10 (38.5%) 49 (35.5%) Non-financial Ratios 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) 0 (0.0%) Panel D: Decision Tree

Activity Ratios 1 (10.0%) 4 (20.0%) 7 (28.0%) 0 (0.0%) 6 (23.1%) 18 (17.1%) Financial Structure Ratios 5 (50.0%) 4 (20.0%) 2 (8.0%) 4 (16.7%) 4 (15.4%) 19 (18.1%) Liquidity Ratios 1 (10.0%) 3 (15.0%) 4 (16.0%) 9 (37.5%) 4 (15.4%) 21 (20.0%) Profitability Ratios 3 (30.0%) 7 (35.0%) 8 (32.0%) 11 (45.8%) 10 (38.5%) 39 (37.1%) Non-financial Ratios 0 (0.0%) 2 (10.0%) 4 (16.0%) 0 (0.0%) 2 (7.7%) 8 (7.6%)

cost of debt for a highly leveraged firm (i.e. high debt to equity ratio) has a much bigger impact on financial health than it has for a low leveraged firm.

For both stepwise methods, liquidity ratios are the dominant ratio class in number of times selected two years prior to bankruptcy. Companies with poor liquidity ratios such as low or negative working capital and/or negative cash flows are more likely to file for bankruptcy in two years. On the other hand, factor analysis and decision tree select for two years prior more profitability ratios. Variable sets for three to five years prior are, except for factor analysis, on average larger than those for the shorter term (1-2 years prior). In addition, for both stepwise methods the proportion of activity and financial structure ratios is higher for longer estimation windows. Activity and financial structure ratios can therefore be helpful in determining long term financial health of companies. All five years combined, the most selected ratio classes are liquidity and profitability ratios. These ratio classes are essential to maintain long term financial health and should be monitored with extra care.

5.2 Classification techniques accuracies

This section compares the predictive performance and consistency of all the models. Table 5 presents the summary results of the prediction accuracy of each model for each year prior using the five random 70-30 splits for training and testing sample. Each method uses the same five random splits for each year prior to enhance the comparability of the prediction accuracies.

Appendix E, table 16 presents the results of the one-tailed student t-test to test whether the superior prediction power of one model over another is significant. One year prior to bankruptcy, the stepwise and hybrid variable selection method yield the best statistics8 for both MDA and logit. Stepwise selection provides the highest mean, median and maximum accuracy. While hybrid selection has lower risk with the highest minimum accuracy and lowest variance of all observations. Stepwise selection significantly outperforms factor analysis at a 1% confidence level for both MDA and logit. Li & Sun (2011) provide similar results however the difference between stepwise and factor analysis for logit is only significant at the 10% level. In addition, stepwise-MDA significantly outperform DT-MDA at the 5% significance levels while hybrid-MDA yield significantly higher mean prediction accuracy then both FA-MDA (1% sig.) and DT-MDA (10% sig.). The hybrid method of variable selection also lead to significant (p-value <1%) higher mean accuracy compared to factor analysis for the logit model. Brezigar-Masten & Masten (2012) report accuracies for hybrid-logit > DT-logit > stepwise logit. In line with their results hybrid-logit outperforms DT-logit at a 10% significance level in this study. On the other hand, Appendix E, table 16

shows that stepwise logit significantly outperforms DT-logit at the 10% significance level. No significant difference is observed between hybrid-logit and stepwise-logit. However, with only one observation of the predictive accuracy of each model, the differences reported of Brezigar-Masten & Masten (2012) are unlikely to be statistical significant.

Table 5

Results of the prediction accuracy (%) of the testing sample for each model using a 50% healthy - 50% bankrupt matched sample and five random 70-30 splits between training and testing sample respectively. Panel A, B, C, D and

E show the results for 1, 2, 3, 4 and 5 year prior respectively.

Model MDA Logit CART

Method Stepwise FA DT Hybrid Stepwise FA DT Hybrid FA DT

Panel A: One year prior

Mean 76.44† 68.62 70.30 74.22 79.22‡ 69.70 73.42 77.94 71.76 71.84† Median 77.30† 68.90 71.80 74.30 79.50‡ 70.20 72.30 77.30 71.80 72.30† Minimum 70.20 63.60 64.40 71.80† 72.30 63.60 65.90 74.40‡ 60.00 65.90† Maximum 80.00† 71.80 77.10 77.80 87.20‡ 74.40 80.00 82.20 83.00† 80.00 Variance 13.60 9.47 26.66 5.27‡ 29.19 15.77 29.32 8.09† 80.23 33.64† Panel B: Two years prior

Mean 71.82 72.32‡ 62.56 64.62 69.28 70.84† 62.92 64.34 66.90† 62.62 Median 75.00‡ 72.90 58.60 65.20 68.80 70.70† 58.60 62.10 66.70† 58.60 Minimum 62.10 66.70‡ 56.90 55.20 59.10 60.60† 56.90 55.20 60.40† 56.90 Maximum 77.60 78.90‡ 75.40 73.70 75.90 78.90‡ 77.20 78.90‡ 71.20 77.20† Variance 39.69 22.35† 57.95 62.11 47.37† 47.63 70.16 86.30 19.30‡ 70.10 Panel C: Three years prior

Mean 54.48 58.00‡ 54.20 54.76 52.66 53.98 54.02 56.02† 55.58† 54.58 Median 54.40 58.10‡ 52.40 56.30 52.10 52.60 52.40 56.30† 54.00 54.40† Minimum 47.60 49.20 50.70† 48.50 45.60 44.40 50.70† 48.50 50.90‡ 49.20 Maximum 60.80 64.80‡ 59.60 61.40 61.90 64.80‡ 61.40 63.20 62.00 63.20† Variance 22.72 35.44 14.68‡ 26.37 53.74 60.89 18.62† 29.15 19.79† 29.75 Panel D: Four years prior

Mean 66.32 65.40 65.24 67.42‡ 66.68† 63.84 65.80 65.02 65.40† 63.62 Median 68.90‡ 68.10 63.10 67.20 67.70† 64.80 63.90 65.30 66.70† 63.80 Minimum 56.90 52.50 60.30 63.10‡ 60.30 52.50 60.30 61.50† 60.00† 60.00† Maximum 70.80 72.40 76.10‡ 72.20 70.80 70.70 76.10‡ 69.00 67.60† 65.60 Variance 33.77 59.38 41.05 11.84† 16.32 48.82 36.82 10.40† 9.74 4.98‡ Panel E: Five years prior

Mean 59.12 57.86 61.66 62.90† 59.06 56.08 61.66 64.78† 65.82‡ 63.08 Median 58.90 57.70 63.40 64.70† 57.40 54.40 63.40† 63.20 66.20‡ 63.40 Minimum 55.90† 53.60 50.00 51.80 54.90 51.80 50.00 57.10† 58.90‡ 58.90‡ Maximum 64.80 61.80 71.00 72.60† 66.20 62.90 71.00 72.60† 74.20‡ 67.70 Variance 12.74‡ 13.57 72.56 57.36 18.82† 23.31 72.56 42.92 46.23 15.93† † Best statistic for within year prior and prediction model.

The different selection methods for CART do not differ substantially in mean predictive accuracy. Even though DT-CART provides four times the best statistic, it does not provide significantly better prediction accuracy. Both logit and CART outperform MDA for each variable selection method. The hypotheses are tested using the two-tail student t-test and the results are presented in Appendix E, table 17. At the 1% significance level stepwise-MDA and hybrid-logit yields significant different prediction accuracies compared to FA-MDA and FA-logit respectively, therefore rejecting both hypothesis 1a (i.e. predictive performance of stepwise-MDA and FA-MDA is not significantly different) and 2e (i.e. predictive performance of FA-logit and hybrid-logit is not significantly different). In addition using a 95% confidence interval stepwise-MDA and DT-MDA; hybrid-MDA and FA-MDA; and stepwise-logit and FA-logit differ significantly from each other, rejecting hypotheses 1b (i.e. predictive performance of stepwise-MDA and DT-MDA is not significantly different), 1e (i.e. predictive performance of FA-MDA and hybrid-MDA is not significantly different) and 2a (i.e. predictive performance of stepwise-logit and FA-logit is not significantly different).

The results for two year prior to the bankruptcy yield interesting outcomes as the superior models are different between one and two year prior. One year prior FA-MDA and FA-logit produced the worst statistics in every category but standard deviation. Contrarily, two years prior both FA models have the best test statistics in all but one category (median for MDA and standard deviation for logit where their stepwise equivalent outperformed them). Both stepwise-MDA and FA-MDA significantly outperformed DT-MDA (5% sig.) and hybrid-MDA (10% sig.). The only significant outperformance between the logit models is a 10% significant higher mean prediction accuracy of FA-logit over DT-logit. Both CART methods again show similar results even though FA-CART now provides all but one best statistic, where one year prior the roles are reversed. The difference in mean prediction accuracy is once again insignificant.

The hypotheses are again tested using the student t-test (see Appendix E, table 17 panel B). At the 5% significance level only FA-MDA provides significant different prediction accuracy compared to DT-MDA, allowing me to reject hypothesis 1d (i.e predictive performance of FA-MDA and DT-MDA is not significantly different) for two years prior. The other significant difference is between stepwise-MDA and DT-MDA, rejecting hypothesis 1b (i.e. predictive performance of stepwise-MDA and DT-MDA is not significantly different).

accuracy. To test whether these models provide significant better results compared a random guess, I perform the one-sample t-test. The test proves that none of the 10 models are at any reasonable significance level better in classifying healthy and bankrupt companies compared to a random guess. Even testing all 50 observations of all 10 models collectively, does not yield significantly higher prediction accuracy then a random guess. This result is not consistent with previous literature and should be checked whether it is sample specific. A possible explanation is that it is period specific, considering that the bankruptcies in this sample are from 2010, 2011 and 2012, the models classify the companies using data from 2007, 2008 and 2009 respectively. This period represents the start and peak years of the financial turmoil in the United States and could explain some of the weak results of the models. However, this does not explain the higher accuracies for the surrounding years (i.e. two and four years prior).

The predictive performance of the models over the longer term (i.e. 4 and 5 years prior) do improve considerably compared to three years prior. The hybrid and to a lesser extent the stepwise selection methods for MDA and logit provide superior test statistics for the longer prediction windows. In terms of significant outperformance only hybrid-logit outperforms FA-logit five years prior at the 5% significance level. In addition, hypotheses 2c and 2e can both be rejected for the five years prior subsample. FA-CART yields better statistics than DT-CART for both years prior.

Table 6 presents the results of the two-tailed student’s t-test comparing the mean predictive accuracies of the best performing selection methods for each of the models one year prior to bankruptcy. The only significant difference is between stepwise-logit and DT-CART at the 90% confidence interval and therefore rejecting hypothesis 4c (i.e. predictive performance of logit and CART using the most effective variable selection method is not significantly different). Mean accuracies of the dominant models for other years prior do not differ significantly.

Table 6

Statistical significances of the difference in prediction accuracy between the best performing selection methods of MDA, logit and CART one year prior to the bankruptcy using the two-tailed student’s t-test.

Stepwise-logit Decision tree-CART

Stepwise-MDA 0.373 0.180

Stepwise-logit - 0.071*

6 Conclusion

This study has tested the performance of utilizing four variable selection methods (stepwise, factor analysis, decision tree and hybrid) for three classification techniques (MDA, logit and CART) to predict bankruptcies of US listed firms between 2010 and 2012. By doing this, the study introduces the performance tests of decision tree-MDA and hybrid-MDA. In addition, all models are tested using variables of one to five year prior to the bankruptcy, while most previous literature is limited to testing the models using variables of one year prior to bankruptcy.

None of the classification techniques, variable selection methods or combinations of the two consistently outperforms the other methods for all five years prior. Stepwise-logit yields the highest mean predictive accuracy (79.22%) one year prior to bankruptcy. Stepwise also proved to be the best variable selection method for MDA (76.44% accuracy) one year prior. Factor analysis yields the best predictive accuracy for CART one year prior, while for two to five years prior decision tree is the best performing variable selection method of the two. However, both CART methods (i.e. factor analysis-CART and decision tree-CART) do not yield significantly different mean prediction accuracy for all five years prior. Two years prior to bankruptcy factor analysis-MDA (72.32%) and factor analysis-logit (70.84%) yield the highest mean accuracy within their respective classification technique. The classification accuracies of each model three years prior to bankruptcy are poor; none of the models significantly outperforms a random guess. None of the previous literature has encountered such a drop in predictive performance for all models, while still delivering reasonable performance four and five years prior to bankruptcy. Four years prior hybrid-MDA (67.42%) and stepwise-logit (66.68%) and five years prior hybrid-MDA (62.90%) and hybrid-logit (64.78%) yield the highest mean predictive accuracies.

Limitations

Using chapter 7 and 11 filings as a definition of a bankrupt company has some limitations. Balcaen & Ooghe (2006) and Haber (2005) point out the following definition problems: Companies can file for bankruptcy for strategic reasons (i.e. filed for bankruptcy but not bankrupt); failure may result in other juridical exits, such as mergers (not filed for bankruptcy but bankrupt). In addition, timing may be postponed due to rescue attempts. In addition, this study treats the cost of type I and type II errors as equal. However, type I and type II error costs in real-world application are not equal as it is more costly to predict a bankrupt company as healthy, than to predict a healthy company as bankrupt.

This study has a limited sample size due to insufficient data availability. In addition, only listed firms are included, while the majority of bankruptcies is non-listed. This limits the robustness of the results for use by practitioners.

Each model has only been tested five times. Due to the limited number of observations and high variation of each model’s predictive power, the differences between the models have somewhat limited significance within each group.

Further research

References

Altman, E., 1968. Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. The Journal of Finance 23, 589-609.

Altman, E., Frydman, H., Kao, D., 1985. Introducing recursive partitioning for financial classification: the case of financial distress. The Journal of Finance 40, 269-291.

Balcaen, S., Ooghe, H., 2006. 35 years of studies on business failure: an overview of the classic statistical methodologies and their related problems. The British Accounting Review 38, 63-93.

Beaver, W., 1996. Financial Ratios as Predictors of Failure. Journal of Accounting Research 4, 71-111.

Brezigar-Masten, A., Masten, I., 2012. CART-based selection of bankruptcy predictors for the logit model. Expert Systems with Applications 39, 10153-10159.

Casey, C., Bartczak, N., 1985. Using operating cash flow data to predict financial distress: some extensions. Journal of Accounting Research 23, 384-401.

Cestari, G., Risaliti, G., Pierotti, M., 2013. Bankruptcy prediction models: preliminary thoughts on the determination of parameters for the evaluation of effectiveness and efficiency. European Scientific Journal 9, 265-290.

Charitou, A., Neophytou, E., Charalambous, C., 2004. Predicting corporate failure: empirical evidence for the UK. European Accounting Review 13, 465-497.

Chen, M., 2011. Predicting corporate financial distress based on integration of decision tree classification and logistic regression. Expert Systems with Applications 38, 11261-11272.

Cho, S., Hong, H., Ha, B., 2010. A hybrid approach based on the combination of variable selection using decision trees and case-based reasoning using the mahalanobis distance: for bankruptcy prediction. Expert Systems with Applications 37, 3482-3488.

Dambolena, I., Khoury, S., 1980. Ratio stability and corporate failure. Journal of Finance 35, 1017-1026.

Dempster, A., Laird N., Rubin, D., 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society 39, 1-38.

Haber, J., 2005. Assessing how bankruptcy prediction models are evaluated. Journal of Business & Economics Research 3, 87-92.

Hoaglin, D., Iglewicz, B., 1987. Fine-tuning some resistant rules for outlier labelling. Journal of the American Statistical Association 82, 1147-1149.

Jackson, R., Wood, A., 2013. The performance of insolvency prediction and credit risk models in the UK: A comparative study. The British Accounting Review 45, 183-202.

Jo, H., Han, I., Lee, H., 1997. Bankruptcy prediction using case-based reasoning, neural networks, and discriminant analysis, Expert Systems with Applications 13, 97-108.

Kaya, M., Gurgen, F., Okay, N., 2008. An analysis of support vector machines for credit risk modeling. Applications of Data Mining in E-business and Finance 177, 25-33.

Keasey, K., McGuinness, P., Short, H., 1990. Multilogit approach to predicting corporate failure – further analysis and the issue of signal consistency. Omega International Journal of Management Science 18, 85-94.

Lacerda, A., Moro, R., 2008. Analysis of the predictors of default for Portuges firms. Banco de Portugal, 1-28.

Lee, T., Chui, C., Chou, Y., Lu, C., 2006. Mining the customer credit using classification and regression tree and multivariate adaptive regression splines. Computational Statistics & Data Analysis 50, 1113-1130.

Li, H., Sun, J., Wu, J., 2010. Predicting business failure using classification and regression tree: an empirical comparison with popular classical statistical methods and top classification mining methods. Expert Systems with Applications 37, 5895-5904.

Li, H., Sun, J., 2011. Empirical research of hybridizing principal component analysis with multivariate discriminant analysis and logistic regression for business failure prediction. Expert Systems with Applications 38, 6244-6253.

O’Gorman, T., Woolson, R., 1991. Variable selection to discriminate between two groups: stepwise logistic regression or stepwise discriminant analysis. The American Statistician 45, 187-193.

Ohlson, J., 1980. Financial ratios and the probabilistic prediction of bankruptcy. Journal of Accounting Research 18, 109-131.

Shin, K., Lee, T., Kim, H., 2005. An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications 28, 127–135.

Tennyson, B., Ingram, R., Dugan, M., 1990. Assesing the information content of narrative disclosures in explaining bankruptcy. Journal of Business Finance and Accounting 17, 391-410.

West, R., 1985. A factor-analytic approach to bank condition. Journal of Banking & Finance 9, 253-266.

West, R., 2000. Neural network credit scoring models. Computers & Operations Research 27, 1131-1152.

Appendix A. Descriptive statistics for each year prior subsample.

Table 7

Descriptive statistics of the healthy and bankrupt subsample for one year prior. Both the healthy and bankrupt subsample consist of 69 observations. Tests for equality for both the mean and variance between the healthy and bankrupt subsample for each variable

Healthy subsample Bankrupt subsample Test for Equality

Mean Median Standard Deviation

Percentiles Mean Median Standard Deviation

Percentiles Variance Mean

Variable 5 95 5 95 Sig. Sig.

Activity Ratios

Accounts Payable to Sales 0.107 0.080 0.079 0.020 0.250 0.133 0.110 0.089 0.020 0.250 .065 .069

Asset turnover 0.931 0.800 0.611 0.175 2.010 1.018 0.720 0.976 0.005 3.170 .000 .531

CAPEX to Sales 0.052 0.030 0.063 0.000 0.210 0.066 0.030 0.075 0.000 0.210 .036 .250

COGS to Sales 0.633 0.660 0.229 0.285 0.955 0.778 0.780 0.332 0.155 1.430 .051 .003

Equity Turnover 1.960 1.450 2.278 -0.810 7.110 0.129 -0.190 3.754 -4.420 7.110 .001 .001

Fixed Asset Turnover 9.172 5.370 10.188 0.295 30.670 9.502 2.960 11.639 0.020 30.670 .131 .860 Inventory turnover 19.104 10.780 17.011 2.660 48.370 19.826 15.228 15.881 0.639 48.370 .347 .797

R&D to Sales 0.098 0.036 0.130 0.000 0.458 0.158 0.077 0.181 -0.019 0.458 .000 .028

Working Capital Turnover 2.110 2.490 6.058 -9.720 13.490 1.096 -0.100 7.783 -12.640 17.580 .161 .395 Financial Structure Ratios

Book Value per Share 7.473 4.670 9.128 -0.195 30.790 -1.454 -0.130 6.831 -16.510 5.765 .002 .000

Current Debt Ratio 0.316 0.260 0.218 0.060 0.760 0.517 0.550 0.244 0.135 0.760 .037 .000

Current to total Assets 0.441 0.410 0.269 0.060 0.910 0.435 0.390 0.279 0.065 0.955 .411 .885

Debt Ratio 0.167 0.120 0.200 0.000 0.790 0.302 0.200 0.317 0.000 0.790 .000 .003

Debt to Equity 0.971 0.860 1.588 -2.210 4.280 -0.343 -1.520 2.809 -2.850 4.280 .000 .001

Equity to Fixed Assets 3.119 1.700 5.912 -11.590 15.330 -1.854 -0.310 5.838 -11.590 9.305 .814 .000

Fixed to Total Assets 0.286 0.170 0.265 0.010 0.880 0.330 0.230 0.300 0.005 0.885 .072 .365

Interest Coverage Ratio 0.872 0.400 11.255 -19.480 21.900 -2.384 -0.520 8.576 -19.480 13.485 .042 .058 Short-term to total debt 2.344 1.260 2.169 0.265 6.590 1.900 1.110 2.118 0.100 6.590 .531 .226

Table 7 continued

Healthy subsample Bankrupt subsample Test for Equality

Mean Median Standard Deviation

Percentiles Mean Median Standard Deviation

Percentiles Variance Mean

Variable 5 95 5 95 Sig. Sig.

Liquidity Ratios

Acid Test Ratio 1.375 1.040 1.230 0.025 4.660 0.597 0.370 0.754 0.000 2.190 .000 .000

Cash Coverage Ratio 2.046 0.710 14.711 -23.820 28.110 -1.782 0.080 10.388 -23.820 17.080 .006 .080

Cash to Total Assets 0.130 0.070 0.142 0.000 0.465 0.103 0.060 0.119 0.000 0.390 .044 .221

Cash flow from Operations to Sales 0.009 0.070 0.183 -0.320 0.230 -0.099 -0.070 0.196 -0.320 0.225 .471 .001 Cash flow to Debt Ratio 0.313 0.230 0.905 -1.370 1.760 -0.339 -0.100 0.566 -1.370 0.235 .004 .000 Cash flow to Long-term Debt Ratio 0.355 0.430 1.134 -1.440 1.930 -0.291 -0.090 0.941 -1.711 1.930 .076 .000 Cash flow to Short-term Debt Ratio 3.467 1.080 11.393 -21.760 25.070 -2.373 -0.190 8.409 -21.760 12.220 .049 .001

Cost of Debt 0.081 0.059 0.181 -0.244 0.376 0.170 0.146 0.150 -0.059 0.376 .371 .002

Current Ratio 2.028 1.730 1.714 0.060 6.300 0.941 0.630 1.072 0.040 3.375 .002 .000

Operating Cash flow Ratio 0.217 0.270 0.767 -1.355 1.695 -0.266 -0.070 0.621 -1.610 0.425 .243 .000

Quick Ratio 1.563 1.220 1.304 0.050 5.000 0.722 0.510 0.830 0.025 2.560 .000 .000

Working Capital to Total Assets 0.115 0.190 0.353 -0.730 0.640 -0.219 -0.120 0.407 -0.730 0.395 .026 .000 Profitability Ratios

Dividend Payout Ratio 0.028 0.000 0.104 -0.195 0.195 -0.008 0.000 0.028 -0.042 0.000 .000 .006

Earnings per Share 0.460 0.010 1.266 -1.640 3.325 -1.402 -1.160 1.312 -3.320 0.040 .089 .000

Gross Profit Margin 0.367 0.342 0.229 0.042 0.718 0.222 0.216 0.333 -0.432 0.848 .049 .003

Gross Return on Assets 0.346 0.295 0.263 0.016 1.084 0.239 0.174 0.418 -0.543 1.084 .005 .074

Net Profit Margin -0.081 0.006 0.203 -0.434 0.174 -0.274 -0.287 0.169 -0.434 0.009 .322 .000

Pretax Profit Margin -0.077 0.007 0.240 -0.509 0.270 -0.304 -0.286 0.205 -0.509 0.014 .870 .000