3.2 Applied Model: Company Case
3.2.4 Application and Results
After applying the agglomerative hierarchical clustering algorithm with using ward’s linkage method, it is also important to interpret the results and performance with experts as the clus-tering algorithms are considered unsupervised. The resulting dendrogram for the overall score can be seen in Figure 13.
As can be seen from the dendrogram, there are 3 clusters exist in subset 3 for the overall score.
Dendrograms for other topics can be seen in Appendix B. After analyzing the clusters, 3 dif-ferent clusters are named as fast learners, lagged fast learners and slow learners. Table 7 rep-resents how many suppliers are in green, red and blue clusters where their learning behaviors named as lagged fast learner suppliers, fast learner suppliers and slow learner suppliers re-spectively. It can be seen from Table 7 that, suppliers can have different learning behaviors in different topics. For example, a supplier can be a fast learner in the environment topic whereas she can be a slow learner in business ethics topics. It can be seen that fast learner suppliers can be considered as the majority in the topics and the overall sustainability score. Table 8
3.2 Applied Model: Company Case
Figure 13: Dendrogram as an output of hieararchical clustering for overall score Table 7: Amount of suppliers that have specific learning behavior in each topic
represents the average clustering features for different learning behaviors in the overall sus-tainability score. As the clustering features represent the improvement amount between the sequences, fast learner suppliers are expected to have the biggest improvement through the sequences. Findings support this point as they have 18%, 5% and 2% increase between the sequences respectively. After creating the dependent variable set for the classification phase, a model can be created to see the relationship between the dependent variables(learning be-haviors of suppliers) and the independent variables(supplier characteristics). As the labeled data set (< 30) is not enough for constructing a classification model, the amount of labeled data must be increased by including the subset 2 which includes sequence 3 suppliers. For that purpose, label spreading algorithm which is a semi-supervised method is used. Label spreading algorithm is an iterative algorithm where the labels of data set propagated to un-labelled points through the data-set (Zhou et al., 2004). The assumptions of semi-supervised learning problems are the assumption of consistency, which means: (1) points that nearby have the same class; and (2) points that are on the same structure most probably have the same. The two assumptions can be considered as valid as suppliers that are nearby in terms of clustering
Table 8: Amount of average sustaianability evolution of suppliers between each sequence
3.2 Applied Model: Company Case features are expected to have the same learning behavior. Therefore, label spreading algorithm can be used in the data-set on hand. For implementation, python package named ‘sklearn’ is used for label spreading algorithm. Details of the algorithm can be seen in Appendix C. The resulting data-set is also examined with experts to confirm the performance and accuracy of the algorithms, and misclassified suppliers are corrected by changing their learning behavior after considering the feedbacks of experts. After including subset 2 to the analysis, another learning behavior named ‘indifferent suppliers’ must be included as another learning behav-ior category in the data-set as there are suppliers that do not show any improvement through the sequences which were categorized as slow learners in hierarchical clustering algorithm.
The resulting labeled data-set for the overall sustainability score can be seen in Table 9. Sup-pliers that are fast learners can be considered as the majority for overall sustainability score, environment, health safety and human capital topic. After fast learner suppliers, depending on the topic, the next majority learning behavior is different per topic. For example, indifferent suppliers are the second majority learning behavior in environment topic whereas it is slow learner suppliers in health and safety topic.
Table 9: Amount of suppliers that have specific learning behavior in each topic after including subset 2
Table 10 demonstrates the effect of including subset 2 on sustainability evolution of each learn-ing behavior in overall score.After includlearn-ing the subset 2, the average of clusterlearn-ing features changes as well. There is another column that appeared in the table that shows indifferent suppliers. It can be seen that average improvement amounts of indifferent suppliers are too small compared to other learning behaviors and approaches to zero. It is expected as those suppliers do not pay attention to SSP and do not spend efforts towards increasing the sustain-ability scores. Including the subset into the data-set for the classification model also increases the data size of the model. The data size increases from 29 to 124 which can be considered as enough for constructing the classification model.
Table 10: Amount of average sustainability evolution of suppliers between each sequence after including subset 2
3.2 Applied Model: Company Case 3.2.5 Model for Classifying Learning Behaviors
The final classification model is constructed by selecting the best model according to data on hand. Detailed explanation for model selection process can be found in appendix A.2. There-fore, XGBoost model is chosen as the main classification model in this research question as it can understand complex non-linear relations and interpretability can be solved by using SHAP values. In addition to the classification model, independent variables and dependent variables are obtained for each topic and overall sustainability score. Independent variables listed in Table 6 are determined for subset 2 and 3. As the supplier characteristics are unique for suppliers, it is not necessary to check whether these characteristics change over different topics. Therefore, the same independent variable set is used for the classification model of each topic. As there are categorical variables in the data, the categories in these type of variables are converted to dummy variables by following the procedure mentioned in § 3.2.2. Furthermore, dependent variables for the classification model are revealed in § 3.2.4. Learning behaviors of suppliers are considered as dependent variables and each supplier has a learning behavior category for each topic and overall score. As there are more than two categories in the data for learning behavior, the classification problem is categorized as a multi-class classification model.
3.2.6 Application and Results of Classifying Learning Behaviors
While classifying the learning behaviors of suppliers, the aim is to find the relationship be-tween supplier characteristics with learning behaviors. Supplier characteristics that are used for this purpose can be seen in Table 6.
For the combination of subset 2 and 3 which are the labeled data-set in § 3.2.4, classification models are constructed for each topic and overall sustainability score. Therefore, there are 5 different models to classify the learning behavior of suppliers in SSP as there are also 4 different topics and overall sustainability score. Model performance is evaluated according to receiver operating characteristics area under curve(ROC - AUC) as it is a performance measure for classification models, it can be used for multiclass classification models and it can be also used when the data is imbalanced. Data is considered as imbalanced as there is the majority class as fast learner suppliers in the data. AUC - ROC is a performance measurement that is used for classification problems by using various thresholds for the classification. ROC refers to probability and AUC refers to separability of classes between each other. The measure shows the capability of the model about distinguishing between classes. Therefore, the logic is higher AUC-ROC means a better classification model.
As there are several problems with the data-set, the model performance can be influenced neg-atively because of those problems. As there are many variables, there can be different types of problems during the modeling phase such as multicollinearity between independent vari-ables and imbalanced data. In addition to that, there are different ratio varivari-ables on a different scale. That is why a standardization or normalization process should be conducted to have all
3.2 Applied Model: Company Case variables sharing the same scale. There may be also irrelevant features that do not provide any insights on the learning behavior of suppliers. Therefore, before providing the data-set into the model and constructing the model, those problems must be solved in order to obtain better performance. Although there are many variables in the data-set, only the most important vari-ables are analyzed. Here, the most important varivari-ables refer to the most influential varivari-ables on various learning behaviors. Some of the variables such as the initial score of suppliers’ is expected to be considered as important variables for most of the topics. For the other variables, the relationship must be carefully interpreted as it can be only a correlation but not causality.
Therefore, to confirm the effects of different supplier characteristics, expert knowledge must be utilized to have the proper motivation for the characteristics.There is a need for data prepro-cessing because of the aforementioned problems. The detailed explanation of methods used for data pre-processing phase can be seen in Appendix D.
Results: The model is run with non-correlated, standardized and eliminated independent vari-ables and dependent varivari-ables which are determined by using the clustering algorithm. Im-portant variables of models can be seen in Appendix E. Hyperparameter tuning for XGBoost is also performed for each model by using k-fold cross-validation. The details of hyperparam-eter tuning and its search space can be seen in Appendix K. The data is split into training and test data using 80%-20% ratio respectively. The model is for multi-class classification problem, that means there are 4 different categories for the dependent variables which are ‘fast learner suppliers’, ‘lagged fast learner suppliers’, ‘slow learner suppliers’, ‘indifferent suppliers’. The performance of the model can be seen in table in terms of ROC-AUC among topics and overall sustainability score. ROC-AUC is used because it is robust against imbalanced data-sets and it can also be used for multi-class classification models (Hand and Till, 2001). For the multi-class classification task, ROC-AUC for ‘one vs one’ and ‘one vs rest’ strategies are examined. Table 11 represents the performance of models for each topic.
Table 11: Performance of the classification model for each topic
Strategy ‘one vs one’ represents converting multi-class classification task into one binary clas-sification problem per class. On the other hand, strategy ‘one vs rest’ represents converting multi-class classification task into one binary classification problem per each pair of classes.
According to threshold values given by Metz (1978), the models for overall sustainability score, environment and business ethics are considered as ‘good’ as they have AUC values between 0.7-0.8. The models for health and safety and human capital are considered as ‘satisfactory’
as they have AUC values between 0.6-0.7. By looking at the AUC values, the models can be
3.3 Results considered to have prediction power as they have AUC values more than 0.5. Although the models are not perfect, they can be used in the prediction of suppliers’ learning behavior, and they can provide insights on the relationship between the learning behaviors and suppliers’
characteristics.
Furthermore, to see the performance difference, binary classification models are also tried to see if the model performance enhances in different topics. The binary classification model is obtained by creating 4 different models per each topic and overall sustainability score. For ex-ample, the first model considers the classification task between fast learners with other learning behaviors whereas the model with sustainability learning behaviors considers the classification between slow learners with other learning behaviors and so on. To compare the performance of models, the average AUC values are calculated for binary classification models per each topic and overall sustainability score. Table 12 represents a comparison of AUC values between binary and multi-class classification models.
Table 12: Comparison of performances between binary and multi-class models
One can look at Table 12 to compare model performances. This comparison demonstrates that the model with multi-class classification performs better than the model with binary classi-fication for each topic and overall sustainability score. Therefore, the detailed analysis to see which characteristics influence which learning behavior is performed with the multi-class clas-sification models.
3.3 Results
The models are obtained by eliminating some of the independent variables using the feature selection algorithm and using dependent variables as learning behaviors.For explaining the models and the influence of independent variables on the dependent variable, SHAP values are explained. It provides insights on the influence of every feature per predicted value. There-fore, the feature influence can be determined by using the aggregated influence of every feature in the prediction model on predicted values. The method is proposed by Lundberg et al. (2018), and it uses game theory as its basis to calculate SHAP values. SHAP values characterize the ef-fect of the independent variable when that independent variable is eliminated from the model.
According to the author, the model can be used for explaining machine learning algorithms which were considered as a black box, it also outperforms other algorithms that are used for explaining complex models. The detailed explanation and interpretation of SHAP values can
3.3 Results be found in Appendix F and SHAP summary plots for each topic and learning behavior can be found in Appendix G. As the number of models analyzed is related to the number of topics and overall sustainability score, there are 5 different analyses for interpreting important supplier characteristics for each topic. Detailed tables of the results can be found in Appendix H. The tables are constructed by analyzing the SHAP values for the relevant independent variable for the relevant model. Columns represent the independent variables; rows represent the learn-ing behaviors. The effects of independent variables on learnlearn-ing behaviors for a specific topic reported for each variable group. It should be noted that the interpreted results are the ones which are discussed and confirmed by experts at Philips. In Table 13, the relationship between activities performed by suppliers and its effects on learning behaviors can be found.
• Suppliers who perform sub-assembly activity are expected to be fast learner suppliers in health and safety topic. As the activity can be considered as a less risky activity, it might be easier for them to improve the issues that may arise.
• Suppliers who perform logistics and distribution activity are expected to be fast learner suppliers. Interviews with experts revealed that logistics and distribution suppliers are expected to care more about sustainability as there are already sustainability develop-ments in the industry such as ’green logistics’ and ’inverse logistics’. Therefore, they are expected to improve more and faster than other suppliers in SSP program. Those suppli-ers are expected to learn fast in environment, health and safety and human capital topic.
It makes sense as logistics and distribution activity is closely related to those topics.
• Suppliers who perform metal stamping activity are more likely to be indifferent in health and safety topic. The conclusion results from that the nature of metal stamping activ-ity is considered as risky. Consequently, those suppliers may take more time to solve health and safety issues. That is why they cannot show significant improvement amounts throughout the sequences.
• Suppliers who perform final assembly are expected to be fast learner suppliers in envi-ronment and business ethics topics. According to expert opinion, it can be because of final-assembly and sub-assembly is closely related and less risky activities. Therefore, applying improvement actions in terms of environment and business ethics topics can be easier and less time consuming for them. That is why they can improve faster than other suppliers that are not performing final assembly.
• Suppliers who perform clean room activity are considered as slow learners in health and safety and human capital topics. Interviews with experts revealed that suppliers that are performing clean room activity are considered as high mature suppliers. Therefore, the room for improvement might be less for them. That is why they are expected to be slow learner suppliers as there is not much improvement potential for those suppliers.
However, the relationship between performing clean room activity with business ethics topics could not be explained by experts.
3.3Results Table 13: Activities that affect learning behavior of suppliers in each topic
Class Subassembly Logistics and
Distribution
Metal Stamping Final Assembly Clean Room Plastic Molding
Lagged Fast Learner
Health & Safety (-) Business Ethics (-) Business Ethics (+)
Fast Learner Health & Safety (+) Environment (+) Business Ethics(+)
Health & Safety (+) Environment (+) Human Capital (-) Human Capital(+)
Slow Learner Human Capital (-) Environment(-) Health & Safety(+)
Human Capital (+)
Indifferent Environment (-) Health & Safety(+) Business Ethics(-)
(+) represents the positive relationship between activities and class whereas (-) represents the negative relationship.
Table 14: Facilities that affect learning behavior of suppliers in each topic
Class Dormitory Chemical
Warehouse
Kitchen Hospital
Lagged Fast Learner
Environment (+) Environment (+)
Health & Safety (+) Fast Learner Environment (-)
Health & Safety (-) Human Capital (-)
Slow Learner Health & Safety (+) Business Ethics (-) Environment (-) Human Capital(+)
Indifferent Business Ethics(+) Health & Safety (-) Environment(+) (+) represents the positive relationship between activities and class whereas (-) represents the negative relationship.
30
3.3 Results
• Suppliers that are performing plastic molding activity are less likely to be indifferent sup-pliers in business ethics topic. As the activity can be considered as more troublesome than other activities, the suppliers are closely monitored by regulations about issues related to business ethics according to experts. That is why they have to show improvement in order not to have problems with regulations. Therefore, those suppliers are not expected to be indifferent suppliers.
When the effects of the facilities of suppliers on learning behaviors interpreted, Table 14 is examined as,
• Suppliers who have dormitory on their site are less likely to be fast learner suppliers on environment, human capital and health and safety topics. According to experts, having a dormitory can be a sign of having more people in the workforce and its labor-intensive.
That is why it results in additional issues in terms of environment, health and safety and human capital topics. As a consequence, those suppliers evolve less than other suppliers that do not have dormitories in their sites.
• Suppliers who have a chemical warehouse on their site are more likely to be indifferent suppliers on business ethics topic. It can result from that managing a chemical ware-house may not be easy in terms of business ethics. Therefore, it may take more time to improve issues that arise from having a chemical warehouse. Consequently, those sup-pliers cannot obtain significant improvement amounts during the process.
• Suppliers who have a kitchen in their sites are expected to be lagged fast learners on en-vironment and health and safety topics. The reason might be that having a kitchen might result in several environmental and health and safety issues that can be solved easily.
However, a delay can be expected to see the effects and implementation of solutions.
That is why they are classified as lagged fast learner suppliers if they have a kitchen in
That is why they are classified as lagged fast learner suppliers if they have a kitchen in