Not everything that counts, can be counted: Customer
value of IT at ABN AMRO
Author: Lisa Blauwendraat Student number: 11413883 Date final version: June 23, 2017 Study: MSc. in Business Administration – Digital Business track Institution: University of Amsterdam Supervisor: prof. dr. H. P. Borgman Second supervisor: dr. H. Heier
Statement of originality
This document is written by Student Lisa Blauwendraat who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
“Everyone designs who devices courses of action aimed at changing existing practices into preferred ones”
Abstract
As today’s data driven economy raises the question how data, analytics, tools, and people come together to delivering value while preventing incidents, the purpose of this article is to bridge the gap between IT and business and in specific show how operational intelligence and monitoring contribute to business value. Design research is carried out to create a proof of concept for the design of a business value dashboard encompassing IT and business metrics to create visibility and monitor business outcomes rather than being in control and monitoring IT outcomes only. It is found that IT-business alignment is essential and there a dashboard should contain both IT and business metrics for delivering a realistic analysis on customer satisfaction. A generic process and artifact are proposed and evaluated. This study is contributing as it provides knowledge and understanding through building and testing a business value dashboard in a real-life context at the ABN AMRO bank. Future research can be performed by testing the outcomes on different cases, in different industries in order to further improve and justify the design proposal. Through this research, managers are
provided a design concept for their dashboard in order to achieve business value visibility of IT run and predictive monitoring. The originality of the case study is seen in the proof of concept of a dashboard for business outcomes instead of IT outcomes as tested on real life cases with the AAB bank.
Table of content
ABSTRACT IV LIST OF FIGURES VII LIST OF TABLES VII ACRONYMS VII 1 INTRODUCTION 1 1.1 INTRODUCTION 1 1.2 ACADEMIC INSIGHTS/TRENDS 3 1.2.1 TECHNOLOGY INCIDENTS 3 1.2.2 SPENDING ON IT RUN 4 1.2.3 ADVANCED ANALYTICS AND MONITORING 5 1.3 ABN AMRO 7 1.4 RESEARCH OBJECTIVES 10 1.4.1 RESEARCH QUESTION 10 1.5 METHODOLOGY 111.5.1 DESIGN SCIENCE RESEARCH 11
1.5.2 PROCESS MODEL 11 1.6 ORIGINALITY / CONTRIBUTION 14 1.7 STRUCTURE OF THE RESEARCH 14 2 STATE-OF-THE-ART 16 2.1 BUSINESS VALUE 16 2.1.1 IT BUSINESS VALUE 16 2.1.2 IT-BUSINESS ALIGNMENT MATURITY 17 2.2 CURRENT STATE IT AND BUSINESS MONITORING 19 2.2.1 IT MEASURES FOR CONTINUOUS AVAILABILITY 19 2.2.2 BUSINESS MEASURES FOR CONTINUOUS AVAILABILITY 20 2.3 CURRENT STATE OF CUSTOMER MEASURES 21 2.3.1 SERVICE QUALITY VS. CUSTOMER SATISFACTION 21 2.3.2 CUSTOMER SATISFACTION 22 2.4 GAP ANALYSIS 23 2.5 REQUIREMENTS 24 2.6 PROCESS 25 3 DESIGN INGREDIENTS 26 3.1 REQUIREMENTS 26 3.2 PERFORMANCE MEASURES 28 3.3 CONNECTIONS 30 3.4 VISUALIZATION 30 3.5 INITIAL DASHBOARD DESIGN 31 4 DESIGN 32 4.1 DIGITAL PAYMENT SERVICE: IDEAL 32 4.2 DESIGN PROCESS 34
4.3 DASHBOARD DESIGN 35 5 EVALUATION 36 5.1 CASE: MI IDEAL 36 5.1.1 WHAT IT SAID 36 5.1.2 WHAT THE BUSINESS SAID 37 5.1.3 CUSTOMER IMPACT 37 5.1.4 IT-BUSINESS ALIGNMENT 38 5.2 DESIGN EVALUATION 38 6 CONCLUSION AND DISCUSSION 39 6.1 CONCLUSION 39 6.2 DISCUSSION 41 6.2.1 LIMITATIONS 41 6.2.2 MANAGERIAL IMPLICATIONS 42 6.2.3 FUTURE RESEARCH 42 APPENDICES 44 APPENDIX A 44 APPENDIX B 44 APPENDIX C 45 APPENDIX D 45 APPENDIX E 46 APPENDIX F 46 APPENDIX G 47 APPENDIX H 47 APPENDIX I 48 APPENDIX J 48 APPENDIX K 49 APPENDIX L 49 APPENDIX M 51 APPENDIX N 51 APPENDIX O 52 REFERENCES 53
List of Figures
FIGURE 1.1THE SPECTRUM OF BI TECHNOLOGIES ... 6
FIGURE 1.2AABCONTROL TOWER ... 8
FIGURE 1.3DESIGN SCIENCE RESEARCH PROCESS MODEL (DSRCYCLE). ... 12
FIGURE 2.1VALUE MEASUREMENTS ... 17
FIGURE 2.2CUSTOMER VALUE MODEL THROUGH IT-BUSINESS ALIGNMENT ... 24
FIGURE 2.3.PROCESS MODEL ... 25
FIGURE 3.1.CONNECTIONS ... 30
FIGURE.3.2.INITIAL PROPOSED DASHBOARD DESIGN ... 32
FIGURE 4.1 IDEAL DASHBOARD DESIGN ... 35
FIGURE 5.1MI IMPACT ON INTERNET ... 36
List of Tables
TABLE 1.1ABNAMROIT MANAGEMENT PROCESSES ... 9TABLE.3.1.IT METRICS ... 28
TABLE.3.2.BUSINESS METRICS ... 29
TABLE 3.3.CUSTOMER MEASURES ... 29
Acronyms
AAB ABN AMRO Bank
BDIM Business-Driven IT Management CES Customer Effort Score
CI Configuration Item CLF Closed Loop Feedback CSAT Customer Satisfaction Score
CMDB Configuration Management Database DSR Design Science Research
IT Information Technology
ITIL Information Technology Infrastructure Library KPI Key Performance Indicator
MI Major Incident
NPS Net Promoter Score
OGSM Objective, Goals, Strategies, Measures SAMM Strategic Alignment Maturity Model UI User Interface
Chapter 1
1 Introduction
1.1 Introduction
Consider your favorite band, Coldplay, has planned a concert tour. Tickets are known to be sold out in seconds, but you got yourself lucky. You are at the point of online payment, but then the transaction gets cancelled due to a system failure. This was the one critical moment you did not want this to happen.
For organizations, it is not the question what, whether or why, but how, to respond to thrive in today’s digital era. For two decades, Information Technology (IT) has been transforming from a back-office function into a service-oriented provider delivering business capabilities managed by both business and technology teams (Raj, Sepple, & Willcocks, 2011). Organizations are more transparent; a small failure can turn into a blast in an instant via social media channels (Maguire et al., 2012). Moreover, customer demands are increasing; desiring continuous availability of personalized services, everywhere, anytime, anyhow. Delivering on customer expectations made business-related IT issues critical components for organizations, requiring continuous effort (Hevner & Chatterjee, 2010). Firms are challenged to manage, measure, and control IT, and ensure its business value.
Organizations responding most effectively to the change will be those that connect the dots and use predictive metrics to excel on customer’s expectations while reducing critical IT incidents (Davenport, 2013). Greater volume and variety of data feed the IT operations analytics (Bhalla & Capelli, 2015). Gartner’s 2016 hype cycle (Schlegel, 2016) shows
predictive analytics to in the early mainstream phase and expected to offering high benefits within five years. Already in 2017, organizational profitability is expected to increase by 20%. Hence, performing data analyses and predictions enables companies to keep up with challenges.
As Internet Banking is one of the most important electronic services, the banking industry is one of the industries mainly influenced by the technological advances. (Ariff et al., 2012; Kirakosyan & Dănăiaţă, 2014). In turn, bank IT costs are higher (7.3% of revenue) than the industry average (3.7% of revenue) (Mai, 2012). To providing continuous availability of services, European banks invest 70% of IT spend on run-the-business, while 30% on change-the-business. However, the real value of run-the-business is hard to quantify, since customers simply expect services to be available. With the majority of investment spend on run, it is common sense that justification for these expenses is sought. As the largest bank in The Netherlands, ABN AMRO also recognizes this problem.
The above resonates with the problem identified at the ABN AMRO Bank (hereinafter: AAB) during a five-month internship. Driven by regulations, customer demands, and digitization, AAB seeks justification of its IT investments. AAB wants to know the customer value of its IT run. As the Chief of IT services mentions: “IT run makes it enjoyable for the customer and it is essential to work”. Accordingly, AAB invested in monitoring tools, with the strategic goals of: achieving efficiency (reducing costs) while maintaining, or increasing, customer satisfaction. However, analytics stops at IT and IT within the bank in locked (J. Ruijs, personal communication, March 9, 2017). Visibility is desired to justify IT spend, while controlling for customer satisfaction through ‘invisible’ service operations. The desired state is used guided analytics and a shorter connection with the customer. The current set of tooling is however fragmented and specialized, and absent and end-to-end view of the IT landscape
disrupts obtaining value and reaching the full potential. Moreover, IT and business are not sufficiently aligned and common metrics and communications lack.
This research takes on the challenge and opportunity for measuring business value of IT run via analytical monitoring. In addition, the relationship between IT services and customer satisfaction needs to be defined as this is not generically determined yet.
1.2 Academic insights/trends
As there is extensive literature available on the relevant topics for the research, the references applied for this research are only indicative rather than descriptive. This section provides trends and academic insights to frame the relevance of the situation.
1.2.1 Technology incidents
Approximately 36% of IT incidents affect the availability or performance of key IT services (Shefford, Holland, & Nagaraj, 2017). In addition, quality or security of services can be affected. Technology incidents impact a business directly (financial) and indirectly (non-financial). Direct impact relates costs for repairing and missed revenues due to failed transactions. Moreover, service level or compliance requirements might not be met, leading to penalties, or worst-case scenario, losing the license to operate. Indirect impact relates to damage on brand reputation, service quality, and customer satisfaction. Based on the number of customers affected, incidents can lead to Major Incidents (MI) with critical business impact. MIs are defined according the priority, which is based on impact and urgency of resolving.
Of ten industries analyzed in 2016, financial services is the seventh most affected industry by incidents (Shefford, Holland, & Nagaraj, 2017). Of these incidents, almost half are caused by system failures. For this matter, root cause analysis and proactive performance are essential to mitigate (future) impact. Hence, there is need for innovative monitoring and
internal control to reduce the risk. In particular the banking sector has higher risks than other sectors, as it relies heavily on vendors and typically has ineffective service management and delivery. As covered below, business analytics offers capabilities to serve this purpose. Communication towards customers is also an important component to reducing the impact of incidents.
1.2.2 Spending on IT run
The recognition of the importance of dealing with IT incidents is reflected in IT investments. IT spend can be divided in run-the-business and change-the-business. Run-the-business, or run, creates value as it is supporting with a focus on efficiency and risk. It refers to non-discretionary (required) expenses to ensure continuous operations. It is less valuable and gains lower priority than change-the-business. Change, namely enables growth and transformation to enhance the organization. While change expenses can be justified and quantified through a business case, run expenses are more difficult to quantify, but as covered below, it is equally, or even more important. For this research, change-the business is out of scope.
Market research companies such as Forrester and Gartner reveal that run-the-business expenses can entail around 80% of total IT expenses. This rate is linked to effectiveness as increased productivity leads to lower costs per unit, indicating optimization of operations. Top-performers ‘only’ spend 55-65% on IT run. Research by Potter (2016) shows that merely 17% of IT organizations can justify their run expenses, mainly because there is no single way to link IT operations to business outcomes. In turn, IT departments, such as AAB, are in need of insights into the productivity of their IT run to increase the effectiveness, deliver business value and reducing costs. For long-term objectives, processes must be adopted to enable translating IT assets into IT service cost components. Subsequently, these can be converted into business capabilities and services.
1.2.3 Advanced analytics and monitoring
IT run spend shifts from IT infrastructure investments to business analytics and monitoring investments. From 2014 onwards, advanced analytics has been the fastest-growing segment as organizations continued to move beyond descriptive - backward-looking - and diagnostic analytics into predictive and prescriptive analytics. This growth was enabled by the improved availability of data, lower-cost compute processing – in the cloud – and high-impact proven use cases, such as a 25% reduction in service desk tickets related to issues (Khan, 2015; Schlegel, 2016). Advanced analytics provide new insights. Next to information, it gives real-time shortcuts to decisions and actions through knowledge extraction of different types of data. A data-driven approach and causality analyses enable to link firms' strategic goals to tactical policies and operational actions (Wang, 2016).
Predictive analytics is a form of advanced analytics that examines data or content with the goal to predict what is likely to happen (Schlegel, 2016). Its aim is ensuring optimal customer experience (Khan, 2015). It is forecasted that by 2020, predictive analytics will impact almost all aspect of everyday life, and raise 40% of a firm’s net new investment in business intelligence and analytics (Kart, Herschel, Linden, & Hare, 2016). Predictive maintenance is possible through using techniques, such as machine learning, artificial intelligence, data mining, and modeling. Figure 1.1 shows increased business value enabled by prediction can be seen. This research, can be placed as moving from monitoring to prediction on the figure.
Figure 1.1 The spectrum of BI technologies
Through predictive analytics organizations can make better decisions and act proactive rather than reactive. For banks in specific, the ability rises to better manage the costs of compliance and the risk of non-compliance. The full potential of analytics is however often delayed due to the difficulty of correctly implementing advanced analytics tools (Amakobe, 2015). Not only the difficulty of design, but also the IT infrastructure for performing analytics needs to be sufficiently supporting: an application is only as good as the IT infrastructure that supports it. This research does not go into further details on IT infrastructure. Moreover, further deploying predictive activities leads to prescriptive analytics, where in addition rules-based decisions are enabled. For this research, however, prescriptive analytics is out of scope.
Summary
Organizations desire insights in the value of their IT spend on run-the-business. Advanced analytics, through big data analyses, allow for new insights on operations and performance. Accordingly, the opportunity to better managing IT incidents and optimize run-the-business performance. It is important to recognize that analytics and managing incidents does not create business value, rather it provides information on and reduces the amount of business value lost.
1.3 ABN AMRO
The paragraph provides information on ABN AMRO Bank relevant for the research. In specific the IT services and service management is described as the research has been performed in this department from where contact with the business have been made for bridging the gap between IT and business. The relevance of value from IT monitoring is described as well.
Responding on the turbulent market, AAB is in a reorganization to become more agile and quicker in change implementations. With the transformation towards Agile (and ultimately DevOps) AAB aims to speeds up IT delivery through more than 300 agile teams. Moreover, heavy investments are made in automation for continuous delivery. The decentralization and automation need support of monitoring tools to stay in control. While the reorganization focuses on changing the bank, monitoring should facilitate run-the-bank.
While IT solution is responsible for change-the-bank, IT services is the department responsible for run-the-bank; all stakeholder rely on their services. Its focus is to improve quality and continuity of its services by enhancing predictiveness and make it easier and reduce the time to resolve. Focus is on tactical SIAM, as operational services rely at vendors, such as IBM. The five main goals of service management are:
1. From firefighting to fire prevention
2. When a crisis occurs, shorten time to resolve and assure the root cause is found (learning organization)
3. Know your IT assets so projects and maintenance will run smoother 4. Automate manual interventions & explore the unknown
For 2016 the IT run budget of AAB was € 436 million. As the budget for 2015 was exceeded due to FTE, change, and vendor expenses, the bank wants to be more in control. To monitor and excel in customer experience and satisfaction, AAB performs end-to-end availability management, entailing online near-real-time monitoring from a customer perspective.
The figure below illustrates the control tower as it is called within AAB.
Figure 1.2 AAB Control tower
IT service uses ITIL best practices for the design of IT management process within the organization. Relevant processes to be named are shown in Table 1.1.
IT service continuity management
Manages the recoverability and continuity of IT services, before any unavailability of IT systems, which can lead to unacceptable financial losses, image damage or to organizational problems. Helps Availability Management to ensure that proactive measures for higher uptime of services are implemented wherever it is cost-justifiable to do so
Availability management
Ensures that the level of availability delivered in all IT services meets the agreed availability needs and/or service level targets in a cost-effective and timely manner. Availability management is
concerned with meeting both the current and future availability needs of the business
Service Level management
Ensures that specific and measurable targets are developed for all IT services
Monitors and improve customer satisfaction with the quality of service delivered
Business Process management (BPM)
Chosen as governance model for the current ABN AMRO
organization. Within BPM, the business processes are incorporated as a central structure. It shows how the bank optimally can arrange its processes and at the same time can meet the organization’s objectives. Business Process Management in order to reach the targets for customer satisfaction and efficiency ratio
Table 1.1 ABN AMRO IT management processes
Business value for IT run is the value of the service for the bank at the moment the service is unavailable, also referred to as the economic damage quantified in Euros (G. Sluijs, personal communication, May 12, 2017). The bank is in need of stability of systems to better serve customers once they are in need of information or help (W. Adriaansen, personal communication, May 18, 2017). Moreover, the bank experiences a customer excellence journey. This entails a transformation of the complete organization, where the customer is placed central and the organization is organized in such a way to serve the customer at its best. Customer satisfaction has, however not been performing sufficient: In 2015, The NPS of AAB was, with -23, lowest compared to other Dutch Banks (Triodos 52, SNS -12, Rabobank -17)1.
Today NPS is -11, while the goal is 8. In Q1 2017, 69% of customer indicated to be able to do their banking anytime and anywhere, 9% experience unavailability and 21% was neutral.
While investing in monitoring, R. Schreurs (personal communication, April 14, 2017) describes it as “you don’t want to see just when there is unavailability, you want to see the
behavior of the services”. Analyzing behavior can deliver value through predictive measures.
The bank is in need of a relationship between run-the-business and customer satisfaction to justify their investment and act on customer needs. As M. Klauwie (personal communication,
March 9, 2017) frames to issue: Monitoring tools are like fire extinguishers; you do not know what they are worth, until they are necessary. This research aims to tackle that problem.
1.4 Research objectives
The research objective of this research is showing business and customer value of IT for run-the-business. A generic dashboard for monitoring IT business value is not readily available as there exists a knowledge in how the dashboard should be organized (Spivak & Naegle, 2016). Borgman, Heier, and Bahli (2012) confirm the need for a balanced dashboard where IT value is demonstrated in terms of the contribution to the business. The research sets first steps towards creating the process for designing a dashboard. A roadmap for success needs to be established that not solely focuses on building capabilities, but also continually demonstrates IT value (Duthoit, Baltassis, Saleh, & Sampieri, 2015). With the trend of advanced analytics, monitoring tools will be used to capture the value of IT run.
In particular, there is focus on IT incidents with customer impact that interrupt service availability and thereby value, both financial and non-financial value. The objective is to predict and ultimately solve UX problems before the business is impacted.
1.4.1 Research question
According to the research objectives defined, the following research question to be answered in this research is formulated:
How does a proof-of-concept dashboard design look like, contributing to showing the customer and business value of IT run through applying analytics and predictive monitoring?
1.5 Methodology
As mentioned, advanced analytics offers opportunities to handle complex business questions. This requires a methodology with the same capabilities. This section describes the research methodology applied and how it provides knowledge. Subsequently, the process model chosen is introduced.
1.5.1 Design Science Research
The business opportunity identified (i.e. the object of study) in this research is designing a monitoring tool for showing customer value of IT run. As such a tool is complex, artificial, and purposefully designed, Design Science Research (DSR) is performed. Moreover, this method selected it allows for changes through the act of building. Design involves creating an innovative artifact, not yet existing with the purpose of solving cases in “we don’t know how to do this yet” areas. Accordingly, DSR creates prescriptive knowledge and understanding of a problem by building and the application of a design artifact (Hevner, March, Park, & Ram, 2004). IS research applies DSR to improve effectiveness and utility of artifacts in solving real business problems. (Peffers, Tuunanen, Rotherberger, & Chatterjee, 2007). While DSR aims at utility of generic means-end-relations, behavioral research aims for truth; it describes, explains, and predicts effects of certain technology use.
1.5.2 Process model
Although several researchers proposed models, there is no commonly accepted procedure for performing DSR. The two essential components in the problem-solving process are building and evaluating. Simultaneously, the design requires relevance to the environment and rigor to the knowledge base (Hevner et al., 2004). Relevance refers to the applicability of the artifact
in the real-life context. This research ensures relevance through testing and face validity within the AAB itself. Rigor concerns about applying appropriate theories and adding value to the knowledge base. Chapter 2 covers rigor in this research. The type of knowledge contributed in this research is through improvement while answering the need for a more effective artifact (Gregor & Hevner, 2013). In specific, the Gregor and Hevner note that improvement entails a new solution for a known problem, where the application domain maturity is high, while the solution maturity is low. A rigorous design process contains multiple iterations. Therefore, this research performs two iterations.
According to the best fit, this research follows the Design Science Research Cycle of Vaishnavi and Kuechler (2004) as can be seen in Figure X.
Figure 1.3 Design Science Research Process Model (DSR Cycle).
*Circumscription is discovery of constraint knowledge about theories gained through detection and analysis of contradictions when things do not work according to theory.
The five steps of the model are:
1. Awareness of the Problem: here the problem is identified and defined. In section XX the challenge of AAB and its relevance in the environment is described. Derived from
this information the research question is formulated to be answered at the end of this DSR Cycle.
2. Suggestion: preliminary solution suggestion based on existing knowledge and theory. This is addressed according to the state-of-the-art of objectives. The knowledge and gap analysis serve as basis for a proposed model. Consequently, requirements for this model to work as a dashboard are described.
3. Development: building the proposed design. In this research, a dashboard for operational monitoring is designed in cooperation with AAB experts. Dashboards provide transparency and enable reflection on past, present and future outcomes of complex information in a minimum amount of time (Borgman et al., 2012; Yigitbasioglu & Velcu, 2012). Moreover, they offer an oversight of information understood by both IT and business functions. The design is affected by several forces (Borgman et al., 2012). In Chapter 3, the forces are described and solutions are created. Moreover, as AAB already adopts dashboards, the research builds on this initiative and make process in the application.
4. Evaluation: reflecting on and learning about the artifact. The essence of evaluation is reflected in its rigor as it demonstrates utility, quality, and efficacy of the relevant artifact (Hevner et al., 2004). Evaluation as continuous process ensures configurations by organizational use and perspectives (Lempinen, 2012). This research evaluates through interviews at the AAB and by the design in a real-life case. In specific, feasibility, validity, and reliability are tested.
5. Conclusion: summarizing on the artifact design and confirming design principles for creating artifacts in the same category. In this phase is reflected on the research question. Moreover, limitations and future research opportunities are given.
Due to the extent of research necessary for generating a generically applicable dashboard, this research is limited to providing the first steps for generic knowledge on a dashboard design. Although economic trends also impact customer satisfaction, this research limits to IT and business measures based on existing literature. This theoretical knowledge serves as basis for the requirements for the dashboard design. A gap analysis in the literature, consistent with the problem identified at AAB, serves as basis for the content design related to the objective.
1.6 Originality / Contribution
Since IT run consumes a significant part of resources, it is important to know its contributing value. In specific, the value creation of separate components is known, but aggregated value is lacking. The research aims to connect the components and proposes relationship, which are tested with real-life data. Therefore, the research contributes through establishing first steps towards the need of a generic dashboard. With the dashboard, the contribution of IT to high-level business outcomes can be obtained. Moreover, it serves as a common language such that value is understood across functions. Hence, a contribution is delivered to the prescriptive knowledge base on how to design and how to determine the design of a dashboard. Prescriptive knowledge comprises sets or rules for design and action. In addition, the opportunity is raised to test and refine also descriptive theory about the research topics through the design phase (Vaishnavi & Kuechler, 2004). In addition, the research is contributing to the ability to gain the potential of value creation through correctly applying advanced analytics.
1.7 Structure of the research
The thesis is structured as follows; Chapter 1 contains the problem formulation according to AAB’s business opportunity and the relevance of the problem. Afterwards, the research
question and research method are given. In Chapter 2 the current state of theoretical knowledge is provided. Subsequently a tentative design is proposed based on proposed relationship. Moreover, the requirements and process steps are described. The next chapter provides relevance and rigor of the model. Afterwards, chapter 4 contains the development of the initial dashboard based on validated model. Chapter 5 provides the evaluation of the artifact in a two iteration rounds. Face validity and case analysis are used. In the final chapter, the conclusion is derived and recommendations are listed. As it is the reflection phase, also limitations of the research are discussed.
Chapter 2
2 State-of-the-Art
In order to design a dashboard where IT and business deliver value and are linked to customer satisfaction, IT-business alignment theory requires maturity. This section describes the need for IT-business alignment and therefore the current state of the three components. Theoretical knowledge and models on IT and business measures is given. Moreover, customer satisfaction and relevant measures are defined. The current states reveal opportunities for improvement wherefore a model indicating interdependencies between the three components is proposed. Finally, requirements for the model are given.
2.1 Business value
2.1.1 IT Business Value
In economic terms, the business value of an investment (an asset) is measured as the net present value in delivers, terms of increased revenue, reduced costs, or both. In order to show intangible business benefits, incorporation in operational business processes is needed, that deliver tangible economic benefits. Accordingly, IT business value is hard to define as the impact of IT spending requires translation into benefits for the business and this often goas wrong (Borgman et al., 2012). IT contributes to value through providing business services and applications, which are built on technology and end-user services addressing the business needs (Tucker, 2016). What is Value, is however determined by the business, not by IT. The basis
for Run-the-business is availability, since when services are not available, no value can be created. Closely linked to business value is IT-business alignment.
2.1.2 IT-business alignment maturity
IT-business alignment shows the relationship between IT and business performance measures and is crucial for the value creation of IT while IT functions according to business demands. Through the Strategic Alignment Model, Henderson and Venkatraman (1993) show that, without strategic alignment, firms are not able to gain sufficient value from their IT investment. This is also confirmed for the banking industry (Dorociak, 2007). Based on the Strategic Alignment model, Luftman (2000) proposed a model to measuring the maturity of alignment: The Strategic Alignment Maturity Model (SAMM).
SAMM is contributing as it combines descriptive and prescriptive aspects of alignment (Luftman, Dorociak, Kempaiah, & Rigoni, 2008). Accordingly, an organization is enabled to evaluate and benchmark its activities and set actions for improvement towards higher maturity. Learning one’s maturity level requires input from both the business and IT. A higher maturity entails a two-way dimension where IT and business adapt their strategies together, aligning service management and support (Luftman, 2000). As most organizations are at maturity level 3, there is substantial opportunity for improvement (Brocke, vom, & Rosemann, 2015).
As can be seen in Appendix B. SAMM comprises five maturity levels, composed of six dimensions: 1. Communications, 2. Competency/Value Measurement,
3. Governance, 4. Partnership, 5. Scope & Architecture, 6. Skills. While all dimensions contribute to an organization’s maturity, in particular Value Measurements (see Figure 2.1) support the ability to demonstrate
IT business value via monitoring tools. High maturity involves a Figure 2.1 Value measurements
balanced dashboard demonstrating IT according to business needs. ITIL2 also confirms the needs for complementing technology metrics with non-technical metrics. However, the availability of balanced business and IT metrics is one of the weakest maturity factors for IT value (Luftman, Dorociak, Kampaiah, & Rigoni, 2008). Another outstanding and relevant dimension is IT governance. IT governance entails that IT is managed both for business outcomes and with IT control. Metrics need a redesign to improve visibility and transparency between IT and the business (Borgman et al., 2012).
Corresponding is the Business-Driven IT management (BDIM) (Moura, Sauvé, & Bartolini, 2007). For bridging the IT-business gap, BDIM contains a business process layer that aids in understanding how IT affects the business (See Appendix C). BDIM emphasizes the importance of business measures and evaluates interdependencies between business and IT on a strategic, operational, or tactical level. The objective functions are business metrics related to IT performance metrics. It has the aim to improve IT quality for better business results. Thereby, offering business value through customer satisfaction and productivity improvements. While BDIM is in need of proof-of-concept experiments for generalization, it can serve as basis in the research.
In addition to metrics, analytical and organizational capabilities determine the success of generating IT business value. Among others, this is described in the Business-Analytics Success Model (Seddon, Constantinidis, & Dod, 2012). This also related to the SAMM Communications dimension. For this research, these capabilities are left out of scope.
Summary
2 ITIL is the Information Technology Infrastructure Library, a framework with an internationally accepted set of
An essential business value of IT run is service availability. It often goes wrong once trying to translate IT services into business value. Therefore, the importance of IT-business alignment to deliver and measure value is highlighted. There is need balanced IT and business measures for high IT value maturity. Metrics and dashboards determining availability need to reflect the ambidexterity to perform well (Borgman et al., 2012). This research contributes to the redesign of metrics.
2.2 Current state IT and business monitoring
2.2.1 IT measures for continuous availability
IT services departments check service availability through proactively monitoring the back-end and the front-back-end of IT applications. Back-back-end encompasses the IT infrastructure that is composed of systems and servers. Technological development of cloud applications offers the opportunity to keep services running while the server is down, this is however not (yet) mainstream. Monitoring individual systems can entail analyses on storage use or capacity according to (adaptive) thresholds. Monitoring a service or application for availability requires an end-to-end view of the Configuration Items (CI) the service is composed of. Therefore, information on the specific CIs must be recorded in for example a Configuration Management Database (CMDB). Monitoring the entire chain of CIs determines the service health status. Monitoring is complemented with simulations. The frequency of such test rounds depends on the priority of the service. Monitoring the front-end entails monitoring the User Interface (UI). Incidents can be predicted based on thresholds for, for example latency, time outs, response time, number of transactions, or conversion.
Besides proactive actions, continuous availability also depends on an organization’s ability to quickly recover from incidents. This, in turn depends on how quickly an incident is
identified and acted upon (Herbane, Elliott, & Swartz, 2004). For this matter, run-the-business is limited to a focus on incidents with customer impact.
2.2.2 Business measures for continuous availability
Based on the above, true availability measures cannot solely depend on IT measures. According to industry-leading analysts, 74% of end-user problems are not detected by current IT infrastructure monitoring tools (Bovy, 2016). Business measures for service availability relate to customer touchpoints. One of these touchpoints is the Service Desk where calls are received. Measures the number of calls against (adaptive) thresholds offer information. Financial impact can directly by linked to costs per Service Desk call. Yet, Bovy (2016) indicates that only one of ten users actually call the Service Desk. Hence, the majority “suffers in silence” or reports via other channels. Other essential customer touchpoints are websites and Social Media platforms. In particular social media impacts the customer relationship as its different from traditional media and increasingly adopted. The platforms facilitate social actors to communicate via Internet, allowing for user-generated content, resembling dynamic, interconnected, and uncontrollable media outages (Peters, Chen, Kaplan, Ognibeni, & Pauwels, 2013). Informative metrics on content are essential to describe the dynamic phenomena through consistent interpretation. In addition, Peters et al. describe motives, network structures, and social roles & interactions, to be essential. Enables by technological advances and analytics, a recent measure is sentiment analysis, i.e. deriving consumers’ feelings based on their content.
Summary
IT and business measures are complementing for measuring value. A balance of metrics need to be in place to represent a realistic customer experience and be able to better act and react on incidents. No generic framework for determining balanced metrics is in place yet.
2.3 Current state of customer measures
Customer satisfaction is influenced by perception of service quality, however the two are different concepts. Therefore, the relationship and distinctions are described first. In addition, the link with availability and reliability is made. Thereafter, customer satisfaction measures are stipulated.
2.3.1 Service quality vs. customer satisfaction
Service quality and customer satisfaction are different in that service quality is more abstract and likely to be influenced by variables such as advertising and other forms of communication. Service quality is defined as to what extent the perceived service experience complies with customers’ expectations (Parasuraman, Zeithaml, & Berry, 1985). Customer satisfaction entails how a customer feels according to its interactions with an organization (Yoon, 2010). However, the two concepts are positively correlated (Sureshchandar, Rajendran, & Kamalanabhan, 2001). It is found that IT-based services, such as internet banking indirectly impact a customer’s perceived service quality and satisfaction (Zhu, Wymer, & Chen, 2002). In particular, perceived IT-based services affect service reliability, responsiveness, and assurance, which in turn affects customer satisfaction and service quality.
Several researchers found that reliability is the most important dimension for measuring quality and predicting customer satisfaction (Liao & Cheung, 2002; Parasuraman et al., 1988; Zeithaml, Parasuraman, & Malhotra, 1999). In addition to service availability, reliability incorporates the ability to perform the service dependably and accurately. Specifics metrics can be determined to measure these concepts and form the bridge to customer satisfaction.
Reliability, availability and serviceability (RAS) is a design philosophy first introduced by IBM in 1998 (Siewiorek & Swarz, 1998). It aims for ensuring high availability by building systems that are reliable and have high mean times between failures (Filks, Zaffos, Cox, & Rao, 2016). While the original RAS applies for IT architecture, in this research IT and business measures can be determined according to the idea for linking to quality of service and customer satisfaction.
2.3.2 Customer satisfaction
Several factors in play when measuring customer satisfaction, such as speed, ease of use, security, design, information content, and customer support service. It is suggested to deploy different complementing customer satisfaction measures in order to better capture the complexity and reflect reality. Based on theoretical knowledge and relevance for the research, three measures are described: CSAT, CES, and NPS.
The Customer Satisfaction Score (CSAT) is the most standard and traditional measure where satisfaction is expressed on a scale, 1-3, 1-5 or 1-10, and presented as mean value or index value. It is useful to measure short-term satisfaction and focuses on specific interactions. The Customer Effort Score (CES) focuses on the effort it took to perform a service, not directly the satisfaction. The rates are measured either on a scale from 1 to 5, or according to agreement/disagreement questions. A low effort rate is preferred (Market Research, n.d.). CES is limited as it addresses service obstacles, but does not specify why one encounters the issues. Moreover, customers are found to be more likely to rate negative experiences, compared to positive ones.
Most recently, the Net Promotor Score was introduced and gained both popularity and criticism. Compared to CSAT, NPS focuses on long-term customer satisfaction and claims to measure customer loyalty and ultimately company growth. The latter relation is, however not
scientifically proved. Moreover, it is part of the Closed Loop Feedback (CLF) measure, which measures for the purposes of listening, learning, and acting. NPS measures on a scale from 0 to 10 how likely customers are to recommend the service/product/company to a friend3. Improving the NPS score entails dealing with dissatisfied customer – detractors - first. Like CES, NPS is also limited by generic questions. To gain specific improvement areas, follow-up questions must be formed. According to the FinTech 2017 Capgemini report (2017), the NPS of banks as is low: -15. Shown is that consumers mainly appreciate ease, reliability, and online and mobile services of banks.
In particular, research shows that CES and NPS are complementary as they reveal correlations. However, each measures different aspects, for what it is recommended to use both. NPS is recognized in particular as part of the CLF.
Summary
Availability and reliability measures form the link to service quality and customer satisfaction. Customer satisfaction is best comprised of several measures, to manage external factors and variables which also affect customer experience.
2.4 Gap analysis
While Luftman at al. (2008) already analyzed the need for balanced IT and business measures for demonstrating IT value maturity, this study investigates the extension to customer value. It is clear that tools integrating IT and business objectives can benefit user experience. The knowledge gap on how to organize a dashboard for business value needs to be addressed. Due to the variety of value determinations, there is no silver-bullet metric compilation ready to
addressing all requirements. Based on the theoretical knowledge and relationships described, the following model is created as basis for generating a generic dashboard. Through applying this model, IT service managers know where to focus their efforts to gain positive impact on the business.
Figure 2.2Customer value model through IT-business alignment
2.5 Requirements
Translating the model into a dashboard brings requirements with it.
One can assume that organizations today have technology and data in house to apply dashboards. Architecture should facilitate data availability. Next to that, Skills are crucial to select the appropriate metrics. These skills entail on one the hand, knowledge on the architecture and security domain, and on the other hand skills for dashboard design. Organizational implementation is required to ensuring appropriate use.
The main requirement addressed is IT-business alignment. Consequently, the IT and business department need to determine metrics in cooperation and with mutual understanding.
Availability is determined by both departments. Metrics should be selected with expert knowledge to ensure validity.
2.6 Process
For the design product, a design process is needed to ensure that the wheel does not need to be reinvented another time. The design process for this research can be seen in Figure 2.3. It is built on theoretical knowledge (i.e. Lempinen, 2012; Pauwels et al., 2009; Yigitbasioglu & Velcu, 2012), and trends and best practices are considered. As there is no generic format available, it is obvious that the process of designing a dashboard is difficult. Each dashboard serves unique strategic objectives in different forms. It is essential to frame a process that the designer can follow to design with minimum effort and in turn is enabled to simply alter metrics for a specified service. While the research aims designing a generic dashboard, it is limited to the point where service specific features are needed. Moreover, business goals change over time. Therefore, the design process should be open for improvements continually.
Figure 2.3. Process model
Demand
Target audience: what does the user
want (e.g.details), consider current knowledge Strategic objectives: know what to measure
Supply:
Ingredients
Determine services facilatating the objective to measure progress Determine what metrics (KPI's) andhow many (only include relevant and high priority metrics
Check KPI feasibility based on data
availability
Connections
Create the connections between
KPI's and structure the dashboard accordingly
Visualization
Create an efficient and effective presentation Functionality features: flexibility, interval frequency, option to drill-down, filter, compare, alert, exportVisual features: Colors, real-time,
3 Design ingredients
In this section, the proposed model in evaluated for applicability in a real-life context; the AAB environment. Enabling a quick design process requires for verification and feasibility of the ingredients. Therefore, based on theory, ITIL standards, best practices, and in cooperation with AAB and partners, below the ingredients are discussed and a selection based on priorities is made. Moreover, the relationships are tested within the environment. At final, the (improved) model is translated into an initial dashboard design as basis for the design process.
3.1 Requirements
The relation between IT and business is prevalent at AAB as incidents are created via an event from IT monitoring, or from a complaint from the business side. Regarding this matter, there is however no alignment as can be seen from IT and business incident and complaint comparisons. In Q1 2017, AAB experiences eight incidents with customer impact, causing 37 hours unavailability and more than 1,2 million customers were harmed. Apparent is that 75% of MIs are caused by changes, which can be prevented through monitoring. AAB estimates to save € 60,000 coordination costs for an averted MI (T. Tauber, personal communication, March 9, 2017).
While there is no alignment, AAB is performing activities to deliver value. Through its value chain, AAB wants to add value through effectiveness, greater flexibility, and increased quality, in specific where Detect to Correct focuses on run-the-business (See Appendix D for the Value Chain). Relevant criteria are cooperation of the business, and process and tooling
optimization (V. Verhaar, personal communication, April 19, 2017). Moreover, a solid knowledge base supplied by both IT and the business is essential to enable better customer service through recognizing and avoiding repeating incidents. Besides solving incidents, there is focus on outweighing the impact on the complete chain process and customer (R. Schreurs, personal communication, April 14, 2017).
AAB wants focus, insights, and grip (P. van Sprundel, personal communication, April 6, 2017). This is translated in the strategy as OGSM, standing for: Objectives (qualitative goal), Goals (quantitative goal), Strategies (strategies or initiatives), Measures (performance indicators). OGSM enables translating organizational goals from higher to lower levels and it ensures that everybody works within the same framework initiative and KPI related. As mentioned, the number of business services identified is an indication of maturity. AAB is deploying the CMDB for this matter.
Data reveals that there is also room for improvement regarding customer satisfaction. For example, transactional NPS over the last three quarters was on average -13 (while the goal for 2016 was +8). The importance, but also vagueness about the NPS is confirmed at the marketing department where they do not know the true value, but estimate one NPS point to be one million Euros (J. van de Ven, personal communication, April 12, 2017). With a reach of 32% people in the CLF, AAB scores well and this is a critical measure as specifications are given. As several customer satisfaction measures are proposed for best reflecting reality, these are analyzed at AAB for overlap. As expected, the measures are complementing and no contradictions are found. Surprisingly, while one event is commented via feedback tool (Usabilla), it is not via the other (CLF). It is important to analyze correlations from both IT, business, and customer measures, the get a full picture of the situation and verify the real impact of incidents.
3.2 Performance measures
Having good performance measures in place contributes to monitoring value delivery. OGSM facilitates the use of Key Performance Indicators (KPIs). Therefore, KPIs are the performance measures applied at in the research. KPIs facilitate measures, actions, automation, prediction, and relevance through presenting numbers. With absent best practices, AAB matches KPIs of ITIL to its value chain (R. Schreurs, personal communication, April 14, 2017). A KPI library is created to avoid different interpretations of the measures. AAB characterizes KPIs according to over-, and underperformance. In addition, the status ‘not measured’ is created to determine the maturity of measurement possibilities, which relates back to IT-business alignment (R. Schreurs,); more ‘not measured’, indicates lower maturity.
As mentioned, KPIs are determined based on AAB documentation and personal communication, AAB partners, ITIL standards, and best practices from literature. The resulting KPIs are displayed in tables 3.1., 3.2., and 3.3. below. For the purpose of measures on the dashboard, there is distinguished between predictive and real-time measures in the table.
METRICS
KPI Description Measurement Predictive / Real-time
Business-critical incidents
Incidents with customer impact # High priority incidents with high reputation and financial impact Real-time
MIs (Major
Incident) Special categorization of business-critical incidents
Per channel and per cause. See AAB measures for MIs in Appendix E (19 MIs
in Q1 2017) Real-time
Changes Changes or upgrades in processes # High risk changes Predictive
Health status IT component status CPU, Memory, Disk space Predictive Real-time /
Synthetic
transactions Simulate customer in application # Successful transaction Real-time UI (User
Interface) Performance measures of the interface Latency, time out, response time
Predictive / Real-time
FCR (First Call
Resolution) Success of incidents closed on time
# tickets resolved on first contact /
(#tickets created - # excludable tickets) Real-time
KPI Description Measurement Predictive / Real-time Complaints:
Service Desk calls
Service desk calls received and logged # incidents type complaint Real-time
Complaints: Social Media analysis
Sentiment analysis through social media
platforms Data mining, example words: iDEAL, error, unavailable, not working, slow Real-time
Complaints: Internet analysis
Collecting feedback from visitors on the website
Through a program based on data mining,
e.g. Usabilla Real-time
Conversion Threshold the number of transactions & of succeeded transactions on the website Real-time
Session replay Record & analyze customer journey from user perspective # failed sessions Real-time
Table.3.2. Business metrics
KPI Description Measurement Predictive / Real-time Availability % uptime and performance % Actual / Expected availability Real-time
MTTR Time to resolve incidents; service quality Av. Solving time Real-time
MTBF Time between incidents; reliability, and service quality Av. Time between incidents Real-time
CSAT Customer satisfaction Scale of percentage Real-time
CES Customer effort score Effort on a scale of 1-3 (lower is better) Real-time
NPS Likeliness of recommendation to others Quarterly Score = Promoters - Detractors Real-time
CLF Immediate feedback by phone after transaction occurs % of people reached Real-time
Table 3.3. Customer measures
In addition to information in the table, KPIs are checked on requirements: Feasibility is checked to verify deployment in the organization. Except for Session Replay and MTBF, all measures are available and hence feasible. MTBF is not yet measured, but MTTR are and therefore one can assume MTBF to be feasible as well. Some interesting example monitoring tooling displays at AAB are summed up:
- Incident monitoring in combination with FCR can be seen in Appendix F. - Appendix E shows a MI overview
- Appendix G shows a MTTR dashboard
- Appendix H shows an example of AAB website monitoring (comments are in Dutch) - Appendix I shows a sentiment analysis of Social Media channels with data mining
words (in Dutch). This dashboard also exists with comparisons with other large Dutch banks.
As mentioned, there is no silver bullet for measuring business performance or customer satisfaction. Hence, several KPIs are selected. Regarding business measures; Service Desk call, Internet website measures, and Social Media complaints respectively decrease in value as measurement, i.e. Service Desk call most valuable, while Social Media offer great volume, but less valuable content. Unfortunately, more customers turn self-service or complain via online channels (A. Bijkerk, personal communication, March 28, 2017).
With KPIs determined the challenge is, however to link them and create a common language.
3.3 Connections
Figure 3.1.Connections
3.4 Visualization
Visual design is essential for the design dashboard success. This paragraph discusses the basic features for dashboards for properly creating the proof-of-concept, based on AAB information and academic knowledge and whitepaper with best practices (Juice, 2015). Just as the design of the artifact, the dashboard lay-out needs to be flexible to change in accordance to specific user needs.
Availability
MI: Negative relation; see App E and J, week 11
Business: Availability complaints: see App. 3.7. (comments in Dutch)
Customer satisfaction: tool enabling business measures
In-depth study on visualization is, however out of scope for this research as focus of this research is on content and determining relationships. Further implementation and application is open for future research or resides within the focal organization.
One can distinguish between functional and visual features (Yigitbasioglu & Velcu, 2012). Functional features describe what the dashboard can do and it is essential that they fit the purpose to generate the right actions (Ghazisaeidi et al., 2015). Most important and basic functional features are:
- Drilling-down option for further details, this is the most important option to serve different layers and levels of detail
- Filtering - Exporting
- Real-time notifications or alerts to triggering action - Scenario analysis
- Frequency, which is based on priority. High priority services requirement (near) real-time monitoring
Visual features; At AAB, among others is advised for measuring performance to not look at a trend, but at a bar chart, ensuring to stay within a standard deviation, because that is what matters in the first place for monitoring (R. Schreurs, personal communication, April 14, 2017). Trend analyses cover another particular activity.
3.5 Initial dashboard design
Figure.3.2. Initial proposed dashboard design
4 Design
This section follows the design process as proposed in paragraph 2.6. and confirmed for relevance and rigor in the previous chapter. The goal of the design process is developing a dashboard design for measuring business value through customer satisfaction. The chosen business service is Internet Banking service iDEAL. A short introduction for iDEAL is given below. Afterwards, the process of design is followed to conclude with an iDEAL dashboard.
4.1 Digital payment service: iDEAL
iDEAL is an IT service offered by banks for online payment. In accordance with the digitization and increased customer demands iDEAL has great impact, is relevant, and experiences continuous growth. Besides customer needs, The Dutch Bank requires availability of 99.88%
in 2018. With availability of 99.75% in the first quarter of 2017, AAB is not reaching this threshold yet. In addition, iDEAL is in the top ten of AAB business services with open incidents. Reviewing most mentioned links over January 2017 shows that iDEAL status is mentioned most (Appendix 4.1.). The importance of Internet and mobile banking is also reflected in the NPS score, as it is the second-best scoring channel and above average of six channels measured (+15 in Q4 2016). Furthermore, 75% of customers had contact with the bank about incidents regarding iDEAL payments4, being the second ranked touchpoint with highest incidents. The NPS for iDEAL payments in 2016 was -7,5, while fore competitors 0. In turn, a Dutch website that reports on incidents (www.allestoringen.nl)5, shows that AAB’s main problems are caused by logging in (38%), iDEAL payments (38%), and money transfers (23%).
While the bank is not the only party involved in the service (next to the consumer, merchant, and issuer/acquirer), it is responsible once an error occurs. Therefore, it is essential to ensure availability. AAB’s scope for increasing availability is on its function as issuing bank; communicating with the customer and acquirer bank. It is important to make this notion that banks or not entirely responsible for the service.
As bank are not entirely responsible for the iDEAL service chain, they can only aid to increase iDEAL availability through monitoring on details and offering continuous service improvement through monitoring while mitigating impact. A monitoring dashboard design can apply in this need.
See Appendix L for more information on iDEAL.
4 AAB internal documentation: Report on customer measurement payments, October 2016 5 Allestoringen.nl/storing/abn-amro. Visited on May 29, 2017.
4.2 Design process
Demand
• Target audience: Both the IT and business department of AAB desire a dashboard for iDEAL (E. Den Hollander, personal communication, March 29, 2017; T. Tauber, personal communication, March 9, 2017). Therefore, the dashboard should account for the different current knowledge and understanding of IT and business. Moreover, the dashboard needs to function for both strategic and operational purposes. Hence, a distinction needs to be made in the level of detail, which is explained in the visualization phase. Details on the operational level are out of scope for this research.
• Strategic objective: IT serves the business; hence iDEAL’s strategic objective is defined by the business. The strategic objective for iDEAL is defined as:
Availability. It should be noted, however, that as the iDEAL service itself costs the AAB money per transaction, it does not lend itself for cost effectiveness directly. Effectiveness can, however be reached through quicker mitigating incident impact, such as quicker repair times. Further details on this notion are out scope for this research.
Supply: KPI selection
• As iDEAL’s strategic objective – availability – corresponds with the availability measure on the initial dashboard design, this design serves as solid basis. IT measures serve for both monitoring real-time and predictive, while the business monitors real-time complaints for mitigating negative impact. As IT and business KPIs complement each other, the dashboard should contribute to increasing availability and awareness.
Connections
• The relationships are based on the proposed model; regarding the strategic objective of Availability, KPIs are described to be related to availability
Visualization
• Regarding visualization, the initial dashboard design still has a broad focus as all relevant KPIs are incorporated. For an iDEAL dashboard with strategic focus on availability, the design can be simplified. Simplification contributes to a mutual understanding for both IT and the business. Through evaluation the dashboard in the next chapter, KPIs can be further prioritized and selected.
4.3 Dashboard design
Based on the process in section 4.2., a dashboard for iDEAL is designed as displayed below in Figure 4.1. The initially design dashboard encompassed more KPIs and more details, tending to an operational dashboard. The dashboard for iDEAL is strategic focused and therefore the simplest design is desired. Simplicity contributes to focus and oversight. For example, details on health score are not desired for strategic purposes. Furthermore, as the dashboard is for an IT and business audience, it essential that for both parties the metrics are relevant and understandable. As the strategic goal of this research is generating first steps for a proof-of-concept of a generic dashboard, the dashboard is not more specific or simplistic. In the next chapter the design is evaluated in detail.
5 Evaluation
For evaluating the iDEAL dashboard design, a case analysis is performed and face validity is collected. Incidents range from the lowest systems – back-end - until the customer interface – front-end. Information and data from a MI of iDEAL is analyzed in this section to gain knowledge and possibly improve the dashboard design.
5.1 Case: MI iDEAL
On Thursday January 26, 2017, a MI occurred at AAB (see the report in Appendix 5.1.). Between 04:30 a.m. and 8:50 a.m., Internet Banking, iDEAL, abnamro.nl, Access Online and more services of AAB were unavailable. At 7:00 a.m. the incident was publicly reported on the Dutch public website www.allestoringen.nl6. At 9:15 a.m. AAB claimed the incident to be
resolved. Approximately 250,000 customers where affected and around € 850,000 in iDEAL payments were lost. With more than 1,2 million views, the impact on Internet was intense, see Figure 5.1.
Figure 5.1 MI impact on Internet. Source: Coosto (M. Bresser, personal communication, May 10, 2017)
5.1.1 What IT said
The earliest signals for the incident were received at 02:43 a.m., nothing that two servers were down. The service continued till the service on the back-up server was also lost. While not deploying advanced analytics or predictive monitoring on this service yet, the data regarding
the incident has been progressed through monitoring tools for analysis. This analysis shows that, through correlations, the server error could have been detected at 1:11 a.m. (T. Tauber, personal communication, March 9, 2017). Thus, incident repairmen could have been 1.5 hours ahead, decreasing the impact. Derived can be that current monitoring is not sufficient as this overutilization of servers was not predicted.
5.1.2 What the business said
AAB IT measures claimed the incident to be resolved at 9:15 a.m. and at 9:20 a.m., AAB announced services to be available again via its social media channels (e.g. Twitter, Facebook). An independent payment website (www.pay.nl) announced iDEAL to transactions to be available again at 12:20 p.m. However, while less often, customers still experienced errors and unavailability until late in the evening, for example via Twitter at 23:23 p.m. (translated from Dutch): “@ABNAMRO, hi, for the umpteenth time error with ideal transfer. When is this
solved, now and forever?”. Other customer comments include ‘error like frequently’, ‘logging
in not possible’ and, ‘cannot transfer money’ (translated from Dutch, see appendix 5.2.). Activity and sentiment analyses performed on social media and the website show peak in activity from 6:15 – 10:00 a.m., but at 21:00 p.m., still there was (negative) activity (see appendix 5.3). Trending topics for the comments were: ‘website, app abn amro, big incident, abn amro hour-long incident, solution’. One can assume this related to the same incident.
5.1.3 Customer impact
There is no daily customer satisfaction measure, however based on the comments, one can derive the level of customer satisfaction. Reviewing comments for January 26, reveals that the majority of comments about iDEAL (also website, Internet -, and Mobile Banking) and negative.
