Figure 1.9: Overview of the analysis approach
• What activities does the FSE perform in the eld during diagnosis? Does he/
she perform the mandatory activities in a corrective maintenance case?
The work ow consists of the activities that are performed by the FSE during diagnosis. The activities that an FSE performs to diagnose the cause of the system failure can vary for each case, but once the cause is found, the engineer has to follow certain procedures to change one or more parts in the system.
In this case, there are mandatory activities that need to be performed and it is important to track if these activities are indeed being performed.
• On which activities does he/she spend more time?
The analysis focuses on retrieving performance information about FSEs actions in order to better understand the work of FSEs: if there are bottlenecks in the process, identifying what activities take more time and need to be optimized, etc.
• Is there an in uence of version of the system or geographical regions on the work ow of the FSE; in other words, do we see a dierence between the
work- ows of FSEs mined on dierent versions or geographical regions?
The Allura Xper systems are constantly improved and new versions of software are updated for these systems. It is expected that with newer versions bugs are xed and the systems are improved. One of the goals of the research is to understand if there is an in uence of these changes on the work of the FSE.
Another aspect is that, being spread globally, the systems are maintained by dierent engineers coming from dierent cultures, who took part in dierent training programs, etc. Considering event logs from systems in dierent ge-ographical regions will help in understanding whether there is an in uence of geography on the work of FSE. Mining work ows from dierent perspec-tives, i.e., system version, geographical region, can help understand the main dierences on the way of work of the FSE.
The answers to the research questions are meant to help understand the work of FSE in order to oer better support to their work, to reduce the time to repair of systems and to avoid false replacements that end up in customer dissatisfaction.
1.6 Research Methodology
To give an answer to the research questions, the following steps are taken:
1. Conduct a case study on event logs taken from Philips X-Ray systems, using existing process mining techniques
1.7. Thesis Overview 17 2. Identify limitations of existing techniques and challenges of mining/analyzing
event logs
3. Develop an approach for discovering and conducting analysis on the work ow of FSE
(a) Mine interactive hierarchical work ows of FSE with potential incorpora-tion of domain knowledge
(b) Analyze the work ow of FSE to answer the research questions (i.e. is there any deviation in the work ow of the FSE considering dierent per-spectives?)
4. Develop a technique for estimating and projecting performance information onto hierarchical process models
(a) Develop an algorithm to replay the event logs onto hierarchical process models
(b) De ne and compute performance metrics
(c) Visualize and project performance metrics on the process models 5. Implement the approach as a new Plugin in ProM framework [2]
6. Evaluate the implemented methods using real life event logs
1.7 Thesis Overview
For supporting this methodology implementations were realized using the ProM framework.
The ProM framework is developed by the process mining group and is a generic framework designed to support process mining. It consists of a range of plugins cor-responding to dierent mining and analysis techniques. For this thesis, a new plugin is developed to annotate process maps expressed as fuzzy maps with performance metrics. The plugin takes as input:
• A hierarchical process model: Hierarchical process models are meant to repre-sent complex processes by abstracting activities to higher levels. The hierar-chical process models are represented by process maps, constructed using the Fuzzy miner [12], [11] based on the technique proposed in [16]. In the context of Philips, these process maps represent the work ow of the FSE.
• A given event log: In Philips context, an event log representing the activities performed by the FSE.
The output of the plugin is a process map annotated with performance information obtained by replaying the event log onto the process map.
1.8 Outline
The remainder of this thesis is structured as follows:
In Chapter 2, we provide the preliminary study case conducted in Philips Healthcare
Chapter 2
Analysis of Field Service Engineer Process Using Existing Process
Mining Techniques
This chapter provides the details of preliminary analysis of the event logs and the observations at Philips Healthcare. Section 2.1 provides an introduction to process mining as an approach to gain insights into processes by analyzing event logs. Sec-tion 2.4 provides a descripSec-tion of the data collecSec-tion phase. SecSec-tion 2.5 presents the pre-processing of logs with a focus on converting and preparing the data for the MXML (Mining XML) format needed by the ProM framework and the correspond-ing ProMImport plugin. The results of the analysis are presented in Section 2.6.
The identi ed problems are presented in Section 2.7.
2.1 Process Mining
Process mining combines process modelling and analysis techniques with data min-ing and machine learnmin-ing. In the context of Work ow Management (WFM) and Business Process Management (BPM), it can help in the diagnosis of business pro-cesses; in the system test area it can be applied to extract test scenarios. Process mining techniques aim at extracting dierent kinds of knowledge based on the event logs. Event logs from dierent environments are dierent in nature but they have one thing in common: they show occurrence of events at speci c moments in time, where each event refers to a speci c process and an instance, i.e. a case [9].
The goal of process mining is to exploit the information recorded in the event logs by using it for a series of analysis techniques. Typically these analysis techniques assume that it is possible to sequentially record events.
In process mining there are three perspectives from which process analysis can be performed: the process perspective, the case perspective and the organizational per-spective [9].
• Process Perspective: The process perspective focuses on the control- ow, i.e., the ordering of activities. The goal of mining this perspective is to nd a good
19
The goal is to either structure the organization by classifying people in terms of roles and organizational units or to show the relation between individual performers.
Figure 2.1: Overview of Process Mining
Figure 2.1 gives an overview of the process mining domain. Depending on the desired outcome of the analysis, there are three basic types of process mining [25]:
• Process discovery: derives information about the original process model, the organizational context, and execution properties from enactment logs. An example of a technique addressing the control ow perspective is the α - al-gorithm [25], which constructs a Petri net model describing the behaviour observed in the event log. In the case of process discovery there is no a priori model, the model will be constructed based on the event log. The process discovery technique used in this chapter is the Fuzzy Miner [12], [11].
• Conformance: compares an a-priori model with the observed behaviour as recorded in the log, i.e. reality is compared with process models or business rules and deviations are detected. Conformance checking based on Linear Temporal Logic (LTL) [22] can be used to detect the deviations, to locate, explain, and to measure the severity of these deviations/violations [19].
2.2. ProM Framework 21
• Extensions: involve extending an a-priori model with additional information that is extracted from the log such as information about timing, resources, de-cisions, etc. An example is the extension of a process model with performance related metrics, i.e. projecting information about bottlenecks in a process on an a-priori process model [19]. Also, there exist approaches that help visual-izing performance information [10].
The environment that includes these process mining techniques and the one that is used in this master's thesis is the ProM framework [1], [2].
2.2 ProM Framework
ProM is an extensible framework that supports a wide variety of process mining techniques in the form of plugins. The latest stable version of the ProM frame-work is ProM6 [2]. Currently it has more than 170 plugins and supports discovery and analysis plugins, such as: plugins supporting control- ow mining techniques, analysing the organizational perspective, mining less-structured exible processes, veri cation of process models, veri cation of Linear Temporal Logic formulas on log les and performance analysis techniques. Besides these techniques, the current version of ProM also supports the implementation of other related subjects such as model conversion and log lters for cleaning event logs.
The ProM framework enables functionalities to be added or removed easily in the form of a plugins. The ProM architecture can be described as depicted in Figure 2.2. The Object Pool component consists of a repository of all objects that are either output or input of the plugins, e.g., various process models, event logs, etc. Connec-tion between these objects are also stored in the repository. The list of objects can be visualized using the User Interface component, limited to the Visible Objects.
Hidden Objects, such as semantics, are not shown by the User Interface component.
Elements of the Plugins component are all ProM plugins, e.g., mining, analysis, etc.
Each of these plugins has its own interface in the form of input and output param-eters. Process mining techniques, like Dotted Chart Analysis [20], Conformance Checking [22], Pattern Abstractions [6], available in the ProM framework, are going to be used for the analysis in this Chapter.
2.3 Process mining in the context of Philips
In the context of Philips, all three basic types of process mining presented before are used for the analysis conducted on the event logs. The problems addressed by the analysis are based on the following research questions presented in Chapter 1: What are the activities that the FSE perform in the eld during diagnosis?,
Does the FSE perform the mandatory activities in a corrective maintenance case?
and What is the work ow of the FSE?. While trying to provide answers to these questions, we have to keep in mind the identi ed problems: low level granularity of event logs and activities considered in isolation. The main goal of the analysis is to discover the work ow of the FSE during a case of corrective maintenance on
Figure 2.2: The ProM framework architecture
an Allura X-Ray system. To mine the control- ow perspective describing the FSE work, the Fuzzy Miner2 is chosen. Using this mining technique a process model is constructed that describes the behaviour of the FSE as observed from the event logs, the series of activities that he/she performs in the eld during diagnosis. To give an answer to the research question Does the FSE perform the mandatory activities in a corrective maintenance case? conformance-checking based on linear temporal logic is used to detect the deviations from the prede ned mandatory activities that an FSE has to perform after replacing an FRU. To get additional insights about the process of the FSE and get a helicopter view of the process, the Dotted Chart Analysis [20] is used. This discovery technique emphasizes the time dimension, helps in discovering performance issues and puts no requirement on the structure of the process. Based on the observations, the main issues discovered in the event logs will be underlined along with the possible solutions considering existing mining techniques and possible improvements.
2.4 Data collection
As mentioned in Chapter 1, on the RADAR system the event logs are stored in an internal database in XML format. These event logs are the main input for the analysis conducted in this chapter.
2A modelling formalism that can provide a highly simpli ed and abstract view on processes [12], [11]
2.4. Data collection 23
Figure 2.3: UML model of XML log le
2.4.1 XML Log Data
Each system that is connected to the PRS system generates log les that record every operation done on the system on a daily basis. These systems are identi ed by an unique identi er. A Log.XML le for each day of the system under operation is stored. Figure 2.3 represents the UML class diagram of XML log les [21].
Each system can create one or more XML log les, which in turn consists of one or more LogEntry XML elements. Each log le has the same XML data structure.
This is shown in the UML model as the classes System, XMLFile, LogEntry, and their relationships.
The elements contained in a LogEntry are explained in Table 2.1.
The LogEntry element can be classi ed into three types:
• User Message: The entry will be logged when a message is shown to the end user on the screen, e.g, Geometry restarting.
• Command: The entry will be logged when a command is invoked in the sys-tem, either by the system itself or by the user. A command always has a Name attribute which represents the corresponding name of the given com-mand. In some cases, a command can also have a Params attribute which gives more information about the logged command, i.e., the voltage values, or even requested, completed type of commands.
• Warning & Errors: These entries are logged when the systems observe devi-ations from the expected behaviour and it reports warnings, usually followed
Elements of Log Entry
Description
Index It refers to the index of the log entry. For each log le, the index starts from 1.
Unit According to the system architecture, the CV system consists of several subsystems, and these subsystems comprise of units, like Session Manager, Reviewing, X-Ray control etc. The value for this element corresponds to the unit that generated this log entry.
Date On which date the message was recorded by the logger.
Time At what time the message was recorded by the logger.
Severity The severity of a log entry. It can either be information, error, or warning. Fatal is another kind of severity though uncommon.
EventID It uniquely identi es each message generated from a particular system unit.
Description Description (information) of the event.
Memo More detailed information about the event is included. Sup-pose a log entry description eld refers to the start of some ser-vice, then the corresponding memo eld could depict the software module related with the started service.
SystemMode The system can have several modes, namely Startup, Shutdown, Shutdown Completed, Warm Restart, Normal Operation, and Field Service.
LogID They are about the logger. LogID refers to the identi er of the logger, while line number and thread describes the properties of the logger from the software perspective.
Line Number Thread
Module They are related to source code of the software inside the system. The module records the directory of the executable le.
SourceFile
Table 2.1: Descriptions of the Elements in a Log Entry
2.4. Data collection 25 by related errors.
Information, User Message and Warnings & Errors LogEntries are not used in the remainder of this chapter, however the Command is used for the conversion of event logs because it represents the actions executed on the system by the user. Out of the system modes presented in Table 2.1, only Field Service mode and Normal Operation mode will be considered. Field Service is the mode in which the FSE is performing the necessary operations/actions for maintenance of a system. Normal Operation is the mode when the user (doctor/laboratory technician and even FSE) operates the system. For example, when the doctor does a surgery on the patient, the system logs commands selected by the doctor and commands/events generated automatically by the system under normal operation mode. The FSE can also exe-cute actions under Normal Operation mode, i.e. when testing the functionality of the system, performing veri cation of its behaviour.
The event logs are selected from the Allura Xper systems considering the replace-ment of a FRU, using dierent perspectives such as version of the system, release, region and geography distribution.
A complicating factor in the case of the X-Ray system's log les is the highly com-plex and exible execution of activities. At the same time, the granularity of events stored in log les is at the lowest level.
2.4.2 Case de nition
In this section we are going to describe in detail the corrective maintenance case which is the basis of the data collection step. In the situation of a system mal-function the result is a corrective maintenance case which, as described in Section 1.2.2 is usually associated with the replacement of a unit in the system (FRU) or software updates. The FSE will visit the hospital and perform diagnostics on the system to identify the cause of the failure and most of the times replace a part or a set of parts. The process of such a replacement represents the focus of our analysis:
what the FSE does on the eld during the diagnosis of a faulty system. Whether it is the part replacement, performing calibration or veri cation of the system, all the activities that the FSE performs represent the FSE process. Every corrective maintenance case is recorded in the job sheets database, CSDWH (described in Sec-tion 1.2.2) that contains informaSec-tion about the customer complaint, replaced FRU, maintenance hours, etc.
A corrective maintenance in which a FSE is required to service a faulty system rep-resents a corrective maintenance call. These calls are coming from the customer who complains about the malfunction of the system. Each call has an unique entry in the CSDWH database and is stored with information like the callID, open date and close date, the id of the system on which maintenance was performed (con gID), etc.
As Figure 2.4 presents, the analysis starts with selecting a part of interest and
iden-Figure 2.4: Data collection
tifying the set of systems on which this part had been replaced, i.e. for this analysis the chosen part is the Converter Velara 8E, presented in detail in Section 2.6.1.
The reason for choosing the Converter Velara 8E is that the mean time to repair (MTTR) for the replacement of this part is much longer than expected, and it also has a large variation across dierent calls.
A part can be replaced on more systems and each system can have associated mul-tiple calls, it can be the case that the same part had to be replaced several times on the same system during a period of time. Furthermore, each call has an associated set of log les. The data collection represents the selection of log les corresponding to each call related to the replaced part.
The time window of a case is de ned to be k days before and after the call open date and call close date. This is a parameter that can be changed in the data collection phase. Because sometimes it happens that the calls are stored in the database by the engineers with dierent dates than they were performed, because the systems store the logged data at later dates and because in some cases the customer may not
The time window of a case is de ned to be k days before and after the call open date and call close date. This is a parameter that can be changed in the data collection phase. Because sometimes it happens that the calls are stored in the database by the engineers with dierent dates than they were performed, because the systems store the logged data at later dates and because in some cases the customer may not