Condition-based maintenance in the cyclic patrolling
repairman problem
M.J.A. Havinga
Master’s Thesis Econometrics, Operations Research and Actuarial Studies. Supervisor: Dr. B. de Jonge
Condition-based maintenance in the cyclic patrolling
repairman problem
Maik Havinga
December 12, 2017
Abstract
We consider the addition of condition-based maintenance to the cyclic patrolling re-pairman problem. In the cyclic patrolling rere-pairman problem, one rere-pairman inspects and repairs a set of machines in a fixed sequence. We introduce the possibility of per-forming preventive maintenance on the machines. In order to find the optimal policy for this problem, we formulate a Markov decision process. Furthermore, this Markov decision process is used to analyze the performance of a commonly used heuristic, called the control-limit policy. We found that the control-limit policy outperforms a policy in which only corrective maintenance is applied. If the machines have a rela-tively stable deterioration process, the optimal policy can reduce costs even further.
1
Introduction
The cyclic patrolling repairman problem originates from the 1950s and 1960s, a period from which the first scientific approaches to maintenance planning originate [10]. Mack et al. [15] were one of the first to study the cyclic patrolling repairman problem. A cyclic patrolling repairman system consists of a number of machines maintained by a single repairman. The repairman travels from machine to machine in a fixed sequence and repairs machines if they have broken down. As with most papers from this period, Mack et al.’s main methodology was renewal theory.
preferred over corrective maintenance, as in most cases it is less expensive to perform and requires less time than corrective maintenance. Most papers on the cyclic patrolling repair-man problem focus only on corrective maintenance, omitting the possibility of adopting preventive maintenance.
Preventive maintenance can be further classified into two categories, time-based main-tenance (TBM) and condition-based mainmain-tenance (CBM). An overview of both methods is given in Ahmad and Kamaruddin [1]. A paper that also compares these two methods is provided by De Jonge et al. [9]. Under TBM, preventive maintenance is performed on a machine after a certain period of time, independent of that machine’s condition. CBM uses the deterioration state of a machine to decide whether maintenance should be performed. Since TBM does not take the state of the machine into account, substantial remaining lifetime of a machine can be wasted. However, TBM is easier to implement than CBM since repair actions can be scheduled beforehand and no investments are needed to monitor the deterioration state of a machine. Nonetheless, we see a trend from TBM towards CBM as monitoring becomes less expensive due to technological advancements. An overview article on CBM is provided by Prajapati et al. [16].
In CBM models, a machine needs to be monitored in order to learn its deterioration state. This can be done by (remote) continuous monitoring or by physical inspection by a repairman. With continuous monitoring, the state of a machine is known at all times. However, this method is very expensive to implement and sometimes not possible at all. This is why in many applications, machines are inspected by repairmen to learn their deterioration state.
In this paper, we combine CBM with inspections in the cyclic patrolling repairman problem. The repairman traverses the machines in a cyclic order. On arrival at a machine, he carries out an inspection to learn its deterioration state. Based on this state and the observed history of the other machines, it is decided whether maintenance should be carried out or not. We formulate a Markov decision process that can be used to determine the optimal policy and to analyze a commonly used heuristic, i.e., the control-limit policy. Under the control-control-limit policy, a machine will be repaired if, after inspection, the deterioration level of machine is exceeds a predetermined threshold.
that can be used to analyze the problem and we introduce the algorithm used to determine optimal policies. In Section 5 we present and discuss case studies to gain insights into the problem. In Section 6 we present our conclusions and recommendations for future research.
2
Literature review
One of the earliest processes in which cyclical repair problems were researched, were wind-ing processes in the textile industry. These processes consists of transferrwind-ing yarn from small packages, produced at the spinning stage of production, to larger cones which are needed for the weaving or knitting processes. During this process, thin, thick and weak points in the yarn are detected, cut out, and the broken ends joined together by an (au-tomated) repairman so that winding can continue. The winding heads are inspected by a repairman in a fixed sequence and are repaired when an irregularity is detected during inspection. Since the repairman has to move between the winding heads and optionally take the time to repair them, repairs regularly have to wait until maintenance can be performed. This incurs extra costs due to downtime. One of the earliest studies on the winding process is published Mack et al. [15]. They focus on the efficiency of a system, given a certain amount of machines and a stochastic failure process. Mack [14] also pub-lished another paper on the problem in which he assumes that repair times are variable. Bunday and El-Badri [4] assume that repairs are not always successful and solve the prob-lem by means of a Markov chain. In more modern times, cyclic probprob-lems are found in computer networks [21]. A very general solution for a patrolling repairman is presented by Das and Wortman [7] in which systems can be asymmetric, meaning that walking distances, repair times and failure times of machines can differ from machine to machine. Patrolling repairman problems with negligible walking times between machines are also studied in the context of machine interference. Machine interference problems occur when two or more machines simultaneously require service, but when there are not enough repairmen to service them all simultaneously. Hence the failure and repair of one machine interferes with a timely repair of another. A review of the machine interference literature is provided by Stecke and Aronson [19] and Haque and Armstrong [11].
jobs and the server travels in a cyclical order from server to server to complete these jobs. Overview articles of polling systems are provided by Takagi [20, 22] and Boon et al. [2].
A combination of a traveling repairman (with walking times) and CBM is studied by Camci [5]. He introduces the traveling maintainer problem, a generalization of the traveling salesman problem. Camci [6] further develops the problem and includes the planning of maintenance over multiple days. An area in which the combination of traveling repairman and CBM is applied is the repair of offshore wind farms. An overview of offshore wind farm maintenance is provided by Shafiee [18]. Brakke [3] researches offshore wind farms in which CBM is applied in combination with a stochastic deterioration process. She finds the optimal policy using a Markov decision process. However, in these papers, no inspections need to be performed to determine the state of the machines and routes and maintenance are planned beforehand for a certain time period.
Our paper introduces CBM with inspections to the cyclical repairman problem. Com-pared to the older cyclical repairman problem, this introduces the possibility to perform preventive maintenance based of the condition of the machines, which could lead to re-duced costs. Compared to the more general traveling maintainer problem, the solution proposed in this papers allows for CBM based on inspection, instead of remote (continu-ous) monitoring. Furthermore, the proposed solution allows for dynamic decision-making, where decisions are taken after every inspection, while previous papers plan ahead for a longer period of time. Lastly, we will determine optimal solutions, whereas most other studies rely on heuristics to find reasonable solutions. However, our approach limits the repairman to a cyclical route with a symmetric system, i.e. all walking times, repair times and stochastic properties are equal for all machines. To the best of our knowledge, there are no studies on the combination of a cyclical repairman problem with CBM.
3
Problem description
is down and can only become operational again if corrective maintenance is performed. If the machine is not in the failed state, preventive maintenance can be carried out. After maintenance, the machine will be in the as-good-as-new state.
There is one repairman available for inspecting and maintaining the machines. This repairman traverses the machines in a cyclic order, i.e., he travels from machine to machine in a fixed sequence and returns to the first machine after visiting the last machine. When the repairman arrives at a machine, he will inspect it to learn the deterioration state of that machine. Time is scaled such that travelling to a machine and inspecting it requires one time period. After an inspection, the repairman can choose to either perform maintenance or to move on to the next machine. If the machine is not in the failed state, the repairman can perform preventive maintenance. Performing preventive maintenance will take Tpm time periods. If the machine is in the failed state, corrective maintenance can be performed. This will take Tcm time periods. Because we consider a discrete-time setting, both Tpm and Tcm are assumed to be positive integers. A machine will not be operational during maintenance. After finishing a maintenance action, the repairman will move on to the next machine.
We consider the problem from a cost perspective and we aim to minimize the long-run cost rate. Performing preventive maintenance will cost cpm and performing corrective maintenance will cost ccm. Furthermore, a downtime cost cd is incurred per machine for every time period that it is in the failed state. Downtime costs also apply to downtime due to maintenance.
4
Problem formulation
In this section we will formulate the problem that we consider as a Markov decision process (MDP). Subsequently, we will introduce the value iteration algorithm that we will use to analyze the problem. We will use both the formulated MDP and the value iteration algorithm to determine optimal policies and analyze the control-limit policy.
4.1 Markov decision process formulation
observed deterioration levels for all of the machines, potential maintenance activities at the last visit to each machine, and the type and remaining time of a possible maintenance action that is currently performed. A machine deteriorates continuously and can fail between visits. When moving to the next machine, the state of the MDP can be used to derive how long it has been since the repairman visited that machine. Combined with the previous deterioration state of that machine, we can calculate the probabilities of finding that machine in a certain state. Furthermore, when we find that the machine is in the failed state, we can calculate its expected downtime since the last inspection.
An MDP consists of a set of decision epochs, states, actions, a reward function and a set of state- and action-dependent transition probabilities. In the next sections, we specify these sets and functions.
4.1.1 Decision epochs
The decision epochs correspond to the start of the time periods and either coincide with the completion of an inspection or lie within the repair process of a machine. We denote the set of decision epochs by T = {1, 2, . . . , N }. Since our aim is to minimize the long-run cost rate, we consider an infinite time horizon, i.e. N = ∞.
4.1.2 States
The entire state of the system can be described by the deterioration states of the machines at their last visits, whether preventive or corrective maintenance has been performed at the last visit and the number of time periods that are remaining until a potential current repair is finished.
We let Xi, i = 1, . . . , n, either take the value PM or CM if, respectively, preventive or
corrective maintenance was performed at the last visit to machine i, or we set it equal to the deterioration state observed at the last visit if machine i has not been maintained. PM and CM are included in the state of the machine such that we can determine how many repairs are performed between two consecutive visits to a machine, and based on that, how much time has past between these visits. Note that Xi equal to either PM or CM implies that machine i was left in the as-good-as-new state.
we let X3, . . . , Xn be the last observed deterioration states of the 2nd, . . . , (n − 1)th next
visited machines by the repairman.
Furthermore, if a repair is performed, we let C ∈ {0, 1, . . . , max(Tpm, Tcm)} denote
the number of time periods that are left until the repairman inspects the next machine. E.g., if the repairman finds a broken machine (i.e., if X1 = m + 1) and chooses to repair
the machine, X1 will be set to CM at the next decision epoch to indicate that corrective maintenance is being performed, and C will be set to Tcm. In every following state, C will
decrease by 1 until C equals 1. Then, repairs are finished and the repairman will move to the next machine. In the next state, C will be set to 0 and the repairman will have finished inspecting the next machine. If C equals 0, there was no ongoing maintenance in the previous time period. The counter C can only have a strictly positive value if X1 equals either PM or CM.
We let the entire state be denoted by S = (X, C). The variable C can take max(Tpm, Tcm) + 1 different values. Furthermore, the number of states per machine equals m + 3, so we have that the state space S consists of (m + 3)n· (max(Tpm, Tcm) + 1) states and equals
S = {S = (X, C) : X = (X1, . . . , Xn), X1, . . . , Xn∈ {1, . . . , m, m + 1, PM, CM}
, C ∈ {0, 1, . . . , max(Tpm, Tcm)}}.
4.1.3 Actions
The action space can be described by two possible actions. Firstly, the repairman can choose to do nothing (DN) and to move on to the next machine. Secondly, he can perform maintenance (MA). We do not distinguish in the actions between preventive and corrective maintenance, as the state of the machine will specify the maintenance type. The repairman is not allowed to interrupt the repair process. This is a reasonable assumption because no new information about the state of the other machines is learned during maintenance, implying that it cannot be optimal to interrupt a maintenance action. In summary, the action space A equals
4.1.4 Costs
We let c(S, A) denote the cost that is incurred if action A is chosen in state S. We distin-guish between two different costs. Firstly, we consider costs of performing maintenance. If a machine is operational, i.e., if X1 ∈ {1, . . . , m}, and action MA is chosen, preventive
maintenance will be performed which has a cost of cpm. If the machine is in the failed state, i.e., if X1= m + 1, the repairman can perform corrective maintenance which has a
cost of ccm.
Secondly, downtime costs are incurred for machines in the failed state and for machines that are being repaired. During maintenance, a machine will be down. If preventive maintenance is performed, this will take Tpmtime periods and this will incur a total cost of
cd· Tpm. For corrective maintenance, this cost will be cd· Tcm. Furthermore, a machine can deteriorate and break down whilst the repairman is working on other machines. However, it is not observed when a machine breaks down. If we find that a machine is in the failed state when inspecting it, we can calculate the expected downtime of that machine since its last visit, and thereby also the expected downtime cost. We can do this based on the time since the last visit of the machine and the deterioration level of the machine at that visit. However, when we inspect the machine and find that it is broken, the state X1 of that machine will be m + 1 and we have no information on the deterioration level of the machine at the previous inspection. We could enlarge the state description to save this information, but this will increase the size of the state space by a factor m + 1. Instead, we make the downtime cost dependent on the state at the next decision epoch.
As described in Puterman [17], we can let c(S,A) depend on costs that are incurred if we move to a certain state at the next decision epoch. Downtime costs are then incurred when the decision to move to the next machine is taken, taking into account the probability that the machine is broken for a certain number of time periods. The advantage of this approach is that we still have the information of the state of the next machine and that we do not have to enlarge the state space.
has been carried out at the last visit and we add Tcm if corrective maintenance has been
carried out. Thus,
t(S) = n + Tpm· 1X1=PM+ n X i=3 1Xi=PM ! + Tcm· 1X1=CM+ n X i=3 1Xi=CM ! .
We let cf(S) denote the incurred cost for unforeseen downtime of the next machine. We let j ∈ {0, 1, . . . , t(S) − 1} denote the number of periods after its last inspection that the next machine has failed. As such, if j = 0 the machine was left in the failed state after the last inspection and if j > 0 the machine degraded to the failed state j time periods after the last inspection. We omit the states in which the machine is operational on inspection or has just broken down in the time period before inspection, since no downtime costs will be incurred in those cases. We let c(j) denote the downtime cost incurred if the next machine broke down j time periods after the last inspection, which is c(j) = cd· (t(S) − j). Furthermore, we let p(j | S) be the probability that the next machine has failed j time periods after the last inspection, given that we are now in system state S. From Puterman [17, Equation 2.1.1], we have that the cost for unforeseen downtime is equal to the sum of expected costs of all possible j, in which the expected cost is equal to the probability that the next machine has failed precisely j time periods after the last inspection, multiplied by the cost associated with breaking down j time periods after the last inspection. Hence,
cf(S) =
t(S)−1
X
j=0
c(j) · p(j | S). (1)
We let pi(D) denote the probability that a machine is in the failed state i time periods after its last inspection, given that its deterioration state was D at that inspection. We have
pi(D) = (Pi)[D, m + 1]
In order to translate the state Xi of machine i, which can also take values PM and CM, to the deterioration level of that machine, we define Xi∗ as the deterioration level
We now have that the probability p(j | S) of the machine breaking down exactly j time periods since the last inspection, is equal to the probability that it is broken j time periods since the last inspection (i.e. pj(X
∗
2)) minus the probability that it was already
broken j − 1 time periods since the last inspection (i.e. p(j−1)(X2∗)). Hence, we have that
p(j | S) = p0(X2∗), if j = 0, pj(X2∗) − p(j−1)(X2∗), if j > 0. Consequently, we have that (1) can now be written as
cf(S) = t(S)−1 X j=0 c(j) · p(j | S) = cd· t(S) · p0(X2∗) + t(S)−1 X j=1 cd· (t(S) − j) · pj(X2∗) − p(j−1)(X2∗) = cd· t(S) · p0(X ∗ 2) + t(S)−1 X j=1 (t(S) − j) · pj(X ∗ 2) − (t(S) − j) · p(j−1)(X ∗ 2) = cd· t(S)−1 X j=0 pj(X ∗ 2).
Hence, the expected downtime cost cf(S) is incurred when the repairman moves to the next machine. To summarize, the cost function c(S, A) equals
c(S, DN) = cf(S) (2) c(S, MA) = 1X 1∈{1,...,m}· (cpm+ cd· Tpm) (3) +1X 1=m+1· (ccm+ cd· Tcm) (4) +1C=1· cf(S). (5)
are finished, we incur expected downtime costs of the next machine when the repairman moves to the next machine (5). This occurs when C = 1.
4.1.5 Transition probabilities
We let S0 = (X0, C0) = (X10, X20, . . . , Xn0), C0 denote the state at the next decision epoch, given that the current state is S. We distinguish between transitions in case that the repairman moves to the next machine and transitions in case that the repairman stays at the same machine. A repairman stays at the same machine if the action MA is chosen and if C = 0 or C ≥ 2. If C = 0 and MA is chosen, the repair process will start and X1
will either be set to PM or CM. Furthermore, C will be set to the corresponding repair time. If C ≥ 2, repairs have already started and C will decrease by 1. This is summarized as follows (X0, C0) = (PM, X2, . . . , Xn), Tpm , if A = MA, C = 0 and X1∈ {1, . . . , m}, ((CM, X2, . . . , Xn), Tcm) , if A = MA, C = 0 and X1= m + 1, ((X1, X2, . . . , Xn), C − 1) , if A = MA, C ≥ 2.
In all other cases, the repairman will move to the next machine. Since in those cases repairs will be finished at the next decision epoch, C will be equal to 0 for all these cases. Since only new information is collected on the next machine, the deterioration states of the other machines will only shift one position. Therefore we have the following transitions for when the repairman moves
(X0, C0) = (X10, X3, X4, . . . , Xn−1, Xn, X1), 0
We only collect new information on X1. Let p(X10 | S) denote the transition probability of machine state X1 to state X
0
1, given system state S. The state of the newly inspected
machine (X10) will depend on the time t(S) since the last visit of the next machine, and the deterioration state of the next machine at that visit (i.e. X2∗). The transition probabilities
are derived from the transition probability matrix P and are given by
p(X10 | S) = (Pt(S))[X2∗, X10], if X10 ∈ {1, . . . , m + 1}, 0, otherwise.
p(S0 | S, A) = 1(X0 ,C0)=((PM,X2,...,Xn),Tpm), if A = MA, C = 0 and X1∈ {1, . . . , m}, 1(X0 ,C0)=((CM,X2,...,Xn),Tcm), if A = MA, C = 0 and X1= m + 1, 1(X0 ,C0)=((X1,X2,...,Xn),C−1), if A = MA, C ≥ 2, p(X10 | X) ·1(X0 ,C0)=((X10,X3,...,Xn,X1),0), otherwise.
4.2 Value iteration algorithm
We use the value iteration algorithm to determine optimal policies for our MDP and to analyze the control-limit policy. Normally, the algorithm generates a sequence of values denoted by v0, v1, v2, · · · ∈ V , where V denotes the space of bounded real-value functions on the state space S. Hence, v : S → R for all v ∈ V . vt can be interpreted as the minimum total expected costs when t time periods are left. However, at iteration i, only the information of vi−1 and vi is used. Storing all the unused functions can restrict the usage of the algorithm in large instances due to computer memory shortages. This is why we only store vi, the most recent function, and vi−1, the function generated in the previous iteration.
An optimal policy denotes for every state S ∈ S an optimal action A ∈ A. We let d(S) denote the optimal action when in state S ∈ S. Furthermore, we let sp(u), the span of u, be defined as
sp(u) = max
S∈S u(S) − minS∈Su(S)
The algorithm finds a stationary -optimal policy, and an approximation to its value.
Value Iteration Algorithm
1. Set v(S) = 0 and v0(S) = 0 for all S ∈ S and specify > 0. 2. Set v(S) = v0(S) for all S ∈ S.
3. For each S ∈ S, compute v0 by
v0(S) = max A∈A c(S, A) + X S0∈S p(S0 | S, A)v(S0) . 4. If sp(v0− v) < (6)
5. For each S ∈ S, choose d(S) ∈ arg min A∈A c(S, A) + X S0∈S p(S0 | S, A)v0(S0) and stop.
The approximated long-run cost rate g∗ now equals g∗= maxS∈S v 0(S) − v(S) + min S∈S v 0 (S) − v(S) 2 . 4.2.1 Control-limit policy
In this section we define our version of the control-limit policy. Jiang [12] provides more background on control-limit policies. Our control-limit policy consists of a simple decision rule; a machine is maintained if, after inspection, the deterioration level exceeds a certain threshold level M ∈ {1, . . . , m + 1}. Otherwise, the repairman will do nothing and move on. In a real-life scenario, this policy would be easy to implement. In order to determine the long-run cost rate of the control-limit policy, given a certain threshold level M , we modify the action space of our MDP. We let Acl be the action space of the transformed
MDP which is defined as Acl = {MA}, if C > 0 or X1∗ ≥ M , {DN}, otherwise.
We let gcl(M ) denote the approximated long-run cost rate of the control-limit policy
given threshold level M . We let M∗denote an optimal threshold level for the control-limit policy, i.e.,
M∗ ∈ arg min
M ∈{1,...,m+1}
(gcl(M )) .
Furthermore, we define gcl∗ as the optimal approximated long-run cost rate of the control-limit policy:
gcl∗ = gcl(M ∗
).
4.2.2 Periodicity
periodic optimal stationary policies. In our problem, we can encounter periodic optimal stationary policies if the number of machines and the repair times are relatively large. In such cases, these policies are for example encountered for the system state in which all machines have undergone corrective maintenance on the last visit. The performed maintenance has taken so much time that the next machine that is inspected is (almost) always in the failed state. In this situation, corrective maintenance will be performed at any machine visit, and the system is again in the same state after Tcm time periods.
We can also encounter periodic optimal stationary policies during the optimization of the control-limit policy. This happens for example if the threshold level M = 1, i.e. machines are always repaired, and machines deteriorate slowly, such that an as-good-as-new machine is very unlikely to fail in n+n·Tpmtime periods. In that case, a machine will never fail and corrective maintenance will never be applied. Hence, the repairman will always perform preventive maintenance and start repairs every Tpm time periods. This periodicity can cause the value iteration algorithm to converge very slowly or not at all.
In order to find the optimal policy for all cases, we apply the transformation described in Puterman [17, Section 8.5.4]. We choose an arbitray τ , satisfying 0 < τ < 1 and we define
˜
c(S, A) = τ c(S, A), for all A ∈ A and S ∈ S, and
˜
p(S0 | S, A) = (1 − τ )1S0
=S+ τ p(S 0
| S, A), for all A ∈ A and S, S0 ∈ S. We now replace c(S, A) by ˜c(S, A) and p(S0 | S, A) by ˜p(S0 | S, A) in our MDP. This translated MDP has the same optimal stationary policies as the original MDP, but ensures that the value iteration algorithm converges. This transformation also leads to shorter calculation times for non-periodic optimal policies. We let ˜g∗ be the approximated long-run cost rate of the translated MDP. The approximated long-long-run cost rate of the original MDP is given by g∗ = 1 τg˜ ∗ .
5
Results
we need to specify is the transition probability matrix of the machines’ deterioration process. We assume that the machines deteriorate by means of a stationary gamma process. However, the gamma process is a continuous-time continuous-state deterioration process. We discretize the gamma process using the approach used by De Jonge [8] and Kremers [13] to create a discrete-time Markov chain with transition probability matrix P . We also use the same parameterization of the gamma process as De Jonge [8]. This parameterization consists of a shape parameter a and a scale parameter b. We assume that a machine is broken if its deterioration exceeds 1. Furthermore, we discretize time in periods of the same length as the period between decision epochs, hence ∆t = 1.
5.1 Two machines
We analyze a case with n = 2 machines that each have m = 10 deterioration states before failure and an eleventh failed state. All parameter values of this case can be found in Table 1. Note that performing preventive maintenance will incur a combined repair and downtime cost of cpm+ Tpm· cd= 3, while performing corrective maintenance will incur a cost of ccm+ Tcm· cd = 14. The transition probability matrix P of the discrete-time
Markov chain is based on a stationary gamma process with a = 5 and b = 0.02. The resulting deterioration process has a relatively low volatility. The transition probability matrix P can be found in appendix A.
Parameter Value n 2 m 10 Tpm 2 Tcm 4 cpm 1 ccm 10 cd 1 0.00005 Table 1: Parameters
the state of the other machine at its last inspection. We omit the states in which X1 is
equal to PM or CM, since the only allowed action in those states is to continue an ongoing maintenance action. If an entry is “MA”, performing maintenance is optimal in that state. A “-” denotes that not performing maintenance is optimal.
First, note that a system state in the grey region of the table will never be reached, since a machine will always be repaired if it has a deterioration level of 8, 9 or 10 at an inspection. Furthermore, if the inspected machine is broken and the other machine is left in a relatively good state (i.e. X2∗ ≤ 4), it will be optimal to repair the inspected machine. This implies that a machine that is being inspected, while the other machine was left broken, can never be in a state with X1 ≤ 4.
If deterioration level X1 of the inspected machine is equal to 1, 2, 3 or 4, it is always optimal to move on to the next machine. If X1 is equal to 5, 6, 7 or 11, it depends on the state of the other machine at its last inspection whether performing maintenance is optimal. If X1 is equal to 5, 6 or 7 and the other machine is left in a good state, it is optimal to first visit the other machine. This is also the case if the other machine is as least as deteriorated as the inspected machine. In the other cases, it is optimal to repair the machine. If the inspected machine turns out to be in the failed state (i.e. X1 = 11) and the other machine was last observed in deterioration level 5, 6, 7, 8 or 9, it is optimal for the repairman to postpone corrective maintenance for the failed machine and first perform preventive maintenance on the other machine to avoid that it will also break. The approximation of the long-run cost rate of this optimal policy is g∗= 0.706.
We compare the optimal policy to our control-limit policy. The optimal threshold level equals M∗ = 7 and the optimal approximated long-run cost rate g∗cl = 0.745. In order to compare the optimal control-limit policy and the optimal policy, we let ∆g∗ = g
∗ cl−g
∗
g∗ ·100%
denote the percentage increase in long-run cost rate increase if we would use the optimal control-limit policy instead of the optimal policy. For the current instance, we find that the long-run cost rate increases by ∆g∗ = 5.50% if the optimal control-limit policy is applied instead of the overall optimal policy.
5.1.1 Number of deterioration states
X1\X2 PM CM 1 2 3 4 5 6 7 8 9 10 11 1 - - - -2 - - - -3 - - - -4 - - - MA MA 5 - - - MA - - - MA MA 6 - - - - MA MA MA - - - - MA MA 7 - - - MA MA MA MA MA - - - MA MA 8 MA MA MA MA MA MA MA MA MA MA MA MA MA 9 MA MA MA MA MA MA MA MA MA MA MA MA MA 10 MA MA MA MA MA MA MA MA MA MA MA MA MA 11 MA MA MA MA MA MA - - - MA MA
Table 2: Optimal policy
shows the long-run cost rate of the different policies as a function of the number of dete-rioration states m. We change P accordingly while all other parameters remain equal to the previously discussed instance. We compare the long-run cost rate of a policy in which only corrective maintenance is applied (CM), the optimal control-limit (C-L) policy and the optimal (OPT) policy.
The long-run cost rate of the CM policy is close to 2 for all m. We observe that the long-run cost rate of the control-limit and optimal policies decrease sharply until m = 5. Increasing m will lead to more information about the machine, as we can estimate the true deterioration level of the machine with greater accuracy. With this information, we can determine more precisely when it is optimal to perform maintenance. From m > 10, both the control-limit and optimal policies have a stable long-run cost rate, implying that we have selected a reasonable number of deterioration states in the previous setting.
5 10 15 20 0.0 0.5 1.0 1.5 2.0 2.5 m long−r un cost r ate CM C−L OPT
Figure 1: Long-run cost rate of policies for different m.
perform maintenance, as we can make a more precise estimate of the deterioration state. This increase in precision favors the optimal policy, which can save up to 6% compared to the control-limit policy.
5.1.2 Gamma process parameters
5 10 15 20 0 1 2 3 4 5 6 7 m ∆ g* (%)
Figure 2: Difference in policies for different m.
This “extra” deterioration is discarded, as a machine will be in the failed state regardless of the amount of overshoot. This causes the expected time until failure to be larger for more volatile processes.
The variance of the deterioration per time unit is equal to a · b2. Since b = 0.1a , the variance is equal to a · (0.1a )2 = 0.1a2. Hence the variance is larger if a is smaller. In Figure 3 we observe that the long-run cost rate of the CM-policy increases if the process becomes less volatile. This increase is caused by the previously noted decrease in time to failure if the process becomes less volatile. If the process is less volatile, the long-run cost rate of the optimal and of the control-limit policy decrease. As the deterioration process becomes more stable, the chance that a machine deteriorates significantly more than expected decreases. By consequence, a machine can be maintained at a more deteriorated state without running an increased risk that it will fail, which allows for postponing preventive maintenance and decreased costs.
0.1 0.5 1.0 5.0 10.0 50.0 100.0 0.0 0.5 1.0 1.5 2.0 2.5 a (log−scaled) long−r un cost r ate CM C−L OPT
Figure 3: Long-run cost rate of policies for different a.
stable. If the stability increases further and a exceeds a certain threshold, the long-run cost rate of the control-limit policy and the optimal policy become more similar. As shown in Figure 3, it is the control-limit policy that decreases in cost more than the optimal policy if the deterioration process becomes more stable, resulting in a lower ∆g∗. An explanation for this stronger decrease is that the optimal threshold M∗ of the control-limit switches from M∗ = 7 to M∗ = 8 around a = 20. This will lead to a decrease in long-run cost rate since maintenance is postponed. Furthermore, if the deterioration process becomes even more stable, the probability that the machine is failed on the next visit will decrease. In comparison, as shown in Table 2, the optimal policy allowed machines to deteriorate more than the control-limit policy for more volatile deterioration processes (e.g. a = 5), reducing its long-run cost rate for those instances.
5.1.3 Corrective maintenance cost
0.1 0.5 1.0 5.0 10.0 50.0 100.0 0 2 4 6 8 a (log−scaled) ∆ g* (%)
Figure 4: Difference in policies for different a.
maintenance. Furthermore, we observe that the control-limit and optimal policy will keep the long-run cost rate relatively low.
Figure 6 shows the percentage difference in long-run cost rate ∆g∗between the control-limit policy and the optimal policy for different values of ccm. We observe that for relatively low and high corrective maintenance costs the control-limit policy performs similarly to the optimal policy. When the corrective maintenance cost is low, the total cost for correc-tive maintenance will be more similar to prevencorrec-tive maintenance. This will lead to policies which apply preventive maintenance less frequent, as the penalty for a failed machine is not very severe. When corrective maintenance costs are low, both policies repair a ma-chine when it has reached a deterioration state of X1∗ ≥ 9. Only for some cases does the
optimal policy dictate the performance of preventive maintenance in an earlier deterio-ration state. This results in only a slight improvement over the control-limit policy. For high corrective maintenance costs, both policies apply a (mostly) identical conservative maintenance policy in which they both apply maintenance when X1∗≥ 3, which results in
1 5 10 50 100 500 1000 0.5 1.0 2.0 5.0 10.0 50.0
Corrective maintenance cost (log−scaled)
long−r un cost r ate (log−scaled) CM C−L OPT
Figure 5: Long-run cost rate of policies for different ccm.
1 5 10 50 100 500 1000 0 2 4 6 8 10
Corrective maintenance cost (log−scaled)
∆
g*
(%)
5.1.4 Other parameters
Similar to the corrective maintenance cost parameter, we varied the other parameters from Table 1, except and n. We found that an increase in preventive maintenance time Tpmwill increase the long-run cost rate of the control-limit policy and of the optimal policy. The optimal policy outperforms the control-limit policy if Tpm is low. If the time to perform corrective maintenance Tcm increases, the long-run cost rate of all policies increases. The
optimal policy outperforms the control-limit policy for all reasonable values of Tcm. If the cost for preventive maintenance cpmincreases, the long-run cost rates of both the
control-limit policy and the optimal policy increase. Furthermore, the difference between these policies ∆g∗ is larger if cpm is lower. Lastly, we varied the costs incurred for downtime
cd. This parameter influences both the cost of preventive and of corrective maintenance, since both incur downtime costs when applied. An increase of cd leads to an increase in
long-run cost rate of all policies and a decrease of ∆g∗. Overall, we observe the trend that the optimal policy performs significantly better than control-limit policy if the cost of preventive maintenance is relatively low.
5.2 Four machines
In this section, we analyze cases with n = 4 machines. We started with the same parame-ters as described in Table 1. We set the number of machines equal to n = 4 and changed the parameters of the gamma process such that the machines would take longer to break down, i.e., a = 2 and b = 0.025. This implies that the expected deterioration per time unit is equal to 0.05. For this instance, the long-run cost rate increases by ∆g∗= 0.756% if the optimal control-limit policy is applied instead of the overall optimal policy. Similar to the previous cases with two machines, we varied all parameters except for and n. However, almost every instance resulted in ∆g∗ < 1.5%. Only when varying the number of deterioration states m, we found that ∆g∗ increased substantially. Figure 7 shows the long-run cost rate of the different policies as a function of the number of deterioration states m. As with the two-machine case, the long-run cost rate of the control-limit and optimal policy decreases sharply for low values of m. However, the long-run cost rates are only stable from m > 20 rather than m > 10 and the optimal policy only starts to improve over the control-limit policy if m > 10.
0 10 20 30 40 0.0 0.5 1.0 1.5 2.0 2.5 3.0 m long−r un cost r ate CM C−L OPT
Figure 7: Long-run cost rate of policies for different m.
0 10 20 30 40 0 1 2 3 4 5 m ∆ g* (%)
0.1 0.5 1.0 5.0 10.0 50.0 100.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 a (log−scaled) long−r un cost r ate CM C−L OPT
Figure 9: Long-run cost rate of policies for different a.
The instance discussed at the beginning of this section with m = 10 yielded ∆g∗ = 0.756%. If we change the number of deterioration states m to 20, ∆g∗ becomes 3.508%. For all subsequent cases in this section we will use m = 20 as a base case, while keeping all other parameters the same as in Table 1 and keeping a = 2 and b = 0.025.
As with to the two-machine case, we varied a and b, keeping the expected deterioration per time period equal. Figure 9 shows the long-run cost rates of the three policies for different values of a. These values follow the same trend as the two-machines case, shown in Figure 3. However, when looking at Figure 10, which shows the ∆g∗ for different a, we observe a different trend than in the two-machines case. Whereas a higher a eventually decreased ∆g∗ in the two-machines case, in the four-machines case ∆g∗ increases if a rises. A possible explanation is that the more stable deterioration process allows for a better estimation of the state of unobserved machines, allowing the optimal policy to better decide when maintenance should be applied and when it should be postponed.
0.1 0.5 1.0 5.0 10.0 50.0 100.0 0 2 4 6 8 10 a (log−scaled) ∆ g* (%)
Figure 10: Difference in policies for different a.
6
Conclusion
In this paper, the addition of condition-based maintenance to the cyclic patrolling re-pairman problem was studied. We considered a rere-pairman who traverses machines in a predefined sequence in order to carry out inspections, learn their deterioration states and possibly carry out maintenance. Performing maintenance requires time and this influences the time until the repairman inspects and repairs subsequent machines. Costs are incurred for repairs and machines that are down. In order to analyze the problem, we have for-mulated it as a Markov decision process (MDP). Based on this model, we analyzed and optimized various policies. Besides the optimal policy, we considered the control-limit policy and a purely corrective policy.
of the machines are relatively stable and when a sufficiently fine discretization is used in the MDP. Furthermore, the improvement is most pronounced if the cost of corrective maintenance is not extremely high or low.
Appendix
A
Transition probability matrix P
References
[1] Ahmad, R. and S. Kamaruddin (2012). An overview of time-based and condition-based maintenance in industrial application. Computers & Industrial Engineering 63 (1), 135– 149.
[2] Boon, M.A.A., R.D. Van Der Mei, and E.M.M. Winands (2011). Applications of polling systems. Surveys in Operations Research and Management Science 16 (2), 67–82. [3] Brakke, R. (2016). Joint optimization of condition-based maintenance and vessel
rout-ing for offshore wind farms. Master’s thesis, University of Gronrout-ingen.
[4] Bunday, B.D. and W.K. El-Badri (1984). A model for a textile winding process. European Journal of Operational Research 15 (1), 55–62.
[5] Camci, F. (2014). The travelling maintainer problem: integration of condition-based maintenance with the travelling salesman problem. Journal of the Operational Research Society 65 (9), 1423–1436.
[6] Camci, F. (2015). Maintenance scheduling of geographically distributed assets with prognostics information. European Journal of Operational Research 245 (2), 506–516. [7] Das, T.K. and M.A. Wortman (1993). Analysis of asymmetric patrolling repairman
systems. European journal of operational research 64 (1), 45–60.
[8] De Jonge, B. (2017). Lecture notes maintenance planning and optimization. University of Groningen.
[9] De Jonge, B., R. Teunter, and T. Tinga (2017). The influence of practical factors on the benefits of condition-based maintenance over time-based maintenance. Reliability Engineering & System Safety 158 (Supplement C), 21 – 30.
[10] Dekker, R. (1996). Applications of maintenance optimization models: a review and analysis. Reliability engineering & system safety 51 (3), 229–240.
[11] Haque, L. and M.J. Armstrong (2007). A survey of the machine interference problem. European Journal of Operational Research 179 (2), 469–482.
[13] Kremers, N. (2017). Optimal dynamic aperiodic inspection and maintenance schedul-ing for sschedul-ingle-unit systems with continuous deterioration processes. Master’s thesis, University of Groningen.
[14] Mack, C. (1957). The efficiency of n machines uni-directionally patrolled by one operative when walking time is constant and repair times are variable. Journal of the Royal Statistical Society. Series B (Methodological) 19 (1), 173–178.
[15] Mack, C., T. Murphy, and N.L. Webb (1957). The efficiency of n machines uni-directionally patrolled by one operative when walking time and repair times are con-stants. Journal of the Royal Statistical Society. Series B (Methodological) 19 (1), 166– 172.
[16] Prajapati, A., J. Bechtel, and S. Ganesan (2012). Condition based maintenance: a survey. Journal of Quality in Maintenance Engineering 18 (4), 384–400.
[17] Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc.
[18] Shafiee, M. (2015). Maintenance logistics organization for offshore wind energy: cur-rent progress and future perspectives. Renewable Energy 77, 182–193.
[19] Stecke, K.E. and J.E. Aronson (1985). Review of operator/machine interference mod-els. International Journal of Production Research 23 (1), 129–151.
[20] Takagi, H. (1988). Queuing analysis of polling models. ACM Computing Surveys (CSUR) 20 (1), 5–28.
[21] Takagi, H. (1991). Application of polling models to computer networks. Computer Networks and ISDN systems 22 (3), 193–211.
