Master’s Thesis
Econometrics, Operations Research and Actuarial Studies
Condition-Based Maintenance with
Accelerated Deterioration
Author
R.B.T. Hermsen
Supervisor: Dr. B. de Jonge
Abstract
In this paper we consider a machine that deteriorates stochastically over time. A common assumption in the literature on maintenance is that a repair brings the machine back to the as good as new state. However, replaced machine parts may break down faster than before. We assume that a repair may have a negative impact on the deterioration rate of the machine. Furthermore, the machine must be replaced to reset the deterioration rate back to the as good as new rate. We formulate our problem as a partially observable Markov decision process and determine the optimal inspection and maintenance schedules for several problem instances to gain insights.
1
Introduction
Nowadays a large portion of the expenses of production companies consists of machine maintenance, especially because of ongoing automatisation. The failure of a machine usually causes extra costs because of longer downtime. If a machine fails, corrective maintenance (CM) has to be performed. Therefore, the goal is to maintain a deteriorated machine before the machine fails, which is cheaper. This is called preventive maintenance (PM). However, PM should not be performed too often as this also causes unnecessary expenses. A balance has to be struck between the number of PMs and the number of failures.
Generally, there are two types of PM: time-based maintenance (TBM) and condition-based main-tenance (CBM). In TBM the scheduling of PM is based on the age of the machine, or PM is performed at regular intervals. An advantage of TBM is the simplicity of the scheduling. However, TBM does not take into account the condition of the machine and because of this maintenance may be performed too early. CBM does take into account the condition of the machine, although this may introduce extra costs, e.g. for monitoring. In some cases the condition of the machine can be observed directly. In other cases an inspection must be performed at a certain cost, to reveal the deterioration level of the machine. A review on TBM and CBM is by Ahmad and Kamaruddin (2012).
deemed too high. We formulate the problem as a partially observable Markov decision process and determine the optimal inspection and maintenance policies for various problem instances to obtain insights.
The remainder of this paper is structured as follows. Section 2 gives an overview of the relevant literature for our problem. We formally describe the problem and the assumptions that we make in Section 3. In Section 4 the problem is formulated as a Markov Decision Process, and the value iteration algorithm that we use is specified. The results of numerical experiments and a sensitivity analysis are given in Section 5. Finally, in Section 6 a summary and recommendations for future research are given.
2
Literature Review
In this section we review the literature on maintenance planning with imperfect repairs. The research on maintenance planning with imperfect repairs is extensive. However, the number of studies in this context with varying deterioration rates is limited.
Pham and Wang (1996) discuss several methods of modelling imperfect repairs. The method that is most like our research uses a failure rate that increases after preventive maintenance. Nakagawa (1986) uses this method to determine the optimal maintenance interval (age-based maintenance). In contrast to our research, they use a failure rate that depends deterministically on the number of repairs, whereas we use a deterioration rate that changes stochastically after a repair. Zhou, Xi, and Lee (2007) consider a failure rate of the machine that depends on the number of repairs performed and the condition of the machine. They also use a failure rate that is a deterministic function of the number of repairs and the condition of the machine. Lugtigheid, Jiang, and Jardine (2008) consider a finite horizon and a component that must be replaced after a fixed number of repairs, where each repair increases the failure rate with a fixed amount.
been studies that discussed maintenance with stochastic effects. One of these papers is by Nakagawa (1979). He considers minimal and perfect repairs. Minimal repairs bring the machine back to the previous functioning state after a failure, and perfect repairs bring the machine back to the as-good-as-new state. In this paper a repair has a probability of p of being perfect and a probability of (1 − p) of being minimal. Do Van, Voisin, Levrat, and Iung (2015) consider imperfect repairs where the reduction of the deterioration level after a repair is generated from a truncated normal distribution with bounds 0 and the current deterioration level.
Lastly, a relevant study for our paper is by Flage, Coit, Luxhoj, and Aven (2012). They use inspections not only to observe the system state, but also to estimate the deterioration parameters, as these are unknown. The uncertainty of the deterioration parameters of a new machine are captured by a prior distribution, and the estimates of these parameters are updated in a Bayesian manner.
In this paper we consider the dynamic optimisation of aperiodic inspections and maintenance of a machine, where a repair may increase the deterioration rate of the machine. The actual deterioration rate remains unobserved. This is opposed to previous studies, where the effect of a repair on the deterioration rate is deterministic. Furthermore, we combine imperfect repairs with condition-based maintenance with aperiodic inspections. To the best of our knowledge this is the first research on imperfect repairs where the effect of a repair on the deterioration rate is unobserved.
3
Problem Description
state (corrective replacement). Replacements also bring the machine back to the as-good-as-new state, but set the deterioration rate back to the minimum rate as well.
The deterioration level of the machine can only be observed through inspections. The failed state is an exception to this, as failures are assumed to be self-announcing. However, inspections do not reveal the current deterioration rate. We assume that inspections are scheduled dynamically based on observed deterioration information and beliefs of the current deterioration rate. This implies that inspections are aperiodic, i.e. the time between inspections is not fixed. All inspection and maintenance actions are assumed to take negligible time. It is possible to perform a maintenance action immediately after an inspection.
The machine can have n different deterioration modes 1, . . . , n. These modes are ordered such that mode 1 corresponds to the lowest deterioration rate and mode n to the highest deterioration rate. We use a transition probability matrix Q ∈ Rn×n to describe the transition probabilities between the deterioration modes when a repair is carried out. Thus, the probability that a repair results in a transition from mode i to mode j equals Q[i, j]. Note that this matrix is upper-triangular, because we assume that repairs cannot result in a lower deterioration rate.
The current deterioration mode determines the probability of a one-step transition from deteriora-tion level i to j. Suppose the current deterioradeteriora-tion mode is k, k = 1, . . . , n, then the probability of transitioning from level i to level j is given by element [i, j] of the (m + 1) × (m + 1) transition probability matrix Pk.
To ascertain that a matrix Pk corresponds to a lower deterioration rate than matrix Pa if a < b,
a = 1, . . . , n, b = 1, . . . , j − 1, we assume that matrix Pb is of a higher stochastical order than matrix
Pa. Using a similar approach to Engelen (2018), we mathematically define this as m+1 X j=i+l Pa[i, j] ≤ m+1 X j=i+l Pb[i, j], i = 1, . . . , m + 1, l = 0, . . . , m + 1 − j.
We give an example of two stochastically ordered matrices below. The sums of the encircled prob-abilities in matrix P1 are smaller than or equal to the corresponding sums of matrix P2, so matrix
P1 = 0.80 0.20 0.10 0.10 0 0.70 0.20 0.10 0 0 0.60 0.40 0 0 0 1.00 , P2 = 0.30 0.30 0.20 0.20 0 0.50 0.30 0.20 0 0 0.50 0.50 0 0 0 1.00
We consider the problem from a cost perspective and we aim to minimise the long-run cost rate. The costs of the respective actions are as follows. Preventive repairs and corrective repairs cost cpr and ccr, respectively. The cost of inspection is ci. Furthermore, the machine can be replaced
preventively at an expense of cprpl and correctively at an expense of ccrpl.
We assume that inspections are cheapest, and that preventive maintenance is cheaper than corrective maintenance, so ci < cpr < ccr and ci < cprpl < ccrpl. Lastly, we assume that replacement is more
expensive than a repair, so cpr < cprpl and ccr < ccrpl.
4
Markov Decision Process Formulation
In this section our problem is formulated as a partially observable Markov decision process (POMDP). This allows us to solve the problem using the value iteration algorithm. The application of this algorithm will be explained in this section as well.
A Markov decision process (MDP) is a sequential decision process with uncertain outcomes. Our description of MDPs is based on Puterman (1990). In an MDP the costs and transition probabilities only depend on the current state and action, and not on the history of states. Time is divided into equally sized intervals and at the times between intervals decisions have to be made. These points in time are called decision epochs. The decision maker observes the current state and, based on that, chooses an action. Following this, the decision maker incurs a cost (or receives a reward) and the state transitions according to some transition probability matrix. This transition probability matrix depends on the action that is chosen.
under-lying system states. They are referred to as belief states or knowledge states.
A (partially observable) Markov decision process is thus described by five main elements: the de-cision epochs, states, actions, rewards, and transition probabilities. We let T denote the set of decision epochs. Since we aim to minimise the long-run cost rate, we consider an infinite time horizon. Therefore the set of decision epochs is T = {1, . . . , N }, with N = ∞.
We will elaborate on the other elements of the POMDP and on the value iteration algorithm in the remainder of this section.
4.1 States
As mentioned before, the machine has m + 1 deterioration levels and can have n different dete-rioration modes. Since we do not observe these at each decision epoch, a state of the POMDP is described as a probability distribution over the possible deterioration levels and a probability distribution over the possible deterioration modes of the machine. We let θ denote the knowledge of the current deterioration level of the machine. This can be written as
θ = [θ1, . . . , θm, θm+1],
where θi is the probability the machine is currently at level i. Let Θ denote the set of possible
knowledge states θ. We have
Θ = θ ≥ 0 : m+1 X i=1 θi = 1 .
We let ζ denote the knowledge of the current deterioration mode of the machine. This can be written as
ζ = [ζ1, . . . , ζn],
where ζj is the probability the machine currently is in deterioration mode j. Let Z denote the set
of possible knowledge states ζ. We have
Z = ζ ≥ 0 : n X j=1 ζj = 1 .
A state s is given by both the knowledge of the current deterioration level and of the current deterioration mode. It is defined as s = (θ, ζ). The state space S is thus given by
S =
s = (θ, ζ) : θ ∈ Θ, ζ ∈ Z
.
Throughout this section we will use the notation ei to denote the ith unit vector of length n, i.e. a
to denote the knowledge state where the deterioration mode is i with certainty.
The set of knowledge states S is infinite. To be able to apply the value iteration algorithm, we will determine a finite approximation of this set. We use the fact that the knowledge state for both the deterioration level and mode follow a fixed path after an inspection or maintenance action. This path is followed until the next inspection or maintenance action and is then (partially) reset. To denote the paths we introduce the information vector z = (i, j, ζ∗). In this vector j denotes the number of time periods since the last inspection, maintenance or failure, at which time the deterioration level was i. Furthermore, ζ∗ denotes the knowledge state of the deterioration mode at that time (j time periods ago). We let sz = (θz, ζz) denote the current knowledge state that follows from z.
Evidently, the set of possible ζ∗ is infinite and we will discretise it. Similar to the approach used by Maillart (2006) we choose some integer M > 0 and define the discrete set Z∗ as
Z∗ = c1 M, c2 M, . . . , cn M
for all sets of integers {ck, k = 1, 2, . . . , n} : n
X
k=1
ck= M and ck ≥ 0 for all k
.
Throughout this paper, if ζ∗ is calculated, ζ∗ takes on the value of the point in Z∗ closest in Euclidean distance to the calculated value. In theory, the number of periods since the last inspection or maintenance j can grow indefinitely, by choosing to perform no inspection or maintenance at each decision epoch. However, the machine will eventually fail. Therefore we can choose a sufficiently large number N as an upper bound on the number of periods that maintenance can be delayed. Now there are finitely many information vectors z possible, so the set of knowledge states that are reached via the paths is finite. We let S0 ⊂ S denote this finite set of knowledge states.
4.2 Decision stages
in the third stage the machine will deteriorate. After each stage a state transition is made. The following sections will define the transition probabilities and immediate costs for each stage.
Stage 1: Inspection
If an inspection is chosen, the deterioration level of the machine becomes known with certainty. Since an inspection is not useful if the deterioration level is already known with certainty (after a failure), the action space for stage 1 is thus given by
A1(sz) = {N I}, if j = 0, {I, N I}, otherwise.
where I denotes an inspection and N I denotes no inspection. The cost at this stage as a function of the current state sz and the action a ∈ A1 is given by
c(sz, a) = 0, if a = N I, ci, if a = I.
Let the current information vector be z = (i, j, ζ∗) and let ˜z = (˜i, ˜j, ˜ζ∗) be the information vector at the next stage. The probability that the inspection reveals deterioration level ` is simply given by element ` of the current knowledge state θz.
Furthermore, the knowledge of the deterioration mode is calculated by Bayesian updating. If the machine was at level i, j time periods ago, then the likelihood that the inspection reveals deterioration level ` when the deterioration mode is k is given by element Pkj[i, `]. This likelihood is multiplied by the prior probability of being in mode k, which is ζk∗. These probabilities are then normalized by dividing them by the sum of these products for all deterioration modes. Hence, we get P1(sz˜| sz, a) = θz`, if a = I, ˜i = `, ˜j = 0, and ˜ζk∗= P j k[i, `] · ζ ∗ k Pn l=1P j l[i, `] · ζ ∗ k , ` = 1, . . . , m, k = 1, . . . , n, 1, if a = N I and ˜z = z, 0, otherwise. Stage 2: Maintenance
these maintenance actions are preventive or corrective. No maintenance is denoted by N M . If the number of decision epochs since the last inspection or maintenance action is N , a maintenance action must be performed. The same holds if the machine is in the failed state. Thus, the action space A2(sz) of stage 2 is given by
A2(sz) = {R, RP L}, if j = N, or θzm+1 = 1, {N M, R, RP L}, otherwise.
The cost at this stage depends on whether the maintenance action is preventive or corrective, and thus depends on whether the machine is in a working state or the failed state. This cost as a function of the current state sz and the action a ∈ A1(sz) is given by
c(sz, a) = 0, if a = N M, cpr, if a = R and θm+1z = 0, ccr, if a = R and θm+1z = 1, cprpl, if a = RP L and θzm+1 = 0, ccrpl, if a = RP L and θzm+1 = 1.
If the machine is repaired, the deterioration level is reset to the ‘as good as new’ level. The repair may increase the deterioration rate so the knowledge of the deterioration mode ζz is multiplied by the deterioration mode transition probability matrix Q.
If the machine is replaced, the deterioration level is reset to the as good as new level and the deterioration mode is reset to mode 1. This is summarised in the following equation:
P2(sz˜| sz, a) = 1, if a = N M and ˜z = z, 1, if a = R, ˜i = 1, ˜j = 0, and ˜ζ∗ = ζz· Q, 1, if a = RP L, ˜i = 1, ˜j = 0, and ˜ζ∗ = e1, 0, otherwise. Stage 3: Deterioration
at the next decision epoch, we need to take the weighted sum over the possible deterioration modes, and we need to sum the probabilities of the functioning states:
R(sz) = R(θz, ζz) = m X i=1 n X k=1 ζkz· (θz· Pk) i .
The probability that the machine fails in the following time period can be calculated in a similar way: r(sz) = m X i=1 n X k=1 ζkz· (θz i · Pk[i, m + 1]) = 1 − R(sz).
Given failure did not occur, the next knowledge state for the deterioration level is calculated as
θ`˜z= Pn k=1ζk· (θz· Pk)` R(sz) , if ` = 1, . . . , m, 0, if ` = m + 1.
The next knowledge state for the deterioration mode is calculated using Bayesian updating. The prior probability for deterioration mode k is ζkz, which is multiplied by the likelihood of no failure given deterioration mode k is R(θz, ek). This product is then divided by the sum of these products
for all deterioration modes. Hence, we get
ζkz˜= ζ z k· R(θz, ek) Pn l=1ζlz· R(θz, el) , k = 1, . . . , n.
If failure does occur, the machine is with certainty in the failed state, i.e. ˜i = m + 1. We then have j = 0 and we recalculate ˜ζ∗ using Bayesian updating:
˜ ζk∗= ζ z k· r(θz, ek) Pn l=1ζlz· r(θz, el) , k = 1, . . . , n.
The transition probabilities are thus defined by
P3(sz˜| sz) = R(sz), if ˜i = i, ˜j = j + 1, ˜ζ∗ = ζ∗, r(sz), if ˜i = m + 1, ˜j = 0, ˜ζk∗ = ζ z k· r(θz, ek) Pn l=1ζlz· r(θz, el) , k = 1, . . . , n, 0, otherwise.
4.3 Value Iteration Algorithm
algorithm computes a sequence of values v0, v1, v2, . . . , where vi : S0 7→ R, for each i. This
com-putation is done iteratively, since each value vn+1 depends on vn. In the case of this study, it is possible to perform multiple actions at one decision epoch. This complicates the computation of each vn+1. We will calculate the values after each stage. As in Tjeerdsma (2018), we let un+1w denote the expected total costs after stage w, w = 1, 2. Furthermore, vn+1 denotes the expected total costs after stage 3.
We start by computing the expected total costs after stage 3, vn+1(sz). We have for all states sz ∈ S0
vn+1(sz) = X
s˜z∈S0
P3(˜s | s) · un1(s˜z).
Using this value we can calculate the value corresponding to the second stage, un+12 (sz). This value depends on the immediate cost of a maintenance action and the total expected cost one stage later. We have for all states sz ∈ S0
un+12 (sz) = min a∈A2(sz) X sz˜∈S0 P2(sz˜| sz, a) · vn+1(sz˜) + c(sz, a) .
Lastly, the expected cost after stage 1 consists of the immediate cost of an inspection and the total expected cost one stage later:
un+11 (sz) = min a∈A1 X s˜z∈S0 P1(sz˜| sz, a) · un+12 (sz˜) + c(sz, a) .
Since this is the first stage of the decision epoch, one iteration of the algorithm is finished. Let the span of v be
sp(v) = max
sz∈S0v(s
z) − min sz∈S0v(s
z).
A stopping criterion for the algorithm is sp(vn+1− vn) < ε, for some small ε > 0. Furthermore, the
final rewards v0 must be specified.
The optimal policy is given by the optimal actions in the final iteration. An approximation of the corresponding optimal cost rate is given by
c∗= minsz∈S0{v
n+1(sz) − vn(sz)} + max
sz∈S0{vn+1(sz) − vn(sz)}
2 .
5
Numerical Experiments
level are specified.
5.1 Deterioration Process
The stationary gamma process is commonly used to model continuous-time continuous-state dete-rioration processes (Van Noortwijk, 2009). Since, we consider a discrete-time discrete-state model, we will discretise several gamma processes to obtain our transition probability matrices Pk. For
each deterioration mode k, we will use a different gamma process to obtain Pk.
Abdel-Hameed (1975) introduced the gamma process in the modelling of deterioration levels. The gamma process is based on the gamma distribution. The density of the gamma distribution f is given by fα,β(t) = 1 Γ(α)βαt α−1e−t β, t > 0,
where α denotes the shape parameter and β the scale parameter. Γ(α) denotes the gamma function and is given by
Γ(α) = Z ∞
0
zα−1e−zdz.
A stationary gamma process has shape function at (a > 0) and scale parameter b > 0. It has the following properties (Van Noortwijk, Van Der Weide, Kallen, and Pandey, 2007):
1. X(0) = 0 with probability 1,
2. X(τ ) − X(t) ∼ fa(τ −t),b for τ > t ≥ 0,
3. X(t) has independent increments,
4. it is a jump process with infinitely many jumps in any time interval.
We assume that after a maintenance action the deterioration rate may increase, but the coefficient of variation (CV) is constant. The expectation and variance of the deterioration level X(t) at time t are
E(X(t)) = abt
and
Var(X(t)) = ab2t, respectively. Therefore its coefficient of variation is given by
CV(X(t)) = pVar(X(t)) E(X(t)) =
1 √
Hence, we assume that the shape parameter a remains constant. We assume that the scale parameter b can have n different values bi (i = 1, ..., n). They are assumed to be ordered as b1 < ... < bk <
... < bn. The deterioration process of a new machine has a scale parameter of b = b1.
5.2 Parameters
To obtain the transition probability matrices Pk, we discretise the continuous-state continuous-time
deterioration process X(t) using the method of Kremers (2017). We examine the case with n = 2 deterioration modes and m = 5 functioning deterioration levels. As gamma process parameters we use the shape parameter a = 0.6, the scale parameters b1 = 1, and b2 = 2, and a failure level of
L = 10. The resulting transition probability matrices are given in Appendix A. Note that matrix P2 is of a higher stochastical order than matrix P1.
The transition probability matrix for the deterioration modes after a repair is given by
Q = 0.5 0.5 0 1 .
An overview of parameters that are used is given in Table 1. The choice of N is based on the other parameters, but may need to be changed if the other parameters are altered. The value N = 11 is sufficiently large as it will turn out that it is never optimal to wait 11 periods before performing an inspection or maintenance action.
Table 1: Parameter values.
s(1,1,0) s(1,3,0) s(1,5,0) s(0.5,1,0) s(0.5,2,0) s(0.38,5,0) s(0.19,1,0) s(0.09,4,0) s(1,1,0)
Delayed inspection after 10 periods
Delayed inspection after 5 periods
Immediate repair
Delayed inspection after 1 period
Delayed inspection after 3 periods
Immediate repair
Delayed inspection after 3 periods
Delayed replacement after 1 period
Figure 1: An optimal decision path. Dashed lines correspond to a stochastic transition.
5.3 Solution
After applying the value iteration algorithm, we found the optimal decisions for all states. We display these optimal decisions in Table 5 in Appendix B, but only for ζ1∗= 1.0, 0.9, . . . , 0.0. A part of this table for i = 1 is given in Table 2. Note that empty cells correspond to states that are never reached, since an inspection or maintenance is performed before reaching that value of j. If at a state both an inspection and maintenance are optimal in their respective decision stages, only the inspection can be performed.
For illustrative purposes an example of a decision path is given in Figure 1. For simplicity’s sake, we now the denote the states as s(ζ∗1,i,j). As seen in Table 2, it is optimal to wait 10 periods and then
inspection is performed after waiting for 5 periods. If this inspection reveals deterioration level 5, we transition to state s(1,5,0). Immediately after the inspection a preventive repair is performed. The updated probability of being in deterioration mode 1 is updated to ζ1∗ = 0.5, so the new state is s(0.5,1,0). After waiting for 1 period, an inspection is performed. If this inspection reveals deterioration level 2, the new state is s(0.5,2,0). Then another inspection is planned after 3 periods. If this inspection reveals deterioration level 5, ζ1∗ is updated to ζ1∗= 0.38, so the new state is s(0.38,5,0). A delayed inspection is planned after 3 periods. If this inspection reveals deterioration level 4, we transition to state s(0.09,4,0). Now the deterioration rate is deemed too high, and a delayed preventive replacement is performed. This brings the machine back to the as good as new state, s(1,1,0).
In Table 2 we see that the time until an inspection is initially decreasing in the estimated probability
Table 2: Optimal decisions for a = 0.6, b1= 1, b2= 2, cprpl= 25, and ccrpl = 35.
↓ j / ζ1∗ → 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 i = 1 0 DN DN DN DN DN DN DN DN DN DN DN 1 DN DN DN DN DN I I DN DN DN DN 2 DN DN DN DN I I DN DN DN 3 DN DN DN DN I DN DN 4 DN DN DN I I DN 5 DN DN DN I 6 DN DN DN 7 DN DN I 8 DN I 9 DN 10 I
of having deterioration mode 2. This is caused by the fact that deterioration is faster. However, for values of ζ1∗ lower than 0.4, inspections are planned later again. Hence, the early inspections for ζ1∗ = 0.4, 0.5 are because in that state there is the most uncertainty about the deterioration mode. With inspections more information can be gained about the deterioration mode.
In Table 5 we can compare the decisions for i = 1 with higher values of i. In general inspections are planned earlier if the last observed deterioration level is higher. This reduces the probability of the machine failing before the inspection. If the estimated probability of having deterioration mode 2 is high, a delayed replacement is already planned if the machine is observed to be at deterioration level 3.
failed, maintenance must be performed immediately. A corrective replacement is performed if ζ1∗ is 0.34 or lower.
5.4 Sensitivity Analysis
In this part we examine the sensitivity of the optimal decisions to changes in several parameters. Firstly, we will determine the effect of a decrease in the replacement costs. Furthermore, we will analyse the effect of changes in the gamma deterioration parameters. Lastly, we will look at the influence of the number of deterioration levels.
5.4.1 The Effect of the Replacement Costs
To examine the effect of changing the replacement costs, we decrease the preventive replacement costs from cprpl= 25 to cprpl= 15 and the corrective replacement costs from ccrpl= 35 to ccrpl = 25.
After applying the value iteration algorithm again, we found new optimal decisions for all states. We display these optimal decisions in Table 6 in Appendix B, again only for ζ1∗ = 1.0, 0.9, . . . , 0.0. A part of this table is given in Table 3. In contrast to the situation with higher replacement costs, maintenance actions are now always planned when an inspection reveals deterioration level 4. In fact, ζ1∗ only needs to be 0.63 or lower for a replacement to be planned. Interestingly, repairs are planned earlier too while only the replacement costs decreased. This might be because after a repair the deterioration rate may increase, but it can be reset more cheaply now with a replacement.
Table 3: Optimal decisions for a = 0.6, b1= 1, b2= 2, cprpl= 15, and ccrpl = 25.
↓ j / ζ1∗ → 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
i = 4 0 DN DN DN DN DN DN DN DN DN DN DN
1 DN DN DN DN RPL RPL RPL RPL RPL RPL RPL
2 R R R R
5.4.2 The Effect of the Gamma Deterioration Parameters
We will now look at the effect of the gamma deterioration parameters. We want to vary the variance of the gamma processes used, while not changing the expected deterioration. To do this, we change parameter b1, while keeping a · b1 constant. Furthermore, we make sure b2 = 2b1 stays true.
We determine the optimal policy for b1 = 3, b2= 6, and a = 0.2. These parameters corresponds to
a 73.2% increase of the coefficient of variation of the gamma process that our transition probability matrices are based on. This means that the machine deteriorates more suddenly. The corresponding transition probability matrices are given in Appendix B. From the matrices we can see that the likelihood of staying at the same deterioration level has increased, but the probability of a failure when at a low deterioration level has also increased. This makes failures indeed more unexpected and sudden. The optimal decisions for the new parameter values are given in Table 7 in Appendix B. A part of this table is given in Table 4 below.
In Table 4 it is seen that with a new machine, it is optimal to have a delayed inspection after 15
Table 4: Optimal decisions for a = 0.2, b1= 3, b2= 6, cprpl= 25, and ccrpl = 35.
↓ j / ζ1∗ → 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 i = 1 0 DN DN DN DN DN DN DN DN DN DN DN .. . ... ... ... ... ... ... ... ... ... ... ... 6 DN DN DN DN DN DN DN DN DN DN DN 7 DN DN DN DN DN I DN DN DN DN DN 8 DN DN DN DN DN I I DN DN DN 9 DN DN DN DN I I DN DN 10 DN DN I I I I 11 DN DN 12 DN DN 13 DN DN 14 DN I 15 I
periods. This longer wait for an inspection could be because the probability of staying at the same deterioration has increased. The fact that the probability of failure has also increased is irrelevant for the planning of inspections, as failures are self-announcing. Furthermore, we observe the same pattern as before: when there is the most uncertainty about the deterioration rate (ζ1∗=0.5), an inspection is planned the earliest.
A graph of the cost rate for values of b1 between 0.25 and 5 is given in Figure 2. We can see that
probabilities matrices. For b1 > 2 however, the cost rate is decreasing in the variance.
0 5
2
b1
Cost
Figure 2: The optimal cost rate for varying values of b1 with a · b1 constant.
5.4.3 The Effect of the Number of Deterioration Levels
0 20
2
m
Cost
Figure 3: The optimal cost rate for a varying number of deterioration states m.
from 1 to 20 is given in Figure 3. Note that the value of N was different for each m. As seen in the figure, the long-run cost rate is decreasing in the number of deterioration states, as expected. However, the marginal effect of adding deterioration states is also decreasing. Doubling the number of functioning states from 5 to 10 results in a cost decrease of 9.7%, while increasing this number from 5 to 20 gives a cost decrease of 14.6%. Hence, expenses can be significantly reduced if more detailed condition information is available and used.
6
Conclusion
In this paper we studied the optimisation of condition-based maintenance with accelerated dete-rioration. We considered a machine of which the deterioration rate may increase after a repair. Repairs are used to reset the deterioration level to the as good as new state, and replacements can be used when the deterioration rate is deemed to high. They reset both the deterioration level and the deterioration rate to the as good as new state.
We formulated the problem as Markov decision process and applied the value iteration algorithm to find an optimal inspection and maintenance schedule. We considered the case with two deterioration modes. We found that inspections are scheduled well in advance if the current deterioration level is low. Furthermore, inspections are scheduled earlier when there is the most uncertainty about the deterioration rate. If the machine is in the failed state, the machine is replaced if the probability of having the low deterioration is rated is 0.34 or lower.
We analysed the sensitivity of the results for several parameters. We found that lower replacement costs may not only cause earlier replacements, but also earlier repairs. Furthermore, we increased the variance of the gamma process on which the transition probability matrices are based, but kept the expected deterioration constant. This resulted in a longer delay for inspections. We also found that the cost rate is initially increasing in the variance, but later decreasing. Lastly, we found that more detailed information about the condition of the machine gives significant cost benefits. Doubling the number of functioning deterioration levels from 5 to 10 gives a cost decrease of 9.7%. A disadvantage of our study is the number of states. The number of states is of order O(m · N · M ). We used 8181 states and this gives transition probability matrices with nearly 67 million elements. Furthermore, the value of N is determined by trial and error for each set of parameters, which is inefficient.
A
Appendices
A Transition Probability Matrices for a = 0.6, b1 = 1, and b2 = 2.
C Optimal Decisions
Table 5: Optimal decisions for a = 0.6, b1= 1, b2= 2, cprpl= 25, and ccrpl = 35.
References
Abdel-Hameed, M. (1975). A gamma wear process. IEEE Transactions on Reliability R-24 (2), 152–153.
Ahmad, R. and S. Kamaruddin (2012). An overview of time-based and condition-based maintenance in industrial applications. Computers & Industrial Engineering 63 (1), 135–149.
Chan, J.-K. and L. Shaw (1993). Modeling repairable systems with failure rates that depend on age and maintenance. IEEE Transactions on Reliability 42, 566–570.
Do Van, P., A. Voisin, E. Levrat, and B. Iung (2015). A proactive condition-based maintance strategy with both perfect and imperfect maintenance actions. Reliability Engineering and System Safety 133, 22–32.
Engelen, G.J.M. (2018). Condition-based optimization of maintenance actions with different quali-ties for system of subsystems. Master’s thesis, University of Groningen.
Flage, R., D. W. Coit, J. T. Luxhoj, and T. Aven (2012). Safety constraints applied to an adaptive bayesian condition-based maintenance optimization model. Reliability Engineering and System Safety 102, 16–26.
Kremers, N. (2017). Optimal dynamic aperiodic inspection and maintenance scheduling for single-unit systems with continuous deterioration processes. Master’s thesis, University of Groningen.
Lugtigheid, D., X. Jiang, and A. K. S. Jardine (2008). A finite horizon model for repairable systems with repair restrictions. Journal of the Operational Research Society 59, 1321–1331.
Maillart, L. M. (2006). Maintenance policies for systems with condition monitoring and obvious failures. IIE Transactions 38 (6), 463–475.
Malik, M. A. K. (1979). Reliable preventive maintenance policy. AIIE Transactions 11 (3), 221–228.
Monahan, G. E. (1982). A survey of partially observable markov decision processes: Theory, models, and algorithms. Management Science 28 (1), 1–16.
Nakagawa, T. (1986). Periodic and sequential preventive maintenance policies. Journal of Applied Probability 23, 536–542.
Pham, H. and H. Wang (1996). Imperfect maintenance. European Journal of Operational Re-search 94 (3), 425–438.
Puterman, M. L. (1990). Markov decision processes. Handbooks in Operations Research and Man-agement Science 2, 1990.
Tjeerdsma, T.C. (2018). Dynamic optimization of aperiodic inspection and maintenance scheduling for two-unit systems. Master’s thesis, University of Groningen.
Van Noortwijk, J. M. (2009). A survey of the application of gamma processes in maintenance. Reliability Engineering and System Safety 94 (1), 2–21.
Van Noortwijk, J. M., J. A. M. Van Der Weide, M. J. Kallen, and M. D. Pandey (2007). Gamma processes and peaks-over-threshold distributions for time-dependent reliability. Reliability Engi-neering and System Safety 92 (12), 1651–1658.
