Condition based spare parts supply
X. Lina, R.J.I. Bastenb,∗, A.A. Kranenburgc, G.J. van Houtumd
aDow Benelux, P.O. Box 48, 4530 AA, Terneuzen, The Netherlands bUniversity of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands cConsultants in Quantitative Methods, P.O. Box 414, 5600 AK, Eindhoven, The
Netherlands
dEindhoven University of Technology, P.O. Box 513, 5600 MB, Eindhoven, The
Netherlands
Abstract
We consider a spare parts stock point that serves an installed base of ma-chines. Each machine contains the same critical component, whose degrada-tion behavior is described by a Markov process. We consider condidegrada-tion based spare parts supply, and show that an optimal, condition based inventory pol-icy is 20% more efficient on average than a standard, state-independent base stock policy. We further propose an efficient and effective heuristic policy. Keywords: Inventory, spare parts, condition monitoring
1. Introduction
Capital goods, such as lithography equipment used in the semiconductor industry, CT scanners that are used in hospitals, or radar systems on board naval vessels, are expensive, technologically complex systems that are used in the primary processes of their users. As a result, their uptime is of utmost importance; each minute of unavailability may be costly, risky, or both. Spare parts are stocked to prevent downtime: upon failure, a defective component can be replaced quickly by a functioning spare part. It is therefore important to have enough stock on hand. However, spare parts are expensive, which
∗Corresponding author: T: +31 53 489 4007
Email addresses: xiaolin1014@gmail.com (X. Lin), r.basten@utwente.nl (R.J.I. Basten), bram.kranenburg@cqm.nl (A.A. Kranenburg), g.j.v.houtum@tue.nl (G.J. van Houtum)
means that stocking too many spare parts is costly. Since making this trade-off poses a challenging problem, there has been a lot of research on spare parts inventory control (see, e.g., Sherbrooke, 2004).
The costs of the spare parts inventories may be reduced by using formation on the condition of the components that are installed in the in-stalled base. To this end, we consider a number of machines, each containing the same one critical component that degrades over time. The degradation evolves according to a Markov chain with a finite state space, with at most one state transition per period. The condition is monitored (perfectly) at the beginning of each period. Since there is at most one transition per period, a component can fail only in a certain period if it is in the last degradation state at the beginning of that period. Upon failure, the component is re-placed immediately by a functioning spare part. One stock point is used to stock these spare parts and the base stock level in each period is dependent on the condition of the installed components and on the inventory pipeline (stock on hand plus outstanding orders). If the stock point has no stock on hand when a demand arrives, an emergency procedure is used to obtain the part from a source with ample supply. For this, emergency costs are paid. The other costs that we consider are inventory holding costs.
We model this problem as a discrete-time Markov decision problem (MDP) and we obtain the optimal policy using value iteration (see, e.g., Tijms, 1986). In an extensive numerical experiment, we find that the optimal policy, which by definition is a state-dependent base stock policy, achieves cost savings of upto more than 30% compared with a state-independent base stock policy. However, using value iteration to obtain the optimal policy is very time con-suming, especially if the number of degradation states, the lead time, or the number of machines is high. Therefore, we propose two heuristic policies, and a third heuristic policy that always uses the best of the former two poli-cies. All three heuristics are easy to compute. We show that the third policy is very close to optimal.
Our main contribution is as follows. First, while the use of condition in-formation for maintenance optimization has been studied extensively, there are only a few studies on the effect on spare parts supply. More specifically,
we study the effect of condition information on spare parts supply without changing the maintenance policy. This is especially relevant if preventive re-placements are undesired because of the loss of a significant part of the useful lifetime of components, or if preventive replacements are (almost) equally ex-pensive as corrective replacements (e.g., in process manufacturing, operating 24/7). The two papers that are most closely related are by Deshpande et al. (2006) and Louit et al. (2011); for the differences with those and other stud-ies, see Section 2. Second, we show that large savings can be obtained via condition based spare parts supply, and we identify under which circum-stances the savings are largest. Third, we derive an efficient and effective heuristic.
The remainder of this paper is organised as follows. We discuss the related literature in Section 2. In Section 3, we introduce our model, and in Section 4 we discuss the resulting Markov decision process. Next, in Section 5, we discuss the optimal base stock policy, and we discuss the heuristics that we propose in Section 6. We then perform an extensive numerical experiment in Section 7. Finally, we draw conclusions and give recommendations for further research in Section 8.
2. Related literature
The relevant literature on spare parts inventory control has started with the paper by Feeney and Sherbrooke (1966). This has led to a huge stream of research on all kinds of spare parts inventory systems. For an overview, we refer to Sherbrooke (2004) and Muckstadt (2005).
In most of the literature on spare parts inventory control, it is assumed that demands that cannot be fulfilled immediately are backordered. In our model, however, we assume that such demands are lost to the stock point under consideration and supplied by an emergency source. A recent overview of the literature on inventory control with lost sales is given by Bijvank and Vis (2011). We discuss two papers in more detail: Zipkin (2008a,b). In these papers, Zipkin considers, as we do, a discrete-time inventory model with deterministic lead time and lost sales. Zipkin (2008a) discusses various heuristics, one of which is the myopic policy (based on Morton, 1971). Our
second heuristic is also a myopic policy, but it is different since we have to make some approximations to cope with our complex demand process. A minor difference is caused by our assumption that holding costs are paid at the beginning of each period, whereas Zipkin (2008a) assumes that they are paid at the end of each period. In both papers, Zipkin assumes that demands in consecutive periods are independent. However, in Zipkin (2008b) he mentions an extension to Markov-modulated demands, resulting in a state-dependent (SD) inventory policy. He explains that the state space of possible supply orders is bounded. In our paper, the total demand process also follows a Markov-modulated process, but it has a special structura that is explicitly exploited in the analysis and the derivation of the heuristics.
There is also literature on varying demand rates and state-dependent (SD) inventory policies in models with backlogging. For example, Song and Zipkin (1993) consider a single stock point that faces demand that follows a Markov-modulated Poisson process. Considering continuous review, holding costs for inventory on hand and penalty costs for backorders, the authors show that the optimal policy is a base stock policy. Although the demand process at each point in time is dependent on an underlying Markov chain, there is no direct link with the state of the components in the installed base. Another stream of research on SD inventory policies uses advance de-mand information (ADI). ADI could result from monitoring the state of the components in the installed base, but this is, to the best of our knowledge, not considered in the literature. Instead, ADI means that customers place orders that will lead to an actual demand only after a certain lead time. The seminal paper in this stream of research is the paper of Hariharan and Zip-kin (1995). The authors consider both a single location system and a serial system. In both cases they assume a continuous review, base stock policy with full backordering. Replenishment orders are triggered by the customers’ orders, which result in actual demands after a certain lead time, thus making perfect ADI. ADI may also be imperfect, see, for example, Tan et al. (2007), who analyze a model in which there is a certain probability (that is generally less than 1) that an order that is placed will lead to an actual demand.
ADI. A related paper is that of Deshpande et al. (2006) in which part-age information is explicitly used in the inventory control policy. The authors assume that a part-age signal can be observed each period, which is then compared with a certain treshold value. Depending on the number of parts that have a signal above the threshold value, the authors calculate a condi-tional mean and variance of a normal distributed lead time demand. These are used to set the base stock level, assuming holding costs per unit on hand and backorder costs per backorder.
Finally, as mentioned in the introduction (Section 1), our paper is related to the stream of literature on condition based maintenance (CBM). In par-ticular, the delay time model is of interest, as introduced by Christer (1982). In this model, if a component becomes defect (which is not self-announcing) there is a certain delay time after which the component actually fails. This makes it whorthwhile to perform inspections to observe the condition of the component. For an overview of the literature on CBM, including a review of diagnostics and prognostics techniques, see Jardine et al. (2006). Within this stream of literature, also the reducing effect of condition based mainte-nance on spare parts supply costs has been studied; see Elwany and Gebraeel (2008), Wang et al. (2008, 2009), and Rausch and Liao (2010). As already stated, we distinguish ourselves from the latter studies by considering the effect of conditioning monitoring on spare parts supply without changing the maintenance policy. To the best of our knowledge, the only paper with the same focus is the one by Louit et al. (2011), but they have different assump-tions. They assume a single system for which at most one spare part is kept on stock, and they assume backordering when a spare part is demanded but not available. In contrast, we consider an arbitrary number of systems, al-low any spare parts inventory level, and assume that an emergency shipment is executed in an out-of-stock situation. This also implies that we have a different cost structure.
3. Model description
We consider a group of N (∈ N) identical machines, each containing one critical component. The component is subject to a degradation process on
a finite state space I0 = {0, . . . , I}, with state 0 representing the perfect working condition and state I representing failure. Time is divided into periods of unit length and we assume an infinite time horizon. We assume that a component can degrade at most one state per period. The state transition can occur at any time during the period, and a transition to state I (failure) is self-announcing. A failed component is replaced in negligible time by a functioning spare part.
This spare part is demanded from one local stock point. If this local stock point has no stock on hand when a demand arrives, an emergency procedure is used and the local stock point faces a lost sale. Using the emergency procedure leads to additional costs of cem (> 0), which may include some
downtime costs and the higher costs (compared to a normal replenishment) that have to be made to achieve a short emergency lead time. We assume that the failed part is still replaced in the same period in which it failed. The stock point can order new components that arrive after a deterministic replenishment lead time of L (∈ N) periods; there are no fixed ordering costs. At the beginning of each period t, we consider the following sequence of events:
1. Spare parts in the pipeline come one period closer; items that were ordered L periods earlier arrive at the stock point.
2. The state of each critical component is observed. Since a failed compo-nent is replaced before the beginning of the next period, a compocompo-nent will never be in state I at the beginning of a period. We therefore introduce the state space I = I0 \ I of states that can be observed at the beginning of a period.
3. A replenishment order may be placed.
4. Holding costs ch (> 0) are paid for the complete inventory position, so
for components on hand and in the replenishment pipeline. The latter assumption can be relaxed since the holding costs that are paid for the components in the pipeline are a constant factor, which is independent of the inventory control policy that is used: given that cem is defined
We define qi,j to be the transition probability of one component’s state
i ∈ I to j ∈ I. For ease of notation, we define qi,I ≡ qi,0 and we refer to
state 0 as the subsequent state of state I − 1. Then, for all i ∈ I it holds that qi,i ≥ 0, qi,i+1 > 0, qi,i+ qi,i+1 = 1, and qi,j = 0 for all j 6∈ {i, i + 1}.
We are interested in finding the inventory policy that minimizes the (undiscounted) average costs per period for an infinite planning horizon. Since the inventory policy does not influence the total number of spare parts that will be used (remember that cem is defined to be additional costs), we
may ignore the variable ordering costs per component.
4. Markov decision process
The behavior of the system that we presented in Section 3 can be de-scribed by a discrete-time Markov decision process (MDP). We will define its states, describe the system transitions between states, and finally show the resulting costs.
We model an MDP with state space S = {(m, s)}, with:
• The degradation vector m = (m0, . . . , mI−1) and mi denotes the
num-ber of parts in degradation state i ∈ I. It holds that P
i∈Imi = N .
• The inventory vector s = (s0, . . . , sL−1), s0 denotes the stock on hand,
and sl (for l ∈ {1, . . . , L − 1}) denotes the number of parts that arrive
in l periods.
Notice that these states describe the situation before a new replenishment order is placed. Therefore, there is no sL in our state description.
The action space of possible actions that can be taken in state (m, s) is denoted by A0(m, s) = N0 = N ∪ 0. Notice that a ∈ A0(m, s) represents
the number of spare parts to order in the current period. In Section 5, we will introduce the action space A(m, s) of actions that can be taken in an optimal policy, and we show that A(m, s) has a finite number of elements.
We define the transition vector d = (d0, . . . , dI−1) with di denoting the
number of parts degrading in a certain period from state i ∈ I to its sub-sequent state. We define the set D(m) = {d | 0 ≤ di ≤ mi, ∀i ∈ I} as the
set of all possible transition vectors, given the current degradation vector m. We use ˆq(m, d) to denote the probability of observing transition vector d ∈ D(m) in a period if the system state with respect to the status of the installed components at the beginning of that period is m:
ˆ q(m, d) =Y i∈I mi di (qi,i+1)di(1 − qi,i+1)mi−di.
The subsequent degradation vector, given the current degradation vector m and the transition vector d is given by f (m, d) = (f0(m, d), . . . , fI−1(m, d)),
with: fi(m, d) = m0− d0+ dI−1 if i = 0; mi− di+ di−1 if i ∈ I \ 0.
We are now ready to define the subsequent inventory vector, given the cur-rent inventory vector s, the transition vector d, and the action a ∈ A0(m, s) that was chosen (notice that a is chosen before d is observed) as ga(s, d) =
ga 0(s, d), . . . , gaL−1(s, d), with: gla(s, d) = (s0− dI−1)+ if l = 0; sl+1 if l ∈ {1, 2, . . . , L − 2}; a if l = L − 1.
Note that (x)+ = max{0, x}.
We next define the transition probability from one system state (m, s) to the next (m0, s0), given that action a ∈ A0(m, s) is taken as:
pa((m, s), (m0, s0)) =
X
{d∈D(m)|f (m,d)=m0;ga(s,d)=s0}
ˆ
q(m, d).
Now that the transition probabilities are defined, we are ready to focus on the costs. The expected one-step costs in the current period, depending on the current system state (m, s) and the action a ∈ A0(m, d) that is taken,
are defined as: ca(m, s) = ch a + X l∈{0,...,L−1} sl + cem X d∈D(m) ˆ q(m, d) (dI−1− s0)+. We denote with Vπ
n(m, s) the total (undiscounted) expected costs with n
periods left to the time horizon, when the current system state is (m, s) and the policy π is used. This policy π = {π(m, s) | (m, s) ∈ S}, with π(m, s) being the function that gives the ordering action a ∈ A0(m, s) given a system state (m, s). Vnπ(m, s) is recursively calculated as follows:
Vnπ(m, s) = cπ(m, s) + X (m0,s0)∈S pπ((m, s), (m0, s0)) Vn−1π (m 0 , s0).
Since we are interested in the long-run average costs in an infinite horizon setting, we define: g(π) = lim n→∞ Vπ n(m, s) n .
Our goal is to find the optimal ordering policy π∗ that minimizes the long-run average costs:
π∗ = arg min
π
g(π).
5. Optimal policy
In this section, we will first give some properties that an optimal ordering policy satisfies. We can use these properties to reduce the action space. We then use value iteration to find the optimal policy.
Lemma 1. It is never optimal to order more than N spare parts in one period.
Proof. Components may degrade at most one state per period. Since there are N components installed, we may observe at most N failures per period. If we then compare policy π1 ordering N0 > N components in a certain
period with policy π2 ordering N components in that period and N0 − N
lead to incurring more emergency costs, but it does lead to incurring less inventory holding costs than applying policy π1. Therefore, a policy that
orders more than N spare parts in one period, may never be optimal.
As a result, we know that the action space of actions that may be taken in an optimal policy is finite. We will introduce such an action space formally after Corollary 3.
Lemma 2. Let Dmax
L+1(m) be the maximum demand over L + 1 periods, given
the current degradation vector m. It can be calculated as follows:
DL+1max(m) = N · L + 1 I + I−1 X i=I+IbL+1I c−(L+1) mi.
Proof. Notice that if I > L + 1, the expression reduces toPI−1
i=I−(L+1)mi and
we are simply counting the number of components that are in the last L + 1 degradation states i ∈ I. If I = L + 1, then the second expression reduces to the summation over an empty set and the first expression reduces to N . If I < L + 1, then machines may experience multiple failures in the next L + 1 periods. b(L + 1)/Ic counts the number of times that a component in one machine may pass through all degradation states in the next L + 1 periods. Each time that all states are passed through, the machine experiences one failure. This number is multiplied by the number of machines N . In addition, more failures may be experienced if components are already in the last couple of degradation states. These additional failures are counted in the second term of the expression.
Corollary 3. Under an optimal policy, the inventory position, given by a +PL−1
l=0 sl (a ∈ A
0(m, s)), will never be increased to a higher level than
DmaxL+1(m).
Proof. The ordering decision is taken at the beginning of the period, before demand in that period has realized. As a result, components that we order at the beginning of a period t, may be used from period t + L on. In the periods t upto and including t+L, we may observe at most Dmax
so we require at most DL+1max(m) spare parts. If our inventory position would be increased to a higher level than DL+1max(m), then the additional spare parts can only be needed from period t + L + 1 on. But, with a similar argument as we used in Lemma 1, we see that it is better to order such spare parts earliest next period.
As a result, we can define the action space of actions that may be taken under an optimal policy to be A(m, s) =
a ∈ N0 | a ≤ DL+1max(m) −PL−1 l=0 sl + . Using value iteration (see, e.g., Tijms, 1986), the optimal policy can be found. The value function Vn(m, s) can be determined recursively as follows:
Vn(m, s) = min a∈A(m,s) ca(m, s) + X (m0,s0)∈S pa((m, s), (m0, s0)) Vn−1(m0, s0) . (1) V0(m, s) = 0 for all (m, s) ∈ S and computation is stopped if:
max (m,s)∈S{Vn(m, s) − Vn−1(m, s)} − min(m,s)∈S{Vn(m, s) − Vn−1(m, s)} ≤ e min (m,s)∈S{Vn(m, s) − Vn−1(m, s)} ,
with e = 10−6. The stationary policy whose actions minimize the right hand side of Equation 1 for all (m, s) ∈ S will be negligibly close in costs to the minimal average costs policy (Tijms, 1986).
Computation of the optimal policy using value iteration is computation-ally inefficient: the size of the state space increases exponenticomputation-ally with the number of machines, the number of degradation states, and the length of the replenishment lead time. Therefore, and because the optimal policy has no clear structure, we propose three heuristic policies in the next section.
6. Heuristic policies
In this section, we propose three heuristic policies. First, however, we present a reference policy that we will use to compare our heuristic policies
with and that we will use as the basis of our first heuristic.
Our reference policy is the optimal state-independent (SID) base stock policy. Comparing with this policy, we can find the value of incorporating degradation information in the inventory control policy. The SID base stock policy has one parameter, the base stock level SSID. If the inventory position
is below this value when an ordering decision is taken, then a number of spare parts is ordered such that the inventory position is raised to SSID again. The
optimal base stock level, SSID∗ , can be found using enumeration, and using the fact that a lower bound on the average costs is given by multiplying the holding costs by the base stock level minus the mean lead time demand
We call our first heuristic the modified SID heuristic (MOD), since it is based on the optimal SID base stock policy. We follow the ordering decisions of the optimal SID base stock policy, except when this would lead to an inventory position that is higher than Dmax
L+1(m). The reason is that we have
shown in Corollary 3 that having an inventory position that is higher than Dmax
L+1(m) can never be optimal. As a result, we sometimes have a lower
inventory position than the SID base stock policy, leading to lower inventory holding costs, while not increasing the emergency costs that we face. This policy is thus at least as good as the optimal SID base stock policy. However, the modified SID heuristic uses a very simple procedure to deviate from the base stock level. As a result, it may sometimes still increase the inventory to a too high level. On the other hand, if the overall condition of the components in the installed base is poor, the level to which the inventory is increased may be too low.
We therefore propose a second heuristic, which we call the myopic heuris-tic (MYO). This heurisheuris-tic uses the degradation information of the compo-nents in the installed base to find an approximated probability distribution of the (L + 1)-period demand. It next makes an explicit trade-off between holding costs and emergency costs, in a myopic way, as described below.
Let us assume that we are at the beginning of period t and that we have to decide how much to order. We need to determine the demand in the periods t upto and including t + L. We make the first approximation by assuming that each machine will face at most one component failure in these periods.
This effectively means that we change the degradation state space back to I0 = I ∪ I, set q
I,0 = 0, and set qI,I = 1. We thus make degradation state I
an absorbing state. We then define Pi,t0 to be the probability that a machine
that is in state i ∈ I0 at the beginning of period t is in state I (has failed) at the beginning of period t + t0, t0 ∈ N0. Obviously, Pi,0 = 0 for all i ∈ I and
PI,t0 = 1 for all t0 ∈ N0. We can then calculate the probabilities for all i ∈ I
and t0 ≥ 1 recursively as follows:
Pi,t0 = qi,i+1Pi+1,t0−1+ qi,iPi,t0−1.
We next define Q(J ) to be the probability of exactly J (out of N ) com-ponents failing in periods t upto and including t + L, under the assumption that each machine will face at most one component failure in those periods. For this, we need to consider only those components that are in the last L + 1 degradation states at the beginning of period t, assuming for now that L + 1 ≤ I. From each of these states i, at most mi components can fail in the
periods t upto and including t + L. We therefore define the set J to be the set of all vectors that can lead to exactly J failures in those L + 1 periods, where ji indicates the number of components that is in degradation state i
at the beginning of period t and that has failed after L + 1 periods (at the beginning of period t + L + 1), as follows:
J = {j ∈ NL+1 0 | ji ≤ mi+I−(L+1); L X i=0 ji = J }.
We can now calculate Q(J ) as follows:
Q(J ) =X j∈J Y i∈{0,...,L} mi+I−(L+1) ji
(Pi+I−(L+1),L+1)ji(1−Pi+I−(L+1),L+1)mi+I−(L+1)−ji.
If L + 1 > I, the calculations simplify:
J = {j ∈ NI 0 | ji ≤ mi; I−1 X i=0 ji = J },
and Q(J ) =X j∈J Y i∈{0,...,I−1} mi ji (Pi,L+1)ji(1 − Pi,L+1)mi−ji.
Now that we have an approximate demand distribution, we make the second approximation by assuming that all demands are fulfilled from stock in the periods t upto and including t + L − 1. In other words, we ignore that some demands are satisfied by an emergency supply in those periods, and that then the inventory position is not reduced by those demands. This is quite a reasonable assumption since emergency costs are usually so high that optimal base stock levels will lead to a low probability of lost sales. Using this assumption, we can calculate the (approximate) marginal (L + 1)-period costs if the inventory position at the beginning of period t is increased to base stock level S(m + 1) instead of S(m), given degradation vector m at the beginning of the first period that we consider:
∆C(S(m)) = ch(L + 1) − cem 1 − S(m) X J =0 Q(J ) . (2)
We have to pay additional holding costs for periods t upto and including t + L (first term). The emergency costs decrease if there are more than S(m) demands (second term). Since this second term at the right hand side of Equation (2) increases in S(m), we see that ∆C(S(m)) is increasing in S(m), which makes the (approximate) optimal costs convex in S(m). Hence, the (approximate) optimal base stock level S∗(m) is the smallest base stock level S(m) for which ∆C(S(m)) ≥ 0, i.e., for which the following inequality holds: S(m) X J =0 Q(J ) ≥ 1 −ch(L + 1) cem .
We have made two approximations in our calculations. Besides, we consider the demand distribution over the next L + 1 periods only, and thus ignore the effect of the decision on the next periods. This makes our heuristic myopic.
Evaluation of the reference policy (the SID policy), the modified SID policy, and the myopic policy can be done using value iteration for smaller
Additional Values used in in test bed
Parameter all test beds 1 2 3
N (Number of machines) 1; 5 — 10 —
L (Replenishment lead time, in weeks) — 1; 2 2; 5 1 I (Number of degradation states) 2 3 5 3; 5 DP V (Degradation probability vector: qi,i+1 for i ∈ I) 100(1); 100(2); 250 — — —
cem (Emergency costs, in Euros) 104; 105 — — —
ch (Inventory holding costs, in Euros/unit/week) 1; 200; 103 — — —
Table 1: Parameters used in each test bed (full factorial); for DP V , see Table 2
problem instances, and using simulation for larger problem instances. The myopic heuristic is expected to lead to lower costs than the modified SID heuristic in most cases. On the other hand, the modified SID heuristic cannot perform worse than the reference policy. Therefore, we propose the best-of-two heuristic (BO2), which is the superior of the other two heuristics: we take the results of the former two heuristics, evaluate them as explained above, and choose the one that leads to the lowest costs.
7. Numerical experiment
We perform an extensive numerical experiment, using three test beds of problem instances. We explain the design of our experiment in Section 7.1 and we discuss the results in Section 7.2.
7.1. Design
The parameters that we use are given in Tables 1 and 2; we explain below how the problem instances are generated using these parameters. Notice that the test beds differ only for N , L, and I. Per test bed we use a full factorial design; test beds 1, 2, and 3 consist of 144, 216, and 108 problem instances, respectively. All parameter values are chosen such that we get a wide range of problem instances that are realistic in practice, e.g., for manufacturers of high-tech systems that sell service contracts. For example, the additional costs for an emergency supply (cem) are much higher than the inventory
holding costs (ch) in practice, and thus in our problem instances.
The aim of test bed 1 is twofold. First, we aim to see how much costs can be saved when using the optimal SD base stock policy instead of the
I (Number of degradation states)
Abbreviation 2 3 5
100(1) (1/50; 1/50) (1/50; 1/35; 1/15) (1/50; 1/20; 1/15; 1/10; 1/5) 100(2) (1/50; 1/50) (1/50; 1/25; 1/25) (1/50; 2/25; 2/25; 2/25; 2/25) 250 (1/125; 1/125) (1/125; 2/125; 2/125) (1/125; 4/125; 4/125; 4/125; 4/125)
Table 2: Degradation probability vectors, DP V : qi,i+1for i ∈ I
optimal SID base stock policy, and second, we aim to see how much of these cost savings are captured by our three heuristics. Test bed 2 is then used to explore the performance of the best heuristic, the best-of-two heuristic, on a wider range of parameter settings. We also use this test bed to investigate in which cases the myopic heuristic does not perform well. Finally, we use test bed 3 to understand what is lost if partial degradation information is used while more degradation information is available.
The degradation probability vectors (DP V ; qi,i+1 for i ∈ I) are
cho-sen such that the expected duration in the perfect state is the same as the total expected duration in the other states. The vector ‘100(1)’ leads to an increasing degradation probability with an increasing degradation state (degradation keeps going faster). The other two vectors have constant degra-dation probabilities, except when they are in the perfect state (state 0). In fact, all values in the ‘250’ vectors can be found by taking the values in the corresponding ‘100(2)’ vector and dividing them by 2.5. Notice that when comparing, for example, ‘100(1)’ for I = 2 and I = 5, then the former vec-tor can be seen as an aggregated version of the latter vecvec-tor: there is less information on the exact degradation state. We use this fact in test bed 3 to understand what is lost if only partial degradation information is used when more is available.
7.2. Results
The results for test beds 1 and 3 are obtained using value iteration. Some problem instances in test bed 2 are too large to use value iteration, which is why we use simulation for all problem instances in that test bed, except for those that are also part of test bed 1. To be more precise, we use the batch means method (see, e.g., Law, 2007, pp. 520–521), as follows:
• Perform a simulation run of length m periods (our choice of m is ex-plained below), resulting in m observations: Yi for i ∈ {0, . . . , m − 1}.
• Divide the run into n batches (we choose n = 10) and calculate the mean value for each batch: ¯Yj = 1kP
(k+1)·m−1
i=k·m Yi for j ∈ {0, . . . , n − 1},
with k = m/n.
• Calculate the grand sample mean: ¯Y =¯ n1 Pn−1
j=0 Y¯j.
• Calculate the 100(1 − α) percent confidence interval (we choose α = 0.1) for ¯Y : ¯¯ Y ± t¯ n−1,1−α/2pS2(n)/n, with tn−1,1−α/2 being the upper
1−α/2 critical point for the t-distribution with n−1 degrees of freedom (t9,0.95 ≈ 1.833), and S2(n) =
Pn−1
j=0 ¯Yj − ¯Y¯
2
/(n − 1).
We have chosen m for each problem instance such that the width of the confidence interval divided by the grand sample mean is less than 1%.
The cost savings that we show in Tables 3 and 4 are calculated as follows:
1 P
PP
p=1
CostsSID(p)−CostsSD(p)
CostsSID(p) , with P being the number of problem instances
in the test bed with the parameter as indicated in the table (e.g., P = 72 in test bed 1 if N ∈ {1, 5}), the problem instances being numbered 1, . . . , P , ‘SID’ referring to the optimal SID base stock policy, and ‘SD’ referring to either Opt. for the optimal SD base stock policy, MOD, MYO, or BO2.
Table 3 gives the results for test bed 1. Using a degradation state-depen-dent policy instead of a state-indepenstate-depen-dent policy leads to drastic cost savings of 19.6% on average and 73.4% at maximum (the latter value is not shown in the table). Most of these savings are also achieved by the best-of-two heuristic (BO2), and it is clear that the performance of that heuristic de-pends heavily on the performance of the myopic heuristic (MYO). We come back to this in our discussion of the results on test bed 2.
Other interesting things to notice are that:
• The cost savings reduce with an increasing number of machines (N ). The reason is that the SID base stock policy improves due to pooling effects.
• The cost savings reduce with an increasing lead time (L). If the lead time is 1, the SD policies can stock spare parts as soon as any machine reaches the last degradation state. If the lead time is higher, the SD policies cannot wait so long and will look more like the SID policy.
• The possible cost savings increase drastically with an increasing number of degradation states (I). We come back to this when we discuss the results for test bed 3.
• The degradation probability vectors (DP V ) have no clear influence on the results.
• The emergency costs (cem) and holding costs (ch) have a huge influence
on the cost savings. It seems that the potential cost savings are minor if the ratio of emergency costs over holding costs is small and that they increase if the ratio increases. However, at a certain point, they start decreasing again.
The results for test bed 2 can be found in Table 4. They basically confirm our findings on test bed 1 on a broader range of parameter values (this is why we have used simulation for this test bed). In addition, from this test bed we learn that there are some problem instances on which the myopic (MYO) heuristic performs badly (not shown in the table). For example, for N = 1, L = 5, I = 5, DP V = 100(2), cem = 104, and ch = 103, we see that the
costs of the MYO heuristic are almost five times the costs of the SID policy. This is caused by one of the approximations underlying MYO: it ignores lost sales during L + 1 periods. When the ratio between emergency costs and holding costs is low, a significant number of demands will lead to lost sales. Another interesting example is that of N = 1, L = 5, I = 2, DP V = 100(1), cem = 105, and ch = 1. Here, the costs resulting from using MYO are almost
2.5 times those of the SID policy. This is caused by another approximation in MYO: it assumes that a machine may fail at most once during L + 1 periods. However, in this case, a machine may fail upto 3 times during L + 1 periods ((L + 1)/I = 3). Such an unanticipated failure leads to huge costs if the emergency costs are high. In total, out of the 216 problem instances in test
# Problem Costs Cost savings
Parameter Value instances SID Opt. MOD MYO BO2
N 1 72 193.7 23.9% 7.6% 23.0% 23.2% 5 72 377.5 15.2% 1.7% 14.0% 14.0% L 1 72 278.9 21.7% 9.3% 21.3% 21.3% 2 72 292.2 17.5% 0.0% 15.6% 15.9% I 2 72 285.6 9.6% 0.0% 8.9% 9.0% 3 72 285.6 29.5% 9.3% 28.1% 28.2% DP V 100(1) 48 327.9 21.6% 5.1% 20.0% 20.0% 100(2) 48 327.9 19.5% 5.1% 18.4% 18.5% 250 48 201.0 17.5% 3.6% 17.0% 17.3% (cem, ch) (104, 103) 24 240.0 0.3% 0.0% 0.1% 0.3% (104, 200) 24 152.5 14.2% 0.2% 14.1% 14.1% (104, 1) 24 1.8 23.4% 7.4% 21.5% 22.1% (105, 103) 24 1035.9 27.2% 4.5% 26.8% 26.8% (105, 200) 24 281.3 32.6% 7.2% 29.6% 29.6% (105, 1) 24 2.1 19.6% 8.6% 18.8% 18.8% Average 144 285.6 19.6% 4.6% 18.5% 18.6%
Table 3: Results for test bed 1
bed 2, 10 lead to a cost increase of more than 10% when using the myopic heuristic instead of the SID policy. That is why our best-of-two heuristic also considers using the results of the modified SID policy.
Test bed 3 consists of 36 problem instances with three different numbers of degradation states (I): 2, 3 or 5. Table 5 shows the results, with the addi-tional costs being computed as follows: 361 P36
p=1
CostsSD, I ∈ {2, 3}(p)−CostsSD, I = 5(p) CostsSD, I = 5(p) ,
with the problem instances being numbered 1, . . . , 36, ‘SD, I ∈ {2, 3}’ refer-ring to the optimal SD policy with I ∈ {2, 3} and ‘SD, I = 5’ referrefer-ring to the optimal SD policy with I = 5.
We see that huge additional costs are incurred if not all degradation information is used. For example, if we distinguish 3 degradation states only (whereas there are 5 present), this leads to additional costs of more than 25% in more than half of the problem instances. Distinguishing only 2 degradation states leads to maximum additional costs of 567%. This means that if more degradation information is available, it is very costly not to use
# Problem Costs Cost savings
Parameter Value instances SID BO2
N 1 72 193.8 24.8% 5 72 428.4 17.9% 10 72 650.3 13.2% L 2 108 393.8 21.6% 5 108 454.5 15.6% I 2 108 424.4 5.4% 5 108 423.9 31.9% DP V 100(1) 72 493.8 20.7% 100(2) 72 493.8 17.7% 250 72 284.9 17.5% (cem, ch) (104, 103) 36 426.7 0.5% (104, 200) 36 232.5 11.2% (104, 1) 36 2.7 23.5% (105, 103) 36 1485.0 24.3% (105, 200) 36 395.0 27.8% (105, 1) 36 3.1 24.6% Average 216 424.2 18.6%
Table 4: Results for test bed 2
it. The other way around, this means that it may be worthwhile to invest in condition monitor equipment if this leads to the ability to distinguish more failure states.
8. Conclusions and recommendations
We have proposed to incorporate degradation state information in the spare parts inventory control policy. In an extensive numerical experiment
Additional costs I = 3 I = 2
Average 52.2% 137.2%
# instances with add. costs in range 0-25% 17 6 # instances with add. costs in range 0-100% 10 12 # instances with add. costs in range 100-567% 9 18
we have shown that using such a degradation state-dependent policy instead of a state-independent policy may lead to costs savings of about 20% on average and over 70% at maximum. We have proposed three heuristics, one of which turns out to be close to optimal.
An interesting extension of our research would be to incorporate a random replenishment lead time. It is unclear whether or not our heuristic would still perform well in that case. Finding the optimal policy would be more difficult since the state space of the Markov decision problem becomes drastically larger.
It may also be whorthwhile to model the degradation behavior differ-ently. In the field of reliability theory, generally other stochastic processes are assumed than the one that we assumed. However, this would complicate the model to such an extent that it may turn out to be difficult to attain interesting results.
Acknowledgements
The second author gratefully acknowledges the support of The Lloyd’s Register Educational Trust, an independent charity working to achieve ad-vances in transportation, science, engineering and technology education, train-ing and research worldwide for the benefit of all.
Bijvank, M. and Vis, I. F. A. (2011). Lost-sales inventory theory: A review. European Journal of Operational Research, 215(1):1–13.
Christer, A. H. (1982). Modelling inspection policies for building mainte-nance. The Journal of the Operational Research Society, 33(8):723–732.
Deshpande, V., Iyer, A. V., and Cho, R. (2006). Efficient supply chain management at the u.s. coast guard using part-age dependent supply re-plenishment policies. Operations Research, 54(6):1028–1040.
Elwany, A. H. and Gebraeel, N. Z. (2008). Sensor-driven prognostic models for equipment replacement and spare parts inventory. IIE Transactions, 40(7):629–639.
Feeney, G. J. and Sherbrooke, C. C. (1966). The (s − 1, s) inventory policy under compound poisson demand. Management Science, 12(5):391–411.
Hariharan, R. and Zipkin, P. H. (1995). Customer-order information, lead-times, and inventories. Management Science, 41(10):1599–1607.
Jardine, A. K. S., Lin, D., and Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 20:1483–1510.
Law, A. M. (2007). Simulation Modeling & Analysis. McGraw-Hill, New York (NY), international edition.
Louit, D., Pascual, R., Banjevic, D., and Jardine, A. K. S. (2011). Condition-based spares ordering for critical components. Mechanical Systems and Signal Processing, 25:1837–1848.
Morton, T. E. (1971). The near-myopic nature of the lagged-proportional-cost inventory problem with lost sales. Operations Research, 19(7):1708– 1716.
Muckstadt, J. A. (2005). Analysis and Algorithms for Service Parts Supply Chains. Springer, New York (NY).
Rausch, M. and Liao, H. (2010). Joint production and spare part inventory control strategy driven by condition based maintenance. IEEE Transac-tions on Reliability, 59(3):507–516.
Sherbrooke, C. C. (2004). Optimal inventory modelling of systems. Multi-echelon techniques. Kluwer, Dordrecht (The Netherlands), second edition.
Song, J.-S. and Zipkin, P. H. (1993). Inventory control in a fluctuating demand environment. Operations Research, 41(2):351–370.
Tan, T., G¨ull¨u, R., and Erkip, N. K. (2007). Modelling imperfect advance demand information and analysis of optimal inventory policies. European Journal of Operational Research, 177:897–923.
Tijms, H. C. (1986). Stochastic Modeling and Analysis. John Wiley & Sons, New York (NY).
Wang, K., Chu, J., and Mao, W. (2009). A condition-based replacement and spare provisioning policy for deteriorating systems with uncertain deteri-oration to failure. European Journal of Operational Research, 194(1):184– 205.
Wang, L., Chu, J., and Mao, W. (2008). A condition-based order-replacement policy for a single-unit system. Applied Mathematical Model-ing, 32(11):2274–289.
Zipkin, P. H. (2008a). Old and new methods for lost-sales inventory systems. Operations Research, 56(5):1256–1263.
Zipkin, P. H. (2008b). On the structure of lost-sales inventory models. Op-erations Research, 56(4):937–944.
