Joint Optimization of Production
Scheduling and Condition-Based
Maintenance Planning
Author:
Christian Scholten, S2216728
University of Groningen
Master’s Thesis Econometrics, Operations Research
and Actuarial Sciences
Specialization: Operations Research
Joint Optimization of Production Planning
and Condition-Based Maintenance Scheduling
Christian Scholten
July 15, 2016
1
Introduction
Preventive maintenance is an important activity in maintaining a system to avoid failures, which can result in costly and dangerous situations [18]. Since the automation of production processes is still growing nowadays, an adequate planning of preventive maintenance actions is crucial. Broadly speaking, preventive maintenance can be divided into two groups: time-based maintenance and condition-time-based maintenance. Time-time-based maintenance is mainte-nance based on calendar schedule, an overview of this maintemainte-nance type is given by Jar-dine and Tsang [15]. In this paper we will focus on condition-based maintenance, where maintenance activities are planned based on degradation information. For an overview of based maintenance, see Ahmad and Kamaruddin [1]. Most studies on condition-based maintenance consider the optimization of maintenance activities in isolation. However, in practice, maintenance generally interferes with other processes as production planning, inventory management and scheduling of repairmen. In this paper we consider joint opti-mization of condition-based maintenance and production planning. Production planning is the process of planning the allocation of work over the available production resources in a production environment [12]. To indicate the relevance of production planning, the total revenue in manufacturing in the USA was $ 2.17 billion in 2015 [26]. Optimal allocation of orders can increase productivity and minimize operating cost even more [13].
In the planning of production and maintenance, we face a conflict of interest. The produc-tion planner desires to have the producproduc-tion equipment continuously available, whereas the maintenance planner regularly has to interrupt operations for performing preventive mainte-nance. Here, independently optimizing a production schedule and maintenance planning can result in conflicting strategies. Therefore, combining these planning activities is essential.
In this paper we consider a single machine that processes incoming jobs that require varying processing times. The condition of the machine declines over time, and this will ultimately lead to machine failure. The machine can process one job at a time, and when it is idle, one of the waiting jobs can be started. The order in which the jobs are processed can be chosen strategically. Selecting a shorter job might for instance be preferred when the machine is further deteriorated. To create an incentive to produce jobs, a cost for jobs in the system is introduced. Furthermore, preventive maintenance can be performed when the machine is deteriorated, and when the machine fails, corrective maintenance is required. Jobs that are interrupted because of machine failure, preventive maintenance or a switch to another job need to start all over again.
The aim of this study is to determine optimal policies, that specify when to perform maintenance on the machine and how to schedule the arriving jobs. Moreover, the model described in this paper can be used as benchmark, due to its general approach it can eas-ily be extended. The formulation and findings can be supportive for a policy maker in manufacturing.
Section 5, a case study is presented and discussed to gain key insights into the problem. Lastly, Section 6 provides conclusions and directions for future research.
2
Literature review
The current study considers simultaneously maintenance optimization and production plan-ning. Although both topics are studied extensively in the literature, only a few combine them. For reviews on condition-based maintenance we refer to Ahmad and Kamaruddin [1], Martin [25] and Prajapati et al. [30]; reviews on production planning are provided by Arisha et al. [2], Brucker and Knust [7] and Pochet and Wolsey [29]. In this section we review existing studies that jointly optimize maintenance scheduling and production planning.
In the past decades joint optimization of the production and maintenance scheduling is discussed already [10, 11, 22]. To set the scene, Lee and Rosenblatt [20] address the problem of joint optimization of lot sizes and production cycles. They include the possibility of process deterioration in the classical Economic Manufacturing Quantity (EMQ) model and introduce the lot size as a decision variable. Furthermore, the effects of adjusting the EMQ are exam-ined and the influence of different parameter values is determexam-ined. Another extension of a classical model is discussed by Batun and Maillart [5], where the First-Come-First-Served (FCFS) approach is extended by finding a production policy based on deterioration level of the machine. They use an experimental design to show that the improvement in optimizing a deterioration dependent production policy increases the benefits significantly.
A more advanced approach is presented by Chelbi et al. [9] and Suliman and Jawad [32], who propose a mathematical model to optimize the planning of preventive maintenance and lot size for a single-unit production system. Chelbi et al. [9] conclude that the introduc-tion of failure possibility decreases the producintroduc-tion lot size for a fixed time-based preventive maintenance planning, whereas Suliman and Jawad [32] show that performing preventive maintenance yields a reduction in inventory and quality related costs. As an extension on these papers, Lin et al. [23] introduce inspection errors, imperfect preventive maintenance and minimal repairs for an imperfect production system. They use a numerical procedure to investigate the influence of the inspection errors and imperfect preventive maintenance on the production planning and maintenance scheduling.
significant gains.
In contrast to the mathematical optimization approach, Assid et al. [3] and Nganga Muchiri et al. [27] present a joint optimization of production, setup and maintenance based on simu-lation. Assid et al. [3] propose four structures of a control policy which combines production, setup and maintenance activities. They found that the block replacement policy with jumps according to the inventory level gives the best results, which reduces the necessary setup cost and reduces the risk of shortages. Nganga Muchiri et al. [27] show a benchmark for evaluating maintenance activities and policy determination regarding machine performance improvement. They show that the success of a condition-based maintenance policy mainly depends on the deterioration duration of the equipment.
The following studies use a similar framework to model the joint optimization of main-tenance and production scheduling problem. Kazaz and Sloan [19] present a semi-Markov Process, where the objective is to determine the optimal maintenance and production policy in order to minimize to long-run expected reward per unit of time. However, they assume that jobs are always waiting to be processed. This in contrast to this study where jobs will arrive according to a stochastic process, where it can occur that particular jobs are not available in the queue. Iravani and Duenyas [14] present a Markov Decision Process frame-work of a make-to-stock production and inventory model. The authors focus on finding an optimal policy on when to make-to-stock, make-to-order, maintain or being idle. Iravani and Duenyas [14] show that policies which do not address maintenance and production indeed result in conflicting strategies and therefore bad performance. However, in this paper the focus lies on a single product, where we consider jobs with different processing times.
Lastly, Borrero and Akhavan-Tabatabaei [6] focus on a single machine and a single prod-uct, where they include the production for inventory as well. Two Markov Decision Process models are proposed, where the models are distinguished based on cost decision criteria. In contrast to this study, Borrero and Akhavan-Tabatabaei [6] consider time-based main-tenance, where this study focus on condition-based maintenance. The main contribution of this study to current literature, is that a queue is introduced where jobs with varying processing times are stored until they are processed. The optimal policy will indicate when to process which job and when to perform maintenance based on the deterioration level of the machine and the total set of jobs in the queue.
3
Model Formulation
it available again. When the machine is in a deteriorated state, it is possible to perform preventive maintenance. Both maintenance types restore the machine to the as-good-as-new state.
Jobs that need to be processed arrive according to a stochastic process. At the start of each time period, a job arrives with probability p, independent of the previously arrived jobs. We assume that there are n different job types, with each type having a different processing time. The length of job type i is denoted by yi, and the probability that an incoming job
is of type i is denoted by zi, i = 1, . . . , n. We let y = (y1, . . . , yn) and z = (z1, . . . , zn).
If, for instance, y = (1, 3, 5) and z = 13,13,13, incoming jobs either have length 1, 3 or 5, all with equal probability. Arriving jobs are placed in a queue and the machine can start to process a job from this queue. We let Qi represent the number of jobs of type i in the
queue, i = 1, . . . , n. We let Q = (Q1, Q2, . . . , Qn) denote the total queue with waiting jobs.
Furthermore, we assume that jobs are non-resumable [21], which means that when a job is interrupted due to machine failure, preventive maintenance or a switch to another job, it should be reprocessed completely.
We consider the problem from a cost perspective and distinguish maintenance costs and costs for jobs that are still in the system. No direct costs for machine processing are incurred, but the maintenance costs are an immediate consequence of machine deterioration caused by machine usage. We prefer to perform preventive maintenance over corrective maintenance, since we assume that performing preventive maintenance is less expensive than corrective maintenance. This assumption is quite common in the literature [4, 16, 17]. We denote ccm
as the cost of performing corrective maintenance and cpmas the cost of performing preventive
maintenance.
The cost for jobs being in the system is included to create an incentive to finish jobs. We determine the Total Weighted Flow-time (TWF), which measures the time being in the system for each job, weighted according to their processing time. Consequently, jobs could be processed in random order when maintenance is not required. For an overview of algorithms to solve TWF problems, see Chekuri et al. [8]. We denote the cost parameter corresponding to the TWF by cq.
4
Markov Decision Process Formulation
4.1
State space
The state of the system can completely be described by the current deterioration level of the machine, the number of jobs per job type that are present in the queue, the remaining processing time of the current job (if any) and the job type of the job that is currently processed (if any). The latter is needed because the job that is processed might be interrupted by a breakdown of the machine, by preventive maintenance or because we choose to switch to another job. In these cases the current job should be returned to the the queue and its original length should be stored.
We let X ∈ {1, 2, . . . , m+1} denote the current deterioration level of the machine. We let L ∈ {0, 1, ..., yn− 1} denote the remaining processing time of the current job on the machine,
where L = 0 represents the machine being idle. The maximum value of L equals yn − 1,
because when we start to process a job with maximum length yn, the remaining processing
time at the start of the next decision epoch equals yn− 1. We let Y ∈ {0, 1, 2, . . . , n} denote
the type of the current job on the machine, where Y = 0 represents an idle machine.
The number of jobs of type i in the queue is denoted by Qi, i = 1, . . . , n, and the state of
the entire queue is described by Q = (Q1, . . . , Qn). We define w as the maximum number of
jobs in the queue for each type of job. Ideally, the queue should not be limited. However, to model the problem as a Markov Decision Process the state space should be finite. Therefore, we assume that no more than w jobs of each job type can be waiting in the queue, i.e. Qi ∈ {0, 1, . . . , w}, i = 1, . . . , n. Here, w should be chosen sufficiently high, to obtain a good
approximation of the original system.
Finally, a state can be described by s = (X, L, Y, Q), and the state space S equals
S = {s = (X, L, Y, Q) : X ∈ {1, 2, . . . , m + 1}, L ∈ {0, 1, . . . , yn− 1},
Y ∈ {0, 1, 2, . . . , n}, Qi ∈ {0, 1, . . . , w}, i = 1, . . . , n}.
4.2
Action space
The action space can be described by all the actions that are possible to perform. Firstly, we can do nothing (DN). Secondly, we can perform preventive maintenance (PM). Thirdly, corrective maintenance (CM) can be performed. Fourthly, we can continue to process the current job on the machine (PJ). Lastly, we can decide to start processing a job of type i from the queue (PQi, i = 1, . . . , n). For example, performing action PQ3 corresponds to
returning the current job on the machine (if any) to the queue and select a job of type 3 from the queue. Since selecting a job cost negligible time, the machine immediately starts producing the job. The durations of all actions are assumed to be equal to one time epoch. Now, the action space A can be described by
A = {DN, PJ, PM, CM, PQ1, . . . , PQn}.
4.3
Reward function
We let r(s, a) denote the cost function of performing action a ∈ A when the system is in state s ∈ S. Note that these costs are independent of time. As mentioned is Section 3, performing preventive maintenance costs cpm and parameter ccm is the cost of performing
corrective maintenance. During each time period, a cost for jobs that are in the system will be incurred. This cost consists of the Total Weighted Flow-time (TWF) multiplied by the corresponding unit cost cq. To determine the TWF, we multiply the total number of jobs in
the system by their length yi, i = 1, . . . , n. Note that the total number of jobs in the system
consists of the current job on the machine and the jobs waiting in the queue. We introduce indicator 1Y =i, which equals 1 when the current job on the machine is of type i, and 0 if
not, i = 1, . . . , n. The indicator adds the current job on the machine to the TWF and will be used to define the transition probabilities. Now, the TWF is given by
TWF =
n
X
i=1
Qi+1Y =iyi.
Furthermore, it can occur that an action is not allowed in a state. Choosing to do nothing is always allowed, but note that the current job on the machine will be interrupted. Contrarily, if the machine is idle (i.e. when L = 0) it will not be possible to process the current job on the machine. We can only perform preventive maintenance when the machine is operational (i.e. when X ≤ m) and corrective maintenance is only allowed when the machine has failed (i.e. when X = m). Starting to process a job of type i is only possible when a job of type i is available in the queue (i.e. when Qi > 0), i = 1, . . . , n. To prohibit actions in states
where these are not allowed, we introduce three indicators. Indicator1X=m+1equals 1 when
the machine in the failed state, and 0 otherwise. Indicator 1L=0 equals 1 when the machine
is idle, and 0 otherwise. Lastly, 1Qi=0 equals 1 when no jobs of type i are currently waiting
in the queue, and 0 otherwise, i = 1, . . . , n. We incur a sufficiently high cost M for actions that are not allowed in a certain state. The cost function r(a, s) is now given by
4.4
Transition probabilities
We let p(j|s, a) denote the transition probability of going from state s ∈ S at decision epoch t to state j ∈ S at decision epoch t + 1, when action a ∈ A is chosen at decision epoch t. Let s0 = (X0, L0, Y0, Q0) with Q0 = (Q01, . . . , Q0n) denote the state at the next decision epoch. We will now define the transition probabilities of going from state s ∈ S to state s0 ∈ S by performing action a ∈ A.
First, we note that the number of jobs in the queue are influenced by two processes. The first consists of jobs that are assigned to the machine and that are returned to the queue because of machine failure, preventive maintenance or a switch to another job. The second is the arrival of new jobs. We use QI
i and QJi to denote the intermediate number of jobs of
type i in the queue. QIi represents the number of jobs of type i after selecting a job from the queue to process and a potential job that has been set back to the queue, but before a potential new arriving job. QJ
i represents the number of jobs of type i after a potential new
arriving job. Note that QJi is not necessarily equal to Q0i, because all Q0i’s are limited by w. We calculate the transition probabilities via intermediate states, because the returning jobs and job arrivals are two independent processes.
Now, if we choose to do nothing, the machine will not deteriorate and the potential current job on the machine will return to the queue. Then, X0 will be equal to X, L0 and Y0 will be set to zero and the corresponding Qi will be increased by 1, using indicator 1Y =i.
Note that an idling machine remains in the same state if we do nothing. Thus, if a = DN,
p(X0, L0,Y0, QI1, . . . , QIn|s, a) = (
1, if (X0, L0, Y0, QI1, . . . , QnI) = (X, 0, 0, Q1+1Y =1, . . . , Qn+1Y =n),
0, elsewhere.
When we select to perform maintenance, the machine will go to the as-good-as-new state X0 = 1, and the potential current job on the machine will move to the queue as well. Thus, if a ∈ {PM, CM}, p(X0, L0,Y0, QI1, . . . , QIn|s, a) = ( 1, if (X0, L0, Y0, QI 1, . . . , QIn) = (1, 0, 0, Q1+1Y =1, . . . , Qn+1Y =n), 0, elsewhere.
machine will return to the queue. So, if a = PJ, we have, p(X0,L0, Y0, QI1, . . . , QIn|s, a) = P [X, j], if (X0, L0, Y0, Q1I, . . . , QIn) = (j, L − 1, Y, Q1, . . . , Qn), j = 1, 2, . . . m, P [X, m + 1], if (X0, L0, Y0, QI1, . . . , QnI) = (m + 1, 0, 0, Q1+1Y =1, . . . , Qn+1Y =n), 0, elsewhere.
The final action is to start processing a job from the queue. When a job is assigned to the machine, the machine will deteriorate, because the machine will start to process this job immediately. Consequently, the current job on the machine will return to the queue. We introduce indicator 1a=PQi, which equals 1 when the current action is to select a job of type
i and 0 elsewhere, i = 1, . . . , n. Indicator 1a=PQi decreases the corresponding queue by 1,
to indicate a job moving from the queue to the machine. Note that, when the machine fails during the process of the selected job, the job will immediately return to the queue. Then, for i = 1, . . . , n, if a = PQi we have, p(X0,L0, Y0, QI1, . . . , QIn|s, a) = P [X, j], if (X0, L0, Y0, QI1, . . . , QIn) = (j, yi− 1, i, Q1+1Y =1−1a=PQ1, . . . , Qn+1Y =n−1a=PQn), j = 1, 2, . . . , m, P [X, m + 1], if (X0, L0, Y0, QI1, . . . , QnI) = (m + 1, 0, 0, Q1+1Y =1, . . . , Qn+1Y =n), 0, elsewhere .
Regardless of the action a, we face the possibility of an arrival. A job arrives with probability p, and the probability that the job is of type i is zi. The transition probability
to QJi depends on the value of QIi, because QIi represents the number of jobs of type i after a potential job has been set back to the queue but before a potential job arrives. So, for all a ∈ A we have p(QJ1, . . . , QJn|QI 1, Q I 2, . . . , Q I n, a) = pz1, if (QJ1, QJ2, . . . , QnJ) = (QI1+ 1, QI2, . . . , QIn), pz2, if (QJ1, QJ2, . . . , QnJ) = (QI1, QI2+ 1, . . . , QIn), .. . ... pzn, if (QJ1, QJ2, . . . , QnJ) = (QI1, QI2, . . . , QIn+ 1), (1 − p), if (QJ 1, QJ2, . . . , QJn) = (QI1, QI2, . . . , QIn), 0, elsewhere.
Limiting the queue by w has implications for the transition probabilities when the queue is occupied. Since we limited the number of jobs in the queue for each job type, an arriving job is rejected when the queue is occupied. This holds for jobs which return from the machine as well. Therefore we have:
However, limiting the queues implies that the queues can be occupied in a certain point of time. An occupied queue gives the advantage of rejecting possible arriving jobs. Therefore, limiting the queues may only have a small effect such that a suitable arrival process will be determined. We now have that the Markov Decision Process is defined by the tuple
{T, S, A, p(·|s, a), r(s, a)}.
The aim is to minimize the long-run expected total cost, by finding an optimal policy π∗. This can formally be stated as
π∗ ∈ arg min π E ∞ X t=1 r(st, π(st)) ! . (1)
In Equation (1), st denotes the state at decision epoch t and π(st) denotes the action given
state stat decision epoch t. An ε-optimal policy π∗ is determined by using the value-iteration
algorithm, see Puterman [31].
4.5
Illustration of Markov Decision Process
To illustrate the Markov Decision Process, a possible development with the most important actions and events of the process is presented in Figure 1. Here, we consider a machine with m = 4 deterioration states and two different job types with lengths y = (1, 4) and z = (12,12). At decision epoch T = t0, the selected action is to process the current job on the machine,
albeit the machine is already in deterioration state X = 3. Unfortunately, during the process the machine fails, which implies that in the succeeding decision epoch the machine is in the failed state X = m + 1. The remaining job length L and job type on the machine Y are forced to zero, because the current job on the machine is returned to the queue. This can be observed in the increase of Q2 as well. Because of performing corrective maintenance, the
machine is in the as-good-as-new state X = 1 at decision epoch t0 + 2. Then, action P Q2
is selected to start a job of type 2, because we have an idling machine and jobs of type 2 waiting.
At the following decision epoch t0+ 3, we can conclude that a job of type 2 is assigned
to the machine and it is processed successfully, because the job type on the machine Y is set to 2 and the remaining job length L equals 3. In the succeeding time epoch we continue to process the job. The state of the machine declines to deterioration level X = m and a job of type 1 arrives, which is added to Q1. Then, we choose to interrupt the current job on the
machine by substituting it for a smaller job. Since the machine is in the most deteriorated state and the current job on the machine has substantial remaining processing time L, it appears to be better to try to finish a smaller job instead of a larger job. The current job on the machine is returned to the queue and a job of type 1 is assigned to the machine. At the next decision epoch t0+ 5, we see that the job of type 1 is processed successfully, because
Figure 1: Possible development of the MDP.
5
Results
In this section a case study is presented, where the main characteristics and implications of the problem will become apparent. First, we will examine some consequences of the modeling choices. We continue to discuss the optimal actions for a machine being idle and the optimal actions for a processing machine. Then, we show the influence of varying the cost of preventive maintenance.
The parameter values, which are used in the case study, are presented in Table 2. Dete-rioration matrix P is used to model the decline of the machine. The values are chosen such that all aspects of the model can be discussed.
Table 1: Parameter values.
Parameter Value Parameter Value
w 6 cpm 0.7 m 3 ccm 1 y (1, 3, 5) p 0.1 z (13,13,13) cq 0.1 P = 0.895 0.1 0 0.005 0 0.7 0.2 0.1 0 0 0.7 0.3 0 0 0 1
a job arrives or returns from the machine and faces an occupied queue. We determine the probability of being in a state where the maximum number of waiting jobs for each job type is higher than a certain threshold. We would like to the discuss the probability of being in
Table 2: Probability of being in state with minimal i waiting jobs.
i p(max(Q) ≥ i) 6 7.30 · 10−7 5 1.01 · 10−5 4 9.94 · 10−5 3 1.02 · 10−3 2 1.13 · 10−2 1 0.20 0 1
a state with more than 5 waiting jobs of one type. We consider these states with an almost occupied queue (Qi ≥ 5) as undesirable, since a returning job results in an occupied queue.
Consequently, this creates an incentive to perform preventive maintenance. Then, arriving jobs of similar job type will be rejected, which implies that no cost will be incurred for this job. Note that this discourages to start processing a job from an occupied as well.
5.1
Policies for an idle machine
Figure 2 provides the optimal policies for a machine being idle as a function of the number of jobs in the queue (per job type) and the deterioration level of the machine. Here, each point indicates an optimal action subject to the number of waiting jobs and deterioration level of the machine. On the axes, we denote the number of waiting jobs in the queue for each job type.
type 1 instead of selecting larger jobs, since we are reluctant to start with large jobs of type 2 or 3. That is, when jobs of type 1 are absent, the optimal action is to perform preventive maintenance. This can be explained by the reluctance of starting a larger job in a deteriorated state and the corresponding high probability of a breakdown during the job. The optimal actions in the most deteriorated state (X = 3) and the failed state (X = 4) are as we might expect. In Figure 2(c), one can see that for all combinations of waiting jobs in the queue, it is optimal to perform preventive maintenance. Lastly, the optimal actions given that the machine has failed are presented in Figure 2(d). Note that the machine is not able to process a job in this state and therefore it is optimal to perform corrective maintenance for all possible combinations of waiting jobs in the queue.
Elaborating on selecting jobs of different types, an example case is presented where we have an idling machine and currently jobs are waiting of each job type. We compare the optimal action across the deterioration state of the machine. Note that the following results hold for comparable cases, where an equal number of jobs are waiting for each job type. When the machine is as-good-as-new, it is optimal to select a job of type 2. Starting a job in the as-good-as-new state has a higher probability to be finished without interruption. Thus, the probability of losing partly finished work is relatively low. Contrarily, when the machine is in a more deteriorated state, X = 2, it is optimal to select a job of type 1 to avoid a breakdown during a job. In the deterioration states X = 3 and X = 4, we prefer to perform maintenance to make the machine as-good-as-new.
Table 3: Idle machine, job selection
X L Y Q1 Q2 Q3 Optimal Action
1 0 0 1 1 1 P Q2
2 0 0 1 1 1 P Q1
3 0 0 1 1 1 P M 4 0 0 1 1 1 CM
5.2
Policies for a machine in progress
To show the difference between performing preventive maintenance for different jobs types, we consider the situation where we have started a job in the last time epoch, see Table 4. We consider only jobs of type 2 and 3, since a job of type 1 is finished after one successful processing time epoch. When the machine is in the as-good-as-new or second deterioration state, X = 1 or X = 2, it is optimal to continue processing the job on the machine for both job types. However, when the machine is in most deteriorated state, X = 3, jobs of type 2 are aimed to be finished and jobs of type 3 are interrupted by performing preventive maintenance. This result can be explained by the fact that the probability of a breakdown and the corresponding corrective maintenance cost ccm are larger than the potential benefits
0 1 2 3 0 1 2 3 0 1 2 3 # of jobs type 1 # of jobs type 2 # jobs of type 3 ● (a)X = 1 0 1 2 3 0 1 2 3 0 1 2 3 # of jobs type 1 # of jobs type 2 # jobs of type 3 (b)X = 2 0 1 2 3 0 1 2 3 0 1 2 3 # of jobs type 1 # of jobs type 2 # jobs of type 3 (c)X = 3 0 1 2 3 0 1 2 3 0 1 2 3 # of jobs type 1 # of jobs type 2 # jobs of type 3 (d)X = 4 ● DN PM CM PJ PQ1 PQ2 PQ3
Table 4: Differences between job types X L Y Q1 Q2 Q3 Optimal Action 1 2 2 0 0 0 PJ 2 2 2 0 0 0 PJ 3 2 2 0 0 0 PJ 4 2 2 0 0 0 CM 1 4 3 0 0 0 PJ 2 4 3 0 0 0 PJ 3 4 3 0 0 0 PM 4 4 3 0 0 0 CM
Since the model is specified such that selecting a new job on the machine during the production of the current job is allowed, Table 5 is presented. We consider the same states as in Table 4, though now we consider waiting jobs in the queue. This allows for selecting a job from the queue to substitute the current job on the machine. One can see the contrast between job types 2 and 3. For a job of type 2, we continue processing the job for X = 1 and X = 2, and we perform maintenance otherwise. The optimal action for a job of type 3 is to continue to process the job when the machine is in the as-good-as-new state X = 1 as well. However, when the machine is in state X = 2, the optimal action is to switch to a smaller job of type 1. The optimal action suggests that the high probability of a breakdown during a large job causes that finishing a smaller job is a better solution, even though we discard the current progress of the large job.
Table 5: Differences between job types 2
X L Y Q1 Q2 Q3 Optimal Action 1 2 2 1 1 1 PJ 2 2 2 1 1 1 PJ 3 2 2 1 1 1 PM 4 2 2 1 1 1 CM 1 4 3 1 1 1 PJ 2 4 3 1 1 1 PQ1 3 4 3 1 1 1 PM 4 4 3 1 1 1 CM
5.3
Influence value of preventive maintenance cost
optimal action. When the machine is idle, preventive maintenance is performed, mainly because it is less expensive than corrective maintenance. We can relax the assumption that preventive maintenance is less expensive than corrective maintenance. In Table 6 we vary the value of cpm, where we consider the same states of the process as in Table 5. As a
logical general observation, the higher the cpm, the less frequently preventive maintenance
is the optimal action. However, setting the preventive maintenance cost equal to corrective maintenance, cpm = ccm, does not imply that it is no longer optimal to perform preventive
maintenance. This suggests that preventive maintenance is mainly used to interrupt a job in an early stage to prevent failure in a later stage. To emphasize the importance of interruption of jobs, the cost of preventive maintenance is increased even more. Then, we still prefer to interrupt a large job, but preventive maintenance is too expensive to perform. Therefore we either continue the job or interrupt it by selecting a smaller job. This implies that we aim to finish smaller jobs, and when the machine fails during the process, the lost progress on the job is relatively small.
Table 6: Optimal action varying cpm
X L Y Q1 Q2 Q3 cpm= 0.4 cpm= 0.7 cpm = 1 cpm= 2 1 2 2 1 1 1 PJ PJ PJ PJ 2 2 2 1 1 1 PJ PJ PJ PJ 3 2 2 1 1 1 PM PM PJ PJ 4 2 2 1 1 1 CM CM CM CM 1 4 3 1 1 1 PJ PJ PJ PJ 2 4 3 1 1 1 PM PQ1 PQ1 PJ 3 4 3 1 1 1 PM PM PM PQ1 4 4 3 1 1 1 CM CM CM CM
6
Conclusion
process balances between the reward of finishing a job and the cost of a possible breakdown of the machine. A breakdown has two consequences: the achieved progress of the current job on the machine will be lost and corrective maintenance must be performed to make the machine operable again. The benefit of finishing a job is a reduction in the total cost for jobs in the system. The model finds an optimal policy between finishing the job on the machine and planning maintenance.
In general, we can conclude that small jobs are aimed to be finished, and large jobs are more likely to be interrupted by breakdowns or preventive maintenance to avoid lost work. Here, the Markov Decision Process suggests to select larger jobs when the machine is considered to be as-good-as-new and tends to select smaller jobs in deteriorated states. When the machine is in the most deteriorated state, it is recommended to first perform preventive maintenance and thereafter start with a large job. To emphasize the importance of early interruption of a job, it can be beneficial to substitute the current large job for a smaller job, since the probability of finishing the smaller job is substantially higher. This is confirmed by performing preventive maintenance to interrupt the large job, even when the cost of preventive maintenance are high.
The model described in this paper can be used in situations where jobs can be differen-tiated based on processing times. The MDP is constructed in such a way that can be used as benchmark to help policy makers in making scheduling decisions, since it can be easily extended. As a consequence of these modeling choices, in theory, jobs can stay in the system permanently, since the jobs are not differentiated by time of arrival. Although this can be an essential part of the problem, the model presented focuses on simplicity and generality. In addition, the model is constructed such that we can indicate the influence of introducing the possibility to select jobs with different processing times. Therefore, we considered the total weighted flow-time to enable analysis based on a MDP. However, other methods are expected to be needed when this criterion or other model properties are considered.
In future research it would be interesting to consider resumable jobs instead of non-resumable jobs. A new action can be determined, which interrupts the current job on the machine. This job will wait in a temporary queue, after which the job can be continued. Then, it should be more profitable to switch between jobs given the deterioration level of the machine. Next, we could add a due date for each job. If this due date is exceeded, a penalty cost will be incurred. This will make the model more complex, since the cost function has to be altered. Note that we could add even more information to a job type, such as precedence constraints or setup costs.
7
Bibliography
[1] Ahmad, R. and S. Kamaruddin (2012). An overview of time-based and condition-based maintenance in industrial application. Computers Industrial Engineering 63, 135 – 149.
[2] Arisha, A., P. Young, and M. El Baradie (2001, July). Job shop scheduling problem: an overview. In International Conference for Flexible Automationand Intelligent Manufac-turing (FAIM 01), pp. 682 693.
[3] Assid, M., A. Gharbi, and A. Hajji (2015). Joint production, setup and preventive main-tenance policies of unreliable two-product manufacturing systems. International Journal of Production Research 53 (15), 4668–4683.
[4] Barlow, R. and L. Hunter (1960). Optimum preventive maintenance policies. Operations Research 8 (1), 90–100.
[5] Batun, S. and L. M. Maillart (2012). Reassessing tradeoffs inherent to simultaneous maintenance and production planning. Production and Operations Management 21 (2), 396–403.
[6] Borrero, J. S. and R. Akhavan-Tabatabaei (2013). Time and inventory dependent opti-mal maintenance policies for single machine workstations: An mdp approach. European Journal of Operational Research 3 (228), 545–555.
[7] Brucker, P. and S. Knust (2012). Complex Scheduling (2 ed.). Springer. The German Society of Operations Research.
[8] Chekuri, C., S. Khanna, and A. Zhu (2001). Algorithms for minimizing weighted flow time. In Proceedings of the thirty-third annual ACM symposium on Theory of computing, pp. 84–93. ACM.
[9] Chelbi, A., N. Rezg, and M. Radhoui (2008). Simultaneous determination of production lot size and preventive maintenance schedule for unreliable production system. Journal of Quality in Maintenance Engineering 14 (2), 161–176.
[10] Chen, J.-S. (2008). Scheduling of nonresumable jobs and flexible maintenance activ-ities on a single machine to minimize makespan. European Journal of Operational Re-search 190 (1), 90 – 102.
[11] Chen, W.-J. (2009). Minimizing number of tardy jobs on a single machine subject to periodic maintenance. Omega 37 (3), 591 – 599.
[13] Herrmann, J.W. (2010). The perspectives of taylor, gantt, and johnson: How to im-prove production scheduling. International Journal of Operations and Quantitative Man-agement 16 (3), 243–254.
[14] Iravani, S.M.R. and I. Duenyas. Integrated maintenance and production control of a deteriorating production system. IIE Transactions 34 (5), 423–435.
[15] Jardine, A.K.S. and A.H.C. Tsang (2013). Maintenance, replacement, and reliability: theory and applications. CRC press.
[16] Jiang, R. and A.K.S. Jardine (2007). An optimal burn-in preventive-replacement model associated with a mixture distribution. Quality and Reliability Engineering Interna-tional 23 (1).
[17] de Jonge, B., A. S. Dijkstra, and W. Romeijnders (2015). Cost benefits of postponing time-based maintenance under lifetime distribution uncertainty. Reliability Engineering & System Safety 140, 15 – 21.
[18] Juang, M.G. and G. Anderson (2004). A bayesian method on adaptive preventive maintenance problem. European Journal of Operational Research 155 (2), 455–473.
[19] Kazaz, B. and J. A. Sloan (2013). The impact of process deterioraton on production and maintenance policies. European Journal of Operational Research 227 (1), 88–100.
[20] Lee, H.L. and M.J. Rosenblatt (1987). Simultaneous determination of production cycle and inspection schedules in a production system. Management Science 33 (9), 1125–1136.
[21] Leung, J.Y.T (2004). Handbook of Scheduling: Algorithms, Models, and Performance Analysis (1 ed.). Chapman and Hall/CRC.
[22] Liao, C.J. and W.-J. Chen (2003). Single-machine scheduling with periodic maintenance and nonresumable jobs. Computers & Operations Research 30 (9), 1335 – 1347.
[23] Lin, Y.-H., J.-M. Chen, and Y.-C. Chen (2011). The impact of inspection errors, imper-fect maintenance and minimal repairs on an imperimper-fect production system. Mathematical and Computer Modelling 53 (9-10), 1680–1691.
[24] Liu, X., W. Wang, and R. Peng (2015). An integrated production, inventory and pre-ventive maintenance model for a multi-product production system. Reliability Engineering & System Safety 137, 76–86.
[25] Martin, K. F. (1994). A review by discussion of condition monitoring and fault diagnosis in machine tools. International Journal of Machine Tools & Manufacturing 34 (4), 527– 551.
[27] Nganga Muchiri, P., L. Pintelon, H. Martin, and P. Chemweno (2014). Modelling maintenance effects on manufacturing equipment performance: results from simulation analysis. International Journal of Production Research 52 (11), 3287–3302.
[28] Panagiotidou, S. and G. Tagaras (2012). Optimal integrated process control and mainte-nance under general deterioration. Reliability Engineering and System Safety 104, 58–70.
[29] Pochet, Y. and L.A. Wolsey (2006). Production Planning by Mixed Integer Programming (1 ed.). Springer.
[30] Prajapati, A., J Bechtel, and S. Ganesan (2012). Condition based maintenance: a survey. Journal of Quality in Maintenance Engineering 18 (4), 384–400.
[31] Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Pro-gramming (1st ed.). New York, NY, USA: John Wiley & Sons, Inc.
[32] Suliman, S. M. A. and S. H. Jawad (2012). Optimization of preventive maintenance schedule and production lot size. International Journal of Production Economics 137 (1), 19–28.
[33] Wang, W. (2012). An overview of the recent advances in delay-time-based maintenance modelling. Reliability Engineering & System Safety 106, 165 – 178.
