Joint Optimization of Condition-Based Maintenance and Vessel Routing for Offshore Wind Farms

Joint Optimization of Condition-Based

Maintenance and Vessel Routing for

Offshore Wind Farms

Author:

Roelien Brakke (2233937) University of Groningen

Master’s Thesis Econometrics, Operations Research and

Actuarial Studies

Specialization: Operations Research

Joint Optimization of Condition-Based

Maintenance and Vessel Routing for Offshore Wind

Farms

Roelien Brakke

September 27, 2016

Abstract

Contents

1. Introduction 4

2. Literature Review 5

3. Problem Description 5

4. Analysis 6

4.1. Markov decision process . . . 7

4.2. Value iteration . . . 9

5. Results 10 5.1. Complexity . . . 10

5.2. Benchmark instance . . . 11

5.3. Other instances . . . 17

5.4. Joint versus separate optimization . . . 18

6. Conclusions and future extensions 23 A. Appendix 24 A.1. Traveling Salesman Problem . . . 24

Nomenclature

n number of turbines

m + 1 number of states per turbine k number of decision epochs

cpm cost of performing preventive maintenance (PM) on a turbine

c_cm cost of performing corrective maintenance (CM) on a turbine c_down downtime cost per day of a turbine

csetup setup cost of using a vessel

c_trav cost per distance unit of using a vessel x_i, y_i x- and y-coordinate of turbine i dij distance between two locations i and j

t_maint amount of time required for performing maintenance on a turbine

t_max length of the time window in which maintenance can be performed β cruising speed of a vessel

1. Introduction

Wind power has been used for centuries and from the late nineteenth century electricity is produced by means of wind turbines. In the early nineties the first offshore wind farm has been built in Europe [13] and the offshore wind farm industry has grown significantly ever since. Offshore wind turbines can generate more electricity than onshore turbines, due to the higher and more consistent wind speeds at sea.

Wind turbines that are in use deteriorate and need to be maintained. Two types of main-tenance can be carried out: preventive and corrective mainmain-tenance. Preventive mainmain-tenance can be performed on turbines that are deteriorated but not failed, corrective maintenance should be performed on failed turbines. Due to the larger damage of a failed turbine than of a deteriorated turbine, the cost of performing corrective maintenance is higher than that of preventive maintenance. Therefore, maintaining turbines preventively is preferred over cor-rective maintenance. Offshore operations require specialized and expensive resources such as service vessels [7]. Consequently, carefully scheduling maintenance actions is more important for offshore than for onshore wind farms.

Preventive maintenance actions can be scheduled based on time or based on conditions of equipment [1]. The advantage of time-based maintenance is the easy implementation as only time has to be recorded. The disadvantage is that maintenance is sometimes performed too early, and that breakdowns are likely to occur when equipment deteriorates faster than expected. Condition-based maintenance policies, on the other hand, schedule maintenance actions based on actual conditions of equipment. In this paper we focus on condition-based maintenance. For performing condition-based maintenance on offshore wind farms, either remote condition monitoring should be available or inspections need to be performed. In this paper we assume that condition information is collected through remote monitoring [18]. Although remote condition monitoring has a high cost, it could lead to a large reduction in the maintenance cost, making it financially viable.

Due to the high setup cost of a service vessel it is likely that it is beneficial to cluster maintenance actions. This benefit is even further increased because a transportation cost has to be paid per distance unit that the vessel is traveling and because the turbines are close to each other relative to the shore. Based on the condition of all turbines it should be determined whether and when maintenance is carried out, which turbines are maintained, and in what order. The order is important since it can reduce the total travel distance, and thus can decrease the transportation cost.

We provide a model that dynamically optimizes the maintenance activities, where the travel cost is also taken into account. We consider a Markov decision process to model the maintenance problem, where each action corresponds to another set of turbines that should be maintained. The cost of an action is the sum of the maintenance cost and the travel cost. For a given set of turbines to maintain, a route along them that minimizes the total travel distance should be used. Such a route is the solution of a traveling salesman problem.

2. Literature Review

The number of studies on the optimization of maintenance planning and routing is signif-icant. However, in most of the studies it is assumed that the locations where maintenance should be performed are known. Anghinolfi et al. [2], D´ıaz-Ram´ırez et al. [9], Zamorano and Stolletz [24], Dhahri et al. [8], Goel and Meisel [10] and Maher et al. [17] all study maintenance planning and routing for settings where it is externally specified on which locations mainte-nance should be performed. Chen et al. [6] study maintemainte-nance planning and routing with given locations, but where the required time to perform maintenance is uncertain. Camci [5] studies maintenance scheduling of geographically distributed assets using prognostics infor-mation. A disadvantage of the methodology used in this study is that it is based on a genetic algorithm, which is a heuristic. L`opez-Santana et al. [16] study combined optimization of maintenance planning and routing, where only preventive maintenance activities are consid-ered. In this study is considered that the time between failures is uncertain, but a two-step iterative approach is used where first the maintenance schedule is determined and then the routing model is optimized, instead of jointly optimizing them.

Shafiee [21] presents an overview of the studies on the optimization of maintenance schedul-ing for offshore turbines. Besnard and Bertlschedul-ing [3] and Scheu et al. [20] study maintenance scheduling for offshore turbines, but they do not take into account the setup and travel cost, that could affect the extent to which maintenance actions are clustered. In these studies the dependence between the turbines arises from the stochastic weather conditions that are con-sidered. Byon et al. [4] modeled the problem as a partially observed Markov decision process, and Scheu et al. [20] study large maintenance strategies for offshore wind farms by means of simulation. Gundegjerde et al. [11] and Li et al. [15] study the problem with uncertain times between failures, but in these studies a static setting is considered, i.e., the mainte-nance schedule is optimized for a single time unit and the costs for postponing a maintemainte-nance action until one of the next time units are not considered. Shafiee et al. [22] study an optimal preventive maintenance strategy for offshore turbine blades, but considered only minor and major damages. They developed a model in order to determine the minimum number of minor damages before sending a maintenance team.

To the best of our knowledge, there are no studies on the simultaneous optimization of maintenance planning and routing for offshore turbines where a stochastic deterioration pro-cess and therefore stochastic times until failures and a dynamic setting, i.e., the maintenance schedule is updated every time step, are considered.

3. Problem Description

Before failure occurs, preventive maintenance can be performed on a turbine; after failure corrective maintenance is required to bring it back to the functioning state. Both maintenance types are assumed to make the turbine as-good-as-new. It is assumed that both preventive and corrective maintenance require the same amount of time t_maint. The cost of preventive maintenance is denoted by c_pmand is equal for all deterioration states; the cost of corrective maintenance is denoted by ccm. Furthermore, because a failed turbine cannot produce energy,

a downtime cost c_down is incurred per day that a turbine is down. Because performing corrective maintenance is at least as expensive as preventive maintenance, i.e., c_cm ≥ c_pm, and because failure results in downtime, preventive maintenance is preferred over corrective maintenance.

At the start of each day, the deterioration states of the turbines are used to determine whether a maintenance activity is initiated, and which turbines are maintained. We assume that a remote condition monitoring system is in use that provides us at the start of each day with the exact deterioration levels of the turbines. The operating costs of this system are fixed and therefore not taken into account.

A vessel with technicians and equipment is used to visit turbines that are to be maintained. Journeys of this vessel start and end at its home base location in the harbor, and maintenance on multiple turbines can be combined into a single journey. We let x_i and y_i denote the x-and y- coordinate of turbine i, i = 0, 1, . . . , n, respectively, where i = 0 represents the home base location. Without loss of generality, we set x₀ = y₀ = 0; the coordinates of the turbines are thus relative to the home base location. The travel distance d_ij between two locations i and j equals the Euclidean distance between these two locations:

d_ij = q

(x_i− x_j)2+ (y_i− y_j)2, i, j = 0, . . . , n.

Maintenance journeys can only take place during daylight and maintenance personnel can only work a certain maximum number of hours per day. Therefore, at each day, a potential maintenance journey should be carried out within a time window with a certain fixed length tmax. Since these maintenance opportunity windows are much shorter than 24 hours and

start at the moment that a maintenance action is chosen, it is assumed that a turbine can still deteriorate on the same day after it has been maintained. Furthermore, no deterioration is assumed to take place during the maintenance opportunity window. A setup cost csetup is

incurred when a maintenance journey takes place and a variable travel cost c_trav is incurred per distance unit the vessel has to travel. The cruising speed of the vessel is denoted by β.

4. Analysis

We model the problem described in Section 3 as a discrete-time discrete-state Markov decision process [19]. The framework of Markov decision processes (MDPs) is appropriate for this problem because of the stochastic conditions of the turbines. Furthermore, when the problem is modeled as an MDP, the maintenance planning can be optimized dynamically, i.e., the maintenance schedule is updated each time unit given the conditions of the turbines. Although the number of states grows exponentially in the number of turbines, which results in large computation times, a big advantage of problems modeled as MDPs is that optimal policies are obtained.

4.1. Markov decision process

The components of an MDP are the set of decision epochs, the set of states, the set of actions, the set of state and action dependent costs and the set of state and action dependent transition probabilities. We continue to define each of these components.

Decision epochs

A decision should be made at the start of each maintenance opportunity window. These moments in time therefore correspond to the decision epochs in our model. The times between consecutive decision epochs are equal to one day. These periods start with the maintenance opportunity window, followed by a period during which deterioration can take place. Our aim is to minimize the long-run cost rate, therefore we consider an infinite horizon.

States

At the start of each decision epoch the state of the system can entirely be described by the deterioration states of the turbines. We let S_i denote the deterioration state of turbine i, i = 1, . . . , n, and we let S denote the state of the system, i.e., S = (S₁, . . . , S_n). The number of states per turbine equals m + 1, hence the state space S consists of (m + 1)n states and equals

S = {S = (S₁, . . . , Sn) : S1, . . . , Sn∈ {1, . . . , m, m + 1}}.

Actions

In the definition of an action we do not distinguish between preventive and corrective maintenance. The difference is expressed in the higher corrective maintenance cost that has to be paid when maintenance is performed on a failed turbine. Thus, for each turbine, the choice is either to maintain it or not. Furthermore, for a given set of turbines to maintain, the turbines are visited in an optimal order such that the travel cost is minimized. Such an order is the solution of a traveling salesman problem (TSP), see Appendix A.1 for details. We thus have that an action is completely specified by the set of turbines that will be maintained.

We let A_i indicate whether turbine i is maintained: A_i = 1 means that turbine i is main-tained and Ai= 0 that it is not, i = 1, . . . , n. Furthermore, we let A denote an action which

is a vector of length n in which entry i indicates whether turbine i is maintained or not, i.e., A = (A₁, . . . , A_n). The action space A consists of 2n actions and equals

A = {A = (A₁, . . . , A_n) : A₁, . . . , A_n∈ {0, 1}}.

set of feasible actions is thus equal to A_feas = ( A ∈ A : f (A) β + tmaint· n X i=1 A_i≤ t_max ) . Costs

We let c(S, A) denote the immediate cost that is incurred if action A ∈ A_feasis chosen when the system is in state S ∈ S and we let A_max= max{A_i : i = 1, . . . , n}. The immediate cost consists of travel costs, maintenance costs, and downtime costs.

The travel cost only depends on the turbines that are maintained, and not on their dete-rioration states. It consists of a fixed and a variable part. The fixed part is only incurred if a maintenance journey is carried out, i.e., if Amax = 1, and is thus equal to csetup· Amax.

The variable part is incurred for each distance unit traveled and equals ctrav· f (A). The total

travel cost, denoted by tc(A), equals

tc(A) = c_setup· A_max+ c_trav· f (A).

The maintenance cost does depend on both the state and the action since an action only indicates which turbines are maintained, but not the involved maintenance types. For each turbine that is maintained, at least the preventive maintenance cost cpmis incurred. The total

preventive maintenance cost equals c_pm·Pn

i=1Ai. For each failed turbine that is maintained,

an additional cost c_cm− c_pm is incurred. The total corrective maintenance cost is equal to (ccm− cpm)

Pn

i=1Ai·1Si=m+1. The total maintenance cost, denoted by mc(S, A), thus equals

mc(S, A) = c_pm· n X i=1 A_i+ (c_cm− c_pm) n X i=1 A_i·1_S i=m+1 = n X i=1 cpm· Ai+ (ccm− cpm)Ai·1Si=m+1
= n X i=1 A_i c_pm+ (c_cm− c_pm)1_S i=m+1
.

The downtime cost c_down is incurred for each turbine that is failed and not yet maintained. The total downtime cost, denoted by dc(S, A), equals

dc(S, A) = cdown· n

X

i=1

(1 − Ai)1Si=m+1.

Note that when a turbine fails during a day, then the downtime cost is incurred for the first time on the next day.

The total immediate cost c(S, A) as a function of the state S and the action A equals c(S, A) = tc(A) + mc(S, A) + dc(S, A).

Transition probabilities We let S0= (S10, . . . , S

0

n) denote the state in the next decision epoch, and we let p(S 0

i | Si, Ai)

to deterioration state S_i0 ∈ {1, . . . , m + 1} when decision A_i ∈ {0, 1} has been taken, i = 1, . . . , n. Since, during a single day, a turbine can deteriorate after it has been maintained, the effect of a maintenance decision and the effect of deterioration can be separated. Since performing maintenance makes a turbine as-good-as-new, the transition probability of turbine i equals the deterioration probability from deterioration state 1 to deterioration state S_i0 if it is maintained, i.e., if Ai= 1, and equals the deterioration probability from deterioration state

S_i to deterioration state S0_i if no maintenance is carried out, i.e., if A_i= 0. Hence, p(Si0 | Si, Ai) =

(

PS_i, S_i0, if A_i= 0, P1, S_i0, if A_i= 1, for S_i, S_i0 ∈ {1, . . . , m + 1} and A_i ∈ {0, 1}, i = 1, . . . , n.

Since we assume that each turbine deteriorates independently of the other turbines, the transition probabilities p(S0 | S, A) of the entire system equal the product of the transition probabilities p(S_i0 | S_i, S_i) of the separate turbines:

p(S0 | S, A) =

n

Y

i=1

p(Si0 | Si, Ai),

where S, S0 ∈ S and A ∈ A_feas.

4.2. Value iteration

We use the value iteration algorithm to determine an optimal policy for our defined Markov decision process. Due to its conceptual simplicity and its ease in implementation, this is the most widely used algorithm for solving MDPs [19]. The algorithm generates a sequence of values v0, v1, v2. . . ∈ R|S|. The value vt is a vector of length |S| and can be interpreted as the minimum total expected costs when t periods are left.

An optimal policy, denoted by π∗, shows for each state S ∈ S an optimal action A ∈ Afeas.

Thus, π(S) is equal to an optimal action for the entire system when it is in state S ∈ S. The span of v, denoted by sp(v), is defined as

sp(v) = max

S∈S v(S) − minS∈Sv(S).

The following value iteration algorithm finds a stationary ε-optimal policy, and an approxi-mation to its value.

Value Iteration Algorithm

1. Set v0 = 0 ∈ R|S| and t = 0, specify ε > 0. 2. For each S ∈ S, compute vt+1(S) by

vt+1(S) = min A∈Afeas    c(S, A) + X S0∈S p S0 | S, A
vt S0
   . 3. If spvt+1− vt< ε,

4. For each S ∈ S, choose π(S) ∈ arg min A∈Afeas    c(S, A) + X S0∈S p S0| S, A
vt+1 _S0
   and stop.

If t∗ is the last iteration, such that spvt

∗

− vt

∗

−1

< ε, then the approximated long-run cost rate equals  max S∈S  vt ∗ (S) − vt ∗ −1 (S)  + min S∈S  vt ∗ (S) − vt ∗ −1 (S)  /2.

5. Results

In this section we analyze various instances of our problem and use the results to derive valuable insights. Our goal is to show how routing affects the optimal maintenance planning and the corresponding costs. In Section 5.1 we briefly discuss the complexity of the problem. In Section 5.2 we observe part of an optimal policy for our benchmark instance and analyze the sensitivity of various parameters. In 5.3 we compare a few instances to show how the maintenance and travel costs affect an optimal policy. Finally, Section 5.4 shows the significant differences in the long-run cost rates between joint and separate optimization of maintenance planning and vessel routing.

5.1. Complexity

The number of states and the number of actions grows exponentially in the number of turbines, that results in large computation times for determining optimal policies, even for small instances. Table 1 shows the cardinality of the set of states and the set of actions for various number of turbines, where we assume four deterioration states, i.e., m = 3.

Table 1: Cardinality of the sets for various instance sizes # turbines n # states |S| # actions |A|

2 16 4

4 256 16

8 65,536 256

16 4,294,967,296 65,536

5.2. Benchmark instance

We consider a set of six turbines, where the turbines are positioned in two rows of three turbines. Furthermore, we assume that the turbines are located on a regular grid. The two centered turbines have the same x-coordinate as the home base location, i.e., x2 = x5 = 0.

The distance between a turbine and its direct neighbor turbines is denoted by δturb. The

distance from the home base location to turbine x₂ is denoted by δ_home. Figure 1 shows a graphical representation of the locations of the turbines. The x- and y-coordinates of the turbines are fully specified by choosing values for the two parameters δhomeand δturb. The

x-and y-coordinate of each of the turbines are as in Table 2.

x y Home base 1 2 3 4 5 6 δhome δturb δturb

Figure 1: Graphical representation of the turbines

Table 2: Turbine locations

i xi yi i xi yi

1 -δ_turb δ_home 4 -δ_turb δ_home+ δ_turb

2 0 δhome 5 0 δhome+ δturb

3 δ_turb δ_home 6 δ_turb δ_home+ δ_turb

The values of the parameters δ_home, δ_turb, β, t_maint and t_maxare chosen such that at most four turbines can be maintained in a single maintenance journey, where it is possible to combine all sets of four turbines. Since m + 1 = 4, the deterioration matrix P of a turbine is a 4 × 4 matrix, P =     0.9 0.06 0.03 0.01 0 0.9 0.06 0.04 0 0 0.9 0.1 0 0 0 1     .

The expected lifetime of one turbine is equal to 1 divided by the deterioration probability of the new state to the failed state, i.e.,

E[lifetime turbine] = 1 P [1, 4].

Hence, with this deterioration matrix the expected lifetime of one turbine is equal to 100 days.

Table 3: Parameter values benchmark instance Parameter Value Parameter Value

m 3 c_down 0.2 ε 0.001 csetup 0.1 δ_home 20 km c_trav 0.01/km δ_turb 5 km t_maint 1h cpm 0.4 tmax 8h c_cm 4 β 20 km/h −10 −5 0 5 10 0 10 20 30 x y Home base 1 2 3 4 5 6

An optimal policy is determined for this instance. Since a system that consists of six turbines can be in 4096 different states, it is not possible to show the entire optimal policy. Therefore, we discuss the main results that holds for this instance and show for some states an optimal action.

It turns out that maintenance is always carried out on bad turbines, if it is allowed to combine them all in a single maintenance journey. Performing maintenance on bad turbines is always preferred over performing maintenance on failed turbines, i.e., if it is not allowed to maintain all bad and failed turbines, maintaining bad turbines has a higher priority than performing maintenance on failed turbines. The reason for this is that it is less expensive to incur a downtime cost for the failed turbine and to prevent that the bad turbine also breaks down than run the risk that the bad turbine is broken down the next day, that results in the higher corrective maintenance cost that are incurred. Furthermore, if all turbines are only in the new or in the good state, maintenance is performed if at least three turbines are good, independent of their locations.

Figure 3 shows an optimal action for eleven different states. Figure 3(a), (b) and (c) show the differences in the optimal actions when one turbine is in various deterioration states. When the turbine that is closest to the home base location is in the good or failed state, no maintenance is performed. If this turbine is in the bad state, it is maintained. The reason for this is that when this turbine is in the good state, the travel cost and maintenance cost are too high in order to bring the good turbine back in the new state. When this turbine is in the failed state, it is less expensive to incur a downtime cost per day and wait until another turbine is deteriorated than incur setup and travel costs for maintaining only one turbine. When the turbine is in the bad state, the turbine is maintained since it is too expensive to run the risk that the turbine is failed the next day, that results in a higher maintenance cost. From Figure (a) and (c) can be derived that if another turbine is in the good or failed state, given the other five turbines are in the new state, also no maintenance is performed since the total costs are then even higher due to the higher travel cost.

Figure 3(d), (e) and (f) show that one failed turbine and one good turbine are clustered in a single maintenance journey if the setup and travel costs are low enough relative to the maintenance cost. Figure 3(g) shows that when the failed turbine cannot be clustered in the next time step with other turbines, it is included in the current maintenance journey. However, Figure (h) shows that if two turbines that are failed and close to each other, such that it is beneficial to cluster these two turbines, in the current maintenance journey a good turbine is included since adding this turbine to the journey has a lower extra cost than adding one of the two failed turbines to the journey. Lastly, Figure (i) shows that if the failed turbines in Figure (h) are good and the good turbine in Figure (h) is bad, then the failed turbine is included in the maintenance journey, since then it is beneficial to cluster the failed turbine instead of one of the good turbines.

−10 −5 0 5 10 0 10 20 30 x y (a) −10 −5 0 5 10 0 10 20 30 x y (b) −10 −5 0 5 10 0 10 20 30 x y (c) −10 −5 0 5 10 0 10 20 30 x y (d) −10 −5 0 5 10 0 10 20 30 x y (e) −10 −5 0 5 10 0 10 20 30 x y (f) −10 −5 0 5 10 0 10 20 30 x y (g) −10 −5 0 5 10 0 10 20 30 x y (h) −10 −5 0 5 10 0 10 20 30 x y (i) −10 −5 0 5 10 0 10 20 30 x y (j) −10 −5 0 5 10 0 10 20 30 x y (k) Home base New state Good state Bad state Failed state Route

Figure 3: Part of an optimal policy for the benchmark instance

We analyze the sensitivity of the cost parameters c_pm, c_setupand c_trav. We analyze how the minimum number of deteriorated turbines before a maintenance journey might be initiated, is affected by the preventive maintenance cost. We analyze for the parameters c_setup and c_trav how sensitive the long-run cost rates are to these parameters.

We determine optimal policies for a sequence of values of cpm, keeping all other parameters

maintenance might be initiated is affected by the preventive maintenance cost. We consider values for c_pm between 0 and 3.

Figure 4 and Figure 5 show how the preventive maintenance cost affects the minimum required number of good and bad turbines, respectively, before maintenance might be initiated in an optimal solution, given that no turbines are failed.

Figure 4 shows that when the turbines are only in the new state or in the good state, even for a very low preventive maintenance cost at least two turbines should be in the good state before maintenance might be initiated. Note that it does depend on the locations of the deteriorated turbines whether for two deteriorated turbines a maintenance journey is initiated. The figure shows also that if one turbine is bad, a maintenance journey is initiated even if no turbines are in the good state. This means that, depending on the location of the bad turbine, a maintenance journey might be initiated when only one turbine is in the bad state and the other turbines are in the new state.

Figure 5 shows that if the number of turbines that are good increases, the minimum required number of bad turbines before maintenance is initiated decreases. The figure shows that when no turbines are in the good state, the preventive maintenance cost should be high before it affects the minimum required number of bad turbines to initiate maintenance. The reason for this is that the preventive maintenance cost is then close enough to the corrective maintenance cost, such that the preventive maintenance cost is too high to perform always preventive maintenance, relative to the corrective maintenance cost.

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 1 2 3 4 0 bad turbines cpm

min # good turbines to initiate maintenance

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0 1 2 3 4 1 bad turbine cpm

min # good turbines to initiate maintenance

0.0 0.5 1.0 1.5 2.0 0 1 2 3 4 0 good turbines cpm

min # bad turbines to initiate maintenance

0.0 0.5 1.0 1.5 2.0 0 1 2 3 4 1 good turbine cpm

min # bad turbines to initiate maintenance

0.0 0.5 1.0 1.5 2.0 0 1 2 3 4 2 good turbines cpm

min # bad turbines to initiate maintenance

Figure 5: Sensitivity of minimum number of bad turbines before maintenance might be initi-ated, given the number of good turbines and given c_pm

We compute the long-run cost rate for a sequence of values of csetup, keeping all other

parameters constant. Similarly, the long-run cost rate is computed for a sequence of values of c_trav, keeping all other parameters constant. Figure 6 shows how different the long-run total cost rate, the long-run maintenance cost rate, the long-run downtime cost rate and the long-run travel cost rate is affected by the setup cost c_setupof a vessel and the travel cost c_trav of a vessel. We consider values for c_setup between 0 and 0.5 and for c_trav between 0 and 0.05. The figure shows the obvious result that the run travel cost rate and hence the long-run total cost rate are more sensitive to c_travthan the long-run maintenance cost rate is. The reason that the long-run maintenance cost rate and the long-run travel cost rate are equally sensitive to csetup is that a small increase in csetup has a much smaller effect on the long-run

cost rates than a small increase in c_trav since that is an increase per distance unit. Thus, a small increase in c_trav results in a large increase in the total travel cost. Furthermore, the long-run maintenance cost rate and the long-run downtime cost-rate are also a little sensitive

to c_setup and c_trav, because an increase in the travel cost ensures that maintenance actions

0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.2 0.4 0.6 0.8 1.0 csetup Long−r un cost per da y 0.00 0.01 0.02 0.03 0.04 0.05 0.0 0.2 0.4 0.6 0.8 1.0 c_trav Long−r un cost per da y Total cost Maintenance cost Downtime cost Travel cost

Figure 6: Sensitivity of c_setup and c_trav

5.3. Other instances

In this section we analyze how sensitive an optimal policy is for changes in the maintenance cost and the travel cost. We analyze various maintenance cost values and travel cost values, where the ratios between c_pm and c_cm and between c_setup and c_trav are kept constant. We consider nine instances where we distinguish between high (H), medium (M) and low (L) maintenance and travel cost. Relative to the bench mark instance, the maintenance cost and the travel cost are once multiplied by 2 and once divided by 2, in Table 4 are the values shown. Each instance name consists of two letters where the first letter indicates the maintenance cost and the second letter indicates the travel cost. Note that the instance M-M is the benchmark instance.

Table 4: Sensitivity of optimal policy instance c_pm c_cm c_setup c_trav

H-H 0.2 2 0.05 0.005 H-M 0.2 2 0.1 0.01 H-L 0.2 2 0.2 0.02 M-H 0.4 4 0.05 0.005 M-M 0.4 4 0.1 0.01 M-L 0.4 4 0.2 0.02 L-H 0.8 8 0.05 0.005 L-M 0.8 8 0.1 0.01 L-L 0.8 8 0.2 0.02

of performing corrective maintenance. The reason for this is that both maintenance costs increase relative to the downtime cost, making it increasingly beneficial to maintain the deteriorated turbines and incur a downtime cost for the failed turbines, instead of maintain the failed turbines and run the risk that the deteriorated turbines also break down.

−10 −5 0 5 10 0 5 15 25 x y

Low Maint Cost − Low Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

Low Maint Cost − Medium Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

Low Maint Cost − High Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

Medium Maint Cost − Low Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

Medium Maint Cost − Medium Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

Medium Maint Cost − High Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

High Maint Cost − Low Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

High Maint Cost − Medium Travel Cost

−10 −5 0 5 10 0 5 15 25 x y

High Maint Cost − High Travel Cost

Figure 7: An optimal action for state S = (4, 3, 2, 1, 2, 4) for several cost settings

5.4. Joint versus separate optimization

This section illustrates the importance of the joint optimization of maintenance planning and vessel routing instead of optimizing them separately. As in section 5.2 and 5.3 the optimal policies are analyzed, we analyze in this section the differences in the long-run cost rate between joint and separate optimization. We first explain what is meant with separate optimization of maintenance planning and vessel routing.

to an action. Such an order is the solution of a TSP.

We determine optimal policies for several instances, where we vary in the cost parameters. For the maintenance cost parameters and the downtime cost parameter two values are chosen and for the setup cost parameter and the travel cost parameter three values are chosen, and optimal policies are determined for all combinations of these parameters. Table 5 shows the parameter values that are considered. We analyze the average long-run cost rates and the changes in the rates when jointly optimized relative to separate optimization. Furthermore, we analyze the minimum and maximum long-run cost rates and the cost parameters that corresponds to these minimum and maximum rates.

Table 5: Parameter values

c_pm c_cm c_down c_setup c_trav

0.4 4 0.2 0.10 0.01/km 0.6 5 0.4 0.15 0.02/km 0.20 0.05/km

Since we have five parameters where three parameters can take two possible values and two parameters can take three possible values, we have solved in total 23· 32 = 72 instances, both for joint and separate optimization. The average long-run cost rates are shown in Figure 8. The long-run total cost per day equals 0.92564 with joint optimization and equals 1.20505 with separate optimization.

0.19665 0.27000 0.41153 0.27001 0.05103 0.00000 0.26644 0.66504 0.00000 0.20000 0.40000 0.60000 0.80000 1.00000 1.20000 1.40000 Joint Separate

PM cost CM cost Downtime cost Travel cost

Figure 8: Average long-run costs per day

jointly optimized instead of separately. The average long-run travel cost rate and the average long-run preventive maintenance (PM) cost rate also decrease, while the average long-run corrective maintenance (CM) cost rate and the average long-run downtime cost rate increase. The long-run PM cost rate decreases with joint optimization, since with joint optimization a maintenance journey is not always initiated when turbines are in the good state, because of the setup and travel costs. With separate optimization only the cost of preventive and corrective maintenance are compared, and since performing PM has a lower cost than performing CM, it is never optimal to wait for a failed turbine before maintenance is carried out. Therefore, no downtime costs are incurred with separate optimization. On the other hand, maintenance journeys are more frequently initiated in case of separate optimization, that results in a higher long-run travel cost rate relative to joint optimization.

The long-run CM cost rate increases with joint optimization, since then a turbine is more often broken down. Deteriorated turbines are not always maintained in case of joint opti-mization, since it is sometimes less expensive to wait for more deteriorated turbines before a maintenance journey is initiated. Then maintenance actions can be clustered such that the cost of initiating a maintenance journey is low enough per maintenance action. However, that results in more failed turbines such that more often CM should be performed.

Since with separate optimization deteriorated and failed turbines are always maintained, as long as it is allowed to combine them in a single maintenance journey, the average long-run downtime cost rate is 0 with separate optimization. The long-run downtime cost rate is larger than 0 with joint optimization, since it is sometimes optimal to let a failed turbine broken down and wait for another deteriorated turbine before a maintenance journey is initiated.

Table 6 shows the changes in the average long-run cost rates when the problem is jointly optimized relative to separate optimization. We let c_lr denote the long-run total cost per day, mcp_lr the long-run preventive maintenance cost per day, mcclr the long-run corrective

maintenance cost per day, dc_lr the long-run downtime cost per day and tc_lr the long-run travel cost per day. For the average long-run downtime cost rate the change is not shown since with separate optimization this rate equals 0.

Table 6: Changes in long-run cost rates when jointly optimized relative to separate optimiza-tion

mcp_lr 27.2% decrease mcclr 54.2% increase

tc_lr 59.9% decrease c_lr 23.2% decrease

Table 6 shows that on average the long-run total cost rate decreases with approximately 23%, when the problem is jointly optimized instead of separately. This shows thus that joint optimization is beneficial relative to separate optimization.

correspond to the minimum and maximum long-run cost rates are shown in Table 7. The parameter values do not affect the minimum and maximum long-run downtime cost rate in case of separate optimization since both rates are close to 0. For all parameter values both long-run downtime cost rates are almost 0, where the rates only differ in the seventh or eight decimal place. This holds also for the minimum long-run downtime cost rate in case of joint optimization. 0.1177 0.3010 0 0.1426 0.6602 0.2160 0.2400 0 0.2699 0.7259 0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000

PM cost CM cost Downtime cost Travel cost Total cost

Joint Separate

Figure 9: Minimum long-run costs per day

0.2804 0.5447 0.1893 0.4470 1.2717 0.3240 0.3000 0 1.2089 1.8329 0.0000 0.2000 0.4000 0.6000 0.8000 1.0000 1.2000 1.4000 1.6000 1.8000 2.0000

PM cost CM cost Downtime cost Travel cost Total cost

Joint Separate

Table 7: Parameter values corresponding to the minimum and maximum long-run costs per day, − means that this parameter does not affect this cost rate

Joint Separate

c_pm c_cm c_down c_setup c_trav c_pm c_cm c_down c_setup c_trav

mcp_lr Min 0.4 4 0.2 0.2 0.05 0.4 - - - -Max 0.6 5 0.4 0.1,0.15 0.01 0.6 - - - -mcclr Min 0.4 4 0.2 0.1 0.01 - 4 - - -Max 0.6 5 0.1 0.2 0.05 - 5 - - -dc_lr Min - - - -Max 0.6 5 0.2 0.15,0.2 0.05 - - - - -tc_lr Min 0.6 4 0.2 0.1 0.01 - - - 0.1 0.01 Max 0.4 5 0.4 0.2 0.05 - - - 0.2 0.05 clr Min 0.4 4 0.2 0.1 0.01 0.4 4 - 0.1 0.01 Max 0.6 5 0.4 0.2 0.05 0.6 5 - 0.2 0.05

In Table 7, the entries with a − mean that these parameters do not affect the correspond-ing optimal long-run cost rates. In case of separate optimization, we observe that the the downtime cost does not affect any long-run cost rate. The reason for this is that in case of separate optimization, it is never optimal to let a turbine broken down, since in the opti-mization of the maintenance schedule the setup and travel costs are not taken into account. Furthermore, the minimum and maximum long-run preventive maintenance cost rates are not affected by the corrective maintenance cost and both long-run corrective maintenance cost rates are not affected by the preventive maintenance cost. You would expect, for instance, that for a higher corrective maintenance cost, earlier preventive maintenance is performed to prevent that a turbine breaks down. On the other hand, you would expect that for a lower preventive maintenance cost, less often corrective maintenance is performed since it is less expensive to perform early preventive maintenance. However, the changes in the parameter values are too small to show this effect in case of separate optimization. With joint optimiza-tion, these long-run cost rates are affected by the other maintenance costs. In order to see this effect with separate optimization, larger values for cpm and ccm should be considered.

With joint optimization, we observe that the minimum long-run PM cost rate is obtained with the highest setup and travel costs, and the maximum long-run PM cost rate is obtained with the lowest setup and travel costs. A low travel cost results in performing more often preventive maintenance, and a high travel cost results in more clustering turbines in a single maintenance journey, such that less often preventive maintenance is performed. Thus, with a lower setup and travel cost turbines are less often broken down since more often preventive maintenance is performed, and with a higher setup and travel cost turbines are more often broken down since less often preventive maintenance is performed. This is also reflected in the effect of the setup and travel costs on the long-run CM cost rate. The lowest setup and travel costs provides the minimum long-run CM cost rate and the highest setup and travel costs provides the maximum long-run CM cost rate. Note that for the maximum long-run PM cost rate it does not matter if csetup equals 0.1 or equals 0.15, since the difference between these

the same maximum long-run downtime cost rate, since the difference between these values is equal to the cost of traveling one distance unit.

The preventive maintenance cost has an opposite effect on the minimum long-run travel cost rate, since a higher preventive maintenance cost results in less often initiating a maintenance journey such that the travel cost decreases. Exactly the other way around holds for the maximum long-run travel cost rate.

6. Conclusions and future extensions

In this paper, the joint optimization of condition-based maintenance actions and vessel routing for offshore wind turbines is studied. The problem is modeled as a Markov decision process, where each action corresponds to another set of turbines that should be visited and maintained during a day. For a given set of turbines to traverse, the order in which they are visited is the solution of a traveling salesman problem. For each action the travel costs are computed, and these costs are taken into account when the maintenance schedule is optimized. Because maintenance journeys should take place within a certain amount of time, not all subsets of turbines can be maintained within a single journey.

We analyzed how an optimal policy is affected by the travel costs. The results show that when the travel costs are significant relative to the maintenance costs, the states and the locations of the turbines have a large effect on the action that is optimal. When a deteriorated or failed turbine is maintained, depends on its location and on the states of the other turbines. Sometimes it is beneficial to wait until more turbines are deteriorated before a maintenance journey is initiated. Then the maintenance actions can be clustered, such that the travel cost per turbine is reduced.

We also considered separate optimization of condition-based maintenance and vessel rout-ing, where first an optimal maintenance schedule was determined and corresponding to the optimal policy, optimal routes are determined. Both models are solved using 72 instances and the results are compared. Although the maintenance cost increases in case of joint opti-mization, a large reduction is found in the travel cost. Consequently, the total cost is reduced as well. This shows the importance of optimizing condition-based maintenance and vessel routing simultaneously.

Now that we have shown that an optimal maintenance planning is affected by the locations of the turbines and the routing costs, future follow-up studies could be devoted to the joint optimization of condition-based maintenance and vessel routing. Although we developed a model to determine an optimal policy, our model can only be applied on wind farms that consist of a few turbines. Future studies could aim to determine decision rules for larger sets of turbines. Exact results can be obtained by, for instance, making use of symmetry in the states to reduce the number of states. Alternatively, heuristics could be developed based on the results that we have found.

Furthermore, we assumed that no planning time is required between the planning of main-tenance actions and initiating a mainmain-tenance journey. This might be unrealistic in certain practical situations. Another extension could be to add a planning time to the model. This extension fits within the framework of Markov decision processes.

maintenance and vessel routing.

A. Appendix

A.1. Traveling Salesman Problem

According to Laporte [14], an instance of the traveling salesman problem (TSP) consists of a set of cities and their corresponding distances and a route should be constructed that minimizes the total distance such that each city is visited exactly once, the route starts and ends at the same city, and the route does not consist of subtours. In our problem the home city is the harbor where the vessel should start and end within one day. Equations (1) to (7) show the mathematical model of the TSP. First a new set that can be derived from A, the decision variables and the objective function are defined.

Set

Let N be the set of turbines for which an optimal route should be computed, N = {i ∈ {1, . . . , n} : A_i = 1}.

Decision variables

z_ij = (

1 if turbine j is visited directly after turbine i, 0 otherwise.

u_i = sequence number in which turbine i is visited. Objective function f (A) = X i,j∈{0,N } d_ijz_ij Mathematical model Minimize f (A) (1) such that X i∈{0,N } z_ij = 1, ∀j ∈ {0, N } (2) X j∈{0,N } z_ij = 1, ∀i ∈ {0, N } (3) u_i− u_j+ |N | · z_ij ≤ |N | − 1, ∀i, j ∈ N (4) 1 ≤ ui≤ |N |, ∀i ∈ N (5) zij ∈ {0, 1}, ∀i, j ∈ {0, N } (6) u_i∈ Z, ∀i ∈ N (7)

References

[1] Ahmad, R. and Kamaruddin, S. (2012). An overview of time-based and condition-based maintenance in industrial applications. Computers & Industrial Engineering 63 (1), 135– 149.

[2] Anghinolfi, D., Paolucci, M., and Tonelli, F. (2016). A vehicle routing problem with time windows approach for planning service operations in gas distribution network of a metropolitan area. IFAC-PapersOnLine 49-12, 1365–1370.

[3] Besnard, F. and Bertling, L. (2010). An approach for condition-based maintenance opti-mization applied to wind turbine blades. IEEE Transactions on Sustainable Energy 1 (2), 77–83.

[4] Byon, E., Ntaimo, L., and Ding, Y. (2010). Optimal maintenance strategies for wind tur-bine systems under stochastic weather conditions. IEEE Transactions on Reliability 59 (2), 393–404.

[5] Camci, F. (2015). Maintenance scheduling of geographically distributed assets with prog-nostics information. European Journal of Operational Research 245, 506–5016.

[6] Chen, L., Gendreau, M., H`a, M.H., and Langevin, A. (2016). A robust optimization approach for the road network daily maintenance routing problem with uncertain service time. Transportation Research Part E 85, 40–51.

[7] Dai, L., St˚alhane, M., and Utne, I.B. (2015). Routing and scheduling of maintenance fleet for offshore wind farms. Wind Engineering 39 (1), 15–30.

[8] Dhahri, A., Zidi, K., and Ghedira, K. (2016). A variable neighborhood search for the vehicle routing problem with time windows and preventive maintenance activities. Trans-portation Research Part E 85, 40–51.

[9] D´ıaz-Ram´ırez, J., Huertas, J.I., and Trigos, F. (2015). Aircraft maintenance, routing, and crew scheduling planning for airlines with a single fleet and a single maintenance and crew base. Computers & Industrial Engineering 75, 68–78.

[10] Goel, A. and Meisel, F. (2013). Workforce routing and scheduling for electricity network maintenance with downtime minimization. European Journal of Operational Research 231, 210–228.

[11] Gundegjerde, C., Halvorsen, I.B., Halvorsen-Neare, E.E., Hvattum, L.M., and Non˚as, L.M. (2015). A stochastic fleet size and mix model for maintenance operations at offshore wind farms. Transportation Research Part C 52, 74–92.

[12] Halvorsen-Weare, E.E., Gundegjerde, C., Halvorsen, I.B., Hvattum, L.M., and Non˚as, L.M. (2013). Vessel fleet analysis for maintenance operations at offshore wind farms. Energy Procedia 35, 167–176.

[14] Laporte, G. (1992). The traveling salesman problem: An overview of exact and approx-imate algorithms. European Journal of Operational Research 59, 231–247.

[15] Li, X., Ouelhadj, D., Song, X., Jones, D., Wall, G., Howell, K.E., Igwe, P., Martin, S., Song, D., and Pertin, E. (2016). A decision support system for strategic maintenance planning in offshore wind farms. Renewable Energy 99, 784–799.

[16] L`opez-Santana, E., Akhavan-Tabatabaei, R., Dieulle, L., Labadie, N., and Medaglia, A.L. (2016). On the combined maintenance and routing optimization problem. Reliability Engineering and System Safety 145, 199–214.

[17] Maher, S.J., Desaulniers, G., and Soumis, F. (2014). Recoverable robust single day aircraft maintenance routing problem. Computers & Operations Research 51, 130–145. [18] M´arquez, F.P.G., Tobias, A.M., P´erez, J.M.P., and Papaelias, M. (2012). Condition

monitoring of wind turbines: Techniques and methods. Renewable Energy 46, 169–178. [19] Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic

Pro-gramming. John Wiley & Sons, Inc., Hoboken, New Jersey.

[20] Scheu, M., Matha, D., Hofmann, M., and Muskulus, M. (2012). Maintenance strategies for large offshore wind farms. Energy Procedia 24, 281–288.

[21] Shafiee, M. (2015). Maintenance logistics organization for offshore wind energy: Current progress and future perspectives. Renewable Energy 77, 182–193.

[22] Shafiee, M., Patriksson, M., and Str¨omberg, A.B. (2013). An optimal number-dependent preventive maintenance strategy for offshore wind turbine blades considering logistics. Ad-vances in Operations Research 2013, 1–12.

[23] St˚alhane, M., Hvattum, L.M., and Skaar, V. (2015). Optimization of routing and schedul-ing of vessels to perform maintenance at offshore wind farms. Energy Procedia 80, 92–99. [24] Zamorano, E. and Stolletz, R. Branch-and-price approaches for the multiperiod