A solution approach for the two item joint replenishment problem using the g-revised cost method

(1)

A solution approach for the two item joint

replenishment problem using the g-revised cost

method

(2)

Master’s Thesis Econometrics, Operations Research and Actuarial Studies Supervisor: dr. N.D. van Foreest

(3)

A solution approach for the two item joint replenishment problem using the g-revised cost method Anne-Mich`ele Woudstra

Abstract

(4)

1 Introduction

In joint replenishment problems the inventory of multiple items is controlled, where the possibility exists to share the fixed order cost. There also exist models where not only a major order cost K has to be paid for each order, but also an additional minor order cost ki is paid for each item i included in the order.

Furthermore, costs are incurred for items on stock and in some models also for backlogged items.

Therefore, instead of controlling the replenishment for the items separately, for example by using separate (s, S) policies for each of the items, joint replen-ishment is considered. For example, it might be beneficial to place an order for item 2 when an order for item 1 is placed anyway, since then the major cost K not has to be paid separately for the order item 2. However, since holding costs are also considered, we do not want to have too many items in stock. As such a balance need to be found between the fixed order and the holding costs to minimize the total costs.

Consider for example product X and product Y. For a truck to transport the items to the colorred store a major cost K is incurred, these cost are for e.g. the driver and the fuel. Furthermore, a minor costs k1and k2 are paid for

loading product X and product Y, respectively, on the truck and the personnel taking care of this transportation. So, when we place an order for product X, the truck needs to be driving to transport product X. It might as well transport product Y, but there still needs to be some sort of payment for loading product Y onto the truck. Additionally, all these products need to be stored and a cost is incurred for, e.g., rent of the storage space, which needs to be paid daily and per square meter.

1.1 Literature review

Commonly used methods for solving such problems are can-order policies, in which (s, S) policies are extended to (s, c, S) policies. Whenever the inventory level of item i hits its must-order level si, an order must be placed and item i is

ordered up to inventory level Si. Every other item j which at that time has an

inventory level at or below its can-order level cj, is included in the order as well.

(6)

Federgruen, Groenevelt, and Tijms (1984) study the problem for a contin-uous review model with compound Poisson demand and nonzero lead times. He does this by solving single item subproblems by a specialized policy itera-tion algorithm. This single item subproblems are generated by a decomposiitera-tion procedure. Using numerical results he finds that the heuristic gives good ap-proximations, which are slightly better than the results from Silver (1964).

Another type of policy commonly used in joint replenishment problems are periodic review policies P (s, S), this policy is introduced by Viswanathan (1997). In such policies at a time interval t for each item is checked if its inven-tory level is at or below its reorder level siand it is order up to level Si if this is

the case. Viswanathan (1997) studies the problem for stochastic demand with constant lead time. The problem is solved by finding (s, S) policies for each item under the assumption that only minor order costs are incurred. He also finds the best review interval t. He finds that the P (s, S) policy performs better than can order policies for relatively high values of K, but can order policies performs better for lower values of K.

Melchiors (2002) compares the policies and proposes a new type of can or-der policy, the compensation approach. The approach compensates the item i placing the order with a value that represents the expected value of the benefit that other items have from a discount opportunity. Such a discount opportunity occurs when other items place an order. through numerical results he finds that the compensation approach has in general lower costs than the can order policy proposed by Federgruen et al. (1984), with a cost difference up to 25%. The compensation approach is in his results also better than the periodic review approach of Viswanathan (1997), also for higher major order cost even though the average cost difference is only 0.75%. As such he concludes that the P (s, S) policy no longer outperforms the can order policies.

Germs, van Foreest, and Kilic (2017) suggest a g-revised cost method for finding an optimal (s, S) policy for the production-inventory system for single item models. In this thesis this method will be used for finding policies for the joint replenishment problem for two item inventory models with major and minor ordering costs. We will answer the following question: can we use the g-revised cost method for finding optimal policies for two item inventory models? The question will first be answered for deterministic demand and only a major order cost. Then we will answer the question for stochastic demand, but with both major and minor cost. Finally, we will find an answer for when the demand is stochastic.

2 Problem Formulation

2.1 Model

A two-item inventory model with constant deterministic demand (d1, d2), with

di being the demand rate for item i (i = 1, 2), will be considered.

(7)

Furthermore, let τij, i = 1, 2, j = 0, 1, 2, . . . denote the (ordered) times at

which item i is ordered. Note that

{τj | j = 1, 2, . . . } = {τ1j | j = 1, 2, . . . } ∪ {τ2j | j = 1, 2, . . . }. (1)

Furthermore, let the total number of orders up to time t and total number of orders for items of class i up to time t, respectively, be given by

n(t) = arg max j {τj: τj ≤ t}, ni(t) = arg max j {τij : τij≤ t}, i = 1, 2. (2)

Orders are assumed to arrive immediately, i.e., there will be no lead time. The inventory levels at time t are denoted as Ii(t) and are reviewed continuously.

Since Ii(τij) is the inventory level of item i right after the arrival of an order

and using the definition of ni(t) from equation (2), the inventory of item i at

time t is given as

Ii(t) = Ii(τi,ni(t)) − di(t − τi,ni(t)), i = 1, 2. (3)

Note that the order sizes are given by Ii(τij) − Ii(τ−ij), where τ −

ij is the time

right before the arrival of a new order. Furthermore, a holding cost of hi per

item per time unit has to be paid for inventory in stock. Backlog is allowed, but at a cost of bi per item per time unit, i.e., when inventory is negative. It is

assumed that bi ≥ hi. As such, the inventory costs are given by

L(I1, I2) = 2 X i=1 (hiIi++ biIi−) (4) where I_i+= max{0, Ii}, (5a) I_i−= (−Ii)+. (5b)

The cost of a (joint) replenishment is fixed and consists of a major cost K and minor costs kifor each item i (i = 1, 2) which is part of the replenishment.

Purchasing costs of the items are not included in the model, since all demand has to be satisfied eventually and so the total purchasing cost will be the same regardless of the policy. Using these definitions, the total cost up until time t under a policy π is Cπ(t) = n(t)K + 2 X i=1 ni(t)ki+ Z t 0 L(I1(s), I2(s))ds. (6)

2.2 Relevant questions

A policy π defines for each inventory level (I1, I2) whether the decision is made

(8)

is placed, what the order up to level is. For such a policy we are interested in the long-run time average cost, defined as

Cπ= lim sup

t→∞

Cπ_(t)

t . (7)

The goal is to find a policy π? which minimizes the long-run average cost, i.e., to find

Cπ?= inf

π C

π_. ₍₈₎

First we will study the problem without the minor costs, i.e., k1= k2= 0. In

order to find the optimum long-run average costs for this case, we first find some necessary conditions for the optimum reorder level and the optimum order up to level. It will be shown that among the policies satisfying these conditions and optimal policy can easily be found. Furthermore, the related long-run average cost Cπ?

will be given. Then the question remains: how can an optimal policy π∗be found when k1, k26= 0, which conditions are still valid? Furthermore, are

there other useful conditions? An algorithm will be found to solve this problem. Finally, we will find an algorithm to solve the problem for stochastic demand.

3 Joint replenishment with only major cost K

We will first study the simplified model in which the minor costs are neglected, i.e., k1 = k2 = 0 and thus a fixed cost K is paid for any order. Suppose we

have found the optimal starting inventory (S₁∗, S∗₂) and after some time the inventory level hits the corresponding order up to level (s∗₁, s∗₂). Since the order cost only consists of the major order cost K, we always order up to (S₁∗, S₂∗) again. The order process is thus cyclic and we only need to consider one such a cycle. Therefore, a policy π only needs to define and order up to level (S1, S2)

and a reorder level (s1, s2), but since we have constant deterministic demand

the reorder level can also be replaced by a cycle time τ. The simple relation between (s1, s2) and τ is given by

(s1, s2) = (S1, S2) − (d1, d2)τ. (9)

With cycle time τ we have,

C(τ) = K + Z τ

0

L(I1(s), I2(s))ds, (10)

Furthermore, equation (7) is then simplified to,

(9)

3.1 Introducing a reward

The current problem will be solved using the g-revised cost method as suggested by Germs et al. (2017).

In order to find the cycle time and order up to level we introduce a reward g. Now not only the order and inventory costs need to be paid, but also a reward of g per time unit is received. With this added reward we define a new cost function given by

V_gτ(x, y) = K + Z τ

0

(L(I1(s), I2(s)) − g)ds. (12)

Instead of trying to solve equation (8) directly, we will solve

Vg= inf (x,y)infτ V

τ

g(x, y). (13)

Note that this is still a function of g and thus the optimal order up to level and cycle time depend on the value of this reward.

First we will show the usefulness of this function and why and how this leads to the optimal long-run average cost, i.e., why this also solves (8). Let Vgbe the

solution of equation (13) and choose g∗such that Vg∗ = 0. With this reward g∗

we then have, for arbitrary policy π,

0 = Vg∗≤ V_gτ∗(x, y) = K + Z τ 0 L(I1(s), I2(s))ds − Z τ 0 g∗ds, (14)

where equality holds for the optimal policy π∗under reward g∗. SinceRτ

0 g ∗_{ds =} g∗τ, this gives g∗≤ K + Rτ 0 L(I1(s), I2(s))ds τ , (15)

where again equality holds for the optimal policy π∗ = ((S₁∗, S₂∗), τ∗). We thus see that g∗ represents the optimal cycle cost Cπ∗ _{and as such, by equation}

(10) and (11), the optimal long-run average cost. Furthermore, the policy π∗ that solves (8) can be found by finding the function Vg that solves (13) and

subsequently finding the reward g∗ that solves the equation Vg∗= 0.

The new task will thus be to find the solution to (13). To do this the first step is to find some necessary conditions on the optimal cycle time. This will be done by finding the optimal cycle time given a starting level (x, y), i.e., by solving the optimal stopping problem

Vg(x, y) = inf τ V

τ

g(x, y). (16)

(10)

corresponding cycle time τ). The conditions found in the first and second steps will show not to be sufficient, but in the third step we see that the solution solving (8) can easily be found in the set of possible solutions found in the first and second step.

3.2 The optimal stopping problem as a free boundary

problem

In this first step we will find the optimal cycle time, given that we start with inventory level (x0, y0). For the inventory at time t we can then define the path

(x(t), y(t)) = (x0, y0) − (d1, d2)t. (17)

With this path and cycle time τ the cost function of equation (12) has the form,

Vτ(x0, y0) = K +

Z τ

0

(L(x(s), y(s)) − g)ds. (18)

Different paths can be defined by choosing different starting inventories (x0, y0), but the first step to finding the solution to (13) will be to solve the

optimal stopping problem

V (x0, y0) = inf τ V τ_(x 0, y0) = K + inf τ Z τ 0 (L(x(s), y(s)) − g)ds, (19)

for each path of the form (17).

Clearly, if τ = 0 the total cost is equal to K. If τ > 0 then, for t ∈ [0, τ), an inventory cost of (L(x(t), y(t)) − g)dt needs to be paid plus the total cost from time t + dt onwards, i.e.,

V (x(t), y(t)) = (L(x(t), y(t)) − g)dt + V (x(t + dt), y(t + dt)). (20)

This leads us to the following dynamic programming equation,

V (x(t), y(t)) = min{K, (L(x(t), y(t)) − g)dt + V (x(t + dt), y(t + dt))}. (21)

the fact that V (x(t + dt), y(t + dt)) = V (x(t), y(t)) + V0(x(t), y(t))dt, together with (21) gives us the idea that the function V (·) is the solution of the constraint optimization problem

max V (x(t), y(t)),

s.t. V (x(t), y(t)) ≤ K, (22a)

(11)

and stopping rule (not necessarily optimal) τ. By the fundamental theorem of calculus we have, using constraint (22b),

v(x(τ), y(τ)) − v(x(t), y(t)) = Z τ t d dsv(x(s), y(s))ds = Z τ t v0(x(s), y(s))ds ≥ − Z τ t (L(x(s), y(s)) − g)ds. (23)

Using v(x(τ), y(τ)) ≤ K, by constraint (22a), this inequality leads to,

v(x(t), y(t)) ≤ v(x(τ), y(τ)) + Z τ t (L(x(s), y(s)) − g)ds ≤ K + Z τ t (L(x(s), y(s)) − g)ds. (24)

This inequality holds in particular for the optimal stopping time τ∗. But then this inequality tells us that for arbitrary function v,

v(x(t), y(t)) ≤ K + Z τ∗

t

(L(x(s), y(s)) − g)ds = V (x(t), y(t)). (25)

Thus V (x(t), y(t)) dominates every other function v that satisfies the constraints of the constraint optimization problem (22) and thus V (x(t), y(t)) together with stopping rule τ∗ is its solution.

The objective in (22) shows that at least one of the constraints needs to be binding. This might mean the constraint satisfaction is solvable as a free boundary problem, i.e., we have,

V (x(t), y(t)) = K, for (x(t), y(t)) ∈ D, (26a)

V0(x(t), y(t)) = −(L(x(t), y(t)) − g), for (x(t), y(t)) ∈ C, (26b) and need to find the subsets C and D such that the function V (x(t), y(t)) is optimal. Or put differently, what is the boundary between sets C and D. As long as our inventory level is within set C we continue. As soon as we hit the boundary and enter set D we stop (i.e., we are at the reorder point).

3.3 The optimal cycle time

We will first give the sets C and D. Then we find V (x0, y0) and the optimal

stop-ping time τ∗according to these sets. After this we show that the free-boundary problem indeed gives the solution of the constraint optimization problem.

(12)

as L(x(t), y(t)) − g < 0. With L(x(t), y(t)) given by equation (4) the optimal continuation set C according to this intuition will be defined by,

C = {(x, y) : L(x, y) − g < 0}. (27)

Furthermore, for each (x0, y0) ∈ C we define

Ω(x0, y0) = {(x, y) : (x0, y0) − (d1, d2)t, t ≥ 0}. (28)

With this definition we define the state space by

Ω = [

(x0,y0)∈C

Ω(x0, y0). (29)

The optimal stopping set D is then defined by the complement of C in state space Ω. As such the stopping set D is given by

D = Ω\C. (30)

The sets state space Ω and the sets C and D are depicted in Figure 1.

(g/h1, 0) (0, −g/b2) (−g/b1, 0) (0, g/h2) C D

Inventory level item 1

In v en tory lev el item 2

Figure 1: The continuation set C and stopping set D.

In order to find V (x(t), y(t)) in the free boundary problem (26), with con-tinuation set C as given in (27) and stopping set D as given in (30), we first need to find τ.

Suppose (x(t), y(t)) is in C. Then we are in C as long as L(x(t), y(t))−g < 0. The stopping time τ is the time at which the inventory level hits D.Thus,

(13)

The inventory level (x(t), y(t)) is thus in the continuation set for t ∈ [0, τ), by (26b) this leads to

V0(x(t), y(t)) = −(L(x(t), y(t)) − g), for t ∈ [0, τ). (32) Furthermore, by equation (26a) V (x(τ), y(τ)) = K, since (x(τ), y(τ)) ∈ D. Using L(x(t), y(t)) given by equation (4), V (x0, y0) is given by

V (x0, y0) = V (x(τ), y(τ)) − Z τ 0 V0(x(s), y(s))ds = K + Z τ 0 (L(x(s), y(s)) − g)ds = K + h1 Z min{τ,max{0,x0/d1}} 0 (x0− d1s)ds − b1 Z τ min{τ,max{0,x0/d1}} (x0− d1s)ds + h2 Z min{τ,max{0,y0/d2}} t (y0− d2s)ds − b2 Z τ min{τ,max{0,y0/d2}} (y0− d2s)ds − gτ. (33)

Performing the calculations, the final form of V (x0, y0) is given as,

V (x0, y0) =K + 1 2(h1+ b1) x2 0 d1 1x0>0≥x(τ)+ 1 2(h2+ b2) y2 0 d2 1y0>0≥y(τ) + (x0τ− 1 2d1τ 2_)(h 11x(τ)>0− b11x(τ)≤0) + (y0τ− 1 2d2τ 2_)(h 21y(τ)>0− b21y(τ)≤0) − gτ, (34)

for (x0, y0) ∈ C, where τ given by equation (31), and V (x0, y0) = K for

(x0, y0) ∈ D.

To see that sets C as defined in (27) and D as defined in (30) indeed solve the free boundary problem and that it is indeed the solution to the constraint optimization problem (22), first remember that the optimal stopping time τ∗

according to these sets is given by the time the inventory level hits D. Further-more, C was defined by the interior of a quadrilateral for which L(x, y) − g < 0 and D as the closure of the exterior of this quadrilateral (thus L(x, y) − g ≥ 0 for (x, y) ∈ D).

Suppose we have a different continuation set C0 containing inventory levels (x0, y0) for which L(x0, y0) − g > 0. Then

V0(x(s), y(s)) = −(L(x(s), y(s)) − g) < 0, for s ∈ [0, τ). (35) As a consequence, by equation (33), this gives

V (x0, y0) = K +

Z τ

0

(14)

And as such, constraint (22a) is not satisfied.

On the other hand, suppose we have an alternative stopping set D0 con-taining inventory levels (x(t), y(t)) for which L(x(t), y(t)) − g < 0. Since these inventory levels are in D0 they satisfy V (x(t), y(t)) = K. By constraint (22b) we have

V0(x(t), y(t)) > −(L(x(t), y(t)) − g) > 0, (37) thus V is increasing in (x(t), y(t)). This gives

V (x(t + dt), y(t + dt)) > V (x(t), y(t)) > K, (38)

but in this case constraint (22a) is violated.

We conclude that the optimal stopping time τ∗ is given by equation (31). The optimal value Vg(x, y) of the optimal stopping problem (16) is given by

(34), for some path (x(t), y(t)) = (x0, y0) − (d2, d1)t with (x0, y0) in the set C

given by (27) and is equal to K for (x0, y0) in the set D given by (30).

3.4 Conditions on optimal reorder and order up to levels

With the optimal stopping time τ∗ for given path (x(t), y(t)) = (x0, y0) −

(d1, d2)t, the next goal is to find the optimal order up to level (x0, y0). We

will first show that this order up to level will be on the boundary of C (with (x(t + dt), y(t + dt)) ∈ C for small values of dt).

Let (x0, y0) be such a point on the boundary of C. Now suppose we would

choose an alternative order up to level (x0₀, y0₀), such that

(x0₀, y₀0) = (x0, y0) − (d1, d2)t, t ∈ (0, τ). (39) Then we have V (x0, y0) = Z t 0 (L(x(s), y(s)) − g)ds + V (x0₀, y₀0). (40)

Since (x0, y0) is on the boundary of C, this means L(x(s), y(s)) − g < 0 for

s ∈ (0, τ) and thus V (x0, y0) < V (x00, y00). On the other hand, suppose we

would choose an alternative order up to level (x00₀, y₀00), such that

(x00₀, y₀00) = (x0, y0) − (d1, d2)t, t ∈ (−∞, 0). (41) This gives V (x00₀, y₀00) = Z 0 t (L(x(s), y(s)) − g)ds + V (x0, y0). (42)

Since (x0, y0) is on the boundary of C, this means L(x(s), y(s)) − g ≥ 0 for

s ∈ (−∞, 0) and thus V (x0, y0) ≤ V (x000, y000). As such we conclude that (x0, y0)

should be chosen on the boundary of C (with (x(t + dt), y(t + dt)) ∈ C for small values of dt).

(15)

the path of the solid arrow. Clearly the path of the dashed arrow gives lower costs. Not only are the inventory costs lower, since the inventory level of item 1 is lower for equal inventory levels for item 2 (and thus the backlog costs are lower). Also, in the dashed path the inventory level is longer in the set C, for which L(x, y) − g < 0. In Figure 2 the inventory level immediately hits D when we choose y0 < 0, but for situations where this is not the case the proof is

similar as we have just shown for x0 < 0, now not the backlog cost but the

holding cost will be lower. We thus conclude that (x∗₀, y∗₀) should always be in the first quadrant.

(g/h1, 0)

(0, −g/b2)

(−g/b1, 0)

(0, g/h2)

Figure 2: Always choose (x0, y0) in the first quadrant.

In a similar way we can show that the reorder level (x(τ), y(τ)) should always be in the third quadrant, i.e., x(τ), y(τ) ≤ 0, see Figure 3. This uses the same line of reasoning as for when we showed that x∗

0, y0∗≥ 0.

3.5 Optimal long-run average costs and the corresponding

parameters

Now that we know (x(τ), y(τ)) is the third quadrant and that τ∗ is given by equation (31). This gives

y(τ∗) = −g b2 −b1 b2 x(τ∗) (x(τ∗), y(τ∗)) = (x0− d1τ∗, y0− d2τ∗)    τ∗= b1x0+ b2y0+ g b1d1+ b2d2 . (43)

(16)

(g/h1, 0)

(0, −g/b2)

(−g/b1, 0)

(0, g/h2)

Figure 3: Always let (x(τ), y(τ)) be in the third quadrant.

Furthermore, since (x∗₀, y∗₀) is in the first quadrant and on the boundary of C, we know that y∗₀= g h2 −h1 h2 x∗₀. (44)

Substituting equations (43) and (44) into equation (34) we have,

Vg(x∗0, y ∗ 0) = K + (h1d1+ h2d2)(h1h2(b1d1+ b2d2) + b1b2(h1d1+ h2d2)) 2d1d2h22(b1d1+ b2d2) x∗2₀ b1(h2+ b2)(h1d1+ h2d2)g d2h22(b1d1+ b2d2) x∗₀+(h2+ b2)(b1d1− h2d2)g 2 2d2h22(b1d1+ b2d2) . (45) By differentiation and setting equal to 0 the optimum S₀∗ is found as

S∗₁= x∗₀= b1d1(h2+ b2)

h1b1d1(h2+ b2) + h2b2d2(h1+ b1)

g (46)

and from equation (44),

S₂∗= y₀∗= g h2 − h1b1d1(h2+ b2)g h2(h1h2(b1d1+ b2d2) + b1b2(h1d1+ h2d2)) = b2d2(h1+ b1) h1b1d1(h2+ b2) + h2b2d2(h1+ b1) g. (47)

Now that all the variables are found as functions of g, remember the cycle cost can be found by solving Vg∗(x∗₀, y₀∗) = 0. We find the cycle cost as

(17)

Finally, the optimal order up to level (S₁∗, S₂∗) is found as S∗₁= x∗₀= b1d1 s 2K(h2+ b2) (h1+ b1)(h1b1d1(h2+ b2) + h2b2d2(h1+ b1)) , (49a) S₂∗= y₀∗= b2d2 s 2K(h1+ b1) (h2+ b2)(h1b1d1(h2+ b2) + h2b2d2(h1+ b1)) . (49b)

Furthermore, the optimal stopping time is found as

τ∗= s

2K(h1+ b1)(h2+ b2)

h1b1d1(h2+ b2) + h2b2d2(h1+ b1)

. (50)

Finally, from equation (9), the optimal reorder level is found as

s∗₁= −h1d1 s 2K(h2+ b2) (h1+ b1)(h1b1d1(h2+ b2) + h2b2d2(h1+ b1)) , (51a) s∗2= −h2d2 s 2K(h1+ b1) (h2+ b2)(h1b1d1(h2+ b2) + h2b2d2(h1+ b1)) . (51b)

4 Joint replenishment with major cost K and

minor costs k

1

and k

2

In this section we will consider the minor costs k1 and k2. Next to the major

cost K for placing an order, an additional cost of ki needs to paid when item

i is replenished. One of the choices to be made is thus whether both items are ordered, or just one and in the latter case, which one. As such it might not be beneficial to always order up to inventory level (S₁∗, S₂∗) or reorder when the inventory level is at or below (s∗₁, s∗₂) as found in the previous section. At this point we do not even know if it is optimal to place a new order every time the inventory level hits D, as found in (30) (the set for which L(x, y) − g ≤ 0). All of this because we might want to wait longer, reorder earlier, or order up to a different inventory level, because this might reduce the order cost in such a way that it outweighs the extra inventory costs or loss in profit that will be made. For example we might want to reorder a little earlier, to prevent an order cost of K + k1+ k2 and instead have an order cost of only K + k1. Furthermore, at

different points in time different decisions might be made. There are no standard cycles anymore, at each reorder point a different cycle might be started. This means that a decision of what to do at the current inventory level, might even be based on what decisions are made in a future cycle.

(18)

For policy π the value function at reorder time τi is given as

Vgπ(x(τi), y(τi)) = K(τi) +

Z τi+1

τi

(L(x(s), y(s)) − g)ds + V (x(τi+1), y(τi+1)),

(52)

where (x(τi), y(τi)) is the order up to level at time τi, the order cost K(τi) is

given by

K(τi) = K + k11item 1 ordered at τi+ k21item 2 ordered at τi. (53)

Furthermore,

(x(s), y(s)) = (x(τi), y(τi)) − (d1, d2)(s − τi), for τi ≤ s < τi+1 (54)

To see that the cost function Vπ

g(x, y) is still useful, suppose that for policy π ,

∃n ∈ N s.t. (x(τn), y(τn)) = (x0, y0). This gives

V_gπ(x0, y0) = n−1 X i=0 K(τi) + Z τn 0 (L(x(s), y(s)) − g)ds + V_gπ(x(τn), y(τn)). (55) But since Vπ

g(x0, y0) = Vgπ(x(τn), y(τn)) this gives

0 = V_gπ(x0, y0)−Vgπ(x(τn), y(τn)) = n−1 X i=0 K(τi)+ Z τn 0 (L(x(s), y(s))−g)ds, (56) and as such g = Pn−1 i=0 K(τi) +R τn 0 L(x(s), y(s))ds τn . (57)

We thus see again that g represents the time average cost. Alternatively, if such an n ∈ N does not exist, set Vπ

g(x(τn), y(τn)) = 0 and let gπ be such that

Vπ gπ(x0, y0) = 0. This gives 0 = V_gππ(x0, y0) = n−1 X i=0 K(τi) + Z τn 0 (L(x(s), y(s)) − gπ)ds. (58)

From which again follows that gπ _{is given by equation (57). We see that g}π

gives the time average cost up to time τn and for n → ∞ we get the long-run

time average cost. Note that again for optimal policy π∗ we have Vπ∗ g∗ ≤ V_gππ

and g∗ thus still represent the optimal long-run time average cost.

(19)

4.1 Conditions on the optimal policy

The solution in this case does not necessarily consist of one repeated cycle, but might consist of multiple different cycles, a solution thus needs to be found in multiple iterations. To solve the problem the value iteration algorithm will be used. Unfortunately there exists an infinite amount of inventory levels and some discretization is needed. We want to prevent the algorithm from becoming too slow due to the presence of too many states, or becoming too inaccurate due to absence of enough states, we therefore first find some conditions on the optimal policy. These conditions will lead to a smaller set of possible states (and a smaller set of possible choices in each state).

4.1.1 What decisions can be made?

At first we find some conditions on the possible choices that might be made, i.e., wait, reorder item 1, reorder item 2 or reorder both items. The first condition is a statement about when not to order certain items.

Theorem 4.1. For ordering only one of the items, with inventory cost given by (4), we find the following.

1. Never order item 1, if x ≥ 0.

2. Never order item 2, if y ≥ 0.

3. Always wait if x, y ≥ 0.

For Theorem 4.1 we will prove the first statement, the proof for the second statement is similar. Furthermore, the third statement is a direct consequence of the first two statements.

Proof. Consider the situation where the current inventory level is (x, y), with x ≥ 0 and we want to place an order for item 1. Consider the following alter-natives,

1. Order up to inventory level (xu, y), a small amount of time dt later the

inventory level is (x0_u, y0) = (xu, y) − (d1, d2)dt.

2. Wait for a small amount of time dt such that we have an inventory level of (x0, y0) = (x, y)−(d1, d2)dt, then order up to (xu0, y0) = (xu, y)−(d1, d2)dt.

The situations are depicted in Figure 4 an the costs are given by

K + k1+ (h1xu+ h2y++ b2y−− g)dt, for action 1, (59a)

(h1x + h2y++ b2y−− g)dt + K + k1, for action 2. (59b)

Since xu > x, the cost in the second situation are lower. It is thus beneficial

to wait for a bit instead of ordering now. Since the statement holds as long as x ≥ 0, we conclude that we do not order item 1 if x ≥ 0.

(20)

(g/h1, 0) (0, −g/b2) (−g/b1, 0) (0, g/h2) (x, y) (xu, y) (x0_u, y0) (x0, y0)

Figure 4: The situation of Theorem 4.1.

(g/h1, 0) (0, −g/b2) (−g/b1, 0) (0, g/h2) (x, y) (xu, y) (x0_u, y0) (x0, y0)

(21)

4.1.2 About the order up to levels

Next, we find some conditions on the order up to level, based on which items are reordered. With regard to ordering only one item we have the following condition.

Theorem 4.2. When ordering only item 1, the order up to level has x ≥ 0. When ordering only item 2, the order up to level has y ≥ 0.

Proof. Again, only the proof for the first situation will be shown, for the second situation the proof will be similar.

Denote the reorder level by (xr, y) and note that xr < 0, by Theorem 4.1.

Consider order up to level (xu, y), we will show that there always exists and

x0

u > xu as long as xu < 0, such that the value function decreases. We have

three situations.

1. Suppose the next order is also only for item 1. In this case, denote the next reorder level by (xr1, y1) and the next order up to level by (xu1, y1), this

gives the path {(xr, y), (xu, y), (xr1, y1), (xu1, y1)}. Note that xr1 < xu.

Now, if |xu| < xu1 − xr1 (note that the right hand side is the amount

ordered in the second order) we could have ordered up to x0_u= 0 instead of up to xu. Furthermore, in this case the second reorder level becomes

x0_r₁ = xr1+|xu|. This does not change the order cost, but since this results

in less backlog for item 1, the inventory costs decrease and as such the value function does too. If |xu| ≥ xu1−xr1, we choose x

0

u= xu+(xu1−xr1)

and x0r1 = xr1+ xu1− xr1 = xu1, the second reorder can thus be skipped.

In this case both the order and inventory cost, and as such the value function, decrease.

2. Suppose the next order is for both items. In this case the reorder level for item 2 also needs to be defined, but note that we can repeat the arguments of the first case , since there is no need to change the order up to level yu.

3. Suppose the next order is for item 2. Since there is no replenishment for item 1 in this case, it can not directly be seen that also in this case we can find an x0_u> xu. Consider a situation where after the first order (the

one for item 1), there are n reorders for only item 2. Since we have to replenish item 1 at some point, after these n replenishments there is a replenishment for either only item 1 or for both items. In this case we can choose x0uas in one of the first two cases. The value function decreases as

denoted in these cases. But since the backlog of item 1 decreases all along the path of the n replenishments for item 2, the value function decreases by an additional amount equal to the decrease in these backlog costs. We thus see that there always exists an x0_u > xu which decreases the value

function, as long as xu > 0. By repetition we can thus remove future orders

until x0_u≥ 0 or change the order up to level to x0

u≥ 0 directly.

Theorem 4.3. When ordering both items, the order up to level (xu, yu) has to

(22)

Proof. In section 3.4 we have shown that the optimal order up to level should be in the first quadrant in the case with only major cost. Clearly this still holds as long as it does not affect any previous cycles negatively, i.e., when we have reorder level (xu, yu) which does not satisfy the condition of Theorem 4.3, we

can choose (x0u, y0u) which does satisfy this theorem instead of (xu, yu) as long

as it does not increase the costs in previous or future cycles. To see that this is indeed the case, let

(x0_u, y0_u) = (xu, yu) − (d1, d2)t, t ∈ R. (60)

We can choose t such that (x0_u, y_u0) satisfies the condition given in Theorem 4.3 if x0_u > xr and y0u > yr, where (xr, yr) is the reorder level, since we can not

order up to inventory levels lower than the current inventory level. This clearly is at least the case if (xu, yu) ∈ C. If either x0u ≤ xr or yu0 ≤ yr we thus have

(xu, yu) /∈ C. But in this case we could have chosen (x0u, yu0) satisfying (60) and

either x0_u = xr or yu0 = yr. This would increase the costs, since we only have

to order one item instead of two and L(x, y) − g > 0 all along the path from (xu, yu) to (x0u, y0u). In this case we thus would not have ordered both items.

4.1.3 About the reorder levels

Now we state a condition on when to reorder, if we know that the next order is only for item 1 or only for item 2.

Theorem 4.4. Suppose the next order is for only item 1 and we consider placing the order at the current inventory level. Let (xr, y) be the inventory of item 1

right before the order arrives, i.e., the current inventory level, and let (xu, y) be

the inventory of item 1 right after the order arrives. The following has to hold,

h1xu≤ −b1xr. (61)

We wait with placing the order if

−b1xr< h1xu. (62)

A similar statement holds for item 2.

Proof. By Theorem 4.1, xr < 0. By Theorem 4.2, xu ≥ 0. The proof will be

done similar as the proof of Theorem 4.1. We have the following alternatives. 1. Order up to inventory level (xu, y) and a small of time dt later the

inven-tory level is (x0_u, y0).

2. Wait for a small amount of time dt such that the inventory level is (x0_r, y0), then order up to (x0_u, y0).

The costs along the given paths are given by,

K + k1+ (h1xu+ h2y++ b2y−− g)dt, for situation 1 (order), (63a)

(23)

If h1xu ≤ −b1xr the costs for the first situation are lower than (or equal to)

the costs for the second situation, we thus prefer to order. If −b1xr< h1xuthe

costs for the second situation are lower and we thus prefer to wait. Note that in both situations we start with inventory level (xr, y) and end with inventory level

(x0u, y0), we thus do not need to take future decisions into consideration.

A consequence of Theorem 4.4 is that a reorder for item 1 is placed when

−b1xr= h1xu. (64)

This is impossible in the case that we need to wait so long that the inventory level of item 2 has dropped below its next reorder level yr, we can choose to

place the order when the inventory level of item 2 is equal to yr. But in this case

we would place two orders immediately after each other, we could thus skip the first one and order up to the next order up to level right away. This would also not be possible if xr is already too big to satisfy this condition at the moment

the previous order was placed, but using the same reasoning as before, we might skip this previous order.

4.1.4 Regarding the replenishments for both items

Finally, we find some conditions on the replenishments for both items. The first theorem is a direct consequence of section 3.

Theorem 4.5. When ordering both items, always reorder when the inventory level hits D, where D is given by the stopping set given in 30.

Proof. In section 3.4 we have already shown that in the case for only major costs the theorem holds. Clearly it still holds here, if it does not affect any future or previous cycles.

Consider an arbitrary reorder level (xr, yr). If this reorder level is in C, we

only have to change the current cycle to choose a reorder level (x0_r, y_r0) which hits D. If the reorder level is in D, we also could have chosen a reorder level (x0_r, y0_r) which hits D. To see this note that once we are in the stopping set D we always have L(x, y) − g > 0, at least until the next reorder. It is thus never optimal to order up to an inventory level that lies in D. This means that the previous reorder level was in C and thus we can choose (x0_r, y0_r) such that it hits D.

4.1.5 The boundaries

(24)

inventory level follows the path according to the solid line. In node a an order is placed for item 1, thus by Theorem 4.1 and Theorem 4.2, a has to have negative inventory for item 1 and b has to have positive inventory for item 1. In node c an order is placed for item 2. Note that if this reorder level is in the exterior of C, the order up to level is also in the exterior of C, as a consequence of Theorem 4.4 (condition (64) is satisfied when the reorder and order up to levels are on the boundary of C). If we however choose the alternative path given by the dashed line, with the reorder and order up to levels for the order for item 2 on the boundary of C, this is clearly a better solution. The inventory levels for item 1 are closer to 0 for equal inventory levels for item 2 on the path from b0 to c0 compared to the path from b to c, and for inventory levels of item 2 on the path from b to c which are not in the path from b0 to c0 we have L(x, y) − g ≥ 0. The latter also holds for the path from d to d0. If the order from node a to node b would have been an order for both items, the same argumentation holds, the only difference is that at node a the inventory level for item 2 would have been lower. If the order from node a to node b would also have been an order for only item 2 we could also have chosen an alternative path with nodes c0 and d0 by simply shifting this order a bit to the left. For the the case where both the path from b to c and the path from d to e intersect the boundary of C for positive inventory level for item 1, we have shown that an alternative path exists for which the reorder level and order up to level are within the closure of C. In case these paths both have negative inventory for item 1, the reasoning is similar. The difference is that we do not look at an order from node a to node b, but an extra order from node e to an extra node f . See Figure 7. In the case where the path from node b to node c intersects the boundary of C for positive inventory level for item 1, and the path from node d to node e intersects the boundary of C for negative inventory level for item 1, we look at both the orders from node a to node b and from node e to node f . Again using similar reasoning we find an alternative path where c0 has inventory level (0, −g/b2) and d0 has inventory

level (0, g/h2). Finally, suppose we have an order for only item 1, again the

reorder and order up to levels are within the closure of C. The reasoning will be along the same lines as the order for only item 2. We thus conclude that all reorder and order up to levels are within the closure of C and as such inventory levels are within the closure of C at all times.

4.2 An algorithm for finding an optimal policy

(25)

(g/h1, 0) (0, −g/b2) (−g/b1, 0) (0, g/h2) a b c d e b0 c0 d0

(26)

(g/h1, 0)

(0, −g/b2)

(−g/b1, 0)

(0, g/h2)

(27)

For given time interval dt we define the set of gridpoints P ⊂ Ω as

P = {(x, y) : x = dtd1m1, y = dtd2m2, m1, m2∈ Z}. (65)

Let Vn(x, y) be the value function in (x, y) after n iterations, with V0(x, y) = 0 ∀(x, y) ∈ P. In every point in P we have four possibilities: wait, order item 1, order item 2 and order both items.

In case we choose to wait the value function is given by

V_waitn (x, y) = (L(x, y) − g)dt + Vn(x − d1dt, y − d2dt). (66)

Note that we thus first have to find Vn_(x0_{, y}0_{) with x}0 _{< x, y}0_{< y. The algorithm}

thus has to find Vn_{(x, y) for the lower inventory levels (x, y) first. Furthermore,}

along the lower boundaries it is not allowed to wait, since this would mean the inventory level is outside of P in the next time step. For these values of (x, y) we thus set Vn

wait(x, y) = ∞.

By condition (64) we know that in the case we choose to order only item 1 the order up to level has to be xu= −b1xr/h1. Since our state space is discrete,

we have two options for the order up to level, the state closest to the left and the closest one on the right. The value function is in this case thus given by

V₁n(x, y) = K + k1+ min ( Vn−1 −b1 h1 x − , y ! , Vn−1 −b1 h1 x + , y !) , (67) where x− denotes the closest gridpoint on the left and x+ _{denotes the closest}

gridpoint on the right. Similar, when we choose to order only item 2 the value function is given by V₂n(x, y) = K + k2+ min ( Vn−1 x, −b2 h2 y −! , Vn−1 x, −b2 h2 y +!) , (68) and when ordering both items the value function is given by

V_bothn (x, y) = K + k1+ k2+ min

(x0_,y0_)∈P:x0_>x,y0_>y{V

n−1_(x0_{, y}0_)}. ₍₆₉₎

The value function in (x, y) will then be given by

Vn(x, y) = min{V_waitn (x, y), V₁n(x, y), V₂n(x, y), V_bothn (x, y)}. (70) We call this algorithm FindOptimalValue, its pseudocode can be found in Algorithm 4.1.

Our interest is the compensation g∗for which the optimal value V∗is equal to 0, sine this will give use the optimal average cycle cost and it leads to the path of the inventory level for the optimal policy. To find g∗ _{we will use the}

binary search approach. Given a lower bound glower and gupper we will run

(28)

Algorithm 4.1 FindOptimalValue, for finding the optimal value for a given g. Input: K, k1, k2, h1, b1, d1, h2, b2, d2, ε, P, δt,g, numiterations Output: V∗, Vn 1: Initialize: Vn−1_{← 1, V}n _{← 0} 2: for n ∈ {1, ..., numIterations} do 3: for (x, y) ∈ P do 4: evaluate Vn

wait(x, y), V1n(x, y), V2n(x, y), Vbothn (x, y)

5: Vn(x, y) = min{V_waitn (x, y), V₁n(x, y), V₂n(x, y), V_bothn (x, y)} 6: end for

7: Vn−1= Vn

8: end for

9: V∗= minx,yVn(x, y)

10: Return V∗, Vn

0, g was to big and is our new upper bound gupper, if V∗ > 0, g was to small

and serves as out new lower bound glower. Furthermore, if FindOptimalValue

returns V∗ = 0, we have found the optimal value g∗. We repeat the process until the difference between the lower and the upper bound is smaller than a given error ε.

The next step is to find the corresponding path. The first order up to level is given by

(x∗, y∗) = arg min

(x,y)

Vn(x, y), (71)

where n is the last iteration of FindOptimalValue. We then look repeatedly at the decision made at the current inventory level (xi, yi). If the decision is

to wait, the next inventory level on the path is given as (xi+1, yi+1) = (xi−

d1dt, yi− d2dt). If the decision is to place an order, the next inventory level

on the path is the order up to level of that order. We can repeat this until we arrive at inventory level (x∗₁, y∗₁) again.

5 Joint replenishment with stochastic demand

In this section we assume a stochastic demand instead of the deterministic demand we used in the previous sections. Since demand is stochastic we can not define a path as we did in the deterministic case. A policy now first needs to define, for each inventory level (x, y), if we wait or order. Furthermore, if we order, it needs to define the order up to level.

(29)

Algorithm 4.2 FindOptimalCost, for finding the optimal value for a given g.

Input: K, k1, k2, h1, b1, d1, h2, b2, d2, ε, P, δt, glower,ghigher

Output: g 1: g ← 1₂(gupper− glower) 2: while g > do 3: V∗_{= FindOptimalValue (K, k}1, k2, h1, b1, d1, h2, b2, d2, ε, P, δt, g) 4: if V∗< 0 then 5: gupper= g 6: else 7: glower= g 8: end if 9: end while 10: Return V∗

which can be reached from inventory level (x, y), i.e.,

P(x, y) = {(x, y) ∈ P : x0≤ x, y0≤ y}. (72) When the inventory level at time t is (x, y) and the decision is made to wait, the value function V (x, y) is given by (L(x, y) − g)dt plus the expected value of the value function at time t + dt. That is,

Vwait(x, y) = (L(x, y) − g)dt +

X

(x0_,y0_)∈P(x,y)

p((x, y), (x0, y0))Vn(x0, y0). (73)

Since p((x, y), (x, y)) may be greater than 0 this gives,

Vwait(x, y) =

(L(x, y) − g)dt +P

(x0_,y0_{)∈P(x,y)\{(x,y)}}p((x, y), (x0, y0))Vn(x0, y0)

1 − p((x, y), (x, y)) .

(74) Note that p((x, y), (x0_{, y}0_{)) = 0 when either x}0_{> x or y}0 _{> y, the algorithm will}

thus again start with calculating the value function for the lower inventory levels in P. Since condition (64) not necessarily holds for stochastic demand, since the proof of Theorem 4.4 does not hold if we do not know what the inventory level in the next time step will be. The value function when an order is placed for item 1 is now thus simply given by

V₁n(x, y) = K + k1+ min (x0_,y)∈P:x0_>x{V

n−1_(x0_{, y)},} ₍₇₅₎

and when an order is placed for item 2 the value function is given by V₂n(x, y) = K + k2+ min

(x,y0_)∈P:y0_>y{V

n−1_{(x, y}0_)}. ₍₇₆₎

The value function for when we order both items remains the same. We can use Algorithm 4.1 to find V∗ for a given value of g, now with Vn

wait given by

equation (74), Vn

1 given by equation (75) and V2n given by equation (76). Also

(30)

6 Results

In this section we will test the algorithms. We will test for correctness by comparing them to a know case. Then we will give an example of a solution for an unknown case.

6.1 The algorithm for only major costs

As a second test we set k1= k2 = 0, we have found the optimal cost and the

corresponding reorder and order up to levels for this case in section 3.5. We use the following parameters, h1 = h2 = 4, b1 = b2 = 8, K = 45, d1 = 2, d2 = 1,

δt= 0.1 and ε = 1e − 5. According to equation (48) the optimal cost should be

given by g∗= s 2K h1b1d1 h1+ b1 + h2b2d2 h2+ b2 = 12√5 ≈ 26.83282. (77)

The optimal order up to level should then be given according to equation (49),

(S₁∗, S₂∗) = (2√5,√5) ≈ (4.5, 2.2), (78) and the optimal reorder level according to equation (51),

(s∗₁, s∗₂) = (√5,1 2

√

5) ≈ (−2.2, −1.1). (79)

According to the algorithm the optimal cost is given by gALG _{= 26.82353, the}

optimal order up to level by (SALG

1 , S2ALG) = (4.4, 2.2) and the optimal reorder

level by (sALG

1 , sALG2 ) = (−2.4 − 1.2). Considering the discretizations in the

algorithm we thus have found a good approximation for the order up to level, since we have chosen the interval δt= 0.1.

6.2 Long-run average costs considering minor costs

Finally we will test a case for which both items have positive demand and both k1 and k2 are positive. We try to show that not in all cases it is optimal to

always order both items. For this purpose we have chosen an extreme case where K and k2 are small compared to k1. The following parameters are used,

h1 = 2, h2 = 4, b1 = 3, b2 = 8, K = 5, k1 = 40, k2 = 5, d1 = 1, d2 = 2 and

δt= 0.1. We find that the optimal cost is found as g = 14.4. The path of the

inventory level is given in Figure 8. We notice that indeed not every order is an order for both items.

7 Conclusion

(31)

−5

0

5

10 −2

0

2

4 Inventory level item 1

In

v

en

tory

lev

el

item

2

Figure 8: The path of the inventory level, for h1 = 2, h2 = 4, b1 = 3, b2 = 8,

(32)

quadrant and the reorder level (s1, s2) should be in the third quadrant.

Fur-thermore, they both are on the boundary of the C, which contains all inventory levels (x, y) such that L(x, y)−g < 0. With these conditions we were able to find the optimal policy. For the case with also minor costs k1 and k2the conditions

found in the case where k1= k2= 0 are still valid in the case the choice is made

to place an order for both items. Furthermore, we found that −b1xr= h1xuhas

to hold when we place an order for only item 1. An equivalent condition holds for an order for only item 2. With the new conditions we found an algorithm to find the optimal policy when considering minor costs. In the case of stochastic demand the previous algorithm can be generalized.

We successfully found an optimal solution to the two item joint replenish-ment problem using the g-revised cost method. The benefit from this is that this solution can now be used as benchmark for e.g. can order policies. Such can order policies are easy to apply in practice, but it has not always been easy to show how good the solutions actually are. With the algorithm found in this thesis these policies can now be benchmarked.

For further research the condition for ordering only one item, −b1xr= h1xu,

with xr the reorder level and xu the order up to level, can be used to find an

alternative algorithm using graph theory. For each pair of states {a, b} we can find the cost for getting from state a to state b by placing an order in between the two states, since we now know at what moment the order should be placed. This can be done for each type of order and the minimum among the options is found. With knowing the lowest cost for each pair of states a graph can be defined. The solution to the problem is then found by finding the shortest cycle in this graph. The solution will be more exact and less states will be needed. It is however in this case necessary to use a good algorithm to find this shortest path.

References

[1] Joseph L. Balintfy. On a basic class of multi-item inventory problems. Man-agement Science, 10(2):287–297, 1964.

[2] A. Federgruen, H. Groenevelt, and H. C. Tijms. Coordinated replenish-ments in a multi-item inventory system with compound poisson demands. Management Science, 30(3):344–357, 1984.

[3] R. Germs, N.D. van Foreest, and O.A. Kilic. Production inventory systems with constant and compound poisson demand. working paper. 2017.

[4] Philip Melchiors. Calculating can-order policies for the joint replenishment problem by the compensation approach. European Journal of Operational Research, 141(3):587–595, 2002.

(33)

A solution approach for the two item joint replenishment problem using the g-revised cost method