Decomposition for dynamic programming in production and inventory control

Decomposition for dynamic programming in production and

inventory control

Citation for published version (APA):

Wijngaard, J. (1979). Decomposition for dynamic programming in production and inventory control. Engineering

and process economics, 4(2-3), 385-388.

Document status and date:

Published: 01/01/1979

Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be

important differences between the submitted version and the official published version of record. People

interested in the research are advised to contact the author for the final version of the publication, or visit the

DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page

numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners

and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.

• You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please

follow below link for the End User Agreement:

www.tue.nl/taverne

Take down policy

If you believe that this document breaches copyright please contact us at:

openaccess@tue.nl

providing details and we will investigate your claim.

OElsevier Scientific Publishing

Company,

Amsterdam ~ Printed in The Netherlands

DECOMPOSITION FOR DYNAMIC PROGRANNING IN PRODUCTION AND INVENTORY CONTROL

JACOB WIJNGAARD

Eindhoven University of Technology Department of Industrial Engineering

P. 0. Box 513 Eindhoven, The Netherlands

INTRODUCTION

Straightforward application of dynamic pro- gramming on production planning is impossi- ble in general by the dimensionality of the

There are two inventory points, both sup- plied by the same production unit P. $1

and 02 are the demand density functions for problem. A calculation of the state values 1 and 2. Production orders are placed at is only possible if the dimensionality can the end of each period and are assumed to be reduced, for instance by decomposition. arrive during the next period. If x, is the

A lot of production planning problems are I

almost decomposable. A system of n inven- inventory on hand at point 1 at the begin- tory points, for instance, with one supply- ning of a period and if an order of z1 units ing production unit gives an (at least) n-

dimensional dynamic programming problem. But if the production capacity is infinite

is placed then the expected inventory and stockout cost during that period is L, (x,+2,). The same for point 2. the problem reduces to n one-dimensional I I I

problems. A system of n inventory points The ordering costs are given by the func- in series, with constrained production ca- tions cl(.) and c,(e). The production pacities in between, also gives an n-dimen- capacity of P is C units per period, that sional problem. But if the production costs means z

1 + z2 2 c. Unsatisfied demand is are linear and the production capacities

infinite then the problem reduces to n one- dimensional problems (see Clark and Scarf

121).

backlogged.

The discounted dynamic programming model for this problem is

In this paper we will show with aid of two

examples how this almost decomposability can k

facilitate the application of dynamic pro- Vol(X 1' x2) = min z1fz2$

Ic,(z,) + c2(z2) gramming. The first example is a system

with two parallel inventory points. The methods used in this case are closely re-

q, z*_1p

lated to the methods used by van Beek [l] in a multi-dimensional production smoothing

+ L1(x1+21) + L2(x2+z2)

problem. The second example is a two- echelon problem.

TWO PRODUCTS WITH A JOINT PRODUCTION CAPACITY

The first system considered is the following

the construction of optimal and good strate-

1.

386

multi-dimensional production smoothing pro- blem.)

$2, L2(x2+z2), C,(')

step 3. Dse V&' x ) as starting vector 2 in the two-dimensional dynamic iteration procedure.

Now there are two interesting questions

a. r

The quality of the strategy f0 2 -?---- _{b. The number of iterations needed}

starting with v', compared with the number of iterations needed starting with the null vector.

The dynamic programming models for these problems are

“:(x1) =

min O~Z,$ {c,(z,) + L1(xl+zl) + 51 1 dl vik-L (xl+zl-dl)+l(dl)~ vZk(xl) = min OZ2~C {c,(z,) + L*(x2+22) + n 1 d2 v;k-1 (x2+s2-d2)0,(d,)l

step 2. Define the two-dimensional function 0

va(., .) by

0

vn(xl, x*) = ve L-(x,) + v~Yx*)

Execute one Z-dimensional iteration step with this function

1 v,(x 1'x2) = min z fz cc Ic,(z,) + c*(z2) 1 2- x110, s*?IO + L1(xl+sl) + L2(x2+z2) +CY _{I O} dl'd2 vd(x +a -d 1 1 L'x2+z2-d2) $(d1)+2(d2)I

This yields a feasible strategy (fo) for

the two-dimensional problem. (Van Beek [l] used this way to construct a strategy for a

With respect to practical problems the first question is the most important one of course. In large production networks it is impossible in general to execute more than one iteration step and even that first step can be difficult.

The second question has to be seen in the framework of numerical methods of dynamic programming. The point is: how can one use the structure of the problem to reduce the computation time?

We have tried to answer these questions with aid of some numerical examples.

0 1 2 3 $2(.) I--- 0.3 0.55 0.1 0.05 E(dl) = 0.9, c*(d$ = 0.59 0.1 0.2 0.5 0.2 E(d2) = 1.8, c2(d2) = 0.76

Inventory cost (linear) 1 Stockout cost (linear) 3

Set-up cost: varying from 1 . . . 10 C=3or4

The inventory levels are restricted to the points -3 . . . +6.

We considered the ratios rl and r2, where r

is the ratio of the costs under strategy f. 1

and the optimal costs rl =

costs(f0)

opt.costs

,

and r2 is the ratio of the number of iterations needed starting with vi(., .) and the number

of iterations starting with the null vector. For all values of the set-up costs we found values for r

1 of about 1.02 and for r 2 of about l/2.

It has to be mentioned here that we did not use standard value iteration but the proce-

dure of Mac Queen, combined with an idea of

van Nunen [ 31, which implies that after each

maximization step a number of value determi- nation steps has to be executed.

TWO ECHELON PROBLEMS

The results for the parallel case are rather hopeful. But if one wants to use this meth- od for whole networks it is necessary to make it suitable for multi-echelon problems. Therefore, we consider here the most simple multi-echelon problem.

22 2

0

0

5 z1 5 min(C, x2-x1)

C,(o) C,(*)

There are two echelons, echelon 2 supplying echelon 1 by a production unit P. The den- sity function of the demand per period is 9(.). Unsatisfied demand is backlogged. Orders are placed at the beginning of each period and arrive at the beginning of the next period. The expected inventory and stockout cost in a period is Ll(xl) + L2(x2)

where x 1 is the inventory on hand at echelon

1 at the beginning of the period and x2 is

the total inventory in the system. The costs for orders from outside are given by the function c,(.), the costs of production

by the function cl(.). The production ca-

pacity of P is C units per period.

The discounted dynamic programming model for this problem is k Va(Xl>X2) = Ll(X1) + L2(x2)

+

min z >o, z >o I+,) + c2(z2) l- 2- zl_lmin(C,x2-xl) (xl+zl-d, x2+z2-d)@(d)) (1)

For the case where C = m, c,(z) = cl'z and

Ll(.) is convex. Clark and Scarf [2] have k

shown that va(x1,x2) = vcI lk(xl) + "tk(x2)

where v lk and v;f" can be found in the a following way

vF(x,) =

min Tcl.zL + L1(xl) z&50 +ucv& d I"-l(x,+z,-d),(J)i (2)

“:(x2) =

min Z2L0 ic2(z2) + L2(x2) + PE(X2) + a r. "a 2k-'(x2+z2-d)$(d)i (3) d

and pi(.) is given by

pi(x,) = 0 for x2 > Sk Pi(X2) = {c1.x2 + 1 v lk-1 d a (x2-d)+(d)} - ic _{1'Sk + 7: "a} d lk-'(Sk-d)$(d)l

for x2 2 Sk and Sk is the point where the

function cl(.) + 1 vik-l(.-d)#(d) attains

its minimum. d

pk(x,) can be interpreted as a penalty for

echelon 2 of being not capable to satisfy the production order of echelon 1. This decomposability for the case C = =, c,(z) = Cl'Z, L1(.) convex suggests that it

is possible to use the same procedure as in the first problem for cases where the decom- position is not exact.

So the first step is the computation of v l-

2- cy

and v

cl from the relations (2) and (3) add- ing of course the restriction z _{15 C.}

The second step is the construction of 0

Va(., .) by va(x1,x2) = vcl 0 1m("l) + v;%$ and the construction of f o by substituting

vz in the right hand side of relation (1).

Then we can make the same comparisons as in the first problem. We have worked out the following numerical examples.

388

0 1 2 3

'$(*) 0.1 0.45 0.3 0.15 E(d) = 1.5 u(d) = 0.75

Inventory cost linear, 1 for echelon 1, 1 for echelon 2.

Stockout cost linear, 3 Set-up cost varying (K) Capacity is varying (C)

The inventory levels are restricted to the points -9 . . . +9.

If the set-up costs are high we have to change the penalty function p somewhat.

cl

In

case the set-up costs are 0, the last echelon, echelon 1, wants to reorder each period up to a level S. If x2 < S this is

not possible and the echelon 2 has to pay a penalty. But in case the set-up costs are positive the optimal strategy for echelon 1 is of the s-S type, with S-s > 0. If the difference between S and s is large echelon 1 does not order each period and it is not correct to penalize echelon 2 for having an inventory less than S if echelon 1 does not order.

To take into account this effect we replaced S in the definition of p, by s + LI. We can

try to choose LI such that echelon 2 is only penalized if echelon 1 wants to order. It is also possible to improve the results by excluding in the construction of the strategy f. possibilities which are certain-

ly not optimal. For instance, if x2 - x1 -

zl~ C then it can never be optimal that

We have the following results.

LJ

C

K 0 2 0 1.04 0.83 0 3 10 1.11 0.33 2 3 10 1.23 0.33 0 3 20 1.10 0.5 0 4 10 1.16 0.33 0 4 20 1.11 0.43 0 = 10 1.18 0.5 0 m 20 1.26 0.67 r1 r2

better results by choosing the penalty func- tion in another way.

REFERENCES

[II van Beck, P., 1977. An Application of Dynamic Programming and the HMMS-rule on Two-level Production Control, Zeitschrift fur Operations Research, 21: B133-B141.

[2] Clark, A. and Scarf, H., 1960. Opti- mal Policies for a Multi-echelon Pro- blem, Management Science, 6: 475-490.

131 van Nunen, J., 1976. A Set of Succes- sive Approximation Methods for Dis- counted Markovian Decision Problems, Zeitschrift fiir Operations Research, 20: 203-208.

Some of the results are reasonable, some rather bad. The difference between !.I = 2 and u = 0 for the case C = 3, K = 10 suggests that it may be possible to get