Stefan Minner
email: stefan.minner@univie.ac.at

Gudrun P Kiesmüller

Dynamic Product Acquisition in Closed Loop Supply Chains

Keywords: REVERSE LOGISTICS, SUPPLY CHAIN Optimal control, Inventory Reverse logistics, Closed loop supply chain management, Inventory, Pricing, Optimal control

We consider a closed loop supply chain where demands can either be satisfied from manufacturing new products or by buying back used products from customers and upgrading their functionality by remanufacturing. A joint buy-back pricing and manufacturing-remanufacturing decision model at the operationsmarketing interface model is presented that allows for dynamic parameters, e.g., product life cycles and seasonal aspects. The model allows to identify beneficial opportunities for buying back and storing used products for immediate and future recovery. We present a new deterministic, dynamic, continuous time optimization model, derive necessary and sufficient optimality conditions, and develop a solution algorithm to find the cost minimizing manufacturing and remanufacturing policies as well as buy back strategies for used products based on Pontryagin's Maximum Principle. It is shown that an optimal policy, in general, will include time intervals where returns are acquired as to synchronize demand and remanufacturing, where returns are acquired and stored for future remanufacturing, and intervals where demand is satisfied by a mix of manufactured and remanufactured products. Furthermore, we discuss several reactive and proactive acquisition and remanufacturing heuristics and show under which conditions they are optimal. The findings are illustrated by numerical examples.

Introduction

In addition to manufacturing and distribution of new products, closed loop supply chains include the processes that take place after the use of a product at the end of its use. The incentives to move from forward to closed loop supply chains are legal ones, i.e. manufacturers are assigned responsible for their products after their usage, and economic ones, i.e. parts and materials of used products still have an economic value and can be used, e.g., as service parts after some remanufacturing and upgrading processes. As a consequence of the economic incentive, a second supply mode for parts and materials becomes available which increases the complexity of operations management. Strategic and operational aspects of return network design, redistribution channel choice, collection, quality and testing, disassembly, and reassembly have to be addressed and combined with forward processes.

Various quantitative models have been developed to provide decision support for these strategic and operational decisions, see [START_REF] Fleischmann | Quantitative models for reverse logistics: A review[END_REF], [START_REF] Fleischmann | Quantitative Models for Reverse Logistics[END_REF], and [START_REF] Dekker | Reverse Logistics -Quantitative Models for Closed-Loop Supply Chains[END_REF] for overviews. The majority of the approaches faces two limitations. First, it is assumed that product returns/waste streams are passively accepted whereas there are practical needs for a product acquisition management (Guide andvan Wassenhove 2001, Sasikumar and[START_REF] Sasikumar | Issues in reverse supply chains, Part I: End-of life product recovery and inventory management -An overview[END_REF] and there is empirical evidence that firms do acquisition management [START_REF] Jr | Product acquisition management: current industry practice and a proposed framework[END_REF]. Second, the majority of the models is static and therefore ignores life cycle aspects, seasonal aspects, trends, and in general time dependencies of model parameters [START_REF] Minner | Optimal control of production and remanufacturing in a simple recovery model with linear cost functions[END_REF]. Instead of following a reactive strategy, to take back products at the end of their use, a proactive strategy appears to be promising. Instruments for such a strategy are to advertise return opportunities, i.e. create awareness for environmentally friendly products and firms, a buy back of used products as it is done, e.g., for toner cartridges, and to adjust contract design, e.g., for the lease of copiers.

A first model that addresses product acquisition management in a single period context is analyzed by [START_REF] Jr | Matching demand and Supply to Maximize Profits from Remanufacturing[END_REF]. The authors consider product returns that can be assigned to different quality classes. Within each class, customers are willing to return their product if a certain buy back price is offered. Therefore, returns for each quality class can be modeled by a return response function. The objective is to maximize the profit given by the sales of remanufactured products (determined from a sales price response function) minus the costs of acquiring respective returns and remanufacturing them. [START_REF] Jr | Matching demand and Supply to Maximize Profits from Remanufacturing[END_REF] determine optimal acquisition and selling prices under the assumption of a perfect remanufacturing process. [START_REF] Bakal | Effects of Random Yield in Remanufacturing with Price-Sensitive Supply and Demand[END_REF] drop this assumption and study the effects of random yield in the remanufacturing process. [START_REF] Galbreth | Optimal Acquisition and Sorting Policies for Remanufacturing[END_REF] allow for scrapping of products if the acquired products are too costly to remanufacture and derive the optimal acquisition and sorting policy. Pricing models for various degrees of reuse of recyclable components are presented in [START_REF] Vadde | Optimal pricing of reusable and recyclable components under alternative product acquisition mechanisms[END_REF]. They consider a single and multiple types of product returns and compare a proactive acquisition policy with a passive policy, where all returns are accepted. All these contributions have in common that they focus on a single period and therefore apply static models.

Dynamic models of pricing are used in the marketing literature to investigate diffusion of innovations (e.g., whether to use a skimming or penetration strategy) and to incorporate cost declines due to accumulated volume of production and learning effects [START_REF] Simon | Price Management[END_REF][START_REF] Rao | Pricing Models in Marketing[END_REF]. [START_REF] Kalish | Monopolist pricing with dynamic demand and production costs[END_REF] uses an optimal control approach in order to determine the optimal pricing policy for a monopolist, [START_REF] Dockner | Optimal advertising policies for diffusion models of new product innovations in monopolistic situations[END_REF] International Journal of Production Research 3 impact of advertising policies (see [START_REF] Feichtinger | Dynamic Optimal Control Models in Advertising: Recent Developments[END_REF] for an overview) and [START_REF] Mesak | Monopolist Optimum Pricing and Advertising Policies for Diffusion Models of New Product Innovations[END_REF] investigate both marketing mix instruments simultaneously. A slightly different model with pricing and warranty period decisions is presented in [START_REF] Mesak | Modelling Monopolist Pricing and Protection Period Decisions for New Products under Warranty[END_REF].

For a general overview on pricing theory and practice we refer to [START_REF] Noble | Industrial Pricing: Theory and Managerial Practice[END_REF].

The main difference to the existing literature is that our model allows for dynamic demands and returns and recoverable items. Used products can be bought back under beneficial circumstances and stored for later use. As a consequence, the combined analysis of waste stream and product acquisition management is possible [START_REF] Guide | Managing Product Returns for Remanufacturing[END_REF]. In contrast to other dynamic pricing models in marketing, we do not model spill over of marketing mix instruments, e.g. past pricing and customer expectations, effects of advertising, and competitive reactions. The only dynamic carry over that our model allows are inventories. This analysis is another contribution to the field of joint production and marketing models (see [START_REF] Eliashberg | Marketing-Production Joint Decision Making[END_REF].

In a dynamic model with product returns and remanufacturing, a different type of cost reductions appears. Returns of products and the opportunity to remanufacture them, which, in general, is assumed to be cheaper than manufacturing a new unit, offers a second supply mode [START_REF] Minner | Multiple Supplier Inventory Models in Supply Chain Management: A Review[END_REF]. However, this opportunity depends on the product's life cycle characteristics and the return behavior of the customers. In the following we analyze a dynamic model of product acquisition and recovery. In contrast to [START_REF] Jr | Matching demand and Supply to Maximize Profits from Remanufacturing[END_REF] we assume that the demands cannot be influenced by pricing, i.e. the firm is a price taker on the sales side and that there is only a single market, i.e. we do not consider market segmentation effects between new and remanufactured products. Further, we consider a single quality level for returns which can be influenced by buy back offers according to a price/effort response function. Note that in the context of our model, buy back has a different meaning than in supply chain coordination where buy back (for new products not sold to customers) contracts are a mechanism to achieve channel coordination (see e.g. [START_REF] Lariviere | Supply Chain Contracting and Coordination with Stochastic Demand[END_REF]).

The paper is organized as follows. In Section 2 we present a static model for a single period problem and linear response functions. This illustration, which is a special case of the model analyzed in [START_REF] Jr | Matching demand and Supply to Maximize Profits from Remanufacturing[END_REF], provides the basics of the effects that will appear in the dynamic model. Then, we present the additional assumptions for the dynamic model and give the optimal control optimization problem. In Section 3, we apply Pontryagin's Maximum Principle to state the necessary (and in our case also sufficient) conditions for an optimal solution. Using these conditions, we further characterize the optimal recovery and buy back policy. These results are exploited to develop an algorithm that constructs an optimal policy for arbitrary demand functions and parameters of a linear return response function. In Section 4 we discuss some simple heuristic policies and show some illustrative examples in Section 5. Finally, in Section 6, we summarize the main findings and discuss streams for further extensions of the framework.

A static demand and acquisition model

Consider a remanufacturer who is selling a single product over a single period. The customer demand d > 0 for this product is deterministic. There are two options available for satisfying demand, manufacturing of x units at a unit cost of c p and remanufacturing u units at a unit cost of c u . We assume that all demands have to be satisfied which leads to d = x+u. Due to legislation the remanufacturer is obliged to accept all offered returns. If more than the required u units are returned, there is a disposal option where z units are salvaged at a unit cost c z . This cost rate might be negative if there exist revenues from selling scrap. For example, the remanufacturer may sell the returns to a material recycling company. Demands to be satisfied using the remanufacturing option require returns. The remanufacturer is applying a proactive acquisition policy and offers a price incentive to customers who return their used products. We assume that returns of used products can be initiated according to a return response function r = f (p) ≥ 0, where p ≥ 0 is the buy back price offered to customers that have the product in use. An alternative, more general, interpretation is that p describes the level of effort put into product acquisition, i.e. advertising or paid transportation for returning the used item. Similar to [START_REF] Bakal | Effects of Random Yield in Remanufacturing with Price-Sensitive Supply and Demand[END_REF] or [START_REF] Galbreth | Optimal Acquisition and Sorting Policies for Remanufacturing[END_REF], we assume a linear return response function

f (p) = a + bp a, b ≥ 0 (1)
which is the most simple type of response function found in marketing and economics literature. For other widely used and more general response functions we refer to [START_REF] Simon | Price Management[END_REF]. The number of returns consists of a number of autonomous, effort/price independent returns a and a parameter b that expresses the price sensitivity of customers. This implies that there are always some customers willing to return their products (if a > 0) and that the return quantity increases with increasing acquisition effort. The remanufacturer determines in the optimal buy back price as well as in the optimal quantities to remanufacture, produce or to dispose of in order to minimize cost. We can formulate the problem as a constrained non-linear minimization problem.

min x,u,z,p c p x + c u u + c z z + p(a + bp) (2) s.t. x + u = d (3) u + z = a + bp (4)
x, u, z, p ≥ 0.

(5)

The first constraint (3) ensures that demand is satisfied while the second constraint (4) reflects that all returns are either remanufactured or disposed of. Obviously, all decision variables have to be non-negative. We can reduce the problem size using x = d -u and z = a+bp-u. Then, the equivalent optimization problem with price and remanufacturing quantity as decision variables is obtained.

min p,u c p (d -u) + (c u + p)u + (c z + p)(a + bp -u) s.t. u ≤ d u ≤ a + bp p, u ≥ 0. ( 6
)
The optimal solution of this problem is characterized in Theorem 2.1:

Theorem 2.1 : The optimal buy back and recovery strategy depends on the recovery cost advantage c p -c u and the disposal cost c z . The optimal decisions p * , x * , u * , z * are

• Region A: -c z ≤ (2d -a)/b, c p -c u ≥ (2d -a)/b,
p * = (d -a)/b, x * = 0, u * = d, z * = 0 • Region B: (2d -a)/b > c p -c u > a/b, c p + c z -c u ≥ 0, -c z ≤ (2d -a)/b, a < d p * = b(c p -c u )-a 2b , x * = d -u * , u * = a+b(c p -c u ) 2 , z * = 0 • Region C: c p + c z -c u ≥ 0, c p -c u ≤ a b , a < d p * = 0, x * = d -a, u * = a, z * = 0 • Region D: c p + c z -c u ≥ 0, c p -c u ≥ (2d -a)/b, -c z ≥ (2d -a)/b p * = -(a + bc z )/(2b), x * = 0, u * = d, z * = a + bp -d • Region E: c p + c z -c u ≤ 0, -c z ≤ a/b p * = 0, x * = d, u * = 0, z * = a • Region F: c p + c z -c u ≤ 0, -c z > a/b p * = -(a + bc z )/(2b), x * = d, u * = 0, z * = a-bc z 2 • Region G: a ≥ d, c p -c u ≥ -c z , -c z ≤ a b p * = 0, x * = 0, u * = d, z * = a -d
The proof of Theorem 2.1 is presented in Appendix A. For the case of practical interest where d > a, Figure 1 illustrates these regions.

E T c p -c u -c z A D B F C E a b a b 2d -a b c p + c z -c u = 0 Figure 1. Illustration of regions for d > a
In cases where remanufacturing (without considering acquisition) is generally profitable, i.e. where the cost of disposing a return and manufacturing a new item is more expensive than remanufacturing (c p + c z > c u ), we have different policies being separated by threshold values for the recovery cost advantage c p -c u . If the recovery cost advantage is positive but low c p -c u ≤ a b , it is optimal to remanufacture all autonomous returns a, not to buy additional returns p = 0 and to manufacture the remaining demand d -a (Case C). If the recovery cost advantage is sufficiently large

c p -c u ≥ 2d-a b ,
E T c p -c u -c z D F a b c p + c z -c u = 0 E G Figure 2. Illustration of regions for d ≤ a
optimal to buy back as many returns as being required to satisfy demand from recovery products in full (Region A). If further the salvage value is positive and large enough (-c z ≥ 2d-a b ), further returns are acquired and beneficially salvaged (Region D). For intermediate recovery cost advantages a b < c p -c u < 2d-a b , it is optimal to buy returns only at a maximum price, to remanufacture all returns and to manufacture the remaining demand.

The threshold values are defined as follows: c p -c u = a b defines the recovery advantage where the optimal price becomes zero and c p -c u = 2d-a b determines the buy back price such that returns equal demands. If autonomous returns a are larger or equal to the demand (see Figure 2), we find that regions E and F remain. If c p + c z -c u ≥ 0 but salvaging returns is not beneficial enough (-c z ≤ a b ), it is optimal to remanufacture for demand and to dispose the remaining autonomous returns (Region G) whereas if -c z > a b remanufacturing for demand and salvaging of excessive returns (Region D) is optimal.

A dynamic demand and acquisition model

Seasonal and life cycle demand is common in Supply Chains. In order to represent these dynamics over a finite planning horizon t ∈ [0, T ], we model demand during the selling season as a continuous and differentiable function of time (d(t) ≥ 0, t ∈ [0, T ]). In order to match supply with demand, the remanufacturing facility needs different quantities of returns at different moments in time. Acquiring the right amount of returns is aggravated by dynamic supply. However, during periods where there is not enough supply, the remanufacturer can encourage customers to return their products by setting economic incentives. We assume a dynamic buy back price function p(t), and continuous and differentiable functions a(t), b(t), resulting in a time dependent return response function

r(t) = a(t) + b(t)p(t).
If supply of returns exceeds demand, the remanufacturer has, besides the disposal option, the possibility to keep returned products in recoverables inventory which is subject to out of pocket inventory holding cost rate h for recoverables y(t) ≥ 0. Opportunity costs for capital investment are incorporated by discounting payments with a constant discount rate of ρ.

In order to avoid trivial solutions, we make the following assumptions concerning the cost structure.

(α1) Remanufacturing a unit is always cheaper than disposing of a return and manufacturing a new one (c p + c z -c u > 0). Otherwise the optimization can be decoupled in a way that all demands are manufactured and all returns are disposed of (see Regions E and F in the static analysis). (α2) It is not advantageous to buy returns for disposal (-c z ≤ a(t)/b(t)). (α3) The interest benefit from delaying disposal is smaller than the holding cost (ρc z < h). Otherwise unnecessary units would never be disposed of but kept in inventory.

The state of the system to be controlled is recoverable inventory y(t). Initial and final inventory levels are assumed to be zero, y(0) = y(T ) = 0. The manager has to decide when to build up recoverables inventory and when to stop economic incentives for product returns. Additionally, he has to determine the amounts to be remanufactured and to be disposed of. The associated movement of the state in continuous time is

ẏ(t) = r(t) -u(t) -z(t) ∀t ∈ [0, T ] (7)
where ẏ(t) denotes the first derivative of the function y(t) with respect to time. The objective is to minimize total discounted cost C over a finite planning horizon of length T subject to satisfying all given demands.

C = T 0 e -ρt c p x(t) + c u u(t) + c z z(t) + p(t)

r(t) + hy(t) dt

Under the assumption that demand always has to be satisfied and cannot be backordered we can replace x(t) = d(t) -u(t) and the remaining control variables at each instant of time are the price p(t) for returns, the remanufacturing decision u(t), and the disposal decision z(t). The resulting optimal control problem is Table 1. Optimal pricing and recovery policy for state y(t) = 0

min p,u,z T 0 e -ρt (c u -c p )u(t) + c z z(t) + p(t)(a(t) + b(t)p(t)) + hy(t) dt s.t. 0 ≤ u(t) ≤ d(t) t ∈ [0, T ] ẏ(t) = a(t) + b(t)p(t) -u(t) -z(t) t ∈ [0, T ] (8) y(0) = y(T ) = 0 y(t) ≥ 0, p(t) ≥ 0, u(t) ≥ 0, z(t) ≥ 0 t ∈ [0, T ].
a(t) 2 -c z b(t) 2 ≤ d(t) Condition d(t) < a(t) 2 -c z b(t) 2 and d(t) > a(t) 2 + b(t) 2 (c p -c u ) d(t) ≤ a(t) 2 + b(t)(cp-cu) 2 Solution u * (t) = d(t) u * (t) = d(t) = r * (t) u * (t) = r * (t) x * (t) = 0 x * (t) = 0 x * (t) = d(t) -u(t) z * (t) = r(t) -d(t) z * (t) = 0 z * (t) = 0 p * (t) = 0 p * (t) = d(t)-a(t) b(t) p * (t) = max 0, b(t)(c p -c u )-a(t) 2b(t) λ * (t) = -cz λ * (t) = 2d(t)-a(t) b(t) λ * (t) = cp -cu

Optimal acquisition and remanufacturing policy

In order to derive the optimal decisions we split the analysis into three parts. We first present the situation where no returns are kept in inventory. Then we analyze the optimal manufacturing and pricing decisions under positive inventories, and in the last step we combine both situations and their transitions.

Theorem 3.1 : Given a state where the inventory level is equal to zero y(t) = 0 and not leaving this state, the optimal decisions are given in Table 1.

Proof : See Appendix B.
As in the static case, the optimal decisions depend on the disposal costs c z and the cost difference between manufacturing and remanufacturing (c p -c u ). Since inventory is empty and it is never beneficial to buy returns for disposal (α1), it is only optimal to dispose of items when autonomous returns are larger than demand and the company is already getting more used products than needed (first column of Table 1). This case corresponds to Region D and a > d in the static case. If the difference between manufacturing and remanufacturing costs is relatively large, then complete synchronization of supply and demand, as described in column 2 of Table 1, is optimal. This case corresponds to region A in the static model. Manufacturing can be optimal in two different situations. First, if the remanufacturing advantage is small (c p -c u ≤ a(t)/b(t)), then it is optimal to set the price equal to zero and just to remanufacture autonomous returns (Region C in the static model). On the other hand manufacturing may be required if not enough returns can be bought at a maximum economical price. (Region B in the static model). Taking a different perspective from the magnitude of demand, small demands d(t) are satisfied from autonomous demands only (if possible), for intermediate demands, return and demand can be synchronized by setting an appropriate buy-back price, whereas for large demands, returns are remanufactured up to an economical level and the remaining demands are manufactured.

Furthermore, λ * (t) has an economic interpretation as the (marginal) value of an additional return at t. If returns are disposed, the value equals the salvage value/disposal cost λ = -c z . In the synchronization case, λ = 2d(t)-a(t) b(t) , i.e., the value follows the development of the demand and buy-back conditions whereas in case manufacturing is required, λ(t) = c p -c u and the value equals the recovery cost advantage.

Under the assumption of positive recoverables inventory and no transition to the zero inventory state, only remanufacturing for demand is optimal.

Theorem 3.2 : Given the inventory level is positive (y(t) > 0) and not leaving this state, the optimal decisions are:

u * (t) = d(t), x * (t) = 0, z * (t) = 0 p * (t) = max{0, λ(t) 2 - a(t) 2b(t) } with λ(t) = - h ρ + λ(t 0 ) + h ρ e ρ(t-t 0 ) < c p -c u Proof : See Appendix C.
Intuitively, disposing of products simultaneously with building up inventories cannot be optimal. Similarly, there is no need for more expensive manufacturing as long as there is inventory of returns. Note that the optimal buy back price can be expressed as a function of the adjoint variable λ(t) which is the shadow price (value) of an additional return at time t. Note that time t 0 denotes either the start or end of an interval with positive inventory. Since there is a maximal buy-back price which is economically reasonable, we have an upper bound for the number of returns to be acquired r * (t) < a(t)+b(t)(cp-cu) 2 . Besides the optimal decisions in one state we require the optimal moments in time of transitions, especially when to start collecting returns for building up recoverables inventory. Let t s denote the start of an optimal time interval with positive inventory and t e the end of such an interval. Proposition 3.3: (Collection conditions) For optimal time intervals [t s , t e ] with positive inventories (y(t) > 0) the following conditions have to hold:

t e t s (d(s) -(a(s) + b(s)p * (s))ds = 0 (9) t ts (d(s) -(a(s) + b(s)p * (s))ds < 0 t s < t < t e (10)
Over the entire interval, cumulative returns have to equal cumulative demands (such that inventory depletes to zero at the end of the interval). For any time t within the interval, inventory has to be positive, i.e. cumulative returns up to t have to exceed cumulative demands up to t.

There are two main reasons for collecting returns to build up recoverables inventory. First, (see Proposition 3.4) when customers are highly motivated to return used products and there is excess of autonomous returns (a(t) > d(t)), the remanufacturer has to chose between the two options of disposing of returned items or keeping products in stock for later use. The optimal decision depends on the disposal cost parameter c z and the return response function.

Proposition 3.4: (Excess autonomous returns) Let us consider time points θ with

d(θ) = a(θ), d(θ -1 ) < a(θ -1 ), and d(θ + 2 ) > a(θ + 2 ) for small values of 1 , 2 > 0.
The optimal inventory at time θ has to satisfy the following condition:

y * (θ) > 0 if 2d(t)-a(t) b(t) > -c z ; t ∈ (θ -1 , θ) = 0 if 2d(t)-a(t) b(t) ≤ -c z ; t ∈ (θ -1 , θ) . ( 11
)
Proof : See Appendix D.

Note that the two cases of ( 11 The second reason for building up inventories (see Proposition 3.5) is motivated by a required price increase in order to acquire the demand quantity related to the costs for buying in advance and storing the item. Instead of synchronizing demand and returns the remanufacturing facility can acquire more returns and build up recoverable inventory.

Proposition 3.5: (Economic incentives) Let us assume that demand satisfies the following inequality

a(t) 2 -c z b(t) 2 ≤ d(t) < a(t) 2 + b(t) 2 (c p -c u ). ( 12
)
For all time points θ satisfying

b(θ)(2 ḋ(θ) -ȧ(θ)) -2d(θ) ḃ(θ) -a(θ) ḃ(θ) b 2 (θ) > ρ d(θ) -a(θ) b(θ) + h (13)
and ( 12), the optimal inventory has to be positive y * (θ) > 0.

Proof : See Appendix E.

This situation occurs if demands and returns are synchronized, Region B in the static model, second column in Table 1). If due to the dynamic of the parameters a(t), b(t), or d(t) further synchronization is more expensive (expressed through the left-hand side in (13) which is the slope of 2d(t)-a(t)

b(t)

, than buying in advance and storing, expressed through the right-hand side, becomes beneficial and results in positive inventory.

Algorithm

For the computation of the optimal price and recovery policy we have to determine the optimal time intervals of zero and positive inventory, respectively the starting points for building up inventories. In the following we sketch an algorithm for constructing the optimal solution. We start with an initial solution as given in Table 1 which means, we assume the three subcases for y(t) = 0 are the status quo. Transitions from having zero recoverables inventory into collecting returns is required in order to satisfy the necessary optimality conditions as stated in Proposition 3.4 and 3.5.

The construction of optimal time intervals [t * s , t * e ] with positive recoverable inventory due to excess autonomous returns starts with determining intervals [t

(0) s , t (0) e ] with d(t) < a(t) ∀t ∈ (t (0) s , t (0) e ), d(t (0) s ) = a(t (0) s ) and d(t (0) e ) = a(t (0)
e ). Since the shadow price λ(t) has to be continuous at the end of the interval, and based on Proposition 3.3 and 3.4, the following procedure computes the optimal interval [t * s , t s and λ(t s ) > -c z , we have to increase t e . If λ(t s ) < -c z , a smaller value of t e is required and the procedure continues with STEP 1 and the new value for t e .

Similarly, we find the starting points of building up inventories due to economic incentives using Proposition 3.5 and applying an iterative procedure. We first determine intervals

[t 1 , t 2 ] with b(t i )(2 ḋ(t i ) -ȧ(t i )) -2d(t i ) ḃ(t i ) -a(t i ) ḃ(t i ) b 2 (t i ) = ρ d(t i ) -a(t i ) b(t i ) + h i = 1, 2 (16)
and ( 13) holds for all θ ∈ (t 1 , t 2 ). Next we have to calculate [t 3 , t 4 ] with

d(t 3 ) = a(t 3 ) 2 -c z b(t 3 ) 2 and d(t 4 ) = a(t 4 ) 2 + b(t 4 ) 2 (c p -c u ). ( 17
)
The following algorithm starts with the initial interval [t

(0) s , t (0) e ] = [t 1 , t 2 ] ∩ [t 3 , t 4
] and constructs intervals with positive recoverable inventories based on Proposition 3.3 and utilizing the continuity of the shadow price λ(t) at the interval boundaries. STEP 1: Choose t e > t (0) e and compute λ(t e ) from ( 14). STEP 2: Compute t s < t e using (15) and

λ(t s ) = 2d(t s ) -a(t s ) b(t s ) . ( 18
)
STEP 3: Evaluate the integral as given in (9). STEP 4: If the value of the integral is equal to zero, then STOP. If it is positive, then there is an excess of demands and t e has to be decreased. Otherwise, if the value of the integral is negative, there are too many returns and we have to increase t e . Continue the procedure with STEP 1 and the new value for t e .

These procedures can easily be implemented because only integrals have to be computed and equations have to be solved within a search and bisection scheme.

Heuristic strategies

Instead of deriving and implementing the optimal policy, alternative heuristic strategies can be applied. The first two heuristics are using a reactive acquisition policy, while heuristic three and four are based on a proactive acquisition policy. We derive conditions, depending on the cost parameters c p , c u , and c z , and the return response function under which these simple strategies provide us with the optimal solution. The propositions in the following can easily be derived using Theorems 3.1 and 3.2. No buy back of returns, collect excess autonomous returns (AUTO2)

F

All returns are collected and no additional price incentives are offered by the remanufacturer to increase the return quantities. Thus, the buy back price is always equal to zero p(t) = 0. Excess returns (a(t) > d(t)) are kept in inventory and disposal does not take place (z(t) = 0). Manufacturing and remanufacturing quantities depend on whether there is recoverables inventory available or not.

y(t) = 0 : x(t) = max{0, d(t) -a(t)}, u(t) = min{d(t), a(t)} y(t) > 0 : x(t) = 0, u(t) = d(t)
If there is positive recoverables inventory, no items are manufactured and demands are met from remanufactured units only. In case of zero inventory, the minimum of demand and returns is remanufactured and missing items are manufactured.

Proposition 4.2: If c p -c u ≤ a(t)/b(t) for all t ∈ [0, T ] and no excessive return has to be stored longer than beneficial, then heuristic AUTO 2 is optimal.

Synchronization of returns and demands (SYN)

Consider a strategy aiming at a complete synchronization of supply of returns and demand. The buy back price is therefore dynamically adapted such that as many returns as needed are acquired p(t) = max{0, (d(t) -a(t))/b(t)}. Demand is completely satisfied from remanufactured items x(t) = 0, u(t) = d(t). Additionally, no returned units are kept on stock (y(t) = 0), and excessive returns are immediately disposed of z(t) = max{0, a(t) -d(t)}. Proposition 4.3: If a(t) ≤ d(t) for all t ∈ [0, T ] and the following two conditions hold for all t ∈ [0, T ]

-c z ≤ (2d(t) -a(t))/b(t) ≤ c p -c u (19) and b(t)(2 ḋ(t) -ȧ(t)) -2d(t) ḃ(t) -a(t) ḃ(t) b 2 (t) ≤ ρ d(t) -a(t) b(t) + h, ( 20
)
then heuristic SYN is optimal.

Optimal static policy (STA) Another simple approach is to implement the optimal static decision for each instant of time. Therefore, no inventories are required and the policy is given in Section 2.

Proposition 4.4: If i) ( 19) and ( 20

) hold or if ii) (2d(t) -a(t))/b(t) ≥ c p - c u for all t ∈ [0, T ], then heuristic STA is optimal.

Numerical examples

We illustrate our results using the following seasonal demand pattern:

d(t) = 20 + 10 sin(t -π), t ∈ [0, 4π] (21)
Demand varies between 10 and 30 units, i.e. 10 ≤ d(t) ≤ 30. The discount rate is ρ = 0.01, i.e. 1% continuous interest per unit of time which in our case is, e.g., a month. For the out of pocket holding cost rate we have chosen h = 0.05.

Example 1: a = 0, b = 5, c p = 10, c u = 4 and c z = 0 Since there are no autonomous returns we only have to determine starting points for building up inventories when buying returns in advance and storing them is less expensive than manufacturing or increasing the buy-back price.

According to Theorem 2 we have to solve the following equation in order to compute the intervals where manufacturing is required in case of no stockkeeping.

resulting in [1.5933,4.6699], [7.8765,10.9531]. This leads to the following intervals where synchronization of returns and demands does not satisfy the necessary optimality conditions: [1.5933, 2.6180] and [7.8765, 8.9012]. At least during these intervals recoverables inventory has to be positive. These initial intervals are enlarged according to the algorithm described above such that the adjoint variable is a continuous function and the collection conditions of Proposition 4 hold. In the optimal solution, inventory is held during [0.6226, 3.6935] and [6.9058, 9.9760]. Therefore, the optimal pricing and recovery policy is as follows:

i) During the time interval [0, 0.5236] buy used items at a constant rate r = 15 for p max = 3, remanufacture them and produce for the remaining demands. ii) During the time interval [0.5236, 0.6226] buy as many returns as being required to meet demand from remanufactured units. iii) During the time interval [0.6226, 3.6935] build up a recoverable inventory and use returns and, if necessary, items from the recoverable inventory to satisfy demands. iv) During the time interval [3.6935, 2π] buy returns at a constant rate r = 15 for p max = 3 and manufacture for remaining demands. v) In the interval [2π, 4π] the optimal policy is the same as in [0, 2π].

For a graphical illustration of the optimal recovery policy, see Figure 3 and for the optimal price, Figure 4.

For the comparison of the optimal strategy with the heuristics mentioned in Section 4 we introduce the relevant costs C rel , since we cannot avoid satisfying demand at least from remanufactured units. Subtracting the portion of the costs which cannot be influenced leads to the following definition of the relevant costs In order to illustrate the impact of different return response function parameters, we investigate different settings for b. The relevant costs are given in Table 2. For b = 2.5 the static policy is optimal since c p -c u < (2d(t) -a(t))/b(t) for all t. Returns are bought back with a constant rate of r = 7.5 units at a price of p max = 3 and the remaining demands are satisfied by production. In case of b = 10, it is never optimal to produce. The synchronization infeasibility interval is [1.606, 4.657], the interval with positive inventory becomes [0.133, 6.145].

C rel = C -c u T 0 d(s) ds.
These examples show the importance of the reaction parameter b on the minimal cost. A higher return response to increasing prices or acquisition efforts significantly reduces costs. Therefore, actions that influence this parameter, e.g. increasing visibility of buy back offers via Internet/E-commerce, creating customer awareness of buy back activities, paid return shipments will be promising.

Example 2: a = 15, b = 5, c p = 10, c u = 8, c z = 1, h = 1, ρ = 0.01 This example illustrates the case where intervals with excess autonomous returns become relevant. For this specific setting of parameters we find that c p -c u ≤ a/b and therefore p(t) = 0 for all t ∈ [0, T ], i.e. it is never optimal to engage in active acquisition efforts. We obtain the following intervals where positive inventory is required due to economic incentives (Proposition 3.5):

[1. 8264, 2.2935] and [8.1096, 8.5767] (24) Time intervals with less demand than autonomous returns are given by [0.5236,2.6180] and [6.8068,8.9012]. Since these intervals include the intervals given in (24), we start with the construction of intervals with positive inventory due to excess autonomous returns. Since the cost for storing returns from t = 0.5236 offset the savings of c p + c z over c u the optimal solution starts keeping returns at t s = 0.8029. In the optimal solution the inventory is running empty at t e = 3.7883 and therefore, the inventory is positive also for the intervals given in (24). The optimal policy is the following: i) Remanufacture autonomous returns with rate a(t) and manufacture the remaining demand in [0, 0.5236]. ii) Remanufacture for demand and dispose excess returns in [0.5236, 0.8029]. iii) Remanufacture for demand and collect excessive returns respectively take required recoverables from inventory in [0.8029, 3.7883]. iv) Remanufacture autonomous returns and manufacture remaining demand [3.7883, 2π]. v) In the interval [2π, 4π] the optimal policy is the same as in [0, 2π].

In Figure 5, the optimal manufacturing, remanufacturing, and disposal policy is depicted. As a consequence of switching from manufacturing to disposal at t = 0.5236 and t = 6.8068 the adjoint variable has a downward jump from c p -c u to -c z , for an illustration see Figure 6. In a second variant of Example 2 we show a similar case where used products are purchased. Therefore, let b = 10. Synchronization is not optimal in the interval [2.099, 4.164], and additional manufacturing under zero inventories is required for [0, 0.253] and [2.889, 2π]. The application of our algorithm to this instance reveals that in the interval [0, 0.253] returns are acquired and remanufactured at the maximum rate of 17.5 and excessive demands are manufactured, returns and demands are synchronized in [0.253, 0.524], demands are remanufactured and excessive returns disposed of in [0.524, 0.847], positive recoverables inventory is optimal for [0.847, 3.833] where in [0.847, 3.342] only autonomous returns are collected and additional returns are bought by acquisition activities in [3.342, 3.833]. In [3.833,2π], again returns are acquired and remanufactured with a rate of 17.5 and excessive demands are manufactured. Thereafter, the same policy is repeated. The performance of the policies is given in Table 3.

Summary and outlook

In this paper we have presented a dynamic optimisation approach to coordinate manufacturing and remanufacturing using both, inventory and pricing mechanisms to control the product return and recovery process. The general framework of optimal control allows to obtain some analytical relationships between the optimal recovery and buy back strategy. However, the overall optimal policy has to be determined numerically. Incentives for storage of returns are either the collection of excessive autonomous returns or a cost beneficial buy-back in expectation of less favorable buy-back conditions or increasing demands. Additionally, our analysis enables us to show under which conditions some simple heuristics are optimal. Using a passive return acquisition policy is in general not close to optimality. This heuristic leads only to good results if the number of products brought back autonomously is already quite large. Otherwise large improvements can be achieved through a proactive return acquisition management.

There are several extensions to generalize the results. Concerning the return response functions, more general functions rather than a linear relationship can be analyzed. Further, carry over effects of past decisions, especially the market size, can be incorporated by introducing a second stock variable (market size) which is influenced by sales and take back and then, the response function depends on the market size and the price. Another extension is to consider joint sales and return pricing where in addition, price bundling effects with a joint price for a sale in case an item is returned can be analysed. The presented approach can also be generalized with respect to the underlying market structure. As a first extension one can think of two separate markets for new and remanufactured products that will have different sales prices and to analyse the associated market segmentation effects. Further, the impact of having several players, e.g. original equipment manufacturers and competing remanufacturing firms that both aim to acquire used products is another stream for future research. (3) The adjoint variable has to satisfy the differential equation:

λ(t) = ρλ(t) + h -µ 1 (t) (B9)
(4) Complementary Slackness conditions have to hold: µ 1 (t)y(t) = 0, µ 1 (t) ≥ 0, µ 2 (t)p(t) = 0, µ 2 (t) ≥ 0, µ 3 (t)u(t) = 0, µ 3 (t) ≥ 0 µ 4 (t)z(t) = 0, µ 4 (t) ≥ 0 µ 5 (t)(d(t) -u(t)) = 0, µ 5 (t) ≥ 0 (B10)

(5) The adjoint variable λ(t) is a piecewise continuous function which can have jumps at certain points in time. Possible candidates are time points θ with a(θ) = d(θ) and y(θ) = 0.

Since the objective function is separable in the decision variables, linear in u and z, and quadratic and convex in p, the necessary conditions are sufficient. Using (B6) the optimal buy back price can be expressed as a function of the adjoint variable λ(t) which is the shadow price (value of an additional return) at time t.

p * (t) = λ(t) 2 -a(t) 2b(t) if λ(t) > a(t) b(t) 0 if λ(t) ≤ a(t) b(t) (B11)
The resulting optimal return quantity becomes r * (t) = d(t). This requires λ(t) ≥ c p -c u which is not possible for an interval with positive length because λ(t) > 0 and λ(t) > c p -c u requires u(t) = 0 which would imply that recoverables inventory will never be depleted to zero which is required as the final inventory level.

Assume z(t) > 0. Then, µ 4 (t) = 0, λ(t) = -c z from (B4), and λ(t) = 0 which contradicts (C2). Summarizing, in an interval with y(t) > 0, we have u * (t) = d(t) and z * (t) = x * (t) = 0.

From λ(t) < c p -c u it follows that there exists an upper limit for the optimal return rate,

r * (t) < a(t) + b(t)(c p -c u ) 2 . ( C3
)
The lower limit is a(t). Depending on the value of λ(t), i.e. whether λ(t) ≤ a(t)/b(t) or λ(t) > a(t)/b(t), the buy back price is either zero or positive.

  20 + 10 sin(t -π) = 15 (22) the resulting intervals are [0,0.5236], [2.6180,6.8068], [8.9012,12.5664]. In order to identify time points satisfying (13) we have to solve 4 cos(t -π)

  Figure 3. Optimal recovery policy

Figure 6 .

 6 Figure 5. Optimal recovery policy

  (A5) and (A6), µ 2 = c p + c z -c u from (A4). Further, c p -c u ≤ a b from inserting µ 2 into (A1) andc p + c z -c u ≥ 0 from µ 2 ≥ 0 which leads to -c z ≤ a/b. This defines Region C. • u = d, u = a + bp, p > 0 p = (d -a)/b. Then, µ 2 = (2d -a + bc z )/b from (A2). µ 2 ≥ 0 ⇔ -c z ≤ (2d -a)/b, µ 1 = c p + c z -c u -µ 2 ≥ 0 ⇔ c p -c u ≥ (2d -a)/b. This defines Region A. • u = d, u < a + bp, p > 0 µ 2 = 0 from (A8), µ 1 = c p + c z -c u ≥ 0 from (A4), p = -(a + bc z )/(2b) from (A1) and (A2), x = 0, z = a + bp -d, -c z > a/b from p > 0 and -c z > (2d -a)/b from u < a + bp. This defines Region D. • 0 < u < d, u = a + bp, p > 0 µ 1 = 0 from (A6), µ 2 = c p + c z -c u from (A4), p = b(cp-cu)-a 2b from (A1) and (A2), u = a+b(c p -c u ) p * (t) = arg max{H(p(t), u * (t), z * (t))} (B5)(2) Maximization of the Lagrange-function (B2) leads to the following equations:∂L ∂p = 0 ⇔ (λ(t) -p(t))b(t) -(a(t) + b(t)p(t)) + µ 2 (t) = 0 (B6) ∂L ∂u = 0 ⇔ c p -c u -λ(t) + µ 3 (t) -µ 5 (t) = 0 (B7) ∂L ∂z = 0 ⇔ -c z -λ(t) + µ 4 (t) = 0 (B8)

  If salvaging of returns is beneficial, no inventories are built whereas if synchronization of returns with demands has a larger value than disposal, than excessive items are collected.

	ctober 23, 2010	18:6	International Journal of Production Research	IJPR
		10	S. Minner and G.P. Kiesmüller
		the static model.
			F o r
			P
			e
			e r
			R
			e
				v i e
				w
				O n l
				y

) correspond to the separation of

Regions A and D in

  Under this strategy, we only collect and remanufacture a maximum of a(t) units. The buy back price is always equal to zero (p(t) = 0). If d(t) < a(t), excess returns are immediately disposed of, i.e. z(t) = max{0, a(t) -d(t)}, otherwise manufacturing is used to obtain missing items to satisfy demand, i.e. x(t) = max{0, d(t) -a(t)}, and u(t) = min{d(t), a(t)}.

	ctober 23, 2010	18:6	International Journal of Production Research	IJPR
		No buy back of returns, zero inventory (AUTO1)
		Proposition 4.1: If d(t) > a(t) and c p -c u ≤ a(t)/b(t) for all t ∈ [0, T ], then heuristic
		AUTO1 is optimal.
			o r
			P
			e
			e r
			R
			e
				v i e
				w
				O n l
				y

Table 2 .

 2 Relevant costs heuristics

		Optimal AUTO1 AUTO2	SYN	STA
	b = 2.5	1180.22			2172.86 1180.22
	b = 5	908.17	1454.26	1454.26 1086.43	916.87
	b = 10	487.93			543.21	543.21

http://mc.manuscriptcentral.com/tprs Email: ijpr@lboro.ac.ukInternational Journal of Production Research

. Further, (2d -a)/b > c p -c u > a/b from u < d and p > 0. This defines Region B.

Appendix A. Proof of Theorem 2.1

In order to solve the optimization problem (6) we have to minimize a non-linear (quadratic), convex function subject to linear constraints. Using a Lagrange-approach we define the Lagrange-function

Due to the convexity of the objective function, the following Kuhn-Tucker conditions are necessary and sufficient for an optimal solution.

∂L ∂p

= (a + 2bp + bc z ) -bµ 2 ≥ 0 (A1)

For the two decision variables u, p we identify the following cases: A3) and (A1). This identifies region E.

from (A3) and due to p > 0. This identifies region F. • 0 < u < min{d, a + bp}, p = 0 µ 1 = µ 2 = 0 from (A6) and (A8), c p + c z -c u = 0, -c z ≤ a/b from (A3) and (A1). All u, x combinations have the same objective function value. Since this is a boundary case, it can be regarded as a special case of region E with u = 0. • 0 < u < min{d, a + bp}, p > 0 µ 1 = µ 2 = 0 from (A6) and (A8), p = -(a + bc z )/(2b) from (A2). Further, c p + c zc u = 0 from (A4) and -c z > a/b from p > 0. All u, x combinations have the same objective function value. Since this is a boundary case, it can be regarded as a special case of region F with u = 0.

Due to Pontryagin's Maximum Principle (for details see [START_REF] Kamien | Dynamic optimization: the calculus of variations and optimal control in economics and management[END_REF] the solution of ( 8) is equivalent to the maximum of the corresponding current value static Hamilton-function subject to the non-negativity constraints for each point in time where the Hamilton-function is defined as

and λ(t) is the adjoint variable to the movement of the inventory state (7). Maximizing (B1) subject to the given non-negativity constraints can be done using a Lagrange approach with the following Lagrange-function

where µ 1 (t), µ 2 (t), µ 3 (t), µ 4 (t), and µ 5 (t) are the Lagrange multipliers associated with the non-negativity of inventory, price, remanufacturing, disposal, and manufacturing. We obtain the following necessary conditions, results, and equations:

(1) Maximization of the Hamilton-function (B1) reveals the optimal decisions:

• optimal remanufacturing decision

International Journal of Production Research 21

If y(t) = 0 for a time-interval with positive length, then ẏ(t) = 0 and r * (t) = u * (t) + z * (t) from ( 7) which implies 0 ≤ u * (t) ≤ r * (t) and 0 ≤ z * (t) ≤ r * (t). We have to distinguish the following 4 subcases. a) u * (t) < min{d(t), r * (t)}, z * (t) > 0 Due to (B10) and (B8) we get λ(t) = -c z . On the other hand it follows from (B3) that λ(t) ≥ c p -c u . This contradicts assumption (α1) and therefore this cannot be optimal. b) u * (t) = r * (t) < d(t), z * (t) = 0.

Then, λ(t) = c p -c u from (B3) and the optimal return rate from (B12) is

Then, λ(t) < c p -c u from (B3) and to ensure r * (t) = d(t) we need

As a consequence of (B4), λ(t) = -c z and the returns are given by

Due to assumption (α2) we have to exclude -c z > a(t) b(t) and to ensure r * (t) > d(t) we require a(t) > d(t).

Appendix C. Proof of Theorem 3.2

In case of y(t) > 0 we get µ 1 (t) = 0 from (B10) and the solution of the differential equation (B9) can be obtained as

where t 0 denotes the starting or ending point of an interval with y(t) > 0. Due to our assumption h > ρc z and using λ(t) ≥ -c z we find λ(t) = (ρλ(t 0 ) + h)e ρ(t-t 0 ) > 0. (C2)

Appendix D. Proof of Proposition 3.4

Under the assumption of y

we get

. This is equivalent with an upward jump of the adjoint variable, which is not allowed due to the optimality conditions. Therefore, collection of returns has to start before θ which means

≤ -c z for all t ∈ [θ, θ + 2 ] there is no need to build up inventories.

Appendix E. Proof of Proposition 3.5

Under the assumption of

and y(t) = 0 we have λ(t) = 2d(t)-a(t) b(t)

and λ(t) = b(t)(2 ḋ(t)-ȧ(t))-2d(t) ḃ(t)-a(t) ḃ(t) b 2 (t)

. Since µ 1 (t) ≥ 0 we know from (B9) that λ(t) ≤ ρλ(t) + h. Therefore, it is clear that y(t) = 0 cannot appear in an optimal solution if λ(t) > ρλ(t) + h (E1)

such that for all time points satisfying (13) a positive inventory is required.


Paolo Renna

Influence of maintenance policies on multi-stage manufacturing systems in dynamic conditions

Keywords: MANUFACTURING SYSTEMS, PREVENTIVE MAINTENANCE, MULTI- manufacturing systems performance, preventive maintenance, multi agent system, discrete event simulation manufacturing systems performance, preventive maintenance, multi agent system, discrete event simulation

Manufacturing systems are subject to a degradation process and if no actions are taken, the degradation leads to machine failures. Machine failures decrease the performance of the manufacturing system with loss of profits. The research concerns the evaluation of the manufacturing system performance in dynamic conditions when different maintenance policies are implemented in a multi-machines manufacturing systems controlled by Multi Agent Architecture. There are two extreme conditions of maintenance policy: no preventive maintenance, the actions are taken on failure state; on the other hand an intensive preventive maintenance can eliminate unforeseen failure, but with high costs. A dynamic policy maintenance is proposed to reduce the number of maintenance operations of the preventive policy. A discrete simulation environment has been developed in order to investigate the performance measures and the indexes of maintenance policies costs. The simulations have been conducted for several levels of fluctuation of mix, products' demand and working time uncertain. The simulation results show that the proposed approach leads to better performance for the manufacturing system and the number of maintenance operations (cost index of the maintenance policy), except in case of mean time between failure characterized by very low standard deviation.

Introduction

In realistic situations, the equipment of the manufacturing systems may not available because of periodical repair, preventive maintenance, or unforeseen breakdowns. Nowadays market conditions are characterized by continuous new products introduction, unforeseen demand fluctuation, reduction of life cycle of the products and margin profit. For the above reasons the maintenance policy needs to be integrated with the scheduling in order to react to the rapidly changes of the market and improve the manufacturing system performance. Two extreme maintenance policies can be considered: no preventive maintenance, therefore the actions are taken on failure state; on the other hand an intensive preventive maintenance can eliminate unforeseen failures, but with high costs. Therefore, a proper maintenance policy in a manufacturing system is necessary for it to improve performance and to reduce production costs. In literature, the following definitions were provided for the above extreme policies: periodic preventive maintenance policy (PM); a unit is preventively maintained at fixed time intervals kT (k=1,2...) independent of the failure history of the unit, and repaired at intervening failures where T is a constant. failure limit policy; maintenance is performed only when the failure rate or other reliability indices of a unit reach a predetermined level and intervening failures are corrected by repairs: corrective maintenance (CM) [START_REF] Wang | A survey of maintenance policies of deteriorating systems[END_REF]. In this paper, it has been investigated the impact of the preventive maintenance on manufacturing system performance, then a proper maintenance policy based on manufacturing system status has been proposed. The maintenance policy is related to production scheduling in order to obtain a balance trade-off between the maintenance and scheduling requirements. In this research, the dynamic actions of maintenance and scheduling are supported by the use of multi agent architecture. The performance measures are evaluated in several environmental conditions: from static to very dynamic. The paper is organized as follows. A literature review of maintenance policies based on manufacturing system status and related to scheduling is discussed in section 2. The manufacturing system context is presented in section 3. The Multi Agent Architecture proposed to support the scheduling and maintenance policy actions is presented in section 4. The simulation environment used to test the proposed approach is described in section 5. In section 6 the simulation results is discussed, while in section 7 the conclusions and future development is discussed.

Literature review

The survey paper [START_REF] Wang | A survey of maintenance policies of deteriorating systems[END_REF] summarized, classified and compared various existing maintenance policies for both single-unit and multi-unit systems. Several studies have been done to evaluate the maintenance policies in a single machine or two parallel machines context. [START_REF] Iravani | Integrated maintenance and production control of a deteriorating production system[END_REF] considered a make-to-stock production/inventory system consisting of a single deteriorating machine which produces a single item. They formulated the integrated decisions of maintenance and production using a Markov Decision Process. [START_REF] Dellagi | Preventive maintenance of manufacturing systems under environmental constraints[END_REF] described a new preventive maintenance approach for manufacturing systems under environment constraints. The manufacturing system under consideration consists of a single machine that produces a single product in a Just-in-Time context. [START_REF] Xu | Parallel machine scheduling with almost periodic maintenance and nonpreemptive jobs to minimize makespan[END_REF] investigated two parallel machines scheduling problem with almost periodic maintenance activities in order to minimize the makespan. They proposed a polynomial time approximation algorithm to solve the problem. They showed that it is unlikely to find a polynomial approximation algorithm that has lower worst-case bound than 2. [START_REF] Jin | Integrating flexible-interval preventive maintenance planning with production scheduling[END_REF] addressed production scheduling and PM planning for a single machine. The purpose of the scheduling problem is to choose an optimal sequence for the jobs as well as an optimal preventive maintenance planning which minimise the total weighted expected completion time of the jobs. Genetic algorithm is used to search for the optimal flexible-interval preventive maintenance planning and job scheduling. [START_REF] Mosheiov | Scheduling a maintenance activity to minimize total weighted completion-time[END_REF] studied a single machine scheduling problem. The objective function is minimum total weighted completion time. The problem is proved to be NP-hard, and an introduction of a pseudo-polynomial dynamic programming algorithm indicates that it is NP-hard in the ordinary sense. They also presented an efficient heuristic, which is shown numerically to perform well. The above approaches have been developed for a simple manufacturing system consisting of a single machine of two parallel machines. Another area of research concerns the performance analysis of multiple-machines manufacturing systems. [START_REF] Gupta | The relationship between preventive maintenance and manufacturing systems performance[END_REF] presented some state-dependent preventive maintenance policies that are consistent with the realities of production environment. They also developed polling models based analysis that could be used to obtain system performance metrics when such policies are implemented. The numerical test was conducted by M/G/1 queues. [START_REF] Savsar | Performance Analysis of an FMS Operating Under Different Failure Rates and Maintenance Policies[END_REF] discussed a procedure that combines simulation and analytical models to analyze the effects of corrective, preventive, and opportunistic maintenance policies on the performance of Flexible Manufacturing Systems (FMS). The FMS performance is measured by its operational availability index, which is determined using the production output rate of the FMS under a variety of time between failure distributions and different operational conditions. The effects of various maintenance policies on FMS performance are simulated and the results are compared to determine the best policy for a given system. The number of machines is limited, the approach proposed is difficult to adapt at different manufacturing systems and only the productivity performance is considered. [START_REF] Guo | Machine scheduling performance with maintenance and failure[END_REF] proposed an experimental model to evaluate the effect of corrective and preventive maintenance schemes on scheduling performance in the presence of machine failure where the scheduling objective is to minimize schedule duration. Further, they showed that parameter values can be chosen for which preventive maintenance does better than corrective maintenance. [START_REF] Yang | Maintenance scheduling in manufacturing systems based on predicted machine degradation[END_REF] propose a new method for scheduling of maintenance operations in a manufacturing system using the continuous assessment and prediction of the level of performance degradation of manufacturing equipment. A genetic algorithm based optimization procedure is used to search for the most cost-effective maintenance schedule, considering both production gains and maintenance expenses. This approach does not include the information concerning the external conditions as the demand fluctuations and product mix changes. Moreover, the approach does not react rapidly to changes of the manufacturing system, in fact the genetic algorithm have to be run every time the conditions change. Lee and [START_REF] Wu | Multi-machine scheduling with deteriorating job and scheduled maintenance[END_REF] investigated a multi-machine scheduling problem in which job processing times are increasing functions of their starting times and machines are not always available. The objective is to minimize the makespan. The problem was resolved by the developed of several heuristics. [START_REF] Levrat | Odds-based decision-making tool for opportunistic production-maintenance synchronization[END_REF] proposed a novel approach for integrating maintenance into production planning. The approach uses the 'odds algorithm' and is based upon the theory of optimal stopping. The objective is to select, among all the production stoppages already planned, those which will be optimal to develop maintenance tasks preserving the expected product conditions. [START_REF] Naderi | Scheduling sequence-dependent setup time job shops with preventive maintenance[END_REF] investigated job shop scheduling with sequence-dependent setup times and preventive maintenance policies in order to minimize makespan. Four metaheuristics based on simulated annealing and genetic algorithms as well as adaptations of two metaheuristics in the literature are employed to solve the problem. The performances of the proposed algorithms are evaluated by comparing their solutions through two benchmarks based on Taillard's instances. Lu and [START_REF] Sy | A real-time decision-making of maintenance using fuzzy agent[END_REF] presented a fuzzy logic approach for decision-making of maintenance. Some linguistic variables and rules-of-thumb are used to form the fuzzy logic models, based on the domain experts' experiences in production line and maintenance department. The historical production data are used to train and tune the fuzzy models. [START_REF] Caputo | Influence of Preventive Maintenance Policy on Manufacturing Systems Performances[END_REF] proposed some approximate queueing models to assess the impact of preventive maintenance interval on Work In Process (WIP). It is shown that WIP value is strongly influenced by the preventive maintenance interval and that maintenance intervals corresponding to a minimum maintenance cost or minimum WIP can be quite different. This kind of analysis can help in making more informed decisions involving WIP and cost trade-offs. In recent years, some studies concern the cost and manufacturing performance analysis of maintenance policies. [START_REF] Gharbi | Maintenance scheduling and production control of multiplemachine manufacturing systems[END_REF] discussed the production and preventive maintenance control problem for a multiple-machine manufacturing system. The objective of such a problem is to find the production and preventive maintenance rates for the machines so as to minimize the total cost of inventory/backlog, repair and preventive maintenance. The simulation experiments allow to determine an approximation of the optimal control policies and values of input factors. [START_REF] Kenne | Simultaneous control of production, preventive and corrective maintenance rates of a failure-prone manufacturing system[END_REF] proposed numerical methods for solving optimal control problem in order to minimize a discounted overall cost consisting of maintenance cost, inventory holding and backlog cost. Safei et al. ( 2010) proposed a multi-objective integer programming approach to investigate the impact of the use-based preventive maintenance policy on the performance of the cellular manufacturing system. The objective is to minimise the machine cost, inter-and intra-cell material handling and maintenance costs. The proposed model was solved by an interactive fuzzy programming; the approach proposed was a centralized approach. The analysis of a dynamic job shop problem is discussed in few papers in literature. Vinod and Sridharan (2006) consider dynamic job shops with sequence-dependent setup times, and a discrete event simulation model of the job shop scheduling was developed. Two types of scheduling rules (ordinary and setup-oriented rules) are applied in simulation model. Their experimental results demonstrate that the setup-oriented rules outperform the ordinary rules. [START_REF] Zhou | Study on job shop scheduling with sequence-dependent setup times using biological immune algorithm[END_REF] propose an immune algorithm to investigate dynamic job shop problem. As far as we reviewed, there is much less literature on integrating job shops scheduling with preventive maintenance. In particular, from the discussion of the literature the following issues can be drawn: -most of the papers proposed mathematical approaches; these approaches can be used practically with reduced number of machines, because the problem of scheduling and maintenance is an NPhard problem; -some papers proposed the queue network approach to analyze the performance of the manufacturing systems and evaluate the maintenance policies.

-few papers proposed methodologies derived by artificial intelligence as genetic algorithm and fuzzy logic to select the best maintenance policies. In summary, the studies concern few performance measures and the dynamicity degree of the external conditions of the manufacturing system was not investigated (demand and mix changes). The originality of the research developed in this paper can be summarized as follows: -a multi agent architecture is developed to support the scheduling and maintenance activities in job shop manufacturing systems with several machines. -a dynamic maintenance policy is proposed in this paper in order to improve the performance of the manufacturing system. -a discrete event simulation environment is developed to evaluate a wide range of performance measures (throughput, throughput time, work in process, tardiness and machines utilizations) for several degree of dynamicity of the market conditions (demand fluctuations and product mix changes).

Problem statement

The manufacturing system consists of a given number of cells; each cell is able to perform a particular set of manufacturing operations. Moreover, each manufacturing cell consists of a given number of machines. In such a system, the parts visit the manufacturing cells according to their The assumptions of the job-shop scheduling problem researched in this paper are the following:

• Each typology part has been given processing order, processing time and due date.

• Orders for production of different parts arrive randomly.

• Operations cannot be preempted.

• Each machine can process only one task at once.

• The queues are managed by the First In First Out policy in order to investigate only the proposed strategy. • Each machine can breakdown randomly.

In this research, the material handling time is included in the machining time, and the handling resources are always available.

Parameters

The parameters considered can be classified in two categories: external and internal parameters. The external parameters concern the market conditions; in particular, it has been considered the workload of the manufacturing system, the degree level of demand fluctuation (from static to very dynamic) and mix product changes (from static to very dynamic changes). In this research, the internal parameters regard the maintenance parameters. In case of PM policy, the parameters are time between Periodic repair actions (Tp) which is related to machine maintenance requirements and it is a fixed value. The Mean Time To Repair (MTTRp) is the time to maintenance the machine; the actions in preventive maintenance are known because no failures are happened. Then, the MTTRp is the minimum possible and it is considered fixed in case of preventive maintenance operations. In case of CM policy the parameters are the Mean Time Between Failure (MTBFc) which is related to machine reliability with mean µF c and standard deviation stdvF c (it is assumed a normal distribution). The preventive maintenance allows to improve the life of the components, therefore the MTBFp>MTBFc, then a parameter d is used to define the ratio:

[0,1] Fc d Fp µ µ = ∈ (1)
The Mean Time To Repair (MTTRc) is not known at priori because the generic machine is in failure state. Therefore, the MTTRc is major of the MTTRp considering two factors, as reported in expression 2:

MTTRc MTTRp K M = (2) ,1 time from last failure K MAX Tp   =     (3) [ ] 1, max M UNIFORM value = (4)
The expression (3) means that the time of a maintenance action is greater when the time from the last maintenance action increases considering Tp as base value. The expression (4) means that the time of the maintenance actions are not known at priori when the machine is in failure state; therefore the increment does not follow a particular distribution. It is assumed a uniform distribution between one and maxvalue for this incremental parameter. The value of Tp is evaluated considering the probability to avoid a failure event during the time between two subsequent maintenance actions. If Tp = µFp, the probability that the machines will be in failure status is the 50%; Table 1 reports the probability of failure state reducing the value of Tp (in simulation environment section the Tp is defined).

[INSERT TABLE 1 ABOUT HERE]

Dynamic policy

The maintenance policy proposed in this paper (DM) concerns the possibility to determine a dynamic time between two interruptions of the machine working state. In order to evaluate if the interruption can be activated, it has been evaluated the state of the manufacturing system: ◊ state of the manufacturing cell; the main index evaluated is the total number of parts in queue, the work in process of the manufacturing cell that specifies the congestion level. ◊ state of the manufacturing system; the indexes evaluated can be the following: the work in process, throughput time and the tardiness performance. The above indexes can be combined in a single function that evaluates the state of the manufacturing system. In this paper, it has been considered the state of the manufacturing cell, and in particular the number of parts in queue of the generic machine. The maintenance strategy performed by the generic machine is the following: IF Time from the last operation is >= Tp AND NQi=0 THEN performs maintenance operation (5)

The expression (5) allows to apply the PM policy if the resource is in idle state, otherwise the maintenance operation is postponed and it can occur a failure of the machine. NQ i is the number of parts in queue of the generic machine i.

The strategy proposed is between the two opposite strategies of PM and CM. Then, the MTBF and MTTR are evaluated by the following index:

Nopf DMf Noptot = (6)
where, Nopf is the number of maintenance operations in failure state of the machine and Noptot is the total number of maintenance operations. DMf assumes values between 0 and 1; if the value is 0 the DM policy is the same of the PM, if the value is one the DM policy is the same of the CM. This index evaluates the position of DM between the extremes PM and CM policies.

The value of MTBFd of the DM policy, is computed by the following expression:

(1 (1 ))

MTBFd MTBFc d DMf = + - (7)
The expression (7) defines a linear dependence between PM and CM policies; If DMf=0 then MTBFd=MTBFp, if the DMf=1 then MTBFd=MTBFc. The MTTRd of the DM policy is computed considering the state of the machine when the maintenance operation is performed. If the generic machine is idle (PM condition) the MTTR is the same of PM policy, otherwise, the machine is in failure state and the MTTR is the same of CM policy (see expression 8).

(1 ( 1))

MTTRd MTTRc d DMf = + - (8)

Multi Agent Architecture

The Multi Agent Architecture consists of three types of agent: a Manufacturing Cell Agent (MCA) is associated to each manufacturing cell; it manages the cell in order to perform the operation requested by the part agent and the scheduling of the maintenance actions. A Machine Agent (MA) is associated to each workstation; it is an intelligent entity whose principle aim is to schedule the resource tasks in order to improve the resource efficiency according to the manufacturing cell objectives. Moreover, when a new part enters the system the corresponding Part Agent (PA) is created; it analyzes the part status locating the following activities to be scheduled and performs the strategy to assign the part to the workstation. The actions performed by the agents for the scheduling problem are the following: (see figure 1):

• part agent analyzes the part status and it locates the next technological operation required by the part process plan; • it sends a message to the MCA informing them that a part requests to perform a manufacturing operation and the typology of the operation required; afterwards, it remains waiting for the MCA agents' answer;

• MCA verifies if it can perform the technological operation required by the PA; in affirmative case the MCA sends a request to the MAs of the manufacturing cell, otherwise the MCA goes in wait state for another request.

• MA evaluates the workstation status at the negotiation time t and it provides the workload of the machine. The work load is computed as the sum of the working time needed by the parts in queue;

• MCA receives such evaluations from all the resources, it assigns the part to the MA that provides the minimum value of workload .

• finally, the MCA communicates the assignment to the MA and PA.

[INSERT FIGURE 1 ABOUT HERE]

Figure 2 shows the interaction between Manufacturing Cell Agent and Machine Agent for the maintenance control policy. The actions performed are the following:

• at the initial state the MCA selects the maintenance policy and the MA monitors the machine's status. • the MCA, in case of PM policy, schedules the next interruption for the maintenance operation. In case of DM policy, the MCA evaluates the next interruption of PM policy and if the queue of the machine is empty, decides to perform the maintenance operations. In case of CM policy, the MCA waits for the signal by the MA that the machine is in failure state. • after the maintenance operation, the MCA updates the information on the manufacturing cell.

[INSERT FIGURE 2 ABOUT HERE]

As the reader can notice, the above procedures define the environmental relations of the autonomous agents involved in the work, but makes no assumption towards the agents' decisionmaking mechanisms. This means that the above protocols can be adapted to different objectives and decision mechanisms of the autonomous cell and job agents.

Simulation environment

The objective of the simulation experiments is to measure the performance of the maintenance policies CM, PM and the proposed dynamic policy DM in a very dynamic environment.

The author selected the Arena® discrete event simulation platform by Rockwell Software Inc. it was used to develop the simulation model of the presented approaches. Discrete event simulation -in many commercial tools and simulation packages, nowadays the simulation model is automatically created from high level modelling languages and notationsallows to validate and optimize dynamic and discrete systems such as production systems, but also workflows such as negotiation mechanisms. These models facilitate evaluating different coordination scenarios and maximizing their potential output and benefits. Arena® -based on the known SIMAN simulation language -is well suited for modelling shop floors of production systems in which each entity (part) follows a manufacturing route through production resources (servers, material handling systems, buffers, and so forth), [START_REF] Law | Simulation Modeling and Analysis[END_REF].

The manufacturing system consists of three manufacturing cells, each cell consists of two workstations that are able to perform the same technological operations. The manufacturing system is called to manufacture a set of two different parts (they differs for the working time required "workload"). Each part needs several disjointed visits to the manufacturing cells; the number of visits for each part is reported in table 2, where, also the production mix is provided.

[INSERT TABLE 2 ABOUT HERE]

In particular, the parts with low workload are characterized by a working time of 10 unit times in each manufacturing cell, while the high workload is 20 unit times for each manufacturing cell. The working time is the same in each manufacturing cell in order to evaluate only the maintenance policies. The due date is obtained by the following expression:

3 1 = = • ∑ ( ) i index i due date
workingtime duedate (9) The due date is obtained by the technological working time multiplied with an index; this index is fixed to 3 in this paper. Parts enter the system following an exponential arrival stream whose inter-arrival times are reported in table 3. The simulations are performed for three congestion levels of the manufacturing system (low, medium and high).

[INSERT TABLE 3 ABOUT HERE]

In order to emulate a dynamic environment the manufacturing characteristics (demand fluctuation and mix changes) changing during the production run consisting of several alternating stages; each stage is characterized by a length that defines the stability of the environment in which operates the manufacturing system. Moreover, the uncertain of the working time is introduced. Table 4 reports the data of demand and mix fluctuations for the three alternating stages considered.

[INSERT TABLE 4 ABOUT HERE]

The inter-arrival exponential parameter changes between 15 and 10, while mix parts changes over the three consecutive stages. Table 5 reports the data related to failures of the manufacturing resources.

[INSERT TABLE 5 ABOUT HERE]

It has been considered that the MTBFp has the same standard deviation of the normal distribution MTBFc (stdvF c ). The parameter maxvalue (see expression 4) is fixed to 1.2; this means that the mean time to repair for the corrective policy has an uncertain of 20% (in corrective policy the actions to perform are not known at priori). The simulations have been conducted for six levels of dynamicity (see table 6), from static (1) to very dynamic ( 6), the simulations are conducted for a 30240 total unit times. The row "Alternating stages" reports how many times the stages are alternated during the length of the simulation.

[

INSERT TABLE 6 ABOUT HERE]

Table 7 reports the five different environmental conditions in which the manufacturing system is tested.

[INSERT TABLE 7 ABOUT HERE]

Moreover, the working time uncertain has been considered as a deviation from the working time of the parts (working time u ). This uncertain can be caused by unforeseen events that affect the manufacturing operation (tool change, part load time, etc.). The working time uncertain follows a uniform distribution as showed in expression 10.

[ ] = , The working time uncertain is considered at two levels 20 % (in expression 10, a=0.9 and b=1.1) and 40 % (in expression 10, a=0.8 and b=1.2) for the case four in table 7. The experimental case five with all fluctuations is simulated with the level of 20% for the working time uncertain.

Therefore, it has been conducted 32 experimental classes. Five cases (table 6) for four cases of table 7, because the case 4 is tested for two level of uncertain.

Simulation results

For each experiment class, a number of replications able to assure a 5% confidence interval and 95% of confidence level for each performance measure have been conducted. The performance measures investigated are the following:

• average throughput time of the parts; it is the average time that the two typology parts spend in the manufacturing system (throughput av). • throughput of the manufacturing system (throughput); • Work In Process (WIP);

• average tardiness of the parts; it is the average time of delay of the parts respect the due date (tardiness av).

• average number of maintenance operations; it is an index of the costs of maintenance related to the calls of the maintenance team (Maintenan op av). Each call is characterized by a fixed cost.

• average time of maintenance operations (Maint time av); it is an index of maintenance costs, but in this case, it is the variable cost related to the time of operation of the maintenance team.

• percentage of maintenance operations in failure state of the machines (Failure av); it is an index of the number of intervention in failures state of the machine that can affect the quality of the parts.

• average of the single maintenance operation (time to repair). it is obtained by the ratio between the average time of the maintenance operations and the number of maintenance operations (Av TTR). The simulation results reported are the percentage difference compared to the PM policy. Figure 3 reports the simulation results over the three levels of congestion of the manufacturing system for the CM maintenance policy and in static environmental conditions. The CM policy leads to reduce the number of maintenance operations on the manufacturing system, and therefore the costs related to this maintenance activity. All the other performance measures are worst compared to the PM policy, in particular the average tardiness of the parts is the performance that is drastically reduced. Moreover, the benefits of the PM policy are more important when the congestion of manufacturing system is low (inter-arrival 15).

[INSERT FIGURE 3 ABOUT HERE]

Figure 4 reports the simulations results over the three levels of congestion of the manufacturing system for the DM maintenance policy and in static environmental conditions. In this case, the proposed approach leads to better performance measures for the manufacturing system. The main benefit regards the average tardiness of the parts. When the congestion level is high (inter-arrival 9) the benefits of DM are very low. Moreover, the DM policy leads to increase the number of maintenance operations in failure state of the machines, but this number is always under the 10% of the total maintenance operations, therefore a limited number. Table 8 reports the simulations results introducing the dynamic condition one at time. The values reported are the average percentage difference compared to the PM policy over all the stage lenghts considered (see table 6). The worst performance are highlighted in the table. From the anaysis of the table the following issues can be drawn:

• the mix changes is the environmental element that leads to worst performance for the CM policy, with the excpetion of the average tardiness of the parts that is more influenced by the working time uncertain. • in case of DM policy, the high level of working time uncertain and the mix changes leads to worst performance.

• the throughput is the performance with less influence by the environmental conditions.

[INSERT TABLE 8 ABOUT HERE]

Figures 5 and6 report the trend of the performance measures over the stage lenght for the CM and DM policy when all the enviropmental conditions change when the stage lenght varying (see table 6).

[INSERT FIGURE 5 ABOUT HERE]

In particular, the performance of the CM policy are very stable, except for the average tardiness of the parts. The tardiness is very high when the environment conditions are stable (stage lenght 10080), it decreases when the the conditions are more dynamic.

[INSERT FIGURE 6 ABOUT HERE]

The DM policy leads to significantly benefit for the tardiness performance in case of stable (stage lenght 10080) or more dynamic (stage lenght 630). The other performance measures have low fluctuation over the stage lenghts. Table 9 reports the simulation results when all the environmental conditions changes for different values of the stdvFc (standard deviation of the mean time to failure). The values are the percentage difference compared to the PM policy and the average over the stage lengths.

[INSERT TABLE 9 ABOUT HERE]

The simulation results show that the performance are the same for low and medium standard deviation; when the standard deviation is high the CM policy has a modest improvement of the performance, while the benefits of the DM are significantly reduced.

Conclusions

The research deals with the performance investigation of different maintenance policies in dynamic conditions. The effects considered are the mix, inter-arrival and working time changes under a scheduling approach base on Multi Agent System architecture. In this research, three maintenance policies are investigated: preventive, corrective and a dynamic policy proposed in this paper. The simulation results show that the mix changes and working time uncertain have significant effect on the performance of the manufacturing system, in particular these changes lead to obtain major benefits when the preventive maintenance policy is used. Moreover, the proposed approach shows significantly improvements in terms of tardiness performance and maintenance operations without increase the average time of the maintenance operations. The tardiness is a performance of the manufacturing system, while the number of the manufacturing operations and average time of its are the indexes of the costs of the maintenance policy. Therefore, the proposed approach improves a particular performance measures (the other performance measures have low difference with preventive maintenance) with a reduction of the maintenance costs. The investigation on the dynamicity of the manufacturing system shows the robustness of the proposed approach. Finally, the effect of the standard deviation of Mean Time to Failure is investigated; the results show that the proposed approach reduced drastically the benefits when the standard deviation is very low. In this case, the preventive policy is the better strategy. Further research paths concern the effect on the parts of the maintenance operations when the resources are in failure state. To be more precisely, if the failure of the machine can cause the waste product or the re-working of the part. This aspect can be connected to the real costs of the maintenance policy, in order to investigate the best policy not only in term of costs of the policy, but also with the effects on the parts. Moreover, the implementation of the proposed approach in a real case study allows to evaluate the real benefits.

  scheduling decision consists in what machine, within the manufacturing cell, the part will perform the next operation.

[

Figure 2 .Figure 6 .

 26 Figure 2. Multi Agent activity diagram interaction -maintenance policies

•

  Table 1. Tp values • Table 2. Mix part • Table 3. Inter-arrival stream • Table 4. demand changes over the stages • Table 5. Failures' data • Table 6. Stage length • Table 7. Stage length • Table 8. Simulation results -dynamic conditions • Table 9. Simulation results -stdvFc changes

		Manufacturing Cell Agent	Machine Agent
			select maintenace policy	monitoring machine status
	1	CM	2 PM	Tp DC CM	1 CM	Probability to failures Part 1 Part 2 low medium High Stage i Stage i+1 Stage i+2 2 3 4 5 6 4 5 CM 3 DM CM DM CM DC CM DM CM Failure state ? DM yes	DM
	state 15.86553% 10 Table 3. Inter-arrival stream µ c -stdvF c 50% Number of visits Mix 3 Inter-arrival 9 Inter-arrival time 15 Mix Part 1 50% 80% 10 MTTRp 45 µFc 450 Static 10080 5040 2520 1260 630 50% 3 15 -20% Alternating stages 1 3 6 12 24 48 No changes Mix fluctuations Inter-arrival Working time All fluctuations and Communicate working time uncertain MIX Inter-arrival Working Working stdvFc=33.75 stdvFc=67.5 stdvFc=135 DM no Wait
	F o r µ c -2*stdvF c workload Low (30 unit times) High (60 unit times) Mix Part 2 50% 20% 80% stdvFc 67.5 Table 6.Stage length fluctuations uncertain time 20% Throughputav 17.35 -2.61 17.75 -2.44 16.21 Every Tp an failure status -1.34 time 40% interruption 2.27501% µ c -3*stdvF c 0.13499% µ c -4*stdvF c 0.00317 Table 1.Tp values F o r Table 2. Mix part F o r F o r Table 4.demand changes over the stages F o r µFp 510 Tp µFp -2 sigma=450 M 1.2 Table 5.Failures' data F o r F o r Table 7.Experimental cases F o r Throughputav 25.39 -2.02 20.54 -2.99 20.98 -2.41 21.41 -1.7 Throughput -0.50 -0.71 -0.03 0 -0.21 0.06 -0.05 -0.08 F o r Throughput -0.13 0.06 0.06 0.13 -0.25 -0.13 WIP 17.43 -2.55 18.00 -2.37 16.31 -1.38 F o r F o r is scheduled Wait evaluate machines' F function state o r yes Interrupttion?
	P e e r P e e r P e e r P e e r P e e r P e e r P e e r P e e r 25.75 -1.66 20.62 87.92 -1.41 88.45 Mainten op av -10.44 -5.86 -12.20 WIP Tardinessav Maint time av 18.18 -0.90 15.81 P e e r Tardinessav 58.50 -7.85 Mainten op av -11.76 -5.02 -11.47 -3.02 -10.73 133.92 -18.70 125.73 20.74 -2.41 17.37 -4.69 -11.75 -4.10 -11.79 -3.89 16.48 -0.74 16.42 59.34 -7.55 53.42 -4.48 -4.67 -4.73 -3.71 Maint time av 16.45 -1.20 16.83 -0.56 14.48 Failuresav 100 3.79 100 4.78 100 8.67 e r e r r e 2.74 P e P e no Communicate interruptions P scheduled Maintenance actions Manufacturing e cell state change	-1.74 5.78 0.31 -0.51
	R e v i e R e v i e R e v i e R e v i e R e v i e R e v i e R e v i e R e v i e 100 5.48 100 31.91 3.60 31.98 Table 8.Simulation results -dynamic conditions 100 7.01 5.95 31.96 5.28 3.56 R e v i e Av TTR Failuresav Av TTR 31.96 4.03 31.97 4.31 31.98 e v i e e v i e e i v Figure 1. Multi Agent activity diagram interaction -scheduling e Table 9.Simulation results -stdvFc changes R R R update manufacturing cell information	100 31.98 5.56	7.05 4.53
							w w w w w w w w w w w w
							O n l O n l O n l O n l O n l O n l O n l O n l O n l O n l O n l O n l
							y y y y y y y y y y y y

http://mc.manuscriptcentral.com/tprs Email: ijpr@lboro.ac.ukInternational Journal of Production Research

List of Figures