Khanh T P Nguyen

Thomas Yeung

Bruno Castanier

Toulouse Archive Ouverte

Keywords: maintenance management, technology management, Markov decision rules, Markov modelling, replacement policies, information acquisition

In this paper, we propose the first model that considers the option to acquire information on the profitability of a new technology that is not yet available on the market for asset maintenance and replacement decisions. We consider the uncertainty of future asset characteristics by incorporating information acquisition decisions into a non-stationary Markov decision process framework. Using this framework, we optimise asset maintenance and replacement decisions as well as the optimal timing of new technology adoption. Through mathematical analyses, the monotone properties and convexity of the value function and optimal policy are deduced. Deeper numerical analyses highlight the importance of considering the acquisition of information on future technology when formulating a maintenance and replacement policy for the asset. We also deduce a non-intuitive result: an increase in the arrival probability of new technology does not necessarily make the acquisition of additional information more attractive.

Introduction

Technology evolution has an important impact on the asset replacement and investment strategy of a firm. It is well known that under technological uncertainty, replacement with current technology or waiting for new technology will be carefully weighted. For recent studies, [START_REF] Büyüktahtakın | A Mixed-integer Programming Approach to the Parallel Replacement Problem under Technological Change[END_REF] proposed a mixed-integer programming approach to solve the parallel replacement problem under technological change that is represented by increasing capacity gains of newer technology assets. [START_REF] Hagspiel | Optimal Technology Adoption when the Arrival Rate of New Technologies Changes[END_REF] focused upon uncertain timing of future technology when the instant rate of new arrivals changes after a fixed or an uncertainty period time. [START_REF] Yatsenko | Algorithms for Asset Replacement under Limited Technological Forecast[END_REF] considered different optimal replacement methods of assets under changing purchase and operation -maintenance cost caused by technology development; and deduced two modified methods using limited technological forecast data but deliver the solutions close to the infinite horizon replacement. However, these above articles only use the cost flow to characterise the asset deterioration state and do not consider the maintenance option for extension of the asset life in order to wait a better technology. Furthermore, the level of knowledge about future technology that is essential for strategic decisions is not examined.

Therefore, it is necessary to develop models that allow decision-makers to combine operational maintenance and strategic investment goals and also take into account information acquisition option for the technology adoption.

In literature, maintenance and replacement decisions with technological change are generally approached in two ways. The first approach focus on the operational maintenance decisions and assumes that new technology is already available on the market. The overall performance of this new technology is known and the question is whether it is worth moving to this new technology given the price of such a change. In this context, the problem is to determine the conditions on the set of characteristics (purchase price, reliability improvements, etc.) which lead to move from one technology to another [START_REF] Borgonovo | A Monte Carlo Methodological Approach to Plant Availability Modeling with Maintenance, Aging and Obsolescence[END_REF]Clavareau andLabeau 2009a, 2009b). The second approach that focuses on the strategic investment goals is mainly found in the management science literature. In this context, the maintenance issue takes even greater importance and may be considered as an economic investment policy as it allows delaying replacement with existing technology in order to await new technology [START_REF] Bethuyne | The Timing of Technology Adoption By a Cost-minimizing Firm[END_REF][START_REF] Büyüktahtakın | A Mixed-integer Programming Approach to the Parallel Replacement Problem under Technological Change[END_REF][START_REF] Hagspiel | Optimal Technology Adoption when the Arrival Rate of New Technologies Changes[END_REF][START_REF] Hopp | Maintenance and Replacement Policies under Technological Obsolescence[END_REF][START_REF] Mauer | Investment under Uncertainty: The Case of Replacement Investment Decisions[END_REF][START_REF] Yatsenko | Algorithms for Asset Replacement under Limited Technological Forecast[END_REF]. These models take into account the uncertainty in unknown future technology. They allow managers to determine the best time for replacement investment of equipment under technological evolution but do not consider the maintenance strategies as well as the impact of technology change on them. Combining operational maintenance and strategic investment goals, Nguyen, Yeung, andCastanier (2011, 2014) show that technology evolution has a significant impact on the optimal maintenance and replacement policy even when the next generation technology has not yet appeared. None of these above articles consider the option to acquire information on the profitability of the new technology. In reality, without gathering the additional information, the risk in planning maintenance and replacement actions under uncertainty of the profitability level of new technology is greatly increased. These risks include the early replacement by a near-obsolete technology or the decision to defer maintenance or replacement while waiting in vain for a new and more efficient technology.

In literature, there are several papers addressed information acquisition option for asset replacement problem under technological change. In an early study, [START_REF] Monahan | Optimal Stopping in a Partially Observable Binary-valued Markov Chain with Costly Perfect Information[END_REF] aimed the question of when to adopt innovation: whether to do nothing while waiting for the market transition or to purchase information and decide based on this newly acquired information. However, the model is limited by assumptions that the information is perfect and that transition matrix is known in advance. Later, McCardle (1985) proposed a dynamic programming model in which a decision-maker decides whether to gather information or to outright adopt/reject a new technology. The study is restrictive in the form of the profitability distribution and signal process. It only allows considering two types of information: positive and negative. [START_REF] Ulu | Uncertainty, Information Acquisition and Technology Adoption[END_REF] considered the probability distribution on new technology's expected benefit as a state variable and showed the limitation of the univariate setting of [START_REF] Mccardle | Information Acquisition and the Adoption of New Technology[END_REF]. That is, a rise in technology's expected benefit or the first-order stochastic dominance improvement in the distribution on benefits does not necessarily imply an increase in the value function nor does it show the necessity of new technology adoption.

These previous models only consider the technology adoption problem based on an available innovative technology and do not consider technologies that may arrive in the future. On the other hand, the information gathering on the new technology's profitability for maintenance policies is an unexplored area. This concern is an important issue in decisionmaking, especially when maintenance requires very high investments and the need to estimate the return of investment of each decision. Furthermore, maintenance can delay in the even greater replacement investment decision. One of the main challenges in this context is to balance the classical operational maintenance optimisation problem with the more challenging real-world strategic investment problem. For this, we propose to develop a new maintenance model with the uncertainty on the profitability of future technology. We define a new option we term 'acquire information' in case the current uncertainty is too high for deciding to maintain or replace with the current technology. In this paper, we propose to extend the preliminary model in [START_REF] Nguyen | Information Acquisition of New Technology Performance for Maintenance/investment Decisions[END_REF] by demonstrating appealing structural properties of the value function and optimal policy in this paper. Deeper numerical analysis also is performed in order to study how maintenance and information gathering policies interact. To our knowledge this paper is the first work to link these above aspects: information acquisition and new technology adoption with operational decisions of asset maintenance and replacement. Compared with previous studies, our model is more complex due to taking into account the system's stochastic deterioration state, the diversity of decisions and the question about gathering information in future decision periods until a new technology's appearance.

The remainder of this paper is structured as follows: Section 2 is devoted to the mathematical formulation. In Section 3, we present some interesting structure properties of the model. The performance of our model is highlighted through numerical examples in Section 4. In Section 5, we discuss the model assumptions in the context of practical applications, especially in the automation and machinery industries. We also extend the results presented in Section 3 by considering a general model for maintenance effect. Finally, conclusions and future work are presented in Section 6.

Construction of the mathematical model

In this paper, we focus on considering the value of the information acquisition option on the uncertain profitability of a new technology (that is not yet available on the market) for maintenance and replacement decisions in a finite horizon N . Past this time interval, if the new technology is not yet on the market, then we do not interest in its appearance. On the other hand, after the new technology becomes available, the exact profitability of the new technology is known to the firm, in other words, the information acquisition option is no longer interested.

Problem statement

On the operational aspect, consider a repairable asset that operates continuously from the new state, X = x 0 , until a failure state, X = z (lower states are better than higher states). The state of asset is changing according to Markov chain. Based on the observed deterioration level x, three different operational actions are considered at the beginning of the decision epoch:

• Do nothing (D N ): the asset continues to deteriorate until the next decision epoch and generates a profit G(x). Note that G(x) is the expected accrued profit within a decision period and depends on the deterioration state x at the beginning of that period. G(x) is decreasing in x. In case of failure, the do nothing action is still allowed but the profit in the next decision epoch is assumed to be negative, i.e. G(z) ≤ 0.

• Maintenance (M): In the paper framework, we redefined the term 'Maintenance' for repair actions as a tacit agreement. The current asset is restored to a given deterioration level, x M . An increasing repair cost in deterioration level, c M (x), is incurred. As we assume that the repair time is negligible, in the next decision interval the asset deteriorates from the level x M and generates a profit G(x M ). • Replacement (R): the asset is immediately replaced by a new one that belongs to the same technology generation (with the cost c 0 and the salvage value b(x)) or that belongs to the new technology (if it is available on the market).

The replacement time is also assumed to be negligible.

Under technological evolution, we assume that only one new technology will appear over an interesting horizon. The new technology has not been available before period n, the probability that it will become available in period n, called p n is increasing in n. Before the new technology's appearance, its expected total profit has an uncertain value. We define θ be the expected value of the total profit (after subtracting the purchasing cost) when replacing the current asset by the new technology. Let be the set of possible values θ i of θ with i be the index on the level of profit. The uncertainty in the θ values or rather the belief in θ i is modelled by a probability distribution π p over . This probability distribution, π p is updated when the additional information on the performance of the new technology is gathered. Indeed, at the beginning of each decision epoch, before choosing the appropriate action (D N , M, R), the manager has the ability to gather additional information on the performance of the new technology performance. The information purchasing time is also assumed to be negligible and this action is described as followed:

• Information acquisition (A): Manager pays information cost c s and receives information s j , s j ∈ S, drawn with the likelihood function L(s|θ ). We define f (s j ; π p ), the predictive distribution for information s j given that the prior distribution over is π p , as a probability mass function on the set of information s j . Note that, in the case of continuous spaces, it is a probability density function.

f (s j ; π p ) = ∀i L(s j |θ i )π p (θ i )
Following the study of [START_REF] Ulu | Uncertainty, Information Acquisition and Technology Adoption[END_REF], we consider the probability distribution on new technology's expected benefit (π p ) as one of state variables of the model. The posterior distribution function π ′ p over is defined through π ′ p (θ i ; π p , s j ) that is the probability of θ i given that the prior distribution is π p and the acquired information is s j .

π ′ p (θ i ; π p , s j ) = L(s j |θ i )π p (θ i ) f (s j ; π p )
With updated information on new technology, the manager can choose an actions (D N , M, R) for the current asset in use.

Because one of our model state variables is the distribution itself, we will frequently suppress the domain of the distribution and write the posterior distribution on as π ′ p (π p , s j ), a function of the prior distribution (π p ) and acquired information (s j ). Similarly, we use the notation f (π p ) instead of f (s j ; π p ).

Model formulation

We model the optimisation problem as a Markov decision process on an infinite time horizon by dividing this problem in three stages:

• In the planning horizon n < N , before the new technology is available, the firm can choose to gather additional information about the new technology's profitability or not; and then based on the current information, an appropriate operational action can be chosen for the current asset. V N n (x, π p ) denotes the maximum expected discounted value from the decision epoch n to the final planning epoch N (see Equations ( 1)-( 6)).

• In the planning horizon N , after the new technology becomes available, its exact profitability is known to the firm.

The firm can choose to adopt it immediately or not, in which case the process ends or continues with the current asset. The corresponding maximum expected discounted value function in this case is represented by V (x, θ i ) (see Equation ( 7)). • At the end of the planning horizon N , if the new technology has not yet appeared, its appearance will be no longer considered: V N n (x, π p ) = Ṽ (x); where Ṽ (x) be the maximum expected discounted value over the infinite horizon of the maintenance optimisation problem when the new technology appearance is not considered (see Equation ( 8)).

In detail, with discount factor β, 0 < β < 1, the MDP formulation of the information acquisition problem for the repair and replacement decisions is given by:

V N n (x, π p ) = max A N n (x, π p ), O N n (x, π p ) (1)
Where:

• the value of choosing directly an action given the current information

O N n (.) = max D N N n (.), M N n (.)1 {x<x M } , R N n (.) (2)
where 1 {x<x M } denotes that the repair action is only considered when x < x M .

• the value of acquiring additional information before the action

A N n (x, π p ) = -c s + j f (π p )O N n (x, π ′ p (π p , s j )) (3)
In Equation ( 3), O N n (.) is a sub-optimisation problem, i.e. if we choose to acquire information, we then must choose the maximum of our operational actions afterwards (D N , M, R).

D N N n (x, π p ) =    G(x) +β (1 -p n ) x ′ P(x ′ |x)V N n+1 (x ′ , π p ) + p n i π p (θ i ) x ′ P(x ′ |x) V (x ′ , θ i ) (4) M N n (x, π p ) = -c M (x) + D N N n (x M , π p ) (5) R N n (x, π p ) = -c 0 + b(x) + D N N n (x 0 , π p ) (6)
where P(x ′ |x) is probability that the deterioration state of the current asset is x ′ at next period given that the deterioration state at the current period is x.

And after the new technology's appearance, the decision-maker can weigh the benefit of continuing to use the current asset or replace by a new available technology through the use of the optimisation model in Equation (7).

V (x, θ i ) = max      D N (.) = G(x) + β x ′ P(x ′ |x) V (x ′ , θ i ) M(.) = -c M (x) + D N (x M , θ i ); ∀x > x M R(.) = max R(x), θ i + b(x)      (7)
The Ṽ (x), maximum expected discounted value over the infinite horizon when we do not consider the possibility of the new technology appearance is given by:

Ṽ (x) = max    D N (x) = G(x) + β x ′ P(x ′ |x) Ṽ (x ′ ) M(x) = -c M (x) + D N (x M ); ∀x > x M R(x) = -c 0 + b(x) + D N (x 0 )    (8)

Structure of the optimal policies

In this section, we study structural properties of the value functions and optimal policies based on state variables of the system. Firstly, we recall basic theorem of [START_REF] Ulu | Uncertainty, Information Acquisition and Technology Adoption[END_REF] in order to provide the foundation for studying the structural properties of the repair and replacement policies in Section 3.1. In Section 3.2, we deduce interesting results in order to examine when the information acquisition option is optimal. Following [START_REF] Ulu | Uncertainty, Information Acquisition and Technology Adoption[END_REF], the likelihood ratio order and monotone properties of the likelihood ratio are defined as follows:

Definition 1

(1) The probability distribution

π p 2 is greater than π p 1 in LR order (π p 2 L R π p 1 ) if and only if for every θ 2 ≥ θ 1 : π p 2 (θ 2 ) π p 1 (θ 2 ) ≥ π p 2 (θ 1 ) π p 1 (θ 1 )
(2) The signal process has the monotone property according to the likelihood ratio order if the signal space S is totally ordered and L(s|θ 2 ) L R L(s|θ 1 ) for all θ 2 ≥ θ 1 , i.e. for all s 2 ≥ s 1 and θ 2 ≥ θ 1 :

L(s 2 |θ 2 ) L(s 2 |θ 1 ) ≥ L(s 1 |θ 2 ) L(s 1 |θ 1 ) Corollary 1 π p 2 L R π p 1 ⇒ θ π p 2 (θ )φ(θ ) ≥ θ π p 1 (θ )φ(θ )
for non-decreasing functions φ(θ ). Theorem 1

(1) Given any signal s j , the posteriors are LR-ordered if and only if the priors are LR-ordered:

π p 2 L R π p 1 ⇔ π ′ p (π p 2 , s j ) L R π ′ p (π p 1 , s j ).
(2) In the case where the signal process satisfies the MLR-property,

• If a prior probability distribution is better than another, then it will be more informative:

π p 2 L R π p 1 ⇒ f (π p 2 ) L R f (π p 1 )
; ∀s j ∈ S • For any prior π , if a signal is more favourable than another, its posterior probability distribution will be better and vice versa:

∀π p , s 2 is more favourable than s 1 ⇔ π ′ p (π p , s 2 ) L R π ′ p (π p , s 1 )
Theorem 1 presented in [START_REF] Ulu | Uncertainty, Information Acquisition and Technology Adoption[END_REF] described the monotone relation between the acquired information and the asset performance characterised by its prior and posterior probability distribution. In the case where the signal process satisfies the MLR-property, if the performance of the new technology is anticipated to be high at the first prediction, then the possibility that beneficial information is acquired to consolidate the prior prediction is also high.

Structure properties of the value functions and optimal policies

In this section, we firstly examine the stationary problems where • technological change is not considered • new technology is already available on the market with profit level, θ i .

Secondly, we consider the non-homogeneous MDP for the non-obsolescence case where new technology has not yet appeared on the market.

Stationary problems of repair/replacement policy

By the Theorem 6.2.10 of [START_REF] Puterman | Markov Decision Processes -Discrete Stochastic Dynamic Programming[END_REF], there exists a deterministic stationary optimal policy when we do not consider technological change or new technology is already available on the market. As the reward functions are bounded, and discount factor 0 < β < 1, their optimal value functions Ṽ (x) and V (x, θ i ) converge for all x and θ i :

lim n→∞ Ṽn (x) = Ṽ∞ (x) or lim n→∞ Vn (x, θ i ) = V∞ (x, θ i ).
We assign numerical values to three possible actions as follows:

1 → Do Nothing, 2 → Maintenance (repair), 3 → Replacement.

Next, the following basic assumption is used to consider the structural properties of the stationary problems.

Assumption 1 [START_REF] Puterman | Markov Decision Processes -Discrete Stochastic Dynamic Programming[END_REF], we derive Lemma 1:

∞ x ′ =ς P(x ′ |x) is non-decreasing in x, for all ς, x ∈ X . Firstly, from Lemma 4.7.2 of
Lemma 1 Let { p j }, { p ′ j } be sequences of non-negative real values satisfying ∞ j=ς p j ≥ ∞ j=ς p ′ j (9)
for all ς , with equality in (9) holding for ς = 1. Suppose v j+1 ≤ v j for j = 1, 2, . . ., then

∞ j=1 v j p j ≤ ∞ j=1 v j p ′ j ( 10
)
where limits in (10) exist, but may be infinite.

Theorem 2 considers the monotone properties of value functions and optimal policies in the case without new technology ( Ṽ (x)) and the case after a technological change ( V (x, θ i )).

Theorem 2

(1) V (x, θ i ) is non-decreasing in θ i ;
(2) Ṽ (x) and V (x, θ i ) are non-increasing in x, ∀x ∈ X ;

(3) ∀a ∈ A : {1, 2, 3}, the optimal policy Ṽ (x) (or V (x,θ i ) ) are non-decreasing in x, ∀x ∈ X with the two following conditions:

• Ṽ (x M ) = 1 (or V (x M , θ i ) = 1) • G(x 1 ) -G(x 2 ) ≥ c M (x 2 ) -c M (x 1 ) ≥ b(x 1 ) -b(x 2 ) ≥ 0; ∀x 2 > x 1 .
On the operational aspect, the Theorem 2 derives the control limit structure for the optimal policy in the case without new technology ( Ṽ (x)) by solving the Equation ( 7) and the case after a technological change ( V (x, θ i )) by solving the Equation (8). Let (y M , y R ) be the respective repair and replacement thresholds, decision rules are (1) replace as soon as the deterioration state becomes upper than y R ; (2) maintain if the deterioration state belongs to (y M , y R ); (3) do nothing if the revenue rate remains lower than y M .

Information acquisition on the profitability of future technology for repair and replacement decisions

In this section, we consider the case when new technology is not yet available; i.e. the non obsolescence case.

From [START_REF] Hopp | A New Optimality Criterion for Non-homogeneous Markov Decision Processes[END_REF] there exists a periodic forecast horizon optimal for the non-homogeneous MDP. That means when the forecast horizon N is large enough, for the first period, the optimal decision remains the same regardless of the horizon size M, M > N . In addition, as the action reward functions are bounded with discount factor 0 < β < 1, we have: lim

N →∞ V N 1 (x, π p ) = V ∞ 1 (x, π p ).
Theorem 3

(1) V N n (x, π p ) is non-increasing in x, ∀x ∈ X .

(2) If the signal process satisfies the MLR property then V N n (x, π ) is non-decreasing in π p according to LR-order (π

p 2 L R π p 1 ⇒ V N n (x, π p 2 ) ≥ V N n (x, π p 1 )
). Theorem 3 presents the monotone properties of the value function V N n (x, θ i ): • the higher deterioration state is, the lower expected discounted value is,

• the more possibility that the new technology will appear with high performance is, the greater expected discounted value is.

In the following, we will demonstrate the convexity of the value function V N n (x, π p ). Theorem 4 For all n, the value function V

N n (x, π p ) is convex in π p : V N n (x, π p α ) ≤ αV N n (x, π p 1 ) + (1 -α)V N n (x, π p 2 ) where π p α = απ p 1 + (1 -α)π p 2 , 0 ≤ α ≤ 1
Theorem 4 describes that if the prior probability distribution (first prediction) of the performance level of the new technology changes, the expected discounted value over a planning horizon does not change linearly, but depends on the second derivative (or, loosely speaking, higher order terms) of the modelling function. This is also a key result to consider the information purchasing (A)-region in the next subsection.

Considering the necessity of information purchasing

In this subsection, we study the A-region, in which system states it is optimal to acquire more information. Firstly, we introduce two following thresholds:

• Low-performance threshold, θ l is defined by the replacement profit in current technology, -c 0 + D N (x 0 ). For new technology profit values below this threshold, it is preferred to continue with existing technology. • High-performance threshold, θ h such as for all θ i > θ h , we will invest immediately in new technology having this expected profit regardless of the deterioration state of the current asset, (θ h > θ l ).

Next, we present Lemma 2 in order to determine the value of θ h .

Lemma 2 Recall that G(x), b(x) are, respectively, the profit accumulated within a decision period and the residual value of the asset at deterioration state x, let θ h be

θ h = G(x 0 ) -b(x 0 ) + β ′ x P(x ′ |x 0 )b(x ′ ) 1 -β ( 11
)
where β is the discount factor and x 0 is the initial deterioration state of the asset; then we will invest immediately in new technology having a profit level θ i > θ h , regardless of the deterioration state of the current asset.

Lemma 2 is an interesting result that helps the decision-maker to determine the profit threshold, θ h , of the new technology when the economic efficiency of the current asset does not need to be considered, new technology should be invested in immediately.

Theorem 5 The selection of the optimal operation for the current asset does not depend on the probability distribution of the new technology's profit (π p ) and we do not gather additional information about the new technology's performance if all expected values of the new technology's profit (θ i ) are:

(1) greater than the high-profit threshold θ h determined by Lemma 2 (2) lower than the low-profit threshold θ l = -c 0 + D N (x 0 ).

Theorem 5 shows an interesting result that, if the technology change is estimated as a breakthrough point with all of expected profit levels greater than θ h (the high-profit threshold), that is determined by the Lemma 2, the information acquisition is not necessary. This option has significant value if and only if we do not know about the nature of technological change, i.e. the estimated profit space includes the low and high values.

Next, we consider the properties of the information acquisition option for the case where the nature of technology change (incremental or breakthrough change) is unknown. From the convexity of the value function, V N n (x, π p ), we obtain directly the following corollary:

Corollary 2 In the case of perfect information, if it is optimal to acquire information at both π

Numerical examples

In this section, we present numerical examples to illustrate the performance of our model. These numerical examples were solved using the classic backward induction algorithm.

Input parameters

We consider an asset that has five degradation levels with the failure state z = 5. After a repair action, the asset is restored to x M = 2. The accumulated profit in a period and deterioration transition matrix for the current technology are, respectively:

G = 200 160 100 20 -70 ; P =      
0.8 0.2 0 0 0 0 0.8 0.2 0 0 0 0 0.8 0.2 0 0 0 0 0.5 0.5 0 0 0 0 1

     
The repair cost is an increasing function while the salvage value is an decreasing function in deterioration state. With constants v, h 1 , h 2 , the respective functions are:

c M (x) = ν + h 1 (x -x M ); b(x) = h 2 (m -x)
We define the appearance probability of new technology at decision epoch n + 1 as an increasing function in time: The ǫ factor reflects the non-appearance probability of new technology at the next decision epoch. Factor δ characterises the increasing rate of the appearance probability of new technology over time.

p n+1 = 1 -ǫδ n ; δ, ǫ ≤ 1
Although the model presented in this paper allow us to consider numerous profit levels of the new technology and also multiple kinds of information, in order to simplify the numerical examples, we assume that the new technology will appear with only two benefit levels θ 1 < θ 2 and that two types of information s j (s 2 is more favourable than s 1 ) will obtained with the following likelihood function (L).

L = L(s 1 |θ 1 ) L(s 2 |θ 1 ) L(s 1 |θ 2 ) L(s 2 |θ 2 ) = a 1 1 -a 1 1 -a 2 a 2
The input parameters for numerical examples are given in Table 1, for a time horizon N = 100.

Sensitivity of the value of the information acquisition option

In this section, we examine the impact of new technology's appearance probability on the value of A-option. Across deterioration x, the influence of the new technology's appearance probability on the A-option value is nonmonotone. This result is illustrated by Figure 1 that is divided into five parts, corresponding to the five deterioration states x. In each part, the probability of a low-profit level (π p (θ 1 )) is distributed from 0 to 1 (respectively, from left to right). The value of the A-option is considered in four cases of the new technology's occurrence probability: p n = [0.3 0.4 0.5 0.6]. When x = 5, the A-option value is increasing in the appearance probability of new technology ( p n ). As shown, the A-option has the greatest value at p n = 0.6 and the smallest value at p n = 0.3. When x = 2, the A-option value is non-monotone increasing when p n is rising from p n = 0.3 to p n = 0.6. Consider for example, when the probability of the low-profit level is greater than 0.5, π p (θ 1 ) > 0.5, the A-values corresponding to the case p n = 0.4 is greater than the one corresponding to the case p n = 0.6. Thus, an increase in the probability of appearance of new technology does not necessarily imply an increase in the A-option value.

Optimal policy when the possible values for new technology are not extreme

Following Theorem 5, the upper and lower threshold of the new technology profit are: θ h = 1884 and θ l = 1192. In this section, we focus on the case when the possible values for new technology are not extreme (θ 1 = 1200 and θ 2 = 2100). After new technology appearance, the repair and replacement policy for asset has monotone property (see Theorem 2). It is illustrated through the following optimal policy:

• At profit level θ 1 = 1200, we do nothing when deterioration x = 1, 2 and replace the asset by new technology when

x ≥ 3. • At profit level θ 2 = 2100, we replace the asset immediately with new technology regardless of its deterioration level.

Figure 2 present us the optimal policies for the first decision epoch when new technology has not yet appeared. We consider:

• Case I: Imperfect information with acquisition cost c s = 10 and likelihood function (L) defined by a 1 = a 2 = 0.8 • Case II: Perfect information with acquisition cost c s = 10 (a 1 = a 2 = 1).

• Case III: Imperfect information with low acquisition cost c s = 1 and L defined by a 1 = a 2 = 0.8 • Case IV: Low-quality information with L defined by a 1 = a 2 = 0.5 For each subfigure in Figure 2, the optimal choice in the action set {A -Information Acquisition, D N -Do nothing, M -repair, R -Replacement} is a function of the current deterioration state, x (vertical axis) and the probability of a low improvement in the new technology, π p (θ i ) (horizontal axis). We find that information acquisition is made at high deterioration levels. For example, in Figure 2(a), we only purchase information when the deterioration state is x = 4, 5. Furthermore, when π p (θ 1 ) is small, the repair option is used for waiting the appearance of new technology with a high-profit level. As the value of π p (θ 1 ) increases, after the M-region, we gather additional information (A-region), finally, a replacement is made. Moreover, through these numerical examples, the following interesting results are obtained:

• When the deterioration state of the asset is bad enough, and if it is optimal to replace the current asset with a prior π p 2 , then, it is also optimal to replace the current asset with less optimistic beliefs, π p 1 , where π

p 2 ≥ L R π p 1 .
• When the deterioration state of the asset is good enough, and if it is optimal to do nothing with a prior π p 1 , then, it is also optimal to do nothing with higher optimistic beliefs, π p 2 , where π

p 2 ≥ L R π p 1 .
• At the same value of the belief in the new technology's expected profitability, if it is optimal to replace the current asset with the deterioration state, x 1 , then it is also optimal to replace the current asset with the higher deterioration states, x 2 where x 2 > x 1 . • At the same value of the belief in the new technology's expected profitability, if it is optimal to do nothing with the deterioration state, x 2 , then it is also optimal to do nothing with the lower deterioration states, x 1 where x 2 > x 1 . • At the deterioration state x, if it is optimal to maintain the asset when the current knowledge on the profitability distribution of the new technology is π p 1 and π p 2 , then it is also optimal to maintain it when

π p ∈ [π p 1 , π p 2 ].
• Increasing the information quality expands the information acquisition region. In fact, the A-region in Case II of perfect information is the largest region, while in Case III of low-quality information, information is not purchased even if its cost is lower than the cost in case 2. In addition, for information having the same quality, a decrease in the information cost also expands the A-region.

Next, we consider the impact of technology change and A-option on the repair and replacement policy. Consider Figure 3, each subfigure, respectively, presents the following cases:

• Case a: Technological change is not considered.

• Case b: Technological change is considered, but A-option is not available.

• Case c: The A option is considered for future technology's profitability Comparing Figures 3, we tend to prolong the current asset's economic life using the D N and M options when considering the possibility of new technology appearance. Consider Figure 3(b), for example, at the deterioration state x = 4 the option M is used when the high-profit level of the new technology is estimated greater than 0.75 (i.e. π p (θ 1 ) ≤ 0.25). If we do not consider technological change (Figure 3(a)), the R-option will exercised at x = 4. Comparing Figures 3(b) and(c), we find that considering the A-option does not affect the optimal decisions at low deterioration states (x < 4). For the high deterioration states, we tend to gather information on the new technology's profitability in order to decide whether to replace the asset by current technology or not.

Discussion on the contributions and the application of the proposed model

The primary motivation of this paper is related to following issues: when the manager, who faces to maintenance decisions for an asset, recognises the possibility of the appearance of a new technology in near future (within a horizon N ), is it necessary to acquire more information about the new technology in order to give a better operational decision? Should he/she replace the current asset by a new one or should he/she try to maintain the current asset to wait for a superior technology? In this context, decisions to maintain or to replace the asset are conventionally based on ratios of cost and system performances, characterised by the profit accumulated during a period time G(x). The investment decision will be based on the level of obsolescence of the current system with respect to known or estimated performance of a new technology that is available or not yet available on the market. It is clear that the asset performance level is also a function of level of degradation. This allows to link with maintenance goals. Moreover, one of the challenges for making investment decision is the acquisition of information on future developments in terms of technological performance. To our knowledge, this paper is the first work to link these above aspects: information acquisition and new technology adoption with operational decisions of repair and replacement. In fact, good relations between maintenance strategies and economic investment strategies are defined in this paper.

Discussion about technology evolution and forecasting

To limit the scope of our study, we focus on technological aspects regardless of the various markets, including supply and demand. Moreover, we do not consider the budget constraints for an investment. This hypothesis may seem restrictive but remains consistent with the views of an entire economist community; whereas, if technological change directly effects on the company's competitiveness, it will be able to justify the introduction of capital for making such investments.

The model developed in this paper is dedicated to machinery industry that is more mature and less dynamic than others, such as the pharmaceutical and telecommunications equipment industries. The times of technological change in the machinery industry are long compared to the component lifetime. Thus, the main concern of the manager can be reduced to the definition of an optimal maintenance policy incorporating the possibility of a unique new technology over an interesting horizon. That is the reason, only one possibility of a new technology's appearance is considered in this paper. In detail, we focus on a stage (e.g. few months or 1-2 years) in which a new technology is not yet occurred but its arrival is warned with uncertain information about its performance. [START_REF] Nguyen | Maintenance and Investissement Policies under Uncertainty Technology Evolution[END_REF] gives us an overview of the methods of technology forecasting, that can be summarised in Table 2. Generally, technology forecasting methods can be classified into two main groups: the exploratory and normative methods. The exploratory forecasting is based on past and present data to estimate changes in future, while the normative forecasting starts with future needs and then identifies the technological performance necessary to meet them. The choice of forecasting method depends on the forecasting period, the required accuracy level and also the forecast target. However, exploratory technological forecasting built on empirical basis presents a greater interest than the normative approach in most practical cases. Specially, the statistical exploratory methods (such as the extrapolation trends or growth curves) are widely used in practice for prediction of technology evolution. The results obtained by these forecasting methods often have the form of statistical data analysis, such as the probability distribution of the asset performance π p over .

Information about the new technology is defined in this paper as the prediction of future machine characteristics such as performance levels (e.g. power, capacity, failure rate). It just gives a picture of the trend in the future and not be regarded as absolute. Conventionally, it is based on four elements: the object (the proposed new technology), the interesting horizon N , the levels of different features (e.g. different performance levels of new machine, θ i ) and appearance time of new technology (characterised by the appearance probability for every period p n ). The predicted performance levels of the new technology, θ i are assumed to be not change over time. This assumption is also consistent with the reality that if new technology is not yet available on the market, there is not enough information about the new technology's profitability to cause significant changes of the original estimation, especially when the estimated values are considered in a finite planning horizon. On the other hand, the value of θ i is expected based on both of information of new technology's characteristics and of the functional policies of the firm. It is also considered as the maximum expected discounted value of a classic replacement problem over an infinite horizon [START_REF] Chan | Optimum Maintenance Policy with Markov Processes[END_REF].

According to [START_REF] Firat | Technological Forecasting, A Review[END_REF], the following information resources (noted s j in this paper) are frequently utilised to forecast technological evolution (in particular θ i ) in the automobile and machinery industries:

• Patent frequency analyses • Benchmarking studies • Portfolio • Scenario analyses • Quality function deployment • Expert interview
In order to obtain these information, the firm can purchase them from market research companies. Higher quality information is, greater investment budget (or greater purchasing cost) is. The information quality could be interpreted by the likelihood function L(s j |θ i ), that is the probability that information s j is obtained if the performance of new technology is θ i . The value of likelihood function L(s j |θ i ) can be predicted by the confidence degree of the manager on the information resource.

Discussion about interesting horizon

In this paper, we assume that if a new technology is not yet on the market after a finite horizon N , the manager do not care anymore about its occurrence. This assumption may be seen as restrictive, however, we justify it by considering the technology breakthrough time interval as an interval of interest after which the impact of the new technology's appearance becomes negligible (Nguyen, Yeung, and Castanier 2014). Furthermore, this assumption is also appropriate to the survey result of [START_REF] Firat | Technological Forecasting, A Review[END_REF], that is, the greater concern of the majority of the managers towards short-term market trend, and then the forecast of new technology is only interested for short-range decisions (under 5 years).

Although we focus on a finite horizon (N periods) when new technology is not yet available on the market with uncertain information about its profitability, but the global decision is optimised over an infinite horizon. In which, the problem is divided into three stages:

• Stage 1: before the new technology is available, information about its profitability is uncertain,

• Stage 2: after the new technology becomes available, its expected value of total profit, θ i (after subtracting the purchase cost) is known to the firm. • Stage 3: after N periods, if the new technology has not yet been on the market, then the decisions are reduced to repair, to do nothing or to replace the asset by the current technology.

All above three stages are combined to evaluate a general objective function in Equation ( 1). Dividing the problem into three stages highlights the following advantages:

• simplifying a complex problem into familiar issues (such as Stage 2 and 3 that were solved by the stationary MDP Hopp and Nair 1994; Nguyen, Yeung, and Castanier 2014) • easily addressing to the change of the action set for every stage.

Discussion about the impact of repair activities

Restoration of the asset to a given deterioration state

In this paper, we assume that after repair, the system is restored to a given degradation threshold. This assumption is not too restrictive in the framework of machinery industry. In fact, considering a drilling CNC machine, its lifetime is expected to be from 15 to 25 years. This lifetime is too long when considering the typical life expectancy of a countersink (that depends on the material types but could probably be counted by days or a few months). On the other hand, the typical life expectancy for grease lubricated spindles below 24,000 RPM max speed is approximatively 6000 h (2-3 years). Repair activities of this CNC machine are performed by the replacement of the CNC tools. Therefore, within short period times of the new technology's arrival, few months or 1-2 years, the effect of a repair action can be considered as the restoration of the CNC machine to a given state. This state does not depend on the current profitable deterioration state (principally caused by the tool deterioration) and is not as good as new (due to the CNC age).

General model of repair effect

In this section, we relax the assumption about repair effect by considering a general model for it. In detail, the repair can restore the asset to any deterioration level between the new state (X = x 0 ) (due to a perfect repair) and a failure state (X = z) (due to a negative impact of repair action on the system). The uncertainty impact of repair activity is characterised by q(x ′ |x), probability that the deterioration state of the current asset is x ′ after repair, given that the deterioration state before repair is x. Consider the transition matrix characterised the repair effect, q(x ′ |x), we find that:

• ∀x, x ′ , x M ∈ X , q(x M |x) = 1 and q(x ′ |x) = 0 (∀x ′ = x M ): the repair restores the machine to a given deterioration state, x M . This is a particular case that has examined in above sections of this paper.

• ∀x, x ′ ∈ X , q(x ′ |x) = 0, ∀x ′ > x: the repair restores the machine to any deterioration level, x ′ , between as-goodas-new (x 0 ) and as-bad-as-old (x) with the probability q(x ′ |x) = 0, ∀x ′ ≤ x. This is a particular case, in which negative impacts of repair is assumed to be negligible, i.e. after repair actions, the system could not get worse than it has been.

Using the general model for repair effect, the repair option can be re-written as follows:

M N n (x, π p ) = -c M (x) + x ′ ∈X q(x ′ |x)D N N n (x ′ , π p ) (12)
where c M (x) is an increasing function in x characterised the repair cost. The rest of our formulation model (Equations ( 1)-( 4)) does not change. Similarly, after the new technology's appearance, the optimisation value, V (x, θ i ) can be re-written by:

V (x, θ i ) = max      D N (.) = G(x) + β x ′ P(x ′ |x) V (x ′ , θ i ) M(.) = -c M (x) + x ′ ∈X q(x ′ |x) D N (x ′ , θ i ); R(.) = max R(x), θ i + b(x)      (13)
And the value function, Ṽ (x), of the case without technology change is given by:

Ṽ (x) = max    D N (x) = G(x) + β x ′ P(x ′ |x) Ṽ (x ′ ) M(x) = -c M (x) + x ′ ∈X q(x ′ |x) D N (x ′ ); R(x) = -c 0 + b(x) + D N (x 0 )    (14)
Using these new formulas of M-option (in every stages), we then verify if all results presented in Section 3 will be hold-on when applying the general model for repair effect or not.

Theorem 6 If ∞ x ′ =ζ q(x ′ |x) is non-decreasing in x for all ζ, x ∈ X , then we have:

(1) V (x, θ i ) is non-decreasing in θ i ;

(2) Ṽ (x), V (x, θ i ) and V N n (x, π p ) are non-increasing in x;

(3) If the signal process satisfies the MLR property then V N n (x, π ) is non-decreasing in π p according to LR-order (π

p 2 L R π p 1 ⇒ V N n (x, π p 2 ) ≥ V N n (x, π p 1
)); (4) There are not enough evidences to conclude about monotone properties of the optimal policy Ṽ (x) (or V (x,θ i ) );

(5) For all n, the value function V N n (x, π p ) is convex in π p ; (6) In the case of perfect information, if it is optimal to acquire information at both π Theorem 6 shows that almost results of structure properties of our model presented in Section 3 are still verified when using the general model of repair effect. However, the monotone property of the optimal policy Ṽ (x) (or V (x,θ i ) ) cannot be assured when assumption of the fixed repair effect is relaxed.

Conclusions

Unlike classical maintenance modelling that only considers a system's degradation states from the operator's point of view, this paper highlights the perspective of maintenance modelling in the context of total productive maintenance (TPM). In detail, we propose a new model to combine the issue of optimising maintenance at the operational level and investment in uncertain technological change. Hence, this model allows optimising several decision-making rules for improving the performance for a long-term horizon with respect to the short-term availability of the system. Moreover, we define a new option 'to acquire information' in the maintenance optimisation domain that allows us to subsequently quantify the level of uncertainty acceptable for a decision. This contribution can help to change models and current maintenance practices that may be considered conservative in the investment and the associated risk management fields. We also believe that such an option could be re-used or extended in other maintenance contexts such as the identification of the real failure causes when diagnosis is considered imperfect, for example. Specially, we use a non-stationary Markov decision process to solve for the optimal policy as a function of performance and cost. Through mathematical analyses, the two major contributions of the optimal policy are highlighted:

(1) There exists the profit threshold of new technology such that for all expected values of new technology which exceed this threshold, it is optimal for the firm to not obtain any additional information and invest immediately in new technology upon its arrival. (2) In the case of perfect information, if the firm decides to acquire more information when the current knowledge on the profitability distribution of the new technology is π In addition, we present numerical examples to illustrate the performance of our model and to consider the influence of the parameters characterising the information quality and the appearance probability of new technology on the optimal decisions. Studying how maintenance and information gathering policies interact, we obtain the following interesting structural results of the optimal policy:

• It is not necessary to acquire additional information about new technology's expected profitability when the arrival probability is increasing. • If it is optimal to replace (or to do nothing) the current asset with a prior distribution of new technology's expected profitability, then it is also optimal to replace (or to do nothing) the current asset with less (or higher) optimistic beliefs. • At the same value of the belief in the new technology's expected profitability, if it is optimal to replace or to do nothing the current asset with the deterioration state, then it is also optimal to replace the current asset with the higher or lower deterioration states.

Finally, we also discuss on the contributions and the assumptions of our model in the context of the machinery industry. A general model for the repair effect was also examined and through it, we showed that almost results of structure properties of our model are still verified when relaxing the assumption of fixed repair effect. However, the monotone property of the optimal policies should be more studied more in an extended version of this work.

In future work, the estimated benefit change of new technology over time could also be considered as well as its impact on the optimal decision. Moreover, in the scope of this paper, we utilised the classic backward induction algorithm to solve the model. Considering a real-world problem with more profit levels generates an immense state space that can cause difficulty in implementation in which case a more efficient algorithm should be developed.

Disclosure statement

No potential conflict of interest was reported by the authors.

  also optimal to acquire the information for any πp α = απ p 1 + (1α)π p 2 , where 0 ≤ α ≤ 1.The result presented in Corollary 2 shows that in the case of perfect information, if the firm decides to acquire more information when the current knowledge on the profitability distribution of the new technology is π p acquire more information when the profitability distribution belongs to the interval [π

Figure 2 .

 2 Figure 2. Numerical example's optimal policies.

Figure 3 .

 3 Figure 3. Impacts of technology change and/or the A-option on the optimal repair and replacement policies.

  0 ≤ α ≤ 1; (7) We do not gather additional information about the new technology's performance if all expected values of the new technology's profit (θ i ) are:(a) greater than the high-profit threshold θ h determined by Lemma 2 (b) lower than the low-profit threshold θ l = -c 0 + D N (x 0 ).

Table 1 .

 1 The input parameters for the numerical examples.

	Discount factor &	β	ǫ	δ
	Appearance probability	0.9	0.7	0.98
	Repair cost &	ν	h 1	h 2
	Salvage value	80	50	20
	Purchase price of current tech	c 0	θ 1	θ 2
	& New tech profit	400	1200	2100

Figure 1. Impact of the new technology's appearance probability on the Value of Information acquisition option.

Table 2 .

 2 Methods of technology forecasting.

	Exploratory

Appendix 1.

A.1 Theorem 2

Proof.

(1) We proceed by induction on the steps of the value iteration algorithm.

Let Vn (x, θ i ) be the maximum expected discounted value over n decision periods, and V (x, θ i ) be its asymptotic value when n tends to infinity. Without loss of generality, let V0 (x, θ i ) = 0, ∀θ i ∈ , ∀x ∈ X . Firstly, for n = 1, we have:

• D N 1 (.) and M1 (.) are constant in θ i .

• R1 (.) is non-decreasing in θ i as maximum functions of non-decreasing functions.

Then, we deduce V1 (.) is non-decreasing in θ i .

We assume now that Vn-1 (.) is non-decreasing in θ i .

As ∀x ′ P(x ′ |x) Vn-1 (.) is non-decreasing in θ i ⇒ D N n (x, θ i ) is non-decreasing in θ i .

As sums of non-decreasing functions, Mn (.) is non-decreasing in θ i .

As maximum functions of non-decreasing functions, Rn (.) is non-decreasing in θ i , ∀n. Finally, Vn (x, θ i ) is non-decreasing in θ i . And while n → ∞, Vn (.) → V (.).

(2) Similarly, from Lemma 1, we can deduce easily Ṽ (x) and V (x, θ i ) are non-increasing in x, ∀x ∈ X .

(3) From Lemma 1, by recurrence we can demonstrate it similar to [START_REF] Nguyen | Maintaining a System Subject to Uncertain Technological Evolution[END_REF]

A.2 Theorem 3

Proof.

(1) We deduce this point by induction. At n = N , V N N (x, π p ) = Ṽ (x), that is non-increasing in x, ∀x ∈ X (Theorem 2) Next, we assume that V N n+1 (x, π p ) is non-increasing in x, ∀x ∈ X and deduce this conclusion is correct at period n. In fact, we have:

) is non-increasing in x as sums of non-increasing functions. M N n (.) and R N n (.) are non-increasing in x due to the non-increasing property of D N N n (.). Then O N n (.) is non-increasing in x as the maximum of non-increasing functions. Finally, A N n (.) is non-increasing in x because of the non-increasing property of O N n (.). So the assertion is proved for all n, V N n (.) is non-increasing in x for all n.

(2) We prove this assertion by induction.

At the last period n = N , we have V N N (x, π p ) = Ṽ (x) for all π p . Then, V N N (x, π p ) is non-decreasing in π p according LR order. Next, we assume that V N n+1 (x, π p ) is non-decreasing in π p , and obtain:

A.3 Theorem 4

Proof. We proceed by induction. The terminal value

We have:

Consider the convexity of A N n (x, π p ), we have:

The posterior probability distribution π ′ p (π p α , s j ) can be written by:

A.4 Lemma 2

Proof. From Equation ( 7), we will invest in new technology if and only if:

Then, the threshold to invest immediately in new technology after its appearance, regardless the deterioration state of the current asset is:

) is non-increasing in x, we have:

. We deduce:

A.5 Theorem 5

Proof.

(1) We deduce this point by induction. From Lemma 2, if θ i ≥ θ h , we invest immediately in new technology regardless the deterioration state of the current asset, then:

We can rewrite (4) by:

Let [h(.)] π p denote the component that depends only on the variable π p of the multi-variable function h(.). We have:

, the option values are independent of the π p and we do not gather additional information about the new technology's performance. Secondly, at n = N -1:

Therefore, we can conclude that the differences between D N , M, R options are constant in π p . In other words, let a * denote the optimal operation for the current asset at state (x, π p ), we have:

)) Now we prove that we do not purchase information. Recall that:

(x, π p ), from (A1), (A5), and (A6) we have:

Hence, with c s > 0, we can deduce: A N N -1 (x, π p ) < O N N -1 (x, π p ) So, at period n = N -1, we conclude:

Next, at period n = N -2:

Then a * at period n = N -2 are constant in π p .

Combining with (a), (b) and Equation (A6), we deduce:

Therefore, we can conclude that at period n = N -2:

Then, for all n ∈ N , due to the recurrence result at period n -1, we deduce:

(2) It is obvious that if all expected values of the new technology's profit are lower than the investment profit in current technology, we will replace the asset by a new identical one. Then, the selection of the optimal operation for the current asset does not depend on the probability distribution of the new technology's profit (π p ) and we do not gather additional information about the new technology's performance.

A.6 Corollary 2

Proof. Having the perfect information s j , we know exactly the profit level θ i of new technology. The posterior probability distribution π p is the unit vector e i defined by: e i = {0 0 . . . 1 at index i . . . 0}.

In addition, e i depends only on the information s j and is independent of the prior probability distribution π p . We can rewrite π ′ p (π p , s j ) by e i (s j ). As f (π

), we have:

The hypothesis that if it is optimal to acquire information at both π p 1 and π p 2 is re-written as follows:

Multiplying, respectively, two inequalities by α and 1α and then summing them:

And thus we deduce the final result:

A.7 Theorem 6

Given that the formula equation of other options (D

do not change, we just consider if any proof result changes with new formula of the repair options, M.

Proof. As ∞

x ′ =ζ q(x ′ |x) is non-decreasing in x for all ζ, x ∈ X , then for all f (x) non-increasing in x, ∀x ∈ X , we have:

g( x) is a non-increasing function in x, ∀x ∈ X (following Lemma 1). Firstly, the monotone properties of values functions, Vn-1 (.) (or Ṽn-1 (.), V N n-1 (.)) will be examined following Theorems 2 and 3. Consider the monotone properties of following functions:

x ′ ∈X q(x ′ |x) D N (x);

x ′ ∈X q(x ′ |x) D N (x, θ i );

x ′ ∈X q(x ′ |x)D N N n (x, π p ) These functions are non increasing in x when D N (x), D N (x, θ i ) and D N N n (x, π p ) are non increasing in x. In fact, similar to the proof of Theorems 2 and 3 using induction demonstration, we assume that: Vn-1 (.) (or Ṽn-1 (.), V N n-1 (.)) is non-increasing in x; and deduce that D N (x, θ i ) is non increasing in x because ∀x ′ P(x ′ |x) Vn-1 (.) is non-increasing in x (similarly for D N (x), D N N n (x, π p )). Therefore, we show that new formula of Mn (.) (or Mn (.), M N n (.)) is also non-increasing in x. Continue in the same way of the proof of Theorems 2 and 3 the monotone properties of value functions are obtained (1) V (x, θ i ) is non-decreasing in θ i ;

(2) Ṽ (x), V (x, θ i ) and V N n (x, π p ) are non-increasing in x, ∀x ∈ X ; (3) If the signal process satisfies the MLR property then V N n (x, π ) is non-decreasing in π p according to LR-order (π

). Secondly, in order to consider the monotone properties (non-decreasing in x) of the optimal policy Ṽ (x) with new formula of M-option, we re-evaluate the difference between M-option and D N -option ( Ṽ 21(x)) or between R-option and M-option ( Ṽ 32(x)). The optimal policy Ṽ (x) is non-decreasing in x when we can prove that Ṽ 21(x) or Ṽ 32(x) are non-decreasing in x. However, with the new formula of M-option given by ( 14), the monotone property of Ṽ 21 (x) is not assured. In fact, for all

) is non-increasing in x, then there are not enough evidences to conclude about the monotone properties (non-decreasing in x) of the optimal policy Ṽ (x) (similar to the optimal policy V (x)).

Thirdly, the convexity of V N n (x, π p ) is verified if new formulas of M N n (x, π p ) is convex in π p . In fact, similar to the induction proof of Theorem 4, we can deduce that D N N n (x, π p ) is convex in π p . Following elementary properties of convex function, we have: g(x, y) convex in y ⇒ g(x, y)d x convex in y In order words, x ′ P(x ′ |x)D N N n (x ′ , π p ) is convex in π p . Therefore, new formulas of M N n (x, π p ) is convex in π p . As the Theorem 4 is verified with new formulas of M N n (x, π p ), then the Corollary 2 is also derived (similar to the Proof of Corollary 2) Fourthly, consider the proof of Lemma 2 and Theorem 4, they are verified if we prove that:

• M(x, θ i )b(x) is non-increasing in x • the difference between D N , M, R are constant in π p . In other words, M N n (x, π p )] π p = βp N -1 i π p (θ i )θ i

In fact, for x 1 ≤ x 2 , we have:

≤ 0 because b(x 1 )b(x 2 ) ≤ c M (x 2 )c M (x 1 ) and x ′ p(x ′ |x) D N (x ′ , θ i ) is non-increasing in x.

On the other hand, we have:

Therefore, Lemma 2 and Theorem 4 is verified with the general model of repair effect.