Prof. Mikhaïl Denis Bosq

Prof. Gérard Nikulin

Prof Robert Duru

Prof Launois

Marina

Keywords: Séquences de Markov, Estimation de paramètre, Estimation-processus du maximum de vraisemblance Markov sequences, Parameter estimation, One-step maximum likelihood estimator-process, Bayesian estimator, Health economics, Cost-effectiveness analysis, Catheter-related bloodstream infection

un-pas, Estimateur de Bayes, Economie de la santé, Analyse coût-efficacité, Bactériémies liées aux cathéters

Résumé

Dans la première partie de cette thèse, nous considérons plusieurs problèmes d'estimation de paramètre de dimension finie pour les séquences de Markov dans l'asymptotique des grands échantillons. Le comportement asymptotique des estimateurs bayésiens et les estimateurs obtenus par la méthode des moments sont décrits. Nous montrons que sous les conditions de régularité ces estimateurs sont consistants et asymptotiquement normaux et que l'estimateur bayésien est asymptotiquement efficace. Les estimateur-processus du maximum de vraisemblance un-pas et deux-pas sont étudiés. Ces estimateurs nous permettent de construire des estimateurs asymptotiquement efficaces sur la base de certains estimateurs préliminaires, par exemple, les estimateurs obtenus par la méthode des moments ou l'estimateur de Bayes et la structure de l'estimateur du maximum de vraisemblance un-pas. Nous proposons notamment des processus autorégressifs non linéaires comme exemple et nous illustrons les propriétés de ces estimateurs à l'aide de simulations numériques. Dans la deuxième partie, nous donnons les applications de processus de Markov en économie de la santé. Nous comparons les modèles de Markov homogènes et non-homogènes pour l'analyse coût-efficacité de l'utilisation de pansements transparents contenant un gel de gluconate de chlorhexidine par rapport aux pansements transparents standard. Le pansement antimicrobien protège les accès vasculaire centrale et réduit le risque de bactériémies liées aux cathéters. L'impact de l'approche de modélisation sur la décision d'adopter des pansements antimicrobiens pour les patients gravement malades est discuté.

Introduction

Cette thèse est consacrée aux problèmes de l'estimation des paramètres de séquences de Markov et l'utilisation des modèles de Markov dans le domaine de l'économie de la santé. Elle est donc composée de deux parties.

La première est dédiée aux problèmes de l'estimation des paramètres multidimensionnels de séquences de Markov X n = (X 0 , X 1 , . . . , X n ) présentées sous la forme de processus autorégressif non linéaire d'ordre 1. Pour ce modèle, nous décrivons le comportement asymptotique de plusieurs types d'estimateurs. Nous montrons la consistance et la normalité asymptotique des estimateurs bayésiens et des estimateurs obtenus par la méthode des moments. Le principal résultat de la première partie est la construction des estimateur-processus du maximum de vraisemblance un-pas et deux-pas (one and two-step MLE-processes). Les résultats sont illustrés par des simulations numériques, qui confirment l'applicabilité de ces méthodes.

La deuxième partie est dédiée aux problèmes d'application des modèles de Markov dans le domaine médico-économique. Le but de cette partie est d'évaluer le rapport coût-efficacité de l'utilisation de pansements transparents contenant un gel de gluconate de chlorhexidine chez les patients gravement malades en se concentrant sur la perspective de l'unité de soins intensifs. Pour mesurer l'impact sur les résultats d'analyse coût-efficacité, deux approches de modélisation ont été considérées : le modèle de Markov homogène et le modèle de Markov non-homogène. Ces deux approches différentes sont fondées sur les chaînes de Markov d'ordre 1, qui est une suite de variables aléatoires X n = (X 0 , X 1 , . . . , X n ). Elle prend ses valeurs dans l'espace dénombrable d'états prédéfinis et les probabilités de transition entre ces états dépendent de la matrice de transition.

Nous abordons ensuite en détail le contenu de ces deux parties.

Dans le premier chapitre nous considérons plusieurs problèmes de l'estimation des paramètres de dimension finie pour les séquences de Markov dans l'asymp-totique de grands échantillons. Supposons que π (ϑ, X j-1 , X j ) est une densité de transition qui dépend d'un paramètre inconnu ϑ ∈ Θ ⊂ R d .

Le modèle considéré est une série temporelle non linéaire (X j ) j≥0 qui est décrit par l'équation X j = S (ϑ, X j-1 ) + ε j , j = 1, 2, . . . , où les variables aléatoires (ε j ) j≥1 sont i.i.d. avec une fonction de densité régulière g (x). La fonction S (ϑ, x) est supposée être connue et régulière par rapport à ϑ ∈ Θ ⊂ R d , où Θ est un ensemble ouvert, borné et convexe.

Le processus (X j ) j≥0 a la densité de transition π (ϑ, x, x ′ ) = g (x ′ -S (ϑ, x)) .

Nous sommes intéressés par les deux méthodes d'estimation des paramètres. Le comportement asymptotique des estimateurs bayésiens θn et les estimateurs obtenus par la méthode des moments θn sont décrits. Sous les conditions de régularité nous montrons que ces estimateurs sont consistants et asymptotiquement normaux

√ n θn -ϑ ⇒ N 0, I (ϑ) -1 , √ n θn -ϑ ⇒ N (0, B (ϑ)) ,
où I (ϑ) est une matrice d'information de Fisher et B (ϑ) est une matrice de covariance.

L'étude des estimateurs bayésiens dans un certain sens est une suite de travail de Varakin et Veretennikov (2002). Ils ont décrit les propriétés asymptotiques de l'estimateur du maximum de vraisemblance dans le cas unidimensionnel (d = 1) à l'aide du résultat général d 'Ibragimov et Hasminskii (1981).

Nous avons montré que dans le cas régulier l'estimateur bayésien possède les mêmes propriétés asymptotiques que l'estimateur du maximum de vraisemblance.

Ensuite nous étudions l'estimateur obtenu par la méthode des moments. Dans le cas régulier il est consistant et asymptotiquement normal, mais avec la covariance limite non optimale.

Les résultats obtenus dans ce chapitre sont illustrés par plusieurs exemples numériques. Nous avons étudié l'exemple de Varakin et Veretennikov (2002), où l'estimateur du maximum de vraisemblance peut être représenté sous la forme explicite. Ensuite, nous avons étudié les deux autres modèles non-linéaires qui vont servir pour la suite dans la construction des estimateurs du maximum de vraisemblance un-pas et deux-pas.

Dans le deuxième chapitre, les estimateurs du maximum de vraisemblance unpas et deux-pas sont étudiés. Ces estimateurs sont récemment introduits par Y. Kutoyants pour les processus de diffusion ergodiques. Les estimateurs du maximum de vraisemblance un-pas et deux-pas nous permettent de construire des estimateurs asymptotiquement efficaces sur la base de certains estimateurs préliminaires, par exemple, les estimateurs obtenus par la méthode des moments ou l'estimateur de Bayes et la structure de l'estimateur du maximum de vraisemblance un-pas.

L'estimation du maximum de vraisemblance un-pas se fait en deux étapes. Premièrement, nous fixons une période d'apprentissage X N = (X 0 , X 1 , . . . , X N ) des observations X n = (X 0 , . . . , X n ), où N = n δ (N est une partie entière de n δ ) et nous estimons le paramètre inconnu par les observations dans l'intervalle d'apprentissage. Le choix du paramètre δ < 1 va être discuté. Cette période d'apprentissage correspond à une partie d'observations relativement courte.

Ensuite nous utilisons l'idée d'estimateur du maximum de vraisemblance unpas pour construire un estimateur-processus ϑ ⋆ n = ϑ ⋆ k,n , k = N + 1, . . . , n qui est asymptotiquement équivalent à l'estimateur du maximum de vraisemblance.

Introduisons en premier lieu la variable s ∈ (τ δ , 1], où τ δ = n -1+δ → 0 et k = [sn]. Nous pouvons écrire ϑ ⋆ k,n = ϑ ⋆ s,n et considérer l'estimateur-processus ϑ ⋆ n = ϑ ⋆ s,n , s ∈ (τ δ , 1] . Notre but est ainsi de construire l'estimateur-processus ϑ ⋆ n qui est asymptotiquement optimal pour toutes s ∈ (τ δ , 1].

L'estimateur du maximum de vraisemblance un-pas peut être calculé avec la procédure suivante :

ϑ * s,n = θN + 1 √ k I θN -1 ∆ k ( θN , X k N ), où ∆ k (ϑ, X k N ) = 1 √ k k j=N +1 l(ϑ, X j-1 , X j ), est une fonction-score et k = [sn] → ∞.
Sous les conditions de régularité nous montrons que √ k(ϑ * s,nϑ) =⇒ N 0, I (ϑ) -1 . Donc l'estimateur-processus ϑ ⋆ n = ϑ ⋆ s,n , τ δ < s ≤ 1 pour toutes s ∈ (τ δ , 1] est asymptotiquement normal avec la matrice de covariance asymptotiquement optimale.

Pour avoir une période d'apprentissage plus courte, nous introduisons un estimateur intermédiaire en plus et l'estimateur du maximum de vraisemblance deuxpas ϑ ⋆⋆ s,n = ϑ ⋆⋆ k,n , k = N + 1, . . . , n . Cette procédure nous conduit à l'estimateurprocessus qui est aussi asymptotiquement efficace. L'estimation du maximum de vraisemblance deux-pas se fait en trois étapes. Premièrement, à la base des observations X N = (X 0 , X 1 , . . . , X N ) nous obtenons l'estimateur préliminaire ϑ N,1 qui est asymptotiquement normal. Ensuite, nous introduisons le second estimateur préliminaire θk,2 = θN,1 + 1

√ k I θN,1 -1 ∆ k ( θN,1 , X k ),
où ∆ k (ϑ, X k ) est la fonction-score définie ci-dessus.

L'estimateur-processus du maximum de vraisemblance deux-pas ϑ ⋆⋆ s,n = ϑ ⋆⋆ k,n , où k = N + 1, . . . , n nous construisons en utilisant ce deuxième estimateur préliminaire :

ϑ ⋆⋆ s,n = θk,2 + 1 √ k I θk,2 -1 ∆ k ( θk,2 , X k ).
Sous les conditions de régularité nous montrons que l'estimateur-processus ϑ ⋆⋆ s,n est asymptotiquement normal √ k(ϑ ⋆⋆ s,nϑ) =⇒ N 0, I (ϑ) -1 et sa matrice de covariance est asymptotiquement optimale.

L'avantage principal des estimateur-processus du maximum de vraisemblance un-pas et deux-pas est qu'ils sont asymptotiquement efficaces pour chaque s et en même temps facile à calculer. Notons que l'estimateur-processus ϑ ⋆ k,n peut être écrit dans la forme récurrente :

ϑ ⋆ k+1,n = k k + 1 ϑ ⋆ k,n + 1 k + 1 θN + 1 k + 1 I θN -1 l θN , X k , X k+1 .
Pour illustrer les résultats de ce chapitre, nous proposons des processus autorégressifs non-linéaires comme les modèles d'observations. Les estimateur-processus du maximum de vraisemblance un-pas et deux-pas sont obtenus dans les deux exemples avec les simulations numériques. Dans le premier cas nous considérons un estimateur du maximum de vraisemblance comme l'estimateur préliminaire et nous cherchons à l'améliorer à l'aide de la procédure du maximum de vraisemblance un-pas. Dans le deuxième cas nous proposons l'estimateur obtenu par la méthode des moments comme l'estimateur préliminaire et nous cherchons à l'améliorer à l'aide de la procédure du maximum de vraisemblance un-pas et deux-pas.

Dans le troisième chapitre, nous considérons les applications de processus de Markov en économie de la santé. L'intérêt médical de ce travail est la gestion des patients gravement malades qui ont eu l'insertion des cathéters intravasculaires (cathéters veineux centraux et cathéters artériels) dans l'unité de soins intensifs. L'insertion des cathéters intravasculaires peut conduire à des complications infectieuses graves, y compris les bactériémies liées aux cathéters, car ils sont une porte d'entrée pertinente pour les micro-organismes dans la circulation sanguine. C'est une complication relativement fréquente (1-5 épisodes / 1000 jours-cathéter) et potentiellement mortelle observée chez les patients gravement malades dans les unités de soins intensifs.

Nous comparons les modèles de Markov homogène et non-homogène de l'analyse coût-efficacité de l'utilisation de pansements transparents contenant un gel de gluconate de chlorhexidine par rapport aux pansements transparents standard. Le pansement antimicrobien protège les accès vasculaire centraux et réduit le risque de bactériémies liées aux cathéters. Les deux modèles simulent les différentes trajectoires observables de la santé des patients relative au risque de contracter une infection liée aux cathéters et évaluent l'incertitude autour des estimations des essais cliniques.

Dans le modèle de Markov homogène nous considérons l'ensemble de 6 états de santé de patients E = (E 1 , E 2 , . . . , E 6 ) :

E 1 = {Aucune infection liée aux cathéters} ; E 2 = {Infection liée aux cathéters}; E 3 = {Dermatite de contact}; E 4 = {Changement du pansement}; E 5 = {Décharge}; E 6 = {Décès} .
Dans le modèle de Markov non-homogène nous considérons 8 états de santé de patients E = (E 1 , E 2 , . . . , E 8 ) : Les états Décharge et Décès sont les états absorbants. Nous considérons l'ensemble de probabilités Θ = (π(E l , E m )) 6×6 ou Θ = (π(E l , E m )) 8×8 comme les matrices de transition inconnues. Les estimateurs de ces matrices π(E l , E m ) sont été obtenus à l'aide de base de données en vie réelle. La matrice de probabilités de transition peut être considérée comme le paramètre multidimensionnel provenant de cette base de données. Et tous les calculs sont fait à partir de ces estimateurs de probabilités de transition π(E l , E m ).

E 1 = {Insertion d'
Ces nouveaux modèles sont profondément différents de toutes les évaluations économiques antérieures de pansements antimicrobiens concernant la prévention des infections liées aux cathéters. Ces évaluations ont utilisé des modèles d'arbre décisionnel représentant les choix thérapeutiques. L'impact de l'approche de modélisation sur la décision d'adopter des pansements antimicrobiens pour les patients gravement malades est discuté.

Les résultats de ces chapitres ont fait l'objet de publications et présentations orales.

Les articles :

Chapter 1

On Parameter Estimation for Markov Sequences

Introduction

We consider the problem of parameter estimation in the case of observations of Markov sequence X n = (X 0 , X 1 , X 2 , . . . , X n ). We suppose that the transitional density π (ϑ, X j-1 , X j ) depends on some unknown finite-dimensional parameter θ and we study the properties of the estimators of this parameter in the asymptotic of large samples (n → ∞). It is known that under regularity conditions the maximum likelihood estimator (MLE) θn is consistent, asymptotically normal

√ N θN -ϑ ⇒ N 0, I (ϑ) -1
and asymptotically efficient (see, [START_REF] Ogata | The weak convergence of the likelihood ratio random fields for Markov observations[END_REF], [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF]). Here I (ϑ) is Fisher information matrix.

In our work we are interested by two methods of the parameter estimation. The first one is Bayesian and the second is the method of moments. In both cases we show that the corresponding estimators are consistent and asymptotically normal.

The model of observations is a nonlinear time series (X j ) j≥0 satisfying the relation

X j = S (ϑ, X j-1 ) + ε j , j = 1, 2, . . . (1.1)
and the initial value X 0 is given too. The random variables (ε j ) j≥1 are i.i.d. with smooth density function g (•). The process (X j ) j≥0 has a transition density

π (ϑ, x, x ′ ) = g (x ′ -S (ϑ, x)) .
It depends on the parameter ϑ and defines the probability of reaching the state x ′ after sojourning in the state x. The parameter ϑ takes its values in some open bounded set Θ ⊂ R d .

We consider two estimators. The first one is constructed following Bayesian approach, i.e., we suppose that the unknown parameter is a random variable with known prior density p (ϑ) , ϑ ∈ Θ. For the simplicity of exposition we take the quadratic loss function W (u) = |u| 2 and therefore the Bayesian estimator (BE) θn is the conditional expectation θn = E (ϑ|X n ). It has the representation

θn = Θ p (θ) V (θ, X n ) dθ -1 Θ θp (θ) V (θ, X n ) dθ,
where the likelihood function

V (ϑ, X n ) = π 0 (X 0 ) n j=1 π(θ, X j-1 , X j ), ϑ ∈ Θ. (1.2)
Here π 0 (x) is the density of the initial value X 0 .

We suppose in this work that the process (X j ) j≥0 is geometrically mixing and has invariant distribution with the density π * (ϑ, x). For simplicity of exposition we put π 0 (x) = π * (ϑ, x). In this case the process (X j ) j≥0 is stationary.

(H) The function π (ϑ, x, x ′ ) is two times continuously differentiable in ϑ and the derivative of the function ℓ (ϑ, x, x ′ ) = ln π (ϑ, x, x ′ ) has polynomial majorants. Moreover we suppose that the invariant density has polynomially decreasing tails, such that the moments of the corresponding functions used in the proof exist.

Of course, this means that the functions g (•) ∈ C 2 and S (•, •) ∈ C 2 ϑ with the corresponding conditions on their derivatives.

Our work is a continuation of the work by Varakin and Veretennikov [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF], where the asymptotic properties of the MLE of the one-dimensional parameter ϑ are described with the help of the general results by Ibragimov and Hasminskii [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF].

We suppose that all conditions of regularity imposed on the function f (ϑ, x, x ′ ) in [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] and providing the ergodicity, the existence of the unique invariant measure π * (ϑ, x, x ′ ) satisfying the mentioned above condition on the tails hold. The conditions of Theorem 1 in [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] are supposed to be fulfilled too. Moreover, our proofs of the asymptotic properties of the likelihood ratio process (Lemmas 1-3) follow the main steps of the corresponding proofs in [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF]. The only difference is in the dimension of the unknown parameter.

We have the law of large numbers with rate: for any p > 0

E ϑ 1 n n j=1 [h (ϑ, X j ) -E ϑ h (X j )] p ≤ C n p/2 . (1.3)
These regularity conditions allow us, for example, differentiate w.r.t. ϑ under sign of mathematical expectation. The Fisher information matrix is

I(ϑ) = E * ϑ l (ϑ, X 0 , X 1 ) l (ϑ, X 0 , X 1 ) T ,
where the dot means the derivation w.r.t. ϑ and T means the transpose of a matrix. The mathematical expectation here is w.r.t. the invariant measure π * (ϑ, •).

We suppose that this matrix is uniformly in ϑ ∈ Θ non-degenerate and bounded

0 < inf ϑ∈Θ inf |λ|=1 λ T I(ϑ)λ, (1.4)
sup ϑ∈Θ sup |λ|=1 λ T I(ϑ)λ < ∞. (1.5)
Here λ ∈ R d .

The maximum likelihood estimator we introduce as usual by the equation

V ( θn , X n ) = sup ϑ∈Θ V (ϑ, X n ). (1.6)
If this equation has more than one solution then we can take any of them as the MLE.

We introduce as well the log-likelihood ratio function

L(ϑ, X n ) = ln π * (ϑ, X 0 ) + n j=1 ln π(ϑ, X j-1 , X j ). (1.7)
It is known that under the regularity conditions the MLE has the following properties 1. It is consistent (here and throughout this work consistent means consistent in probability): θn -→ ϑ.

Asymptotically normal:

√ n( θnϑ) =⇒ N (0, I(ϑ) -1 ), (1.8) where I(ϑ) is the Fisher information matrix (defined above).

3.

Asymptotically efficient, i.e., it satisfies the relation: for all ϑ 0 ∈ Θ lim

δ→0 lim n→∞ sup |ϑ-ϑ 0 |<δ E ϑ W √ n θn -ϑ = EW I(ϑ 0 ) -1/2 ζ .
(1.9)

The same time for all estimators ϑ n the following Hajek-Le Cam's type lower bound

lim δ→0 lim n→∞ sup |ϑ-ϑ 0 |<δ E ϑ W √ n (ϑ n -ϑ) ≥ EW I(ϑ 0 ) -1/2 ζ (1.10)
holds [START_REF] Veretenikov | Parametric and Non-Parametric Estimation of Markov Chains[END_REF].

Here W (u) = |u| p , u ∈ R d with p > 0 is a loss function (see, e.g., [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF]) and ζ is a Gaussian vector ζ ∼ N (0, J), J is a unit d × d matrix.

The proofs of these properties can be found in [START_REF] Ogata | The weak convergence of the likelihood ratio random fields for Markov observations[END_REF](properties 1 and 2) and in [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] (property 3) in the one-dimensional case (d = 1). See [START_REF] Taniguchi | Asymptotic Theory of Statistical Inference for Time Series[END_REF] for the discussions of related works. Note that there one can find the study of the properties of another well studied estimator called conditional least square estimator.

Our goal is to show that the Bayesian estimator has the same asymptotic properties.

The second estimator studied in this work is the estimator of the method of moments (EMM) θn . Recall that these estimators (under regularity conditions) are consistent and asymptotically normal, but their limit covariance function is different of the inverse Fisher information matrix and therefore these estimators are not asymptotically efficient.

Nevertheless the study of these estimators can be interesting because the construction of them can be more simple than that of the MLE or BE. Moreover, we show in the forthcoming work [START_REF] Kutoyants | On multi-step MLE-process for Markov sequences[END_REF] that using these estimators and multi-step procedure it is possible to obtain estimators asymptotically equivalent to MLE and BE (see [START_REF] Kutoyants | On multi-step MLE-processes for ergodic diffusion[END_REF] where this construction is discussed in details for some models of stochastic processes). This is the first part of the study devoted to the construction of multi-step MLE-processes for Markov sequences [START_REF] Kutoyants | On multi-step MLE-process for Markov sequences[END_REF]. The studied here two estimators are supposed to be used in the next work as preliminary estimators. The goal is to propose an estimator-process ϑ ⋆ j,n , j = N, . . . , n, which can be easily calculated for all N ≤ j ≤ n with N = n δ , δ < 1 and which has asymptotically optimal properties.

Motivation

Suppose we have some discrete-time random process (in economics, in medicine, etc.) that we can describe with a well-known autoregressive model (AR) :

AR(1) : X j+1 = ϑX j + ε j+1 , j = 1, 2, . . . or AR(p) : X j+1 = p l=1 ϑ l X j-l + ε j+1 , j = 1, 2, . . . .
Here ϑ l are the parameters of the model and ε j is white Gaussian noise, i. e.,

Eε j = 0, Eε 2 j = σ 2 , Eε j ε i = 0, i = j .
There are many ways to estimate these parameters. Recall some results of the methods of maximum likelihood estimator, Bayes estimator and estimator method of moments.

Maximum Likelihood Estimator

Let us denote θn the maximum likelihood estimator of the true value ϑ. Recall the definition of this estimator

V ( θn , X n ) = sup ϑ∈Θ V (ϑ, X n ).
(1.11)

If ε j are i.i.d. r.v.'s N (0, σ 2 ) and we have the AR(1) model then the MLE is easy to calculate:

θn = n-1 j=1 X j+1 X j n-1 j=1 X 2 j .
The properties of MLE follow from the representation:

√ n( θn -ϑ) = 1 √ n n-1 j=1 X j ε j+1 1 n n-1 j=1 X 2 j =⇒ N 0, 1 -ϑ 2 .
This limit follows from the law of large numbers

1 n n-1 j=1 X 2 j -→ E ϑ X 2 1 = σ 2 1 -ϑ 2
and from the Central Limit Theorem

1 √ n n-1 j=1 X j ε j+1 =⇒ N 0, σ 4 1 -ϑ 2 .
The Fisher information in this problem is

I (ϑ) = 1 1 -ϑ 2
and therefore the MLE is asymptotically efficient.

Hajek-Le Cam's lower bound

The asymptotic efficiency is understood in the following way. For all estimators ϑ n and all ϑ 0 ∈ Θ we have

lim δ→0 lim n→∞ sup |ϑ-ϑ 0 |<δ E ϑ W √ n (ϑ n -ϑ) ≥ EW I(ϑ 0 ) -1/2 ζ . (1.12)
This is Hajek-Le Cam's lower bound on the risks of all estimators, which is valid under regularity conditions for any statistical model. Here W (u) is some loss function and ζ ∈ N (0, J).

It is known that for the AR(1) model with |ϑ| < 1 the MLE θn is asymptotically efficient, i.e., it satisfies the relation: for all ϑ 0 ∈ Θ lim

δ→0 lim n→∞ sup |ϑ-ϑ 0 |<δ E ϑ W √ n θn -ϑ = EW I(ϑ 0 ) -1/2 ζ .

Bayes Estimator

In our work we will study the BE that can be calculated much easier than the MLE in some cases. Note that in the regular cases the asymptotic properties of the BE and MLE are equivalent [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF].

Suppose that the unknown parameter ϑ is a random vector with the density a priory p (ϑ) , ϑ ∈ Θ.

Then the BE θn is calculated as follows .13) This estimator under regularity conditions is consistent, asymptotically normal

θn = Θ ϑp (ϑ) V (ϑ, X n ) dϑ Θ p (ϑ) V (ϑ, X n ) dϑ . ( 1
√ n θn -ϑ =⇒ N 0, I (ϑ) -1
and asymptotically efficient [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF].

Suppose that we observe the AR(1) model, where the unknown parameter ϑ ∈ [α, β] and -1 < α < β < 1 is a r. v. with the prior density p (ϑ) , ϑ ∈ [α, β]. From (1.13), the BE θn has the following representation:

θn = β α ϑp (ϑ) exp -1 2σ 2 n j=1 (X j -ϑX j-1 ) 2 dϑ β α p (ϑ) exp -1 2σ 2 n j=1 (X j -ϑX j-1 ) 2 dϑ . We have √ n θn -ϑ =⇒ N 0, 1 -ϑ 2 .

Estimator of the Method of Moments

Let us consider the same example as previously. Suppose that we have the observations X n of AR (1). Then by the law of large numbers

1 n n j=1 X 2 j -→ σ 2 1 -ϑ 2 .
Hence the estimator of the method of moments is

θn = 1 - σ 2 1 n n j=1 X 2 j + -→ ϑ.
Here A + = max (A, 0). Usually the limit variance of the EMM is greater than that of the MLE or BE, but in some problems the calculation of this estimator can be easier and its use by this reason can be preferable.

Recall the definition of EMM. Suppose that we have some autoregressive time series X j+1 = S (ϑ, X j ) + ε j+1 , j = 0, 1, 2, . . . with invariant density function π * (ϑ, x).

Let us take such function q (x) that the limit function m (ϑ) defined by the relation

1 n n j=1 q (X j ) -→ m (ϑ) = q (x) π * (ϑ, x) dx
is strictly monotone, i.e., the equation m (ϑ) = t has a unique solution ϑ = h (t). Then we define the EMM

θn = h 1 n n j=1 q (X j ) -→ h (m (ϑ)) = ϑ.
This estimator is asymptotically normal

√ n θn -ϑ =⇒ N (0, D (ϑ))
with some covariant matrix D (ϑ).

Bayes estimator

Suppose that we observe the time series (1.1), where the unknown parameter ϑ ∈ Θ is a random vector with the prior density p (ϑ) , ϑ ∈ Θ. The function p (•) is continuous, bounded and positive. We are interested by the behavior of Bayes estimator for the quadratic loss function, which has the following representation:

θn = Θ ϑp (ϑ) V (ϑ, X n ) dϑ Θ p (ϑ) V (ϑ, X n ) dϑ .
Recall that as usual in the regular cases the asymptotic properties of the BE are equivalent to the properties of the MLE [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF].

Theorem 1 Let the conditions of regularity be fulfilled. Then the BE θn is consistent, asymptotically normal √ n( θnϑ 0 ) =⇒ N (0, I(ϑ 0 ) -1 ) (1.14)

and asymptotically efficient for the polynomial loss functions.

Proof. We have to check the conditions H1-H4 of the Theorem 3.2.1 in [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF], but first we recall the main steps of the proof. Note that we follow the main steps of the proof given in [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF], where just changed the dimension of the parameter ϑ.

Let us introduce the normalized likelihood ratio function

Z n (u) = V (ϑ 0 + u √ n , X n ) V (ϑ 0 , X n ) , u ∈ U n , (1.15)
where the set

U n = u : ϑ = ϑ 0 + u √ n ∈ Θ .
Note that for the BE we have the relations (below

ϑ u = ϑ 0 + u √ n ) θn = Un ϑ 0 + u √ n p (ϑ u ) V ϑ 0 + u √ n , X n du Un p (ϑ u ) V ϑ 0 + u √ n , X n du = ϑ 0 + 1 √ n Un u p (ϑ u ) V ϑ 0 + u √ n ,X n V (ϑ 0 ,X n ) du Un p (ϑ u ) V ϑ 0 + u √ n ,X n V (ϑ 0 ,X n ) du = ϑ 0 + 1 √ n Un u p (ϑ u ) Z n (u) du Un p (ϑ u ) Z n (u) du . Hence √ n θn -ϑ 0 = Un u p (ϑ u ) Z n (u) du Un p (ϑ u ) Z n (u) du .
Let n → ∞, then from continuity of the function p (ϑ u ) at the point ϑ 0 it follows that p (ϑ u ) → p (ϑ 0 ). Suppose that we already proved that the process Z n (u) , u ∈ U n converges in distribution to the random process

Z (u) = exp u, ∆ (ϑ 0 ) - 1 2 u T I(ϑ 0 )u , u ∈ R d ,
where the vector ∆ (ϑ 0 ) ∼ N (0, I(ϑ 0 )) (we denote •, • the scalar product in R d and • is the related norm) and that we can justify the convergence of the corresponding integrals.

Then

√ n θn -ϑ 0 =⇒ R d u Z(u) du R d Z(u) du . Further, denote v = I(ϑ 0 ) 1/2 u and ∆ = I(ϑ 0 ) -1/2 ∆ (ϑ 0 ).
Using these notations we can write

u, ∆ (ϑ 0 ) - 1 2 u T I(ϑ 0 )u = v, ∆ - |v| 2 2 = - 1 2 v -∆ 2 + 1 2 ∆ 2 and R d u Z(u) du = I(ϑ 0 ) -1 ∆ (ϑ 0 ) R d Z(u) du + I(ϑ 0 ) -1/2 R d v -∆ exp - 1 2 v -∆ 2 dv e - ∆ 2 2 = I(ϑ 0 ) -1 ∆ (ϑ 0 ) R d Z(u) du. Hence √ n θn -ϑ 0 =⇒ I(ϑ 0 ) -1 ∆ (ϑ 0 ) ∼ N 0, I(ϑ 0 ) -1
and we obtain the asymptotic normality of the estimator. This convergence together with the uniform integrability of M n provides the convergence of moments.

For example, if we show that for some p > 0 sup

ϑ 0 ∈Θ E ϑ 0 √ n θn -ϑ 0 p ′ < C. (1.16)
Then for any p < p ′ we have the convergence of the moments

lim n→∞ E ϑ 0 √ n θn -ϑ 0 p -→ E ϑ 0 I(ϑ 0 ) -1 ∆ (ϑ 0 ) p .
To verify the convergence

M n = Un u p (ϑ u ) Z n (u) du Un p (ϑ u ) Z n (u) du =⇒ R d u Z(u) du R d Z(u) du
we have to prove

Un u p (ϑ u ) Z n (u) du =⇒ p (ϑ 0 ) R d u Z(u) du, Un p (ϑ u ) Z n (u) du =⇒ p (ϑ 0 ) R d Z(u) du
and for some m ≥ p ′ + 2 the estimate

P ϑ 0 M n ≥ 1 N m ≤ C N m .
(1.17)

As it was shown by Ibragimov and Hasminskii [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF], the conditions H1 -H4 allow to justify these limits and the estimate (1.17).

The next lemma provides the verification of H1.

Lemma 1 Let the conditions of regularity be fulfilled, then we have the representation

Z n (u) = exp u, ∆ n (ϑ 0 , X n ) - 1 2 u T I(ϑ 0 )u + r n , (1.18)
where r n → 0 and

∆ n (ϑ 0 , X n ) = 1 √ n n j=1 l (ϑ 0 , X j-1 , X j ) =⇒ ∆ (ϑ 0 ) ∼ N (0, I(ϑ 0 )) . (1.19)
Proof. The representation (1.18)-(1. [START_REF] Maki | The risk of bloodstream infection in adults with different intravascular devices: a systematic review of 200 published prospective studies[END_REF]) is known as local asymptotical normality (LAN) of the family of measures corresponding to this model of observations (see, e.g., [START_REF] Roussas | Asymptotic inference in Markov processes[END_REF], [START_REF] Roussas | Contiguity of Probability Measures: Some Applications in Statistics[END_REF], [START_REF] Kushnir | Asymptotically optimal tests for a regression problem of testing hypotheses[END_REF], [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF]).

Below we use the expansions ln (1 + x) = x -x 2 2 + O (x 3 ) and

π(ϑ 0 + u √ n , X j-1 , X j ) = π(ϑ 0 , X j-1 , X j ) + 1 √ n u, π( θ, X j-1 , X j ) .
We have

ln Z n (u) = n j=1 ln π(ϑ 0 + u √ n , X j-1 , X j ) π(ϑ 0 , X j-1 , X j ) = n j=1 ln π(ϑ 0 , X j-1 , X j ) + 1 √ n u, π( θ, X j-1 , X j ) π(ϑ 0 , X j-1 , X j ) = n j=1 ln 1 + u, π( θ, X j-1 , X j ) π(ϑ 0 , X j-1 , X j ) 1 √ n = u, 1 √ n n j=1 π( θ, X j-1 , X j ) π(ϑ 0 , X j-1 , X j ) -u T n j=1 π( θ, X j-1 , X j ) π( θ, X j-1 , X j ) 2nπ(ϑ 0 , X j-1 , X j ) T u + o(1) = u, 1 √ n n j=1 l(ϑ 0 , X j-1 , X j ) -u T 1 2n n j=1 l(ϑ 0 , X j-1 , X j ) l(ϑ 0 , X j-1 , X j ) T u + o(1) = u, ∆ n (ϑ 0 , X n ) -u T 1 2n n j=1 l(ϑ 0 , X j-1 , X j ) l(ϑ 0 , X j-1 , X j ) T u + o(1).
Here and in the sequence o(1) means convergence to zero in probability. By the central limit theorem

∆ n (ϑ 0 , X n ) = 1 √ n n j=1 l(ϑ 0 , X j-1 , X j ) =⇒ ∆ (ϑ 0 ) ∼ N (0, I(ϑ 0 ))
and by the law of large numbers

1 n n j=1 l(ϑ 0 , X j-1 , X j ) l(ϑ 0 , X j-1 , X j ) T -→ I(ϑ 0 ).
Lemma 2 Let the conditions of regularity be fulfilled. Then there exists the constant C > 0 such that

E ϑ 0 |Z n (u 2 ) 1/2 -Z n (u 1 ) 1/2 | 2 ≤ C|u 2 -u 1 | 2 .
Proof. Following Lemma 3.1.1 in [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF] we can write (below

u s = u 1 + s(u 2 -u 1 )) that Z n (u 2 ) 1/2 -Z n (u 1 ) 1/2 = 1 2 √ n 1 0 Z 1/2 n (u s ) n j=1 l ϑ 0 + u s √ n , X j-1 , X j , (u 2 -u 1 ) ds.
Hence

E ϑ 0 |Z n (u 2 ) 1/2 -Z n (u 1 ) 1/2 | 2 ≤ 1 4n 1 0 E ϑ 0 Z n (u s ) n j=1 l ϑ 0 + u s √ n , X j-1 , X j , (u 2 -u 1 ) 2 ds ≤ 1 4n 1 0 E ϑu s n j=1 l ϑ 0 + u s √ n , X j-1 , X j , (u 2 -u 1 ) 2 ds = n i,j=1 (u 2 -u 1 ) 4n 1 0 E ϑu s l (ϑ us , X j-1 , X j ) l (ϑ us , X i-1 , X i ) T ds (u 2 -u 1 ) = 1 4 (u 2 -u 1 ) T 1 0 I (ϑ us ) ds (u 2 -u 1 ) ≤ C u 2 -u 1 2 ,
because the information matrix satisfies (1.4).

Lemma 3 Let the condition of regularity be fulfilled. Then for any q > 0 there exist constant B q such that

E ϑ 0 (Z n (u) 1/2 ) ≤ B q |u| q .
(1.20)

Proof. First we follow the proof of the Theorem 1 in [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] and write

E ϑ 0 ,x Z n (u) 1 2 ≤ exp -c |u| 2 + C k (1 + n) k . We have u = √ n (ϑ -ϑ 0 ). Hence |u| ≤ √ nD (Θ) , D(Θ) = sup ϑ 1 ,ϑ 2 ∈Θ |ϑ 1 -ϑ 2 | . Further n ≥ |u| 2 D (Θ) 2k , C k (1 + n) k ≤ C k D (Θ) 2 |u| 2k = B q |u| q .
with some constant B q > 0.

Therefore the conditions H1-H4 of the Theorem 3.2.1 in [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF] are fulfilled and according to this theorem the BE has the mentioned in Theorem 1 properties.

Method of moments

The properties of the estimators constructed by the method of moments are wellknown. Nevertheless we recall here the conditions of the asymptotic normality and convergence of moments because in the forthcoming work these estimators will be used as preliminary in the construction of the asymptotically efficient estimators using the one-step and two-step MLE procedures [START_REF] Kutoyants | On multi-step MLE-process for Markov sequences[END_REF].

Let q (x) be a such vector-function that the function m(ϑ) = E ϑ q(X) = g (x) π * (ϑ, x) dx admits a unique solution for all ϑ ∈ Θ of the equation

m(ϑ) = t, t ∈ R d .
To have this property we introduce the condition of Identifiability: For any ν > 0 and any ϑ 0 we have

κ (ν) = inf |ϑ-ϑ 0 |>ν |m (ϑ) -m (ϑ 0 )| > 0.
Note that if this condition is not fulfilled then the consistent estimation of the parameter ϑ with such q (•) is impossible and we have to seek another function q (•). Indeed, if for some ν > 0 we have κ (ν) = 0 then it follows that there exists at least one ϑ 1 such that m (ϑ 1 ) = m (ϑ 0 ) and ϑ 1 = ϑ 0 . For example, the condition of Identifiability is fulfilled in one-dimensional case d = 1 if the function m (ϑ) is strictly monotone.

Moreover we suppose that this solution ϑ can be written as ϑ = h (t), where h (•) ∈ R d is some smooth vector-function.

Recall that π * (ϑ, x) is the one-dimensional invariant density and we denote its distribution function as Π * (ϑ, x).

The solution ϑ = m -1 (t) = h(t) we write as

ϑ = h q (x) dΠ * (ϑ, x) .
Let us introduce the empirical distribution function

Π * n (x) = 1 n n j=1 1I {X j <x} .
Now the estimator of the method of moments (EMM) θn we obtain by substitution Π * n (•) on the place of Π * (ϑ, x). This yields

θn = h q (x) d Π * n (x) = h 1 n n j=1 q (X j ) .
The calculation of EMM can be simpler than that of the MLE or BE, but its limit covariance

B (ϑ) in √ n θn -ϑ =⇒ N (0, B (ϑ))
is usually greater than that of the MLE:

B (ϑ) ≥ I (ϑ) -1 . This means that B (ϑ) - I (ϑ) -1 is positive definite.
Remind that by the central limit theorem for stationary strongly mixing sequences ξ j = q (X j ) -E ϑ q (X j ) we have

1 √ n n j=1 ξ j =⇒ N (0, D (ϑ)) ,
where the covariance matrix

D (ϑ) = E ϑ ξ 0 ξ T 0 + 2 ∞ j=1 E ϑ ξ 0 ξ T j .
Introduce as well the matrix

M (ϑ) = ṁ (ϑ) ṁ (ϑ) T
and suppose that it is uniformly nondegenerate

inf ϑ∈Θ inf |λ|=1 λ T M (ϑ) λ > 0. (1.21)
To verify the convergence of moments we need one technical lemma.

Lemma 4 Under regularity conditions for any L > 0 and m > there exist a constant C > 0 such that

P ϑ 0 √ n θn -ϑ 0 > L ≤ C L m . (1.22)
Proof. We can write

P ϑ 0 √ n θn -ϑ 0 > L = P ϑ 0 inf |ϑ-ϑ 0 |<Ln -1/2 | mn -m (ϑ)| > inf |ϑ-ϑ 0 |≥Ln -1/2 | mn -m (ϑ)| ≤ P ϑ 0 inf |ϑ-ϑ 0 |<Ln -1/2 (| mn -m (ϑ 0 )| + |m (ϑ 0 ) -m (ϑ)|) > inf |ϑ-ϑ 0 |≥Ln -1/2 (|m (ϑ 0 ) -m (ϑ)| -| mn -m (ϑ 0 )|) ≤ P ϑ 0 | mn -m (ϑ 0 )| > κ Ln -1/2 .
For the function |m (ϑ)m (ϑ 0 )| we can obtain two estimates. The first local estimate is

|m (ϑ) -m (ϑ 0 )| 2 = ṁ( θ), (ϑ -ϑ 0 ) 2 = (ϑ -ϑ 0 ) T ṁ( θ) ṁ( θ) T (ϑ -ϑ 0 ) ≥ κ |ϑ -ϑ 0 | 2 for |ϑ -ϑ 0 | ≤ ν sufficiently small ν. Remind that the matrix M ( θ) = ṁ( θ) ṁ( θ) T is uniformly nondegenerate (1.21). Outside of the circle |ϑ -ϑ 0 | ≤ ν we have |m (ϑ) -m (ϑ 0 )| ≥ κ (ν) ≥ κ (ν) |ϑ -ϑ 0 | D (ϑ) ,
where we denoted D (ϑ) the diameter of the set Θ.

Therefore we have the following estimate for all ϑ ∈ Θ

|m (ϑ) -m (ϑ 0 )| ≥ κ |ϑ -ϑ 0 | (1.23)
with some constant κ > 0.

This estimate allows us to write

P ϑ 0 | mn -m (ϑ 0 )| > κ Ln -1/2 ≤ P ϑ 0 | mn -m (ϑ 0 )| > κ Ln -1/2 ≤ κ-m n m/2 E ϑ 0 | mn -m (ϑ 0 )| m L -m ≤ C L m ,
because we suppose that the moments converge (1.3).

Let us denote

B (ϑ) = h ′ (m (ϑ)) T D (ϑ) T D (ϑ) h ′ (m (ϑ)) .
Theorem 2 Let the conditions of regularity be fulfilled, then the EMM θn is consistent, asymptotically normal

√ n θn -ϑ 0 =⇒ N (0, B (ϑ 0 )) (1.24)
and we have the convergence of moments: for all p > 0

lim n→∞ n p 2 E ϑ 0 θn -ϑ 0 p = E ϑ 0 |ζ (ϑ 0 )| p , (1.25)
where ζ ∼ N (0, B (ϑ 0 )).

Proof. The consistency follows immediately from the estimate (1.22) as follows:

for any ν > 0 we have

P ϑ 0 θn -ϑ 0 > ν ≤ C ν m n m/2 -→ 0.
From the law of large numbers and continuity of the function h (•) we obtain the consistency of this estimator

mn = 1 n n j=1 q (X j ) -→ E ϑ q(X), θn -→ h (m (ϑ)) = ϑ.
To verify the asymptotic normality of this estimator first note that by the central limit theorem for Markov sequences we have

1 √ n n j=1 [q (X j ) -E ϑ q(X)] =⇒ N (0, D (ϑ)) .
Hence, if we suppose that the function h (•) is continuously differentiable at the point m (ϑ) then by Taylor's expansion we obtain

θn = h m (ϑ) + 1 √ n 1 √ n n j=1 [q (X j ) -E ϑ q(X)] = h m (ϑ) + η n √ n = h (m (ϑ)) + h ′ (m (ϑ)) η n √ n + o (1) = ϑ + h ′ (m (ϑ)) η n √ n + o (1)
.

Here h ′ (m (ϑ)) is d × d matrix. Therefore √ n θn -ϑ = h ′ (m (ϑ)) η n + o (1)
and

√ n θn -ϑ =⇒ N (0, B (ϑ)) .
To prove the convergence of moments (1.25) it is sufficient to verify that the family of random variables is uniformly integrable: for any p > 0 there exists a constant C > 0 such that

n p/2 E ϑ 0 θn -ϑ 0 p ≤ C. (1.26)
Let us denote

G n (u) = P ϑ 0 √ n θn -ϑ 0 < u
and take m = p + 1. We have

n p/2 E ϑ 0 θn -ϑ 0 p = ∞ 0 u p dG n (u) = L 0 u p dG n (u) - ∞ L u p d [1 -G n (u)] ≤ C 1 + L p + p ∞ L u p-1 [1 -G n (u)] du ≤ C 1 + L p + ∞ L u p-1 C u m du ≤ C 1 + L p + C L ≤ C.
Hence the condition (1.26) is fulfilled and we have the convergence of moments.

Examples

In this section we introduce three different examples and we studied the behavior of the parameter estimators in each case. In the next chapter examples 2 and 3 will be used to illustrate the one and two-step MLE-processes.

Example 1

This first example was formulated in the work [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF].

X n+1 = X n -(2 + ϑ) sgn(X n ) ln(1 + |X n |) 1 + |X n | + ε n+1 , ϑ ∈ (-1, +1), (1.27)
ε n ∼ N (0, 1), i.i.d.
This example is one for which the general theorem 2 (MLE asymptotic minimax efficiency) of [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] works.

We can not obtain analytical expression for the invariant density that is why we use the kernel type density estimation. Let us estimate the invariant density of observations of Markov sequence X n = (X 1 , X 2 , . . . , X n ) with the help of kernel density estimation which is a non-parametric method.

fn (x) = 1 nh n n j=1 K( X j -x h n ),
where h n is the step and K is the symmetric kernel that satisfy the following conditions:

K(x) ≥ 0, R K(x) dx = 1, R xK(x) dx = 0, R x 2 K(x) dx < ∞.
For the estimation of fn (x) we need to chose the smoothing kernel to be used. This may be, for example, gaussian, rectangular, triangular, epanechnikov, etc.

The classic example of K is the gaussian kernel:

K(x) = 1 √ 2π e -x 2 2 ✶ [-∞;+∞] .
We use the function density to perform kernel density estimation in R. We can show that for the large samples there is no difference of the type of smoothing

The next step is to calculate the maximum-likelihood estimator (MLE).

Let us denote from equation (1.27) Considering (1.28) we can write the equation (1.27) in the following form

A(X j-1 ) = sgn(X j-1 ) ln(1 + |X j-1 |) 1 + |X j-1 | (1.28) -6 -4 - 2
X j = X j-1 -(2 + ϑ)A(X j-1 ) + ε j , ϑ ∈ (-1, +1). (1.29)
Let us consider the likelihood function for our example

V (ϑ, X n ) = n j=1 f (ϑ, X j-1 , X j ), ϑ ∈ (-1, +1),
where Gaussian kernel theta=-0.9

f (ϑ, X j-1 , X j ) = 1 √ 2π e -1 -6 -4 - 2

Figure 1.3: Kernel density estimation for different theta

And the log-likelihood ratio function is

L n (ϑ, X n ) = n j=1 - 1 2 [X j -(X j-1 -(2 + ϑ)A(X j-1 )] 2 = n j=1 ℓ(ϑ, X j-1 , X j ).
The partial derivative of the log-likelihood ratio function with respect to ϑ is

∂L ∂ϑ = n j=1 l(ϑ, X j-1 , X j ) = 1 2 2 n j=1 [X j -(X j-1 -(2 + ϑ)A(X j-1 ))]A(X j-1 ) = n j=1 [X j -X j-1 -2A(X j-1 ) -ϑA(X j-1 )]A(X j-1 ).
We need to find ϑ for which

∂L ∂ϑ = 0. n j=1 [X j -X j-1 -2A(X j-1 )]A(X j-1 ) = ϑ n j=1 A(X j-1 ) 2 . So the maximum-likelihood estimator θMLE = n j=1 [X j -X j-1 -2A(X j-1 )]A(X j-1 ) n j=1 A(X j-1 ) 2
.

With the help of R we draw the distribution of log-likelihood ratio function L n (ϑ, X n ) and we find its maximum. We compare the results for n = 1000 and n = 100000.

On the Figures 1.4 and 1.5 we present the distribution of log-likelihood ratio function L n (ϑ, X n ) where n = 1000 and n = 100000. The red line is the true value of the parameter and the dotted line is the maximum-likelihood estimator for this parameter.

For the large samples we observe that the estimation of the parameter θMLE is very close to it true value.

Example 2

In our first example taking from the work of Varakin A.B., Veretennikov A.Yu. (2002) [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] we have shown that θMLE has an explicit expression. That's why we Therefore let us introduce another example for which the problem of MLE calculation is evident.

We propose to study the Markov sequence X n = (X 1 , X 2 , . . . , X n ) defined as

X n+1 = (X n ) 2 1 + ϑ | X n | + ε n+1 , ϑ ∈ (2, 5), (1.30)
ε n ∼ N (0, 1), i.i.d.
This example is one for which the general theorem 2 (MLE asymptotic minimax efficiency) of [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] works.

Let us estimate the invariant density of observations of Markov sequence X n = (X 1 , X 2 , . . . , X n ) with the help of kernel density estimation we have already done in previous example. where h n is the step and K is the symmetric kernel that satisfy the conditions indicated in example 1.

fn (x) = 1 nh n n j=1 K( X j -x h n ), - 1
We use the function density to perform kernel density estimation in R. We can show that for the large samples there is no difference of the type of smoothing kernel to be used.

On the Figure 1.6 we draw the density estimation for the sample of n = 10000 observations. As in the previous example, the next step is to calculate the maximum-likelihood estimator (MLE).

Let f (ϑ, X j-1 , X j ) be the density function of Markov sequence X n defined in (1.30)

f (ϑ, X j-1 , X j ) = 1 √ 2π e -1 2 X j - (X j-1 ) 2 1+ϑ|X j-1 | 2 . (1.31)
Let us consider the likelihood function for our example And the log-likelihood ratio function is

V (ϑ, X n ) = n j=1 f (ϑ, X j-1 , X j ), ϑ ∈ (2, 5).
L n (ϑ, X n ) = n j=1 - 1 2 ln 2π - 1 2 X j - (X j-1 ) 2 1 + ϑ | X j-1 | 2 = n j=1 ℓ(ϑ, X j-1 , X j ).
The partial derivative of the log-likelihood ratio function with respect to ϑ is

∂L ∂ϑ = n j=1 l(ϑ, X j-1 , X j ) = n j=1 | X j-1 | 3 (1 + ϑ | X j-1 |) 2 -X j + (X j-1 ) 2 1 + ϑ | X j-1 | .
We need to find ϑ for which

∂L ∂ϑ = 0.
With the help of R we draw the distribution of log-likelihood ratio function L n (ϑ, X n ) and we find its maximum. We compare the results for n = 1000 and n = 10000 .

On the Figure 1.7 we present the distribution of log-likelihood ratio function L n (ϑ, X n ) where n = 1000. The red line is the true value of the parameter and the dotted line is the maximum-likelihood estimator for this parameter. On the Figure 1.8 we present the distribution of log-likelihood ratio function L n (ϑ, X n ) where n = 10000. The red line is the true value of the parameter and the dotted line is the maximum-likelihood estimator for this parameter.

For the large samples we observe that the estimation of the parameter θMLE is very close to it true value.

In the following section we will use this example for constructing the one-step MLE-process. Let us consider another example for which there is no explicit MLE expression.

Example 3

Let us consider the following model of observations

X j+1 = X j + 3 ϑ -X j 1 + (X j -ϑ) 2 + ε j+1 , j = 0, 1, . . . , n -1,
where (ε j ) j≥1 are i.i.d. standard Gaussian random variables. The unknown parameter ϑ ∈ Θ = (-1, 1). The initial value X 0 is given.

The time series (X j ) j≥1 has ergodic properties with the density of invariant distribution presented on the 1.9. The vertical line represents the true value of parameter.

It is easy to see that ϑ is the shift parameter. Indeed, we have The invariant distribution of Y j does not depend on ϑ.

X j+1 -ϑ = X j -ϑ + 3 ϑ -X j 1 + (X j -ϑ) 2 + ε j+1 and if we put Y j = X j -ϑ then Y j+1 = Y j -3 Y j 1 + Y 2 j + ε j+1 .
The Fisher information does not depend on ϑ because ϑ is a shift parameter, i.e., I (ϑ) = I.

We have

ℓ (ϑ, x, x ′ ) = ln π(ϑ, x, x ′ ) = - 1 2 x ′ -x -3 ϑ -x 1 + (ϑ -x) 2 2 - 1 2 ln 2π and l (ϑ, x, x ′ ) = 3 x ′ -x -3 ϑ -x 1 + (ϑ -x) 2 1 -(ϑ -x) 2 (1 + (ϑ -x) 2 ) 2 .
Therefore the Fisher information has the representation

I = 9 E ϑ   X j+1 -X j -3 ϑ -X j 1 + (ϑ -X j ) 2 2 × 1 -(ϑ -X j ) 2 (1 + (ϑ -X j ) 2 ) 2 2   = 9 E ϑ (ε j+1 ) 2 × E ϑ 1 -(ϑ -X j ) 2 (1 + (ϑ -X j ) 2 ) 2 2 = 9 E ϑ=0 1 -ζ 2 0 (1 + ζ 2 0 ) 2 2
.

It can be approximated as follows

I n = 1 n n j=1 1 -X 2 j 1 + X 2 j 2 2
.

Numerical simulations with n = 1000 gives us the value I 1000 = 2.134 and I 1000 -1 = 0.4686.

Before studying the limit variances of estimator of the method of moments and Bayes estimator let us represent on the Figure 1.10 the three estimators of parameter ϑ for n = 1000. Here we used continuous line for drawing the true value of our parameter which is ϑ 0 = 0.5, dash line for Bayes estimator θ = 0.4765, dotted line for estimator of the method of moments θ = 0.4163 and dash-dotted line for maximum-likelihood estimator θ = 0.5146. Both BE and MLE are quite close to the true value ϑ 0 and the EMM gives us the worst estimation of the parameter.

Bayesian estimator.

Let us take the uniform prior distribution p (ϑ) = 1 2 1I {|ϑ|≤1} and calculate the Bayes estimator

θn = 1 -1 ϑ V (ϑ, X n ) dϑ 1 -1 V (ϑ, X n ) dϑ . The true value is ϑ = 0, 5. Denote ũn = √ n θn -ϑ . We know that √ n θn -ϑ =⇒ N 0, I -1 .
Numerical simulations with n = 1000 repeated 1000 times give us the data ũ1000,l = √ 1000 θ1000,lϑ , l = 1, . . . , 1000 and allow to calculate the empirical variance 1 1000 1000 l=1 (ũ 1000,l ) 2 = 0.482 which is in accordance with the waited value I -1 = 0.4686.

Estimator of the method of moments.

As ϑ is the shift parameter we can take the estimator of the method of moments as follows

θn = 1 n n j=1 X j .
The numerical simulations of ū1000,l = √ 1000 θ1000,lϑ , l = 1, . . . , 1000 yields the value

B n = 1 1000 1000 l=1 (ū 1000,l ) 2 = 1.208.
We see that the limit variance of the EMM is greater than that of the BE. Note that both estimators are used in the work [START_REF] Kutoyants | On multi-step MLE-process for Markov sequences[END_REF] as preliminary for the construction of the asymptotically efficient estimator-processes.

Chapter 2

On Multi-step MLE-process for Markov Sequences

Introduction

This chapter is devoted to the problem of finite-dimensional parameter estimation in the case of observations of Markov sequence in the asymptotics of large samples. Suppose we observe the process X n = (X 0 , X 1 , X 2 , . . . , X n ). For simplicity of exposition we take as a model of observations a nonlinear time series satisfying the relation

X j = S (ϑ, X j-1 ) + ε j , j = 1, 2, . . . (2.1)
and the initial value X 0 is given too. The random variables (ε j ) j≥1 are i.i.d. with some known smooth density function g (x). The function S (ϑ, x) is supposed to be known and smooth with respect to ϑ.

Our goal is to construct a sequence (we say process) of estimators ϑ ⋆ n = (ϑ k,n ), where k = N + 1, N + 2, . . . , n and N ≪ n. By the first N + 1 observations X N = (X 0 , X 1 , . . . , X N ) we estimate the parameter ϑ and the obtained preliminary estimator θN we use in the construction of the estimator process ϑ ⋆ n . This construction is based on the modification of the well-known one-step maximum likelihood estimator (MLE) procedure introduced by Le Cam in 1956 [START_REF] Cam | On the asymptotic theory of estimation and testing hypotheses[END_REF]. In the proofs we follow the similar work [START_REF] Kutoyants | On multi-step MLE-processes for ergodic diffusion[END_REF] devoted to parameter estimation in the case of ergodic diffusion process.

As the initial estimator is constructed by a relatively small number of observations N ∼ n δ with δ < 1 the rate of convergence of the preliminary estimator is

√ N ∼ n δ/2 √ N θN -ϑ ⇒ N (0, D (ϑ))
and we have to improve this rate up to the optimal √ n and to improve the limit variance up to the optimal.

Note that the idea to improve the rate of convergence of preliminary estimator using the Newton-Raphson procedure was realized by Kamatani and Uchida [START_REF] Kamatani | Hybrid multi-step estimators for stochastic differential equations based on sampled data[END_REF] in the different situation. They considered the problem of parameter estimation by the discrete time observations of the diffusion process in the asymptotics of the observations of high frequency , i.e., they supposed that the step of discretization tends to zero.

Another particularity of the presented work is the following. We propose a sequence of estimators, which can be easily calculated and the same time it has the same asymptotic properties as the asymptotically efficient MLE. This means that these estimators are asymptotically normal and that its limit variance is the inverse Fisher information matrix.

The properties of the parameter estimators for nonlinear time series and Markov sequences, of course, are well-known. Let us mention here the works by Roussas [START_REF] Roussas | Asymptotic inference in Markov processes[END_REF], Ogata and Inagaki [START_REF] Ogata | The weak convergence of the likelihood ratio random fields for Markov observations[END_REF], Varakin and Veretennikov [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF]). More about statistical problems for time series can be found in the monographs by Veretennikov [START_REF] Veretenikov | Parametric and Non-Parametric Estimation of Markov Chains[END_REF], Taniguchi and Kakizawa [START_REF] Taniguchi | Asymptotic Theory of Statistical Inference for Time Series[END_REF], Fan and Yao [START_REF] Yao | Nonlinear Time Series: Nonparametric and Parametric Methods[END_REF], and the references therein. Note that we take the time series (2.1) just for simplicity of expositions. The proposed results can be generalized on the more general Markov sequences defined by their transition density if we suppose that this density satisfies to the corresponding regularity conditions.

The process (X j ) j≥0 has a transition density

π (ϑ, x, x ′ ) = g (x ′ -S (ϑ, x)) .
It depends on the parameter θ and defines the probability of reaching the state x ′ after sojourning in the state x. The parameter ϑ takes its values in some open bounded set Θ ⊂ R d .

We suppose that the time series (X j ) j≥1 is geometrically mixing, has invariant distribution with the density function π * (ϑ, x) and for simplicity of exposition we put π 0 (x) = π * (ϑ, x). In this case the process (X j ) j≥0 is stationary.

The construction of the one-step MLE-process in this work is done in two steps. On the first step we estimate the unknown parameter by the observations X N = (X 0 , X 1 , . . . , X N ) on the learning interval j ∈ [0, N ]. As preliminary estimator we can take the MLE, Bayes estimator (BE), estimator of the method of moments (EMM) or any other estimator, which is consistent and asymptotically normal.

Let us recall some of them. The MLE estimator is defined as follows. Introduce the likelihood function

V (ϑ, X n ) = π 0 (X 0 ) n j=1 π(θ, X j-1 , X j ), ϑ ∈ Θ, (2.2)
where π 0 (x) is the density of the initial value X 0 .

The maximum likelihood estimator we introduce as usual by the equation

V ( θn , X n ) = sup ϑ∈Θ V (ϑ, X n ). (2.3)
If this equation has many solutions then we can take any of them as the MLE.

It is known that under the regularity conditions the MLE is consistent, asymptotically normal:

√ n( θn -ϑ) =⇒ N (0, I(ϑ) -1 ), (2.4)
where I(ϑ) is the Fisher information matrix

I(ϑ) = E * ϑ l (ϑ, X 0 , X 1 ) l (ϑ, X 0 , X 1 ) T ,
where ℓ (ϑ, x, x ′ ) = ln π (ϑ, x, x ′ ), the dot means the derivation w.r.t. ϑ and T means the transpose of a matrix. The mathematical expectation here is w.r.t. the invariant measure π * (ϑ, •).

As π (ϑ, x, x ′ ) = g (x ′ -S (ϑ, x)) we can write

I(ϑ) = E * ϑ ġ (X j -S (ϑ, X j-1 )) ġ (X j -S (ϑ, X j-1 )) T = E * ϑ ġ (X j -S (ϑ, X j-1 )) ġ (X j -S (ϑ, X j-1 )) T g (X j -S (ϑ, X j-1 )) 2 = E g ′ (ε) g (ε) 2 E * ϑ Ṡ (ϑ, ξ) Ṡ (ϑ, ξ) T = I g E * ϑ Ṡ (ϑ, ξ) Ṡ (ϑ, ξ) T ,
where we used the equality X j -S (ϑ, X j-1 ) = ε j and denoted

I g = g ′ (x) 2 g (x) dx.
Moreover the MLE is asymptotically efficient. There are several definitions of the asymptotically efficient estimators. One of them is the following : an estimator ϑ * n is called asymptotically efficient if it satisfies the relation: for all ϑ 0 ∈ Θ lim

δ→0 lim n→∞ sup |ϑ-ϑ 0 |<δ E ϑ W √ n (ϑ * n -ϑ) = EW I(ϑ 0 ) -1/2 ζ . (2.5)
Here W (u) , u ∈ R d is a loss function satisfying the usual conditions. Note that it can be bounded, polynomial W (u) = |u| p , u ∈ R d with p > 0 or other (see, e.g., [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF]) and ζ is a Gaussian vector ζ ∼ N (0, J), J is a unit d × d matrix. Remind that for all for all estimators θn the following Hajek-Le Cam's type lower bound

lim δ→0 lim n→∞ sup |ϑ-ϑ 0 |<δ E ϑ W √ n θn -ϑ ≥ EW I(ϑ 0 ) -1/2 ζ (2.6)
holds (see, e.g. [START_REF] Ibragimov | Has'minskii R. Statistical Estimation -Asymptotic Theory[END_REF]). That is why (2.5) indeed defines the asymptotically efficient estimator.

Note that these properties of the MLE were established in several works. We mention here [START_REF] Ogata | The weak convergence of the likelihood ratio random fields for Markov observations[END_REF] and [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] (in the one-dimensional case d = 1).

As preliminary estimator we can use as well the BE. Recall its definition and properties. Suppose that the unknown parameter ϑ ∈ Θ is a random vector with the prior density p (ϑ) , ϑ ∈ Θ. The function p (•) is continuous, bounded and positive.

The Bayes estimator for the quadratic loss function has the following representation:

θn = Θ ϑp (ϑ) V (ϑ, X n ) dϑ Θ p (ϑ) V (ϑ, X n ) dϑ .
This estimator under regularity conditions is consistent, asymptotically normal

√ n( θn -ϑ 0 ) =⇒ N (0, I(ϑ 0 ) -1 ) (2.7)
and asymptotically efficient for the polynomial loss functions. For the proof see [START_REF] Motrunich | On parameter estimation for Markov sequences[END_REF].

Recall also the properties of the estimator of the method of moments. Suppose that the vector-function q (x) ∈ R d is such that the system of equations

m (ϑ) = t, ϑ ∈ Θ,
where

m (ϑ) = E * ϑ q (ξ)
has a unique solution ϑ = ϑ (t).

Introduce the function h (t) inverse to the function m (ϑ), i.e., ϑ = m -1 (t) = h (t). Then the EMM is defined as follows

θn = h 1 n n j=1 q (X j ) .
It is known that under regularity conditions this estimator is consistent, asymptotically normal

√ n θn -ϑ =⇒ N (0, C (ϑ)) ,
where C (ϑ) is some matrix. Moreover the moments of the EMM converge too (see [START_REF] Motrunich | On parameter estimation for Markov sequences[END_REF]).

In this work we use the score-function which can be calculated as follows. We introduce the log-likelihood ratio function

L(ϑ, X n ) = ln π * (ϑ, X 0 ) + n j=1 ln π(ϑ, X j-1 , X j ).
(

The normalized vector score-function

∆ n (ϑ, X n ) = 1 √ n ∂L(ϑ, X n ) ∂ϑ = 1 √ n n j=1
g ′ (X j -S (ϑ, X j-1 )) g (X j -S (ϑ, X j-1 )) Ṡ (ϑ, X j-1 ) .

If we denote the true value ϑ = ϑ 0 , then we have

∆ n (ϑ 0 , X n ) = 1 √ n n j=1 g ′ (ε j ) g (ε j ) Ṡ (ϑ, X j-1 ) .
Note that (i < j)

E ϑ g ′ (ε i ) g (ε i ) g ′ (ε j ) g (ε j ) Ṡ (ϑ, X i-1 ) Ṡ (ϑ, X j-1 ) T = E ϑ g ′ (ε i ) g (ε i ) Ṡ (ϑ, X i-1 ) E ϑ g ′ (ε j ) g (ε j ) Ṡ (ϑ, X j-1 ) T F j-1 = 0, because E ϑ g ′ (ε j ) g (ε j ) Ṡ (ϑ, X j-1 ) T F j-1 = E g ′ (ε j ) g (ε j ) E ϑ Ṡ (ϑ, X j-1 ) T F j-1 and E g ′ (ε j ) g (ε j ) = ∞ -∞ g ′ (x) dx = 0.
Therefore by the central limit theorem

∆ n (ϑ 0 , X n ) =⇒ N (0, I (ϑ 0 )) ,
where the Fisher information matrix

I (ϑ 0 ) = I g E ϑ Ṡ (ϑ, ξ) Ṡ (ϑ, ξ) T .

Main result

Suppose that we have a Markov sequence X n = (X j ) j=0,n with the transition density π (•) depending on some unknown finite-dimensional parameter ϑ ∈ Θ.

The set Θ ⊂ R d is open, bounded.
Our goal is to construct on-line recurrent estimator of this parameter. Therefore we need for each j to have an estimator ϑ * j,n with good properties, i.e., this estimator can be easily calculated and the same time it has to be asymptotically optimal in some sense. We call such sequence of estimators ϑ * j,n , j = 1, . . . , n estimatorprocess.

We propose a construction of such estimator in two steps. We slightly change the statement of the problem. Introduce the learning part X N = (X 0 , X 1 , . . . , X N ) of observations X n = (X 0 , X 1 , . . . , X n ), where N = n δ (N is the integer part of n δ ) and the parameter δ < 1 will be chosen later.

Throughout the entire paper we suppose that the following Regularity conditions are fulfilled.

1. The time series (X j ) j≥0 is geometrically mixing with the density of invariant law π * (ϑ, x) and such that the law of large numbers

1 n n j=1 h (X j ) -→ E ϑ h (ξ) ≡ h (x) π * (ϑ, x) dx
and the central limit theorem

1 √ n n j=1 [h (X j ) -E ϑ h (ξ)] =⇒ N 0, σ (ϑ) 2
hold. Here we suppose that the function h (•) is quadratically integrable w.r.t. the invariant measure and σ (ϑ) 2 < ∞ is the corresponding limit variance.

2.

The preliminary estimator θN is consistent and asymptotically normal

√ N θN -ϑ =⇒ N (0, B (ϑ))
with some covariance matrix B (ϑ).

The function

S (•, •) ∈ C 3 ϑ , the density g (•) > 0 and g (•) ∈ C 3 .
4. The function ℓ (ϑ, x, x ′ ) = ln π (ϑ, x, x ′ ) ∈ C 3 ϑ and its derivatives uniformly on ϑ are majorated by absolutely integrable functions, i.e., (2.9)

sup ϑ∈Θ ∂ i ℓ (ϑ, x, x ′ ) ∂ϑ i ≤ R i (x, x ′ ) , i = 1, 2, 3, where E ϑ |R i (X j-1 , X j )| 2 < C.
Here λ ∈ R d .
Note that as preliminary estimator θN we can take the MLE, the BE or the EMM. All of them have the required properties (under additional regularity conditions, which we do not mention here). The details can be found in [START_REF] Ogata | The weak convergence of the likelihood ratio random fields for Markov observations[END_REF], [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF], [START_REF] Motrunich | On parameter estimation for Markov sequences[END_REF] or any other work describing their properties. The conditions 3-4 allow us differentiate the function ℓ (ϑ, x, x ′ ) with respect to ϑ and by condition 5 these derivatives have bounded moments.

We construct the one-step MLE-process ϑ ⋆ k,n , k = N + 1, . . . , n as follows. Introduce the variable s ∈ (τ δ , 1], where τ δ = n -1+δ → 0 and put k = [sn], where [a] means the integer part of a. Let us write ϑ ⋆ k,n = ϑ ⋆ s,n and consider the estimator-

process ϑ ⋆ n = ϑ ⋆ s,n , s ∈ (τ δ , 1] .
Our goal is to construct an estimator process ϑ ⋆ n asymptotically optimal for all s ∈ (τ δ , 1]. Recall that the MLE θs,n constructed by the first k = [sn] observations is asymptotically efficient and for example,

√ sn θs,n -θ =⇒ N 0, I (ϑ) -1 , s ∈ [δ, 1] .
Note that to solve the equation sup ϑ∈Θ V ϑ, X [sn] = V θs,n , X [sn] for all s ∈ (τ δ , 1] is computationally rather difficult problem, except some particular examples. Therefore it is better to seek another estimators, which have the same property to be asymptotically efficient for all s ∈ (τ δ , 1]) and which can be calculated in more simple way.

We consider two different situations depending on the length of the learning interval [0, N ]. If N = n δ with 1 2 < δ < 1 then we construct the one-step MLEprocess and if we take the preliminary interval shorter, i.e., N = n δ with 1 4 < δ ≤ 1 2 , then we introduce an intermediate estimator and only after that we can construct the two-step MLE-process. Therefore we consider below these two situations separately.

One-step maximum likelihood estimator-process

We proceed as follows. Let us fix s ∈ (τ δ , 1] and slightly modify the vector scorefunction

∆ k (ϑ, X k N ) = 1 √ k k j=N +1 l(ϑ, X j-1 , X j ),
where

k = [sn] → ∞. Introduce the one-step MLE ϑ * s,n = θN + 1 √ k I θN -1 ∆ k ( θN , X k N ).
Here and below for simplicity of notation this writing means that N is the integer part of n δ . Theorem 3 Suppose that the conditions of regularity are fulfilled, then

√ k(ϑ * s,n -ϑ) =⇒ N 0, I (ϑ) -1 .
(2.10)

Proof. We can write √ k(ϑ * s,n -ϑ) = √ k(ϑ N -ϑ) + I θN -1 ∆ k ( θN , X k N ) = √ k(ϑ N -ϑ) + I( θN ) -1 ∆ k (ϑ, X k N ) + I( θN ) -1 ∆ k ( θ, X k N ) -∆ k (ϑ, X k N ) .
We have

∆ k ( θ, X k N ) -∆ k (ϑ, X k N ) = 1 0 ( θN -ϑ), ∆k (ϑ + v θN -ϑ , X k N ) dv.
Hence (below

ϑ v = ϑ + v θN -ϑ ) √ k(ϑ N -ϑ) + I( θN ) -1 ∆ k ( θ, X k N ) -∆ k (ϑ, X k N ) = √ k(ϑ N -ϑ)I( θN ) -1 I( θN ) + 1 √ k 1 0 ∆k (ϑ v , X k N ) dv .
Further

I( θN ) + 1 √ k 1 0 ∆k (ϑ v , X k N ) dv = I(ϑ) + 1 √ k ∆k (ϑ, X k 0 ) - 1 √ k ∆k (ϑ, X N -1 0 ) + I( θN ) -I(ϑ) + 1 √ k 1 0 ∆k (ϑ v , X k N ) -∆k (ϑ, X k N ) dv = 1 k k j=1 l (ϑ, X j-1 , X j ) + I(ϑ) + O N k + O n -δ 2 , because 1 √ k ∆k (ϑ, X N -1 0 ) = 1 k N -1 j=1 l (ϑ, X j-1 , X j ) = O N k = O n -1+δ , I( θN ) -I(ϑ) = O θN -ϑ = O n -δ 2 and 1 √ k 1 0 ∆k (ϑ v , X k N ) -∆k (ϑ, X k N ) dv = O θN -ϑ = O n -δ 2 .
By the central limit theorem we have

1 √ k k j=1 l(ϑ, X j-1 , X j ) + I(ϑ) =⇒ N (0, C (ϑ))
with some matrix C (ϑ). Remind that E ϑ l(ϑ, X j-1 , X j ) = -I(ϑ).

Therefore √ k(ϑ * s,n -ϑ) = I( θN ) -1 ∆ k (ϑ, X k N ) + n δ 2 (ϑ N -ϑ) n 1-δ 2 O n -1 2 + n 1-δ 2 O n -1+δ + n 1-δ 2 O n -δ 2 = I(ϑ) -1 ∆ k (ϑ, X k 0 ) + o (1) =⇒ N 0, I(ϑ) -1 ,
where we used once more the central limit theorem

1 √ k k j=1 l (ϑ, X j-1 , X j ) =⇒ N (0, I(ϑ)) .
Therefore the one-step MLE-process ϑ ⋆ n = ϑ ⋆ s,n , τ δ < s ≤ 1 for all s ∈ (τ δ , 1] is asymptotically normal (2.10) with asymptotically efficient covariance matrix.

Two-step maximum likelihood estimator-process

The choice of the learning period of observations N = n δ with δ ∈ (1/2, 1) allows us to construct an estimator process for the values s ∈ (τ δ , 1] only. It can be interesting to see if it is possible to take more short learning interval. Our goal is to show that the learning period can be N = n δ with δ ∈ (1/4, 1/2]. Below we follow the construction which was already realized in [START_REF] Kutoyants | On multi-step MLE-processes for ergodic diffusion[END_REF] in the case of ergodic diffusion process.

Suppose that N = n δ with δ ∈ (1/4, 1/2). The asymptotically efficient estimator we construct in three steps. By the first N observations as before we obtain the preliminary estimator θN which is asymptotically normal with the rate √ N , i.e., n

δ 2 θN,1 -ϑ =⇒ N (0, B (ϑ)) .
This can be the same estimator as in the preceding case. It can be, for example, the EMM, BE or MLE.

The two-step MLE-process ϑ ⋆⋆ n = ϑ ⋆⋆ k,n , k = N + 1, . . . , n we construct as follows. Fix some s ∈ (τ δ , 1], τ δ = n -1+δ and introduce the second preliminary estimator-process (below k =

[sn]) θk,2 = θN,1 + 1 √ k I θN,1 -1 ∆ k ( θN,1 , X k ), (2.11)
where

∆ k (ϑ, X k ) = 1 √ k k j=1 l (ϑ, X j-1 , X j ) .
Then we show that the random sequence n 1/4+ε θk,2ϑ with some ε > 0 is bounded in probability (tight).

Finally, using this estimator-process and the one-step procedure of the Theorem 3 we obtain asymptotically efficient estimator

ϑ ⋆⋆ k = θk,2 + 1 √ k I θk,2 -1 ∆ k ( θk,2 , X k ).
(2.12)

In the next theorem we realize this program.

Theorem 4 Suppose that the conditions of regularity are fulfilled, then the estimator ϑ ⋆ n defined (2.11) and (2.12) is asymptotically normal

√ k(ϑ ⋆⋆ n -ϑ) =⇒ N 0, I (ϑ) -1 .
Proof. The only thing to proof is the tightness of the sequence of random vectors n 1/4+ε θk,2ϑ , because if it is tight, then the proof of Theorem 4 follows from the Theorem 3. Let us fix some ε > 0.

For the estimator-process θk,2 defined by (2.11) we can write

n 1 4 +ε θk,2 -ϑ = n 1 4 +ε θN -ϑ + n 1 4 +ε √ k I θN -1 ∆ k ( θN , X k ) = n 1 4 +ε θN -ϑ + n 1 4 +ε √ k I θN -1 ∆ k (ϑ, X k ) + n 1 4 +ε √ k I θN -1 θN -ϑ ∆k ( θk , X k ).
Note that ∆ k (ϑ, X k ) is asymptotically normal and therefore

n 1 4 +ε √ k I θN -1 ∆ k (ϑ, X k ) -→ 0, because n 1 4 +ε k -1 2 → 0. Further n 1 4 +ε θN -ϑ + n 1 4 +ε √ k I θN -1 θN -ϑ ∆k ( θk , X k ) = n 1 8 + δ 2 θN -ϑ R n ,
where

R n = n 1 8 +ε-δ 2 J + I θN,1 -1 1 k k j=1 l θ, X j-1 , X j .
We have by the law of large numbers

1 k k j=1 l (ϑ, X j-1 , X j ) -→ -I (ϑ) .
From the regularity conditions it follows that

I θN,1 -1 -I (ϑ) -1 ≤ C θN,1 -ϑ , ∆k ( θk , X k ) -∆k (ϑ k , X k ) ≤ C θN,1 -ϑ .
Therefore we verified the tightness of the sequence n

Examples

We consider below two examples. Both of them was already discussed in the previous chapter in the context of the study of the Bayesian estimators and the estimators of the method of moments. In the first example we construct the preliminary MLE and the one-step MLE-process. In the second example we construct the preliminary EMM, the second preliminary estimator-process and then the two-step MLE-process.

Example 1. MLE as preliminary estimator

Let us consider the problem of the construction of the one-step MLE-process in the case of observations X n = (X 0 , X 1 , . . . , X n ) of the time series

X j = (X j-1 ) 2 1 + ϑ | X j-1 | + ε j , ϑ ∈ (2, 5), (2.13)
where (ε j ) j≥1 ∼ N (0, 1).

Note that for this model the conditions of the Theorem 2 of the work [START_REF] Varakin | On parameter estimation for "polynomial ergodic" Markov chains with polynomial growth loss functions[END_REF] can be verified. The time series has ergodic properties. The density of invariant law we estimate with the help of kernel-type estimator:

πn (x) = 1 nh n n j=1 K X j -x h n ,
where the width h n = n -1/5 and K (•) is the gaussian kernel:

K(x) = 1 √ 2π e -x 2 2 .
The estimator of the invariant density in the case n = 10 5 and ϑ = 2, 5. could be found in the previous example-section.

First we define the MLE constructed on the learning sequence X N = (X 0 , X 1 , . . . , X N ). For the conditional density function π(ϑ, X j-1 , X j ) of the Markov sequence (2.13), we have the representation

π(ϑ, X j-1 , X j ) = 1 √ 2π e -1 2 X j - (X j-1 ) 2 1+ϑ|X j-1 | 2 . (2.14)
Hence the likelihood function is

V (ϑ, X N ) = π 0 (X 0 ) N j=1 π(ϑ, X j-1 , X j ), ϑ ∈ (2, 5).
And the log-likelihood ratio function is

L N (ϑ, X N ) = ln π 0 (X 0 ) + N j=1 - 1 2 ln 2π - 1 2 X j - (X j-1 ) 2 1 + ϑ | X j-1 | 2 = ln π 0 (X 0 ) + N j=1 ℓ(ϑ, X j-1 , X j ).
On the Figure 2.1 we present the result of simulations of this log-likelihood ratio function L N (ϑ, X n ), where N = n 3/4 ∼ 5623. The continuous vertical line corresponds to the true value ϑ 0 = 2.5 of the parameter and the vertical dotted line corresponds to the maximum-likelihood estimator θN .

To find the MLE we have to solve the maximum likelihood equation

∂L ∂ϑ = N j=1 l (ϑ, X j-1 , X j ) = 0, ϑ ∈ (2, 5) ,
which has the following form The normalized score-function is

N j=1 | X j-1 | 3 (1 + ϑ | X j-1 |) 2 -X j + (X j-1 ) 2 1 + ϑ | X j-1 | = 0, ϑ ∈ (2, 5) .
∆ k (ϑ, X k ) = 1 √ k k j=1 | X j-1 | 3 (1 + ϑ | X j-1 |) 2 -X j + (X j-1 ) 2 1 + ϑ | X j-1 | ,
where N + 1 ≤ k ≤ n. Finally the one-step MLE-process that has the following representation

ϑ ⋆ k,n = θN + 1 I k ( θN )k k j=1 | X j-1 | 3 (1 + θN | X j-1 |) 2 -X j + (X j-1 ) 2 1 + θN | X j-1 | ,
where N + 1 ≤ k ≤ n and I k ( θN ) = 0.001 is the Fisher information calculated as follows

I k ( θN ) = - 1 k k j=1 l θN , X j-1 , X j .
More detailed analysis shows that with such definition of the empirical Fisher information the main result of this work Theorem 2 is valid. Therefore the estimatorprocess ϑ ⋆ n is asymptotically normal and asymptotically efficient. The realization of the simulated one-step MLE-process for n = 10 5 is shown on the Figure 2.2. We can see that the initial estimator θN is far from the true value and that the trajectory of one-step MLE-process approaches to the true value.

Example 2. EMM as preliminary estimator

Let us consider another example, where it will be much more easy to take the EMM as preliminary one. Our goal is to illustrate the convergence of the one-step MLE-process when the initial estimator is not asymptotically efficient. It be can, for example, the EMM which has"bad" rate and "bad" limit variance.

Introduce the time series

X j = X j-1 + 3 ϑ -X j-1 1 + (X j-1 -ϑ) 2 + ε j , j = 1, . . . , n, (2.15)
where (ε j ) j≥1 are i.i.d. standard Gaussian random variables. The unknown parameter ϑ ∈ Θ = (-1, 1). The initial value X 0 is supposed to be given too. Note that this example was already used in the work [START_REF] Motrunich | On parameter estimation for Markov sequences[END_REF] and in the previous chapter to illustrate the properties of the BE and EMM. This process has ergodic properties and its invariant density can be estimated as in the Example 1 with the help of the kernel-type estimator. The result of such estimation can be found in [START_REF] Motrunich | On parameter estimation for Markov sequences[END_REF] and in the previous example-section.

In this example our goal is to two estimator-processes: one-step and two-step. Our goal is to construct the estimator-processes ϑ ⋆ n and ϑ ⋆⋆ n , which are asymptotically equivalent to the MLE and therefore are asymptotically efficient. The same time their calculation is much more simple than that of the MLE.

We start with the one-step MLE-process. As described before we construct this estimator in two steps. First we need to calculate a consistent preliminary estimator θN by the initial observations X 1 , . . . , X N , where N = n δ with δ ∈ ( 1 2 , 1). As preliminary we can take the MLE, BE or EMM. Note that the unknown parameter for this model of observations is the shift parameter and that the invariant density function is symmetric with respect to ϑ. Hence we can take the EMM θN

= 1 N N j=1 X j -→ ϑ, N = n 3/4 .
Of course, the limit variance of the EMM θN is greater than that of the BE, but this estimator is much more easier to calculate.

The score-function process is

∆ k (ϑ, X k ) = 1 √ k k j=1 l (ϑ, X j-1 , X j ) , N + 1 ≤ k ≤ n, where l (ϑ, x, x ′ ) = - 1 2 2 x ′ -x -3 ϑ -x 1 + (ϑ -x) 2 × 3 1 + (ϑ -x) 2 -(ϑ -x) 2 (ϑ -x) (1 + (ϑ -x) 2 ) 2 = 3 x ′ -x -3 ϑ -x 1 + (ϑ -x) 2 1 -(ϑ -x) 2 (1 + (ϑ -x) 2 ) 2 .
Therefore we can calculate the one-step MLE-process as follows

ϑ ⋆ k,n = θN + 1 I k √ k ∆ k ( θN , X k ) = θN + 3 I k k k j=1 X j -X j-1 -3 ϑ -X j-1 1 + (ϑ -X j-1 ) 2 1 -(ϑ -X j-1 ) 2 (1 + (ϑ -X j-1 ) 2 ) 2 .
Here N + 1 ≤ k ≤ n I k is the empirical Fisher information. Its calculation in this example can be found in [START_REF] Motrunich | On parameter estimation for Markov sequences[END_REF]. Note that I (ϑ) = I as usual with the shift parameter.

Remind that by the Theorem 2 this estimator asymptotically normal .

The simulated one-step MLE-processes are shown on the Figure 2.3 and 2.4 for n = 1000 and n = 10000.

On the Figure 2.3 the preliminary EMM θN = 0.45 that is quite close to the true value of parameter ϑ = 0.5. We obtain this estimator based on the learning interval of N = 178 observations. And we can observe the estimator-process ϑ ⋆ n = ϑ ⋆ k,n , k = N + 1; . . . , n that tends to the true value. On the Figure 2.4 the preliminary EMM θN = 0.56 that is quite close to the true value ϑ = 0.5. We obtain this estimator based on the learning interval of N = 1000 observations. And we can observe the sequence of estimators ϑ ⋆ n = ϑ ⋆ k,n , k = N + 1; . . . , n that tends to the true value.

Let us illustrate the two-step MLE-process. Now we take N = n 3/8 .

We consider two cases: one with n = 1000 observations and the second with n = 10000 observations. On the Figure 2.5 the preliminary EMM θN = 0.4 that is quite far from the true value ϑ = 0.5. We obtain this estimator based on the learning interval of N = 1000 3/8 ≈ 13 observations. Then we obtain the second preliminary estimatorprocess ϑ ⋆ n = ϑ ⋆ k,n , k = N + 1; . . . , n (continuous line) and see that it tends to the true value. The two-step MLE-process ϑ ⋆⋆ n (dashed line) is closer to the true value and as well tends to the true value.

On the Figure 2.6 the preliminary EMM θN = 0.54 that is quite close to the true value ϑ = 0.5. We obtain this estimator based on the learning interval of N = 10000 3/8 ≈ 32 observations. Then we obtain the second preliminary estimator-process ϑ ⋆ n = ϑ ⋆ k,n , k = N + 1; . . . , n (continuous line) and see that it tends to the true value. The two-step MLE-process ϑ ⋆⋆ n (dashed line) is closer to the true value and as well tends to the true value.

Discussion

Two-step MLE-process allows us to estimate the parameter θ for the values k satisfying the condition n 1/4 < k ≤ n]. If we need a shorter learning interval, say, 1, n δ with δ ∈ ( 1 8 , 1 4 ], then we have to study the three-step MLE-process, i.e., we use a preliminary estimator θN and two estimator-processes like (2.11).

Note that the proposed one-step MLE-process can be written in the recurrent form.

ϑ ⋆ k+1,n = θN + 1 √ k + 1 I θN -1 ∆ k+1 θN , X k+1 = θN + 1 k + 1 I θN -1 k j=1 l θN , X j-1 , X j + l θN , X k , X k+1 = k k + 1 θN + 1 k I θN -1 k j=1 l θN , X j-1 , X j + 1 k + 1 θN + 1 k + 1 I θN -1 l θN , X k , X k+1 = k k + 1 ϑ ⋆ k,n + 1 k + 1 θN + 1 k + 1 I θN -1 l θN , X k , X k+1 .
The obtained presentation

ϑ ⋆ k+1,n = k k + 1 ϑ ⋆ k,n + 1 k + 1 θN + 1 k + 1 I θN -1 l θN , X k , X k+1
allows us to calculate ϑ ⋆ k+1,n using the values θN , ϑ ⋆ k,n and observations X k , X k+1 only.

The similar structure can be obtained for the two-step MLE-process too. Note that this is not a particular case of the well-known algorithms of stochastic approximation (see, for example the works of Duflo Random Iterative Models).

Chapter 3

On applications of Markov chains in health economics

Introduction

The management of critically ill patients is strongly dependent of intravascular catheters, most of them being central venous catheters (CVC) and arterial catheters. However, intravascular catheters can lead to serious infectious complications including Catheter-Related Bloodstream infections (CRBSIs) [START_REF] Maki | The risk of bloodstream infection in adults with different intravascular devices: a systematic review of 200 published prospective studies[END_REF], [START_REF] Raad | Intravascular-catheter-related infections[END_REF] as they are a relevant entry door allowing microorganisms into the bloodstream [START_REF] Crnich | The promise of novel technology for the prevention of intravascular device-related bloodstream infection. Pathogenesis and short-term devices[END_REF]. CRBSI is still a frequent (1-5 episodes/1000 catheter-days) and life-threatening complication observed in critically ill patients (ICU) [START_REF] Byers | Case fatality rate for catheterrelated bloodstream infections: a meta-analysis[END_REF], [START_REF] Kluger | The relative risk of intravascular device related bloodstream infections in adults[END_REF], [START_REF] Pittet | Nosocomial bloodstream infection in critically ill patients: Excess length of stay, extra costs, and attributable mortality[END_REF], [START_REF] Arnow | Consequences of intravascular catheter sepsis[END_REF], [START_REF] Mermel | Prevention of intravascular catheter-related infections[END_REF] even though in the last years different strategies have been proven to efficiently reduce its risk. The patient's own skin flora is very likely the most important source of catheter colonization and infection for central venous catheters in place for 10 days or less, and responsible for 60% of CRBSIs [START_REF] Mermel | The pathogenesis and epidemiology of catheter-related infection with pulmonary artery swanganz catheters: A prospective study utilizing molecular subtyping[END_REF], [START_REF] Maki | Infections associated with intravascular lines[END_REF]. The skin flora microorganisms more often causing CRBSI, Staphylococcus aureus and S. epidermidis) [START_REF] Wenzel | Hospital-acquired infections in intensive care unit patients: An overview with emphasis on epidemics[END_REF], [START_REF] Muñoz | Clinical-epidemiological characteristics and outcome of patients with catheter-related bloodstream infections in Europe[END_REF], [START_REF] Blot | Clinical and economic outcomes in critically ill patients with nosocomial catheter-related bloodstream infections[END_REF], will regrow after skin antisepsis [START_REF] Bashir | Suppression of regrowth of normal skin flora under chlorhexidine gluconate dressings applied to chlorhexidine gluconate-prepped skin[END_REF] and colonize the outer surface of the catheters [START_REF] Bjornson | Association between microorganism growth at the catheter insertion site and colonization of the catheter in patients receiving total parenteral nutrition[END_REF]. CR-BSIs frequently trigger sepsis which causes a great deal of morbidity and deaths, and increases health care costs [START_REF] Coopersmith | Effect of an education program on decreasing catheter-related bloodstream infections in the surgical intensive care unit[END_REF], [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF].

A large proportion of the CR-BSIs are preventable through careful control of the factors responsible for colonization of intravascular catheters by microorganisms [START_REF] Eggimann | Impact of a prevention strategy targeted at vascular-access care on incidence of infections acquired in intensive care[END_REF], [START_REF]National Nosocomial Infection Surveillance (NNIS) System report: data summary from January 1990 -May 1999[END_REF]. Several interventions based on better education, training and staffing [START_REF] Coopersmith | Effect of an education program on decreasing catheter-related bloodstream infections in the surgical intensive care unit[END_REF] or on the implementation on evidenced-based bundles of care [START_REF] Pronovost | An intervention to decrease catheter-related bloodstream infections in the ICU[END_REF], [START_REF] Pronovost | Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: Observational study[END_REF] were proven to be able to reducing the CRBSI rates. Also, the use of CHG antiseptic solutions for prepping the skin before the central line insertion and bathing or cleansing the patients during their ICU stay have a positive impact in preventing CRBSIs [START_REF] Chaiyakunapruk | Chlorhexidine compared with povidone-iodine solution for vascular catheter-site care: A meta-analysis[END_REF], [START_REF] Bleasdale | Effectiveness of chlorhexidine bathing to reduce catheter-associated bloodstream infections in medical intensive care unit patients[END_REF], [START_REF] Popovich | Effectiveness of routine patient cleansing with chlorhexidine gluconate for infection prevention in the medical intensive care unit[END_REF]. Implementation of more sophisticated technologies, as catheters impregnated with antiseptics and antibiotics, also contribute to further reduce the risk of CRBSIs [START_REF] Casey | Antimicrobial central venous catheters in adults: A systematic review and meta-analysis[END_REF].

Antimicrobial catheter dressings are one of the available medical technologies designed to prevent skin flora re-growth and, as a consequence, to reduce the incidence of CR-BSIs. The use of a CHG-containing antimicrobial sponge at the catheter insertion site significantly reduced the CRBSI rate in Intensive Care Unit (ICU) population in France [START_REF] Timsit | Chlorhexidine-lmpregnated sponges and less frequent dressing changes for prevention of catheter-related infections in critically III adults: A randomized controlled trial[END_REF], even when the baseline was already low (below 2 episodes/1000 catheter-days). More recently, the clinical efficacy of a novel transparent CHG-containing catheter dressing, combining antimicrobial activity and transparency for easy visual observation of the insertion site, has been also evaluated by the same team, in a French multicenter randomized controlled trial [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF]. In this study, the impact of the antimicrobial transparent dressing on reducing the CRBSI rate was also highly statistically significant.

Subsequently, two multi-state models were constructed. A homogeneous (H-MCMC) and non-homogeneous Markov Model with Markov Chain Monte Carlo simulation (NH-MCMC) models have been developed [START_REF] Maunoury | Cost-Effectiveness of the TLC-NOSF Dressing in Venous Leg Ulcers[END_REF], based on individual patient data collected during the French RCT [START_REF] Dimick | Increased resource use associated with catheter-related bloodstream infection in the surgical intensive care unit[END_REF]. The methods and results of these two approaches are discussed and compared.

The aim of these works, focusing on the ICU perspective, was to evaluate the costeffectiveness of routine use of the CHG-containing dressings in critically ill patients. The both models attempted to simulate the various observable health trajectories of ICU patients regarding the risk of acquiring CRBSIs and to evaluate all the uncertainty around the estimations of the RCT. These new models are profoundly different from all previous economical evaluations of antimicrobial dressings for intravascular access related to the prevention of CRBSI [START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the dressing study[END_REF], [START_REF] Ye | Economic impact of use of chlorhexidine-impregnated sponge dressing for prevention of central lineassociated infections in the United States[END_REF], [START_REF] Crawford | Cost-benefit analysis of chlorhexidine gluconate dressing in the prevention of catheter-related bloodstream infections[END_REF]. Those evaluations used decisiontree models representing the therapeutic choices (antimicrobial vs. non-antimicrobial dressings) and the incidence of catheter-related infections as clinical outcome.

The results of this chapter have been the subject of several publications and oral presentations. The principal article named "Cost-effectiveness analysis of a transparent antimicrobial dressing for managing central venous and arterial catheters in intensive care units" [START_REF] Maunoury | Cost-effectiveness analysis of a transparent antimicrobial dressing for managing central venous and arterial catheters in intensive care units[END_REF] was submitted and accepted for the publication in PLOS ONE journal of science and medicine (the latest impact factor: 3.534).

Besides, several posters and presentations were closely linked with this topic. The posters were presented during the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) annuals conferences (2012-2014) and the corresponding abstracts were published in Value in Health:

1. Non-homogeneous cost-effectiveness modeling of a new CHG-dressing for preventing catheter-related bloodstream infections for patients in intensive care units [START_REF] Maunoury | Non-homogeneous costeffectiveness modeling of a new CHG-dressing for preventing catheterrelated bloodstream infections for patients in intensive care units[END_REF]. 2. Cost-Effectiveness of the TLC-NOSF Dressing in Venous Leg Ulcers [START_REF] Maunoury | Cost-Effectiveness of the TLC-NOSF Dressing in Venous Leg Ulcers[END_REF]. 3.2 Healthcare decision-making

Medical interest

Catheter-related bloodstream infections (CRBSIs) are associated with attributable mortality rates of up to 11.5% and additional length of stay in the intensive care unit (ICU) of up to 12 days [START_REF] Soufir | Attributable morbidity and mortality of catheter-related septicemia in critically ill patients: a matched, riskadjusted, cohort study[END_REF], [START_REF] Renaud | ICU-Bacteremia Study Group. Outcomes of primary and catheter-related bacteremia. A cohort and case-control study in critically ill patients[END_REF]. The universally accepted method for minimizing CRBSIs is a bundle of care combining maximal sterile barrier precautions for insertion, an appropriate antiseptic solution for skin antisepsis and line access, preferential subclavian catheterization, and immediate removal of unnecessary catheters [START_REF] O'grady | Guidelines for the prevention of intravascular catheter-related infections[END_REF], [START_REF] Timsit | New materials and devices for preventing catheter-related infections[END_REF].

Combining this catheter-care bundle with continuous quality improvement programs can decrease the CRBSI rate below 2 per 1,000 CVC-days [START_REF] Eggimann | Impact of a prevention strategy targeted at vascular-access care on incidence of infections acquired in intensive care[END_REF], [START_REF] Coopersmith | Effect of an education program on decreasing catheter-related bloodstream infections in the surgical intensive care unit[END_REF]. In Europe, the incidence of CRBSIs ranges from 1 to 3.1 per 1,000 patient-days [START_REF] Suetens | European surveillance of ICUacquired infections (HELICS-ICU): methods and main results[END_REF] and according to the French surveillance network, less than one CRBSI occurred per 1,000 CVCdays in 2010 [START_REF] Poujol | Notification of healthcareassociated infections[END_REF]. However, rates below 2 per 1,000 CVC-days are difficult to achieve in all ICUs [START_REF] Davidoff | Heterogeneity Is Not Always Noise[END_REF], [START_REF] Timsit | A multicentre analysis of catheter-related infection based on a hierarchical model[END_REF] and in the long term [START_REF] Pronovost | Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: observational study[END_REF].

Most organisms responsible for short-term CRBSIs originate from the insertion site [START_REF] Mermel | What is the predominant source of intravascular catheter infections?[END_REF]. It was demonstrated previously that the risk of developing CRBSIs can be dramatically reduced (60% decrease) by the systematic use of a new antimicrobial transparent dressing [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF] containing a Chlorhexidine Gluconate (CHG) gel even though bundles of care are appropriately followed and CRBSI level is lower than 1.5 per 1,000 catheter-days in the control group.

The purpose of this work is to evaluate the advantages of the routine use of the new CHG dressing to secure central lines of patients in ICU from a medico-economic viewpoint compared to non-antimicrobial transparent dressings, in settings where bundles of care practices are appropriately followed and where incidence of infection is already low (1.5 per 1,000 catheter-days). Both medical and economic criteria are embedded into a decision-analytic model to support the choice of the best dressing strategy from an ICU perspective.

Health economic context

The main question of the health economics we can formulate as follows:

-What is the additional cost required to obtain a supplementary benefits of a therapeutic intervention or treatment?

-How much should society spend to increase the life expectancy for one year? These questions are crucial because they emphasize the fact that resources are limited. In health system it is necessary to consider the both medical and economic variables in terms of efficacy and in terms of expected cost.

The health economic evaluation provides explicit information on the costs and consequences of different interventions (health products or technologies, therapeutic sequences, screening programs, etc.) to improve decision-making and to promote efficient resource use. It is intended to facilitate the public decisions about resource allocation. The countries with a National Health Service or National health insurance generally leave the political authorities to decide on new drugs, new therapies and medical devices to be covered by the plan. It is clear that the innovative product with the proven therapeutic benefits, often associated with the reducing the risk of premature death to a certain population at risk, induces the extra cost (additional expenses) compared to existing products.

Cost-effectiveness analysis

The cost-effectiveness analysis (CEA) is a method of medico-economic evaluation to assess the costs and medical benefits for the various medical technologies, therapeutic sequences or concurrent clinical strategies. Generally, CEA is used to compare a therapeutic innovation with the most widely used current strategy.

The general rule in solving a problem of medical and economic decision is simple. The decision should be made taking into account two factors: in terms of efficiency and in terms of cost. Medical and budgetary approaches are opposed to each other naturally. Physicians think in terms of the expected medical service or in terms of efficiency. Managers try to minimize the expected expenditure.

The efficiency criterion is a way to bring together these two terms. In CEA the goal is to calculate the incremental cost-effectiveness ratio (ICER). The ICER means that the additional cost generated by the new product is rewarded with medical effect.

= = - - .
The substitution or replacement of a strategy by another leads to the difference in cost and the difference in efficiency. By calculating ICERs, we can classify the different strategies with respect to another based on the efficiency criterion. Obviously, the best strategy is the one that is the least expensive and that is most effective. On the other side, the strategy is dominated by another if it is more expensive and less effective or equally effective but more expensive.

It should not be considered in any way that the medical and economic assessment is limited to simple calculation of the ICER. The real challenge for the evaluation is to incorporate into the model the proposed uncertainty concerning this calculation.

And finally, if the cost-effectiveness of a strategy is proven, the positive or negative consequences of the decision can be assessed on the budget of a payer structure. To analyze these budgetary consequences, a budget impact analysis (BIA) should be proposed by the health economists. By cons, there is no sense in BIA if the differential cost-effectiveness ratio is not favourable to the innovative strategy.

The objective of this medico-economic study is to demonstrate the advantages for patient of the routine use of CHG dressing for central lines in intensive care unit (ICU) patients, compared to non-antimicrobial dressings, related to the prevention of catheter-related bloodstream infections. The demonstration will take in consideration both medical and economic criteria and will be founded on an analytic decision model (multi-state homogeneous and non-homogeneous Markov models). The model outcomes will support the choice of the best dressing strategy based on a retroprospective cost-effectiveness analysis (the transition probabilities are estimated from the real-life individual database).

Presentation of database

Data Collection

The main data source was the database assembling all patient data collected during the RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF]. This multicentre randomized-controlled study compared the impact of the antimicrobial CHG dressing (referred in the current dissertation as CHG dressings) and of non-antimicrobial transparent dressings (referred as non-CHG dressings) on the rate of catheter related infections.

The main objective of the RCT transposed in this cost-effectiveness analysis was to determine if the use of the new transparent CHG dressing decreased CRBSI rates. The RCT was not blinded to the investigators or ICU staff due to the obvious visual differences between the dressings, but was blinded to the microbiologists processing the skin and catheter cultures and to the committee adjudicating on the CRBSI cases.

The two groups receiving different types of non-antimicrobial transparent dressings in the RCT were pooled together as "non-antimicrobial transparent dressings" for the purpose of the modeling presented in this work.

Study Population

The multicentre RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF] enrolled adult patients (>18 years) admitted to 12 French ICUs in seven universities and four general hospitals, from 31 May 2010 to 29 July 2011, and expected to require intravascular catheterization for 48 hours. Patients with known allergies to chlorhexidine or transparent dressings were excluded. Of 2,054 screened patients with at least one catheter, 1,898 could be enrolled in the study and 1,879 were assessable for the intention-to-treat analysis, for a total of 4,163 catheters and 34,339 catheter-days. Patients and catheters characteristics are reported in the next sections.

Due to a short time horizon (30 days), the patient characteristics such as age and the proportion of males/females were not incorporated in the model. However, the analysis of adjustment on covariates between the subgroups was conducted in order to ensure the comparability between two strategies (see section 3.3.4 and 3.3.5).

Study Catheters

In the RCT, all central venous catheters inserted at subclavian, jugular and femoral veins, as well as arterial catheters inserted at radial and femoral arteries for a given patient, were managed according to the randomized dressing assignment. Pulmonary arterial, hemodialysis, and peripherally-inserted venous catheters and catheters inserted before ICU admission were excluded from the study. All study centers followed French recommendations for catheter insertion and care, which are similar to Center for Disease Control (CDC) recommendations [START_REF] Timsit | Updating of the 12th consensus conference of the Société de Réanimation de langue française (SRLF): Catheter related infections in the intensive care unit[END_REF].

Additional ICU Length of Stay (LOS) due to CR-BSI

The analyses presented in this work were conducted on the "Global" population, comprising patients who, during their ICU stay remained alive, died or discharged from the ICU. The "global" patient is the main statistical unit of the study (see section Results of cost-effectiveness analysis).

In order to assess the impact of CR-BSI on extending ICU LOS, a subgroup analysis was performed within the "Global" population, comparing patients having developed a CR-BSI during the ICU stay with those not having developed a CR-BSI. The comparison was made through independent non-homogeneous MCMC simulations for each dressing strategy. These NH-MCMC simulations were based on observed patient data, collected during the Dressing 2 clinical study. In this study, all patients were randomly assigned to one of the two dressing strategies, what allow us to assume comparability of the groups.

Discussing further on the comparability of these two subgroups, we can present two additional adjustments.

First, a "natural" adjustment, which is linked to the main statistical unit of our modeling which is the "global" patient was considered. It means that the probability of developing a CR-BSI in the "Global population", that corresponds to the clinical trial population, follows the same plausible statistical distribution law (due to the randomization).

The statistical analysis for all confounding covariates, such as age, sex, severity (SOFA score), duration of catheterization, number of dressing change per day, shows the comparability between these subgroups (see section 3.3.5).

A further adjustment from NH-MCMC simulation was performed, considering covariates which could impact mainly the values linked to the cost-effectiveness results. The rate of catheter change and the number of additional ICU days due to CRBSI in each dressing strategy were taken into account.

Adjustments on covariates between the subgroups

A statistical analysis for all confounding covariates, such as age, sex, Sequential Organ Failure Assessment severity score (SOFA, a score predicting ICU mortality based on lab results and clinical data [START_REF] Ferreira | Serial evaluation of the SOFA score to predict outcome in critically ill patients[END_REF]), duration of catheterization, number of dressing change per day, was performed in order to demonstrate the comparability between the subgroups (see Table 1). Four subgroups of patients (CHG/CRBSI, CHG/No-CRBSI, Non-CHG/CRBSI, Non-CHG/No-CRBSI) were compared with these covariates. Mann-Whitney tests between subgroups were performed. The results of Mann-Whitney tests on these covariates between subgroups (CRBSI/No CRBSI) show no statistically significant difference at the 0.05 level.

Main Assumptions

1. In the cases where the "Discharge" state was reported, and a CRBSI was observed for this patient up to two days after the event, the infection was considered to occur the day of discharge from the ICU. 2. The transitional probability from the health state "Contact Dermatitis" to "Dressing Gauze and Tape" state was considered the same for both groups. By entering to the "Dressing Gauze and Tape" the patient followed probabilities of transition corresponding to the non-CHG dressings arm. 3. The cost of CRBSI is independent from the outcome (survival or death or discharge). 4. Catheter colonization with or without CRBSIs was considered as having negligible diagnosis costs and was excluded from the model for not being considered as a "health state" per se. 5. The costs related to replacement of a catheter suspected to be colonized (and causing CRBSI) were comprised in one of the health states including the need for a new central line. The cost per ICU day was considered as identical for each dressing group. The cost of a gauze and tape dressing is identical in both groups. 6. Health states including CRBSIs were assumed to last a single day because it was not technically possible to identify the termination of a CRBSI in the patient database. However, the costs of treating the complete episode, as well as the total costs associated with the extra length of stay due to the CRBSI were accounted on the day when the CRBSI was diagnosed. 7. With the current knowledge of publicly available data sources, there is no direct CR-BSI related risk of dying.

Medico-economic evaluation using Markov models

The use of Markov model is specially required in following cases:

clinical trials have insufficient periods of follow-up to assess the impact of therapy in the long-term. The Markov model extrapolates from trial results in terms of transitions and rates for all comparators and estimates the costeffectiveness of new interventions over a life-time horizon;

to determine the influence of uncertainty surrounding input parameters. It can be achieved using several univariate and multivariate analyses (as deterministic and probabilistic sensitivity analysis where the values for parameter estimates vary within the uncertainty distributions that best reflect the nature of each specific parameter);

when needed an accurate representation of the evaluated clinical structure by modeling repetitive events and time dependence of probabilities.

Markov homogeneous model

Markov models consider the patients in a discrete state of health, and the events represent the transition from one state to another. Such type of model permits a more accurate representation of the evaluated clinical structure by modeling repetitive events and time dependence of probabilities (see: time-inhomogeneous Markov Chains).

In this section we will consider the discrete-time Markov chain () = ( 0 , 1 , … ), where () is the health state in time . In the following epidemiological applications the state space is discrete. The Markov property states that the conditional probability distribution for the system at the next step depends only upon the present state, not on the sequence of events that preceded it (it is so called memoryless property).

A random process X possesses the Markov property, and is called a Markov Chain, if

P(Xn+1 = j | X0 = i0, X1 = i1,… , Xn = in )= P(Xn+1 = j | Xn = in)
depends only on in and j, not on any past values. Time-homogeneous Markov chains are processes where

P(Xn+1 = j | Xn = in )= P(Xn = j | Xn-1 = in)
for all n. It implies that the probability of the transition is independent of n or the probability of the transition is the same after each step.

Markov models have limitations that must be overcome as models become more sophisticated, especially when dealing with time-dependent probabilities of transitions and different states of disease.

Study design

The adopted modeling approach complies with the guidelines of French National Authority for Health (Haute Autorité de Santé -HAS) [START_REF] Health | Choices in Methods for Economic Evaluation: a methodological guide[END_REF]. The 30-day ICU-time homogeneous Markov model [START_REF] Maruotti | A mixed non-homogeneous hidden Markov model for categorical data, with application to alcohol consumption[END_REF], [START_REF] Liquet | Investigating hospital heterogeneity with a multi-state frailty model: application to nosocomial pneumonia disease in intensive care units[END_REF] structure was based on observed data of a multicentre RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF], conducted by the Grenoble University Hospital -CHU Grenoble. The model has been programmed using Visual Basic Application with the Excel® 2007 software. This model consists of six health states described in Table 2. The statistical unit of the study is the ICU patient within a time horizon of 30 days (discharged alive from the ICU, alive but still at the ICU, or deceased during the ICU stay). Patient data from the multicentre RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF], comparing the CHG dressing to non-antimicrobial transparent dressings, were translated into a patient transition matrix among the different possible health states, for both the antimicrobial and nonantimicrobial dressings groups. This transitional matrix was used to perform homogeneous Markov-Chain Monte Carlo (H-MCMC) simulations [START_REF] Hastings | Monte Carlo Sampling Methods Using Markov Chains and Their Applications[END_REF] representing the observed daily evolution of patients in ICU. 1,000 Monte Carlo simulations of 1,000 patients were used for probabilistic sensitivity analysis and 95% confidence intervals (CI) calculations.

Model structure

Markov models consider patients in a discrete state of health [START_REF] Weinstein | Clinical Decision Analysis[END_REF], [START_REF] Beck | The Markov Process in Medical Prognosis[END_REF], [START_REF] Sonnenberg | Markov models in medical decision making: a practical guide[END_REF], [START_REF] Janssen | Semi-markov models and applications[END_REF], and events representing the transition from one health state to another. This type of modeling can take into account iterative occurrences for each Markov state. The Markov property refers to the fact that the conditional probability distribution of future states of the health depends only upon the present state, not on the sequence of events that preceded it.

The hypothesis of this homogeneous model is that the transition probabilities do not change with time spending in ICU. This assumption is very strong and therefore the non-homogeneous model was developed (section 3.4.2).

Building up the transition matrix

The overall percentages of patient in each health state per comparator as reported in the RCT database [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF] were transformed to 1-day (cycle length) transition probabilities. Each patient will be in one of the six health states described in Table 2 at each day in ICU.

Healthcare resource use and costs

Costs used within the model reflect the ICU perspective in France and consist of following components:

• drug acquisition costs,

• cost of treating the adverse events ,

• direct costs of treating the CR-BSI,

• costs due to ICU stay.

catheter suspected to be colonized (and causing CR-BSI) will be absorbed in the health-states CT new (the dressing arm with NO CHG); • The estimated cost per ICU day at the Grenoble University Hospital is identical in each Dressing Group; • The G+T (gauze and tape) cost for CHG-group is identical to No-CHG group.

Costs items for each Markov state

Table 3 reportes the costs included in the analysis for each Markov state. The results showed in Tables 5 and6 refer to the base case scenario of the costeffectiveness modeling. The main difference between CHG group and Non-CHG group are based on % of states 3 and 4, e.g. number of CRBSIs. A ratio of 1 to 4.67 is observed for the average number of CRBSIs between dressing groups. The number of ICU-days, the number of days before discharging and the number of days before dying are comparable in the two groups. The table 6 shows the cost result for the average patient in each dressing group. The statistical significance of this result will be discussed in Sensitivity analyses section.

Sensitivity analyses

Sensitivity analyses are performed to vary each parameter of the model in order to determine what levels will result in a change of preference for the therapeutic strategy. This is a way to test the boundaries of the model and identify the main parameters driving cost differences.

One-way Sensitivity Analysis

For the one-way sensitivity analysis/tornado diagram, we varied the parameters under the base-case assumptions. The resulting tornado diagram is shown in Figure 123.

Results are most sensitive to the additional ICU LOS due to CRBSI, CHG dressing cost, dressing change schedule, CRBSI, death and discharge rate changes.

Probabilistic Sensitivity Analysis (PSA)

A probabilistic sensitivity analysis [START_REF] Weinstein | Principles of good practice of decision analytic modeling in health care evaluation: Report of the ISPOR Task Force on Good Research Practices-Modeling Studies[END_REF] was performed with 1,000 homogeneous MCMC simulations of 1,000 patients for both CHG dressing and non-CHG dressing groups. Each group of 1,000 patients depicts an average patient representing all patients for each dressing group studied in the RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF]. The method used was the Gibbs sampling [79], a commonly used Markov Chain Monte Carlo algorithm. It allowed to retrace 10 6 health trajectories (1000*1000 patients for each dressing strategy), based on the probabilities observed in the RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF] day-after-day (during 30 days) for each patient to change from one health-state to another. Repeating the algorithm 1,000 times allows the calculation of 95% confidence intervals for the costeffectiveness criterion (here, number of CRBSI avoided and cost per patient).

The health states including CRBSI (CRBSI/No new catheter and CRBSI/new catheter) as rare events for both strategies are in the area of low probabilities. On the other hand, the "discharge" and "death" states as frequent events for both strategies are in the area of high probabilities. This corresponds to the reality observed in the RCT (higher frequency of discharge and death than CRBSI).

The CHG-dressing prevents 6. 7 and Table 8). We can see that the difference of CRBSI events between the strategies is statistically significant at the 0.05-level. So the difference of costs between the strategies is not statistically significant at the 0.05-level.

The PSA cost-effectiveness plan (Figure 4) describes the effectiveness difference on the x-axis and the cost difference on the y-axis between the two groups of dressings, for 1,000 H-MCMC simulations of 1,000 patients in each group. The (0,0)-point indicates the reference dressing strategy (Non-CHG group). All the points observed on the graph represent the incremental cost-effectiveness ratio (ICER) of CHGdressing strategy versus reference dressing. This PSA supports the decision to adopt CHG dressing for critically ill patients since the strategy is 97.95% more effective than the comparator at the same cost per patient in the intensive care unit. The mean ICER calculated from the PSA and defined as the cost per patient treated with chlorhexidine dressing to prevent one patient experiencing a CRBSI, is €12,094.

Markov non-homogeneous model

In non-homogeneous case the Markov property is still unchanged. We can state that

P(Xn+1 = j | X1 = i1 , X2 = i2 ,… , Xn = in )= P(Xn+1 = j | Xn =
in) depends only on in, j and n. It means that the probability of the transition depends on n (time) or the probability of the transition changes after each step.

Study design

The adopted modeling approach complies with the guidelines of French National Authority for Health (Haute Autorité de Santé -HAS) [START_REF] Health | Choices in Methods for Economic Evaluation: a methodological guide[END_REF]. The 30-day ICU-time nonhomogeneous Markov model [START_REF] Maruotti | A mixed non-homogeneous hidden Markov model for categorical data, with application to alcohol consumption[END_REF], [START_REF] Liquet | Investigating hospital heterogeneity with a multi-state frailty model: application to nosocomial pneumonia disease in intensive care units[END_REF] structure was based on observed data of a multicentre RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF], conducted by the Grenoble University Hospital -CHU Grenoble. The model has been programmed using Visual Basic Application with the Excel® 2007 software. This model consists of eight health states described in Table 9: four states combining either occurrence, or no occurrence of CRBSI, and the need, or no need, of a new central line (CT); one state for contact dermatitis; one for changing to an alternative dressing (gauze and tape) in case of dermatitis, and two absorbing states (death and discharge of the ICU). The statistical unit of the study is the ICU patient within a time horizon of 30 days (discharged alive from the ICU, alive but still at the ICU, or deceased during the ICU stay). Patient data from the multicentre RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF], comparing the CHG dressing to non-antimicrobial transparent dressings, were translated into a daily patient transition matrix among the different possible health states, for both the antimicrobial and nonantimicrobial dressings groups. The transition matrixes were used to perform nonhomogeneous Markov-Chain Monte Carlo (NH-MCMC) simulations [START_REF] Hastings | Monte Carlo Sampling Methods Using Markov Chains and Their Applications[END_REF] representing the observed daily evolution of patients in ICU. 1,000 Monte Carlo simulations of 1,000 patients were used for probabilistic sensitivity analysis and 95% confidence intervals (CI) calculations.

The final health outcome of the cost-effectiveness analysis is the number of CRBSIs avoided and the cost-effectiveness criterion is the cost per patient with CRBSI avoided resulting from chlorhexidine dressing use.

Model structure

Markov models consider patients in a discrete state of health [START_REF] Weinstein | Clinical Decision Analysis[END_REF], [START_REF] Beck | The Markov Process in Medical Prognosis[END_REF], [START_REF] Sonnenberg | Markov models in medical decision making: a practical guide[END_REF], and events representing the transition from one health state to another. This type of model allows an accurate representation of the evaluated clinical structure by modeling repetitive events and time dependence of probabilities (time-nonhomogeneous Markov Chains). The Markov property refers to the fact that the conditional probability distribution of future states of the health depends only upon the present state, not on the sequence of events that preceded it.

In a non-homogeneous modeling approach, the transition probability from one state to the next will change with time, as observed in real life of patients in ICU. The probability of changing from one state to the other can be assembled into a transition matrix.

Building up the transition matrix

A transition matrix for each day in the ICU was built based on transition probabilities reported in the RCT database [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF]. Each patient will be in one of the eight health states described in Table 9 at each day in ICU. The model structure (see Figure 5) is somewhat based on that published by the independent assessments for managing the CR-BSI. However, more detailed levels are added looking at the "day to day" individual data during the ICU stay. In the Markov model there is no CR-BSI related death state as no evidence was found to support this direct elevated risk of dying.

Unlike the published model, an attempt has been to segregate the rather heterogeneous "day to day" patient health state in CR-BSI control.

estimated. For patients with CRBSI, an additional cost [START_REF] Dimick | Increased resource use associated with catheter-related bloodstream infection in the surgical intensive care unit[END_REF] due to an extra ICU length of stay (LOS) was calculated (see next section).

Main Assumptions Used for the Cost Analysis

• The cost of CR-BSI is independent from the outcome (survival or death or discharge). For the analysis the main statistical unit is the "global" (survival or death or discharge) patient; • Catheter colonization with or without CR-BSI had no costs (after the diagnosis)

or adverse outcomes (colonization has been initially excluded from the model because it's not a "health-state". The costs for diagnosis are negligible compared to costs related to additional LOS. The costs related to replace a catheter suspected to be colonized (and causing CR-BSI) will be absorbed in the health-states CT new (the dressing arm with NO CHG); • The estimated cost per ICU day at the Grenoble University Hospital is identical in each Dressing Group; • The G+T (gauze and tape) cost for CHG-group is identical to No-CHG group. The results showed in Tables 12-16 refer to the base case scenario of the costeffectiveness modeling. Tables 12 and13 show that the main difference between CHG group and Non-CHG group are based on % of states 3 and 4, e.g. CRBSIs. A ratio of 1 to 5 is observed for the average number of CRBSIs between dressing groups. The number of ICU-days, the number of days before discharging and the number of days before dying are comparable in the two groups. 12 and13). This difference was highly statistically significant as indicated by the non-overlapping 95% confidence intervals (see Figure 6). The percentage of patients in the two absorbing states, coded as 7 (discharge from ICU) and 8 (death), is comparable in both groups of dressings, (see Table 14, Table 15 and Figure 7). Considering a defined global average patient cohort, the average cost per patient per ICU-day decreases with time due to the increasing percentage of discharged or dead patients during the ICU stay (the cost for a discharged or dead patient is considered as zero in the model). The curves are showing the average cost considering an initial population of defined sized (Figure 8). Table 16 shows the cost results for the average patient in each dressing group. 16). Subgroup analyses supported by the comparability test compared the average total costs for patients with CRBSI versus patients without CRBSI for each study group (CHG and Non-CHG dressings). This comparison revealed no significant differences in costs among the subgroups (Table 16). Figure 9 shows the non-overlapping 95% confidence intervals for each dressing strategy.

Costs items for each Markov state

Sensitivity analyses

Sensitivity analyses are performed to vary each parameter of the model in order to determine what levels will result in a change of preference for the therapeutic strategy. This is a way to test the boundaries of the model and identify the main parameters driving cost differences.

One-way Sensitivity Analysis

One-way sensitivity analyses were performed varying the main input parameters (additional ICU LOS due to CRBSI (days), CHG Dressing cost, number of CHG dressing per day, number of Non-CHG dressing per day, and cost per ICU day) of the model around the base case assumptions.

A tornado diagram (Figure 10) shows the variation in the mean cost difference between the CHG and non-CHG strategies around the one calculated for the base-case (€141). The model was most sensitive to the variation of the number of extra ICU LOS due to CRBSIs. The cost difference varied of approximately €370, when accounting from a single extra ICU day (cost difference of €251) to 26 extra ICU days (cost difference of €-115). The next three influential parameters were the CHG-dressing cost, the interval for dressing change, and the cost per ICU-day. However, the variation in the cost differences obtained by changing these parameters was less pronounced (differences between upper and lower limits of 88, 85 and €83, respectively).

Probabilistic Sensitivity Analysis (PSA)

A probabilistic sensitivity analysis [START_REF] Weinstein | Principles of good practice of decision analytic modeling in health care evaluation: Report of the ISPOR Task Force on Good Research Practices-Modeling Studies[END_REF] was performed with 1,000 non-homogeneous MCMC simulations of 1,000 patients for both CHG dressing and non-CHG dressing groups. Each group of 1,000 patients depicts an average patient representing all patients for each dressing group studied in the RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF]. The method used was the Gibbs sampling [79], a commonly used Markov Chain Monte Carlo algorithm. It allowed to retrace 10 6 health trajectories (1000*1000 patients for each dressing strategy), based on the probabilities observed in the RCT [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF] day-after-day (during 30 days) for each patient to change from one health-state to another. Repeating the algorithm 1,000 times allows the calculation of 95% confidence intervals for the costeffectiveness criterion (here, number of CRBSI avoided and cost per patient).

The health states including CRBSI (CRBSI/No new catheter and CRBSI/new catheter) as rare events for both strategies are in the area of low probabilities. On the other hand, the "discharge" and "death" states as frequent events for both strategies are in the area of high probabilities. This corresponds to the reality observed in the RCT (higher frequency of discharge and death than CRBSI).

The PSA cost-effectiveness plan (Figure 11) describes the effectiveness difference on the x-axis and the cost difference on the y-axis between the two groups of dressings, for 1,000 NH-MCMC simulations of 1,000 patients in each group. The (0,0)-point indicates the reference dressing strategy (Non-CHG group). All the points observed on the graph represent the incremental cost-effectiveness ratio (ICER) of CHGdressing strategy versus reference dressing. This PSA supports the decision to adopt CHG dressing for critically ill patients since the strategy is 99.8% more effective than the comparator at the same cost per patient in the intensive care unit. The mean ICER calculated from the PSA and defined as the cost per patient treated with chlorhexidine dressing to prevent one patient experiencing a CRBSI, is €12,046.

Convergence of MCMC

A large number of methods of convergence diagnostic methods for Markov Chain Monte Carlo algorithm (MCMC) is available in the literature [START_REF] Brooks | MCMC Convergence Diagnosis via Multivariate Bounds on Log-Concave Densities[END_REF], [START_REF] Asmussen | Stationarity Detection in the Initial Transient Problem[END_REF], [START_REF] Brooks | Alternative Methods for Monitoring Convergence of Iterative Simulations[END_REF].

To illustrate the convergence of MCMC consider that we will estimate only the cumulative number of patients that develop a CR-BSI during 30 days in the ICU using CHG dressing. This event correspond to the rarest event represented in our study. Note that the true value of this parameter is 3 patients for 934. The convergence of all other events in our work could be treated in the same way.

In order to reduce the possibility of bias due to the effect of starting values for Markov Chains and "burn in" period we conducted a Monte Carlo simulations of 1,000 replicates, each with N=1,000 individuals (1,000,000 individuals overall).

Here we cannot perform just one long run of single chain as the convergence times are not available [START_REF] Geyer | Practical Markov Chain Monte Carlo[END_REF], [START_REF] Tierney | Markov Chains for Exploring Posterior Distributions[END_REF]. The option is to take several shorter runs of a number of independent chains and form a sample from these observations [START_REF] Gelman | Inference from Iterative Simulation using Multiple Sequences[END_REF]. In the database the number of patients in CHG and No-CHG group was 934 and 935. So the choice of sampler for Markov Chain that is 1,000 patients is coherent. The number of independent replications to be run should be estimated.

We propose to run several chains in parallel and to estimate the parameter of interest (see Figure 12 and Figure 13). The convergence can be achieved when the estimations of parameter from the individual chains become indistinguishable. We can see that had we only 100 or 500 iterations of our Markov chain we might easily underestimate or overestimate the occurrence of CR-BSI event. Based on 1,000 iterations the estimator of our parameter converge to the true value. By continuing the number of replications beyond 1,000 (in graphs 2,000 and 5,000 iterations) we can be reasonably certain that convergence of our Markov chain was achieved after 1,000 iterations.

Incremental net monetary benefit

The average incremental net monetary benefit (iNMB) of €344 per patient induced by a daily CHG dressing use in patients resident in intensive care units was calculated, (Table 17). The iNMB is calculated from the difference between the cost induced by CHG dressing use (mean cost CHG patient -mean cost Non-CHG patient) and the cost averted by CHG use (differential of effectiveness per patient x mean cost per CRBSI Non-CHG patient). If mean cost per CRBSI Non-CHG patient is considered as the current willingness to pay (WTP) for treating one patient with CRBSI, the incremental net monetary benefit of CHG dessing use for the ICU is as follow:

= ∆ * -∆ . If the "incremental net monetary benefit" is positive, this indicates that the assessed technology is cost-effective.

Comparison of two models. Discussion

In this work, we performed the two approaches homogeneous and non-homogeneous Markov models.

The first homogeneous (time-independent) approach which is closely associated with the non-homogeneous Markov model developed after. In each dressing strategy (CHG, No-CHG), homogeneous MCMC simulations were performed to estimate the cost-effectiveness results. This homogeneous Markov chain Monte Carlo (H-MCMC) simulation was carried out with the statistical unit which is described in this study report as "global patient". The "global patient" is closest to the observed "ICU patient's health trajectories", he could be alive, discharged or dead during his ICU stay.

The CHG-dressing prevents 6. This non-homogeneous Markov Chain Monte Carlo (NH-MCMC) simulation was carried out with the statistical unit which is described in this study report as "global patient". Therefore, to take into account the "all causes death" state in our modeling decreases the effectiveness of the assessed new CHG-dressing because of the death rate is very important in ICU setting. Indeed, the NH-MCMC simulation on the "global patient" estimated 7 No-CHG died patients and 9 No-CHG discharged patients more than in the CHG group. As these two states are the "absorbent" states of the model, their cost are zero. As a consequence, our results could be considered as a "conservative" scenario.

For taking into account this matter, we performed a NH-MCMC simulation subgroup analysis with alive patients only. In each Dressing Strategy (CHG, No-CHG), a non-homogeneous MCMC simulation was performed for a subgroup analysis: "Global" patients with CR-BSI versus "Global" patients without CR-BSI. These NH-MCMC simulations were based on observed patient data, collected during the Dressing 2 clinical study. In this study, all patients were randomly assigned to one of the two dressing strategies, what allow us to assume comparability of the groups.

Discussing further on the comparability of these two subgroups, we can present two additional adjustments.

First, a "natural" adjustment, linked to the main statistical unit of our modeling which is the "global" patient was considered. The probability of developing a CR-BSI in the "Global population", that corresponding to the clinical trial population, follows the same plausible statistical distribution law (due to the randomization).

The statistical analysis for all confounding covariates, such as age, sex, severity (SOFA score), duration of catheterization, number of dressing change per day, shows the comparability between these subgroups. Nevertheless, a potential limit of the model can come from the covariates which are not available in the database. Such variables as smoking status, obesity, diabete, etc. could induce the important bias.

A further adjustment from NH-MCMC simulation was performed, considering covariates which could impact mainly the values closely linked to the costeffectiveness results. The rate of catheter change and the number of additional ICU days due to CRBSI in each dressing strategy were taken into account.

Comparison of two approaches

The difference in clinical outcomes between each dressing strategies was statistically significant with both models while cost differences were not. The PSA with the NH-MCMC resulted in 11.8 infections avoided per 1,000 patients (95%CI: Comparing the results of two models the effectiveness result of CHG is lower than in No-CHG dressings in the first model. In the H-MCMC model the costs difference is in favor of CHG dressing because the base case scenario eliminates all the differences between the strategies that correspond to the death and discharge health states for ICU patients. In the non-homogeneous model a "conservative" scenario was considered.

The NH-MCMC simulation on the "global patient" estimated 7 No-CHG died patients and 9 No-CHG discharged patients more than in the CHG group.

Nevertheless it is still possible to enter the different rates as the inputs in the homogeneous model. Thus, the principal limit of this study is the time-independence of the Markov process that means that we have accepted the same transition probabilities over the time spending in ICU.

Figure 1 . 1 :

 11 Figure 1.1: Kernel density estimation for small samples

Figure 1 . 2 :

 12 Figure 1.2: Kernel density estimation for large samples

Figure 1 . 4 :

 14 Figure 1.4: Log-likelihood function for 1 000 observations and ϑ = 0.25

Figure 1 . 5 :

 15 Figure 1.5: Log-likelihood function for 100 000 observations and ϑ = 0.25

Figure 1 . 6 :

 16 Figure 1.6: Density for Markov sequence

Figure 1 . 7 :

 17 Figure 1.7: Log-likelihood function for 1 000 observations and ϑ = 2.5

Figure 1 . 8 :

 18 Figure 1.8: Log-likelihood function for 10 000 observations and ϑ = 2.5

Gaussian kernel density for theta=0. 5 Figure 1 . 9 :

 519 Figure 1.9: Gaussian kernel density for 100 000 observations

Figure 1 . 10 :

 110 Figure 1.10: Three estimators of theta

5 .

 5 The information matrix I (ϑ) is uniformly in ϑ ∈ Θ non-degenerate and bounded 0 < inf ϑ∈Θ inf |λ|=1 λ T I(ϑ)λ, sup ϑ∈Θ sup |λ|=1 λ T I(ϑ)λ < ∞.

2 -

 2 ϑ . Now the proof of the Theorem 4 follows from the proof of the Theorem 3.

Figure 2 . 1 :

 21 Figure 2.1: Log-likelihood function for 1 000 observations Further we construct the one-step MLE-process ϑ ⋆ n = ϑ ⋆ k,n , N + 1 ≤ k ≤ n based on this preliminary estimator θN as follows.

Figure 2 . 2 :

 22 Figure 2.2: One-step MLE-process for 100 000 observations

Figure 2 . 3 :

 23 Figure 2.3: One-step MLE-process for n=1 000

Figure 2 . 4 :

 24 Figure 2.4: One-step MLE-process for n=10 000

Figure 2 . 5 :

 25 Figure 2.5: Second preliminary and two-step MLE-processes for n=1 000 and ϑ = 0.5

Figure 2 . 6 :

 26 Figure 2.6: Second preliminary and two-step MLE-processes for n=10 000 and ϑ = 0.5 Indeed, the estimator ϑ ⋆ k,n we can represent in the following way

3 .

 3 Modeling cost-effectiveness of antimicrobial dressings for preventing catheter-related bloodstream infection: homogeneous vs. non-homogeneous Markov approaches[START_REF] Maunoury | Modeling costeffectiveness of antimicrobial dressings for preventing catheter-related bloodstream infection: homogeneous vs non-homogeneous Markov approaches[END_REF]. 4. Cost-effectiveness analysis of an antimicrobial transparent dressing for protecting central vascular accesses in critically ill patients versus standard transparent dressings in France: A comparison of two modeling approaches: Decision-Tree versus Non-Homogeneous Markov Model (NHMM)[START_REF] Palka-Santini | Cost-effectiveness analysis of an antimicrobial transparent dressing for protecting central vascular accesses in critically ill patients versus standard transparent dressings in France: A comparison of two modeling approaches: Decision-Tree versus Non-Homogeneous Markov Model (NHMM)[END_REF].

Figure 1 .

 1 Figure 1. One-way Sensitivity Analysis: Mean cost difference per global patient between CHG and No-CHG strategies, in euros; part 1.

Figure 2 .

 2 Figure 2. One-way Sensitivity Analysis: Mean cost difference per global patient between CHG and No-CHG strategies, in euros; part 2.

Figure 3 .

 3 Figure 3. One-way Sensitivity Analysis: Mean cost difference per global patient between CHG and No-CHG strategies, in euros; part 3.

Figure 4 .

 4 Figure 4. Cost-effectiveness results for the probabilistic sensitivity analysis using 1,000 homogeneous Markov-Chain Monte Carlo simulations of 1,000 patients.

  The possible transitions among health states from one day to the next are represented in the Markov diagram (see figure below) was censored beyond 30 days. The current model comprises a time horizon of 30 days in ICU, each day corresponding to 1 Markov cycle.

Figure 5 .

 5 Figure 5. The structure of the Markov Model showing the possible transition between health states from one Markov cycle to the next cycle.

Figure 6 .

 6 Figure 6. Number of CRBSIs for 1,000 patients in each dressing group (results for the first 100 of the 1,000 simulations).

Figure 7 .

 7 Figure 7. Percentage of discharged or dead patients in each dressing group.

Figure 8 .

 8 Figure 8. Cost per patient per ICU-day for each dressing group.

Figure 9 .

 9 Figure 9. Mean cost per patient in each dressing group (results for the first 100 of the 1,000 simulations).

Figure 10 .

 10 Figure 10. Tornado diagram for the one-way sensitivity analysis.

Figure 11 .

 11 Figure 11. Cost-effectiveness results for the probabilistic sensitivity analysis using 1,000 nonhomogeneous Markov-Chain Monte Carlo simulations of 1,000 patients.

Figure 12 Figure 13 .

 1213 Figure 12. Convergence of Markov Chain

  Mean net saving is calculated as follow: Cost induced -Cost averted = (Mean cost CHG patient -Mean cost No-CHG patient) -(mean effectiveness difference per patient x mean cost per CR-BSI No-CHG patient).

  [ 3.85; 19.64]) and a mean extra cost of €141 per patient (95%CI: [€-975; €1,258]) when using antimicrobial dressing. The PSA with the H-MCMC resulted in 6.45 infections avoided per 1,000 patients (95%CI: [0.15; 12.75]) and the mean extra cost of €252 per patient (95%CI:[€-924; €1,428]).

  un premier cathéter, aucune infection liée aux cathéters}; E 2 = {Insertion d'un nouvel cathéter, aucune infection liée aux cathéters}; E 3 = {Infection liée aux cathéters sans l'insertion d'un nouvel cathéter}; E 4 = {Infection liée aux cathéters avec l'insertion d'un nouvel cathéter}; E 5 = {Dermatite de contact}; E 6 = {Changement du pansement}; E 7 = {Décharge}; E 8 = {Décès} .

1 .

 1 Motrunich, A. On parameter estimation for Markov sequences. Soumis, 2015.

	Les posters et présentations orales :
	1. On parameter estimation for Markov sequences. Asymptotical Statistics of
	Stochastic Processes. Workshop : S.A.P.S. X, Le Mans, 2015.
	2. Modeling cost-effectiveness of antimicrobial dressing for preventing catheter-
	related bloodstream infection : homogeneous vs non-homogeneous Markov
	approaches. International Society for Pharmacoeconomics and Outcomes Re-
	search (ISPOR), 17th Annual European Congress, Amsterdam, 2014.
	3. Cost-effectiveness analysis of an antimicrobial transparent dressing for pro-
	tecting central vascular accesses in critically ill patients versus standard
	transparent dressing in France : A comparison of two modeling approaches :
	Decision-tree versus non-homogeneous Markov model. International Society
	for Pharmacoeconomics and Outcomes Research (ISPOR), 17th Annual Eu-
	ropean Congress, Amsterdam, 2014.
	2. Kutoyants, Y.A. and Motrunich, A. On multi-step MLE-process for Markov
	sequences. Soumis à Metrika. (Accepté pour publication avec les modifica-
	tions légères) 2015.
	3. Maunoury, F., Motrunich, A., Palka-Santini, M., Bernatchez, S.F., Ruckly,
	S., Timsit, J.F. Cost-effectiveness analysis of a transparent antimicrobial
	dressing for managing central venous and arterial catheters in intensive care
	unit. PLoS ONE 2015 ; 10(6) : e0130439. doi :10.1371/journal.pone.0130439.

4. Non-homogeneous cost-effectiveness modeling of a new CHG-dressing for preventing catheter-related bloodstream infections for patients in intensive care units. International Society for Pharmacoeconomics and Outcomes Research (ISPOR), 16th Annual European Congress, Dublin, 2013. 5. Cost-effectiveness of the TLC-NOSF dressing in venous leg ulcers. International Society for Pharmacoeconomics and Outcomes Research (ISPOR), 15th Annual European Congress, Berlin, 2012.

Table 1 . Comparability of subgroups on covariates

 1 CHG group frequencies: 9 patients with CRBSI, 925 patients without CRBSI ** Non-CHG group frequencies: 21 patients with CRBSI, 914 patients without CRBSI ⱡ The results (p value) of Mann-Whitney tests on these covariates between subgroups show no

	Dressing group	CHG * Mean (std)	Non-CHG ** Mean (std)	Comparison p-value ⱡ
	SOFA score (severity)
	CRBSI	7.89 (4.08)	10.29 (3.39)	0.1459
	No CRBSI	8.17 (3.76)	8.17 (3.83)	0.8737
	Age (years)
	CRBSI	58.78 (13.73)	62.57 (19.08)	0.5262
	No CRBSI	61.97 (15.71)	62.17 (16.42)	0.6043
	Number of males
	CRBSI	5 (55.56%)	12 (57.14%)	1.0000
	No CRBSI	630 (68.11%)	603 (65.97%)	0.3460
	Catheterization time (days)
	CRBSI	39.67 (22.58)	28.43 (31.56)	0.0984
	No CRBSI	11.01 (11.52)	10.92 (11.01)	0.9934
	Number of dressings per day
	CRBSI	0.59 (0.29)	0.73 (0.37)	0.2675
	No CRBSI	0.67 (0.52)	0.65 (0.58)	0.2653

*

Table 2 . Health states for H-MCMC model defined from a multicentre randomized controlled trial

 2

	Health States	Definition
	1. No CRBSI / No new CT	Insertion of a first catheter, no diagnosed CRBSI and no
	needed	contact dermatitis
	2. CRBSI / No new CT	CRBSI diagnosed without neither contact dermatitis nor
	needed	the need for inserting a new catheter
	3. Contact dermatitis	No diagnosed CRBSI, and no need for new catheter
		inserted but occurrence of contact dermatitis
	4. Dressing Gauze and Tape Change to an alternative dressing strategy (gauze and
		tape) due to contact dermatitis
	5. Discharge	Patient leaves the ICU alive
	6. Death	Patient dies during the ICU stay
	CRBSI: Catheter-related Bloodstream Infections; CT Catheter: Central venous or radial / femoral
	arterial

Table 3 . Costs items for each Markov state

 3

	Main costs	Detailed costs	no CR-BSI/ no CT new	CR-BSI/ no CT new	Contact dermatitis	Dressing G&T	Death Discharge
	Cost of dressings		X	X	X	X
	Cost of
	treating contact				X
	dermatitis
		Four
		standard			X
		dressings
		Removal
		of the			X
		catheter
		Insertion
		of a new			X
		catheter
	Cost of
	treatment			X
	of CR-BSI
	Additional ICU -LOS			X

Table 4 . Costs per Markov state per patient from the base case scenario

 4

	Markov State	Costs for 1 patient CHG, Euro 2013	Costs for 1 patient No-CHG, Euro 2013
	No CRBSI/noCTnew	1,268	1,266
	CRBSI/noCTnew	13,659	13,657
	Contact dermatitis	1,387	1,385
	Dressing G+T	1,266	1,266
	Discharge	0	0
	Death	0	0
	3.4.1.4 Results of cost-effectiveness analysis
	The results presented below are from the base case scenario of the cost-effectiveness
	modeling. The structure follows the next principal sections:
	• Base case scenario results for 1,000 H-MCMC of 1,000 patients
	CHG group
	• Base case scenario results for 1,000 H-MCMC of 1,000 patients
	No-CHG group

Table 5 . Mean number of CRBSI for 1,000 patients in each dressing group -Horizon: 30- days ICU

 5

	Stat.	CHG group (1) No-CHG group (2)	Diff. (1-2)
	Number of CRBSI	1.8	8.42	-6.62
	CRBSI occurred for almost 2 and 9 patients in each CHG and non-CHG groups
	respectively (considering 1,000 patients in each group).

Table 6 . Mean Cost for 1 patient in each dressing group -Horizon: 30-days ICU

 6

	Stat.	CHG group (1) No-CHG group (2)	Diff. Cost (1-2)
	Mean cost	€ 21,748	€ 21,803	€ -55

Table 7 . Mean Number of CR-BSI for 1,000 patients in each dressing group -Horizon: 30-days ICU -1,000 MCMC simulations of 1,000 patients

 7

	Stat.	CHG group (1)	No-CHG group (2)	Diff. Effectiveness (1-2)
	Mean	1.8	8.30	-6.50
	Lower 95%CI	0	2.77	-12.57
	Upper 95%CI	4.44	13.83	-0.43

Table 8 . Mean Cost for 1 patient in each dressing group -Horizon: 30-days ICU -1,000 MCMC simulations of 1,000 patients Stat. CHG group (1) No-CHG group (2) Diff. Cost (1-2)

 8

	Mean	€21,770	€21,819	€-49
	Lower 95%CI	€20,925	€20,935	€-1,252
	Upper 95%CI	€22,616	€22,704	€1,153

Table 9 . Health states for NH-MCMC model defined from a multicentre randomized controlled trial

 9

	Health States	Definition
	1. No CRBSI / No new CT	Insertion of a first catheter, no diagnosed CRBSI and no
	needed	contact dermatitis
	2. No CRBSI / new CT	No diagnosed CRBSI, no contact dermatitis and a new
	needed*	catheter inserted (not as a replacement)
	3. CRBSI / No new CT	CRBSI diagnosed without neither contact dermatitis nor
	needed	the need for inserting a new catheter
	4. CRBSIs / new CT	CRBSI diagnosed without contact dermatitis but the need
	needed*	for inserting a new catheter
	5. Contact dermatitis	No diagnosed CRBSI, and no need for new catheter
		inserted but occurrence of contact dermatitis
	6. Dressing Gauze and	Change to an alternative dressing strategy (gauze and tape)
	Tape
	7. Discharge	Patient leaves the ICU alive
	8. Death	Patient dies during the ICU stay
	* New CT needed can mean either the replacement of the existing catheter, or the need for an
	additional catheter at a new site.
	CRBSI: Catheter-related Bloodstream Infections; CT Catheter: Central venous or radial / femoral
	arterial

Table 10

 10 reportes the costs included in the analysis for each Markov state.

Table 10 . Costs items for each Markov state Main costs Detailed costs / Data Provider

 10 Cost of treatment of CRBSI[START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF] and additional ICU-LOS due to CRBSI[START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF],[START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF] were taken into account for health states 3 and 4;• Cost per catheter change (venous, arterial)[START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF] was taken into account for health states 2 and 4.The costs per patient for each health state were calculated in both CHG and No-CHG dressing and presented in Table11.

			no CR-	no CR-	CR-BSI/	CR-	Contact	Death Disch
			BSI/ no	BSI/ CT	no CT	BSI/	dermatiti	arge
			CT new	new	new	CT new	s
	Cost of dressings	CHU Grenoble	X	X	X	X	X
		Time needed per dressing	X	X	X	X	X
		Number of
		nurses	X	X	X	X	X
		involved
		Material used	X	X	X	X	X
	Cost of treating dermatitis contact	CHU Grenoble/ 2012 Schwebel Dressing 1/					X

Table 11 . Costs per Markov state per patient from the base case scenario Costs for 1 patient CHG, Euro 2013 Costs for 1 patient No- CHG, Euro 2013

 11

	NoAE/noCRBSI/noCTnew	1,270	1,266
	NoAE/noCRBSI/CTnew	1,364	1,361
	NoAE/CRBSI/noCTnew	13,661	13,658
	NoAE/CRBSI/CTnew	13,756	13,752
	Contact dermatitis	1,388	1,385
	Dressing G+T	1,266	1,266
	Discharge	0	0
	Death	0	0
	3.4.2.4 Results of cost-effectiveness analysis
	The results presented below are from the base case scenario of the cost-effectiveness
	modeling. The structure follows the next principal sections:
	• Base case scenario results for 1,000 NH-MCMC of 1,000 patients CHG
	group
	• Base case scenario results for 1,000 NH-MCMC of 1,000 patients No-
	CHG group

Table 12 . Base case scenario results for 1,000 NH-MCMC of 1,000 patients CHG group

 12

	Statistics	Mean	Lower 95%CI	Upper 95%CI
	State 2 No AE/no CRBSI/CT new	27.82%	24.18%	31.45%
	State 3 No AE/CRBSIs/no CT new	0.00%	0.00%	0.00%
	State 4 No AE/CRBSIs/CT new	0.31%	0.00%	6.48%
	State 5 AE/no CRBSI/no CT new	2.88%	1.46%	4.30%
	Number of ICU-days	12.91	12.30	13.52
	Number of days before State 7 Discharge	18.74	18.05	19.43
	Number of days before State 8 Death	25.17	24.49	25.85

Table 13 . Base case scenario results for 1,000 NH-MCMC of 1,000 patients Non-CHG group

 13

	Statistics	Mean	Lower 95%CI	Upper 95%CI
	State 2 No AE/no CRBSI/CT new	25.16%	21.88%	28.44%
	State 3 No AE/CRBSIs/no CT new	0.53%	0.07%	0.98%
	State 4 No AE/CRBSIs/CT new	0.95%	0.33%	1.57%
	State 5 AE/no CRBSI/no CT new	1.27%	0.44%	2.09%
	Number of ICU-days	12.72	12.12	13.32
	Number of days before State 7 Discharge	18.43	17.72	19.16
	Number of days before State 8 Death	25.28	24.64	25.92
	CRBSI occurred for 3 and 14 patients in each CHG and non-CHG groups respectively
	(1,000 patients in each group; Tables

Table 14 . Percentage of events for absorbing states for 1,000 NH-MCMC of 1,000 patients CHG group -Time Horizon: 30-days ICU Statistics % of State 7 (ICU Discharge) % of State 8 (Death)

 14

	Mean

Table 15 . Percentage of events for absorbing states for 1,000 NH-MCMC of 1,000 patients Non- CHG group -Time Horizon: 30-days ICU

 15

	Statistics	% of State 7 (ICU Discharge)	% of State 8 (Death)
	Mean	61.34%	27.07%
	Lower 95%CI	58.28%	24.24%
	Upper 95%CI	64.41%	29.90%

Table 16 . Mean Cost for one patient in each dressing group. Time Horizon: 30-days ICU -1,000 NH-MCMC simulations of 1,000 patients

 16

	Groups /Statistics	Mean	Lower 95%CI	Upper 95%CI
		ALL PATIENTS
	CHG (1)	€16,461	€15,659	€17,265
	Non-CHG (2)	€16,320	€15,538	€17,103
	Diff. Cost (1-2)	€141	€-975	€1,258
		PATIENTS with CRBSI in ICU
	CHG (1)	€39,071	€17,384	€60,758
	Non-CHG (2)	€41,424	€36,213	€46,635
	Diff. Cost (1-2)	€-2,353	€-24,984	€20,277
		PATIENTS without CRBSI
	CHG (1)	€16,385	€15,584	€17,186
	Non-CHG (2)	€15,946	€15,177	€16,715
	Diff. Cost (1-2)	€439	€-664	€1,542
	For a 30-day time horizon in ICU, the mean cost per patient for CHG group was of
	€16,461, versus €16,320 for the non-CHG strategy. The mean cost per patient with
	CRBSI was of €39,071 and €41,424 in CHG and non-CHG dressing groups while the
	mean cost per patient without CRBSI was of €16,385 and €15,946 in CHG and non-
	CHG dressing groups, respectively (see Table

Table 17 . Incremental net monetary benefit (iNMB) for 1 patient induced by CHG use - Horizon: 30-days ICU -1,000 NH-MCMC simulations of 1,000 patients Statistics (iNMB) Cost-effective, if positive sign

 17

	Mean

  5 infections / 1,000 patients (95% CI: [0.43; 12.57]) as estimated via probabilistic cost-effectiveness sensitivity analysis in the proposed CEA. The mean adjusted cost per "global" patient is €21,770 (95% CI: [€20,925; €22,616]) for the CHG-dressing group and €21,819 (95% CI: [€20,935; €22,704]) for the reference dressing. Mean cost difference per "global" patient is €-49:(95% CI: [€-1,252; €1,153]).The second model that we performed is a non-homogeneous Markov model which is very linked to the Dressing 2 individual observed data. This time-dependent transition model allows simulating, with a high goodness of fit, the "health trajectory" of individuals observed in the Dressing 2 study.The CHG-dressing prevents 11.75 infections /1,000 patients (95% CI:[3.85;19.64], number needed to treat = 85) as estimated via probabilistic costeffectiveness sensitivity analysis. The mean adjusted cost per "global" patient is € 16,462 (95% CI: [€15,659; €17,265]) for the CHG-dressing group and €16,320 (95% CI: [€15,538; €17,103]) for the reference dressing. Mean cost difference per

"global" patient is +€141 (95% CI: [€-975; €1,258] and mean net saving per patient is €-344.88 (95% CI: [€-1,572.77; €883.01]).

  The effectiveness result is -9.44 /1,000 patients (95% CI: [-16.58; -2.29]) CR-BSI averted; cost difference per alive patient is +€618.43 (95% CI: [€-725.16; €1,962.01]) and mean net over cost per alive patient is +€226.39 (95% CI: [€-1,215.32; €1,668.11]), comparatively to the No-CHG group. This result is based on the fact that mortality is higher in the No-CHG group (see the results above) and this outcome is considered as zero-cost absorbing state. Therefore, we could achieve another NH-MCMC simulation subgroup analysis with patients neither dead nor discharged, but, in this sample, no CRBSI event was recorded.

[X j -(X j-1 -(2+ϑ)A(X j-1 )] 2is the density function of X n .

Dr. Maria Palka-Santini for her expert medical explications concerning the application part of my research.

Finally, I appreciate the financial support from Statésia. Also this work was done under partial financial support of the grant of RSF number 14-49-00079. , Dmitry Tolstoy, thank you, for your love.

--------------------

Base case input parameters considered in the cost analysis

The base case analysis is the most representative case of the real life, considering French ICU settings, and depending on expert opinions, literature, and RCTs. The main input parameters considered in the cost analysis are the following:

• Dressing costs per day: CHG dressing which is 20 times more expensive than non-antimicrobial transparent film and 60 times costly than gauze and tape. Direct costs for the treatment of CRBSIs were obtained from a micro-costing study [START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF]. ICU costs were based on an observational (real life) study [START_REF] Garrigues | Étude CRRéa: Évaluation médico-économique du Coût Réel d'une journée en RÉAnimation (Medico-economic evaluation of the Real Cost of a day in ICU)[END_REF] that assessed all resources consumed during a patient day in the ICU. This twenty-four hours multicentre prospective medico-economic study provides a complete overview and estimation of the actual average cost for medical and surgical ICUs in different hospital types in France: Hospitals (CH), University Hospitals (CHU) and Regional Hospitals (CHR). Twenty-two ICUs were selected randomly and all costs for 109 patients were estimated. For patients with CRBSI, an additional cost [START_REF] Dimick | Increased resource use associated with catheter-related bloodstream infection in the surgical intensive care unit[END_REF] due to an extra ICU length of stay (LOS) was calculated (see next section).

Main Assumptions Used for the Cost Analysis

• The cost of CR-BSI is independent from the outcome (survival or death or discharge). For the analysis the main statistical unit is the "global" (survival or death or discharge) patient; • Catheter colonization with or without CR-BSI had no costs (after the diagnosis)

or adverse outcomes (colonization has been initially excluded from the model because it's not a "health-state". The costs for diagnosis are negligible compared to costs related to additional LOS. The costs related to replace a

Costs per Markov state per patient

The calculation of the cost for each Markov state per patient was done as follows (using the base case input parameters listed above):

• Dressing costs (including time needed per dressing, number of nurses involved, and materials used [START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF]) and cost per ICU day [START_REF] Garrigues | Étude CRRéa: Évaluation médico-économique du Coût Réel d'une journée en RÉAnimation (Medico-economic evaluation of the Real Cost of a day in ICU)[END_REF] were taken into account for health states 1-6; • Cost of treating contact dermatitis [START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF] -(including catheter removal, four alternative dressings, and insertion of a new catheter) was taken into account only for health state 5; • Cost of treatment of CRBSI [START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF] and additional ICU-LOS due to CRBSI [START_REF] Timsit | Randomized controlled trial of chlorhexidine dressing and highly adhesive dressing for preventing catheter-related infections in critically ill adults[END_REF], [START_REF] Schwebel | Economic evaluation of chlorhexidine-impregnated sponges for preventing catheter-related infections in critically ill adults in the Dressing Study[END_REF] were taken into account for health states 2.

The costs per patient for each health state were calculated in both CHG and No-CHG dressing and presented in Table 4.

Healthcare resource use and costs

Costs used within the model reflect the ICU perspective in France and consist of following components:

• drug acquisition costs,

• cost of treating the adverse events ,

• direct costs of treating the CR-BSI,

• costs due to ICU stay,

• cost per catheter change.

Base case input parameters considered in the cost analysis

The base case analysis is the most representative case of the real life, considering French ICU settings, and depending on expert opinions, literature, and RCTs. The main input parameters considered in the cost analysis are the following:

• Dressing costs per day: CHG dressing which is 20 times more expensive than non-antimicrobial transparent film and 60 times costly than gauze and tape. Direct costs for the treatment of CRBSIs were obtained from a micro-costing study. ICU costs were based on an observational (real life) study [START_REF] Garrigues | Étude CRRéa: Évaluation médico-économique du Coût Réel d'une journée en RÉAnimation (Medico-economic evaluation of the Real Cost of a day in ICU)[END_REF] that assessed all resources consumed during a patient day in the ICU. This twenty-four hours multicentre prospective medico-economic study provides a complete overview and estimation of the actual average cost for medical and surgical ICUs in different hospital types in France: Hospitals (CH), University Hospitals (CHU) and Regional Hospitals (CHR). Twenty-two ICUs were selected randomly and all costs for According to the base case scenario, the CHG-dressing significantly more efficacious to prevent CRBSI when compared to the reference dressing for the ICU patients, contributes to preserve patients' health capital at the same cost for the ICU.

The antimicrobial dressings are consistently more efficacious in preventing CRBSIs whatever the model used. The H-MCMC is less sensitive to simulate the real life of the ICU patients. Regardless the model approach chosen the antimicrobial strategy is more efficacious than the comparator, but its probability of being cost-effective is comparatively reduced with the H-MCMC. Time dependent approach (NH-MCMC) seems to be better adapted to model rare events as CRBSIs.