A Markov process is a stochastic process that satisfies the Markov property, in which the future is independent of the past given the present. We first consider a Markov process over the real line with values on a finite set, where the law is defined by exponentially distributed jumps and a transition measure according to which the location of the process at the jump time is chosen; or indistinctly by the generator matrix. We also study Piecewise deterministic Markov processes, a more complex process that consists on two sub-processes: one on a continuous-space and the other on a discrete-space, and together are a Markov process involving a deterministic motion punctuated by random jumps. In the case when there are multiple weakly irreducible classes and the generator matrix can be rewritten as a double scales generator for a small parameter , we present a method to approximate the process to a two-scales process: a slow-process on a reduced state space and fast-process inside each new class, and we prove an approximation error of the laws of order . We present simulation examples and an application to the sodium channel in the Hodgkin and Huxley model, where we separate the voltage-gates of type h and m into two different time scales.

• Π : I × I → [0, 1] is a Markov transition kernel such that Π(i, •) ∈ P(I), a probability measure in I for all i ∈ I, then, the process follows i) , for all n ≥ 0 and P X T n+1 = j| X Tn = i = Π(i, j), for all n ≥ 0 Theorem 1.1.1 (Markov property) The process (X t ) t is a Markov process; i.e. for all n ≥ 0 and 0 ≤ s 1 < . . . < s n < t and s ≥ 0 and i 1 , . . . , i n , i, j in I,

And it is homogeneous in time; i.e. for all t, s ≥ 0

The proof can be found in [3] (Theorem 31). In order to better understand the process, we present the following operator:

Definition The semigroup P t of the process (X t ) t is an operator defined by

for all bounded function f : I → R and where E i [f (X t )] stands for the expectation value of process f (X t ) with starting condition X 0 = i.

Definition

The infinitesimal generator L of the process (X t ) t is an operator defined by

for all bounded function f : I → R. Theorem 1.1.2 (Markov Process characterization) The law of the Markov process (X t ) t with states in I is either characterized by A. the local characteristics (λ, Π), B. the infinitesimal generator L, which is identified by the generator matrix Q

and it satisfies Q ij ≥ 0, for all i, j ∈ I and i = j

Markov process in discrete space and continuous time

A Markov process is a stochastic process that satisfies the Markov property, in which the future is independent of the past given the present. We consider a Markov process (X t ) t≥0 defined over the real line with values on a finite set, where the law is defined by exponentially distributed jumps with rate λ and a transition measure Π according to which the location of the process at the jump time is chosen; or indistinctly by the generator matrix Q.

In the case when there are multiple weakly irreducible classes and the generator can be rewritten as a double scale generator for a small parameter , we present a method to approximate the process to a two-scales process: a slow-process on a reduced state space and fast-process inside each new class.

Definition Markov process

Let consider a filtered probability space (Ω, F, (F t ) t , P) and let (X t ) t≥0 be a Markov process with values on I = {1, . . . , N } for some N ∈ N. Constructing the Markov process as in [START_REF] Graham | Stochastic Simulation and Monte Carlo Methods[END_REF], we define (T n ) n∈N the series of jumps time of X t such that T 0 = 0 and

T n = inf {t ≥ T n-1 |X t = X t-}
for all n ≥ 1; then (X t ) t≥0 is defined by:

     0 = T 0 < T 1 < T 2 < • • • < ∞ as lim n→∞ T n = ∞ X t = n≥0 X Tn 1 {Tn<t≤T n+1 }
and initial condition i ∈ I. The law of the process (X t ) t is defined by the its local characteristics (λ, Π), where

• λ : I → R + is the jump rate,
The generator and the local characteristics are related according to

Q ij = -λ(i)1 {i=j} + λ(i)Π(i, j)1 {i =j}
for all i, j ∈ I.

Proof (B. ⇒ A.) As presented in [START_REF] Graham | Stochastic Simulation and Monte Carlo Methods[END_REF] (Theorem 5.3), let consider the process X δ kδ k , equal to (X t ) t over a discretized grid-mesh of size δ > 0 and T 1 the first jump-time of X t , then it follows that

{T 1 > t/δ δ} ⊂ X δ 0 = X δ δ = • • • = X δ t/δ δ = x ⊂ {T 1 > t/δ δ} ∪ {T 2 -T 1 < δ}
where {T 2 -T 1 < δ} control the event of jumps between the mesh-grid, and as lim δ→0 P i (T 2 -T 1 < δ) = 0 by right continuity, we have

P i (T 1 > t) = lim δ→0 P X δ 0 = X δ δ = • • • = X δ t/δ δ = i
by the Markov property, it holds that

P X δ 0 = X δ δ = • • • = X δ t/δ δ = i = P X δ t/δ δ = i|X δ 0 = • • • = X δ ( t/δ -1) = i P X δ 0 = • • • = X δ ( t/δ -1)δ = i . . . = t/δ k=1 P X δ kδ = i|X δ 0 = • • • = X δ (k-1)δ = i = t/δ k=1 P X δ kδ = i|X δ (k-1)δ = i = P X δ δ = i|X δ 0 = i t/δ
= e ( t/δ log(P[X δ δ =i|X δ 0 =i])

and as around x ≈ 1 we have log(x) ≈ x -1, it follows that lim

δ→0 log P X δ δ = i|X δ 0 = i δ = lim δ→0 P X δ δ = i|X δ 0 = i -1 δ = lim δ→0 E i 1 {i} (X δ δ ) -1 {i} (i) δ = L1 {i} (i) = Q ii and we conclude P i [T 1 > t] = lim δ→0 e ( t/δ log(P[X δ δ =i|X δ 0 =i]) = e tQ ii
We deduce that λ(i) := -Q ii corresponds to the jump rate. On the other hand, for all j = i,

t/δ k=1 X δ 0 = X δ δ = • • • = X δ (k-1)δ = i, X δ kδ = j ⊂ {T 2 -T 1 < δ} ∪ T 1 ≤ t/δ δ, X 0 = i, X δ T 1 = j ⊂ {T 2 -T 1 < δ} ∪ t/δ k=1 X δ 0 = X δ δ = • • • = X δ (k-1)δ = i, X δ kδ = j
and hence the limit

P i [T 1 ≤ t, X T 1 = i] = lim δ→0 t/δ k=1 P X δ 0 = X δ δ = • • • = X δ (k-1)δ = i, X δ kδ = j
by the Markov property, it holds that

t/δ k=1 P X δ 0 = X δ δ = • • • = X δ (k-1)δ = i, X δ kδ = j = t/δ k=1 P X δ 0 = X δ δ = • • • = X δ (k-1)δ = i P X δ kδ = j|X δ 0 = X δ δ = • • • = X δ (k-1)δ = i = t/δ k=1 P i [T 1 > (k -1)δ] P X δ δ = j|X δ 0 = i δ δ
we observe that

lim δ→0 P X δ δ = j|X δ 0 = i δ = lim δ→0 E i 1 {j} (X δ δ ) -1 {j} (i) δ = L1 {j} (i) = Q ij
and we conclude

P i [T 1 ≤ t, X T 1 = j] = lim δ→0 t/δ k=1 P i [T 1 > (k -1)δ] P X δ δ = j|X δ 0 = i δ δ = t 0 e Q ii s Q ij ds = -P i [T 1 ≤ t] Q ij Q ii
and it follows that

P i [X T 1 = j| T 1 ≤ t] = P i [T 1 ≤ t, X T 1 = j] P i [T 1 ≤ t] = - Q ij Q ii We conclude that P i [X T 1 = j] = -Q ij /Q ii ; then -Q i• /Q
ii is a probability measure in I \ {i} for all i ∈ I, and immediately we have that

Q ij ≥ 0, for all i, j ∈ I and i = j Q ii = - i =j Q ij , for all i ∈ I
We deduce that the Markov kernel Π of the process is equal to

Π(i, j) :=    - Q ij Q ii 1 {Q ii =0} if i = j 1 {Q ii =0} if j = i (A. ⇒ B.
) As shown in [START_REF] Kourkova | Processus de Markov, applications[END_REF] (Theorem 31), when h ↓ 0

P i [X h = i] ≥ P i [T 1 ≥ h] = e -λ(i)h = 1 -λ(i)h + o(h)
and for all j = i

P i [X h = j] ≥ P i [T 1 ≤ h, X T 1 = y, T 2 -T 1 ≥ h] =
1 -e -λ(i)h Π(i, j)e -λ(j)h = λ(i)Π(i, j)h + o(h)

We observe that as the sum over y on the left side has to be 1, last inequalities have to be equalities. To prove this let suppose that one of the quantities (P i [X h = i] -1 + λ(i)h) /h or (P i [X h = j] -λ(i)Π(i, j)h) /h does not tend to 0. Let suppose it is the second one, then there exists δ > 0 and a sequence (h k ) k≥0 such that h k ↓ 0 and

|P i [X h k = j] -λ(i)Π(i, j)h k | /h k > δ;
it follows that exists K such that for all k ≥ K,

P i [X h k = j] -λ(i)Π(i, j)h k h k > δ
Then, as j∈I P i [X h = j] = 1 and j =i λ(i)Π(i, j) -λ(i) = 0, for all k > K it holds that 0

= P i [X h k = i] -1 + λ(i)h k h k + j =i P i [X h k = j] -λ(i)Π(i, j)h k h k > o(1) + (N -1)δ
where o(1) ↓ 0 when k ↑ ∞; and we arrive to the contradiction δ < 0. We conclude that

P i [X h = i] = 1 -λ(i)h + o(h) P i [X h = j] = λ(i)Π(i, j)h + o(h), for all j = i
Now, for all bounded f and h ↓ 0, we have

E i [f (X h )] = f (i)P i [X h = i] + j∈I\{i} f (j)P i [X h = j] = f (i) -f (i)λ(i)h + h j∈I\{i} f (j)λ(i)Π(i, j) + o(h)
and in order to compute the infinitesimal generator we have that

E i [f (X h )] -f (i) h = -λ(i)f (i) + j∈I\{i} f (j)λ(i)Π(i, j) + o(1)
taking the limit and o(1) ↓ 0 when h ↓ 0, it follows that

Lf (x) = lim h→0 E x [f (X h )] -f (i) h = j∈I f (j)Q ij where Q ij = -λ(i)1 {j=i} + λ(i)Π(i, j)1 {j =i} .
Remark Theorem 1.1.2 allows us to use indistinctly the local characteristics or the matrix generator when referring to the characterization of a Markov process.

An important property of the generator is that it satisfies the Dynkin formula,

P t f (i) = f (z) + t 0 P s Lf (i)ds (1.1)
And if we define the distribution p(•, t) for all t ≥ 0, such that

p(i, t) = P p 0 [X t = i]
for all i ∈ I and initial distribution p 0 , we can rewrite the expectation as the sum over the probability in equation (1.1), we have

d dt z∈I f (z)p(z, t) = z∈I j∈I f (j)p(z, t)Q zj
and setting f (z) = δ i , we derive the forward equation,

     d dt p(i, t) = j∈I p(j, t)Q ji p(i, 0) = p 0 (i) (1.2)
for all i ∈ I. Equation (1.2) can be solved directly, with a solution given by

p(•, t) = p 0 (•) exp (Qt)
in the sense that p(•, t) = (p(1, t), . . . , p(N, t)) and exp (tQ) = ∞ n=0 (tQ) n n! .

Simulation

In order to simulate a Markov process (X t ) t∈[0,T ] with values in I = {1, . . . , N }, initial distribution p 0 and local characteristics (λ, Π), the idea is:

• draw a sample i 0 from p 0 for the initial value,

• then iteratively, if the process is in state i,

1. draw a sample from E(λ(i)) exponential law for the duration at i, 2. draw the next state j = i from the law Π(i, •).

If λ(i) = 0 then the process is absorbed at i. To achieve this, we consider the following lemmas that allows us to draw samples of the distributions from the uniform distribution.

Lemma 1.1.3 Let λ > 0 and U ∼ U [0, 1], then the random variable ξ = -1 λ log (1 -U ) ∼ E(λ). Proof P [ξ ≤ t] = P - 1 λ log (1 -U ) ≤ t = P [U ≤ 1 -exp (-λt)] = 1 -exp (-λt) Remark As U ∼ 1 -U , we have also that ξ = -1 λ log (U ) ∼ E(λ) Lemma 1.1.4 Let U ∼ U [0, 1
] and i ∈ I, and we consider the random variable Y ∈ I defined by

Y =      1 if U ≤ Π(i, 1) i if i-1 w=1 Π(i, w) < U ≤ i w=1 Π(i, w), i = 2, . . . , N Then, it follows that Y ∼ Π(i, •).
Proof It holds immediately that P [Y = z] = Π(x, z), for all z ∈ I.

Multiple weakly irreducible classes

With the objective to perform a state reduction approximation for processes with multiple weakly irreducible classes, let consider a filtered probability space (Ω, F, (F t ) t , P) and let (X ) t≥0 be a Markov process with values on I = {1, . . . , N } for some N ∈ N, initial condition x ∈ I and characterized by a generator Q ∈ R N ×N . A process is considered to have multiple weakly irreducible classes if its generator depends on a small parameter > 0 and two generators Q and Q ∈ R N ×N such that

Q = 1 Q + Q (1.3)
where Q governs the rapidly changing part and Q describes the slowly changing components. We consider that Q can be put into block-diagonal form,

Q =      Q1 0 0 0 Q2 . . . 0 0 0 Q N     
where each Qk ∈ R m k ×m k is a generator for some integer m k , such that N k=1 m k = N , and it defines the class sk = {i k1 , . . . , i km k } ⊂ I that denotes the states associated with Qk . We define the new set S = {s 1 , . . . , s N } that will be space of the slow process; where we notice that N k=1 sk = I.

Assumption 1.2.1 For all k = 1, . . . , N , the generator Qk is weakly irreducible; that is the system of equations

           m k j=1 ν k j Qk ji = 0, for i = 1, . . . , m k m k i=1 ν k i = 1
has a unique non-negative solution.

We consider for all t ≥ 0 the probability measure p (•, t) ∈ P(E, E), such that

p (i, t) = P p 0 [X t = i] (1.4)
for all i ∈ I and initial probability p(•, 0) = p 0 . It holds that p is solution of (1.2), the forward equation

     d dt p (i, t) = j∈I p (j, t) 1 Q ji + Q ji p (i, 0) = p 0 (i) (1.5)
for all i ∈ I. The slow and fast components are coupled through weak and strong interactions in the sense that the underlying Markov chain fluctuates rapidly within a single group sk and jumps less frequently between groups sk and sp for k = p. The states in sk , k = 1, . . . , N , are not isolated or independent of each other; more precisely, if we consider the states in sk as a single state, then these states are coupled through the matrix Q, and transitions from sk to sp , k = p are possible. In fact Q, together with the quasi-stationary distributions ν k of Qk , determines the transition rates among states in sk , for k = 1, ..., N .

Asymptotic expansion

To start we present the following lemmas.

Lemma 1.2.2 Consider the matrix differential equation For each i = 1, . . . , m, denote the ith row of P by Pi . The weak irreducibility of Q then implies that the system of equations Pi Q = 0, Pi 1 = 1 has a unique solution. Since P is the limit of the transition matrix, Pi ≥ 0. As a result, Pi is the quasi-stationary distribution ν and P has identical rows with P = 1(ν 1 , . . . , ν m ).

d ds P (s) = P (s)Q P (0) = I (1.6) where P (s) ∈ R N ×N . Suppose Q ∈ R N ×N is
Using the Jordan canonical form, there is a nonsingular matrix U such that exp(Qs) = U diag (exp(J 0 s), exp(J 1 s), . . . , exp(J q s)) U -1

where J 0 , J 1 , . . . , J q are the Jordan blocks satisfying that J 0 is a diagonal matrix having appropriate dimension (if λ i is a simple eigenvalue of Q, it appears in the block J 0 ), and that

J k ∈ R m k ×m k , k = 1, . . . , q.
Since lim s→∞ exp(Qs) exists, all the nonzero eigenvalues λ i , for 1 ≤ i ≤ m -1, must have negative real parts. Moreover, in view of the weak irreducibility of Q, the eigenvalue zero is a simple eigenvalue (having multiplicity 1). Then it is easily seen that exp(Qs) -P ≤ K exp(-κs)

where κ = (1/2) max 1≤i≤m1 Re(λ i ).

Lemma 1.2.3 (Gronwall's lemma) If f is a positive locally bounded Borel function on R + such that

f (t) ≤ a + b t 0 f (s)ds
for every t and two constants a and b, then f (t) ≤ a exp(bt).

Proof We have

f (t) ≤ a + b t 0 a + b s 0 f (u)du ds = a + abt + b 2 t 0 (t -u)f (u)du ≤ a + abt + b 2 t t 0 f (u)du
Proceeding inductively we get

f (t) ≤ a + abt + . . . + ab n t n n! + b n+1 t n n! t 0 f (u)du
Since f is locally bounded, the last term on the right converges as n tend to infinity and the result follows.

Now we present the most important theorem of the section, that presents a characterization for the probability measure of the process (X t ) t , based on the results given in [START_REF] Yin | Continuous-Time Markov Chains and Applications[END_REF].

Theorem 1.2.4 (Asymptotic Expansion) The probability measure p (•, t) (1.4) of the process (X t ) t can be expanded in the form:

p (i, t) = ϕ(i, t) + γ i, t + e (i, t) (1.7)
In this approach ϕ is set to be an approximation on the slow-scale t away from 0, γ approximate the fast-scale τ = t/ and e corresponds to the error of the expansion. The functions ϕ, γ and e are such that

• ϕ(i, t) is differentiable for all t ∈ [0, T ] • there is a κ 0 > 0 such that |γ (i, τ )| ≤ K exp (-κ 0 τ )
uniformly for all i ∈ I

• and the following estimate holds

sup t∈[0,T ] |e (i, t)| ≤ K uniformly for all i ∈ I
To proceed, we define an operator L by

L f = df dt -f Q + Q (1.8)
for any smooth row-vector-valued function f ; then L f = 0 iff it is a solution to the forward differential equation (1.2). We set that both ϕ and γ are solution to the forward equation, then they satisfy

L ϕ(t) = 0 and L γ t = 0 that is, d dt ϕ(i, t) = j∈I ϕ(j, t) Qji + Qji d dt γ i, t = j∈I γ y, t Qji + Qji (1.9)
and we write a new time scale τ = t/ for the second equation

d dτ γ(i, τ ) = j∈I γ(j, τ ) Qji + Qji
And if we identify the terms over 0 and 1 in the time-scale t and setting that γ(τ ) must not depends on , we have the following set of equations

         d dt ϕ(i, t) = j∈I ϕ(j, t) Qji j∈I ϕ(j, t) Qji = 0 and    d dτ γ(i, τ ) = j∈I γ(j, τ ) Qji
Remark From the last term comes the approximation error of this expansion, and due the fact that Q is not weakly irreducible.

In order to match asymptotic expansion, we have necessarily at t = 0 that

p 0 (i) = ϕ(i, 0) + γ(i, 0)
Sending → 0 in the asymptotic expansion (1.9), the fast-scale and error disappear and only remains the slow-scale, then as i∈I p (i, t) = 1 for all > 0, we have lim

→0 i∈I p (i, t) = i∈I ϕ(i, t) = 1
Remark What it is presented here is a zero level asymptotic expansion for a generator that does not depend on time. On a different case as shown in [START_REF] Yin | Continuous-Time Markov Chains and Applications[END_REF], when Q (t) depends on time we would do an n-level asymptotic expansion on the form

p (i, t) = Φ n (i, t) + Γ n t + e n (i, t) = n i=0 i ϕ n (i, t) + n i=0 i γ n i, t + e n (i, t)
with a remainder of order n . It would be necessary to add condition to Q (t), such that it is n-times continuously differentiable on [0, T ] with each derivative Lipschitz continuous.

Determining ϕ

We need to determine ϕ(i, t) for i ∈ I and t ∈ [0, T ] such that

                 j∈I ϕ(j, t) Qji = 0 d dt ϕ(i, t) = j∈I ϕ(j, t) Qji i∈I ϕ(i, t) = 1 (1.10) Since Q = diag Q1 , . . . ,
Q N where each Qk is weakly irreducible, therefore if we consider ϕ k (i, t), the function ϕ(i, t) restricted on i ∈ sk , we have that it satisfies

j∈s k ϕ k (j, t) Qk ji = 0
which solution is ϕ k (i, t) = θ(k, t)ν k i , the product of the invariant measure ν k of Qk and a scalar multiplier θ(k, t), a function defined for k = 1, . . . , N and t ∈ [0, T ]. We observe that as ν k is a distribution in sk , it holds that

i∈s k ϕ k (i, t) = i∈s k θ(k, t)ν k i = θ(k, t)
and by consequence of the third equation in (1.10), it also holds that N k=1 θ(k, t) = 1. Setting all this with the second equation in (1.10), it follows that

d dt θ(k, t) = i∈s k d dt ϕ(i, t) = i∈s k j∈I ϕ(j, t) Qij = i∈s k N p=1 j∈sp θ(p, t)ν p j Qji = N p=1 θ(p, t)   j∈sp x∈s k ν p y Qji   (1.11)
where we observe the emergence of a new generator

Q ∈ R N × N such that Qpk = j∈sp i∈s k ν p j Qji , (1.12)
for all p, k = 1, . . . , N . To determine the initial condition θ(k, 0), we first observe that in the asymptotic expansion it has to hold that

i∈s k ϕ(i, 0) = lim δ→0 lim →0 i∈s k p (i, δ) (1.13)
moreover, in view of the forward equation (1.5) and that i∈s k Qk ji = 0, which comes from the property of generators, we have

i∈s k p (i, t) = i∈s k p 0 (i) + δ 0 i∈s k j∈s k p (i, s) Qji ds and since p (i, t) is bounded it follows that lim δ→0   lim sup →0 δ 0 i∈s k j∈s k p (i, s) Qji ds   = 0 therefore by (1.13) it yields i∈s k ϕ(i, 0) = lim δ→0   lim →0 i∈s k p (i, δ)   = i∈s k p 0 (i)
and we finally have θ(k, 0) =

x∈s k p 0 (i)
To conclude this section we present the following Corollary, that allows us to understand more clearly the solution to ϕ.

Corollary 1.2. [START_REF] Davis | Markov Models and Optimization[END_REF] The system (1.10) for ϕ(i, t), is equivalent to the system

                           j∈s k ϕ(j, t) Qji = 0, for i ∈ sk i∈s k ϕ(i, t) = θ(k, t) d dt θ(k, t) = N p=1 θ(p, t) Qpk θ(k, 0) = i∈s k p 0 (i)
for k = 1, . . . , N ; where Q is the generator defined in (1.12).

Remark In Corollary 1.2.5 appears the function θ(k, t), which can be interpreted as the probability of a new Markov process (Z t ) t≥0 defined over the aggregate states space S = {s 1 , . . . , s N } with infinitesimal generator Q.

Determining γ

We consider γ(i, τ ), for all i ∈ I and t ∈ [0, T ], solution to

   d dτ γ(i, τ ) = j∈I γ(j, τ ) Qji (1.14)
To match the asymptotic expansion, we have at t = 0 that

p 0 (i) = ϕ(i, 0) + γ(i, 0)
and as Q is constant, we can solve (1.14) directly and together with the above initial condition, we obtain

γ(•, τ ) = (p 0 (•) -ϕ(•, 0)) exp Qτ (1.15)
Considering that for each k = 1, . . . , N , Qk is weakly irreducible, we need to prove that γ(i, τ ) can be obtain by equation (1.15), and there is a positive number κ 0 such that

|γ(i, τ )| ≤ K exp (-κ 0 τ )
uniformly for i ∈ I. To prove this, let ν k be the stationary distribution corresponding to the generator Qk . We define the column vector 1 m = (1, 1, . . . , 1) ∈ R m×1 and the matrix

π =      1 m 1 ν 1 0 0 0 1 m 2 ν 2 . . . 0 0 0 1 m N ν N     
where

1 m k ν k =    ν k 1 • • • ν k m k . . . ν k 1 • • • ν k m k    Noting the block-diagonal structure of Q, we have exp Qτ =         exp Q1 τ 0 0 0 exp Q2 τ . . . 0 0 0 exp Q N τ        
Furthermore, we see that for k = 1, . . . , N it holds

i∈s k (p 0 (i) -ϕ(i, 0)) = i∈s k p 0 (i) - i∈s k ϕ(i, 0) = i∈s k p 0 (i) -θ(k, 0) = 0
we conclude that the initial condition (p 0 (•)-ϕ(•, 0)) is orthogonal to π, and by virtue of Lemma 1.2.2, for each k = 1, . . . , N there exists

κ k > 0 such that exp Qk τ -1 m k ν k ≤ K exp (-κ k τ ) then we have |γ(•, τ )| = (p 0 (•) -ϕ(•, 0)) exp Qτ -π ≤ K sup k≤ N exp Qk τ -1 m k ν k ≤ K exp (-κ 0 τ ) where κ 0 = min k≤ N κ k .

Analysis of remainder

The remainder of the asymptotic expansion (1.7) corresponds to

e (i, t) = p (i, t) -ϕ(i, t) -γ i, t
where e (0) = 0, and if we consider the operator L as in (1.8),

L f = df dt -f Q + Q it holds that L p (t) = 0 and then L e (i, t) = - d dt ϕ(i, t) + d dt γ i, t + j∈I ϕ(j, t) + γ j, t Qji + Qji
and from equations (1.10) and (1.14), we have

L e (t) = γ i, t Q
expanding the operator and as

Q = Q/ + Q, we have d dt e (x, t) = j∈I e (j, t)Q ji + j∈I γ j, t Qji
taking the norm, integrating and making use of the exponential decay property of γ, it holds that

|e (i, t)| ≤ |e (i, 0)| + t 0 j∈I e (j, t)Q ji ds + t 0 j∈I γ j, t Qji ds ≤ C t 0 |e (i, s)| ds + K t 0 exp -κ 0 s ds ≤ C t 0 |e (i, s)| ds + K κ 0 1 -exp -κ 0 t ≤ C t 0 |e (i, s)| ds + K
and by Gronwall's lemma (1.2.3), we conclude

|e (i, t)| ≤ K exp (Ct)
and taking the supreme on time, the remainder satisfies

sup t∈[0,T ] p (i, t) -ϕ(i, t) -γ i, t = K
where K is a positive constant that depends on T .

Two-scales approximation

We now present a corollary that allows us, under the conditions already discussed, to represent the Markov process as a two-scales process: one on a slow-scale that goes over the state classes and a fast-scale that acknowledge the dynamic inside each class.

Corollary 1.2.6 Let (X ) t∈[0,T ] be a Markov process over I = {1, . . . , N }, initial distribution p 0 and with a two-scales generator Q ∈ R N ×N that depends on > 0 and two generators Q and Q such that

Q = 1 Q + Q where Q = diag( Q1 , . . . , Q N ) with each sub-generator Qk ∈ R m k ×m k weakly irreducible and
determines the class sk ⊂ I. Then there exists positive constants K T , K and κ 0 such that

P (X t = i kx ) -ν k x θ(k, t) ≤ K T + K exp -κ 0 t
for all i kx ∈ sk , for x = 1, . . . , m k and k = 1, . . . , N . Here ν k is the stationary distribution for class sk given by Qk ; and θ(k, t) is a function over k = 1, . . . , N and t ∈ [0, T ], that satisfies

           d dt θ(k, t) = N p=1 θ(p, t) Qpk θ(k, 0) = i∈s k p 0 (i) for the generator Q ∈ R N × N defined by Qpk = j∈sp i∈s k ν p j Qji for p, k = 1 . . . , N .
In this corollary, we can interpret θ(k, t) as the probability distribution of a Markov process (Z t ) t∈[0,T ] defined over S = {s 1 , . . . , s N } with generator Q, which holds the dynamics on the slow-scale. We can think of this process as

Z t = si ⇐⇒ X t ∈ si
and that θ(k, t) = P (Z t = sk ) for i = 1, . . . , N . On the other hand, the dynamic in the fast-scale will be defined punctually on the position of Z t by and random variable X fast such that

P X fast = i kx | Z t = sk = ν k x
for each i kx ∈ sk . We notice that its value depends on the slow process for each time when we want to evaluate it.

Example 1

Consider a two machine flow-shop with machines that are subject to breakdown and repair. The production capacity of the machines is described by a finite-state Markov chain. If the machine is up, then it can produce parts with production rate u(t); its production rate is zero if the machine is under repair. For simplicity, suppose each of the machines is either in operating condition (denoted by 1) or under repair (denoted by 0). Then the capacity of the workshop becomes a four-state Markov chain (X t ) t∈[0,T ] with state space I = {(0, 0), (1, 0), (0, 1), (1, 1)} and initial distribution p 0 . Suppose that the first machine breaks down much more often than the second one. To reflect this situation, consider that (X t ) t is generated by Q as (1.3) for a small > 0, with Q and Q given by

Q =     -λ 1 λ 1 0 0 µ 1 -µ 1 0 0 0 0 -λ 1 λ 1 0 0 µ 1 -µ 1     , Q =     -λ 2 0 λ 2 0 0 -λ 2 0 λ 2 µ 2 0 -µ 2 0 0 µ 2 0 -µ 2    
where λ i and µ i are the rates of repair and breakdown, respectively. We consider the probability

p (i, t) = P p 0 (X t = i)
that denote the probability distribution of the underlying chain at time t and it is solution of (1.2), the forward equation

   d dt p (•, t) = p (•, t) 1 Q + Q p (•, 0) = p 0
which can be solved directly; in particular for λ 1 = λ 2 = µ 1 = µ 2 = 1 and p 0 = (1, 0, 0, 0), the solution is

p (1, t) = 1/4 e -2 t + 1/4 e -2 t + 1/4 e -2 t(1+ ) + 1/4 p (2, t) = -1/4 e -2 t + 1/4 e -2 t -1/4 e -2 t(1+ ) + 1/4 p (3, t) = 1/4 e -2 t -1/4 e -2 t -1/4 e -2 t(1+ ) + 1/4 p (4, t) = -1/4 e -2 t -1/4 e -2 t + 1/4 e -2 t(1+ ) + 1/4
The matrices Q and Q are themselves generators of Markov chains. Note we that

Q = diag -λ 1 λ 1 µ 1 -µ 1 , -λ 1 λ 1 µ 1 -µ 1
with both generator weakly irreducible, with invariant distribution ν =

µ 1 µ 1 +λ 1 , λ 1 µ 1 +λ 1 . From generator Q we identify the classes s0 = {(0, 0), (1, 0)} = {i 01 , i 02 } s1 = {(0, 1), (1, 1)} = {i 11 , i 12 } (1.16)
that form the slow-scale states S = {s 0 , s1 }. The slow-scale generator is then given by Q = diag (ν, ν) Qdiag ((1, 1), (1, 1)), then

Q = -λ 2 λ 2 µ 2 -µ 2
By Corollary 1.2.6, we can approximate the law of process (X t ) t to

P p 0 [X t = i kx ] ≈ ν x P [Z t = sk ]
where (Z t ) t∈[0,T ] is a Markov process on the slow-scale with states in S and generator Q. We consider the probability θ(k, t) = P [Z t = sk ], and it is solution of the forward equation

d dt θ(•, t) = θ(•, t) Q θ(•, 0) = (p 1 (0) + p 2 (0), p 3 (0) + p 4 (0))
which can be solved directly; in particular for λ 2 = µ 2 = 1 the solution is

θ(s 1 , t) = 1/2 + (1/2)e -2t θ(s 2 , t) = 1/2 -(1/2)e -2t
Finally, the approximation, with ν = (1/2, 1/2), is equivalent to

p(1, t), p(2, t) ≈ 1/4 + 1/4 e -2 t p(3, t), p(4, t) ≈ 1/4 -1/4 e -2 t
In all four cases, the reminder is given by

|e (t)| = 1/4 e -2 t + 1/4 e -2 t(1+ ) ≤ 1/4 e -2t/ 1 + 1 1 + 2t ≤ /4 (2t ) e -2 t + 1/2 e -2 t ≤ /4 + 1/2 e -2 t

Example 2

Let define the Markov process (X t ) t∈[0,T ] with state space I = {1, . . . , 9}, initial distribution p 0 and generated by Q as in (1.3) for a small > 0, with Q and Q given by

Q = diag     -λ 1 λ 1 0 λ 1 -(λ 1 + µ 1 ) µ 1 0 µ 1 -µ 1   ,   -λ 1 λ 1 0 λ 1 -(λ 1 + µ 1 ) µ 1 0 µ 1 -µ 1   ,   -λ 1 λ 1 0 λ 1 -(λ 1 + µ 1 ) µ 1 0 µ 1 -µ 1    
with each sub-matrix weakly irreducible that solve for ν = (1/3, 1/3, 1/3), and

Q =               -λ 2 0 0 λ 2 0 0 0 0 0 0 -λ 2 0 0 λ 2 0 0 0 0 0 0 -λ 2 0 0 λ 2 0 0 0 λ 2 0 0 -(λ 2 + µ 2 ) 0 0 µ 2 0 0 0 λ 2 0 0 -(λ 2 + µ 2 ) 0 0 µ 2 0 0 0 λ 2 0 0 -(λ 2 + µ 2 ) 0 0 µ 2 0 0 0 µ 2 0 0 -µ 2 0 0 0 0 0 0 µ 2 0 0 -µ 2 0 0 0 0 0 0 µ 2 0 0 -µ 2              
We consider the probability function p (x, t) = P p 0 (X t = x) for x ∈ I, that denote the probability distribution of the underlying chain at time t, and it is solution of (1.2), the forward equation

   d dt p (•, t) = p (•, t) 1 Q + Q p(•, 0) = p 0 (1.17)
which can be solved directly; in particular for λ 1 = λ 2 = µ 1 = µ 2 = 1 and p 0 = (1, 0, 0, 0, 0, 0, 0, 0, 0), the solution is equal to

d dt p (1, t) = 1/
(X t = i kx ) ≈ ν k x P (Z t = sk ) = 1 3 P (Z t = sk ) (1.19)
for all i kx ∈ I, where (Z t ) t∈[0,T ] is a Markov process on the slow-scale with generator Q = diag (ν, ν) Qdiag ((1, 1), (1, 1)), then

Q =   -λ 2 λ 2 0 λ 2 -(λ 2 + µ 2 ) µ 2 0 µ 2 -µ 2  
We consider the probability θ(k, t) = P (Z t = sk ), which is solution the forward equation

d dt θ(•, t) = θ(•, t) Q θ(•, 0) = (p (1, 0) + p (2, 0) + p (3, 0), p (4, 0) + p (5, 0) + p (6, 0), p (7, 0) + p (8, 0) + p (9, 0))
which can be solved directly; in particular for λ 2 = µ 2 = 1 the solution is

θ(s 1 , t) = 1/2 e -t + 1/6 e -3 t + 1/3 θ(s 2 , t) = -1/3 e -3 t + 1/3 θ(s 3 , t) = 1/6 e -3 t -1/2 e -t + 1/3
Finally, the approximation is equivalent to In this section, we will compare the exact distribution function of the process (X t ) t∈[0,T ] , defined in Example 2, to a Monte Carlo approximation of its law; and also, to a Monte Carlo approximation of the law of the two-scales process derived on equation (1.19).

p (1, t), p (2, t), p (3, t) ≈ 1/6 e -t +
For the simulation of the process (X t ) t∈[0,T ] in I = {1, . . . , 9}, we follow the steps on Section 1.1.1. We set a simulation time T ≥ 0 and a time step ∆t and we define the number of steps N = T /∆t . We consider the jump rate λ(x) and the Markov kernel Π ∈ R 9×9 as defined in Theorem 1.1.2; in particular for λ 1 = λ 2 = µ 1 = µ 2 = 1 and = 0.2, we have

λ =               6 11 6 7 12 7 6 11 6               , Π =               0 5/
             
We set an array (X k ) N +1 k=1 for the process and a variable S for the time of jumps. The pseudo-code goes as follow:

1. For the initial condition we set X 1 = 1, k = 2 and S = 0.

2. While k ≤ N + 1 and S ≤ T do:

(a) With U 1 ∼ U (0, 1) set S = S - 1 λ(X k-1 ) log (U 1 ) (b) While k ≤ N and k∆t ≤ S do: i. X k = X k-1 ii. k = k + 1 (c) With U 2 ∼ U (0, 1) i. if U 2 ≤ Π (X k-1 , 1) then X k = 1 ii. if j-1 w=1 Π (X k-1 , w) < U 2 ≤ j w=1 Π (X k-1 , w) for j = 2, . . . , 9, then X k = j (d) k = k + 1
For the process (Z t ) t∈[0,T ] over S = {s 1 , s2 , s3 }, we consider its jump rate λ(k) and the Markov kernel Π ∈ R 3×3 ; in particular for

λ 2 = µ 2 = 1, we have λ =   1 2 1   , Π =   0 1 0 1/2 0 1/2 0 1 0  
And for the simulation, we consider the array (Z k ) N +1 k=1 t and the same pseudo-code as before.

In order to approximate the law p of (X t ) t∈[0,T ] via the Monte Carlo method, we take into account M realizations of (X m k ) N +1 k=1 for m = 1, . . . , M , then we have that

p M C (i, T ) = 1 M M m=1 1 {X m N =i} , i = 1, . . . , 9 with standard deviation p M C (x,T )(1-p M C (x,T ))

M

. And similarly, for the two-scales approximation (1.19), we set M realizations of (Z m k ) N +1 k=1 for m = 1, . . . , M , and then it follows

p * M C (i, T ) = 1 3 θ M C (k, T ) = 1 3M M m=1 1 {Z m N =s k } , i = 1, . . . , 9
for k such that x ∈ sk , and standard deviation

1 3 p * M C (x,T )(1-p * M C (x,T )) M . Solving for λ 1 = λ 2 = µ 1 = µ 2 = 1, p 0 = (1,
0, 0, 0, 0, 0, 0, 0, 0), = 0.2 and M = 1000, the results are the followings: Chapter 2

T = 1 p p M C std p * M C std 1 0.

Piecewise Deterministic Markov Process

Piecewise deterministic Markov processes, or PDMP, are a family of càdlàg Markov processes involving a deterministic motion punctuated by random jumps. The motion of the PDMP (Z t ) t≥0 depends on three local characteristics, namely the jump rate λ, the flow φ solution of an ordinary differential equation, and the transition kernel Π according to which the location of the process at the jump time is chosen. The process starts from z and follows the flow φ(z, t) until the first jump time T 1 , then the location of the process at T 1 is selected by the transition measure Π(φ(x, T 1 ), •) and the motion restarts from this new point as before.

In order to perform the state reduction approximation on PDMP, we consider a subclass called Markov switching model where the jumps occurs only on the discrete part of the process. In the case when there are multiple weakly irreducible classes and the generator can be written as a double scale generator for a small parameter , we present a method to approximate the process to a two-scale process: a slow-process on a reduced state space and fast-process inside each new class.

Ordinary differential equations and vector fields

d dt x(t) = ψ(x(t)) x(0) = x ∈ D (2.1)
has a unique solution φ(x, t) determined for all t ≤ t D , where t D is the time at which the solution exit from D, i.e.

t D := inf{t ≥ 0|φ(x, t) ∈ ∂D}
and ∂D is the boundary of D. The unique solution φ has the following properties 1. The map φ(•, t) : x → φ(x, t) is one-to-one and onto.

2. The family (φ(•, t)) t≥0 is a semi-group, i.e. for any t, s ≥ 0 it holds that φ(x, t + s) = φ(φ(x, s), t) for all x ∈ D Remark It is possible that t D fails to converge to ∞ as D ↑ R d , in which case there is said to be an explosion.

Let f : D → R be a C 1 function. Then with x(t) = φ(x, t),

d dt f (x(t)) = d l=1 ∂f (x(t)) ∂x l ψ l (x(t))
where ψ l is the lth component of ψ. Let denote F the first order differential operator

F f (x) = d l=1 ∂ ∂x l f (x)ψ l (x)
Then x(t) satisfies (2.1) if and only if it satisfies

d dt f (x(t)) = F f (x(t)), for all f ∈ C 1 (R d )
the operator F is the vector field.

Remark It is interesting to think ODEs as Markov Process. Assuming that F is locally Lipschitz and that there are no explosion, the trajectory of the process is X t = φ(x, t) and the semi-group is P t f (x) = f (φ(x, t)) for f ∈ C 1 . Then

P t f (x) -f (x) = t 0 F f (X s )ds and P t f (x) -f (x) - t 0 F f (X s
)ds = 0 and certainly a martingale. Thus F is the infinitesimal generator of the deterministic process.

Definition PDMP

We present a formal definition of a PDMP as shown by Davis (see [START_REF] Davis | Piecewise-Deterministic Markov Processes: A General Class of Non-Diffusion Stochastic Models[END_REF], [START_REF] Davis | Markov Models and Optimization[END_REF]). We start with the state space E defined as follows. Let consider the finite set I = {1, . . . , N } for a N ∈ N, and for every i ∈ I let D i be an open subset of R d , then

E = i∈I ({i} × D i ) = {(i, x) : i ∈ I, x ∈ D i } (2.2)
Let E denotes the following class of measurable sets in

E E = i∈I A i : A i ∈ M i
where M i denotes the Borel sets of D i ; then (E, E) is a Borel space. Let consider a filtered probability space (Ω, F, (F) t , P) and we define

(Z t ) t≥0 = (I t , X t ) t≥0
a Piecewiese deterministic Markov process with states on E and initial condition Z 0 = z = (i, x) ∈ E. The process I t corresponds to the discrete part and has states in I and X t is the continuous part and has states in R d . The process (Z t ) t is completely defined by ((F i ) i∈I , λ, Π) called the local characteristic of the PDMP, where

• F i is a locally Lipschitz continuous vector field characterized by the Lipschitz function ψ(i, x) for all i ∈ I, determining a flow φ(i, x, t) that set the motion of the PDMP between the jumps as shown in section 2.1. On a point of notation, any function f :

E → R is identified with its component functions f i : D i → R and we write F f (i, x) instead of F i f (x) for all (i, x) ∈ E.
We will suppose that if t ∞ (x) denote the explosion time of the flow φ(i, x, •) then we assume that t ∞ (x) = ∞ when t D i (x) = ∞, thus excluding explosions.

• λ : E → R + is the jump rate, a measurable function where for each (i, x) ∈ E and all t > 0 it holds that

t 0 λ(i, φ(i, x, s))ds < ∞
and we suppose that λ = sup z∈E λ(z) is bounded.

• Π : E×E → [0, 1] is a Markov transition kernel that maps E into the set P(E) of probability measures on (E, E), with properties that:

for each A ∈ E the map z → Π(z, A) is measurable and

-∀z ∈ E, Π (z, {z}) = 0
The dynamic of the process (Z t ) t with initial condition (i, x) ∈ E is constructed as follows.

Consider a random variable T 1 such that

P (i,x) [T 1 > t] =    exp - t 0 λ (i, φ(i, x, s)) ds for t < t D i (x) 0 for t ≥ t D i (x)
If T 1 is equal to infinity then

Z t = (i, φ(i, x, t)) for all t ≥ 0 Otherwise if T 1 < ∞, we select independently a E-valued random variable θ 1 having distribution Π ((i, φ(i, x, T 1 )), •), namely P (i,x) [θ 1 ∈ A] = Π ((i, φ(i, x, T 1 )), A) for any A ∈ E.
The trajectory of (Z t ) t for t ∈ [0, T 1 ], is given by

Z t = (i, φ(i, x, t)) for t < T 1 θ 1 for t = T 1
Starting from X T 1 = θ 1 , we now select the next inter-jump time T 2 -T 1 and post-jump location X T 2 = θ 2 in a similar way.

Lemma 2.2.1

The times between jumps of process (Z t ) t , satisfy the lack of memory property; i.e. the random variable T n+1 -T n is such that

P (I Tn ,X Tn ) [T n+1 -T n > t + u| T n , T n+1 -T n > u] = P (I Tn ,φ(I Tn ,X Tn ,u)) [T n+1 -T n > t]
for all n.

Proof Let T 1 be the first time-jump of the process (Z t ) t , then for u > 0 and t + u < t D i (x) we have

P (i,x) [T 1 > t + u|T 1 > u] = P (i,x) [T 1 > t + u, T 1 > u] P (i,x) [T 1 > u] = P (i,x) [T 1 > t + u] P (i,x) [T 1 > u] = exp - t+u 0 λ (i, φ(i, x, s)) ds / exp - u 0 λ (i, φ(i, x, s)) ds = exp - t+u u λ (i, φ(i, x, s)) ds = exp - t 0 λ (i, φ(i, x, s + u)) ds = exp - t 0 λ (φ(i, φ(i, x, u), s)) ds = P (i,φ(i,x,u)) [T 1 > t]
where we have used the semigroup property of φ.

We now present that PDMP satisfies the strong and normal Markov property.

Theorem 2.2.2 (Markov property) The process (Z t ) t≥0 is a homogeneous strong Markov process; i.e. for any z ∈ E, a stopping time T and function f , it holds that

E z f (Z T +s )1 {T <∞} |F T = E Z T [f (Z s )] 1 {T <∞}
where E z [f (Z t )] corresponds to the expectation value of the process f (Z t ) with starting condition Z 0 = z.

The proof can be found in [START_REF] Davis | Markov Models and Optimization[END_REF] (theorem 25.5).

Remark In particular, the process also satisfies the normal Markov property; i.e. for all n ≥ 0 and 0 ≤ s 1 < . . . < s n < t and s ≥ 0 and z 1 , . . . , z n , z, w in I,

P [Z t+s = w| Z s 1 = z 1 , . . . , Z sn = z n , Z t = z] = P [Z s = w| Z 0 = z]
This comes form the fact that any t > 0 is a stopping time.

In order to better understand the process, we define the following operators.

Definition For all t ≥ 0, the semigroup P t of the process (Z t ) t is defined for all measurable function f such that

P t f (z) = E z [f (Z t )]
We continue with the definition of another operator closely related to the semigropup.

Definition The infinitesimal generator L of the process (Z t ) t is defined such that

Lf (z) = lim h→0 P h f (z) -f (z) h
for all f ∈ D(L), where D(L) is the set of all measurable function f : E → R d such that t → f (i, φ(i, x, t)) is absolutely continuous on R + for all (i, x) ∈ E.

Lemma 2.2.3

The infinitesimal generator of the PDMP with local characteristics ((F i ) i∈I , λ, Π) corresponds to the operator L such that

Lf (z) = F f (z) + λ(z) E (f (v) -f (z)) Π(z, dv) (2.3)
defined for all f ∈ D(L) and z ∈ E.

Proof To prove this result, we observe that when h ↓ 0

P (i,x) [Z h = (i, φ(i, x, h))] ≥ P (i,x) [T 1 ≥ h] = exp - h 0 λ (i, φ(i, x, s)) ds = 1 - h 0 λ (i, φ(i, x, s)) ds + o(h)
and that for (i, x) = (j, y)

P (i,x) [Z h = (j, φ(j, y, h))] ≥ P [T 1 ≤ h, Z T 1 = (j, y), T 2 -T 1 ≥ h] = 1 -exp - h 0 λ (i, φ(i, x, s)) ds Π((i, x), {(j, y)}) exp - h 0 λ (j, φ(j, y, s)) ds = h 0 λ (i, φ(i, x, s)) ds Π((i, x), {(j, y)}) + o(h)
We observe that as the sum over all (j, y) ∈ E on the right side is 1, last inequalities have to be equalities. To prove this, let (T k ) n∈N denotes the sequence of time-jumps of the process (Z t ) t , then we have that

N t = k∈N 1 {t≥T k }
is a Poisson process; and if we consider λ = sup z∈E λ(z) and h ↓ 0, we have

P (i,x) [N h = 0] = 1 -h λ + o(h) P (i,x) [N h = 1] = h λ + o(h) P (i,x) [N h ≥ 2] = o(h)
The inequality ≤ is due the bound of the Poisson law; and the equality comes form the fact that the sum at both side of the inequality has to be 1 at any h small. This allow us to control the number of jumps in a small interval, and we can conclude that

P (i,x) [Z h = (i, φ(i, x, h))] = 1 - h 0 λ (i, φ(i, x, s)) ds + o(h) P (i,x) [Z h = (j, φ(j, y, h))] = h 0 λ (i, φ(i, x, s)) ds Π((i, x), {(j, y)}) + o(h), for (i, x) = (j, y)
Now, for all f ∈ D(L) and h ↓ 0, we have

E (i,x) [f (Z h )] = f (i, φ(i, x, h))P (i,x) [Z h = (i, φ(i, x, h)]) + j∈I\{i} D j \{x} f (j, φ(j, y, h))P (i,x) [Z h = (j, φ(j, dy, h))] = f (i, φ(i, x, h)) - h 0 λ (i, φ(i, x, s)) ds f (i, φ(i, x, h)) + h 0 λ (i, φ(i, x, s)) ds j∈I\{i} D j \{x} f (j, φ(j, y, h))Π((i, x), {(j, dy)}) + o(h) = f (i, φ(i, x, h)) + h 0 λ (i, φ(i, x, s)) ds j∈I D j (f (j, φ(j, y, h)) -f (j, φ(i, x, h))) Π((i, x), {(j, dy)}) + o(h)
as Π((i, φ(i, x, h)), •) is a probability measure where Π(z, {z}) = 0. In order to compute the infinitesimal generator we observe that

E (i,x) [f (Z h )] -f (i, x) h = f (i, φ(i, x, h)) -f (i, x) h + 1 h h 0 λ (i, φ(i, x, s)) ds j∈I D j
(f (j, φ(j, y, h)) -f (j, φ(i, x, h))) Π((i, x), {(j, dy)}) + o(1)

taking the limit and taking o(1) ↓ 0 when h ↓ 0, it follows that

Lf (i, x) = d l=1 ∂f (i, x) ∂x l ψ l (i, x) + λ(i, x) j∈I D j (f (j, y) -f (i, x)) Π((i, x), {(j, dy)})
and considering v = (j, y) ∈ E and dv = (j, dy), we have (2.3).

An important property of the generator is that it satisfies the Dynkin formula:

P t f (z) = f (z) + t 0 P s Lf (z)ds (2.4)
that it is equivalent to the statement that the process

C f t := f (Z t ) -f (z) - t 0 Lf (Z s )ds
is an F t -martingale. To prove this, we use the Markov property and the time-homogeneity; we see that for all t > s

E C f t -C f s |F s = E f (Z t ) -f (Z s ) - t s Lf (Z u )du|F s = E f (Z t ) -f (Z s ) - t s Lf (Z u )du|Z s = P t-s f (Z s ) -f (Z s ) - t s P u-s Lf (Z s )du = 0
With the objective to calculate the probability measure for the process (Z t ) t , we define for all t ≥ 0 the distribution p(•, t) ∈ P(E, E) such that

p(A, t) = E 1 {Zt∈A}
for A ∈ E and some initial distribution p(•, 0) = p 0 . If we rewrite the expectation as the integral over the probability measure in equation (2.4), we have

∂ ∂t E f (z)p(dz, t) = E F f (z) + λ(z) E (f (v) -f (z)) Π(z, dv) p(dz, t) Now, with z = (i, x) ∈ E, replacing F f (i, x) for d l=1 ∂f (i,x)
∂x l ψ l (i, x), integrating by parts and by Fubini's lemma, it leads to

i∈I D i f (i, x) d dt p(i, dx, t) = - i∈I D i f (i, x) d l=1 ∂ ∂x l [ψ l (i, x)p(i, dx, t)] - i∈I D i f (i, x)λ(i, x)p(i, dx, t) + i∈I D i λ(i, x)   j∈I D j f (j, y)Π((i, x), {(j, dy)})   p(i, dx, t)
we conclude that in a weak sense, the distribution of the process solves the Fokker-Planck equation,

∂ ∂t p(i, x, t) = - d l=1 ∂ ∂x l [ψ l (i, x)p(i, x, t)] -λ(i, x)p(i, x, t) + j∈I D j λ(j, y)Π((j, y), {(i, x)})p(j, dy, t) (2.5)
for all (i, x) ∈ E, t ≥ 0 and for some initial condition distribution p(•, 0) = p 0 .

Remark If we define the generator matrix Q = (Q ij ) i,j∈I such that for each i, j ∈ I, Q ij :

D i × D j → R where Q ii (x, x) = -λ(i, x)
for all (i, x) ∈ E Q ij (x, y) = λ(i, x)Π((i, x), {(j, y)}) for all (i, x), (j, y) ∈ E, (i, x) = (j, y). the equation (2.5) for the probability measure can be rewritten as

∂ ∂t p(i, x, t) = - d l=1 ∂ ∂x l [ψ l (i, x)p(i, x, t)] + j∈I D j p(j, dy, t)Q ji (y, x)
for all (i, x) ∈ E, t ≥ 0 and initial condition p(•, 0) = p 0 .

Example 1

Let consider a PDMP (Z t ) t = (I t , X t ) t with states in E = {0, 1} × R and local characteristics given by ((F i ) i=1,2 , λ, Π), where

• F i = 1 is the vector field for i = 1, 2, determining a flow φ(i, x, t) = x + t,

• λ : E → R + is the jump rate, such that λ(i, x) = α(x) if i = 0 β(x) if i = 1 • Π is a Markov transition kernel such that Π((0, x), {(1, 0)}) = 1 for all x ∈ R Π((1, x), {(0, x)}) = 1 for all x ∈ R
Then, the infinitesimal generator is given by

Lf (i, x) = ∂ ∂x f (i, x) + λ(i, x) j=0,1 R (f (j, y) -f (i, x))Π((i, x), {j, dy}) = ∂ ∂x f (i, x) + α(x)(f (1, 0) -f (0, x))1 {i=0} + β(x)(f (0, x) -f (1, x))1 {i=1}
for f ∈ D(L). And the Fokker-Plank equation is given by

∂ ∂t p(i, x, t) = - ∂ ∂x p(i, x, t) -λ(i, x)p(i, x, t) + j=0,1 R
λ(j, y)Π((j, y), {(i, x)})p(j, dy, t)

= - ∂ ∂x p(i, x, t) + [β(x)p(1, x, t) -α(x)p(0, x, t)] 1 {i=0} + α(x)p(0, x, t)1 {x=0} -β(x)p(1, x, t) 1 {i=1}
Remark In this model, X t could represent the time since the last spike of a neuron denote by I t , such that I t = 0 when the neuron is resting and I t = 1 during the spike.

Simulation

In order to simulate a PDMP (Z t ) t∈[0,T ] with values in E, initial distribution p 0 and local characteristics ((F i ) i∈I , λ, Π, ), the idea is:

• draw a sample z 0 from p 0 for the initial value,

• then iteratively, if the process is in state (i, x),

A. determine the flow φ from vector field F i , B. draw a sample from an E(λ(i, φ(i, x, t))) exponential law for the duration time, C. between jumps set Z t = (i, φ(i, x, t)), D. draw the next state w = (i, x) from the law Π((i, x), •).

To achieve this, we consider the following lemmas that allows to draw samples of the distribution needed from the uniform distribution.

Lemma 2.2.4 (Rejection Method) Let λ : [0, T ] → R + be a bound rate function and a random variable τ ∼ exp(λ(t)). Let consider λ = sup t∈[0,T ] λ(t) and the auxiliary random variables ξ ∼ exp( λ) and U ∼ U (0, 1), then

P ξ ∈ dt U ≤ λ(ξ) λ = λ(t)dt
Proof Let define the functions

f δ (t) = P [τ ∈ [t, t + δ)] = t+δ t λ(s) exp - s 0 λ(r)dr ds = λ(t)δ + o(δ 2 ) f δ (t) = P [ξ ∈ [t, t + δ)] = t+δ t λ exp -λs ds = λδ + o(δ 2 )
Now, noting that

P [A|B] = P [B|A] P [A] /P [B] we have P ξ ∈ [t, t + δ) U ≤ f δ (ξ) f δ (ξ) = P U ≤ f δ (ξ) f δ (ξ) ξ ∈ [t, t + δ) f δ (t) P U ≤ f δ (ξ) f δ (ξ)
where

P U ≤ f δ (ξ) f δ (ξ) = ∞ k=0 P U ≤ f δ (kδ) f δ (kδ) f δ (kδ) = ∞ k=0 f δ (kδ) f δ (kδ) f δ (kδ) = ∞ k=0 f δ (kδ) = 1
and

P U ≤ f δ (ξ) f δ (ξ) ξ ∈ [t, t + δ) = 1 f δ (t) P U ≤ f δ (ξ) f δ (ξ) , ξ ∈ [t, t + δ) = 1 f δ (t) ∞ k=0 P U ≤ f δ (kδ) f δ (kδ) , kδ ∈ [t, t + δ) f δ (kδ) = 1 f δ (t) P U ≤ f δ ( t) f δ ( t) f δ ( t) = f δ ( t) f δ (t)
for some t ∈ [t, t + δ). Finally, we obtain

P ξ ∈ [t, t + δ) U ≤ f δ (ξ) f δ (ξ) = f δ ( t)
and when δ → 0, we have the result wanted.

Where we use Lemma 1.1.3 to simulate ξ ∼ E( λ). For draw a sample of Π((i, x), •), it will depend on the form of the distribution. In the case it has a finite support, we can use Lemma 1.1.4.

Markov switching model with multiple weakly irreducible classes

In order to perform the state reduction approximation, we will consider a subclass of PDMP known as the Markov switching model (see [START_REF] Azais | Piecewise deterministic Markov Process -recent results[END_REF], [START_REF] Benaim | Quantitative ergodicity for some switched dynamical systems[END_REF]). Let consider the state space E = I × R d for a finite space I = {1, 2, . . . , N }, the Borel space (E, E) and a filtered probability space (Ω, F, (F t ) t , P); we define for a T < ∞

(Z t ) t∈[0,T ] = (I t , X t ) t∈[0,T ]
a Markov switching process with states on E, initial condition Z 0 = z = (i, x) ∈ E and local characteristics given by ((F i ) i∈I , λ, Π), where

• F i is a continuous vector field characterized by the Lipschitz functions ψ(i, x), for all (i, x) ∈ E; that determines a flow f (φ(i, x, t)) via the ODE

d dt f (x(t)) = F f (i, x(t)) = d l=1 ∂f (x(t)) ∂x l ψ l (i, x(t)) • λ : E → R + is the jump rate, • Π : E × E → [0, 1
] is a Markov transition kernel that allows only jumps in I; i.e. we demand that Π ((i, x), {(j, x)}) ≥ 0, for all (i, x) ∈ E and j ∈ I, Π ((i, x), {(j, y)}) = 0, otherwise

The Markov switching model is a process Z t = (I t , X t ) ∈ I × R d can be described such that A. X t is driven by the vector field F It , B. If I t = i and X t = x, then I t jumps to j with rate λ(i, x)Π((i, x), {(j, x)}) for j = i,

C. Its generator infinitesimal is given by

Lf (i, x) = F f (i, x) + λ(i, x) j∈I (f (j, x) -f (i, x)) Π((i, x), {(j, x)})
with a generator matrix Q(x) ∈ R N ×N defined by

Q ij (x) = -λ(i, x)1 {i=j} + λ(i, x)Π((i, x), {(j, x)})1 {i =j} (2.6)
for i, j = 1, . . . , N .

In this model, the trajectory of the continuous part X t does not jump and it evolves continuously; the jumps only occur on the discrete part I t . Let us also note that there are no boundary in this model for the continuous part, so we don't have to consider border condition and there is no risk of explosion times.

Remark In general, (I t ) t is not a Markov process on its own since its jump rates depend on (X t ) t . If the jump rate λ does not depend on (X t ) t , then (I t ) t is a Markov process on the finite space I and X t is a function of (I s ) s≤t .

We consider that the process (Z t ) t has multiple irreducible classes, which means that its generator Q (x) for any fix x ∈ R d depends on a small parameter > 0 and two generators Q(x) and Q(x), such that

Q (x) = 1 Q(x) + Q(x) (2.7)
where Q(x) governs the dynamic on the slow-scale and Q(x) on the fast-scale. The generator Q(x) can be put into block-diagonal form

Q(x) =      Q1 (x) 0 0 0 Q2 (x) . . . 0 0 0 Q N (x)     
where each Qk (x) is a generator of dimension m k ×m k for some integer m k , such that N k=1 m k = N , and it defines the class sk = {i k1 , . . . , i km k } ⊂ I that corresponds to the states associated with the generator Qk (x). We define the new set S = {s 1 , . . . , s N } that will be the space of the slow process, and that satisfies N k=1 sk = I.

Assumption 2.3.1 For all k = 1, . . . , N and x ∈ R d , Qk (x) is a weakly irreducible generator; that is for any fix x ∈ R d the system of equations

           m k j=1 ν k j (x) Qk ji (x) = 0, for i = 1, . . . , m k m k i=1 ν k i (x) = 1
has a unique non-negative solution.

We consider for all t ≥ 0 the distribution p (•, t) ∈ P(E, E) such that

p (A, t) = E p 0 1 {Z∈A} for A ∈ E, then the Fokker-Planck equation (2.5) corresponds to      ∂ ∂t p (i, x, t) = - d l=1 ∂ ∂x l [ψ l (i, x)p (i, x, t)] + j∈I p (j, x, t) 1 Qji (x) + Qji (x) p (i, x, 0) = p 0 (i, x) (2.8)
for all (i, x) ∈ E, t ∈ [0, T ] and initial distribution p 0 .

Asymptotic expansions

Now we present the most important theorem of the section, that presents a characterization for the probably measure of the process (Z t ) t , based on results given in [START_REF] Yin | Continuous-Time Markov Chains and Applications[END_REF] and [START_REF] Pakdamana | Asymptotic expansion and central limit theorem for multiscale piecewise-deterministic Markov processes[END_REF].

Theorem 2.3.2 (Asymptotic expansion) The probability measure p (•, t) (2.8) of the process (Z t ) t can be expanded in the form:

p (i, x, t) = ϕ(i, x, t) + γ i, x, t + e (i, x, t) (2.9)
In this approach ϕ is set to be an approximation on the slow-scale t away from 0, γ approximate the fast-scale τ = t/ and e corresponds to the error of the expansion. The functions ϕ, γ and e are such that

• ϕ(i, x, t) is differentiable for all t ∈ [0, T ].
• there exist a constant κ 0 > 0 such that

|γ (i, x, τ )| ≤ K exp (-κ 0 τ )
uniformly for all (i, x) ∈ E.

• and the following estimate holds

sup t∈[0,T ] |e (i, x, t)| ≤ K
uniformly for all (i, x) ∈ E.

To proceed, we define an operator L by

L f (i, x, t) = d dt f (i, x, t) + d l=1 ∂ ∂x l [ψ l (i, x)f (i, x, t)] - j∈I f (i, x, t) Qji (x) + Qji (x) (2.10)
for any smooth function f , then L f = 0 iff f is a solution to the Fokker-Planck equation (2.8). We set that both ϕ and γ are solution to the forward equation, then they satisfy L ϕ(i, x, t) = 0 and L γ i, x, t = 0 that is, we have

∂ ∂t ϕ(i, x, t) = - d l=1 ∂ ∂x l [ψ l (i, x)ϕ(i, x, t)] + j∈I ϕ(j, x, t) Qji (x) + Qji (x) ∂ ∂t γ i, x, t = - d l=1 ∂ ∂x l ψ l (i, x)ϕ i, x, t + j∈I γ j, x, t Qji (x) + Qji (x) (2.11)
and we write a new time scale τ = t/ for the second equation

∂ ∂τ γ(i, x, τ ) = - d l=1 ∂ ∂x l [ψ l (i, x)ϕ (i, x, τ )] + j∈I γ(j, x, τ ) Qji (x) + Qji (x)
And if we match the terms over 0 and 1 in the time scale t and setting that ψ(τ ) must not depends on , we have the following set of equations

         j∈I ϕ(j, x, t) Qji (x) = 0 ∂ ∂t ϕ(i, x, t) = - d l=1 ∂ ∂x l [ψ l (i, x)ϕ(i, x, t)] + j∈I ϕ(j, x, t) Qji (x) and ∂ ∂τ γ(i, x, τ ) = j∈I γ(j, x, τ ) Qji (x)
Remark From the last term comes the approximation error of this expansion, and due the fact that Q is not weakly irreducible.

In order to match the asymptotic expansion, we have necessarily at t = 0 that

p (i, x, 0) = ϕ(i, x, 0) + γ(i, x , 0)
Sending → 0 in the expansion (2.11), the fast-scale and error disappear and only remains the slow-scale, then as i∈I D p (i, x, t)dx = 1 for all > 0, we conclude lim

→0 i∈I D p (i, x, t)dx = i∈I D ϕ(i, x, t)dx = 1

Determining ϕ

We need to determine ϕ(i, x, t) for (i, x) ∈ E and t ∈ [0, T ] such that

                   j∈I ϕ(j, x, t) Qji (x) = 0 ∂ ∂t ϕ(i, x, t) = - d l=1 ∂ ∂x l [ψ l (i, x)ϕ(i, x, t)] + j∈I ϕ(j, x, t) Qji (x) i∈I R d ϕ(i, x, t)dx = 1 (2.12) Since Q(x) = diag Q1 (x), . . . , Q N (x)
where each Qk (x) is weakly irreducible, therefore if we consider ϕ k (i, x, t), the function ϕ(i, x, t) restricted on i ∈ sk , we have that it satisfies

j∈s k ϕ k (j, x, t) Qk ji (x) = 0, for all i ∈ sk then its solution is ϕ k (i, x, t) = θ(k, x, t)ν k i (x)
, the product of the invariant measure ν k (x) of Qk (x) and a function multiplier θ(k, x, t) with values in k ∈ {1, . . . , N }, x ∈ R d and t ∈ [0, T ]. And as ν k (x) is a distribution in sk , we have that

i∈s k R d ϕ k (i, x, t)dx = i∈s k R d θ(k, x, t)ν k i (x)dx = R d θ(k, x, t)dx
and by consequence of the third equation on (2.12), it also holds that

N k=1 R d θ(k, x, t)dx = 1
Setting all this with the second equation in (2.12), for each k = 1, . . . , N it follows that

∂ ∂t θ(k, x, t) = i∈s k d dt ϕ(i, x, t) = - i∈s k d l=1 ∂ ∂x l [ψ l (i, x)ϕ(i, x, t)] + i∈s k j∈I ϕ(j, x, t) Qji (x) = - i∈s k d l=1 ∂ ∂x l ψ l (i, x)ν k i (x)θ(k, x, t) + i∈s k N p=1 j∈sp θ(p, x, t)ν p j (x) Qji (x)
reorganizing the terms, we have that for k = 1, . . . , N , it holds

∂ ∂t θ(k, x, t) = - d l=1 ∂ ∂x l     i∈s k ν k i ψ l (i, x)   θ(k, x, t)   + N p=1 θ(p, x, t)   j∈sp i∈s k ν p j (x) Qji (x)   (2.
13) where we observe the emergence of a new generator Q

(x) ∈ R N × N such that Qpk (x) = j∈sp i∈s k ν p j (x) Qji (x) (2.14)
for k, p = 1, . . . , N , and a new function ψ define by

ψ(k, x) = i∈s k ν k i (x)ψ(i, x) (2.15)
for k = 1, . . . , N . In order to determine the initial conditions θ(k, x, 0), we fist observe that in the asymptotic expansion it has to hold that

i∈s k ϕ(i, x, 0) = lim δ→0 lim →0 i∈s k p (i, x, δ) (2.16)
moreover, in view of the forward equation (2.8) and that i∈s k Qk ji (x) = 0 for all j ∈ sk , we have

i∈s k p (i, x, δ) = i∈s k p (i, x, 0)+ δ 0   - i∈s k d l=1 ∂ ∂x l [ϕ l (i, x)p (i, x, s)] + i∈s k j∈s k p (j, x, s) Qji (x)   ds and since p (i, x, t) is bounded it follows that lim δ→0 lim sup →0 δ 0   - i∈s k d l=1 ∂ ∂x l [ϕ l (i, x)p (i, x, s)] + i∈s k j∈s k p (j, x, s) Qji (x)   ds = 0
therefore by (2.16) it yields

i∈s k R d ϕ(i, x, 0)dx = lim δ→0   lim →0 i∈s k R d ϕ(i, x, δ)dx   = i∈s k R d p (i, x, 0)dx
and we finally have θ(

To conclude, we present the following Corollary that synthesized the results of this section.

Corollary 2.3.3 The system (2.12) for ϕ(i, x, t), is equivalent to the system

                           j∈s k ϕ(j, x, t) Qji (x) = 0, for i ∈ sk i∈s k ϕ(i, x, t) = θ(k, x, t) ∂ ∂t θ(k, x, t) = - d l=1 ∂ ∂x l ψl (k, x)θ(k, x, t) + N p=1 θ(p, x, t) Qpk (x) θ(k, x, 0) = i∈s k p 0 (i, x)
for k = 1, . . . , N ; where the function ψ is defined in (2.15) and the generator Q in (2.14).

Remark These θ(i, x, t) can be interpreted as the probability measure of an PDMP (M t ) t≥0 defined over the aggregate states space S × R d , where S is the finite space {s 1 , . . . , s N }, and defined by its local characteristics given by the generator Q and the vector field Fs s∈ S given by

F f (s k , x) = d l=1 ∂f (x) ∂x l ψ(s k , x) (2.17)
Determining γ

We consider γ(i, x, τ ) for (i, x) ∈ E and t ∈ [0, T ] solution to

∂ ∂τ γ(i, x, τ ) = j∈I γ(j, x, τ ) Qji (x) (2.18)
To matched asymptotic expansion, we have necessarily at t = 0 that ϕ(i, x, 0) + γ(i, 0) = p (i, x, 0)

In order to solve equation (2.18), we observe that we can solve it directly and together with the above initial condition, we obtain

ψ(•, x, τ ) = (p 0 (•, x) -ϕ(•, x, 0)) exp Q(x)τ (2.19)
Considering that each Qk (x) is weakly irreducible, we need to prove that γ(i, x, τ ) can be obtain by (2.19), and there is a positive number κ 0 such that

|ψ(i, x, τ )| ≤ K exp (-κ 0 τ )
uniformly for (i, x) ∈ E. To prove this, let ν k (x) be the stationary distribution corresponding to the generator Qk (x), and we define the column vector

1 m = (1, 1, . . . , 1) ∈ R 1×m , then π(x) =      1 m 1 ν 1 (x) 0 0 0 1 m 2 ν 2 (x) . . . 0 0 0 1 m N ν N (x)     
where

1 m k ν k (x) =    ν k 1 (x) • • • ν k m k (x) . . . ν k 1 (x) • • • ν k m k (x)    Noting the block-diagonal structure of Q, we have exp Q(x)τ =         exp Q1 (x)τ 0 0 0 exp Q2 (x)τ . . . 0 0 0 exp Q N (x)τ        
Furthermore, we see that for k = 1, . . . , N it holds

i∈s k (p 0 (i, x) -ϕ(i, x, 0)) = i∈s k p 0 (i, x) - i∈s k ϕ(i, x, 0) = i∈s k p 0 (i, x) -θ(k, x, 0) = 0
and then, the initial condition (p 0 (•, x) -ϕ(•, x, 0)) is orthogonal to π(x). By virtue of Lemma (1.2.2), for each k = 1, . . . , N there exists κ k > 0 such that exp Qk (x)τ -

1 m k ν k (x) ≤ K exp (-κ k τ ) then we have |γ(•, x, τ )| = (p 0 (•, x) -ϕ(•, x, 0)) exp Q(x)τ -π(x) ≤ K sup k≤ N exp Qk (x)τ -1 m k ν k (x) ≤ K exp (-κ 0 τ ) where κ 0 = min k≤ N κ k .

Analysis of remainder

The remainder of the asymptotic expansion (2.9) correspond to e (i, x, t) = ϕ(i, x, t) + γ i, x, t -p (i, x, t)

where e (0) = 0, and if we consider the operator L as in (2.10), it holds that L p (t) = 0 and from equations (2.12) and (2.18), we have

L e (i, x, t) = - d l=1 ∂ ∂x l ψ l (i, x)γ i,
p (i, x, t) -ϕ(i, x, t) -γ i, x, t = K
where K is a positive constant that depends on T .

Two-scales approximation

We now present a corollary that allows us, under the conditions already discussed, to represent the PDMP as a two-scale process: one on a slow-scale that goes over the state classes and a fast-scale that acknowledge the dynamic inside each class.

Corollary 2.3.4 Let be (Z ) t∈[0,T ] a Markov switching process defined over E = I × R d with I = {1, . . . , N }, initial distribution p 0 , and local characteristics given by the functions ψ and a two-scales generator Q (x) ∈ R N ×N that depends on > 0 and two generators Q(x) and Q(x) such that

Q (x) = 1 Q(x) + Q(x)
where

Q(x) = diag( Q1 (x), . . . , Q N (x)
           ∂ ∂t θ(k, x, t) = - d l=1 ∂ ∂x l ψl (k, x)θ(k, x, t) + N p=1 θ(p, x, t) Qpk (x) θ(k, x, 0) = i∈s k p(i, x, 0) for the generator Q(x) ∈ R N × N defined by Qpk (x) = j∈sp i∈s k ν p j (x) Qji (x)
for p, k = 1 . . . , N ; and function ψ given by

ψ(k, x) = i∈s k ν k i (x)ψ(i, x)
for k = 1 . . . , N .

In this corollary, we interpret θ(k, x, t) as the probability measure of an PDMP (M t ) t∈[0,T ] defined over S × R d with S = {s 1 , . . . , s N } and local characteristics given by the functions ψ and generator Q(x), which holds the dynamics on the slow-scale. We can think of this process as M t = (s k , x) ⇐⇒ Z t = (i, x) for some i ∈ sk and that θ(k, x, t) = P [M t = (s k , x)] for i = 1, . . . , N . On the other hand, the dynamic in the fast-scale will be defined punctually on the position of M t by the random variable I fast such that P I fast = i kj | M t = (s k , x) = ν k j (x) for each i kj ∈ sk . We observe that the value of the fast process depends instantaneously on the prosition of the slow process M t .

Example 2

Continuation of Example 1 Chapter 1 (1.2.2). Let consider now that the proper functioning of the two machines depends on the heat in the factory. If the heat on the factory is too high, the machines are more likely to malfunction; and at the same time by working they contribute to the rise of the heat. For this we present the following model: let (Z t ) t∈[0,t] = (X t , H t ) t∈[0,t] be a Markov switching model, where X t represents capacity of the workshop that correspond to a four-state chain with state space I = {(0, 0), (1, 0), (0, 1), (1, 1)}, and H t represents the heat in the factory with values in R. Suppose that the first machine breaks down much more often and produce more heat than the second one; to reflect this situation, consider that the process is generated by Q (h) as (2.7) for a small > 0, with Q(h) and Q(h) given by

Q (h) = 1 Q(h) + Q(h)
where

Q(h) =     -λ(h) λ(h) 0 0 µ(h) -µ(h) 0 0 0 0 -λ(h) λ(h) 0 0 µ(h) -µ(h)     , Q(h) =     -λ(h) 0 λ(h) 0 0 -λ(h) 0 λ(h) µ(h) 0 -µ(h) 0 0 µ(h) 0 -µ(h)    
where λ(h) and µ(h) are the rates of repair and breakdown respectively, and follow the logistic equations

λ(h) = 1 1 + exp(2h) ∈ [0, 1] µ(h) = 1 1 + exp(-2h) ∈ [0, 1]
The heat is determined vector field functions

ψ(x, h) = α(2x 1 -1) + β(2x 2 -1)
with α ≥ β, which determines the flow φ(x, h, t) = h + α(2x 1 -1)t + β(2x 2 -1)t; and we define the matrix Ψ ∈ R 4×4 by

Ψ =     ψ(i 1 ) 0 0 0 0 ψ(i 2 ) 0 0 0 0 ψ(i 3 ) 0 0 0 0 ψ(i 4 )     =     -α -β 0 0 0 0 α -β 0 0 0 0 -α + β 0 0 0 0 α + β    
We consider the probability p (x, h, t) = P p 0 [Z t = (x, h)], that denotes the probability distribution of the underlying process at time t, and it is solution of the forward equation

∂ ∂t p (•, h, t) = - ∂ ∂h p (•, h, t)Ψ + p (•, h, t)Q (h) p (•, h, 0) = p 0 (•, h) (2.20)
where p 0 is the initial distribution.

In order to find the two-scale approximation, we notice that Q can be rewritten as

Q(h) = diag -λ(h) λ(h) µ(h) -µ(h) , -λ(h) λ(h) µ(h) -µ(h)
with both sub-generator weakly irreducible, with invariant measure

ν(h) = µ(h) µ(h) + λ(h) , λ(h) µ(h) + λ(h) = (µ(h), λ(h))
and we identify the classes s0 = {(0, 0), (1, 0)} = {i 01 , i 02 } s1 = {(0, 1), (1, 1)} = {i 11 , i 12 } that form the slow-scale states S = {s 0 , s1 }. By Corollary 2.3.4, we approximate the law of process (Z t ) t = (X t , H t ) t to the law of a two-scale process (W t ) t = ( Xt , H t ) t with states in S × R, and X fast with states in s Xt , such that

P [Z t = (i kx , h)] ≈ P X fast = i kx P [W t = (s k , h)] (2.21)
with a magnitude error of order . On the fast-scale, X fast ∈ s Xt and it follows that

P X fast = i k1 = µ(H t ), for k = 0, 1 P X fast = i k2 = λ(H t ), for k = 0, 1
On the other hand, ( Xt , H t ) t ∈ S × R is a Markov switching process on the slow-scale with generator Q(h) given by

Q(h) = ν(h) 0 0 ν(h) Q(h) 1 2 0 0 1 2 = -λ(h) λ(h) µ(h) -µ(h)
and vector field functions ψ that satisfies

ψ(k, h) = x∈s k ν x (h) [α (2x 1 -1) + β (2x 2 -1)] = µ(h) [-α + β (2k -1)] + λ(h) [α + β (2k -1)] = α (λ(h) -µ(h)) + β (2k -1)
We consider the probability θ(k, h, t) = P [W t = (s k , h)] and it is solution of the forward equation

         ∂ ∂t θ(k, h, t) = - ∂ ∂h ψ(k, h)θ(k, h, t) + 2 p=1 θ(p, h, t) Qpk (h) θ(k, h, 0) = i∈s k p 0 (i, h) Example 3 -Monte Carlo
In this section, we will compute the distribution function of the process (Z t ) t∈[0,T ] = (X t , H t ) t∈[0,T ] , defined in Example 2, via a Monte Carlo estimate of its law; and also, via a Monte Carlo estimate of the law of the two-scale approximation process derived on equation (2.21). For the simulation of the process X t in I = {1, . . . , 4} and H t in R, we follow the steps on Section 2.2.1. We set a simulation time T ≥ 0 and a time step ∆t and we define the number of steps N = T /∆t. We define jump rate Λ(x, φ(x, h, t)) and the Markov Kernel Π(h) = (Π(x, y, h)) x,y∈I as

Λ(•, h, t) =                    1 + λ(h -(α + β)t) 1 µ(h + (α -β)t) + λ(h + (α -β)t) 1 λ(h + (-α + β)t) + µ(h + (-α + β)t) 1 + µ(h + (α + β)t) and Π(h) =         0 1 +1 +1 0 µ(h) λ(h) +µ(h) 0 0 λ(h) λ(h) +µ(h) µ(h) λ(h)+µ(h) 0 0 λ(h) λ(h)+µ(h) 0 +1 1 +1 0        
We also need to compute Λ(x, h), a rate that bound of Λ(x, φ(x, h, t)) for all t ≥ 0. As λ(h), µ(h) ≤ 1 for all h, we have Λ(x, h) = 1+ . We set an array (X k , H k ) N +1 k=1 for the process and a variable S for the time of jumps. The pseudo-code goes as follow:

1. For the initial condition we set X 1 = x, H 1 = h, k = 2 and S = 0.

While k ≤ N + 1 and S ≤ T do:

Chapter 3

Hodgkin and Huxley model

The Hodgkin and Huxley model (H&H) [START_REF] Hodgkin | A quantitative description of membrane current and its application to conduction and excitation in nerve[END_REF], describes the membrane potential V of a typical neuron on the mean behavior of the potassium (K + ) and calcium (Na + ) ion channels, through its voltage-gated processes.

                   dV (t) dt = f (V (t), m(t), h(t), n(t)) dm(t) dt = α m (V (t)) (1 -m(t)) -β m (V (t))m(t) dh(t) dt = α h (V (t)) (1 -h(t)) -β h (V (t))h(t) dn(t) dt = α n (V (t)) (1 -n(t)) -β n (V (t))n(t) (3.1)
for all t ≥ 0 and initial condition V (0) = V 0 , m(0) = m 0 , h(0) = h 0 and n(0) = n 0 ; where

f : R 4 → R such that f (V, m, h, n) = 1 C (I ext - I Na g Na m 3 h (V -E Na ) - I K g K n 4 (V -E K ) - I L g L (V -E L ))
The dimensionless variables m, h, and n describe the probability of open voltage-gates on each ion-channel. The rate functions that appear in the equations were also determined by Hodgkin and Huxley and are given by

α m (V ) = (25 -V )/10 e (25-V )/10 -1 , β m (V ) = 4e -V /18 α h (V ) = 0.07e -V /20 , β h (V ) = 1 e (30-V )/10 + 1 α n (V ) = (10 -V )/100 e (10-V )/10 -1 , β n (V ) = 0.125e -V /80 (3.2)
The parameters provided in the original paper correspond to the membrane potential shifted by approximately 65 mV so that the resting potential is at V ≈ 0. The equilibrium potentials and Theorem 3.1.1 Law of large number.

Let {e i } i=1...N be a succession of Markov processes defined in (3.3), and let Z 0 = (v 0 , g 0 ) ∈ R × [0, 1] be a initial condition of (3.9). For all δ, ε > 0, there exists an initial condition Z N 0 = V N 0 , u N 0 for (3.8) and N 0 = N 0 (δ, ε) such that for all N ≥ N 0 the solution Z N t = V N t , u N t satisfies:

P sup 0≤t≤T |V N t -v(t)| > δ < ε (3.10) and P sup 0≤t≤T |u N t -g(t)| > δ < ε (3.11)
for all fixed T > 0.

First we decompose the difference between the stochastic and the deterministic processes as a sum of a martingale part M N and a finite variation part F N as follows

[u N t -u N 0 ] -[g(t) -g(0)] = M N (t) + t 0 F N (s)ds
where we define:

F N (t) := α(V N t )(1 -u N t ) -β(V N t )u N t - dg(t) dt M N (t) := u N t -g(t) -u N 0 -g(0) - t 0 F N (s)ds
For the proof of Theorem 3.1.1, we need the following lemmas.

Lemma 3.1.2 M N (t) is a {F t }-martingale
Proof First we notice that for all i and h ↓ 0 On another side, considering E [e i (t) -g(t)] = 0 for all i, and then

E [e i (t + h)|F t ] = E [e i (t + h)|e i (t)] = P [e i (t + h) = 1|e i (t)] = α(V N t )(1 -e i (t))h + 1 -β(V N t ) e i (t)h + o(h) and 1 h E u N t+h -u N t |F t = α(V N t )(1 -u N t ) -β(V N t )u N t + o(1) = E α(V N t )(1 -u N t ) -β(V N t )u N t |F t + o(1) Then we have that 1 h E [M N (t + h) -M N (t)|F t ] = 1 h E u N t+h -u N t |F t - 1 h E t+h t α(V N s )(1 -u N s ) -β(V N s )u N s ds|F t - 1 h [g(t + h) -g(t)] + 1 h t+h t dg(s) dt ds Solving gives lim h→0 1 h E [M N (t + h) -M N (t)|F t ] = 0 and therefore d ds E [M N (t + s)|F t ] | s=0 = 0. Finally E [M N (t + h)|F t ] = cte = E [M N (t)|F t ] = M N (t).
E u N t -g(t) 2 = E 1 N N i=1
e i (t) -g(t)

2 = 1 N 2 N i=1 E [e i (t) -g(t)] 2 ≤ 1 N
as E [e i (t) -g(t)] 2 = g(t) (1 -g(t)) ≤ 1 for all i, and

E F 2 N (t) = E α(v(t))g(t) -α(V N t )u N t -β(v(t))g(t) -β(V N t )u N t 2 ≤ 2E α 2 ∞ g(t) -u N t 2 + β 2 ∞ g(t) -u N t 2 ≤ 4 max{ α 2 ∞ , β 2 ∞ }E u N t -g(t) 2 ≤ 4 N max{ α 2 ∞ , β 2 ∞ } Finally E M N (t) 2 ≤ C 1 T 2 N max{ α 2 ∞ , β 2 ∞ }
where α ∞ and β ∞ are finite because α and β are continuous by assumption H1. Then by Chebychev inequality and Doob inequalities for martingales:

P sup 0≤t≤T M N (t) 2 ≥ δ ≤ 1 δ E sup 0≤t≤T M N (t) 2 ≤ 4 δ E M N (t)
F N (t) = α(V N t )(1 -u N t ) -α(v(t))(1 -g(t)) -β(V N t )u N t + β(v(t))g(t)
In order to use the Lipschitz property of α and β, we separate last equation into two,

F 1 N (t) := β(V N t )u N t -β(v(t))g(t) F 2 N (t) := α(V N t )(1 -u N t ) -α(v(t))(1 -g(t))
We define the process (M N t ) t , function of (V N t , u N t ) such that and again, the second term of the inequality, said J N (θ) converges to 0 as N → ∞, because of the convergence of (V with initial condition ZN 0 = Z N 0 .

M N t = 1 √ N R N t = u N t -u N 0 - t 0 b(V N s ,
φ N (θ, t) -1 = E h(M N t ) -h(0) = t 0 E Lh(M N s ) ds = t 0 E   λ N (V N s , u N s ) w∈E N h(w -u N s + M N s ) -h(M N s ) -(w -u N s )h (M N s ) Π (V N s , u N s ), {(V N s , w)}   ds = t 0 E   exp iθR N s λ N (V N s , u N s ) w∈E N ξ θ √ N w -u N s Π (V N s , u N s ), {(V N s ,

State reduction in sodium channels

Let present an H&H model with and alternative characterization of the sodium channels. Let (Ω, F, {F t } t , P) be a complete filtered probability space and a finite T > 0. Let consider a neuron with N sodium channels an we define the PDMP (Z N t ) t∈[0,T ] = (V N t , u N i=1 be the sequence of sodium ion channels with states in {0, 1} × {0, 1, 2, 3}, and it can be characterized by the chain

m = 0 m = 1 m = 2 m = 3 3α m 2α m α m h = 0 ---→ ← --- ---→ ← --- ---→ ← --- β m 2β m 3β m C i (t) : α h ↓↑ β h α h ↓↑ β h α h ↓↑ β h α h ↓↑ β h 3α m 2α m α m h = 1 ---→ ← --- ---→ ← --- ---→ ← --- β m 2β m 3β m
where the generator Q(V ) is given by We define for the slow-scale the generator matrix Q(V ), that satisfies

Q(V ) =             -α h 0 0 0 α h 0 0 0 0 -α h 0 0 0 α h 0 0 0 0 -α h 0 0 0 α h 0 0 0 0 -α h 0 0 0 α h β h 0 0 0 -β h 0 0 0 0 β h 0 0 0 -β h 0 0 0 0 β h 0 0 0 -β h 0 0 0 0 β h 0 0 0 -β h             and Q(V ) = diag Q1 (V ), Q2 (
Q(V ) = ν(V ) 0 0 ν(V ) Q(V ) 1 4 0 0 1 4 = -α h α h β h -β h (3.18)
With this results, we approximate the law of the process Z N t t = (V N t , u N t ) t to a two-scale process (W t ) t = ( Ṽ N t , ũN t ) t and {m N i,fast } N i=1 . The dynamic on fast-scale corresponds to the

First

  we study the deterministic part of the process. Let D be an open set of R d and ψ : D → D be a Lipschitz continuous function, i.e. there exists a constant C D such that |ψ(x) -ψ(y)| ≤ C D |x -y| for all x, y in D. Then the differential equation

  ) with each sub-generator Qk (x) ∈ R m k ×m k weakly irreducible and determines the class sk ⊂ I. Then there exists positive constants K T , K and κ 0 such thatP [Z t = (i kj , x)] -ν k j (x)θ(k, x, t) ≤ K T + K exp -κ 0 tfor all x ∈ R d and i kj ∈ sk for j = 1, . . . , m k and k = 1, . . . , N . Here ν k (x) is the stationary distribution in class sk given by Qk (x), and θ(k, x, t) is a function defined for k = 1, . . . , N , x ∈ R d and t ∈ [0, T ] that satisfies

Lemma 3 . 1 . 3

 313 Let T > 0, ε > 0, δ > 0. Then there exists N 0 such that ∀N ≥ N 0P sup 0≤t≤T M N (t) 2 ≥ δ ≤ ε Proof Using (a + b + c) 2 ≤ 4(a 2 + b 2 + c 2 )and CauchySchwarz inequality we have thatE M N (t) 2 ≤ 4E u N t -g(t)

  u N s )ds with infinitesimal generator L that satisfiesLh(M ) = h (M )b(V, u) + λ N (V, u) w∈E N (h(w -u + M ) -h(M )) Π ((V, u), {(V, w)})for all h ∈ D(L), and due the definition of λ N in (3.4) and Π in (3.5), we have thatb(V, u) = λ N (V, u) w∈E N (w -u) Π ((V, u), {(V, w)}) If we define functions h(M ) = exp iθ √ N M , ψ(u) = exp(iu) -1 -iu + u 2 /2 /u 2 ,and ξ(u) = exp(iu) -1 -iu = u 2 ψ(u) -u 2 /2; then it follows that

θ 2 b 2 t 0 θ 2 E

 2202 N s , u N s ), {(V N s , w)} ds the second term in the last equality, said K N (θ), converges to 0 as N → ∞ by dominated convergence, and because ψ θ√ N w -u N s = ψ ±θ/ √ N → 0 as lim u→0 ψ(u). So we have φ N (θ, t) = -(v(s), g(s)) φ N (θ, s)ds + 1 b (v(s), g(s)) -b V N s , u N s exp iθR N s ds + K N (θ)

  k = 1, 2, and rates αm (V ) and βm (V ) given by αm(V ) = α m (V ) βm (V ) = β m (V )Each sub-matrix Qk (V ) has invariant distribution given by

  Contents 1 Markov process in discrete space and continuous time 1.1 Definition Markov process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Multiple weakly irreducible classes . . . . . . . . . . . . . . . . . . . . . . . . . .

	Chapter 1
	4 Conclusions
	1

1.2.1 Asymptotic expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Two-scales approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Piecewise Deterministic Markov Process 2.1 Ordinary differential equations and vector fields . . . . . . . . . . . . . . . . . . . 2.2 Definition PDMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Markov switching model with multiple weakly irreducible classes . . . . . . . . . 2.3.1 Asymptotic expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Two-scales approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Hodgkin and Huxley model 3.1 H&H as limit of PDMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Langevin approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 State reduction in sodium channels . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Two-scales approximation . . . . . . . . . . . . . . . . . . . . . . . . . . .

  Proof As it is presented on[START_REF] Yin | Continuous-Time Markov Chains and Applications[END_REF] (Lemma A.2 p.374), we first notice that the solution of (1.6) is P (s) = exp (Qs). By virtue of Theorem II.10.1 of[START_REF] Lai | Markov Chains with Stationary Transition Probabilities[END_REF], lim s→∞ P (s) exists and is equal to a constant matrix P . Then we observe that

	lim s→∞	exp(Qs) = lim s→∞	P (s) =	P
	and so lim

a generator of a (homogeneous or stationary) finite-state Markov chain and is weakly irreducible. Then P (s) → P as s → ∞ and exp (Qs) -P ≤ K exp -ks for some k > 0, where P = 1 N (ν 1 , . . . , ν N ) ∈ R N ×N and (ν 1 , . . . , ν N ) is the stationary distribution of the Markov process with generator Q.

  that form the slow-scale states S = {s 1 , s2 , s3 }. By Corollary 1.2.6, we can approximate the law of process (X t ) t to P

	-1/6 e -t	-1/12 e -t( +3)	-1/12 e -t(3 +1)	+ 1/9
	From generator Q, we identify the classes
		s1 = {1, 2, 3} = {i 11 , i 12 , i 12 }
		s2 = {4, 5, 6} = {i 21 , i 22 , i 23 }	(1.18)
		s3 = {7, 8, 9} = {i 31 , i 32 , i 33 }

18 e -3 t + 1/18 e -3 t + 1/36 e -3 t(1+ ) + 1/6 e -t + 1/12 e -t( +3)

+1/6 e -t + 1/12 e -t(3 +1) + 1/4 e -t(1+ ) + 1/9, d dt p (2, t) = 1/18 e -3 t -1/9 e -3 t -1/18 e -3 t(1+ ) + 1/6 e -t -1/6 e -t( +3) + 1/9, d dt p (3, t) = 1/18 e -3 t + 1/18 e -3 t + 1/36 e -3 t(1+ ) + 1/6 e -t + 1/12 e -t( +3) -1/6 e -t -1/12 e -t(3 +1) -1/4 e -t(1+ ) + 1/9, d dt p (4, t) = -1/9 e -3 t -1/18 e -3 t(1+ ) + 1/18 e -3 t + 1/6 e -t -1/6 e -t(3 +1) + 1/9, d dt p (5, t) = -1/9 e -3 t + 1/9 e -3 t(1+ ) -1/9 e -3 t + 1/9, d dt p (6, t) = -1/9 e -3 t -1/18 e -3 t(1+ ) + 1/18 e -3 t -1/6 e -t + 1/6 e -t(3 +1) + 1/9, d dt p (7, t) = 1/18 e -3 t + 1/36 e -3 t(1+ ) + 1/18 e -3 t -1/6 e -t -1/4 e -t(1+ ) +1/6 e -t -1/12 e -t( +3) + 1/12 e -t(3 +1) + 1/9, d dt p (8, t) = 1/18 e -3 t -1/18 e -3 t(1+ ) -1/9 e -3 t -1/6 e -t + 1/6 e -t( +3) + 1/9, d dt p (9, t) = 1/18 e -3 t + 1/36 e -3 t(1+ ) + 1/18 e -3 t -1/6 e -t + 1/4 e -t(1+ )

Table 1 .

 1 1: Comparatives result for the distribution function of X t at t = 1, between the exact solution p , the Monte Carlo approx. p M C and the Monte Carlo approx. of the two-scales process p * M C .

		1769 0.177 0.0120694 0.172 0.0039779
	2	0.1751 0.179 0.0121227 0.172 0.0039779
	3	0.1734 0.167 0.0117945 0.172 0.0039779
	4	0.1066 0.097 0.0093590 0.105 0.0032314
	5	0.1056 0.115 0.0100884 0.105 0.0032314
	6	0.1046 0.095 0.0092723 0.105 0.0032314
	7	0.05307 0.054 0.0071473 0.0563 0.0024304
	8	0.05256 0.057 0.0073315 0.0563 0.0024304
	9	0.05203 0.059 0.0074511 0.0563 0.0024304
	T = 2	p	p M C	std	p * M C	std
	1	0.1338 0.132 0.0107040 0.135 0.0036021
	2	0.1338 0.145 0.0111344 0.135 0.0036021
	3	0.1337 0.144 0.0111024 0.135 0.0036021
	4	0.1108 0.113 0.0100115 0.1113 0.0033156
	5	0.1108 0.103 0.0096120 0.1113 0.0033156
	6	0.1108 0.103 0.0096120 0.1113 0.0033156
	7	0.08869 0.094 0.0092284 0.087 0.0029708
	8	0.08869 0.086 0.0088659 0.087 0.0029708
	9	0.08868 0.08 0.0085790 0.087 0.0029708

Table 1 . 2

 12

	T = 5	p	p M C	std	p * M C	std
	1	0.1122 0.107 0.0097750 0.1126 0.0033329
	2	0.1122 0.101 0.0095289 0.1126 0.0033329
	3	0.1122 0.109 0.0097750 0.1126 0.0033329
	4	0.1111 0.109 0.0098549 0.1086 0.0032806
	5	0.1111 0.107 0.0097750 0.1086 0.0032806
	6	0.1111 0.117 0.0101642 0.1086 0.0032806
	7	0.1099 0.123 0.0104940 0.112 0.0033243
	8	0.1099 0.108 0.0098151 0.112 0.0033243
	9	0.1099 0.118 0.0102018 0.112 0.0033243

: Comparatives result for the distribution function of X t at t = 2, between the exact solution p , the Monte Carlo approx. p M C and the Monte Carlo approx. of the two-scales process p * M C .

Table 1 .

 1 3: Comparatives result for the distribution function of X t at t = 5, between the exact solution p , the Monte Carlo approx. p M C and the Monte Carlo approx. of the two-scales process p

* M C .

  Each channel contains four separate voltage-gates that are open or closed depending on the voltage variation; and it is in an open-state (conductance) if all four gates are open, and close-state (non-conductance) if at least one gate is close. The voltage equation is given by

  2 ≤ C 4T2 δNand E M N (t) 2 ≤ εδ 4 for all N ≥ N 0 = C T 2 ε .Lemma 3.1.4 For the finite variation term F N (t), there existsC > 0 independent of N such that |F N (t)| ≤ C |u N t -g(t)| + |V N t -v(t)|

	Proof First we notice that

  N t , u N t ) t to (v(t), g(t)) proved in Theorem (3.1.1). By Gronwall lemma (1.2.3) we conclude that φ N (θ, t) → φ(θ, t) with

		φ(θ, t) = exp -	1 2	θ 2	0	t	b (v(s), g(s)) ds
	where (v(t), g(t)) t is solution of the deterministic system (3.9).
	It follows from Theorem 3.2.1 that when N is large enough, u N t degenerates and behaves in law
	equal to the process
	u N 0 +	0	t	b V N s , u N s ds +	0	t	1 N	b (V N s , u N s )dW s
	and thus when N is large enough Z N t = V N t , u N t , the solution of the system
					      	d dt u N V N t = f V N t , u N t t = i=1 N e i (t) 1 N	(3.15)
	degenerates and tends identical in law to the diffusion approximation ZN t = Ṽ N t , ũN t , solution
	of the system
	    	d Ṽ N t dũ N t		= f Ṽ N t , ũN t = b Ṽ N t , ũN t	dt dt +	1 N	t , ũN t dW t b Ṽ N	(3.16)

  N t ) t∈[0,T ] , where V N t is the membrane potential with values in R, and u N t corresponds to the proportion of open sodium ion channels with values in E N . Each channel contains four separate voltage-gates: three of type m and one of type h; and it is in an open-state if all four gates are open, and closed-state if at least one gate is closed. Let consider {C i (t)} N i=1 = e h

i (t), e m i (t)

Acknowledgements

This document corresponds to Rapport du Stage of the Master Mathématiques de

This Mémoir e was financed by The Republic of Chile via CONICYT.

(a) To compute the sejour time i. With U 1 ∼ U (0, 1) set ξ = -Λ-1 log (U 1 )

ii. With U 2 ∼ U (0, 1), if U 2 ≤ Λ(X k-1 ,H k-1 ,ξ) Λ set S = S + ξ iii. else return to i.

(b) While k ≤ N and k∆t ≤ S do: i.

For the process (W t ) t∈[0,T ] = Xt , H t t∈[0,T ] over S×R, we then define the jump rate Λ(k, φ(k, h, t))

and the Markov Kernel Π(h) = Π(k, p, h)

We also need to compute Λ(x, h), a rate that bound of Λ(x, φ(x, h, t)) for all t ≥ 0. As λ(h), µ(h) ≤ 1 for all h, we have Λ(x, h) = 1. We set an array Xk , H k N +1 k=1

for the process the pseudo-code is identical to the one already described.

In order to approximate the law p of (X t , H t ) t∈[0,T ] via the Monte Carlo method, we take into account M realizations of (X m k , H m k ) N +1 k=1 for m = 1, . . . , M , then we have that

M

. And similarly, for the two-sale approxima-

for m = 1, . . . , M , and then it follows

and standard deviation

Solving for p 0 = (δ 0 , 0, 0, 0), = 0.2, α = 2, β = 1, = 0.2 and M = 3000, the results are the followings: typical conductance are E K = -12 mV, E Na = 120 mV, E L = 10.6 mV, g K = 36 mS/cm 2 , g Na = 120 mS/cm 2 , g L = 0.3 mS/cm 2 ; and C = 1 µF/cm 2 is the membrane capacitance.

H&H as limit of PDMP

Let (Ω, F, {F t } t , P) be a complete filtered probability space and a finite T > 0. We present a neuron model (see [START_REF] Pakdamana | Asymptotic expansion and central limit theorem for multiscale piecewise-deterministic Markov processes[END_REF], [START_REF] Pakdamana | Fluid limit theorems for stochastic hybrid systems with application to neuron models[END_REF]) with N K potassium channels and N Na sodium channels (N = N K + N Na ), as a PDMP

where

with rates α u and β u as defined in equations (3.2). Then, the variables u Nu t , for u ∈ {m, h, n}, are defined by

characterized by

which is time-dependent through V ,

• Markov transition kernel Π such that:

The corresponding membrane equation is

where

Under suitable initial conditions, the solution

) converges in probability as N grows to infinity, uniformly on bounded intervals [0, T ], to the solution Z(t) = (v(t), n(t), m(t), h(t)) of the deterministic equation:

when the following conditions are satisfied, H1 α u and β u ∈ C 1 for u ∈ {m, h, n}

H3

The process v from (3.7) is bounded on [0, T ] with T > 0, and for all N ≥ 1 the process

Since the process of opening and closing by assumptions are asymptotically independent among species, we can consider the study of only one gate type. Thus, we will consider the equation

where {e i } i=1...N are analogously defined as in (3.3), and the deterministic system is given by

We assume the conditions H1 -H3 are satisfied for this reduced case. Thus, we have the following theorem as it is presented in [START_REF] Pakdamana | Fluid limit theorems for stochastic hybrid systems with application to neuron models[END_REF].

For the first term we have that

given that g(t) ∈ [0, 1]. Identically we have that

Now we have all the necessary tools to prove Theorem 3.1.1.

Proof Theorem 3.1.1. We want to apply the Gronwall lemma to the function

s)ds and using the last lemma, (a + b + c) 2 ≤ 4(a 2 + b c + c 2 ) and CauchySchwarz inequality we have that

Wee need now to work on V N t -v(t) 2 , using hypothesis H1 between the jumps, we have

Then, by CauchySchwarz inequality and ab ≤ 1 2 (a 2 + b 2 ) holds

Putting together both inequality we obtain:

where B = max 2K 1 + K 2 , 8T C 2 that does not depend on N , and

We control the initial condition and we control the martingale, and A can be chosen arbitrarily small with great probability, then for ε > 0 there exists N 0 such that A(N ) ≤ ε for all N ≥ N 0 . By Gronwall lemma 1.2.3 we have f (t) ≤ ε exp(BT ) for all t ∈ [0, T ] and N ≥ N 0 . The proof conclude with Chebychev and Doob inequalities for each term.

Remark In order to include the 3 different voltage-gates of type u ∈ {m, h, n}, one should just write the same arguments for all the 3 processes u N t , and include all the |u N t -g(t)| 2 in the function f (t) of the Gronwall lemma.

Langevin approximation

A second result corresponds to a central limit theorem that provides a way to build a diffusion or Langevin approximation to the solution of the stochastic system (3.8). As before let

solution of the system (3.8), and we define the process (R N t ) t∈[0,T ] such that

where b(V, u) := (1 -u)α(V ) -uβ(V ).

Theorem 3.2.1 Under the same hypothesis of Theorem 3.1.1, the process (R N t ) t defined in (3.13), converges in law as N → ∞ to the process (R t ) t∈[0,T ] with

where Z(t) = (v(t), g(t)) is solution of the deterministic system (3.9) with initial condition Z(0) = Z N 0 for all N , and (W t ) t∈[0,T ] is a standard Brownian motion in R.

Proof First, we notice that

We want to compute the characteristic function φ N (θ, t) of R N t , defined by

where α, β depend on V and are defined in (3.2) for voltage-gate of type h and m. For any V ∈ R, C i (t) is a stochastic process with states in I = {(0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1), (1, 2), (1, 3)} and generator

Then, the variable u N t is defined by

that is, the ion channel in only open in state [START_REF] Graham | Stochastic Simulation and Monte Carlo Methods[END_REF][START_REF] Davis | Piecewise-Deterministic Markov Processes: A General Class of Non-Diffusion Stochastic Models[END_REF]; and the corresponding membrane equation is

with initial condition V N 0 = V 0 and u N 0 = u 0 .

Two-scales approximation

In order to perform the approximation, we first need to find the time-scale separation of the two sub processes. For that, we notice that given the rate function given by H&H, the voltagegates of type m fluctuate much more rapidly than the voltage-gate h, so we look for some such that

for all V ≈ [0, 80], that is around where the membrane potential fluctuate; that gives us a mean value of = 0.2051. Then, we have that Q (V ) depends on and two generator Q(V ) and Q(V ) that satisfies

dynamic of each voltage-gates of type m, {e m i,fast } N i=1 with sates in {0, 1, 2, 3}, which satisfies

for all i = 1, . . . , N . And the slow-scale process (W t ) t = ( Ṽ N t , ũN t ) t , defined over R × [0, 1], is a Neuron model that only depends on the sequence of voltage-gate {e h i (t)} N i=1 , each one with values in {0, 1} and generator Q(V ), that is

Then the proportion of open sodium channels ũN t , depends dynamically on the voltage-gates of type h and statically on the voltage-gates of type m, that affects the process from the fast-scale, such that

and the membrane equation is equal to

with initial condition Ṽ N 0 = V 0 and ũN 0 = u N 0 .

Numerical results

First we present a simulation of the diffusion approximation (3.16) for the original H&H model (3.6); and next, to the same model after the state-reduction method applied on the sodium channel as shown in secction 3.3. Chapter 4

Conclusions

When we identify fast moving components on a Markov process and a small parameter that separates the fast component from the slow ones, we can expect that in short intervals of time the process fluctuates only on a subset of the state space, and in long intervals of time we observe the emergent of a new, slow Markov process on a reduced state space which ignores the fast fluctuation inside each class. The generator matrix Q of this process can be rewritten as a double scales generator that depends on as in the form Q = Q/ + Q, where Q holds the information of the fast process and Q of the slow one, this characterization together with the asymptotic expansion method of the law presented in Yin [START_REF] Yin | Continuous-Time Markov Chains and Applications[END_REF], allows us to properly identify the components of the slow and fast processes and it gives us an approximation error of the laws of order . This method creates an opportunity for the simulation and numerical analysis of Markov processes, because it turns a complicated problem into two simpler ones, and instead of simulating the process on the complete state space we can simulate only the slow process on a reduced space, much easier to simulate, and then use the fast process to identify the position of the Markov process at each time needed.

Markov process on a finite set are very important in their modeling capabilities but there are not sufficient for processes with more complicated dynamics, so we introduce Piecewise deterministic Markov processes or PDMP. PDMP were proposed by Davis [4] and are a family of non-diffusion processes that consists on two sub-processes, one on a continuous-space and the other on a discrete-space, and together are a Markov process involving a deterministic motion given by the solution of a differential equation, and exponentially distributed random jumps. As the rapidly moving component occurs on the finite-state process we applied the two-scales approximation method on a sub-group of PDMP called the Markov switching model, where the jumps only happen in the discrete-state process, by applying the asymptotic expansion method of the law.

By the work of Pakdamana et al [START_REF] Pakdamana | Fluid limit theorems for stochastic hybrid systems with application to neuron models[END_REF], PDMP have a natural application in neuroscience via the PDMP interpretation of Hodgkin and Huxley model (H&H) of the membrane potential of a neuron. And as H&H model has several components moving at different rates it is well suited to perform a two-scales approximation; so we apply it to the sodium channel where we separate the voltage-gates of type h and m into two different time scales.