Nesrine Ben

Mohamad Assaad

Mérouane Debbah

Risk-Sensitive Reinforcement Learning for URLLC Traffic in Wireless Networks

Keywords: URLLC, risk-sensitivity, resource allocation, constrained MDP, reinforcement learning

In this paper, we study the problem of dynamic channel allocation for URLLC traffic in a multi-user multichannel wireless network where urgent packets have to be successfully received in a timely manner. We formulate the problem as a finite-horizon Markov Decision Process with a stochastic constraint related to the QoS requirement, defined as the packet loss rate for each user. We propose a novel weighted formulation that takes into account both the total expected reward (number of successfully decoded packets) and the risk which we define as the QoS requirement violation. First, we use the value iteration algorithm to find the optimal policy, which assumes a perfect knowledge of the controller of all the parameters, namely the channel statistics. We then propose a Qlearning algorithm where the controller learns the optimal policy without having knowledge of neither the CSI nor the channel statistics. We illustrate the performance of our algorithms with numerical studies.

I. INTRODUCTION

In the fifth generation (5G) wireless networks, there are new service categories with heterogeneous and challenging requirements, among them the Ultra Reliable Low Latency (URLLC) traffic [START_REF] Bennis | Ultra-reliable and low-latency wireless communication: tail, risk and scale[END_REF], designed for delay and reliability sensitive applications like real-time remote control, autonomous driving, and mission-critical traffic. In URLLC traffic, the Endto-End (E2E) latency defined by 3GPP is lower than 1 ms along with a reliability requirement of 1 -10 -5 to 1 -10 -9 [START_REF] Bennis | Ultra-reliable and low-latency wireless communication: tail, risk and scale[END_REF], [START_REF] Ji | Ultra-reliable and low-latency communications in 5G downlink: Physical layer aspects[END_REF].

A plausible solution to address the latency requirement issue is to make transmissions without Channel State Information (CSI) knowledge at the transmitter side. To increase reliability, exploiting frequency diversity is beneficial, and occurs by making parallel transmissions of the same packet over different subcarriers in an Orthogonal Frequency Division Multiplexing (OFDM) system where each subcarrier experiences different channel characteristics.

However, this solution is costly in terms of system capacity. Therefore, the number of parallel transmissions should not be fixed in advance but should rather be variable and depending on many parameters such as the position of a user in the cell, or the statistics about his packet losses over the previous time slots. For example, if a user experienced a high number of packet losses in the previous time slots, it should be allocated a high number of subchannels to increase his success probability, whereas a user with a low number of dropped packets may be assigned a low number of subcarriers. Hence, it is crucial to design efficient dynamic schemes able to adapt the number of parallel transmissions for each user to his experienced QoS.

In this work, we study the problem of dynamic channel allocation for URLLC traffic in a multi-user multi-channel wireless network under QoS constraints. A channel here refers to a frequency band or a subcarrier in an OFDM system, and the QoS is related to the packet loss rate for each user, defined as the average number of dropped packets. Besides, we introduce the notion of risk related to the violation of the QoS requirements; more precisely, a risk occurs or equivalently, a risk state is reached when the QoS requirement is violated for a user. Furthermore, we consider that the transmitter does not have neither the CSI nor the channel statistics at the transmission moment. In fact, due to the urgency of URLLC packets mentioned previously, there is not enough time for the BS to make channel estimation and probing techniques like in conventional wireless communications.

A. Related Work

The issue of deadline-constrained traffic scheduling has been investigated by several works including [START_REF] Dua | Downlink wireless packet scheduling with deadlines[END_REF]- [START_REF] Cayci | Learning for serving deadline-constrained traffic in multi-channel wireless networks[END_REF]. For example, in [START_REF] Cayci | Learning for serving deadline-constrained traffic in multi-channel wireless networks[END_REF], the authors study the problem of dynamic channel allocation in a single user multi-channel system with service costs and deadline-constrained traffic. They propose online algorithms to enable the controller to learn the optimal policy based on Thompson sampling for multi-armed bandit problems. The MDP framework and reinforcement learning approaches for downlink packet scheduling are considered in [START_REF] Dua | Downlink wireless packet scheduling with deadlines[END_REF], [START_REF] Destounis | Scheduling URLLC users with reliable latency guarantees[END_REF], [START_REF] Aggarwal | OFDMA downlink resource allocation via ARQ feedback[END_REF]- [START_REF]Joint resource optimization and relay selection in cooperative cellular networks with imperfect channel knowledge[END_REF]. In [START_REF] Dua | Downlink wireless packet scheduling with deadlines[END_REF], the authors propose an MDP for deadline-constrained packet scheduling problem and use dynamic programming to find the optimal scheduling policies. The authors do not consider QoS constraints in the scheduling problem.

Most risk-sensitive approaches consist in analyzing higher order statistics than the average metric such as the variance of the reward [START_REF] Bennis | Ultra-reliable and low-latency wireless communication: tail, risk and scale[END_REF], [START_REF] Gosavi | Finite horizon Markov control with one-step variance penalties[END_REF]- [START_REF] Assaad | Risk sensitive resource control approach for delay limited traffic in wireless networks[END_REF]. For instance, a risk-sensitive reinforcement learning is studied in [START_REF] Vu | Ultrareliable communication in 5G mmWave networks: A risk-sensitive approach[END_REF] in millimeter-wave communications to optimize both the bandwidth and transmit power. The authors consider a utility (data rate) that incorporates both the average and the variance to capture the tail distribution of the rate, useful for the reliability requirement of URLLC traffic. The authors do not exploit frequency diversity.

In this work, we consider an alternative approach to the risk which consists in minimizing the risk state visitation probabil-ity. In fact, due to the stochastic nature of the problem (timevarying channel and random arrival traffic in our context), giving a low reward to an undesirable or a risk-state may be insufficient to minimize the probability of visiting such state [START_REF] Geibel | Risk-sensitive reinforcement learning applied to control under constraints[END_REF]. Therefore, in addition to the maximization of the total expected reward, we propose to consider a second criterion which consists in minimizing the probability of visiting risk states where a risk state here is related to the violation of QoS requirements.

B. Addressed Issues and Contribution

In this work, we address the following issues:

• We formulate the dynamic channel allocation problem for URLLC traffic as a finite-horizon MDP wherein the state represents the QoS of the users, that is, the average number of dropped packets or packet loss rate of the users. The decision variable is the number of channels to assign to each user. We define a risk state as any state where the QoS requirement is violated for at least one user. Besides, we define a stochastic constraint related to the risk state visitation probability. • Assuming the channel statistics are known to the controller, we use the finite-horizon value iteration algorithm to find the optimal policy to the weighted formulation of the problem, which takes into account both the total expected reward over the planning horizon and the risk criterion (QoS requirement violation probability). • When the channel statistics are unknown to the controller, we propose a reinforcement learning algorithm (Q-learning) for the weighted formulation of the problem, which enables the controller to learn the optimal policy. We illustrate the performance of our algorithms with numerical studies.

C. Paper Structure

In Section II, we present the system model for the multiuser multi-channel wireless network with URLLC packets and time-varying channels along with the QoS definition. In Section III, we introduce the constrained MDP formulation with all its components. In Section IV, we present both the finite-horizon value iteration algorithm and the reinforcement learning algorithm. Section V is devoted to numerical results. Finally, we conclude the paper in Section VI.

II. SYSTEM MODEL

We consider a multi-user multi-channel wireless network where URLLC packets have to be transmitted over timevarying and fading channels. Due to the strict latency requirement of URLLC packets in 5G networks mentioned previously, there is not enough time for the BS to estimate the channel, and the packets are then immediately transmitted in the absence of CSI at the transmitter side. When a packet is successfully decoded, the receiver sends an acknowledgment feedback, which is assumed to be instantaneous and errorfree. We consider a centralized controller which dynamically Controller user 1

user K ρ 1 (t + 1) ρ K (t + 1)
. . . distributes the channels to the users based on their QoS (see Fig. 1).

a 1 (t) a K (t) ℓ 1 (t) ℓ K (t)
Furthermore, we make the following assumptions: Packet arrival process: the packet arrival process is considered as an independent and identically distributed (i.i.d.) random process over a finite set I = {0, 1, .., A max }, where A max is a positive constant, and is identical for all the users. Let α a denote the probability that a ∈ I packets arrive for a given user at the beginning of a time slot.

Deadline-constrained traffic: regarding the strict URLLC latency requirement specified by 3GPP (lower than 1 ms), each packet has a lifetime of one time slot and can either be served or dropped; if there are available channels, the packet will be transmitted, otherwise, it will be dropped because after one time slot it becomes outdated and useless. Furthermore, one packet is transmitted per channel.

Channel model: we consider i.i.d. Bernoulli channels with a mean µ ∈ [0, 1]. In millimeter-wave communications, the links are characterized by their intermittence and high sensitivity, and this channel model reflects the existence of a light-of-sight (LOS) channel state [START_REF] Cayci | Learning for serving deadline-constrained traffic in multi-channel wireless networks[END_REF], [START_REF] Akdeniz | Millimeter wave channel modeling and cellular capacity evaluation[END_REF]. To increase reliability, a user can be assigned more channels than the number of waiting packets (depending on his experienced QoS). Some packets are then simultaneously sent over multiple parallel channels.

Channel split: for each user, all the packets are equally important: when the number of available channels is larger than that of waiting packets, we assume that some packets are picked uniformly at random to be replicated. A packet is obviously more likely to be successfully received when sent over many channels simultaneously. However, assigning more channels to a user will affect the QoS experienced by the other users. Note that channel split across the packets (which occurs in the same manner for all the users) should not be confused with the channel split across the users (which takes into account the QoS perceived by the users).

For user k, the distribution of available channels ℓ k over the waiting packets a k occurs as follows: each packet is transmitted over (ℓ k ∧ a k ) channels and may be furthermore replicated once with a probability ( ℓ k ∨a k a k ), where the symbol ℓ k ∧ a k denotes the larger integer m such that ma k ℓ k , and ℓ k ∨ a k denotes the remaining integer of the division of ℓ k by a k .

The probability that a packet is successfully received given that there are a k waiting packets at the transmitter and ℓ k assigned channels can then be expressed by

ν k (a k , ℓ k ) = 1 - ℓ k ∨ a k a k 1 -(1 -µ) (ℓ k ∧a k ) + ℓ k ∨ a k a k 1 -(1 -µ) 1+(ℓ k ∧a k ) .(1)
The expected number of successfully received packets for user k is then given by

E [N k (ℓ k )] = a k ∈I a k α a k ν k (a k , ℓ k ). ( 2
)
QoS criterion: for each user k, we define the packet loss rate at time slot t, ρ k (t), as follows

ρ k (t) = 1 t t-1 i=0 n k (i) a k (i) , t 1, (3)
where n k (t) denotes the number of lost packets for user k at time slot t. Note that

ρ k ∈ [0, 1] (n k (t) a k (t)).
A packet is lost when either of the two following events occurs: (i) it is not transmitted because of insufficient available channels, (ii) is transmitted but ACK feedback is not received. The parameter ρ k reflects the QoS perceived by user k: higher values of ρ k mean a higher number of lost packets and poor QoS whereas lower values of ρ k mean good QoS. To ensure good QoS for the users, the resource allocation scheme should take account of their experienced QoS and keep this parameter values for all users within an acceptable range.

Finally, the decision variable is the number of channels associated to each user k at each time slot, denoted by ℓ k , which satisfies

K k=1 ℓ k (t) = L, (4)
where L denotes the number of available channels.

III. CONSTRAINED MDP FRAMEWORK

The stochastic nature of the wireless channel incites us to consider an MDP framework to solve the decision problem. In this section, we first introduce the constrained MDP formulation along with its components. We then derive the optimality equations.

A. Model Formulation

We define the following finite-horizon MDP • State Space: is the finite set T × S where T = {0, .., T }, S = {ρ 1 × .. × ρ K }, ρ k for k = 1, .., K is defined in (3), and the symbol × stands for the Cartesian product. • Action Space: is the finite set L = {(ℓ 1 , .., ℓ K ) satisfying (4)}, where ℓ k denotes the number of channels assigned to user k.

• Reward: we define the reward r at time slot t, when the controller chooses action ℓ ∈ L in state s t , as the expected total number of successfully received packets over all the users, that is,

r(s t , ℓ) = E K k=1 N k (ℓ k ) . (5)
Note that the reward depends only on the number of channels allocated for each user (the action), and not on the current state s t . Besides, the reward is a non-linear function of the action.

• Transition Probabilities:

First, we define the probability that n packets are lost for user k as a function of the number of waiting packets a k and the number of assigned channels ℓ k at a given time slot as follows

σ k (n, a k , ℓ k ) = a k n 1 -ν k (a k , ℓ k ) n ν k (a k , ℓ k ) a k -n ,
where n a k and a k n denotes the binomial coefficient. The state transition probability for user k is given by

p(ρ ′ k | ρ k (t), ℓ k ) = α a k σ k (n, a k , ℓ k ), (6)
where

ρ ′ k = t t + 1 ρ k + 1 t + 1 n a k . (7)
Finally, let s t+1 = ρ ′ 1 × .. × ρ ′ K and s t = ρ 1 × .. × ρ K , the transition probability from state s t to state s t+1 given when action l is taken, is then given by

p(s t+1 | s t , ℓ) = K k=1 p(ρ ′ k | ρ k (t), ℓ k ). (8)
Regarding the strict requirements of URLLC packets described earlier, we introduce in the following the notion of a risk-state.

Definition 1. We define a risk state any state where ρ k > ρ max for any k ∈ {1, .., K} with ρ max > 0 is constant fixed by the controller. The set of risk states Φ is then,

Φ = {ρ 1 × .. × ρ K where there ∃ k such that ρ k > ρ max }.
Besides, a risk-state is an absorbing state, that is, the process ends when it reaches a risk state [START_REF] Geibel | Risk-sensitive reinforcement learning applied to control under constraints[END_REF].

A deterministic policy π assigns at each time step and for each state an action. Our goal is to find an optimal deterministic policy π * which maximizes the total expected reward V π T (s) given by

V π T (s) = E π T t=0 r(s t , π(s t ))| s 0 = s , (9)
with the reward r is defined in (5), while satisfying the QoS constraint given by

η π (s) < w, (10)
where η π (s) denotes the probability of visiting a risk state over the planning horizon, given that the initial state (at time slot 0) is s and policy π is followed, and w is a positive constant. Formally,

η π (s) = P π (∃ t such that s t ∈ Φ|s 0 = s). (11)
In order to explicitly characterize η π (s), we introduce in the following the risk signal r.

Definition 2. We define a risk signal r as follows

r(s t , ℓ t , s t+1 ) = 1 if s t+1 ∈ Φ 0 otherwise, ( 12
)
where s t and ℓ t denote the state and action at time slot t, respectively, and s t+1 denotes the subsequent state.

Proposition 1. The probability of visiting a risk-state, η π (s), is given by

η π (s) = V π T (s), (13)
where we set

V π T (s) = E π T t=0 r (s t , π(s t ), s t+1 ) | s 0 = s . ( 14
)
Proof. The random sequence r(t = 0), r(t = 1),.., r(t = T ) may contain 1 if a risk state is visited, otherwise all its components are equal to zero (recall that a risk state is an absorbing state). Therefore, T t=0 r(t) is a Bernoulli random variable with a mean equal to the probability of reaching a risk state, that is, relation (13) holds.

B. Optimality Equations

By virtue of Proposition 1, we associate a state value function V π T to the probability of visiting a risk state. Now, we define a new weighted value function V π ξ,T , which incorporates both the reward and the risk, as follows

V π ξ,T (s) = ξV π T (s) -V π T (s), (15)
where ξ > 0 is the weighting parameter, determined by the risk level the controller is willing to tolerate. The function V π ξ,T can be seen as a standard value function associated to the reward ξrr. The case ξ = 0 corresponds to a minimum-risk policy whereas the case ξ → ∞ corresponds to a maximum-value policy.

Let Π denote the set of deterministic policies, and define

V * T (s) = max π∈Π V π T (s), V * T (s) = min π∈Π V π T (s), V * ξ,T (s) = max π∈Π V π ξ,T (s).
Besides, we define u π t , u π t , and u π ξ,t for 0 t T respectively by

u π t (s) = E π T i=t r(s i , π(s i ))| s t = s , (16)
u π t (s) = E π T i=t r(s i , π(s i ), s i+1 )| s t = s , (17)
u π ξ,t (s) = ξu π t (s) -u π t (s). (18)
Note that V π T incorporates the total expected reward over the entire planning horizon whereas u t incorporates the rewards from decision epoch t to the end of the planning horizon only. Besides, u t (s) is the probability of visiting a risk state given that at time t the system is in state s ∈ {S/Φ}, and is thus a measure of the risk.

The optimality equations are given by (the proof is similar to that in [START_REF] Puterman | Markov Decision Processes: Discrete Stochastic Dynamic Programming[END_REF], chap. 4 and skipped here for brevity)

u * t (s) = max ℓ∈L r(s t , ℓ) + j∈S p(j|s t , ℓ)u * t+1 (j) (19)
u * t (s) = min ℓ∈L j∈S p(j|s t , ℓ) r(s t , ℓ, j) + u * t+1 (j) (20)
u * ξ,t (s) = max ℓ∈L j∈S p(j|s t , ℓ) ξr(s t , ℓ) -r(s t , ℓ, j)

+u * ξ,t+1 (j) , (21)
for t = 0, .., T -1. For the boundary conditions, that is at time slot T , u * T (s), u * T (s), and u * ξ,T (s) are set to zero for each s ∈ S.

In a non-risk state, the reward r is given in ( 5) and the risk signal is equal to zero whereas in a risk state the reward r is set to zero and the risk signal r is set to one.

IV. ALGORITHM DESIGN

In this section, we present two algorithms: (i) finite-horizon value iteration algorithm which assumes that all the model parameters are known to the controller, namely the channel statistics (model-based algorithm), and (ii) reinforcement learning algorithm which does not require the controller's knowledge of channel statistics (model-free algorithm).

A. Value Iteration Algorithm

In order to find a policy that maximizes the weighted value function defined in [START_REF] Geibel | Risk-sensitive reinforcement learning applied to control under constraints[END_REF], we use the value iteration algorithm [START_REF] Puterman | Markov Decision Processes: Discrete Stochastic Dynamic Programming[END_REF]. In this algorithm, we proceed backwards: we start by determining the optimal action at time slot T for each state, and successively consider the previous stages, until reaching time slot 0 (see Algorithm 1).

Algorithm 1 Finite-Horizon Value Iteration Algorithm

B. Risk-Sensitive Reinforcement Learning Algorithm

During the learning phase, the controller gets estimates of the value of each state-action pair. It updates its estimates through the interaction with the environment where at each iteration it performs an action and then observes the reward r, the risk signal r, and the next state (see Fig. 2).

The learning controller chooses an action at each learning step following the ε-greedy policy, that is, it selects an action that maximizes its current estimate with probability 1 -ε, or a random action with probability ε. The parameter ε captures the exploration-and-exploitation trade-off: when ε → 0, the controller tends to choose an action that maximizes its current state's estimated value; whereas when ε → 1, the controller tends to choose randomly an action and to favor the exploration for optimality.

The state-action value function is given by [START_REF] Watkins | Q-learning[END_REF], [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF]]

Q π (s t , ℓ) = r(s t , ℓ) + j∈S p(j|s t , ℓ) u π t+1 (j),
where the first term denotes the immediate reward, that is the number of successfully received packets over all the users, when the action l is performed in state s t ; and the second term denotes the expected reward when the policy π is followed in the subsequent decision stages. Similarly to the state-action value function associated to the reward, we define the stateaction value function associated to the risk

Q π as Q π (s t , ℓ) = j∈S p(j|s t , ℓ) r(s t , ℓ, j) + u π t+1 (j) .
Note that the introduction of the signal risk r enabled us to define a state-action value function, Q to the risk. Besides, the state-action value function associated to the weighted formulation, Q π ξ , is given by

Q π ξ (s t , ℓ) = ξQ π (s t , ℓ) -Q π (s t , ℓ).
Finally, the Q-function updates at the learning step n (which should not be confused with the decision epoch t) are given by [START_REF] Watkins | Q-learning[END_REF]]

Q (n+1) (s t , ℓ) ← 1 -α n (s t , ℓ) Q (n) (s t , ℓ) + α n (s t , ℓ) r + max ℓ∈L {Q (n) (s t+1 , ℓ)} , (22)
Q (n+1) (s t , ℓ) ← 1 -α n (s t , ℓ) Q (n) (s t , ℓ) + α n (s t , ℓ) r + min ℓ∈L {Q (n) (s t+1 , ℓ)} , (23) and,
Q (n+1) ξ (s t , ℓ) ← 1 -α n (s t , ℓ) Q (n) ξ (s t , ℓ) + α n (s t , ℓ) ξr -r + max ℓ∈L {Q (n) ξ (s t+1 , ℓ)} , (24)
where α n (s t , ℓ) denotes the learning rate parameter at step n when the state s t and action ℓ are visited.

The learning algorithm converges to the optimal stateaction value function when each state-action pair is performed infinitely often and when the learning rate parameter satisfies for each (s t , ℓ) (the proof is given in [START_REF] Watkins | Q-learning[END_REF], [START_REF] Bertsekas | Neuro-dynamics Programming[END_REF] and skipped here for brevity),

∞ n=1 α n (s t , ℓ) = ∞, (a) and ∞ n=1 α 2 n (s t , ℓ) < ∞ (b) . ( 25
)
In this case, the Q-functions are related to the value functions as follows

max ℓ∈L {Q(s t , ℓ)} = u * t (s t ), min ℓ∈L Q(s t , ℓ) = u * t (s t ), max ℓ∈L {Q ξ (s t , ℓ)} = u * ξ,t (s t ).
When a risk state is reached during the learning phase, the system is restarted according to the uniform distribution to a non-risk state. In addition, when t T , we consider that an artificial absorbing state is reached and we reinitialize t (see Algorithm 2).

Algorithm 2 Q-learning Algorithm 1: Initialization t ← 0, s 0 ← s, n ← 1, 2: for each ℓ ∈ L 3: Q(s 0 , ℓ) ← 0, Q(s 0 , ℓ) ← 0, Q ξ (s 0 , ℓ) ←

V. PERFORMANCE EVALUATION

In this section, we present the numerical results obtained with the value iteration and the learning algorithms in a variety of scenarios. We consider the setting of two users along with a number of channels L = 5. For the arrival traffic, we consider the following truncated Poisson distribution

Prob(a = m) = λ m /m! Amax i=0 λ i /i! if m A max zero otherwise, (26)
where λ = 3 and A max = 6. The mean of the Bernoulli channel µ and the value of the parameter ρ max throughout this section are fixed to 0.6 and 0.55 respectively.

A. Minimum-risk vs maximum-value policy

First, we compare the performance of the minimum-risk policy (obtained when ξ = 0), maximum-value policy (obtained when ξ → ∞), weighted policy (when ξ > 0), and the fixed policy which consists is assigning the same number of channels for each user at each time slot (ℓ 1 = 2 and ℓ 2 = 3).

We depict in Fig. 3-top the reward u t (s) given in [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF] as a function of time when s = 0.3 × 0 and different policies are followed. We observe that the maximum-value policy clearly outperforms the fixed and the minimum-risk policy. In Fig. 3-bottom showing u t (s) given in [START_REF] Bertsekas | Neuro-dynamics Programming[END_REF], we observe that the probability of visiting a risk-state when the fixed policy is followed is much higher than that obtained when the minimum-risk policy π * is performed. For example, at the time step t = 5, u t (s) is equal to 0.42 when the policy π f is performed whereas this value reduces to 0.02 when the policy π * is followed. In fact, the fixed policy does not take account of the experienced QoS of the users, and therefore, it is the policy which results in the highest risk-state visitation probability. Besides, this probability decreases over time for all the policies. In fact, as time goes on, the probability of entering a risk-state over the remaining time steps decreases.

The reward u t (s) increases for the lower values of t until reaching a maximum value and then it decreases, for all the policies. In fact, for the lower values of t, the probability of visiting a risk-state is high, and this affects the expected value of the reward (recall that in the risk state, the reward is equal to zero). As time goes on, this probability decreases, and thus the expected reward increases. However, at the further time steps, the number of remaining decision stages is low and hence the expected reward (total number of successfully received packets over the remaining time slots) decreases.

B. Learning

In the learning algorithm, we simulate the wireless channel with a Bernoulli random variable with a number of trials equal to the number of channels associated to each packet for each user. For the learning rate parameter α n , we considered the following expression [START_REF] Even-Dar | Learning rates for Q-learning[END_REF]: Fig. 3: Performance of the minimum-risk policy π * , the maximum-value policy π * , the weighted-policy π * ξ with ξ = 0.1, and the fixed policy π f . On the top, u t (s), on the bottom, u t (s) where s = 0.3 × 0 and T = 9.

α n = 1 (1 + n(s t , ℓ)) γ , (27)
where n(s t , ℓ) denotes the number of times the state-action pair (s t , ℓ) was visited until iteration n, and γ is a positive parameter ∈ [0.5, 1] [21].

We depict in Fig. 4 the optimal (minimum-risk) policy (number of channels to assign to user 1 , ℓ 1 ∈ [0, .., 5]) computed by the learning algorithm, as a function of time steps (decision epochs) and ρ 1 , when ρ 2 is fixed to 0. The figure shows a monotony property: the number of channels to assign to user 1 increases with time and with ρ 1 . In fact, as the QoS of user 1 degrades (ρ 1 increases), more channels are assigned to it to compensate for this degradation; and as time goes on, this policy is more sensitive to this degradation as more channels are assigned for the same values of ρ 1 , but at further time steps.

VI. CONCLUSION

In this work, we studied the problem of dynamic channel allocation for URLLC traffic in a multi-user multi-channel wireless network within a novel framework. Due to the stochastic nature of the problem related to time-varying, fading channels and random arrival traffic, we considered a finite-horizon MDP framework. We determined explicitly the probability of visiting a risk state and we wrote it as a cumulative return (risk signal). We then introduced a weighted global value function which incorporates two criteria: reward and risk. By virtue of the value iteration algorithm, we determined the optimal policy. Furthermore, we used a Q-learning algorithm to enable the controller to learn the optimal policy in the absence of channel statistics. Besides, we examined the impact of the learning rate parameter on the accuracy of the learning algorithm.

In the future work, we would like to take account of spatial diversity in the dynamic allocation scheme where both the BS and the user terminals can be equipped with multiple antennas to enhance the system performance.

Fig. 1 :

 1 Fig. 1: Dynamic allocation of channels (ℓ 1 , .., ℓ K ) to the users based on their QoS (ρ 1 , .., ρ K ).

Fig. 4 :

 4 Fig. 4: Optimal policy ℓ * 1 as a function of time steps and ρ 1 with ρ 2 = 0 and T = 5.

0 4 :

 4 End for 5: Repeat 6: observe current state s t 7: select and perform action ℓ in state s t 8: observe the new state s t+1 , reward r and the risk r 9: update the Q-functions Q(s t , l), Q(s t , l), Q ξ (s t , ℓ) if t = T , then t ← 0 artificial absorbing state reached 14: if s t ∈ Φ, then s t ∼ Unif{S/Φ} absorbing state reached 15: until convergence

	according to (22), (23), (24) respectively
	10: t ← t + 1
	11: n ← n + 1
	12: update α n
	13:


Yosu Luque

M.D Dominique Cathelin

Ph.D Sophie Vandermeersch

Xiaoli Xu

M.D Julie Sohier

M.D Sandrine Placier

Yi-Chun Xu-Dubois

M.D Kevin Louis

Alexandre Hertig

Ph.D Jean-Christophe Bories

Ph.D Florence Vasseur

Fabien Campagne

Ph.D James P Di Santo

Ph.D Christian Vosshenrich

Ph.D Eric Rondeau

Dr Laurent Mesnard
email: laurent.mesnard@aphp.fr

Glomerular common gamma chain confers B-and T-cell-independent protection against glomerulonephritis

Keywords: Epithelial common gamma chain attenuates crescentic glomerulonephritis glomerulonephritis, gC chain, lymphocyte, podocyte, IL-2Rβ, IL-15 C Urea

Introduction

Human crescentic glomerulonephritis (CG) and rapidly progressive glomerulonephritis (RPGN) are life threatening immune-mediated diseases 1 . For many years, the pathogenesis of human RPGN has been studied using the experimental model of passive anti-glomerular basement membrane glomerulonephritis (anti-GBM-GN) whereby heterologous anti-glomerular basement membrane (GBM) serum is injected into rats or mice [START_REF] Nahas | Masugi nephritis: a model for all seasons[END_REF] . Following injection, there is a strong inflammatory response, which in a few days leads to severe glomerular and tubulo-interstitial injuries [START_REF] Salant | Experimental glomerulonephritis[END_REF][START_REF] Tipping | T cells in crescentic glomerulonephritis[END_REF] : renal cells become surrounded by lymphocytes, a feature reminiscent of human CG 5- 10 . The development of anti-GBM-GN lesions in mice has been shown to involve a CD4 + T cell response through either Th1 [START_REF] Tipping | T cells in crescentic glomerulonephritis[END_REF][START_REF] Kitching | Crescentic glomerulonephritis--a manifestation of a nephritogenic Th1 response?[END_REF][START_REF] Kitching | Endogenous interleukin-10 regulates Th1 responses that induce crescentic glomerulonephritis[END_REF][START_REF] Kitching | IL-12p40 and IL-18 in crescentic glomerulonephritis: IL-12p40 is the key Th1-defining cytokine chain, whereas IL-18 promotes local inflammation and leukocyte recruitment[END_REF][START_REF] Tipping | Glomerulonephritis, Th1 and Th2: what's new?[END_REF] or Th17 [START_REF] Paust | The IL-23/Th17 axis contributes to renal injury in experimental glomerulonephritis[END_REF][START_REF] Steinmetz | The Th17-defining transcription factor RORγt promotes glomerulonephritis[END_REF][START_REF] Turner | IL-17A production by renal γδ T cells promotes kidney injury in crescentic GN[END_REF] pathways, while the T cell subset of invariant natural killer T (iNKT) cells exerts a protective role independently of T cell polarization [START_REF] Mesnard | Invariant natural killer T cells and TGF-beta attenuate anti-GBM glomerulonephritis[END_REF] .

Interestingly, other studies have reported that mice or rats born athymic or "nude", i.e. constitutively lacking T cells, still develop classical features of GN after anti-GBM serum injection, which is hypothesis-generating, and provides an opportunity to explore the particular role potentially played by renal cells, independently from T cells [START_REF] Kusuyama | Nephrotoxic nephritis in nude mice[END_REF][START_REF] Sato | Nephrotoxic serum nephritis in nude rats: the roles of host immune reactions[END_REF] , and by intra-renal signaling pathways or local factors normally used by lymphocytes. We thus induced an experimental glomerulonephritis in several immunodeficient mice strains: Rag2 -/-, Rag2 -/-Il2rg -/-and Rag2 -/- Il2rb -/-which are devoid of either T/B cells or T/B/NK cells. Compared to Rag2 -/-mice, Rag2 -/- Il2rg -/-and Rag2 -/-Il2rb -/-strains carry the additional deletion of either the interleukin IL-2 receptor common gamma chain (gC) gene (Rag2 -/-Il2rg -/-) or the IL-2 receptor β subunit (IL-2Rβ) gene (Rag2 -/-Il2rb -/-). Several interleukin receptor complexes share the gC subunit (IL-2, IL-4, IL-7, IL-9, IL-15, and IL-21) [START_REF] Ring | Mechanistic and structural insight into the functional dichotomy between IL-2 and IL-15[END_REF] . Importantly, IL-15 and IL-2, together with an IL-15Rα or IL-2Rα co-receptor respectively, have their action further restricted to the β subunit. The trimeric IL-15 receptor (IL-15Ra, IL-2Rβ, gC) is necessary for NK cell ontogeny and explains the absence of NK cells in Rag2 -/-Il2rg -/-or Rag2 -/-Il2rb -/-mice. In this paper we show that gC/ IL-2Rβ complexes expressed at the surface of intrinsic renal cells play an important role in the organization of renal lesions during severe immune-mediated glomerulonephritis.

Results

Rag2 -/-Il2rg -/-mice are more susceptible than Rag2 -/-mice to anti-GBM serum Experiments were first carried out to test the effect of anti-GBM serum in Rag2 -/-mice devoid of T/B cells, Rag2 -/-Il2rg -/-mice devoid of T/B/NK cells, and wild type (WT) mice from the same C57BL/6J background. Administration of anti-GBM serum impaired renal function parameters and induced lesions typical of glomerulonephritis in WT, Rag2 -/-and Rag2 -/-Il2rg -/-mice (Figure 1A) evaluated at day 9. No marked differences were observed in the levels of blood urea nitrogen (BUN) and percentage of pathologic glomeruli between anti-GBM treated WT mice and Rag2 -/- mice (Figures 1B &1C). However, the score of interstitial lesions, assessed by tubular dilations and epithelial atrophy, was lower in anti-GBM treated Rag2 -/-mice compared to anti-GBM-GN treated WT mice (Figure 1D). The levels of plasma urea, proteinuria and the severity of renal histological lesions were even greater in anti-GBM treated Rag2 -/-Il2rg -/-mice than in anti-GBM treated Rag2 -/-mice (Figure 1B-E). Anti-GBM-GN treated Rag2 -/-Il2rg -/-mice consistently exhibited more significant tubulo-interstitial lesions, CD44 de novo staining (Supplemental Figure 1), a higher percentage of pathologic glomeruli (Figures 1C &1D) and a more important decrease in the podocytic marker podocalyxin (Figure 1F) than anti-GBM treated Rag2 -/-mice.

Therefore, Rag2 -/-mice appeared only partially protected against anti-GBM-GN, while Rag2 -/- Il2rg -/-mice were found to be more susceptible to anti-GBM serum than WT or Rag2 -/-mice.

NK cells have no protective role against renal lesions induced by anti-GBM serum

Rag2 -/-mice have been shown to have a significant proportion of circulating NK cells (around 25% vs 5% in WT mice), while Rag2 -/-Il2rg -/-mice totally lack NK cells [START_REF] Shinkai | RAG-2-deficient mice lack mature lymphocytes owing to inability to initiate V(D)J rearrangement[END_REF] . To determine the role of NK cells in the results shown above, Rag2 -/-Il2rg -/-mice were replenished by NK cells (1x10 6 cells/mouse), isolated from the spleen of WT C57BL/6J mice, and injected together with anti-GBM serum. At day 7, flow cytometry analysis showed that ~ 2% of circulating cells in Rag2 -/- Il2rg -/-mice were NK cells (Figure 2A). At day 9, NK cells were also detected in the mice kidneys (Figure 2B), where an increase in mRNA expression of the Nkg2b/Cd159a NK cell receptor (Figure 2C) was detectable. Despite that, plasma urea levels and renal lesions were similar to non-replenished Rag2 -/-Il2rg -/-mice (Figures 2D to 2F). These findings suggest that NK cells do not account for the severity of glomerulonephritis in Rag2 -/-Il2rg -/-mice.

The lack of γC expression in renal cells worsens renal lesions induced by anti-GBM serum

Using bone marrow graft experiments we then analyzed the role of kidney-expressed gC in the development of renal lesions in anti-GBM-GN. Bone marrow cells isolated from WT or Rag2 -/- CD45.1 + mice were grafted into irradiated Rag2 -/-Il2rg -/-CD45.2 + (referred to as ep.gc -/-) mice, in order to obtain chimeric mice knockout only for the gC chain in non-immune cells. Six weeks after bone marrow transplantation at least 90% of circulating cells expressed the CD45.1 + marker in Rag2 -/-Il2rg -/-CD45.2 + recipients (Supplemental Figure 2A). NK cell counts measured after bone marrow transplantation are represented in Supplemental Figure 2B. In all anti-GBM-GN induced ep.gc -/-chimeric mice, plasma urea levels were consistently higher than in mice with WT epithelia (Figure 3A). All ep.gc -/-chimeric mice exhibited significantly severe glomerular lesions with numerous extra-capillary crescents (Figures 3B and3E) and proteinuria (Figure 3D). In addition, the tubulo-interstitial lesions were more severe in ep.gc -/-mice than in mice with WT epithelia, independent of the origin of the bone marrow (WT or Rag2 -/-) (Figures 3C and3E).

These findings suggest that gC expressed on renal cells plays a protective role during anti-GBM-GN.

Despite the absence of lymphocytes, IL-15 appears as the main effective ligand of γC in the renal cortex during glomerulonephritis

We next analyzed relative expression levels of cytokines (e.g. IL-2, IL-4, IL-7, IL-9, IL-15, and IL-21) acting through receptor complexes containing the common gC subunit in the renal cortex of WT, Rag2 -/-or Rag2 -/-gc -/-mice (Figure 4A) after anti-GBM-GN. In all cases, IL-21 and IL-4 mRNA were undetectable. The level of IL-2 mRNA remained very low in all mouse strains (Figure 4A). IL-7 and IL-9 mRNA were detectable in the renal cortex, but no difference was observed between WT, Rag2 -/-, or Rag2 -/-Il2rg -/-mice. Interestingly, IL-15 mRNA was found to be highly expressed in the renal cortex after anti-GBM-GN in the absence of T/B lymphocytes (Rag2 -/-mice) or even in the absence of T/B/NK cells in Rag2 -/-Il2rg -/-animals (Figure 4A). Thus, and as described by others [START_REF] Shinozaki | IL-15, a survival factor for kidney epithelial cells, counteracts apoptosis and inflammation during nephritis[END_REF] , IL-15 is abundantly produced by intrinsic renal cells, which we confirmed by immunohistochemistry of the renal cortex (Figure 4B). We also showed that from cytokines classically associated with the gC response, only IL-15 plasma levels were found to be higher in Rag2 -/-Il2rg -/-mice compared to WT animals (Supplemental Figure 3A). To explore the role of IL-15, we challenged Rag2 -/-Il15 -/-and Rag2 -/-littermates with anti-GBM serum: plasma urea and proteinuria were found to be higher in Rag2 -/-Il15 -/-mice compared to Rag2 -/-mice (Supplemental Figures 4 A,B,C). Next, we induced an anti-GBM-GN in Rag2 -/- Il2rb -/-mice lacking the gC co-receptor IL-2Rβ. Again, Rag2 -/-Il2rb -/-mice were found to be highly sensitive to anti-GBM serum when compared to Rag2 -/-mice. Both strains developed acute renal failure, yet plasma urea levels (Figure 4C), tubulo-interstitial lesions ( Figures 4E/G) and proteinuria (Figure 4F) were significantly greater in Rag2 -/-Il2rb -/-than in Rag2 -/-mice (Figure 4D). Taken together, these results suggest that in the absence of lymphocytes, IL-15 and its co-receptors gC and IL-2Rβ have a protective role during anti-GBM-GN. IL-15Rα, gC and IL-2Rβ receptor subunits are expressed in renal cells during anti-GBM-

GN

Immunohistochemical studies evidenced noticeable intra-renal expression of gC and its coreceptor complexes, IL-2Rβ and IL-15Rα. Interestingly, γC and IL-15 co-receptors IL-15Rα and IL-2Rβ were not expressed at the basal state in glomeruli. Nevertheless, they are induced during anti-GBM-GN (Figures 5A to C). Co-localization studies in Rag2 -/-mice showed that gC is mainly induced in podocytes during anti-GBM-GN (Figure 5D). These findings indicate that gC, IL-2Rb and IL-15Rα are expressed in renal epithelium and in particular by injured glomeruli during anti-GBM-GN. We also demonstrated the presence of IL15Rα and IL2Rβ proteins from the renal cortex by immunoblot (Supplemental Figure 3B).These receptors were not kidneyspecific Il2rg (gC) or Il2rb splice variants. In particular, RNA-Seq analysis revealed that all "classical" transcripts and exons of gC, originally described in lymphocytes, are expressed in the kidneys of Rag2 -/-mice (Supplemental Figures 5 A,B,C).

Renal epithelial IL-15 signaling is impaired in the absence of gC or IL-2Rβ

In vitro experiments were carried out on primary cultured podocytes in which gC has been demonstrated to be highly expressed. Mice primary podocytes from the Rag2 -/-Il2rb -/-, Rag2 -/- Il2rg -/-or WT kidney cortex were cultivated from glomerular extracts and stimulated by IL-15 (Figure 6A). The purity of glomerular extracts was also confirmed by microscopy (Figure 6B).

To confirm the functionality of IL-15 receptors we studied downstream effectors classically known in lymphocytes: phosphorylated JAK 1/3 and spleen tyrosine kinase (SYK). As described in immune cells, IL-15 induced JAK1/3 and SYK phosphorylation in primary cultured podocytes (Figures 6A/C/D/E). However, despite the absence of gC, JAK1/3 was still induced in primary podocytes from Rag2 -/-Il2rg -/-mice (Figures 6A/C/D), while SYK phosphorylation dramatically decreased (Figures 6A and6E). There was no JAK and SYK phosphorylation in the absence of IL-2Rβ (Figures 6A/C/D/E). Thus, IL-15 signaling on primary podocytes appears to be more dependent on IL-2Rβ than on gC.

Common γ chain is also induced in human podocytes during glomerulonephritis

We finally analyzed the expression of gC on human renal biopsies from patients with crescentic glomerulonephritis, using pristine 3-month post-transplant renal biopsies as controls.

Interestingly, γC was weakly expressed in control biopsies, but was largely induced in glomeruli during glomerulonephritis. Despite being weaker than in its severe form, the expression of gC appears at the early stage of glomerulonephritis and is visible on fibrotic glomerular lesions (Figure 7A). We determined that γC was expressed by human podocytes during glomerulonephritis, indeed gC co-localized with nephrin and synaptopodin (Figure 7B). These results suggest that γC could be an early danger signal expressed by podocytes during glomerulonephritis.

Discussion

Previous studies have described the role of immune cells during the development of renal injury in the passive experimental model of anti-GBM-GN [START_REF] Tipping | T cells in crescentic glomerulonephritis[END_REF][START_REF] Kitching | IL-12p40 and IL-18 in crescentic glomerulonephritis: IL-12p40 is the key Th1-defining cytokine chain, whereas IL-18 promotes local inflammation and leukocyte recruitment[END_REF][START_REF] Turner | IL-17A production by renal γδ T cells promotes kidney injury in crescentic GN[END_REF][START_REF] Holdsworth | Th1 and Th2 T helper cell subsets affect patterns of injury and outcomes in glomerulonephritis[END_REF][START_REF] Kurts | Role of T cells and dendritic cells in glomerular immunopathology[END_REF] . To date, many studies support a central role for T lymphocytes in the generation of renal lesions [START_REF] Tipping | T cells in crescentic glomerulonephritis[END_REF][START_REF] Tan | Thymic deletion and regulatory T cells prevent antimyeloperoxidase GN[END_REF] . Because renal lesions are observed despite the absence of B and T lymphocytes in mice, we propose that intrinsic renal cells play a central role in the development of anti-GBM-GN [START_REF] Akagi | Inhibition of TGF-beta 1 expression by antisense oligonucleotides suppressed extracellular matrix accumulation in experimental glomerulonephritis[END_REF][START_REF] Couser | Basic and translational concepts of immune-mediated glomerular diseases[END_REF][START_REF] Huang | Mechanisms of T cell-induced glomerular injury in anti-glomerular basement membrane (GBM) glomerulonephritis in rats[END_REF][START_REF] Hir | Histopathology of humorally mediated anti-glomerular basement membrane (GBM) glomerulonephritis in mice[END_REF] . Our findings are also in agreement with those of earlier studies that evaluated passive anti-GBM-GN lesions in nude animals [START_REF] Kusuyama | Nephrotoxic nephritis in nude mice[END_REF][START_REF] Sato | Nephrotoxic serum nephritis in nude rats: the roles of host immune reactions[END_REF] . Similar observations regarding glomerular injury in the absence of T cells were also made in Rag1 -/-mice in another experimental model (Alport's Syndrome) in which inflammatory cells are believed to contribute to renal lesions. In this latter study, T cells were not required for the emergence of glomerular injury, but may have contributed to the development of tubular lesions [START_REF] Lebleu | Lymphocytes are dispensable for glomerulonephritis but required for renal interstitial fibrosis in matrix defect-induced Alport renal disease[END_REF] . In our study we find similar results in the passive anti-GBM-GN model, as tubulo-interstitial lesions appear less severe in Rag2 -/-mice compared to those observed in WT mice.

When re-examining the role of T lymphocytes during the generation of anti-GBM-GN lesions it appears that several points could in fact reconcile our findings with previous studies.

First, active and passive experimental anti-GBM-GN models are often unintentionally scrambled. The passive form, used in our study, results from the injection of a heterologous anti-GBM serum, whereas the active model is the result of the "active" immunization of the host against glomerular extracts or specific GBM epitopes breaking self-tolerance [START_REF] Arends | T cell epitope mimicry in antiglomerular basement membrane disease[END_REF][START_REF] Reynolds | Strain differences and the genetic basis of experimental autoimmune antiglomerular basement membrane glomerulonephritis[END_REF][START_REF] Steblay | Glomerulonephritis induced in sheep by injections of heterologous glomerular basement membrane and Freund's complete adjuvant[END_REF] . The active model is largely dependent on cell-mediated immunity (CMI) whilst our work demonstrates that the passive model is not dependent in this way. Second, the very first studies demonstrating the role of CMI during passive anti-GBM-GN were made using either depleting antibodies, with potential off-target effects, or an unstable genetic background [START_REF] Kitching | Interleukin-4 and interleukin-10 attenuate established crescentic glomerulonephritis in mice[END_REF][START_REF] Kitching | Interleukin-4 deficiency enhances Th1 responses and crescentic glomerulonephritis in mice[END_REF][START_REF] Tipping | Crescentic glomerulonephritis in CD4-and CD8deficient mice. Requirement for CD4 but not CD8 cells[END_REF][START_REF] Tipping | T lymphocyte participation in antibody-induced experimental glomerulonephritis[END_REF] . Third, some passive-anti-GBM-GN phenotypes, acquired in mutant mice engineered with defective CMI effectors, could be explained by a defect in renal cells per se or the unsuspected direct action of immunosuppressive drugs on renal epithelial cells [START_REF] Faul | The actin cytoskeleton of kidney podocytes is a direct target of the antiproteinuric effect of cyclosporine A[END_REF] . This defect in intra renal CMI effectors would be evident only during the disease state and has been observed in STAT3 or B7-1 knock out animals. STAT3 signaling has been shown to be engaged in the immune process and during experimental anti-GBM-GN [START_REF] Dai | Podocyte-specific deletion of signal transducer and activator of transcription 3 attenuates nephrotoxic serum-induced glomerulonephritis[END_REF] . B7-1/CD80 was first demonstrated to be an important CMI effector expressed on immune cells acting as costimulatory molecules. Later, B7-1 was found expressed on podocytes and regulating the filtration barrier in immune-mediated kidney diseases [START_REF] Reiser | Induction of B7-1 in podocytes is associated with nephrotic syndrome[END_REF] . Our work also illustrates this point. Indeed, we demonstrate that gC plays an important role in epithelial cells only during glomerulonephritis.

Using double knockout animals (Rag2 -/-Il2rg -/-) and bone marrow graft experiments, we demonstrate that the expression of gC in the kidney plays a major role in the development of passive anti-GBM-GN. To our knowledge ours is the first study to show a role of a tissueexpressed γC chain. This result is obtained in the absence of lymphocytes. Several interleukin receptor complexes (IL-2, IL-4, IL-7, IL-9, IL-15, and IL-21) share the gC subunit [START_REF] Ring | Mechanistic and structural insight into the functional dichotomy between IL-2 and IL-15[END_REF] . As with gC, IL-2Rβ is also part of the co-receptor complex that signals only for IL-2 and IL-15 [START_REF] Lin | The role of Stat5a and Stat5b in signaling by IL-2 family cytokines[END_REF] . But IL-2 or IL-15 signaling requires an additional α subunit leading to functional receptor complexes formed with IL-2Rα/IL-2Rβ/g, or IL-15Rα/IL-2Rβ/g. To date, gC or IL-2Rβ expressions have been reported in primary cultured tubular epithelial cells or renal tumoral cells but not in glomerular cells (e.g. endothelial, mesangial or even visceral epithelial cells) in mice or humans [START_REF] Giron-Michel | Interleukin-15 is a major regulator of the cellmicroenvironment interactions in human renal homeostasis[END_REF][START_REF] Tejman-Yarden | Renal cells express a functional interleukin-15 receptor[END_REF] .

Given either their absence or very low level of expression during anti-GBM-GN, we did not consider that IL-2, IL-4, IL-7, IL-9 -or even IL-21 -played an important role during anti-GBM-GN. These results and the severity of the anti-GBM-GN observed in Rag2 -/-Il2rb -/-mice pointed to IL-15 in particular. We demonstrate that IL-15 per se confers a powerful protective effect against anti-GBM-GN in the absence of lymphocytes. These results are partially in line with the original study by Shinozaki et al. [START_REF] Shinozaki | IL-15, a survival factor for kidney epithelial cells, counteracts apoptosis and inflammation during nephritis[END_REF] Using Il15 -/-mice in the same model, this study demonstrated that IL-15 produces an important anti-apoptotic effect in epithelial cells. However, as opposed to the hypothesis of Shinozaki et al, our study highlights that the IL-15 epithelial protective effect appears marginally T cell driven.

Our in vitro data suggest that podocytes use the classical IL-15/JAK/SYK signaling pathways described in immune cells [START_REF] Ring | Mechanistic and structural insight into the functional dichotomy between IL-2 and IL-15[END_REF][START_REF] Giron-Michel | Interleukin-15 is a major regulator of the cellmicroenvironment interactions in human renal homeostasis[END_REF][START_REF] Tejman-Yarden | Renal cells express a functional interleukin-15 receptor[END_REF][START_REF] Alves | Common gamma chain cytokines: dissidence in the details[END_REF][START_REF] Giron-Michel | Interleukin-15 Plays a Central Role in Human Kidney Physiology and Cancer through the γc Signaling Pathway[END_REF] . Impaired JAK/SYK activation on epithelial cells could therefore explain the severe renal lesions in mice lacking γC or IL-2Rβ compared to their WT counterparts. Some publications suggest a role for SYK during glomerulonephritis [START_REF] Smith | A Spleen Tyrosine Kinase Inhibitor Reduces the Severity of Established Glomerulonephritis[END_REF][START_REF] Ryan | Myeloid cell-mediated renal injury in rapidly progressive glomerulonephritis depends upon spleen tyrosine kinase[END_REF] .

Despite the absence of gC, IL-15 unexpectedly induces JAK phosphorylation. We hypothesize that IL-2Rβ dimerization could occur in the absence of γC but yet still lead to JAKs phosphorylation and thus permit beneficial IL-15 downstream events. Inhibition of SYK with a non-selective SYK inhibitor (fostamatinib) has been shown to confer protection in WKY rats, both during nephrotoxic nephritis [START_REF] Smith | A Spleen Tyrosine Kinase Inhibitor Reduces the Severity of Established Glomerulonephritis[END_REF] and experimental autoimmune nephritis [START_REF] Mcadoo | Spleen tyrosine kinase inhibition attenuates autoantibody production and reverses experimental autoimmune GN[END_REF] . One of the hypotheses put forward in these papers has been the reduction of SYK activity following Fc gamma Receptor I and III activation. In both studies we cannot exclude the possibility that, in spite of serious SYK inhibition, IL-15 levels have been augmented by favorable off-target effects.

Moreover, given its action on FLT3, JAK, LCK and c-Kit [START_REF] Braselmann | R406, an orally available spleen tyrosine kinase inhibitor blocks fc receptor signaling and reduces immune complex-mediated inflammation[END_REF] , fostamatinib appears as a nonspecific inhibitor of SYK, limiting its value. Thus, according to these data, it is hard to reconcile our findings with previous works, given that IL-15 levels were not evaluated coupled with the pleiotropic effects of SYK. Moreover, our results show that IL-15 induces a protecting signal by activating SYK in the epithelium. We think that global inhibition could affect both immune cells and the epithelium which play different roles in shared SYK signaling. Further studies are required in order to analyze the nature of SYK-associated epithelial signaling.

All these findings contribute to our understanding of epithelial-specific IL-15 signaling and suggest that gC and IL-2Rβ may modulate different downstream survival events. IL-2Rβ/JAK interactions have been studied mainly in vitro as part of the IL-2 response rather than that of IL-15 [START_REF] Fujii | Activation of Stat5 by interleukin 2 requires a carboxyl-terminal region of the interleukin 2 receptor beta chain but is not essential for the proliferative signal transmission[END_REF] . Similar to the observed effects in anti-GBM-GN-induced renal lesions, IL-2/IL-15 receptors have been reported to be a key anti-apoptotic factor in both immune and epithelial mammary cells [START_REF] Hughes | The spectrum of STAT functions in mammary gland development[END_REF] . Altogether, the intra-renal expression of IL-15 signaling may represent a key functional modus operandi that limits epithelial lesions associated with glomerulonephritis. These effects may be related to STAT5 action or to other signaling proteins such as SYK which are involved in the IL-15 response [START_REF] Fehniger | Interleukin 15: biology and relevance to human disease[END_REF][START_REF] Huntington | IL-15 trans-presentation promotes human NK cell development and differentiation in vivo[END_REF] . Further studies are needed to determine more precisely the intra-renal role of downstream IL-15 signaling effector events in the model of anti-GBM-GN.

NK cells are notably expanded in Rag2 -/-animals and have been shown to produce large amounts of IL-15 [START_REF] Ranson | IL-15 is an essential mediator of peripheral NKcell homeostasis[END_REF] , which may lead to some beneficial effects, in tubular cells in particular. This might explain why Rag2 -/-mice present fewer tubulo-interstitial injuries than their WT counterparts. But the effect, if any, should be local and dependent on IL-2Rβ. Indeed, the reintroduction of NK cells in Rag2 -/-Il2rg -/-mice was insufficient to protect them against anti-GBM serum in glomeruli but seems to have a protective effect in tubular cells in which only IL-2Rβ could have relayed the IL-15 signal. Similarly, in Rag2 -/-animals, we cannot exclude a possible role for innate lymphoid cells (ILC) which might afford protection by promoting IL-15 production in NK cells as hypothesized above. Further studies are needed to specifically address these questions.

In conclusion we demonstrated that significant renal lesions occur in the absence of T and B or even NK cells during experimental passive anti-GBM-GN. We also showed that epithelialexpressed immune receptors such as gC and its IL-2Rβ co-receptor can largely modulate the protective effect of IL-15, an interleukin which is highly expressed by the kidney epithelium. We also confirmed that human podocytes express γC during glomerulonephritis suggesting that renalexpressed immune receptors such as gC/IL-2Rβ per se might represent unsuspected targets during glomerulonephritis.

Methods

Animals

C57BL/6J WT mice were obtained from the Charles Rivers Laboratory (L'Arbresle, France). C57BL/6J Rag2 -/-Il2rg -/-CD45.1 mice were kindly provided by S. Ezine and F.Vasseur (Inserm U1020, Paris, France). C57BL/6J Rag2 -/-CD45.2 mice and Rag2 -/-Il2rg -/-mice were provided by J.C. Bories (Centre Hayem EA 3963, Paris, France). C57BL6J Rag2 -/-Il2rb -/-mice and Rag2 -/-Il15 - /-mice were provided by J. Di Santo (Institut Pasteur, Paris, France). Description of the immunological phenotypic of these different mice strains has been previously described [START_REF] Ranson | IL-15 is an essential mediator of peripheral NKcell homeostasis[END_REF][START_REF] Disanto | Lymphoid development in mice with a targeted deletion of the interleukin 2 receptor gamma chain[END_REF] .

Mice submitted to sanitary control tests to ensure proper pathogen-free status were housed in the same animal facility before any experiments. All mice were backcrossed at least 12 times into a C57BL/6J background to ensure a similar genetic background. We provide body weights and basal renal phenotypes in Supplemental Table 1. All mice were held in the same SPF conditions during breeding.

Experimental anti-GBM glomerulonephritis

Passive experimental anti-GBM-GN was induced using decomplemented sheep anti-rat GBM serum prepared as previously described [START_REF] Mesnard | Invariant natural killer T cells and TGF-beta attenuate anti-GBM glomerulonephritis[END_REF] . Passive non-accelerated anti-GBM-GN was induced by daily intravenous administration of 1.5 mg total protein/ g body weight for three consecutive days as previously described [START_REF] Salant | Experimental glomerulonephritis[END_REF] . Renal injury was evaluated on day 9 following the last injection of anti-GBM serum. As controls, mice were injected with phosphate buffered saline (PBS) or normal sheep serum. BUN, urinary protein and creatinine levels were measured using a Konelab enzymatic-spectrophotometric analyzer (Thermo Scientific).

NK cells isolation and transfer in

Rag2 -/-Il2rg -/-mice NK cells were isolated from C57BL/6J WT mice spleen cell suspension using the NK Cell Isolation Kit, an LS Column, and a MidiMACS™ Separator according to the manufacturer's instructions (Miltenyi Biotec). The NK cells were then injected intravenously (1.10 6 cells in 100 µl in sterile PBS per mouse) to Rag2 -/-Il2rg -/-mice.

Bone marrow transplantation

CD45.2 + WT C57BL/6J mice and Rag2 -/-Il2rg -/-mice were X irradiated (700 rad/mouse) to deplete bone marrow progenitors, and then reconstituted 24 h later by intravenous retro-orbital injection of 10 × 10 6 bone marrow white cells obtained from flushed femurs and tibias derived from donor CD45.1 + mice. Mice were then left for six weeks for bone marrow reconstitution.

Histological and immunohistochemical studies

Kidneys were fixed in formalin solution (4%), and embedded in paraffin. Sections (4-µm thick) were stained by Masson's trichrome to determine proportion of positive glomeruli which were defined by crescent formation (defined as glomeruli exhibiting ≥2 layers of cells in Bowman's space, with or without podocyte injury, as indicated by ballooning, necrosis, or cyst formation) and/or fibrin deposition. The proportion of glomeruli affected was determined by a blinded examination on at least 50 glomeruli per section. Tubular lesions were evaluated semiquantitatively as follows: 0, no tubular dilation; 1, dilation in 1-25% of the tubules analyzed; 2, dilation in 26-50% of the tubules analyzed; 3, dilation in 51-75% of the tubules analyzed; 4, dilation in >76% the tubules analyzed.

Immunohistochemical studies were performed on dewaxed kidney 4-µm sections which were incubated for 30 minutes at 95°C in the target retrieval solution (Dako), then in peroxidase blocking reagent (Dako), blocked in PBS containing 5% BSA and immunostained against IL-15 (R&D Systems), γC (Biolegend), IL-2Rβ (Novus Biologicals), IL-15Rα (Novus Biologicals), CD44 (Abcam) and podocalyxin (R&D Systems). Specific staining was revealed using Histofine reagents, which contained anti-rabbit or anti-rat immune-peroxidase polymer.

For NK cell and IL-2Rγc stainings, 4-µm kidney cryosections were blocked with 5% BSA-PBS for 10 minutes at room temperature Kidney sections were then incubated with anti-Ly49G2 (BD Biosciences Pharmingen) or anti-IL-2Rγc (Biolegend) antibodies overnight. For Ly49G2 staining kidney sections were then incubated sequentially with biotinylated secondary antibody, avidinbiotin-peroxidase complex (ABC-Elite; Vector Laboratories), and 3,3'-diaminobenzidine (with nickel chloride enhancement). For IL2Rγc specific staining was revealed using Histofine anti-rat secondary antibody and AEC (Dako). Images were obtained with optical microscopy (Olympus, Hamburg, Germany).

Immunofluorescence assay and confocal microscopy

Rabbit anti-IL2Rγc (Biolegend), rabbit anti-synaptopodin (Santa Cruz) and rabbit anti-nephrin (LifeSpan BioSciences Inc.) antibodies were incubated overnight on frozen kidney slides after blocking with PBS containing 5% BSA. Kidney sections or human tissues were then incubated with anti-rat Alexa Fluor 488, or anti-rabbit Alexa Fluor 546 conjugated secondary antibodies (Invitrogen). The nuclei were stained using DAPI (Invitrogen). Images from mice tissues were obtained with laser scanning confocal microscopy (TCS SP2 AOBS, Leica, Wetzlar, Germany).

Acquisition software used was Leica Confocal Software and images (x630) were processed using Image J®. Images from human tissues were obtained with a wide-field light microscope (IX83, Olympus) and processed using Image J®.

Flow cytometry

Blood samples (50µl) from mice were incubated for 1 h at 4°C with anti-mouse CD45.1, CD45.2 or Ly49G2 primary antibodies or corresponding isotype controls (BD Biosciences Pharmingen).

After washing, cells were then incubated with alexa488-coupled anti-rat IgG (Invitrogen) for 30 min at 4°C. Red cells were then lysed and resuspended cells were analyzed using a FACScanto II flow cytometer (BD Biosciences Pharmingen).

Quantitative RT-PCR

RNA was extracted from whole kidneys using TRIzol solution (Life Technologies). Residual genomic DNA was removed by DNase I treatment (Fermentas). RNA samples (1 µg) were reverse-transcribed using a Revert Aid H minus First Strand DNA Synthesis kit (Fermentas).

Beta-Glucuronidase (Gusb) and Hypoxanthine-guanine phosphoribosyltransferase (Hprt1) genes were used as endogenous reference housekeeping genes. The specific PCR primer sequences used are listed in Supplemental Table 2. Normalization of the quantitative PCR (qPCR) data PCR analysis obtained using a LightCycler 480 (Roche Diagnostics) was performed using the Roche LightCycler 2.0 software (Roche Diagnostics). Data were expressed as 2 -DCp, where Cp corresponds to the cycle threshold number.

In vitro assays in cultured podocytes

Freshly isolated renal cortex from WT, Rag2 -/-Il2rg -/-or Rag2 -/-Il2rb -/-mice were mixed and digested by collagenase I (1 mg/ml; Gibco) in RPMI 1640 (Gibco) for 2 minutes at 37°C. Next, collagenase I was inactivated with RPMI 1640 plus 10% FBS (Hyclone). Tissues were then passed through a 100-µm cell strainer and a 40-µm cell strainer (BD Falcon) in PBS (Euromedex) with 0.5% BSA (Euromedex). Glomeruli, adherent to the 40-µm cell strainer, were removed from the cell strainer with PBS with 0.5% BSA injected under pressure, and finally washed twice in PBS. Freshly isolated glomeruli purity was analyzed by light microscopy (Figure 6B) and then were plated in six-well dishes in RPMI 1640 with 10% FBS, 2% HEPES buffer (Gibco), and 1% penicillin/streptomycin (Gibco). Two days after seeding, glomeruli became adherent to the dish, and podocytes spread out from glomeruli.

Primary podocytes were cultured on RPMI 1640 with 1% FBS for 12 hours and then stimulated 7 days after seeding with mouse recombinant IL-15 (ref. PMP47, Bio-Rad AbD Serotec), at 50 ng/ml for 30 minutes and then mixed in RIPA buffer (Santa Cruz) with orthovanadate, PMSF, protease inhibitor cocktail (Santa Cruz) and NaF and frozen at -80°C for protein extraction.

Western blot analysis from cultured primary podocytes

After extraction from podocytes with lysis buffer, samples were fractionated by SDS-PAGE NuPAGE 4/12% gels (Invitrogen) under reducing conditions and then transferred to a polyvinylidene difloride membrane. Membranes were incubated with the appropriate primary antibodies: anti-phosphorylated JAK1 (Cell Signaling), JAK3 (Cell Signaling), SYK (Novus Biologicals), anti-IL15Rα (Novus Biologicals), anti-IL2Rβ (Novus Biologicals). Protein loading was monitored by using the rabbit anti-GAPDH antibody (Sigma). Secondary antibody was horseradish peroxidase-conjugated goat anti-rabbit or anti-rat IgG (Amersham). Antigens were revealed with enhanced chemiluminescence (ECL)-plus (Amersham) on autoradiography films (Fuji). Relative quantification was made using Image J® software.

Plasma cytokine measurement

Frozen plasma from WT, Rag2 -/-and Rag2 -/-Il2rg -/-mice were collected at day 9 following anti-GBM-GN induction. γC-associated cytokine plasma levels were measured using a ProcartaPlex Mix&Match Mouse 6-Plex (Ebioscience SAS, Affimetrix) according to the manufacturer's protocol.

Human tissues

Frozen renal biopsy specimens were obtained from the Hôpital Tenon, Assistance Publique-Hôpitaux de Paris, Paris, France, according to local procedures. Only renal biopsy specimens with a sufficient amount of tissue remaining after completion of diagnostic workup were included for immunohistochemical evaluation.

High throughput RNA-Sequencing, demonstration of absence of specific kidney splice variants

RNA samples from non-nephritic kidneys from Rag2 -/-(and Rag2 -/-Il2rg -/-kidney cortex as a negative control) (n=2 per group) of high quality (RNA Integrity Number >6) were processed to create a mRNA library using TrueSeq reagents (Illumina). Clusters were created for Illumina RNA-Sequencing using a CBot system and sequencing was carried out on one lane of HiSeq2000 to produce paired-end reads (75 bases length, 29 to 35 million of reads per sample). After demultiplexing, bar-coding removal, quality control and paired-end reads were processed from Fastq Sanger format to compact-reads format and then aligned with STAR (default parameters) to the mouse reference genome (mGC37/mm9) in order to seek putative kidney specific splice variants. Data alignment and analysis were performed using the GobyWeb interface and examined visually at the Il2rg locus in the Integrative Genomic Viewer (http://www.broadinstitute.org/igv/) respective of Ensembl Annotation (Ensembl version 55). Values are means ± SEM from 6 mice in each condition tested. In D to F no significant differences were observed between groups. Bone marrow (BM) cells from 10-week-old wild type (WT) or Rag2 -/-mice (WT BM or Rag2 -/- BM, respectively) were transferred into sub-lethally irradiated 10-week-old WT (WT BM => WT, Rag2 -/-BM => WT) or into Rag2 -/-Il2rg -/-mice (WT BM => Rag2 -/-Il2rg -/-, Rag2 -/-BM => Rag2 -/-Il2rg -/-mice). Il2rg were not shared by other proxy genes, such as Gm20489-001 (red on the left in A), excluding any kidney-specific variants with an Il2rg similarity. Other processed transcripts, such as Il2rg-002 or 003 (blue in A), which do not have a computationally predicted protein coding expression, were not expressed. Conversely, classical transcripts (in red in A, on the right) were expressed at the protein coding level in the kidney of a naïve Rag2 -/-mouse. Rag2 -/-Il2rg -/-+NK cells

Supplemental

+NK cells

Rag2 -/-Il2rg -/- Rag2 -/-Il2rg -/- Rag2 -/-Il2rg -/- Urea (mmol/L)

Figure 1 .

 1 Figure 1. Mice lacking lymphocytes develop typical lesions of glomerulonephritis following

Figure 2 .

 2 Figure 2. Natural killer (NK) cells are not sufficient to protect Rag2 -/-Il2rg -/-mice from anti-

Figure 3 :

 3 Figure 3: Tissue common gamma chain (gC) is required to maintain kidney function and

  (A to C) Plasma urea levels (the horizontal bar in the graph represents the mean plasma urea level in C57BL/6J mice at basal state) (A), percentage of crescentic glomeruli (C) and semi-quantitative score of tubulo-interstitial lesions (B) in chimeric mice (n=5 per group) 9 days after the administration of anti-GBM serum. (D) Proteinuria to creatininuria ratio in chimeric mice (n=5 per group) 4 days after the administration of anti-GBM serum. Values are expressed as means ± SEM (n=5 per group) and * p < 0.05. (E) Representative images of Masson trichrome stained sections of renal cortex from 16-week-old WT or Rag2 -/-Il2rg -/-mice grafted with WT BM, 9 days after the last injection of anti-GBM serum. Scale bar: 50 µm.

Figure 4 :

 4 Figure 4: IL-15 is highly expressed in kidney cortex in a lymphocyte-free environment and

Figure 5 :

 5 Figure 5: Expression of IL-15R subunits in the mouse kidney during glomerulonephritis.

Figure 1 :Figure 2 :

 12 Figure 1: Luque et al.

Figure 3 :Figure 4 :Figure 6 :

 346 Figure 3: Luque et al.

Figure 7 :

 7 Figure 7: Luque et al.

Table 1 : Basal phenotype of the different immunodeficient mice strains used

 1 Data represents means ± SEM of at least 5 mice per group. Kruskal-Wallis non-parametric test. * p<0.05, ** p<001.

	Supplemental

Table 2 : Polymerase chain reaction (PCR) primer sequences used for quantitative PCR.

 2

Acknowledgments

This work was supported by Inserm and also in part by the Faculté de Médecine Pierre et Marie Curie (Sorbonne Universités, UPMC Univ Paris 06). Y.L. is a recipient of Fonds d'Etudes et de Recherche du Corps Médical (Assistance Publique des Hôpitaux de Paris), Société Française de Bienfaisance et d'Enseignement de San Sebastian-Donostia, Espagne and Société de Néphrologie grants. D.C. is a recipient of a European Renal Association -European Dialysis and Transplant Association (ERA -EDTA) grant and the support from Agence Nationale de la Recherche (ANR) of France. We thank Edith Bouget, Chloe Cakin and Miftah Madi for their technical assistance, Benedita Rocha and Sophie Ezine (Inserm U1020, Hôpital Necker, Paris, France) for providing

Statistics

Values are given as means ± SEM. Statistical analysis was performed using the GraphPad Prism program. The Mann-Whitney test was used to compare two groups. Kruskal-Wallis test analyzed the distribution of three or more groups. A p value<0.05 was considered significant.

Study approval

All mouse studies were submitted to, and approved by, the French Institutional Animal Care and Use Committee (Hôpital Tenon, AP-HP, Paris, France) and the Ethics Committee Charles Darwin (Paris, France) (ref n° 5 ECCD).

Disclosures

The authors have declared that no conflict of interest exists.

Representative images of the expression of IL-15Rα (A), IL-2Rβ (B) or common gamma chain (γC) (C) by immunohistochemistry in 12-week-old Rag2 -/-or Rag2 -/--glomerular basement membrane (GBM) mice at day 9. Images are representative of at least 5 mice. Scale bar: 50 µm.

IL-15Rα has been expressed in some in the tubules during normal conditions, but was largely induced in glomeruli following anti-GBM serum injection. IL-2Rß expression was not detected under normal conditions but was observed at day 9 of glomerulonephritis, mainly in glomeruli.

The gC chain expression has been found in glomeruli after anti-glomerular basement membrane glomerulonephritis (GBM-GN). (D) Immunofluorescent staining of nephrin (in red) and gC chain (in green) on kidney sections from Rag2 -/-or Rag2 -/-Il2rg -/-mice at day 9 of the anti-GBM-GN. DAPI was used to stain nuclei. The gC has been found expressed mainly by podocytes. Rag2 -/- Il2rg -/-mice are positive for nephrin (red) but negative for gC. Representative images from confocal microscopy observations. Scale bar: 50 µm (A) Western blot analysis of phosphorylated JAK1/3 and spleen tyrosine kinase (SYK) proteins from primary cultured podocytes extracted from kidney of 12-week-old wild type (WT), Rag2 -/- Il2rg -/-, Rag2 -/-Il2rb -/-mice. GAPDH was chosen as loading control. (B) Glomerular extracts used for primary podocyte cultures analyzed in light microscopy at 4x magnification (left) and 10x magnification (right). (C, D and E) Ratio of phosphorylated JAK1/3 and SYK on GAPDH is reported with respect to basal conditions using densitometric values from at least 5 mice from each group tested. Values are means ± SEM* p< 0.05 between groups. Bone marrow (BM) cells from wild type (WT) or Rag2 -/-mice (WT BM or Rag2 -/-BM, respectively) were transferred into either sub-lethally irradiated WT (WT BM => WT, Rag2 -/- BM => WT) or Rag2 -/-Il2rg -/-mice ( WT BM => Rag2 -/-Il2rg -/-, Rag2 -/-BM => Rag2 -/-Il2rg -/- mice). Il2rg transcripts according to the Ensembl database (v55) at the Il2rg gene locus (GRCm37/mm9). (B and C) Paired-end reads of naïve Rag2 -/-(B) and Rag2 -/-Il2rg -/-(C) mouse kidneys. All "classical" Il2rg exons were represented in the kidney of a Rag2 -/-mouse, while exons 2 to 7 were absent in the Rag2 -/-Il2rg -/-mouse kidney. Despite numerous transcript variants described in the Ensembl database for Il2rg (represented in blue or red in A), exons belonging to Supplemental Table 1


Eric Lichtfouse
email: eric.lichtfouse@dijon.inra.fr

Gérard Bardoux

André A Mariotti

Jérôme Balesdent

Donna C Ballentine

Stephen A Macko

Éric Lichtfouse

Molecular, 13C, and 14C evidence for the allochthonous and ancient origin of C16-C18 n-alkanes in modern soils

published or not. The documents may come   L'archive ouverte pluridisciplinaire

INTRODUCTION

n-Alkanes are widely occurring in geological samples such as soils, sediments, coals and petroleums. Like most molecular fossils, the precice origin of these substances, their role within the biogeochemical carbon cycle and the nature of processes involved in their transformation are still under speculation [START_REF] Kissin | Catagenesis and composition of petroleum: origin of n-alkanes and isoalkanes in petroleum crudes[END_REF][START_REF] Grimalt | Sources and occurrence of C 12 -C 22 n-alkanes distributions with even carbon-number preference in sedimentary environments[END_REF][START_REF] Tegelaar | Possible origin of n-alkanes in high-wax crude oils[END_REF][START_REF] Sicre | Airborne and vapor phase hydrocarbons over the Mediterranean Sea[END_REF][START_REF] Dinel | Soil lipids: origin, nature, content, decomposition, and effect on soil physical properties[END_REF][START_REF] Lichtfouse | Tracing biogenic links of natural organic substances at the molecular level with stable carbon isotopes: n-alkanes and nalkanoic acids from sediments[END_REF][START_REF] De | A review of macromolecular organic compounds that comprise living organisms and their role in kerogen, coal, and petroleum formation[END_REF][START_REF] Freeman | Predictive isotopic biogeochemistry: hydrocarbons from anoxic marine basins[END_REF], COLLISTER et al., 1994[START_REF] Spooner | Stable carbon isotopic correlation of individual biolipids in aquatic organisms and a lake bottom sediment[END_REF], AMBLÈS et al., 1994). In recent studies we have shown that plants of distinct 13 C/ 12 C compositions, the so-called C 3 and C 4 plants, can be used as natural tracers in field experiments to study the origin and the dynamics of organic carbon from soil substances [START_REF] Balesdent | Natural 13 C abundance as a tracer for studies of soil organic matter dynamics[END_REF], LICHTFOUSE et al., 1994a[START_REF] Lichtfouse | 13 C analysis of molecular organic substances, a novel breakthrough in analytical sciences[END_REF], LICHTFOUSE, 1995[START_REF] Lichtfouse | Heterogeneous turnover of molecular organic substances from crop soils as revealed by 13 C labelling at natural abundance with Zea mays[END_REF]. Indeed, those studies were based on isotopic variations observed after a change from C 3 to C 4 vegetation. Here, we show that the absence of isotopic variation of individual n-alkanes in such experiments can also provide some information on their source.

EXPERIMENTAL Soil analysis

Zea mays was grown in two experimental fields at one crop per year on soils which have previously been cultivated only with C 3 plants. Soils were cropped during 5 years at La Minière and 23 years at Boigneville, France. Geological information on these soils is available elsewhere (BALABANE andBALESDENT, 1992, BALESDENT et al., 1990). Vertical cores through the 0-35 cm ploughed horizon were taken at nine random locations, well-mixed, dried at 20°C then sieved to 2 mm.

A similar fractionation procedure is described by LICHTFOUSE et al. (1994b). Drastic precautions were taken to prevent contamination. Typically, 100 g of sieved dried soil (total organic carbon ~ 1 %) were finely ground and extracted three times by sonication at 30°C with CHCl 3 -MeOH (3:1 v/v) to give about 18.0 mg of extract. The extract was fractionated into neutral (~9.0 mg) and acid (~3.6 mg) fractions by column chromatography on KOH impregnated silica gel [START_REF] Mccarthy R | A rapid qunatitative method for the separation of free fatty acids from other lipids[END_REF]. The neutral fraction was then fractionated by thin layer chromatography (TLC) on fluorescein impregnated silica gel (0.25 or 0.5 mm thickness, CH 2 Cl 2 ), with 1,2:3,4-dibenzanthracene, cholesterol and friedelin as side-references, to give a hydrocarbon-ester fraction (~3.2 mg, R F > ~0.85), a ketone fraction (~2.0 mg, R F ~ 0.28-0.85), an alcohol fraction (~2.0 mg, R F ~ 0.06-0.28) and a polar fraction (~1.8 mg, R F < ~0.06). The hydrocarbon-ester fraction was then fractionated by TLC on fluorescein impregnated silica gel (0.25 mm thickness, n-hexane), with n-octacosane, 1-phenyldodecane, 2-methylphenanthrene and 1,2:3,4-dibenzanthracene as sidereferences, to give an alkane fraction (~1.5 mg, R F > ~0.87), an aromatic fraction (~ 0.9 mg, R F ~ 0.15-0.87), and an ester fraction (~0.8 mg, R F < ~0.15).

Gas chromatography (GC) analyses of n-alkanes from alkane fractions were performed using a Hewlett Packard GC 5990 fitted with an on-column injector and, either a flame ionization detector (FID) or a HP 5989A quadrupole mass spectrometer. Relative concentrations were obtained by integration of FID peak areas. Absolute concentrations were calculated by comparison with the area of an internal standard, 5α-androstane. Isotopic analyses were carried out under continuous helium flow using an HP 5990 GC coupled to a CuO furnace (850°C), and a cryogenic trap (-100°C) coupled to a VG Optima mass spectrometer, monitoring continuously ion currents at m/z = 44, 45 and 46. Isotopic compositions are expressed in per mil. relative to the Pee Dee Belemnite standard: δ 13 C = [( 13 C/ 12 C sample -13 C/ 12 C std)/( 13 C/ 12 C std)] x 10 3 , where 13 C/ 12 C std = 0.0112372. The δ 13 C values of maize wax n-alkanes are -19.0‰ (C 25 ), -19.1‰ (C 27 ), -18.4‰ (C 29 ), -20.6‰ (C 31 ) and -22.2‰ (C 33 ) (LICHTFOUSE et al., 1994c).

An alkane fraction from a soil cultivated 4 years at La Minière field was 14 C-dated as described elsewhere (EGLINTON et al., 1996, LICHTFOUSE andEGLINTON, 1995). Accelerator Mass Spectrometry dating indicated a radiocarbon age of 8510 ± 120 yrs before the present (BP) corresponding to a fraction Modern (fM) of ~ 0.35.

Diesel analysis

A sample of diesel from an automobile as well as a sample of exhaust from the same vehicle were obtained. Particulate exhaust was collected on an ashed (550°C, 1 hour) glass fiber filter using a low volume (60-100 L/min) air sampler. The filter was immediately immersed in dichloromethane and stored in a sealed vial at 4°C until analysis.

Approximately 1 mL of diesel fuel was charged to an ash glass column fitted with a teflon stopcock and containing 2.5 g silica (J. T. Baker, 40-140 mesh) and 1.6 g alumina (Fisher Scientific, in a hexane slurry. Both adsorbents were activated at 200°C for 4 hours and stored under desication prior to use [START_REF] Durand | Analyse géochimique de la matière organique extraite des roches sédimentaires. III. Accroissement de la rapidité du protocole opératoire par l'amélioration de l'appareillage[END_REF]. The alkanes were isolated from the aromatic components by elution from the column using 18 mL hexane and were concentrated under vacuum desiccation prior to instrumental analysis. The exhaust aerosol filter was extracted in 150 mL dichloromethane for 16 hours using a Soxhlet procedure. The dichloromethane in which the aerosol filter was immersed was filtered gravimetrically to remove soot particles and was combined with the aerosol filter extract. The combined extract was concentrated by rotoevaporation at < 30°C and the alkanes were isolated from the aromatic fraction using the same procedure as described above.

(years) (‰) (‰) (‰) (‰) (‰) (‰) (‰) (‰) ______________________________________________________________________ La

RESULTS AND DISCUSSION 13 C Labelling of Soil Organic Molecules at Natural Abundance

A labelling experiment with carbon 13 at natural abundance has been designed to follow the fate of plant carbon into soil organic matter. Thus, maize, a C 4 plant, has been cultivated at two sites on soils which were previously cultivated with isotopically distinct C 3 plants. Maize has been cultivated at one crop per year over a 5-year period at the La Minière site and over a 23-year period at the Boigneville site. The carbon isotopic composition of n-alkanes from those soils are reported in Table 1.

The isotopic compositions of C 25 -C 33 n-alkanes from soils increase with increasing time of maize cultivation (Figure 1). It has been shown that this increase resulted from the input of 13 C-enriched n-alkanes from maize waxes into soil organic matter (LICHTFOUSE et al., 1994a, LICHTFOUSE, 1995[START_REF] Lichtfouse | Heterogeneous turnover of molecular organic substances from crop soils as revealed by 13 C labelling at natural abundance with Zea mays[END_REF]. It is even possible to calculate the contribution of maize n-alkane to each soil n-alkane at any time of cultivation, using the following equation:

M = 100.(δ -δ 0 )/(δ m -δ 0 )
where M is the carbon percentage (or mass percentage) of maize n-alkane within soil n-alkane, δ is the isotopic composition of soil n-alkane at a given time of cultivation, δ 0 is the isotopic composition of soil n-alkane before maize cultivation and δ m is the isotopic composition of leaf wax n-alkane. For instance, after 23 years of maize cultivation, M reaches 50.3 % for the C 31 n-alkane. These experiments under natural conditions can therefore give access to the dynamics of individual organic substances.

Isotopic Compositions of Soil C 16 -C 18 n-alkanes

In contrast, C 16 , C 17 and C 18 soil n-alkanes show, within the experimental error, constant δ 13 C values over the 5-year experiment at La Minière, averaging at -29.4‰, -30.0‰ and -29.2‰ respectively (Table 1, Figure 1). Moreover, even after 23 years of maize cultivation at the Boigneville field, no isotopic increase is apparent for C 16 -C 18 n-alkanes (~ -28.4‰). This absence of isotopic variation on these long-term experiments demonstrates that these n-alkanes must derive from an external source, as explained below. The absence of isotopic variation shows that C 16 -C 18 n-alkanes are not derived from soil biomass. Indeed, if those substances were to derive from certain soil organisms, they should have integrated at least a small portion of heavy maize-derived carbon. This is especially true after 23 years of cultivation, where the maize carbon content of C 31 n-alkane is 50.3%. Moreover, because short-chain n-alkanes are more biodegradable, more volatile and more susceptible to weathering and water-washing than homologues [START_REF] Tissot | Demethylated hopanes in crude oils and their applications in petroleum geochemistry[END_REF][START_REF] Bossert | The fate of petroleum in soil ecosystems[END_REF][START_REF] Peters | The Biomarker Guide[END_REF], one would expect a higher turnover and thus a higher maizederived carbon content for C 16 -C 18 soil n-alkanes. This rapid degradation of shortchain n-alkanes is well supported by an incubation of C 20 n-alkane into a rendzina soil, which show that 25% of this component is mineralized only after 8 weeks (AMBLÈS et al., 1994). On the contrary, the lack of isotopic variation of C 16 -C 18 soil n-alkanes in our experiments indicates that their carbon does not derive from the soil biomass. These results are also supported by incubation of soils from Grignon, France, with 13 Clabelled glucose, showing that all n-alkanes in the C 16 -C 35 range do not incorporate heavy carbon from glucose (LICHTFOUSE et al., 1995).

The distribution of C 15 -C 18 soil n-alkanes do not show any strong predominance of odd (or even) carbon-number homologues (Figure 2). This feature is typical of hydrocarbons occurring in mature, ancient sedimentary organic matter such as fossil fuels [START_REF] Tissot | Origin and evolution of hydrocarbons in Early Toarcian Shales, Paris Basin, France[END_REF][START_REF] Tissot | Demethylated hopanes in crude oils and their applications in petroleum geochemistry[END_REF]. To check this possible contribution of ancient carbon, a soil alkane fraction was analysed for 14 C. AMS dating indicated a radiocarbon age of 8510 ± 120 yrs before the present (BP) corresponding to a fraction Modern (fM) of ~ 0.35. If we assume that this fraction is a mixture of modern C and fossil fuel C (no 14 C), then the fraction Modern indicates that there is 35 % of modern C and 65% of fossil-fuel C. In fact, 65% represents a minimum value for fossil-fuel carbon because incoporation of some bomb-derived 14 C into the biomass gives rise to overestimation of fM values (LICHTFOUSE and EGLINTON, 1995). This additional isotopic evidence therefore demonstrates unambigously the occurrence of ancient carbon in the soil alkane fraction.

Diesel n-alkanes

The n-alkanes from diesel fuel and diesel automobile exhaust range from C 12 to C 24 with no even or odd predominance (Figure 3 and4). The n-alkanes from the fuel showed an abundance maximum at C 15 , whereas those from the exhaust showed a maximum at C 20 , suggesting a preferential loss during combustion and sampling. 13 C values of diesel n-alkanes range from -29.1‰ to -26.5‰ (Table 2, Figure 5). Pristane and phytane show lighter values averaging at -29.8‰ and -31.0‰, respectively. 13 C values. These findings suggest that fuel products, like those emitted by vehicles, are a likely source of soil n-alkanes through aeolian transport then deposition onto the soil surface. Removal of pollutants and particulate matter from the atmosphere by adsorption on waxy surfaces of plants (SMITH, 1977, SIMONICH and[START_REF] Simonich | Importance of vegetation in removing polycyclic aromatic hydrocarbons from the atmosphere[END_REF]) could also play a major role because cropping practices induce a high flux of plant debris into the soil. Crop fields could therefore act as a major sink of pollutants carried by aerosols.

Origin of Soil C 16 -C 18 n-Alkanes

Several external sources can be considered. First, C 16 -C 18 n-alkanes may be derived from the pre-existing sedimentary rock. However, they should have been biodegraded, or water-washed rapidly after the soil formation, especially after the beginning of tillage where water-washing and biodegradation are strongly enhanced by cropping practices. This view is strengthened by consideration of n-alkane absolute concentrations (Table 3). Indeed, at La Minière, absolute concentrations of C 16 , C 17 and C 18 n-alkanes average at 33, 27 and 49 ng per g of soil, respectively, and do not show any systematic decrease nor increase with increasing time of cultivation. They must therefore represent a rather regular input in order to sustain their absolute concentrations in soils. Noteworthy, the absolute concentrations of C 16 -C 18 soil n-alkanes (27-49 ng/g) is in the same concentration level as C 25 -C 33 n-alkanes (25-232 ng/g). This suggests that, given their higher degradability, the amount of C 16 -C 18 soil n-alkanes introduced into the soil over a fixed time period should be even higher than the amount of C 25 -C 33 soil nalkanes derived from maize waxes. Secondly, C 16 -C 18 soil n-alkanes could be derived from algae growing at the soil surface, because atmospheric CO 2 represents a carbon source of relatively constant isotopic composition. There are indeed several reports of C 15 -C 23 n-alkanes and n-alkenes in microalgae [START_REF] Gelpi | Olefins of high molecular weight in two microscopic algae[END_REF], 1970[START_REF] Weete | Algal and fungal waxes. In Chemistry and Biochemistry of Natural Waxes[END_REF]. However, those algal substances show a strong predominance of odd-carbon number homologues, e.g. at C 17 , which should be retained, at least partly, when algal constituents are transfered to soil organic matter. Such a predominance is not apparent in the C 16 -C 18 n-alkanes distributions (Figure 2, Table 3). Moreover, soil algae fix both atmospheric CO 2 and CO 2 produced by the degradation of soil organic matter. This soil CO 2 must show a significant isotopic increase with increasing time of cultivation due to the input of maize matter. Therefore algal constituents should also show an isotopic shift toward heavier values with increasing time of maize cultivation. A contribution of C 16 -C 18 soil n-alkanes can thus be reasonably excluded here.

The most probable origin of C 16 -C 18 soil n-alkanes is an input from fossil fuels, either by aeolian input, by cropping practices, e.g. fumes from exhausts, or by continuous hydrocarbon seepage from the subsurface. Indeed, their weak even carbon number predominance is usually encountered in petroleums and sediments of medium to high maturity, whereas soil n-alkanes in the C 25 -C 35 range show a strong predominance of odd carbon number homologues, indicative of an immature contribution (Figure 2, TISSOT and WELTE, 1984). Moreover, the isotopic values of C 16 -C 18 soil n-alkanes range from -28.1 to -31.6 (Table 1), which is in good agreement, 10 with only few exceptions, with isotopic ranges reported in various mature sedimentary rocks and petroleums [START_REF] Clayton | Effect of maturity on 13 C/ 12 C ratios of individual compounds in North Sea oils[END_REF][START_REF] Wilhems | A comparative study of the stable carbon isotopic composition of crude oil alkanes and associated crude oil asphaltene pyrolysate alkanes[END_REF], BJORØY et al., 1994, LICHTFOUSE et al., 1994b), and in diesel fuel products (this study). On the other hand, the two crop fields studied are both lying in the Paris basin, which is well known for the occurrence of mature Liassic organic matter and petroleums in deep formations [START_REF] Tissot | Origin and evolution of hydrocarbons in Early Toarcian Shales, Paris Basin, France[END_REF], LICHTFOUSE et al., 1994b). Continuous hydrocarbon seepage from the subsurface may thus alternatively account for the persistence of shortchain alkanes in soils at low but detectable levels.

CONCLUSION

Molecular, 13 C and 14 C information has been used to decipher the origin of nonpoint source pollutants occurring in soil. These findings show that short-chain n-alkanes from crop soils represent a regular input of fossil fuel hydrocarbons either from aerosols, e.g. vehicle exhausts, or from deep sedimentary rocks. Furtehrmore, the absence of isotopic variation during long term labelling experiments can be used to assess the indigenous origin of individual organic substances. This principle may apply to any media that has recorded a well-marked isotopic change : sediments, soils, tree rings, peat, ice, shells, etc.

Figure 1 .

 1 Figure 1. δ 13 C isotopic compositions of n-alkanes from soils cultivated maize. Note the absence of significant variations of C 16 -C 18 n-alkanes δ 13 C values over the 5-year period.

Figure 2 .

 2 Figure 2. Typical distribution of n-alkanes from alkane fractions of soil organic (this chromatogram from the La Minière field experiment after 4 years of maize cultivation). IS: Internal Standard, 5α-androstane. Carbon number of n-alkanes are drawn on top of peaks. Conditions: gas chromatography fitted with a flame ionization detector; fused-silica capillary column HP Ultra 1 (25 m x 0.20 mm); 100% methylsilicone phase (0.11 µm); helium carrier gas (1 ml/min). Temperature: 50°C (2 min), 50-320°C at 3°/min, 320°C (60 min).

Figure 3 .

 3 Figure 3. Gas chromatogram of alkanes extracted from the diesel fuel. Pr : pristane, Ph : phytane.

Figure 4 .

 4 Figure 4. Gas chromatogram of alkanes extracted from the diesel exhaust. Pr : pristane, Ph : phytane.

Figure 5 .

 5 Figure 5. Carbon isotopic data for alkanes extracted from diesel fuel and exhaust. Pr : pristane, Ph : phytane.

Table 1 .

 1 Carbon isotopic composition of soil n-alkanes

	______________________________________________________________________
	TIME	C 16	C 17	C 18	C 25	C 27	C 29	C 31	C 33

  The alkanes were characterized with a Hewlett Packard (HP) 5890 series II gas chromatograph equipped with an HP-1 (12m x 0.2mm i.d.) column which was interfaced to a HP 5971 mass selective detector (GC/MS). The GC temperature program was as follows : hold at 70°C for 2 min, increase at a rate of 6°C/min to 280°C, hold at 280°C for 20 min. Compound specific carbon isotope data were obtained using a VG Prism series II isotope ratio mass spectrometer (IRMS) and a Hewlett Packard 5890 series II GC equipped with the same column as above and coupled through a combustrion furnace and water trap to the VG Prism ratio mass spectrometer (GC/C/IRMS).

	Minière field experiment
	0.00	-29.0	-30.2	-29.2	-34.6	-34.2	-35.7	-35.7	-35.7
		(0.8)	(0.8)	(0.6)	(0.3)	(0.3)	(0.3)	(0.3)	(0.3)
	0.42	-29.6	-29.8	-29.7	-32.3	-32.5	-35.3	-35.2	n.d.
		(0.7)	(0.7)	(0.8)	(0.3)	(0.3)	(0.5)	(0.5)
	1.00	-28.9	-29.6	-28.4	-31.6	-32.7	-35.5	-35.8	-35.7
		(0.5)	(0.7)	(0.7)	(0.3)	(0.3)	(0.3)	(0.3)	(0.3)
	1.42	-29.5	-29.4	-28.7	-31.7	-32.1	-35.4	-35.6	-35.7
		(0.6)	(0.8)	(0.6)	(0.3)	(0.3)	(0.3)	(0.3)	(0.3)
	1.75	-29.5	-30.1	-29.0	-31.6	-32.3	-35.5	-35.5	-35.4
		(0.5)	(0.5)	(0.5)	(0.3)	(0.3)	(0.3)	(0.3)	(0.3)
	2.42	-29.7	-30.3	-29.5	-30.6	-31.1	-35.1	-35.0	-35.2
		(0.5)	(0.5)	(0.8)	(0.5)	(0.3)	(0.3)	(0.3)	(0.3)
	3.42	-29.5	-30.4	-29.1	-29.9	-29.5	-34.7	-34.0	-33.9
		(0.5)	(0.5)	(0.6)	(0.3)	(0.3)	(0.3)	(0.4)	(0.4)
	4.00	-29.4	-30.3	-29.6	n.d.	-29.7	-34.5	-34.4	-33.6
		(0.8)	(0.5)	(0.6)		(0.3)	(0.3)	(0.3)	(0.5)
	4.33	-29.3	-30.0	-29.5	-30.2	-29.5	-34.3	-33.7	-33.4
		(0.5)	(0.6)	(0.6)	(0.3)	(0.3)	(0.3)	(0.3)	(0.3)
	Average † -29.4	-30.0	-29.2
		(0.3)	(0.3)	(0.4)
	______________________________________________________________________
	Boigneville field experiment
	Wheat *	-29.7	-31.6	-29.8	n.d.	-33.0	-35.7	-35.7	-33.7
		(0.6)	(0.7)	(0.6)		(0.3)	(0.3)	(0.3)	(0.3)
	20.0	-29.7	-31.5	-29.8	n.d.	-24.1	-28.8	-28.7	-26.7
		(0.8)	(0.8)	(0.8)		(0.3)	(0.3)	(0.3)	(0.3)
	23.0	-28.3	-28.9	-28.1	n.d.	-24.0	-28.3	-28.1	-25.6
		(0.5)	(0.5)	(0.5)		(0.3)	(0.3)	(0.3)	(0.3)
	______________________________________________________________________

Note: δ 13 C values of soil n-alkanes versus time of maize cultivation. * Soil cultivated 23 years with wheat. The average deviation of 3-4 replicates are given in parenthesis. † Calculated mean deviation in parenthesis. n. d. = not determined.

Table 2 .

 2 Carbon isotopic composition (‰) of alkanes isolated from diesel fuel and automobile exhaust.

	___________________________________
	Alkane	Fuel	Exhaust
	___________________________________
	C 12	-28.3	n.d.
	C 13	-27.9	n.d.
	C 14	-27.0	n.d.
	C 15	-26.9	n.d.
	C 16	-27.2	-27.4
	C 17	-26.5	-28.0
	C 18	-27.0	-29.1
	C 19	-28.3	-26.8
	C 20	-28.5	-27.4
	C 21	-28.9	-27.6
	C 22	-27.7	-28.0
	C 23	-27.6	-27.5
	C 24	n.d.	-27.2
	Pristane	-29.6	-30.1
	Phytane	-30.7	-31.3
	___________________________________
	Short-chain soil n-alkanes and n-alkanes from diesel products show close
	distributions and

Table 3 .

 3 Absolute concentration of soil n-alkanes

	______________________________________________________________________
	TIME	C 16	C 17	C 18	C 25	C 27	C 29	C 31	C 33
	(years)			(ng/g dried soil)
	______________________________________________________________________
	La Minière field experiment
	0.00	35	40	39	21	50	155	214	59
	0.42	28	25	n.d.	24	66	198	284	77
	1.00	17	16	26	18	36	108	153	42
	1.42	44	29	96	22	50	139	198	53
	1.75	19	18	34	19	41	121	174	50
	2.42	62	39	68	31	72	191	280	79
	3.42	24	21	40	29	67	173	262	71
	4.00	33	26	41	22	61	150	212	64
	4.33	31	25	49	37	88	213	312	89
	Average † 33	27	49	25	59	161	232	65
		(13)	(8)	(21)	(6)	(15)	(34)	(51)	(14)
	______________________________________________________________________
	Boigneville field experiment
	Wheat *	24	11	39	14	41	140	188	61
	20.0	10	5	6	10	30	61	101	46
	23.0	28	28	53	15	30	58	97	40
	______________________________________________________________________
	Note:

* Soil cultivated 23 years with wheat. n. d. = not determined. † Calculated mean deviation in parenthesis.

Acknowledgements

We thank Dr Timothy Eglinton from Woods Hole Oceanographic Institution for 14 C measurements. Dr Michael Engel is greatly acknowleged for his helpful comments in reviewing the manuscript. We are indebted to the Institut National de la Recherche Agronomique, Secteur Environnement Physique et Agronomie, Département de Science du Sol, for financial support.