Seif Eddine Hammami

Hossam Afifi

Hassine Moungla

Ahmed Kamel

Drone-assisted cellular networks: A Multi-Agent Reinforcement Learning Approach

Drone-cell technology is emerging as a solution to support and backup the cellular network architecture. cell-drones are flexible and provide a more dynamic solution for resource allocation in both scales: spatial and geographic. They allow to increase the bandwidth availability anytime and everywhere according the continuous rate demands. Their fast deployment provide network operators with a reliable solution to face sudden network overload or peak data demands during mass events, without interrupting services and guaranteeing better QoS for users. With these advantages, drone-cell network management is still a complex task. We propose in this paper, a multiagent reinforcement learning approach for dynamic drones-cells management. Our approach is based on an enhanced joint action selection. Results show that our model speed up network learning and provide better network performance.

I. INTRODUCTION

The continuous growth of data demand and the increase in wireless traffic rates in new mobile networks needs intelligent and dynamic technologies for telecommunication management. Recent studies predict that the new generation cellular standards (like 5G) will rely much more heavily on a dense and less power consuming networks to serve dynamically requested data rates to the user [START_REF] Andrews | What will 5g be?[END_REF]. Small cells mounted on unmanned aerial vehicles (UAVs) or drones (we call them drone-cells hereafter) are proposed, as an alternative to fixed femto-cells, to support existing macro-cell infrastructure. The deployment of these mobile small cells consists on move these small cells toward a target positions (usually within the range of the macro-cell to support) based on the decision made by a mobile network operator. Since drone operation costs money, the drone-cells movement toward adequate positions must be correlated with the amount of data requests, i.e. drone-cells should support overloaded cells (Figure 1). Hence, an intelligent entity should be added to the network that instantly monitors the network state and finds the optimal decision to control drone-cells. Several studies have discussed the theoretical ability of cellular networks to use drone-cells to support their existent macro-cells [START_REF] Alsharoa | Energy management in cellular hetnets assisted by solar powered drone small cells[END_REF], [START_REF] Kalantari | On the number and 3d placement of drone base stations in wireless cellular networks[END_REF] but the readiness of those networks to integrate such dynamic entities has not been discussed. For example, drone-cells require coordinated insertion to the network infrastructure while serving subscribers and smooth de-association at the end of their activity (low battery, low data demand ...). This requires capabilities for efficient network configuration and management flexibilities. It needs self-organizing capabilities for the networks. Massive amounts of information about users behavior and network load must be continuously collected and analyzed by intelligent algorithms. This should trigger the usage of UAVs on field. On the other hand, the promising development of artificial intelligence techniques and machine learning tools may speed up the development of cellular networks and leed to develop such dynamic and self-adaptive tools that help the network operators to better manage their infrastructure and integrate new techniques. Machine learning techniques have the advantage of exploiting the plethora of data generated by cellular networks to improve these dynamic deployment techniques [START_REF] Hammami | Network planning tool based on network classification and load prediction[END_REF].

In this work, we propose a dynamic reinforcement learning solution for drone-cells networks that exploits real traces of demand profiles and adapts in real time the deployment of dronecells according these demands. We choose an efficient machine learning paradigm instead of classical linear programming models. Our solution is based on a multi-agent reinforcement learning (MARL) approach which is applied in two steps: an off-line exploration step and an on-line one. The off-line step consists in a learning phase where the drone-cells of the multiagent topology learn, in a cooperative reinforcement learning context, how to react and move according to different scenarios of bandwidth demands. Real Traces of network time-series load are used for this phase. In the on-line phase, the prelearning agents demonstrate how they are adapted instantly on a real scenario of load peak signaled by the STAD framework. The cooperative reinforcement learning is based on a joint action selection that aims to choose the optimal set of agents action. We propose a joint action selection algorithm and we compare it against the classical grid selection algorithm.

The paper is organized as follows. In section 2 we present related work, while in section 3 we present a highlight of multi-agent reinforcement learning approach. Section 4 develops our MARL-based model. Section 5 presents simulations uses-cases and results and we conclude in section 6.

II. RELATED WORK

Smart deployment of drone-cells has been a topic of recent research; examples of such works include [START_REF] Kalantari | Backhaul-aware robust 3d drone placement in 5g+ wireless networks[END_REF], where the authors use binary integer linear programming (BILP) to selectively place drone-cells in networks and integrate them to an existent cellular network infrastructure while temporal increase of user demands occurs. They design the air-toground channel for drone-cells as the combination of two [START_REF] Kalantari | On the number and 3d placement of drone base stations in wireless cellular networks[END_REF] investigate the adequate number and the 3D placement topology for dronecells formation. The study aims to optimize the geographic coordinates of drone-cells and their total number while serving users. In [START_REF] Park | Drone formation algorithm on 3d space for a drone-based network infrastructure[END_REF], authors propose a framework for drone-cells formation. The framework aims to optimize the coverage (by increasing or decreasing the drone altitude) in order to remove dead zones and reach network capacity goals.

In contrary to previous studies that aim to optimize the placement of drone-cells before deployment, [START_REF] Jun | Path planning for unmanned aerial vehicles in uncertain and adversarial environments[END_REF] investigates path optimization issues during drone-cells formation.

Our proposed contribution develops new methods not only in optimal network deployment for drone-cells, but also in how to dynamically manage this deployment in order to enhance the existent infrastructure using a multi-agent reinforcement learning approach.

III. REINFORCEMENT LEARNING

In this section, we introduce the background on "multiagent" reinforcement learning. We also discuss using multiagent approach in our contribution and we present its benefits/advantages against a simple single-agent.

A. Multi-agent reinforcement learning approach

A typical agent has its set of states, actions and rewards. As the wireless topology is composed of multiple drone-cells, and since modeling a single reinforcement learning model for each single drone-cell leads to an exponential growth in the (state, action, rewar d) space, we prose to employ a multiagent reinforcement learning model. It will reduce the space and have a better global federated goal.

The multi-agent reinforcement learning approach [START_REF] Soua | Adaptive data collection protocol using reinforcement learning for vanets[END_REF] , [START_REF] Hammami | Network planning tool based on network classification and load prediction[END_REF], is a generalization of the single model and is modeled by a stochastic game. The multi-agent RL employs a joint action, which is the combination of actions to be executed by each agent at state k.

The stochastic game is represented by the tuple < X, A 1 , ..., A n , f , r 1 , ..., r n > with n is the total number of agents. A i with 1 ≤ i ≤ n is the set of possible actions of agent i.

So we can define the joint action as

A : A 1 × A 2 × ... × A n . r i : X × A × X → R is the reward function of agent i that is assumed to be bounded. f : X × A × X → [0, 1]
is the transition probability function. In the multi-agent approach, the transitions between states are the result of a joint action of all the agents which is expressed as: ja k = [a 1,k , ..., a n,k ] where a i,k ∈ A n . The reward r i,k+1 (r i (x k ) = r i,k+1 ) resulting from executing the action a i,k is computed according to the joint policy j h and the joint expected return is expressed as follows:

R jh i (x) = E ∞ i=1 (γ k r k+1 |x 0 = x, j h) (1)
The Qlearning function depends also on the joint action and the joint policy and it is expressed as follows:

(2)

Q jh k+1 (x k , ja k ) = Q k (x k , ja k ) + α k [r k+1 + γ max ja Q k (x k+1 , ja )Q k (x k , ja k )]
In our model, we are adopting the fully coordinated multiagent RL, and in this case all reward functions are the same for all agents, r 1 = ... = r n = r

B. Motivations of using MARL approach

Along with advantages due to the distributed feature of the multi-agent systems [START_REF] Nour | M2hav: A standardized icn naming scheme for wireless devices in internet of things[END_REF], as the acceleration become possible by parallelizing the computation, MARL approaches emerge with the benefit of sharing experiences between agents. Experience sharing helps agents to learn faster and reach better performances. Hence, the agents can communicate between each other to share experience and learning so that the better learned agents may speed up the learning phase of other agents. Moreover, multi-agent systems facilitate the insertion of new agents into the system which leads to came up with scalability issues affecting classic method such as linear programming.

IV. DRONE-CELLS NETWORK AGENT MODEL

In this section we present our proposed model for dronecells optimization and management and we explain the mapping between our model parameters and the theoretical model presented in the previous section.

A. Model Framework Description

Our drone-assisted network model is based on a centralized multi-agent reinforcement learning approach. The framework consists of a multi-agent system implementation based on MESA package. It is composed of entities that describe the system parameters and manage the interaction between the different agents. The framework is described in figure 2.

Since the approach is centralized, we need a coordinator agent that collects information from different agents and intervenes in the actions selection process. In our framework, this coordinator agent is represented by the network operator agent (which can be also a centralized entity in the network architecture). The framework is composed also of a set other agents. We distinguish two types of agents: Active agents executing the model actions and passive agent that participate in the system interaction but without executing any action. The active agents are the drone-cells that are represented by the drone entity in our framework. The action to be executed by a drone is either to move to a cell location in order to serve it or to go back to the facility location (in case of battery running out or no cell left to serve). This entity defines the battery level BT d,t of a drone d at time-stamp t, the bandwidth offer level BW d,t , the status of drone d (whether is serving, idle or charging) and other parameters.

The passive agents are the cell (represented by Cells entity) and the facility (represented by Facility entity) agents. The cell agent entity defines the bandwidth demand rate at each time-stamp, the geographic location of the cell and the status of the cell (whether is served or not). The facility agent constitutes the initial location of all drones and where drones can be charged. If a drone is not serving a cell must move to the facility location. We suppose that the facility location is situated in the middle of the service area in order to optimize the flight time toward cells.

B. Model States

The model states provide information about the network status at each time-stamp t such as drones parameters, Facility actual capacity or cells demand rates. In our MARL model, each drone agent d has it own state S t which describes the actual parameters at time t. These parameters are the drone battery level BT d,t , the bandwidth offer capacity level BW d,t .

Moreover, each cell agent C state is represented by the aggregated throughput demand T d C,t at time t that exceeds the maximum capacity of the macro base station that covers it, and that should be assisted by drones. The facility agent state is also described by the current number drone that are placed in it and either they are charged or not. All these information are gathered by the network operator agent (The coordinator agent) in order to choose the joint action to take for all drones in order to maximize the total network' QoS.

C. Model Actions

The action that can a drone agent execute, at each state, is either to move and serve a cell and serve it or to move to the facility (to charge its battery or in case of no cell left to serve). Unlike single agent models, the action of each drone agent is taken by the coordinator (centralized approach) to maximize the global network QoS. Hence, it is not always straightforward that the chosen agent action maximize its own reward. Otherwise, the coordinator agent choose a joint action (set of action for all drones) that maximize the global network performance.

D. Reward function

The reward function for each drone agent is measured at each state and after choosing the action to execute. The reward function measures the the local network service ratio (NSR) and it is the fraction of the served throughput by the drone d at time t and the throughput request of the cell to be served by d at the same time. Its expression is as follows:

(3)

r d,t = BandwidthServed d,t BandwidthRequest C,t

E. Coordinated multi-agent RL

Choosing the optimal joint action is a critical step for the multi-agent RL model. The criteria of choosing a better joint action helps to reduce the computational cost of searching the joint actions, which is exponentially proportional to the agent number and to provide a better performance. We present in this section the hill climbing search algorithm (HCS) which is proposed in [START_REF] Proper | Scaling model-based average-reward reinforcement learning for product delivery[END_REF]. Then, we propose an enhancement of the Hill climbing search algorithm for an opltimal joint action selection (eHCS) algorithm that speeds up the action search. The model is then compared to the HCS algorithm.

1) Hill climbing search algorithm: The main objective of hill climbing algorithm is to examine the neighboring agents one by one and to select the first neighboring agent action which optimizes the current reward as next node. The algorithm is resumed by the following steps:

1-Initialization step: Select an initial joint action JA by picking the action of each agent that maximize the reward.

2-Choosing randomly an agent and its action in the previous

JA is changed by a neighboring action. 3-The new formed action is evaluated. If it provides better network service ratio (NSR) so it is stored as optimal JA. 4-Repeat steps 2 and 3 until testing all combinations.

2) Enhanced Joint optimal action selection algorithm : Our multi-agent reinforcement learning model is based on a semi-centralized cooperative solution. The centralized part consists of adding a coordinator agent (Which is the network operator agent here) which is able to collect all drone agents and other model agents information in order to select the "joint optimal action" to be executed by drones. The centralized approach reduces then the complexity of the system, speed up the selection process and alleviate the system information sharing among all agent. Unlike the distributed approach which consists on forwarding all information between all agents, that may be a time consuming. Plenty of centralized joint action selection algorithms exists in the literature such as hill climbing search [START_REF] Proper | Scaling model-based average-reward reinforcement learning for product delivery[END_REF], Stackelberg Q-Learning [START_REF] Cheng | A multi-agent reinforcement learning algorithm based on stackelberg game[END_REF] etc. In our model we propose a modified version of hill climbing search algorithm which fit our drone actions selection problem.

The main objective of this algorithm is described as follows: At first, each drone agent, At a state S t at time-stamp t, choose its own optimal action to execute without taking into consideration other drones configuration. Then, the coordinator agent collected all drone information (Battery level, remaining bandwidth) as well as their chosen actions. It collects also cells and facility agents information. After that, the coordinator sorts decreasingly the set of cell demands (respectively the set of drone offers). Then, it tries, at each iteration, to find a match between the drone actions (the position to move toward) and the demand positions (cell positions) and assign the adequate drone to the cell. Then, if the cell is totally served, the coordinator looks for drone agents whose action is identical to the served cell position and re-select a new action for the drone according a new Q-table. The new Q-table is formed with the remained possible actions (except the selected priorly).

The coordinator repeat the same process over all drones. This selection method avoid assigning many drones to the cells with higher demand and try to share drones all over the network.

The semi-centralized joint action selection algorithm is presents in algorithm 1. Add NewAction(d) to JAction List return JAction List Note that the joint action selection algorithms are used only during the exploration phase (training phase). Once the Qtables are calculated and their entries are the optimal values, they are used for the online exploitation phase.

Algorithm 1 Centralized Joint action selection

V. SIMULATION AND RESULTS

A. Use-case Scenario

We validate our framework by using real datasets of cell demand time-series extracted from the Milan CDRs dataset. We reproduce in this contribution a mass event where macrocells are assisted by drone-cells managed by our framework in order to cover the unusual peaks of cellular user demands. As shown and analyzed aforementioned, we detected some unuasual behavior of demand arround the SanSiro stadium during football match compared to workdays. Figure 3 represents the cells segmentation of the Sansiro stadium and areas around. Figure 4 depicts the daily demands time-series of these cells and we can notice that the SanSiro stadium cells have a high peak of user demand around 8pm (football match time) while supporters and their departure time after the match. We can notice that these cells are covering a parking area and some metro stations which make our assumptions believable. We apply our framework to these demand time-series in order to manage the abnormal increase of users demand during this period of the day and we show results in the next part.

B. Results

The framework is applied on cell demands between 6pm and 11pm, a period of time when the abnormal demand peak are occurring. During this period, drone-cells will attempt to assist the existent macro-cell and serve the additional demand. Table 1 resumes the simulation parameters. As performance metric, we choose to use the network service ratio (NSR) which is defined as follows:

(4) N SR = T otal Bandwidthserved T otalbandwidthdemand
where T otal Bandwidthserved is the sum of the served bandwidth by all drones and T otalbandwidthdemand is the sum of all cell demands. Our MARL model based on the enhanced hill climbing algorithm is compared also to a MARL model with the classic hill algorithm. Figures 5678910shows the simulations and comparison results between both models. Figure 5 We use in this scenario only 8 drones to cover the area for both models. We can notice that after finishing the learning period (after 300 steps) the network service ratio converges to 1. During the learning period, the drones are exploring several options of possible actions and communicating their decisions to the coordinator. At the end of this period, the coordinator selects the optimal joint action for all drones. All cells are fully served. We also notice that 8 drones are sufficient for our eHCS˙based model, to serve the network. Whereas, the concurrent model based on HCS performs worse than our model. We notice that in this scenario, and after the exploration phase, the model fails to converge to 1 and the network service ratio converges to 0.9 and 0.6 at 6pm and 7pm respectively. Hence, the HCS based model performs 10% lower than our model at 6pm and %40 lower at 7pm, when demand is much higher. So, we can say that our model performs much better than the other model during peak hours. Moreover, the exploration period was not sufficient for the HCS based model to achieve the optimality and we conclude that our model is faster with 8 drones for the period between 6pm and 7pm. Figures 6-10 show simulation and comparison results using 14 drones. We notice that for all time periods, the network service ratio converges to 1 after the learning period for our model (the learning is very fast in this situation). Whereas, the HCS based model success to converge to 1 only for three time periods; at 6pm, 7pm and 11pm while its network service ratio is lower than 1 for the rest of periods. Furthermore, for scenarios at 6pm, 7pm and 11pm, even that the HCS based model converges to 1 after the exploration period, it is slower that our model.

Globally, we notice that the HCS model fails to reach an optimal steady state contrary to our model. The exploration period was not sufficient to learn and to find the optimal joint actions within these periods. The other model does not manage battery in a performant way. Most of drone batteries are lost in an unefficient way (Battery life-time is 2 slots and they need one time slot to recharge). That is why the other model reaches optimality at 11pm again (after recharge). Hence, in order to satisfy the global demand during the exploration period with the same period size, the HCS based model needs more drones to serve all demands. MARL learns these constraints and succeeds to efficiently serve the network. With this configuration, our enhanced model only needs 11 drones at total to cover the whole area during the football match event. These 11 drones alternate to serve the requested amount of data according to their battery life level.

Figure 11 depicts the deployment of drone-cells using our enhanced model at 9pm , time-slot corresponding to a heavy demand behavior. The colors represents the demand intensity at each cell. We can notice that 9 drones of 14 are deployed at cell locations that guarantee serving the total required bandwidth. Note that at this time 4 drones are out of charge while 1 drone is not serving at all. Drones-cells management is still complex and needs advanced and intelligent algorithms. Even with their fast deployment, drone-cells networks suffer from coordination issues. On the other hand, battery technology is still limited and since it is split between navigation and antenna beaming/transmission, energy presents a major constraints for drones-cells deployment. We present in this paper a solution based on dronecells to support macro cells of the classic cellular network during mass events when data rate demand explodes. We solve this complex navigation/coverage management by a multi-agent reinforcement learning approach for this dynamic network deployment. We also propose an enhanced joint action selection algorithm to alleviate the coordination complexity between drone-cells agents and also speed up the search phase of the optimal joint action. Our model takes into consideration the battery life constraints while aiming to maximize the network service ratio. Our solution is validated with real network traces and we provide a bench marking analysis. Our model based on the enhanced joint action selection that we propose is compared with a model based on hill climbing search algorithm. Results show that our model outperforms the second model not only when the rate demand is lower but especially at peak time service.

Our model offers hence a better solution for network operators to dynamically manage their network and to provide efficiently a better QoS for users during mass events. A future perspective will introduce the deep aspect that can add classification and memory of early situations to the multi agent system.

Figure 1 :

 1 Figure 1: Graphical illustration of a Drone-assisted network

Figure 2 :

 2 Figure 2: Graphical illustration of the MARL Drone-assisted network Framework and the relationship between the Agents

Figure 3 :Figure 4 :

 34 Figure 3: Graphical illustration of cell segmentation of SanSiro area

  depicts the scenario simulation at 6pm (Fig 5a) and 7 pm (Fig 5b), the time of supporters arrival before starting the football match.

Figure 5 :Figure 6 :Figure 7 :

 567 Figure 5: Network service ratio evolution in function of execution steps using 8 drones

Figure 9 :Figure 10 :

 910 Figure 8: QoS, 14 drones, 9pm

Figure 11 :

 11 Figure 11: Illustration of drone-cells deployment at 9pm

1 :

 1 Input: (Action,State) space, Q-tables, ServedCell˙list 2: Output: Joint Action J A t 3: Collect system in f ormation 4: for d in Drones do:

	5:	if Action d,t ∈ ServedCell list then
	6:	E xtractNewQ -table(d)
	7:	ActionSelection(NewQ table(d))
	8:	Verify NewAction(d,t)
	9:	Find MatchAction(Cell)
	10:	if Matched Action then
	11:	Actualize ServedCell list
	12:

Table I :

 I Simulation Parameters and Values

	Parameters	Values
	Drone-cells max battery	100
	Drone-cells max rate	50 Mb/s
	Cell max rate demand at peak hours 120 Mb/s
	Battery life-time factor	0.5
	Number of cells	6
	Learning rate	0.75
	Learning period (exploration)	300
	exploitation period	50


J L Autran

D Munteanu

Atmospheric radiation and COTS at ground level

This tutorial will survey single event effects (SEE) induced by terrestrial cosmic rays on current commercial CMOS technologies. After describing the natural radiation environment at ground and atmospheric levels, the tutorial will describe the physics of SEEs, from the main mechanisms of interaction between atmospheric radiation (neutrons, protons, muons) and circuit materials to the electrical response of transistors, cells and complete circuits. SEE characterization using accelerated and real-time tests will be examined, as well as modeling and numerical simulation issues. Special emphasis will finally concern the radiation response of advanced technologies, including deca-nanometer bulk, FD-SOI and FinFET families.

Microelectronics industry has experienced tremendous progress in the last forty years, especially with regard to the evolution of the products (i.e. integrated circuits) performances, and at the same time, concerning the drastic reduction of manufacturing costs per elementary integrated function. So far, this considerable growth of the semiconductor industry has been due to its technological capability to constantly miniaturize the elementary components of circuits, namely the MOSFET (metal-oxidesemiconductor field effect transistor), the basic building block of VLSI (very large scale integration) integrated circuits. However, the conventional bulk MOSFET scaling encountered this last decade serious physical and technological limitations, mainly related to the gate oxide (SiO 2 ) leakage currents [START_REF] Gusev | Advanced high-k dielectric stacks with polySi and metal gates: Recent progress and current challenges[END_REF], the large increase of parasitic short channel effects and the dramatic mobility reduction [START_REF] Fischetti | Long-Range Coulomb Interactions in Small Si Devices. Part I: Performance and Reliability[END_REF] due to highly doped silicon substrates necessarily used to reduce these short channel effects. Technological solutions have been proposed in order to continue MOSFET miniaturization, such as: (i) the introduction of highpermittivity gate dielectric stacks [START_REF] Houssa | Fundamental and Technological Aspects of High-k Gate Dielectrics[END_REF], midgap metal gate and strained silicon channels [START_REF] Rim | Transconductance Enhancement in Deep Submicron Strained-Si n-MOSFETs[END_REF]; (ii) the replacement of the conventional bulk MOSFET architecture by alternative solutions including the use of new materials (e.g. SOI materials) or new devices architectures (e.g. Multiple-Gate devices [START_REF] Park | Multiple-gate SOI MOSFETs: device design guidelines[END_REF], silicon nanowire MOSFETs) allowing better electrostatic control, and, as a result, intrinsic channels with higher mobilities and currents.

As MOSFET scales have reduced, the sensitivity of the integrated circuits to radiation coming from the natural space or present in the terrestrial environment has been found to seriously evolve [START_REF] Dodd | Basic mechanisms and modeling of single-event upset in digital microelectronics[END_REF][START_REF] Baumann | Radiation-Induced Soft Errors in Advanced Semiconductor Technologies[END_REF][START_REF] Autran | Soft Errors: from particles to circuits[END_REF]. Nowadays, for ultra-scaled devices, natural radiation is inducing one of the highest failure rates of all reliability concerns for devices and circuits in the area of nanoelectronics [START_REF] Baumann | Radiation-Induced Soft Errors in Advanced Semiconductor Technologies[END_REF]. In particular, ultra-scaled memory integrated circuits have been found to be more sensitive to single-event-upset (SEU) and digital devices more affected by digital single-event transient (DSETs). This sensitivity is a direct consequence of the reduction of device dimensions and spacing within memory cells combined with the reduction of supply voltage and node capacitance, resulting in a decrease of both the critical charge (i.e. the minimum amount of charge required to induce the flipping of the logic state) and the sensitive area (i.e. the minimum collection area inside which a given particle can deposit enough charge to induce a change in the cell) [START_REF] Baumann | Radiation-Induced Soft Errors in Advanced Semiconductor Technologies[END_REF], [START_REF] Mitra | Soft Errors: Technology Trends, System Effects and Protection Techniques[END_REF].

This tutorial is structured in four main parts. In part 2, we will briefly describe the natural radiation environment at ground and atmospheric levels. The physics of SEEs will be summarized in part 3, from the main mechanisms of interaction between atmospheric radiation (neutrons, protons, muons) and circuit materials to the electrical response of transistors, cells and complete circuits. Part 4 will explain SEE characterization using accelerated and real-time tests, as well as modeling and numerical simulation issues. Finally, part 5 will present and discuss the radiation response of advanced commercial technologies, including deca-nanometer bulk, FD-SOI and FinFET families.


Nicolas Casaretto

Dominik Schaniel

Paul Alle

Emmanuel Wenger

Pascal Parois

Bertrand Fournier

El-Eulmi Bendeif

Cyril Palin

Sébastien Pillet
email: sebastien.pillet@univ-lorraine.fr

Paul Alle Ab

In-house time-resolved photocrystallography on the millisecond timescale using a gated X-ray hybrid pixel area detector

Keywords: Time-resolved X-ray diffraction, photocrystallography, hybrid pixel detector

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

There is a strong demand in broad areas of today's physics, chemistry, and biology on the precise characterization of the structure of materials and its evolution in time. The structural dynamics of complex systems built from many atoms involve intermediates and transition states on a multidimensional energy landscape. In the solid state, the whole processes may cover 15 orders of magnitude in time, from femtoseconds to seconds. Electronic motions are the fastest processes in the femtosecond and sub-femtosecond regime, atomic motions and molecular vibrations occur in the subpicosecond time scale (Bertoni et al., 2015;Collet et al., 2012;Lorenc et al., 2009;Lorenc et al., 2012), optical and acoustic phonons (lattice vibration) occur in the ps time scale (Lindenberg et al., 2000;Sokolowski-Tinten & von der Linde, 2004), while heat diffusion and thermal transfers in crystalline solids happen in slower time scale (Bertoni et al., 2015;Collet et al., 2012;Lorenc et al., 2009;Lorenc et al., 2012). The μs-ms range is dominated by macroscopic transformations in crystals, such as domain and domain wall dynamics in ferroic crystals (polarization switching in ferroelectrics) (Grigoriev et al., 2006;Issenmann et al., 2012), propagation of macroscopic elastic deformations (ref); this timescale corresponds also to the lifetime of long-lived molecular metastable states. In the past decade, timeresolved laser pump/X-ray probe techniques proved to be important approaches for investigating structural dynamics in gases, solids and liquids down to the femtosecond time scale (Chen, 2004;Kim et al., 2015;Dwayne Miller, 2014;Chergui & Zewail, 2009, Patterson, 2014). The common measurement method consists in aligning jointly a pulsed laser and pulsed X-ray beam on the sample, and synchronizing the two pulses with an adjustable time delay ∆t between laser pump and X-ray probe. Varying the time delay, the corresponding X-ray scattered or absorbed signal provides a direct monitoring of the light-induced structural evolution of the sample as a function of time delay. Dedicated time-resolved X-ray absorption and X-ray diffraction instruments have been developed at synchrotron radiation facilities, where the effective highest time resolution is defined by the synchrotron bunch duration, typically in the 50-100ps range (Chollet et al., 2015;Ejdrup et al., 2009;Ichiyanagi et al., 2009;Wulff et al., 2002;Nozawa et al., 2007). The frequency of the pump-probe cycles is given by the orbit time (2.82 s at ESRF, 1.18 s at SOLEIL, 3.68 s at APS for instance) and the filling mode of the storage ring (uniform filling, single bunch mode, …), and make use of phaselocked mechanical choppers to select individual X-ray pulses (Cammarata et al., 2009;Gembicky et al., 2009). Recently, time resolution in the femtosecond time regime has been achieved using the slicing technique at synchrotron radiation facilities, (Beaud et al., 2007;Laulhé et al., 2015) or X-ray free electron lasers (XFEL) operating in the hard X-ray regime (Patterson, 2014). Efforts have been made also on the laboratory scale, with laser-driven plasma table-top sources offering an attractive alternative to accelerator-based sources (Zamponi et al., 2009;Elsaesser & Woermer, 2014). Monochromatic X-ray pulses are generated by focusing very intense femtosecond optical laser pulses onto a metallic target (Cu or Ti), producing characteristic K hard X-ray pulses of typically 100 fs duration. These plasma sources are however rather complex and laborious to operate.

Two different instruments for measuring the X-ray diffraction pattern of optically excited crystals based on a conventional in-house diffractometer have been reported by the group of P. Coppens. The very first experiment reported in 1994 used a high power continuous laser, a conventional X-ray diffractometer and a gated X-ray scintillation point detector (White et al., 1994). The laser beam was chopped with a mechanical chopper, which consisted of a slotted blade rotating in front of the laser to produce periodic laser pulses of frequency and width adjustable through the blade rotation speed and slot width respectively. The X-ray detector was synchronized with the rotation of the chopper to adjust the pump-probe time delay ∆t. The technique was appropriate for 0.1-10 ms time resolution.

The second technique, reported in 2014, used a pulsed laser, a conventional X-ray diffractometer equipped with an integrating Charge Coupled Device detector (CCD), and an optical chopper located in the X-ray beam path (Kaminski et al., 2014). Contrary to the initial approach, the X-ray beam was chopped to produce X-ray pulses of adjustable frequency and time width. The laser pulses were synchronized with the X-ray pulses to adjust the pump-probe time delay ∆t. The advantage of this setup is the use of an area X-ray detector which covers reciprocal space more efficiently than a point detector.

The last years have seen the revolution in X-ray diffraction introduced by the novel generation of photon counting Hybrid Pixel Area Detectors (HPAD) (XPAD, PILATUS…) (Henrich et al., 2009;Pangaud et al., 2007;Medjoubi et al., 2010;Dinapoli et al., 2011;Johnson et al., 2012;Alle et al., 2016). These detectors can be considered as a matrix of hundred thousands of independent point detectors. Their electronics allow frame rates up to the kHz regime, and an internal time-resolution down to a hundred nanoseconds. This kind of detector may therefore be efficiently used to isolate single X-ray bunches from the pulse train of synchrotron radiation sources in stroboscopic pump-probe measurements (Lauhé et al., 2012;Jacques et al., 2016). We implemented a hybrid pixel detector on our laboratory diffractometer, originally for the study of electric-field induced structural changes (Fertey et al., 2013), and recently demonstrated also its excellent potential for high-resolution X-ray diffraction studies (Wenger, 2015;Wenger et al., 2014). Here, we report on the extension of this setup to laser pump/Xray probe time-resolved crystallography in the millisecond time scale. We demonstrate the feasibility of this kind of experiments using the photo-induced linkage-isomerisation in Na 2 [Fe(CN) 5 NO].2H 2 O (sodium nitroprusside, SNP) as a prototype case. This example allows for a thorough test of the capabilities of our setup: photo-induced variations of Bragg intensities of the order of 1% have been detected within acquisition windows as short as 6 ms, limited only by counting statistics.

The paper is organized as follows. In the first section, we describe the different elements of the experimental setup and the appropriate measurement methods for millisecond time-resolved photocrystallography. In the second part, the results of photo-induced linkage-isomerisation in SNP is reported.

X-ray diffractometer for in-house time-resolved photocrystallography

We have initially designed an experimental approach to perform X-ray diffraction on single crystal under in situ electric field perturbation using a specific pump-probe mode called the field-switching method (Fertey et al., 2013). It consists in the application of a periodic four-step electric field (E=E + , 0 + , E -, 0 -) on the sample, and measuring the diffraction signal in a quasi-static mode sequentially during the four steps. The corresponding structural response of the crystal can be investigated through the electric field induced position shift ∆θ and relative intensity variation ∆I/I of Bragg peaks. A setup based on a laboratory X-ray diffractometer was elaborated using a fast hybrid pixel area detector XPAD gated in synchronization with the four electric field steps applied on the crystal. A dedicated firmware allowing adapted architecture of the internal electronics of the detector permitted multiplexing and stacking inside the detector of the four different diffraction frames corresponding to the four measurement steps. An appropriate statistics was achieved by the internal summation of numerous pump-probe cycles leading to four images read out of the detector as a final result of the experiment.

Based on this operational experiment and methodology, a new setup has been developed for lightinduced time-resolved crystallography. The approach combines two experiments performed simultaneously on the same single crystal sample on the diffractometer : (1) time-resolved lightinduced X-ray diffraction, and (2) transient optical absorption spectroscopy. These two experiments provide the optical and structural response of the crystal under in situ pulsed laser excitation as a function of time.

The pump-probe approach using a gated X-ray photon counting XPAD detector

Time resolved X-ray photocrystallographic experiments are conveniently performed using a sequential stroboscopic scheme. Depending on the type of X-ray source, laser pump, and X-ray detector, there are essentially two different schemes for performing pump-probe measurements, as illustrated in figure 1.

(1) In the first case (fig. 1 left), the X-ray probe beam consists of a sequence of short X-ray pulses X(t), whose frequency defines the overall time scale of the experiment. During an individual cycle, a dynamical process is initiated by applying a short laser pulse to the crystal, whose response is monitored after a tunable time delay Δt through the X-ray scattered signal. The sample response may be of different kinds, such as transient population of metastable state species, change of crystal symmetry, whose signature is detected by a transient modification of Bragg peak intensity, shape or position. Pump laser pulses S(t) are accurately synchronized with the train of X-ray pulses, adjusting the desired time delay Δt. The signal scattered by the crystal sample occurs only during the X-ray pulses X(t), so that a conventional integrating CCD or Image Plate detector is convenient. The pump-probe cycles are repeated until a satisfactory signal to noise ratio is achieved to provide on the detector a summed scattered signal, signature of the transient structure of the sample averaged over X(t). These experiments are typically based on the intrinsic time structure of synchrotron radiation, plasma sources, or XFELs, and make use of phase-locked mechanical choppers to select X-ray pulses ; the X-ray pulse duration X(t) is in the range of hundreds of picoseconds to femtoseconds. The time resolution is controlled by the pump and probe pulse widths 𝜏 𝑝𝑢𝑚𝑝 and 𝜏 𝑝𝑟𝑜𝑏𝑒 respectively), the mismatch in the temporal overlap between the pump and probe pulses (𝜏 𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ ), and the timing (2) In the second approach (fig. 1 right), a continuous X-ray beam is delivered by a rotating anode or a conventional X-ray tube. During the pump-probe cycle, the structure of the crystal sample is continuously changing, as well as the scattered signal on the X-ray detector. In order to probe the transient structure, the scattered signal is sampled by using a fast-gated detector such as a hybrid pixel photon counting detector, a streak camera, or an avalanche photodiode. The effective recording of the signal by the detector occurs only during a short adjustable window X(t). In that case, the laser pulse serves as a trigger for the gating of the detector, and synchronization with a tunable time delay

Δt. The maximum time resolution is in that case also dependent of the electronic response time of the X-ray detector:

𝜏 = (𝜏 𝑝𝑢𝑚𝑝 2 + 𝜏 𝑝𝑟𝑜𝑏𝑒 2 + 𝜏 𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ 2 + 𝜏 𝑗𝑖𝑡𝑡𝑒𝑟 2 + 𝜏 𝑑𝑒𝑡𝑒𝑐𝑡𝑜𝑟 2 ) 1 2 ⁄ Figure 1
Principle of time-resolved X-ray scattering pump-probe measurement with (a) an intrinsically pulsed X-ray beam (e.g. synchrotron, plasma source or XFEL), (b) a quasi-continuous Xray beam with a scattered signal sampled by a fast-gated detector (only one delay is depicted here). The sample response may be considered as a metastable state population

These two schemes have a different impact on the X-ray scattered signal-to-noise ratio (noise is here considered as the estimated standard deviation on the measured signal). In the first case, the X-ray detector acquires the signal continuously, but scattering from the sample occurs only during X(t) (blue areas in figure 1). If a CCD detector is used, the statistics of the experiments suffers from CCD readout noise and dark current. On the contrary, a gated hybrid pixel detector, such as our XPAD detector, is governed by photon counting statistics, and does not suffer from dark current and read-out noise.

Signal acquisition is defined by tuneable measurement time windows (X(t) in figure 1 right) whose temporal width is only limited by the detector response time (150 ns for XPAD). For a laboratory based time-resolved experiment, a hybrid pixel detector is therefore very advantageous.

Another major difference between the two schemes is the possible number of simultaneous measurement windows. In the first case, only one X(t) window, corresponding to one delay Δt can be acquired by the detector at the same time. The experiment should be reproduced for different delays to collect a complete temporal dynamics. In the case of the XPAD detector, the firmware allows collecting and stacking simultaneously four different images, as described above for the sequential four-step electric field measurement. The adaptation to the present photocrystallographic method leads to four different time delays being acquired simultaneously and stacked inside the detector; figure 1 right illustrates only one measurement window, the principle for the four-delays measurement is shown in figure 2 and3.

Layout of the instrument

Our time-resolved light-induced diffraction setup is following the second scheme described in the previous section, using a gated hybrid pixel area detector XPAD. The instrument consists of several elements combined in two simultaneous experiments : time resolved diffraction and transient absorption spectroscopy. A layout of the instrument is given in figure 2. The pump system consists of a Surelite II-10 Nd-YAG pulsed laser equipped with an optical parametric oscillator (OPO, model SLOPO+). The system delivers laser pulses with 3-5 ns duration at a frequency of 10 Hz. The OPO is tuneable in the range 400-700 nm. The laser beam is directed to the single crystal sample on the diffractometer using a series of silver-coated mirrors and precisely adjusted and focussed through appropriate lenses to illuminate the crystal uniformly (i.e. the beam is larger than the crystal). A pump-probe cycle is defined by the time between two laser pulses (100 ms at 10 Hz frequency of the Surelite II-10 system). During 100 ms, the single crystal sample is excited once at t=0, followed by a temporal evolution. The molecules in the sample are for instance excited by the pump laser pulses at t=0, and then recover the ground state within 100 ms through possible intermediate transient states, so that the sample is recovering the ground state before the next laser pulse; this is mandatory for such a stroboscopic scheme. If a lower repetition rate, corresponding to a longer pumpprobe cycle is required, a fast shutter can be installed at the output of the pump laser to select a fraction of the pulses out of the pump laser sequence (e.g. one pulse over two, …).

Using a pinhole of 110 µm in diameter (typical of a single crystal sample size and X-ray beam size of our diffractometer), the pump laser photon flux on the sample position has been evaluated to 9.10 18 photons.cm -2 per pulse at 450 nm. For the SNP crystal that we chose for our subsequent validation, this corresponds to nearly 2000 photons per molecule per laser pulse.

Single crystal transient absorption spectroscopy

It is usually convenient to completely characterize the photo-physical properties of the sample using transient optical absorption spectroscopy or luminescence as a function of temperature in a prior experiment to prepare the subsequent photocrystallographic investigation. Such optical studies of the photoexcitation process allow defining important parameters such as : photoexcitation efficiency, thermal relaxation properties (activation energy, frequency factor), number of transient excited states, population evolution of the excited states, reversibility of the complete process, bleaching effects. It is especially important to assess the reversibility of the photoexcitation with respect to the frequency of the envisioned pump-probe measurement (repetition rate in figure 1). For our purpose, we decided to develop a transient optical absorption spectroscopy experiment centred on the crystal mounted on the X-ray diffractometer, allowing performing the optical measurement simultaneously with the photocrystallographic one. The system consists of a continuous optical probe laser aligned on the crystal sample, and focussed to a diameter smaller than the diameter of the pump laser at the sample position, in order to achieve a perfect spatial overlap. The transmitted light is detected with a 200 MHz Si-Pin diode from Femto Messtechnik GmbH. Interference filters are mounted in front of the probe diode to attenuate the pump laser light.

Time-resolved photocrystallography with the XPAD detector

The X-ray diffraction experiment is based on a modified Bruker-Nonius Mach3 four-circle diffractometer equipped with a molybdenum Incoatec microfocus X-ray source, and a nitrogen gas flow cryogenic system (Wenger, 2015). The typical X-ray beam size at the sample position is of the order of 100 µm, and the photon flux is estimated to 2.10 9 ph.s -1 .cm -2 . The complete goniometer movements are controlled by a prototype software to avoid any collisions with all the optics, optics holders, and X-ray source elements. This is automatically taken into account in the preparation of the optimized diffraction data collection strategy.

The most important element of the setup is the X-ray hybrid pixel area detector XPAD S270. We have previously presented the working principle of hybrid pixel detectors and compared their characteristics with CCD and CMOS detectors (Allé et al., 2016), and we described the specific stacking pump-probe mode of the detector (Fertey et al., 2013), so that only the main characteristics for the present photocrystallographic concerns are recalled here. The XPAD consists of 4 modules arranged horizontally in a flat geometry. Each module contains seven chips each consisting of 9600 pixels (pixel size 130×130 µm 2 ). The resulting images have a size of 75×73 mm 2 , including blind areas separating the different modules. The X-ray photons are absorbed by the upper 300 µm thick silicon sensor layer with 37% absorption efficiency for Mo energy. Each X-ray photon is converted into one electrical pulse that is individually counted for each pixel by an independent counting channel (12-bit counter value).

Each pixel counting channel is controlled by an electronic gate, so an electrical gate signal (logical value 1 or 0) enable or disable the counting process (open or close the gate), this function is the so-called electronic shutter and can switch as fast as 150 ns (which sets the shortest possible image X-ray exposure time and is short enough for single bunch experiment at synchrotron). Once the acquisition is over, the counter values of the pixels (9600 for each electronic chip) are transferred (read) to the electronic board in t R = 1.7 ms. Once the reading is done, the image can be processed on the board inside the detector, independently of what occurs on the chip, so that the chip can start immediately a new acquisition while the previous image is processed. The electronic board consists of a FPGA (Field Programmable Gate Area) and fast memory (SRAM). A dedicated firmware configures the architecture of the electronic inside the FPGA to allow controlling and reading of all the pixels as well as multiplexing, summing and storing images in four different channels modulo 4 in synchronization with the gate signal (fig 3). This is the so-called IPI (Image Plus Image) mode. Hereafter, we consider a complete acquisition as a repetition of N pump-probe cycles. Four frames are collected during each cycle, separately summed (in the FPGA) with the frames measured during the previous cycles and stored in the SRAM, this is called stacking. The output of the measurement is therefore simply four different summed images. This stacking processing in the FPGA is performed in t S = 3.7 ms. This process is illustrated in figure 4. The stacking is independent of the photon counting process in the chip, so that the stacking stage can be initiated while the next image is exposed to X-rays. The blind time between two successive image measurements is the read time t R (1.7 ms). The reading-stacking process is triggered by the end of the gate and before reading a new image the FPGA must have finish the stacking of the previous one so that the minimum delay between the end of two successive images (minimum sampling period) is the read time t R plus the stacking time t S 1.7+3.7 = 5.4 ms. For the timeresolved X-ray diffraction experiment, the XPAD computer is in slave mode and is driven by a master computer that also controls the goniometer and a specific electronic to generate the gate signals (fig 3). The pump signal is monitored with a Si-Pin diode located in the OPO system. For each laser pump pulse the diode generates an output voltage which triggers the electronic to generate the four gate signals with adjustable time and delay. The investigation of light-induced structural dynamics requires a precise measurement of the Bragg intensities I(hkl) as a function of time. As explained above, the XPAD detector allows collecting four different frames sampled between one pump-probe cycle. One frame (F#4 in figure 4) is systematically adjusted exactly at the end of the cycle, just before the next laser pulse. This frame is used as a reference for the ground state, for instance to compute I#1(hkl)/I#4(hkl) intensity ratios, where I#4(hkl) (resp.

I#1(hkl)) Bragg intensities are obtained from integration of the Bragg peaks in frame F#4 (resp. F#1).

Two scanning modes are implemented, corresponding to two different types of measurement as illustrated in figure 5.

In the first type of measurement, the goniometer holding the single crystal sample is continuously rotated while the pump-probe cycles are collected (continuous rotation scan mode). This is similar to the crystal rotation method in conventional single crystal X-ray crystallography. The maximum rotation speed of our MACH3 goniometer is nearly 2deg/min, so that even a very narrow Bragg peak with a mosaïcity of 0.1deg will be accurately sampled at 10 Hz : the complete peak profile will span at minimum 30 pump-probe cycles, and therefore 30 times (F#1…F#4).

For the second type of measurement, the crystal is rotated step by step (M' different steps), and for each step, N' complete pump-probe cycles are recorded (the total number of cycles is therefore M'×N'). A very fine slicing of the Bragg peak profile may thus be achieved.

Intensity statistics of time-resolved photocrystallography with the XPAD detector

The XPAD detector is a photon counting hybrid pixel detector. Using diffraction measurement tests, we have verified that the intensity statistics of the XPAD detector obeys the main characteristic of Poissonian statistics, in particular the intensity variance is equal to the mean intensity (Wenger, 2015).

As a matter of fact, the measured intensity on a given pixel 𝐼 𝑖 corresponds to the number 𝑁 𝑖 of absorbed photons (37% efficiency for 300 µm thick silicon sensor at Mo Kα radiation) above the discrimination threshold collected during the X-ray exposure time X(t), and the signal variance is 𝜎 2 (𝐼 𝑖 ) = 𝑁 𝑖 (Alle et al., 2016). The total intensity of a Bragg reflection is in a first approximation a direct summation of the intensity of all the pixels under the peak

I 𝑝𝑒𝑎𝑘 = ∑ 𝐼 𝑖 𝑝𝑖𝑥𝑒𝑙𝑠 = ∑ 𝑁 𝑖 𝑝𝑖𝑥𝑒𝑙𝑠
, and the corresponding variance is 𝜎 2 (𝐼 𝑝𝑒𝑎𝑘 ) = ∑ 𝑁 𝑖 𝑝𝑖𝑥𝑒𝑙𝑠 = 𝐼 𝑝𝑒𝑎𝑘 if one assumes no pixel to pixel correlations. This is true if one works directly on the raw frames without applying any distortion or flat field calibrations.

In the context of time-resolved photocrystallography using a photon counting hybrid pixel detector, the important issue is the detection of a significant scattered (relative) intensity difference. The total intensity of a Bragg peak is directly proportional to the exposure time and angular rotation speed of the crystal, which are tuneable parameters of the experience. It is also common to repeat the pumpprobe cycles until a satisfactory statistics is achieved. In our experience, we want to detect an intensity difference between frame F#1, F#2, F#3 with respect to frame F#4. For a given Bragg peak, if one wants to detect a 1% intensity difference with a 3σ confidence level, the minimum intensity of the ground state (F#4) peak is nearly 180000 counts.

𝐼 #1 -𝐼 #4 = 0.01 × 𝐼 #4 𝜎 2 (𝐼 #1 -𝐼 #4 ) = 𝐼 #1 + 𝐼 #4 if one assumes no correlation between I #1 and I #4 .

𝐼 #1 -𝐼 #4 > 3 × 𝜎(𝐼 #1 -𝐼 #4 ) 𝐼 #4 > 180900
This value is quite general, and may be used systematically as an indicator for the preparation of a real experiment with our system.

Experimental validation : time resolved crystallography of photo-induced linkage isomerism in sodium nitroprusside Na2[Fe(CN)5(NO)].2H2O

To assess the possibilities and limitations of such pump-probe X-ray diffraction experiments based on a laboratory setup, we have designed a validation experiment using a prototype molecular photoswitchable material, namely sodium nitroprusside Na 2 [Fe(CN) 5 (NO)].2H 2 O (SNP). SNP exhibits light-induced linkage isomerism, a photochemical effect, which results in the rotation of the nitrosyl ligand NO upon visible photon absorption, and formation of two long-lived metastable states. MS2 is formed by a rotation of the nitrosyl by nearly 90 degrees, leading to a side-bonded coordination mode, while MS1 results from a nearly 180 degrees rotation to an isonitrosyl Fe-ON coordination mode as revealed by photocrystallographic techniques (Carducci et al., 1997;Fomitchev & Coppens, 1996;Schaniel et al., 2005a;Schaniel et al., 2006). The complete photophysics of SNP is well understood from time-resolved UV-visible absorption spectroscopy and time-resolved infrared spectroscopy (Schaniel & Woike, 2009;Schaniel et al., 2010). The mechanism involves a direct Fe(3d) to π*(NO) metal-to-ligand charge transfer transition (MLCT) or indirect transition as a relaxation from excited higher lying d-states into the π*(NO). After such an electronic excitation, the NO ligand undergoes a rotation by about 90° within 300 fs (Schaniel et al., 2010). This fast internal conversion from the excited GS potential into the MS2 potential is followed by a vibrational relaxation to the MS2 minimum, which occurs within about 10 ps (Gallé et al., 2012). MS1 and MS2 are local minima on the potential energy surface of the ground electronic state, they lay about 1.05 eV (MS1) and 0.95 eV (MS2) above the GS minimum, and they are separated by an energy barrier of 0.69 eV (MS1) and 0.43 eV (MS2) from the GS (Woike et al., 1993).

The validation measurement has been conducted as follows. In the first step, X-ray diffraction measurements have been performed at T=100 K in the ground state and long lived metastable state MS2 in order to get complete X-ray diffraction data sets which could be used to simulate the timeresolved photocrystallographic experiment, and select the Bragg intensities which exhibit the highest GS to MS2 intensity contrast. In the second step, a complete stroboscopic time-resolved measurement was conducted to derive the optical response (transient absorption spectroscopy measurement) and structural response of the sample.

X-ray diffraction of the GS and MS2 states in photostationnary conditions

In a first step, a conventional measurement in the ground state (GS) has been performed at T=100 K.

The diffraction frames have been indexed and integrated using the Crysalis software (Agilent, 2014) yielding 12675 (2658 unique) measured reflections up to a maximum resolution of sin(θ)/λ = 0.82 Å -1 (R int = 0.013). The corresponding crystal structure has been refined using the Shelxl software (Sheldrick, 2008). In a second step, the crystal was irradiated with a photodiode at 445 nm during 4 hours, populating MS1 and MS2 states at the same time. The crystal was subsequently irradiated at 1064 nm during 10 minutes to transfer MS1 molecular states to MS2 and to the ground state. The final single crystal contains therefore only molecules in the ground state and molecules in the metastable state MS2 [START_REF] Schaniel | [END_REF]. Diffraction data have been collected in this state up to a maximum resolution of sin(θ)/λ = 0.82 Å -1 (9586 measured reflections, 2630 unique reflections, R int = 0.016). A

Simulation of structural changes and diffracted intensity contrast factors

In order to assess the contrast in Bragg peak intensity difference we might expect from the timeresolved X-ray diffraction experiment, we have performed the following analysis. Starting from the measured Bragg intensities in the ground state and in the metastable state MS2 in the photostationnary state presented in the previous section, we have calculated the corresponding normalized intensity difference:

∆𝐼 𝜎(∆𝐼) = 𝐼 𝑀𝑆2 (ℎ𝑘𝑙) -𝐼 𝐺𝑆 (ℎ𝑘𝑙)
𝜎(𝐼 𝑀𝑆2 (ℎ𝑘𝑙)) + 𝜎(𝐼 𝐺𝑆 (ℎ𝑘𝑙))

As discussed above, the intensity statistics on the hybrid pixel detector XPAD follows counting statistics, so that the previous equation is simply:

∆𝐼 𝜎(∆𝐼) = 𝐼 𝑀𝑆2 (ℎ𝑘𝑙) -𝐼 𝐺𝑆 (ℎ𝑘𝑙) √(𝐼 𝑀𝑆2 (ℎ𝑘𝑙)) + √(𝐼 𝐺𝑆 (ℎ𝑘𝑙))
This experimental intensity contrast is shown in figure 7. A similar analysis was performed for the simulated intensity contrast, calculated from the ground state and MS2 state refined structural models for hypothetical 1%, 5% and 10% populations of the metastable state MS2. As can be seen in figure 7 (right), at P MS =10% MS2, the intensity contrast is significant for sodium nitroprusside between the ground state and metastable state MS2. 104 reflections over 2616 analysed reflections have intensity variation greater than the estimated standard deviation. This becomes much more challenging at P MS =1%, at which a very high accuracy is required to detect the structural reorganisation characterizing MS2. No reflections exhibit signal to noise ratio better than 0.2. This simulation shows that an extremely high accuracy will be required in the time-resolved measurement.

According to this simulation, we identified the (240) reflection (see figure 7 left), which is one of the strongest Bragg reflection, at low angle, and with a high intensity difference between the ground state and the metastable MS2 state. This reflection will be monitored precisely in the time-resolved photocrystallographic experiment.

Results from transient absorption spectroscopy

The optical absorption spectra of the three states of SNP (GS, MS1, and MS2) are given in figure 8. The absorption spectrum of the ground state is dominated by two bands located at 388 nm and 495 nm, corresponding to Fe(3d) to π*(NO) metal to ligand charge transfer [START_REF] Schaniel | [END_REF]. In the MS1 and MS2 states, new bands appear at 755 nm and 560 nm respectively owing to the shift in energy of the molecular orbitals [START_REF] Schaniel | [END_REF]. In view of these spectra, appropriate wavelength for the pump and optical probe lasers can be chosen. The OPO of the pump system has been tuned to 450 nm, while a probe laser at 633 nm was selected, at which MS2 state exhibits a high absorption and at the same time MS1 and GS exhibit a low absorption. 633 nm is therefore very sensitive to the population of MS2.

Figure 8 Optical absorption spectra of SNP in the different states. (adapted from [START_REF] Schaniel | [END_REF].

Transient optical measurements have been conducted as a function of temperature over the 150-250K temperature range with the pump laser polarization direction oriented along the crystalline c axis in order to enhance the optical response (Schaniel et al., 2005b); the corresponding trace at 150K is shown in figure 9. As can be seen, a rapid decrease of transmitted intensity occurs, which recovers in a second step. The decrease can not be resolved with the resolution of our setup; it has been shown by Schaniel et al. (2010) that the population of MS2 state occurs on the femtosecond time scale. The intensity decrease at 633 nm corresponds to an increase of absorption, consistent with a population of MS2 state (see figure 8). The subsequent intensity increase corresponds to the thermally activated relaxation of MS2 to the ground state. For each temperature, the intensity as a function of time was adequately fitted to an exponential dependence I(t) = I 0 + ∆I max e -t τ with τ the time constant corresponding to a temperature dependent relaxation rate k MS2→GS (T)=1/; at T=150K, the fitted time constant is τ = 14 ms. The rate constants are plotted in figure 9 in the form of an Arrhenius plot

ln ( 1 τ ) = ln(𝑍) - 𝐸𝑎 𝑘 𝑏 𝑇
with Ea the activation energy of the relaxation process, and Z the frequency factor. As seen in figure 9, the relaxation process follows a thermally activated process (linear trend), with refined parameters Ea=0.42(2) eV, and Z = 15.10 13 s -1 . These values are in perfect agreement with values reported in the litterature (Schaniel et al., 2005b). The transient optical absorption setup is therefore validated. ). The red line is a linear fit with refined parameters Ea=0.42(2) eV, and Z = 15.10 13 s -1 .

Time-resolved x-ray diffraction experiments

The results of the transient absorption spectroscopy measurements shows that the dynamics of SNP at T = 150 K is appropriate for a time resolved photocrystallographic investigation permitted by our instrument. The relaxation is complete within 100 ms at this temperature, and the most important evolution occurs within the first 50 ms. Accordingly, the molecules pumped to the MS2 state by the laser pulses relax to the ground state before the next laser pulse 100 ms later at 10 Hz frequency, the recovery condition is fulfilled. Using the goniometer of the diffractometer, we have oriented the crystal so that the reflection (240) is in diffraction condition. This reflection has been selected as one of the most intense reflection with a high intensity difference between the ground state and MS2 state, according to our simulation described above. Time-resolved diffraction has been conducted using the continuous rotation scan mode, with acquisition windows of 6 ms whose centres are located at time delays ∆t=3, 13, 23, 93 ms, that is 3 windows sampling the beginning of the pump-probe cycle, and one window at the end of the cycle as a reference for the ground state. In other words, the diffracted X-ray signal is integrated over 6 ms for each acquisition window (each time delay). Omega scans of 5 degrees, with 118.4 seconds per degree (scan speed ratio 32 with the MACH3 goniometer software). The complete exposure time per scan is 592 seconds. At 10Hz, 1184 pump-probe cycles occur within one degree rotation, the sampling is therefore very good. This scan has been repeated 300 times to increase statistics, giving a total of nearly 1800000 pump-probe cycles, that is 7.2 million total images acquired (F#1…F#4). Owing to the stacking procedure, 300 times four summed diffraction frames result from the measurement. The total duration of the experiment is nearly 49 hours. photoswitching. This is a further indication that the light-induced intensity changes are not due to a gradual crystal deterioration; in that case, a systematic intensity decrease would have been obtained.

This result is very important, because using our experimental conditions and setup, an intensity change as low as 1% was accurately detected.

In our time resolved experiment, the metastable state population follows an exponential relaxation The influence of the total number of measured scans on the accuracy of the derived X-ray diffraction data is evidenced in figure 11 left. As we can see, the accuracy of one single scan (nearly 6000 pumpprobe cycles) is not satisfactory, the standard deviation is much too high to show a conclusive trend.

Averaging over 100 scans already presents the correct trend.

The limited Bragg intensity change measured in the time-resolved diffraction experiment results from a low photoconversion efficiency. As stated in reference (Schaniel et al., 2005b), where an intensity dependent study of the light-induced absorption was made, the saturation population at a pump wavelength of 532 nm is 0.8% (using 4ns long laser pulses). The ration of the absorption cross sections at 532 nm between MSII and GS was calculated to 1.25•10 -3 . As a consequence, MSII absorbs about 1000 times more than GS at this wavelength. At 450 nm this ratio is smaller, but still MSII has a higher absorption cross section than GS. Therefore, obtaining a slightly higher population at 450 nm than at 532 nm pulse wavelength is consistent. Furthermore, these results are in agreement with the populations determined for MSII from low-temperature measurements using differential calorimetry (Woike et al., 1993), where for MSII saturation populations of 3% for a pump wavelength of 457.9 nm and about 0.5% for the pump wavelength of 532nm were obtained. This implies also that with the pump fluence available in our experiment we are close to saturation of the MSII population, i.e. there is an equilibrium between GS→MSII and MSII→GS processes. This again, is compatible with the fact that the population of MSII occurs on a 300 fs time-scale (Schaniel et al., 2010), so within the pulse duration of 2-4 ns there is plenty of time to establish a "photostationnary" state.

Conclusions and perspectives

We have presented a new experimental setup based on a laboratory diffractometer, dedicated to time-resolved X-ray diffraction measurements for investigating light induced structural changes in single crystals. For that purpose, we have implemented a laser pump/X-ray probe scheme using a fast gated hybrid pixel area XPAD detector, which is triggered by the pump laser pulses, and accurately synchronized with tuneable time delays ∆t between pump and probe.

First experiments on the photoswitching dynamics of sodium nitroprusside at T=150 K are very conclusive. 1% intensity changes have been detected with a 6 ms acquisition window. According to the characteristics of the XPAD detector, much shorter acquisition windows could be envisioned, at the expense of the limited X-ray flux, and therefore estimated scattered signal, but the sampling time cannot be shorter than 5.4 ms. The present experiment overall duration was nearly 49 hours for only 5 degrees of reciprocal space covered, and showed an adequate long-term stability for 1% intensity change detection. For our prototype experiment on sodium nitroprusside, we have focussed on the reflection (240) selected according to its high intensity, and high intensity change upon GS→MS2 photoswitching. A complete time-resolved measurement covering a complete sphere of reciprocal space seems out of reach with the present setup. An appropriate selection of relevant reflections, which are very sensitive to the structural changes would be necessary, possibly using a leverage analysis in case of a least-squares structural refinement of the light-induced model (Parson et al., 2012;[START_REF] Prince | Mathematical Techniques in Crystallography and Materials Science[END_REF]. Improvements to the setup are foreseen for the near future. A more powerful and stable pump laser system is currently being implemented in order to reach higher photoconversion per pulse, and therefore higher photoinduced Bragg intensity change. A new generation of XPAD with a 10 measurement windows firmware instead of only 4 is commissioned, with a faster electronic allowing sampling period of 4 ms as well as a 1 mm thick silicon sensor in order to improve the X-ray photon absorption efficiency. With these progresses, we anticipate a shortening of the measurement time per frame for appropriate statistics, so that a complete data collection of a photo-induced state for structural analysis would be possible.

Although many current time-resolved structural studies focus on ultrafast phenomena, there are plenty of much slower solid state processes developing in the microsecond to second time scale. The investigation of the structural response of crystals to external perturbation (electric field, or pulsed laser) in this time range is fundamental for understanding many physical properties of current interest, such as the switching dynamics of domain patterns in ferroelectric or ferroelastic materials, the nucleation and growth processes at phase transitions, the development of long range modulated structures, the propagation of lattice distortion and strain just to name a few. Our instrument is appropriate for such time-resolved structural studies, and may be used for a wide range of scientific applications.

Table 1 Selected characteristics of the time-resolved X-ray diffraction experiment, and hybrid pixel The structural difference between GS and MSII is limited so that the difference between 𝐹 𝑀𝑆𝐼𝐼 (𝐻 ⃗ ⃗ ) and 𝐹 𝐺𝑆 (𝐻 ⃗ ⃗ ) is also quite small. In the previous equation the second term on the right hand side is dominant with respect to the last term for small initial population 𝑝 0 , so that in a first approximation, the time resolved Bragg intensity follows a single exponential dependence with time constant 𝜏.

The

Figure 2

 2 Figure 2 Schematic of the combined time-resolved X-ray diffraction and transient absorption spectroscopy experiments.

Figure 3

 3 Figure 3 Instrumental setup. A master computer drives the goniometer, the XPAD computer and a synchronization electronic. The pump laser works at 10 Hz, the master computer positions the XPAD detector in stacking mode (waiting for acquisition) and then sends the start scan to the goniometer and the start acquisition to the electronic. The electronic triggered by the diode pulse generates the gate signal to open and close the electronic shutter, and to synchronize the stacking processing. At the end of the scan the XPAD computer reads out the four images resulting from the internal stacking.

Figure 4

 4 Figure 4 Schematic time structure of the laser pump-X-ray probe measurement strategy. The laser frequency is 10Hz, corresponding to 100 ms between two laser pulses. The laser pulse duration is 4 ns.The Image Plus Image (IPI) mode of the XPAD offers at present the acquisition and internal stacking of 4 frames (noted F#1…F#4 and colour coded red, green, blue and pink in the figure). F#4 is taken as the reference frame just before the laser pulse. "R" is the read time of each frame (t R = 1.7 ms). "S" is the internal stacking of each frame (t S = 3.7 ms). t f is the exposure time, which could be as short as 150 ns, the time resolution of the internal electronics of the detector. The minimum sampling period is 5.4 ms.

Figure 5

 5 Figure 5 Chronogram of the measurement sequence in the (left) continuous rotation scan (CRS) mode, and (right) step scan (SS) mode. In the CRS mode, the images are measured "on the fly" during the rotation of the crystal, therefore corresponding to N pump-probe cycles. In the step scan mode, an acquisition consisting of N' pump-probe cycles is measured for each of the M' goniometer rotation positions.

  figure 6.

Figure 6

 6 Figure 6 Refined structural models of the ground state (GS, left) and metastable state (MS2, right) of SNP.

Figure

  Figure 7 (left) Experimental contrast ∆𝐼 𝜎(∆𝐼) as a function of sin (𝜃) 𝜆 for SNP calculated from the measured ground state and metastable state MS2 in the photostationnary state. The arrow points to the (240) reflection whose intensity is monitored in the time-resolved photocrystallographic experiment. (right) simulated contrast calculated from the ground state and MS2 state refined structural models for hypothetical 1%, 5% and 10% populations of the metastable state MS2.

Figure 9

 9 Figure 9 left : transient intensity on the detection Si-pin photodiode as a function of time at T=150K. Th red line is an exponential fit I(t) = I 0 + ∆I max e -t τ . right : Arrhenius plot ln ( 1 τ ) as a function of

Figure 10 (

 10 Figure 10 (left) Experimental time resolved diffraction image (image #1 and #4 of scan 43 among 300 scans). (right) zoom on the selected reflection (240), and pixel to pixel colour coded difference intensity.

  with initial population 𝑝 0 and lifetime 𝜏 of the metastable state, so that the corresponding Bragg intensity is given by:𝐼(𝐻 ⃗ ⃗ , 𝑡) = (𝑝(𝑡) * 𝐹 𝑀𝑆𝐼𝐼 (𝐻 ⃗ ⃗ ) + (1 -𝑝(𝑡))𝐹 𝐺𝑆 (𝐻 ⃗ ⃗ )) × (𝑝(𝑡) * 𝐹 𝑀𝑆𝐼𝐼 * (𝐻 ⃗ ⃗ ) + (1 -𝑝(𝑡))𝐹 𝐺𝑆 * (𝐻 ⃗ ⃗ ))For a centrosymmetric crystal like SNP, 𝐹 𝑀𝑆𝐼𝐼 (𝐻 ⃗ ⃗ ) and 𝐹 𝐺𝑆 (𝐻 ⃗ ⃗ ) are real quantities and𝐼(𝐻 ⃗ ⃗ , 𝑡) = 𝐹 2 𝑀𝑆𝐼𝐼 𝐹 𝐺𝑆 )We have simulated this equation for the (240) reflection, the results are given in the supplementary material. It shows that the first and second terms of this equation are in all cases dominant with respect to the last term, so that the intensity evolution of a Bragg reflection should follow in a good approximation a single exponential dependence.The intensity ratio I(240)#1/I(hkl)#4 as a function of time has therefore been quantitatively fitted with an exponential decay 𝑦(𝑡) = 1 + 𝐴𝑒 -𝑡 𝑡1 giving a time constant of t1 = 10(1) ms, in perfect agreement with the time constant derived from the optical absorption at T = 150 K : τ = 14 ms.

Figure 11 (

 11 Figure 11 (left) Normalized intensity of the (240) reflection averaged over one single scan, and averaged over 100 and 300 repeated scans. (right) Normalized intensity of the (240) reflection as a function of time averaged over 300 repeated scans. The insert shows the histogram of the normalized intensity distribution over the 300 scans. The continuous line is a Gaussian fit to the distribution with average position 1.0097(4). The dashed line indicate the I1/I4=1.0 position.

  Photon flux (ph.s -1 .cm -2 ) 2.10 9 Beam size at sample position (µm)

  following figure illustrates the simulated normalized intensity of the (240) reflection as a function of time with different values of the initial population 𝑝 0 . As can be seen, all the curves follow in a first approximation a single exponential decay with time constant𝜏 very close to 14 ms.

Figure S2 .

 S2 Figure S2. Simulated normalized intensity of the (240) reflection as a function of time with different values of the initial population 𝑝 0 . All the curves have been adequately fitted to a single exponential decay.

Acknowledgements

We are grateful to Claude Lecomte for discussions, Theo Woike and Mirco Imlau for tremendous help in the design of the transient optical spectroscopy and pump laser systems. This work was supported by the CNRS, the Université de Lorraine, the French 'Agence Nationale de la Recherche' (grant Bistamat ANR-12-BS07-0030-02), and "Lorraine Université d'Excellence".

Funding information Agence Nationale de la Recherche (award No. ANR-12-BS07-0030-02

Supplementary material of

Dependence of the diffracted intensity as a function of time delay

The scattering factor for a crystal containing a fraction 𝑝 of molecules in the metastable state, and 1 -𝑝 molecules in the ground state with a random spatial distribution can be written :

So that the scattered intensity is simply:

In a time resolved experiment, the metastable state population is time dependent so that the intensity becomes:

For an exponential relaxation with lifetime 𝜏 of the metastable state and initial population 𝑝 0 :

For a centrosymmetric crystal like SNP, 𝐹 𝑀𝑆𝐼𝐼 (𝐻 ⃗ ⃗ ) and 𝐹 𝐺𝑆 (𝐻 ⃗ ⃗ ) are real quantities.