Chinmay Singhal

Yann Ponty

Isaure Chauvot

Isaure Chauvot De Beauchene

Pseudo-atoms (coarse-graining

A hybrid combinatorial method for docking single stranded RNA on proteins at the thermodynamic equilibrium

Introduction

We recently proposed a fragment-based approach to model ssRNA local conformations and assemble them on the protein surface. By docking then re-assembling overlapping fragments, we could select a pool of most-connected poses that delineates the RNA-binding site (Fig. right) [1]. But the number of possible assemblies is beyond the reach of brute force approaches.

Protein-RNA complexes participate in many aspects of cell regulation. Their atomistic structural description is crucial to understand the recognition mechanism, but the experimental resolution of their structure is arduous. Computational docking methods aim at modeling a 3D assembly, by assembling structures of each isolated constituent. Yet for highly flexible objects like single-stranded RNA (ssRNA), the isolated structure of the whole molecule can adopt an ensemble of conformations too large to be experimentaly solved or computationaly modeled. For the fragment assembly to succeed, all positions in the sequence must have been sampled correctly. We previously used the ATTRACT software [3] to dock RNA on the protein surface. Here we adapted ATTRACT for docking inside deep cavities. As we deal with a poly-U, one single docking of a UUU fragment was performed.

ssRNA-protein complex

Disordered ~ ∞ conformations

Fragment deeper in pocket

B. Fragments assembly by stochastic backtracking

We then search for chains of compatible docking poses with low total energy (approximating poses energy as additive). 10 6 docking poses have to be kept to ensure retaining enough correct solutions at each sequence position. => 10 72 possible assemblies ! => How can we retrieve the most probable assemblies while avoiding a brute-force enumeration?

By an unbiased sampling of paths: For each sample, at each position in sequence, the next pose is chosen among all poses with a probability given by the partition function Z, which defines the probability of each pose to participate in a downstream path. Z is pre-calculated by dynamic programming. The stochastic backtracking provied 100,000 models containing 0.3% correct solutions, and a best solution at 2.2 Å from the reference position (Xray).

U U U U Connectivity : tight overlap (RMSD < 2 Å ) Z fwd (k ,i)= ∑ j /connect ( j , i) exp( E(i , j) RT )×Z fwd (k-1, j)
The scoring function of ATTRACT is not precise enough to select the best models. Instead, we used some knowledge on the system to define geometric constraints used as filters: a. Aromatic rings are known for establishing stacking interactions with RNA bases. One is present at the entrance of the protein pocket → we impose the 4 th base to be at < 5 Å from the aromatic ring.

Conclusion

[ 1 ] Isaure Chauvot de Beauchêne, Sjoerd J. de Vries, Martin Zacharias (2016) PloS.Comput.Biol.

[ 2 ] Martin Zacharias ( 2003) Proteins [ 3 ] Sjoerd J. de Vries, Christina E.M Schindler, Isaure Chauvot de Beauchêne, Martin Zacharias (2015) Biophys. J.

Results

Most connected poses X-ray

Loop conformations : Enumerate local conformations inside domain, check compatibilities pairwise and with domains orientation.

Rotamers (local geometries)

L1 1 L2 1 L1 2 L2 2 L3 1 L3 2

Domains orientation :

Discretize the ensemble of possible pairwise orientations, then assess which are geometrically compatible From RNA sequence, predict a Boltzmann ensemble of secondary structures (2D), dock all fragment of possible structure at each position in sequence, then incorporate the 2D structure probability in the probabilistic assembly.

Coassemble w. protein conformations

Fold the RNA on the protein

E(i) = -1 Kcal/mol P(i) = 0.2
RMSD 1.0 Å P(i, j) = 0.9 RMSD 2.5 Å P(i, j) = 0.5

Weight the edges and nodes of the graph by : We present an improved method capable of modeling protein-bound ssRNAs solely based on the protein structure and ssRNA sequence. The main improvements include:

Combine probabilities

• A new docking protocol for docking RNA fragments inside deep pockets with systematic sampling.

This shows improved sampling quality (lower RMSDs) for the nucleotide fragments buried inside the binding pocket.

• A stochastic backtracking algorithm that can perform unbiased sampling from the connectivity graph of the docked fragments. This algorithm generates near-native RNA chains among 100,000 samples.

• An efficient and effective filtering procedure to incorporate experimental knowledge on the protein-ssRNA system. The filters can be combined, leading to an enrichment of up to x163 after filtering As a first proof of principle, the method could model ab initio a protein-bound ssRNA with a length of 10 nucleotides, an unprecedented length far beyond the reach of standard small-molecule docking programs.

reference position (X-ray) Last model after filtering

  space : 10 7 states (position, orientation, conformer) Energy minimisation in empirical force field Select 10 6 poses of lowest energy Detection of pocket points (POCASA [4], 2 Å probe, 1 Å grid) Selection of points cloud centers (> 500 points within 7Å) Cluster fragment library to keep few representative Coarse filter: Sample orientations * points * few conformers Retain low-energy poses Fine filter: For each pose, test all conformers in same cluster as the representative => search space 10 9 Keep low-energy poses, minimize further, keep 10 6 lowest energy . Cluster w

•

  Compute all pairwise connectivities • Eliminate non-connected poses (propagate) • Compute partition function Z backward, by dynamic programming • Sample paths forward, w. proba = Z(k,i) / ∑Z(k) • Iterate 100,000 times Example with T = ∞ ( simply sum over paths) This stochastic backtracking results in a canonical sub-ensemble of assemblies backtracking Dynamic programming Here, we present an improved method capable of modeling the ssRNA solely based on the protein structure and ssRNA sequence. Improvements include (A) a new docking protocol for systematic sampling in deep pockets, (B) a stochastic backtracking algorithm for unbiased sampling of assemblies, and (C) a combination of filters of the ensemble of sampled models based on biophysical caracteristics of the binding site. As a proof-of-principle, we applied this method on a 11-mer poly-U ssRNA inserted in the deep cavity of an exonuclease None c b a bc ac ab abc combination of filters

b.

  Mg 2+ ions are known for establishing ionic interactions with RNA phosphate groups. One such ion is present in the bottom of the protein pocket → We impose the last phosphate to ba at < 7 Å from the Mg 2+ . c. We consider the single-stranded RNA as linear → nucleotides not in same fragment are at > 6 Å from each other Applying each filter separately leads to an enrichment of x 1.6 to x34 in correct solutions. Combining two filters leads to an enrichment of x 2.7 to x 163. Combining the three filters retains only one model, which is at 4.0 Å from the reference position (cf Figure above).

  Account for protein flexibility : Proteins also change conformation upon binding, albeit less drastically than RNAs. To avoid sampling many of their possible global conformations and use all of them for docking, one can apply to the protein the same principle of decoupled local sampling as for RNA. " set of conformations can be used for docking, then the compatibility of adjacent conformations can be assessed together with the connectivity of RNA poses.


Jérôme Ermont
email: jerome.ermont@enseeiht.fr

Sandrine Mouysset
email: sandrine.mouysset@irit.fr

Jean-Luc Scharbarg
email: jean-luc.scharbarg@enseeiht.fr

Christian Fraboul
email: christian.fraboul@enseeiht.fr

Message scheduling to reduce AFDX jitter in a mixed NoC/AFDX architecture

Current avionics architecture are based on an avionics full duplex switched Ethernet network (AFDX) that interconnects end systems. Avionics functions exchange data through Virtual Links (VLs), which are static flows with bounded bandwidth. The jitter for each VL at AFDX entrance has to be less than 500 µs. This constraint is met, thanks to end system scheduling. The interconnection of many-cores by an AFDX backbone is envisioned for future avionics architecture. The principle is to distribute avionics functions on these many-cores. Many-cores are based on simple cores interconnected by a Network-on-Chip (NoC). The allocation of functions on the available cores as well as the transmission of flows on the NoC has to be performed in such a way that the jitter for each VL at AFDX entrance is still less than 500 µs. A first solution has been proposed, where each function manages the transmission of its VLs. The idea of this solution is to distribute functions on each many-core in order to minimize contentions for VLs which concern functions allocated on different many-cores. In this paper, we consider that VL transmissions are managed by a single task in each many-core. We propose to construct a scheduling table executed by this task using an Integer Linear Program. The access to the Ethernet interface is then only allowed to one VL leading to a significant reduction of the jitter.

INTRODUCTION

Aircrafts are equipped with numerous electronic equipment. Some of them, like flight control and guidance systems, provide flight critical functions, while others may provide assistance services that are not critical to maintain airworthiness. Current avionics architecture is based on the integration of numerous functions with different criticality levels into single computing systems (monocore processors) [START_REF] Do | Software considerations in airborne systems and equipment certification[END_REF]. Such an architecture is depicted in Figure 1. Computing systems are interconnected by an AFDX (Avionics Full Duplex Switched Ethernet) [START_REF]Aircraft Data Network, Part 7: Avionic Full Duplex Switched Ethernet (AFDX) Network[END_REF]. The End System (ES) provides an interface between a processing unit and the network. The continuous need for increased computational power has fueled the on-going move to multi-core architectures in hard real-time systems. However, multi-core architectures are based on complex hardware mechanisms, such as advanced branch predictors whose temporal behavior is difficult to master. Many-core architectures are based on simpler cores interconnected by a Network-on-Chip (NoC). These cores are more predictable [START_REF] Nélis | The Challenge of Time-Predictability in Modern Many-Core Architectures[END_REF]. Thus, many-cores are promising candidates for avionics architecture.

A typical many-cores architecture provides Ethernet interfaces and memory controllers. For instance, Tilera Tile64 has 3 Ethernet interfaces and 4 memory controllers [START_REF] Wentzlaff | On-Chip Interconnection Architecture of the Tile Processor[END_REF], while Kalray MPPA has 8 Ethernet interfaces and 2 memory controllers [START_REF] Dupont De Dinechin | Time-critical Computing on a Single-chip Massively Parallel Processor[END_REF].

An envisioned avionics architecture is depicted in Figure 2. A set of many-cores are interconnected by an AFDX backbone, leading to a mixed NoC/AFDX architecture. Avionics functions are distributed on these many-cores. Communications between two (1) end-to-end transmission delay has to be upper-bounded by an application defined value, (2) frame jitter at the ingress of the AFDX network has to be smaller than a given value (typically 500 µs).

In single core architectures the latter constraint is enforced by the scheduling implemented in the End System. In many-cores architectures, frame jitter mainly depends on the delay variation between the source core and the source Ethernet interface. This variation is due to two factors. First, the frame can be delayed on the NoC by other frames transmitted between avionic functions. Second, the Ethernet controller can be busy, transmitting another frame. [START_REF] Abdallah | Towards a mixed NoC/AFDX architecture for avionics applications[END_REF] proposes a mapping strategy which minimizes the first factor, i.e. the variation of this NoC delay. Each core is allocated at most one function. Each VL is managed by its source function.

In this paper, we mainly address the second factor. We propose a static scheduling of Ethernet transmissions, based on a table. Each transmission is allocated a periodic slot. The scheduling is managed by a dedicated core. The periodic slots are allocated to the applications in order to guarantee that the access to the Ethernet interface is allowed to one application. We formulate the mapping of the applications to the slots as an Integer Linear Program (ILP). The objectives of the approach are:

(1) to allocate the slots for the transmission of the outgoing flows within the respect of their periodic constraint ; (2) to oversample the slots for outgoing flows with period > n ms, with n = 1, 2, 4, . . . The remainder of the paper is as follows. Section 2 introduces current AFDX and NoC architectures. Section 3 explains the addressed problem. The new approach is described in Section 4. Section 5 shows the proposed scheduling solution on a case study and give some results compared to the distributed solution of [START_REF] Abdallah | Towards a mixed NoC/AFDX architecture for avionics applications[END_REF]. Finally, Section 6 concludes with some future works.

SYSTEM MODEL

We summarize the main features of both a AFDX flows and manycores considered in this paper.

AFDX flows

A VL defines a unidirectional connection between one source function and one or more destination functions. Each VL is characterized by two parameters:

• Bandwidth Allocation Gap (BAG). Minimal time interval separating two successive frames of the same VL. The value of the BAG is ranging from 1 to 128 ms. • L min and L max . Smallest and largest Ethernet frame, in bytes, transmitted on the VL. In current architectures, each ES performs a traffic shaping for each VL to control that frames are transmitted in accordance with BAG and authorized frame size. The queued frames, which are ready to be transmitted, are then selected depending on a strategy configured in the VL scheduler. Therefore, it is possible that more than one VL has a packet ready and eligible for transmission. In this case, a queuing delay (jitter) is introduced. This jitter, computed at the transmitting ES, is the time between the beginning of BAG interval and the first bit of the frame to be transmitted in that BAG. This jitter must not be greater than 500µs.

NoC Architecture and Assumptions

In this paper, we consider a Tilera-like NoC architecture, i.e. a 2D-Mesh NoC with bidirectional links interconnecting a number of routers. Each router has five input and output ports. Each input port consists of a single queuing buffer. The routers at the edge of the NoC are interconnected to the DDR memory located north and south of the NoC via dedicated ports. The first and last columns of the NoC are not connected directly to the DDR. Besides, the routers at the east side connects the cores to the Ethernet interfaces via specific ports. Many applications can be allocated on a NoC. Each application is composed of a number of tasks, where one core executes only one task. These tasks do not only communicate with each other (core-to-core flows), but also with the I/O interfaces, i.e. the DDR memory and Ethernet interfaces (core-to-I/O flows). These flows are transmitted through the NoC following wormhole routing [START_REF] Mohapatra | Wormhole routing techniques for directly connected muti-computer systems[END_REF], an XY policy and a Round-Robin arbitration. Besides, a credit-based mechanism is applied to control the flows. A flow consists of a number of packets, corresponding to the maximal authorized flow size on the NoC. Indeed, a packet is divided into a set of flits (flow control digits) of fixed size (typically 32-bits). The maximal size of a NoC packet is of 19 flits as in Tilera NoC. The wormhole routing makes the flits follow the first flit of the packet in a pipeline way creating a worm where flits are distributed on many routers. The credit-based mechanism blocks the flits before a buffer overflow occurs. The consequence of such a transmission model is that when two flows share the same path, if one of them is blocked, the other one can also be blocked. Thus, the delay of a flow can increase due to contentions on the NoC. The Worst-case Traversal Time (WCTT) of a flow can be computed using different methods proposed in the literature [START_REF] Ayed | Towards a unified approach for worst-case analysis of Tilera-like and KalRay-like NoC architectures[END_REF][START_REF] Ferrandiz | Using Network Calculus to compute end-to-end delays in SpaceWire networks[END_REF]. In this paper, we choose RC N oC [START_REF] Abdallah | Wormhole networks properties and their use for optimizing worst case delay analysis of many-cores[END_REF] to compute the WCTT as it leads to tightest bounds of delays compared to the existing methods on a Tilera-like NoC. This method considers the pipeline transmission, and thus computes the maximal blocking delay a flow can suffer due the contentions with blocking flows.

PROBLEM STATEMENT

Actually, an outgoing I/O flow is transmitted following three steps:

(1) A core sends data to the nearest port of DDR memory, (2) then it sends a DMA command to the Ethernet interface on a separate network. This DMA command indicates that all the data are in the DDR memory. This command is stored into a DMA command FIFO queue. (3) When the Ethernet interface executes the DMA command, data packets are sent from the same port of DDR memory to the same Ethernet interface. The packets of an outgoing I/O flow will incur a contention with different types of communications on the NoC which could lead to a jitter.

Let us illustrate the delays of these steps with the example in Figure 3. Two VLs VL 1 and VL 2 are respectively generated by tasks t1 DDR and t2 DDR . At the beginning of VL 1 first BAG period, t1 DDR sends VL 1 data to the nearest port of DDR memory. This transmission can take up to WCTT t oDDR . Step 2 (transmission of the DMA command to the Ethernet interface) is done after this worst-case delay. Thus step 1 duration is constant and does not generate any jitter. Similarly, step 2 duration is assumed to be constant, since the DMA command is sent on a separate network. Thus all the jitter comes from step 3 (transmission of the data from DDR memory to the Ethernet interface). Considering VL 1 first BAG period in Figure 3, the jitter is the delay d1 of this transmission, which is between 0 and its worst-case duration. The jitter can be much higher. Indeed, for VL 1 second BAG period, the Ethernet interface is busy with VL 2 when it receives VL 1 DMA command. The delay due to VL 2 has to be added to the jitter, leading to an overall value of d

+ d V L2 + d2.
The goal is to avoid that the Ethernet interface is busy when it receives a DMA command (like for VL 1 second period in Figure 3).

A DEDICATED CORE FOR OUTGOING I/O FLOWS

We propose to dedicate a specific core of the NoC to the transmission of VL through the Ethernet interface. The idea is to execute only one task t DDR for all the applications. This task can be executed on a core of the NoC that do not execute any other task. The transmission of the command of the DDR uses another internal network and does not impact the communications of other tasks. The behaviour of the task t DDR is as follow:

(1) Reception of a message from the final task of the function: the data that need to be sent are in the DDR. In such a way, we do not change the behaviour of the functions presented in section 2.2. The transmission of the message is constant and can be considered as a part of the execution time of the sending task of the function. (2) Transmission of a DDR command to the DDR. The corresponding data are then transmitted from the DDR to the Ethernet interface.

In this paper, we consider that the schedule of the DDR commands, and so the VLs sent by the functions, can be done by using a scheduling table.

A scheduling table

The goal of this method is to reduce the jitter induced by the transmission of other VLs from the memory to the Ethernet interface through the NoC and the transmission of these VLs through the Ethernet interface. The considered scheduling table is composed of slots of 31.25µs. The global duration of the table is 128 ms. So the number of slots is 4096. The table is composed of 128 lines of 1 ms, each line contains 32 slots. A set of slots is allocated to each VL sent by the applications by considering the BAG duration: a VL will obtain a slot at exactly each BAG. Such a scheduling is represented in Table 1. In this example, we denoted by the name of the application the slot when the corresponding VL should be sent. As an example, a VL from the application App 1 has a BAG duration equal to 2 ms. It can be located in column 1 of lines 0, 2, . . . , 126. In the same manner, VL from the application App 4 has a BAG of 8 ms and is located in column 10 of lines 1, 9, 17, . . . , 121.

For each application App i which send a VL i , the number of allocated slots ω i is defined as:

ω i = WCTT(VL i ) + ft(VL i ) sd ( 1
)
where WCTT(VL i ) is the WCTT of VL i from the DDR to the Ethernet interface, ft(VL i ) is the transmission delay of the frame through the Ethernet interface and sd = 31.25µs is the slot duration. The WCTT(VL i ) depends on the contention on the NoC, while ft(VL i ) depends on the size of the frame that is transmitted by the Ethernet interface. For example, if we consider that the WCTT of VL 2 , sent by App 2 , from the memory to the Ethernet interface is 30 µs and the transmission of the VL 2 by the interface needs 120 µs, the resulting value of ω 2 is 5 as depicted in Table 1. This scheduling guarantees that the VLs are sent from the memory to the Ethernet interface at different times, leading to a reduction of the jitter induced by the transmission of other VLs. In such a way, the only delay that a VL can suffer when it is transmitted from the DDR is due to the interferences from the transmission of internal communications through the NoC that share the same path as the VL.

The goal is now to define this scheduling table, i.e. the allocation of the slots to the applications.

Allocation of the slots to applications

We propose an allocation solution with two objectives:

(1) The main objective of the allocation of the slots is to guarantee that the VLs sent by the applications respect their BAG constraints. The approach needs to guarantee that an application can send a data at exactly every BAG period and will find the next transmission slot at exactly one BAG from the We have then formulated these objectives into an ILP. The proposed solution is a sequence per line of slots allocated to the transmission of VLs that is repeated every n ms, with n = 1, 2, 4, . . . The mathematical definition is given in the following.

Formulation.

With the notations in the table 2, we consider m different possible applications and their associated BAG, the scheduling table provides free slots that can be addressed for applications.

We have to treat the special case of BAG equal to 1ms because it impacts all the lines of the scheduling table and so the capacity of each line is reduced by the number of slots dedicated to the

C = 32 - j ∈J ω j , (2)
where J = {j ∈ {1, .., m}|BAG j = 1}. The capacity C should be in [|0,32|]. So negative values will indicate that the configuration cannot ensure the transmission.

From the computation of the real capacity of each BAG by taking into account the 1ms period of application, the allocation of each application can be formulated as an Integer Linear Program (ILP) called Bin Packing Problem [START_REF] Martello | Knapsack Problems[END_REF]. The bin packing problem can be described informally as follows: by considering m applications, each having an integer number of slots ω j (j=1,..,m) and a limited number of lines N of integer capacity C. The objective is to pack all the applications into the minimum number of lines in the table so that the total number of slots per application packed in any line of the table does not exceed its capacity. This will provide an optimal way to assign slots in the scheduling table by the applications and thus the filling of the table will permit transmitting applications with long BAG several times within its own BAG.

We have to fix the limited number of lines N to allocate all the applications. In fact, in order to repeat the most the allocations in the table, we take into account all the BAG j (j=1..m), once. So the number N is equal to the minimum BAG among all the applications except 1ms (which are already considered by equation ( 1)) and is defined as follows:

N = min j=1...m, BAG j 1 BAG j . (3)
Formally, let's introduce two binary decision variables y i for i ∈ {1, .., N } which indicates the considered line of the table and x i j , for i ∈ {1, .., N } and j ∈ {1, .., m} which indicates the place of the application j in the line i of the table as defined in the table 2. From these notations, as the objective is to pack all the applications into the minimum number of lines in the scheduling table, the objective function will be formulated as: min N i=1 y i defined in Table 2. This function has to be minimized so that the total number of slots per application, noted m j=1 ω j x i j for a line i ∈ {1, .., N }, packed in any line i ∈ {1, ..N } of the table does not exceed its capacity C and this constraint can be expressed for all the lines i ∈ {1, .., N } as:

m j=1 ω j x i j ≤ C y i , ∀i = 1, .., N .
Finally, as the aim is to fill the table by considering all the m applications, this leads to define these following constraints:

N i=1 x i j = 1, ∀j = 1, .., m.
Thus, we can formulate the problem of filling the scheduling table by applications as the following ILP problem [START_REF] Kang | Algorithms the variable sized bin packing problem[END_REF]:

min N i=1 y i (4) s.t m j=1 ω j x i j ≤ C y i , ∀i = 1, .., N (5)
N i=1

x i j = 1, ∀j = 1, .., m (6)
y i ∈ {0, 1} , ∀i = 1, .., N (7)
x i j ∈ {0, 1} , ∀i = 1, .., N , ∀j = 1, .., m. (8)
Objective function [START_REF] Abdallah | Wormhole networks properties and their use for optimizing worst case delay analysis of many-cores[END_REF] means that the applications should be stored in a minimum of lines of table N defined by equation [START_REF] Abdallah | Towards a mixed NoC/AFDX architecture for avionics applications[END_REF]. Constraints (4) impose that the capacity of each line defined by equation ( 1) is not exceeded while constraints [START_REF] Ayed | Towards a unified approach for worst-case analysis of Tilera-like and KalRay-like NoC architectures[END_REF] ensure that all the applications are packed. This ILP problem is solved by using CPLEX [START_REF]V12. 1: User's Manual for CPLEX[END_REF].

Example.

Let's consider the application configuration of Table 1. It is composed of 6 applications. App 2 is the first mapped into the table as the BAG is equal to 1ms. With equation (1), the capacity C is equal to 27 free slots. Equation ( 2) computes the number of lines in which the applications has to be allocated is 2. The resulting scheduling table is given in Table 3. As all the applications can be packed in two lines, lines 0 and 1 are repeated in the table. So, applications App 3 , App 4 , App 5 and App 6 , for which

CASE STUDY

In this section, we evaluate the approach to realistic case studies.

Considered applications

The considered case study is composed of critical and non critical applications:

• Full Authority Digital Engine (FADEC) application: It controls the performance of the aircraft engine. It receives 30 KBytes of data from the engine sensors via an Ethernet interface and sends back 1500 Bytes of data to the engine actuators. The application FADEC n is composed of n tasks denoted t f 0 to t f n-1 . t f n-1 is dedicated to send the commands to the engine actuators via the Ethernet interface. Except t f n-1 , all other tasks exchange 5 KBytes of data through the NoC. They also send 5 KBytes of data to t f n-1 . Figure 4a shows the tasks graph of FADEC 7 . This graph illustrates the core-to-core and core-to-I/O communications between the tasks of the FADEC application. • Health Monitoring (HM) application: It is used to recognize incipient failure conditions of engines. It receives through an Ethernet interface, a set of frames of size 130 KBytes and sends back 1500 bytes of data actuators. The application HM n is composed of n tasks, denoted t h0 to t hn-1 . The last task t hn-1 is dedicated to send the data actuators to the Ethernet interface. The task t hi sends 2240 bytes of data to t hi+1 through the NoC, with i ∈ [0, n -2]. All these tasks finish their processing by storing their frames into the memory. Figure 4b shows the tasks graph of HM 6 .

FADEC applications are critical, while HM applications are noncritical. In this case study, we consider 2 configurations: one composed of 8 applications (3 FADEC and 5 HM) and one composed of 9 applications (3 FADEC and 6 HM). BAGs of the sent data by these applications are indicated in Table 4. HM 7 is the added application between the two configurations.

Mapping applications into the manycore

The described applications are the mapped into the many-core. Different strategies can be used such as:

• Smart Hill Climbing (SHiC) [START_REF] Fattah | Smart hill climbing for agile dynamic mapping in many-core systems[END_REF]: this approach maps the applications without fragmented regions, generated by the methods in the literature (as in [START_REF] Luiz De Souza Carvalho | Dynamic task mapping for MPSoCs[END_REF], [START_REF] Zimmer | Low contention mapping of real-time tasks onto tilepro 64 core processors[END_REF], [START_REF] Fattah | CoNA: Dynamic application mapping for congestion reduction in many-core systems[END_REF]). This method searches a region of size equal to the size of the application to be allocated. The tasks of this application are allocated in the selected region in such a way to reduce the distance between the communicated tasks. Thus, SHiC allocates the task with the maximum number of communications at the center of this region and around it the tasks communicating with it to form a square shape. • Map IO [START_REF] Abdallah | Reducing the contention experienced by real-time core-to-I/O flows over a Tilera-like Network on Chip[END_REF]: this approach performs the mapping into two steps. The first step splits the NoC into regions and then allocates primarily critical applications in a dedicated region close to memory and Ethernet controllers by following a circular direction and using rectangular shapes. The second step consists to allocate the tasks within each application where some rules are used to minimize the contentions on the path of the core-to-I/O flows. These rules are based on the principle of allocating the tasks which generate perpendicular communications on the path of the core-to-I/O flows. • ex_Map IO [START_REF] Abdallah | Towards a mixed NoC/AFDX architecture for avionics applications[END_REF]: this approach is an extension of Map IO in order to minimize the delay of outgoing I/O flows on the NoC (such as d, d1 and d2 in Figure 3). Several mapping rules have been defined. As an example, one rule minimizes the number of flows that can delay an outgoing flow on its path to the Ethernet interface.

As an example, Figure 5 shows the mapping of the two configurations (8 applications and 9 applications) on a 10x10 NoC using ex_Map IO .

The number of occupied cores is 89 for an 8 applications configuration and 96 for a 9 applications configuration. This means that there are 11 free cores for the first configuration and 4 in the other one, as we can see in Figure 5. These free cores can execute a t DDR . As an example, t DDR can be located in column 2 of line 4 in Figure 5a and can be located in column 3 of line 3 in Figure 5b.

Once the applications are mapped, we can compute the WCTT of the transmission of the VLs from the memory to the Ethernet interface. The results for the two considered configurations are given in Table 4. In this case study, we consider that for each application a maximum size frame is sent through the AFDX network. So, the frame transmission duration is 123 µs for all the applications of the case study. The number of slots allocated for each application can be computed and is given in Table 4, by considering Equation ??. Note that there are no value for SHiC considering 9 applications since mapping 9 applications using SHiC is not possible.

Obtained scheduling table

Figure 6 shows the resulting scheduling tables for the different mapping approaches. As explain in section 4, the number of lines in which all the applications have to be allocated is 2. This is why we only represent the 2 first lines of the scheduling tables in Figure 6. The oversampling of slots is then 2ms. This means that applications with a BAG greater than 2ms will get a transmission slot every 2ms. If they miss the first transmission slot, they will have to wait a maximum of 2ms before the data will be transmitted.

Results

The results are shown in Table 5. Distributed t DDR corresponds to the solution described in Section 3. In this case, SHiC can not respect the jitter constraint while Map IO and ex_Map IO allows a reduction of the jitter leading to a jitter value less than 500µs for an 8 applications mapping. But, the jitter is still greater than 500µs when mapping 9 applications.

As expected, the solution proposed in the paper significantly reduces the jitter of the VLs. The jitter becomes lower than 200µs when mapping both 8 and 9 applications on the NoC for all the mapping strategies.

CONCLUSION

In this paper, we proposed to replace the mono-core processors in avionics architecture by a NoC-based many-cores architecture. Thus, End Systems are replaced by many-cores.

The main contribution of the paper is that it proposes a new VL transmission strategy which considers one dedicated node in the many-core architecture to shape the traffic and schedules the outgoing I/O flows. This dedicated node executes a scheduling table. The applications are allocated into the table using an ILP formulation. The two objectives of the strategy is to guarantee the BAG regulation and to oversample the slots to applications with a BAG greater than 2ms.

The results show that the jitter is significantly reduced and only depends on the interferences that can occur on the transmission path between the DDR and the I/O interface.

The ILP formulation imposes a strong constraint that can be relaxed: all the applications should be stored in the minimum period (different to 1ms). As an example, we can notice that it remains a very little amount of free slots in Figures 6d and6e (respectively 3 and 6). But applications with longer BAG can be allocated after this minimum period. This leads to consider variable capacity size and to formulate a variant of bin packing [START_REF] Kang | Algorithms the variable sized bin packing problem[END_REF] or a cutting stock problem with periodic items and thus defining a more complex strategy for filling the scheduling table.

The worst case transmission delay of a VL from one application from a many-core to another application executed on another manycore should be mastered in order to certify the avionic system. AFDX network uses Network Calculus to compute the maximum worst case delay [START_REF] Charara | Methods for bounding end-to-end delays on an AFDX network[END_REF]. For a NoC used in processors like Tilera, the method is Recursive Calculus [START_REF] Ayed | Towards a unified approach for worst-case analysis of Tilera-like and KalRay-like NoC architectures[END_REF]. The problem that could be addressed now is how to compute the worst case end-to-end delay of the global communication.

Figure 1 :

 1 Figure 1: An AFDX network.

Figure 2 :

 2 Figure 2: A mixed NoC/AFDX architecture

Figure 3 :

 3 Figure 3: A possible transmission on a given VL.

Figure 4 :

 4 Figure 4: Task graph of core-to-core and core-to-I/O communications of the: (a) FADEC application, (b) HM application.

Table 1 :

 1 Scheduling table of the VL i sent by each application App i (VLs are represented by their sender applications). current transmission. For example, if App 1 starts its transmission at slot located in column 5 of line 2, then App 1 can send a new data 2ms (the BAG of VL 1 is 2ms) later, at slot located in column 5 of line 4, as depicted in Table 1. (2) In Table 1, App 6 can send VL 6 at column 20 in line 15. Considering the scheduling and the execution of App 6 in the many-core, VL 6 can be ready to be sent at 100µs, for example. So, the transmission of this VL through the Ethernet interface will start 15.5 ms later (15 lines of the table is 15

	1

ms, 16 slots of 31.2µs is 0.5 ms). Thus, we propose to reduce this delay by allocating more transmission slots for each VLs. Our second objective is then to allocate slots every n ms for VLs, with n = 1, 2, 4, . . . In our approach, we propose to fix n to the minimum value of the BAG of the VLs sent by the applications. This is what we call oversampling.

Table 2 :

 2 Notations for Bin Packing Problem applications. So the capacity, noted C, in terms of free slots for each line of the scheduling table is defined as follows:

		Constants
	m	Number of applications
	BAG j	period of the application j noted App j
	ω j	number of slots for application j
	C	Capacity of each line of scheduling table by removing 1ms BAG
	N	Number maximum of lines considered for allocation
		Boolean Variables
	y i	equal 1 if line i of the table is used in the solution, 0 otherwise
	x i j	equal 1 if application j is packed into line i of the table, 0 otherwise
		Objective Function
	N
	y i	minimize the number of lines to pack all the applications
	i=1

Table 3 :

 3 Scheduling table obtained by the mapping method.

	1

Table 4 :

 4 Figure 5: Mapping 8 (left) and 9 (right) applications on a 10x10 many-core using ex_Map I O . WCTT on the NoC of the transmission of VL flows of each application from the memory to the Ethernet interface, (in µs)

		port 18	port 13 port 14 port 15 port 16 port 17	port 11 port 12
	t h3 t h5		t h15	t f11	t f10	t f7	t f5 t f2	t f0	t f1	10
	t h13	t h12		t h14	t f12	t f9	t f8	t f4 t f3	FADEC 13 t f6	9
	t h11	t h10		t h2	t h3	t h2	t h1 t h0	t h9	FADEC 7 t f2 t f6	8
	t h9 t h8 t h6 t h7 t h4 t h9	t h1 t h0 HM 16 t h0	t h4 t h1 HM 9 t h3	t h5 t h6 t h2	t h6 t h7 t h0 FADEC 11 t h8 HM 10 t f1 t h7 t f8 t f9	t f3 t f4 t f5	t f0 t f1	(0,5)
	t h4	t h3		t h1	t h5	t h4	t h8	t f7	t f10	t DDR
	t h6	t h5		t h10	t h4	t h5	t h6 HM 11	t f4	t f5
	t h8 t h7		t h11	t h7	t h8	t h9	t f3	t f2
	t h2 HM 12	t h3		t h10	t h2	t h1	t h0	t f0	t f6
		port 8		port 7	port 6
	10	9		8

Table 5 :

 5 Maximum jitter experienced by the transmission of VL, in µs


Shaping the spatial and spectral phase with reflectives nanoresonators

Boher Micke 1 , Julien Jaeck *1 , Patrick Bouchon 1 , Jérôme Primot 1 and Riad Haïdar 1,2 1 DOTA,ONERA,Université Paris Saclay F-91123 Palaiseau -France 2 Ecole Polytechnique, Département de Physique, 91128 Palaiseau, France, * micke.boher@onera.fr

Since the work of Federico Capasso in 2011 [START_REF] Yu | Light propagation with phase discontinuities: Generalized laws of reection and refraction[END_REF], metasurfaces for optics, or how to structure a surface to provide optical properties (focusing, deflection, dispersion, etc.), have been proposed for numerous applications. The function of an optical component is determined by the local modification it induces on an incident electromagnetic field. The goal is to control the phase of a transmitted or reflected field by texturing the matter at the sub-wavelength scale and minimizing the influence on the amplitude (at least in the spectral band of interest).

A solution to this problem is the use of MIM (Metal-Insulator-Metal) nano-antennas [START_REF] Zhang | High efficiency near diffraction-limited mid-infrared flat lenses based on metasurface reflectarrays[END_REF], which consists in metallic rods deposited on a layer of transparent material itself deposited on a metal mirror. Such a structure can only work in reflection because of the metallic mirror and behaves like an horizontal Fabry-Perot resonator whose resonance is controlled mainly by the length of the rod and which is accompanied by a significant phase jump up to 2π. Here we use two properties of these structures, the chromaticism induced by the resonance and the interaction surface which is greater than the geometrical surface of the nano-rod [START_REF] Bouchon | Wideband omnidirectional infrared absorber with a patchwork of plasmonic nanoantennas[END_REF], to combine the different dimensions of nano-antennas locally in order to create two optical functions on the same surface. Our proposed structure is illustrated in figure (a), it deflects the incident beam at 3µm in one direction and the one at 5µm in the opposite direction. This prism is composed of four periodic areas, each of them adding a π/2 delay to the previous one by modifying the length of antennas. As shown in figure (b) these nano-antennas create two opposite phase ramps at two different wavelengths. This is possible thanks to the chromaticism of MIM structures and the fact that they interact with the light included in an effective section much larger than their dimensions.

As we can see on figure (c) the phase response of a given structure varies with the wavelength. We use the fact that when the resonance is localized at 5µm (respectively at 3µm) the introduced phase at 3µm (respectively at 5µm) is negligible. That is to say that the three smallest patterns will modify locally the phases at 3µm only and the three greatest patterns will do the same for 5µm only. This dual prism paves the way to the creation of more complicated dual optical functions metasurfaces.

The authors acknowledge the financial support from a DGA-MRIS scholarship.

Figure

  Figure. (a) Illustration of the proposed dual prism. (b) Illustration of one period of the dual prism and associated phase ramps. (c) Phase response of the nano-antennas according to their lengths in function of the wavelength. The colors correspond to the ones of previous figure.


G Fabbris
email: gfabbris@anl.gov

N Jaouen

D Meyers

J Feng

J D Hoffman

R Sutarto

S G Chiuzbȃian

A Bhattacharya

M P M Dean
email: mdean@bnl.gov

Emergent c-axis magnetic helix in manganite-nickelate superlattices

The nature of the magnetic order in (La 2/3 Sr 1/3 MnO3)9/(LaNiO3)3 superlattices is investigated using x-ray resonant magnetic reflectometry. We observe a new c-axis magnetic helix state in the (LaNiO3)3 layers that had never been reported in nickelates, and which mediates the ∼ 130 • magnetic coupling between the ferromagnetic (La 2/3 Sr 1/3 MnO3)9 layers, illustrating the power of x-rays for discovering the magnetic state of complex oxide interfaces. Resonant inelastic x-ray scattering and x-ray absorption spectroscopy show that Ni-O ligand hole states from bulk LaNiO3 are mostly filled due to interfacial electron transfer from Mn, driving the Ni orbitals closer to an atomic-like 3d 8 configuration. We discuss the constraints imposed by this electronic configuration to the microscopic origin of the observed magnetic structure. The presence of a magnetic helix in (La 2/3 Sr 1/3 MnO3)9/(LaNiO3)3 is crucial for modeling the potential spintronic functionality of this system and may be important for designing emergent magnetism in novel devices in general.

Interfaces between complex oxide materials exhibit remarkably rich physics driven by the interplay of various types of charge, orbital and spin couplings [1][START_REF] Bhattacharya | Magnetic Ox-ide Heterostructures[END_REF][START_REF] Hellman | Interface-induced phenomena in magnetism[END_REF]. Particularly fascinating is their proven ability to host new types of emergent magnetic order that do not exist in either of the bulk constituents, representing a challenge to our understanding of electron correlations, as well as an opportunity for exploitation in spintronic devices [START_REF] Bhattacharya | Magnetic Ox-ide Heterostructures[END_REF][START_REF] Hellman | Interface-induced phenomena in magnetism[END_REF][START_REF] Hoffman | Tunable Noncollinear Antiferromagnetic Resistive Memory through Oxide Superlattice Design[END_REF]. In fact, complex magnetic phases are potentially common in transition metal oxide superlattices given the severe effects of interfacial symmetry breaking in localized 3d orbitals [START_REF] Banerjee | Ferromagnetic exchange, spin-orbit coupling and spiral magnetism at the LaAlO3/SrTiO3 interface[END_REF][START_REF] Li | Spirals and Skyrmions in Two Dimensional Oxide Heterostructures[END_REF][START_REF] Fust | Realizing topological stability of magnetic helices in exchange-coupled multilayers for all-spinbased system[END_REF][START_REF] Moetakef | Carrier-Controlled Ferromagnetism in SrTiO3[END_REF][START_REF] Brinkman | Magnetic effects at the interface between non-magnetic oxides[END_REF], but scrutiny over the microscopic magnetic interactions at such interfaces is often constrained by limited direct evidence of the resulting magnetic structure. Superlattices composed of manganites and nickelates are a paradigmatic venue for efforts to discover and understand emergent interface magnetism, motivated by the complex magnetic order and phase diagram of its bulk constituents [START_REF] Zener | Interaction between the d-Shells in the Transition Metals. II. Ferromagnetic Compounds of Manganese with Perovskite Structure[END_REF][START_REF] Torrance | Systematic study of insulator-metal transitions in perovskites RNiO3 (R=Pr,Nd,Sm,Eu) due to closing of charge-transfer gap[END_REF]. In fact, this system harbors fascinating phenomena, such as exchange bias and interfacial electronic reconstructions [START_REF] Nikolaev | Indications of antiferromagnetic interlayer coupling in La 2/3 Ba 1/3 MnO3/LaNiO3 multilayers[END_REF][START_REF] Nikolaev | Oscillatory Exchange Coupling and Positive Magnetoresistance in Epitaxial Oxide Heterostructures[END_REF][START_REF] Rojas Sánchez | Exchange-bias effect at La0.75Sr0.25MnO3/LaNiO3 interfaces[END_REF][START_REF] Gibert | Exchange bias in LaNiO3 -LaMnO3 superlattices[END_REF][START_REF] Hoffman | Charge transfer and interfacial magnetism in (LaNiO3)n/(LaMnO3)2 superlattices[END_REF][START_REF] Taekyung | Charge transfer, confinement, and ferromagnetism in LaMnO3/LaNiO3 (001) superlattices[END_REF][START_REF] Gibert | Interfacial Control of Magnetic Properties at LaMnO3/LaNiO3 Interfaces[END_REF][START_REF] Piamonteze | Interfacial properties of LaMnO3/LaNiO3 superlattices grown along (001) and (111) orientations[END_REF][START_REF] Ning | Fermi Level shifting, Charge Transfer and Induced Magnetic Coupling at La0.7Ca0.3MnO3/LaNiO3 Interface[END_REF][START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF][START_REF] Gibert | Interlayer coupling through a dimensionality-induced magnetic state[END_REF][START_REF] Kitamura | Spatial distribution of transferred charges across the heterointerface between perovskite transition metal oxides LaNiO3 and LaMnO3[END_REF][START_REF] Zang | The Exchange Bias of LaMnO3/LaNiO3 Superlattices Grown along Different Orientations[END_REF][START_REF] Flint | Tuning interfacial ferromagnetism in LaNiO3/CaMnO3 superlattices by stabilizing nonequilibrium crystal symmetry[END_REF][START_REF] Flint | Role of polar compensation in interfacial ferromagnetism of LaNiO3 superlattices[END_REF]. Recently, a highly unusual magnetic coupling between ferromagnetic (LSMO) 9 layers was observed in [001]-oriented (La 2/3 Sr 1/3 MnO 3 ) 9 /(LaNiO 3 ) n [(LSMO) 9 /(LNO) n , n = 1 -9], in which the coupling angle between (LSMO) 9 layers varies between zero and 130 • as a function of n (Fig. 1) [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. Despite the demonstrated ability of this system to be used as resistive memory devices [START_REF] Hoffman | Tunable Noncollinear Antiferromagnetic Resistive Memory through Oxide Superlattice Design[END_REF], there is no experimental evidence for how or if the non-collinear magnetism of the (LSMO) 9 layers is mediated through (LNO) n . Such lack of information critically hampers the ability to understand the physical mechanism driving the (LSMO) 9 magnetic coupling, which consequently inhibits the design of new superlattices with optimized magnetic properties.

In this work we focus on the (LSMO) 9 /(LNO) 3 superlattice because it displays the largest magnetic coupling angle between the manganite layers (≈ 130 • at 25 K [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]). We probe its magnetic and electronic structure using resonant soft x-ray techniques. The magnetic structure of both (LSMO) 9 and (LNO) 3 are studied using Mn and Ni L-edge x-ray resonant magnetic reflectivity (XRMR). We find that the nickelate layers contain a magnetic helix in which the Ni moments are confined to the basal plane and rotate around the c-axis. Such magnetic structure is not present in any simple perovskite nickelate nor manganite. The XRMR ability to provide a precise description of the (LSMO) 9 /(LNO) 3 magnetic ordering allows for a careful analysis of the magnetic interactions in this system. To this end, the nature of the Ni 3d and O 2p orbitals is probed through Ni L 3 -edge resonant inelastic x-ray scattering (RIXS) and O K-edge x-ray absorption spectroscopy (XAS) measurements. The RIXS spectra are composed of well defined orbital excitations, which are reproduced by a model of localized Ni 3d 8 orbitals under the effect of an octahedral crystal field. Even though the Ni 3d orbital is predominantly localized, O K-edge XAS show a small Ni 3d -O 2p ligand hole concentration, indicating that the NiO 2 planes are lightly hole-doped. While further work is needed to elucidate the magnetic couplings acting in this system, our analysis suggests that the (LSMO) 9 /(LNO) 3 magnetic helix emerges from a combination between the strong pinning of Ni and Mn moments at the interfaces with a spin density wave (SDW) instability within the (LNO) 3 layers.

High-quality 14×[(LSMO) 9 /(LNO) 3 ] superlattices were grown on [001] single-crystalline SrTiO 3 substrates using molecular beam epitaxy as described previously [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]27]. XRMR measurements were performed at the Ni and Mn L-edges [27] at the REIXS beamline of the Canadian Light Source [START_REF] Hawthorn | An in-vacuum diffractometer for resonant elastic soft x-ray scattering[END_REF] and at the SEXTANTS beamline of the SOLEIL synchrotron [START_REF] Jaouen | An apparatus for temperaturedependent soft X-ray resonant magnetic scattering[END_REF]. Data were collected using horizontal scattering geometry, as well as both linear (σ and π) and circular [left (CL) and right (CR)] x-ray polarizations (Fig. 1). Magnetic field was applied parallel to the sample surface and within the scattering plane using permanent magnets at REIXS and an electromagnet at SEXTANTS. In the former the data was collected at remanence after removing a 600 mT field, while in the later the sample was field cooled, and measured, under 1.2 mT. The temperature was kept at ∼25 K throughout the experiments. Data analysis was performed with the Dyna [START_REF] Elzo | X-ray resonant magnetic reflectivity of stratified magnetic structures: Eigenwave formalism and application to a W/Fe/W trilayer[END_REF] and ReMagX [START_REF] Macke | Magnetic reflectometry of heterostructures[END_REF] softwares using the magnetic matrix formalism. Resonant optical constants were obtained by combining the tabulated values with averaged XAS and x-ray magnetic circular dichroism measurements [27,[START_REF] Chantler | Theoretical Form Factor, Attenuation, and Scattering Tabulation for Z =192 from E =110 eV to E =0.41.0 MeV[END_REF] Figure 2 displays a summary of the XRMR data col-lected on (LSMO) 9 /(LNO) 3 . The XRMR signal measured with circular light is primarily sensitive to components of the magnetization of each layer along the xray direction, while π polarization probes the transverse magnetization, and σ polarization is dominated by the lattice response [START_REF] Elzo | X-ray resonant magnetic reflectivity of stratified magnetic structures: Eigenwave formalism and application to a W/Fe/W trilayer[END_REF][START_REF] Macke | Magnetic reflectometry of heterostructures[END_REF]. It is thus clear in Figs. 2 (a) through (d) that magnetic peaks appear at 1/2 order positions in (LSMO) 9 /(LNO) 3 at both Mn and Ni edges indicating that the magnetic structure unit cell along the c-axis has twice the length of the chemical repetition. This observation is further demonstrated by the disappearance of such magnetic peaks when a field of 0.6 T is applied, which is sufficient to saturate the magnetic moments along the field [Fig. 2(e)] [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. While the Mn edge results are consistent with the previous report [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF], the data at the Ni edge indicates that a non- L3-edge (641 eV) XRMR data using circular and linear polarization respectively. Data from both edges show half-order magnetic peaks (black arrows). This is consistent with a noncollinear magnetic structure between both LNO layers and LSMO layers as shown in Fig. 1. (e) Field dependence of the Ni L3 XRMR data. A field of 0.6 T is sufficient to force a ferromagnetic alignment between every layer, which suppresses the half-order peaks, consistent with previous results [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. collinear magnetic structure is also present in the (LNO) 3 layer. These results also demonstrate that the magnetic coupling within NiO 2 planes is not antiferromagnetic as this would lead to the absence of magnetic response in the CR/CL XRMR. Consequently, the magnetic structure of (LNO) 3 is different from the ( 1 4 1 4 1 4 )-type observed in [111]-grown LNO heterostructures and bulk RENiO 3 [START_REF] Torrance | Systematic study of insulator-metal transitions in perovskites RNiO3 (R=Pr,Nd,Sm,Eu) due to closing of charge-transfer gap[END_REF][START_REF] Gibert | Interlayer coupling through a dimensionality-induced magnetic state[END_REF]33]. However, this in itself does not distinguish between other (LNO) 3 magnetic structures such as the zigzag and (reversed) helical models (Fig. 1).

The (LNO) 3 magnetic structure is solved by modeling the XRMR data as shown in Fig. 3. Note that fits were performed at the Ni L 2 edge because there is a substantial overlap between the La M 4 and Ni L 3 edges, which makes the XRMR data analysis very difficult. Despite its magnetic sensitivity, the XRMR signal is largely dominated by the lattice and charge responses [Fig. 3(a)-(d)]. In fact, the disagreements between simulation and experimental data seen at the Mn L 3 -edge are at least par-tially driven by the expected Mn 4+/3+ valence modulation within the (LSMO) 9 layer [START_REF] Hoffman | Charge transfer and interfacial magnetism in (LaNiO3)n/(LaMnO3)2 superlattices[END_REF][START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF][START_REF] Zwiebler | Electronic depth profiles with atomic layer resolution from resonant soft x-ray reflectivity[END_REF]. We can overcome this challenge by isolating the magnetic response using the XRMR circular polarization asymmetry shown in Fig. 3(e) [START_REF]The XRMR assymetry is defined here as the difference between right and left polarized XRMR times q 4 z[END_REF]. It is clear that only the helical model is able to reproduce all features in the data, particularly at high-q z . Magnetic models with LNO spins away from the basal plane are not shown here but were also unable to reproduce the experimental data. Modeling of both Mn and Ni edges is consistent with an angle of 140 ± 20 • between (LSMO) 9 magnetic moments, in agreement with neutron reflectivity [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. The (LNO) 3 c-axis magnetic helix result is surprising as such structures are uncommon in bulk transition metal oxides and have not previously been observed in nickelates. This demonstrates that in (LSMO) 9 /(LNO) 3 the interfacial symmetry breaking and charge transfer drive an emergent form of magnetism, the underlying electronic structure of which we now explore.

We first address the character of the nickelate layer 3d orbital using Ni L 3 -edge RIXS. Data was collected at 30 K using the AERHA end-station of the SEXTANTS beamline at the SOLEIL synchrotron [START_REF] Sorin | Design and performance of AERHA, a high acceptance high resolution soft x-ray spectrometer[END_REF]. The experimental scattering geometry is detailed in Fig. 1. Figure 4 shows the RIXS spectra as a function of incident x-ray energy and linear polarization. The experimental data [Fig. 4 consistent with localized orbital excitations of 3d 8 (Ni 2+ ) ions [START_REF] Fabbris | Orbital engineering in nickelate heterostructures driven by anisotropic oxygen hybridization rather than orbital energy levels[END_REF][START_REF] Ghiringhelli | NiO as a test case for high resolution resonant inelastic soft x-ray scattering[END_REF][START_REF] Ghiringhelli | Observation of Two Nondispersive Magnetic Excitations in NiO by Resonant Inelastic Soft-X-Ray Scattering[END_REF][START_REF] Fabbris | Doping Dependence of Collective Spin and Orbital Excitations in the Spin-1 Quantum Antiferromagnet La2-xSrxNiO4 Observed by X Rays[END_REF]; a distinctively different broad diagonal feature is seen in RIXS spectra of bulk RENiO 3 and holedoped La 2-x Sr x NiO 4 (for x ≥ 1/3) [START_REF] Fabbris | Orbital engineering in nickelate heterostructures driven by anisotropic oxygen hybridization rather than orbital energy levels[END_REF][START_REF] Fabbris | Doping Dependence of Collective Spin and Orbital Excitations in the Spin-1 Quantum Antiferromagnet La2-xSrxNiO4 Observed by X Rays[END_REF][START_REF] Bisogni | Ground-state oxygen holes and the metalinsulator transition in the negative charge-transfer rare-earth nickelates[END_REF], in which the significant amount of extra holes in the NiO 2 plane populate the hybridized Ni 3d -O 2p ligand hole states. This result is consistent with Ni L 2 -edge XAS data [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. The RIXS spectra was simulated using the Kramers-Heisenberg equation combined with atomic multiplet calculations performed with the Cowan and RACER codes [START_REF] Cowan | The Theory of Atomic Structure and Spectra[END_REF][START_REF] De | Core level spectroscopy of solids[END_REF][START_REF] Fabbris | Orbital engineering in nickelate heterostructures driven by anisotropic oxygen hybridization rather than orbital energy levels[END_REF]. These simulations [Fig. 4 (c)&(d)] nicely reproduce the observed signal [27], confirming that these are orbital dd excitations, thus demonstrating the localized character of the Ni 3d state in (LSMO) 9 /(LNO) 3 . Furthermore, the excellent agreement between experimental data and simulation implies high-spin (S = 1) Ni 3d 8 orbitals under a nearly octahedral crystal field with 10D q = 1.1 eV and ∆e g = 0.05 eV [START_REF]with ∆t2g = 0 and reduced Slater-Condon parameters[END_REF].

While the RIXS results establish a predominantly localized Ni 3d orbital, it has minimal sensitivity to the potential presence of a small amount of Ni 3d -O 2p ligand holes. Figure 5 displays the pre-edge of the O K-edge XAS data collected in total electron yield mode at the REIXS beamline of the Canadian Light Source. The xrays were incident at 15 • and the sample was field cooled under 0.6 T, but the magnetic field was removed prior to the data collection. The pre-edge is dominated by features related to Mn-O and Ni-O ligand holes. Comparing this data with the literature [START_REF] José | Influence of the Synthetic Pathway on the Properties of Oxygen-Deficient Manganese-Related Perovskites[END_REF][START_REF] Cui | Tuning the entanglement between orbital reconstruction and charge transfer at a film surface[END_REF], it is clear that the Mn-O states dominate the signal. This result is consistent with the large response expected for Mn 3d 4 /3d 3 (Mn 3+/4+ ) and small or absent pre-edge structure for Ni 3d 8 (Ni 2+ ) [START_REF] Kuiper | Isotropic O 1s prepeak as evidence for polarons in La2NiO 4+δ[END_REF][START_REF] Pellegrin | O 1s near-edge xray absorption of La2-xSrxNiO 4+δ : Holes, polarons, and excitons[END_REF][START_REF] Cao | Engineered Mott ground state in a LaTiO 3+δ /LaNiO3 heterostructure[END_REF]. It is also consistent with the position of the magnetic circular dichroism signal, which is expected to be dominated by LSMO [Fig. 5(a)&(c)]. However, a small pre-edge shoulder is seen at an energy that is consistent with it originating in the LNO layer (grey dashed line). Interestingly this feature displays a marked linear dichroism, implying an anisotropic ligand hole state in which the more holes populate the 3d 3z 2 -r 2 2p z states in detriment of 3d x 2 -y 2 2p σ . This result is consistent with the tensile strain applied by the SrTiO 3 substrate and is in-line with the concept that strain primarily controls the Ni-O hybridization in LNO heterostructures [START_REF] Fabbris | Orbital engineering in nickelate heterostructures driven by anisotropic oxygen hybridization rather than orbital energy levels[END_REF][START_REF] Parragh | Effective crystal field and Fermi surface topology: A comparison of d-and dp-orbital models[END_REF][START_REF] Grisolia | Hybridization-controlled charge transfer and induced magnetism at correlated oxide interfaces[END_REF].

We now discuss the consequences of the observed electronic configuration to the potential mechanisms driving the complex magnetic structure of (LSMO) 9 /(LNO) 3 . We start by noting that non-collinear magnetic order in transition metal oxides may derive from the Dzyaloshinskii-Moriya interaction [START_REF] Dzyaloshinsky | A thermodynamic theory of weak ferromagnetism of antiferromagnetics[END_REF][START_REF] Moriya | Anisotropic Superexchange Interaction and Weak Ferromagnetism[END_REF], but this interaction is inconsistent with the symmetry of the (LNO) 3 magnetic structure [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. The interfacial electronic reconstruction in (LSMO) 9 /(LNO) 3 induces magnetic frustration in the nickelate layer since the antiferromagnetic Ni similar to the dominant ferromagnetic Ni 2+ -O-Mn 4+ [START_REF] Taekyung | Charge transfer, confinement, and ferromagnetism in LaMnO3/LaNiO3 (001) superlattices[END_REF][START_REF] Fabbris | Doping Dependence of Collective Spin and Orbital Excitations in the Spin-1 Quantum Antiferromagnet La2-xSrxNiO4 Observed by X Rays[END_REF][START_REF] Nakajima | Spin-wave excitations in two dimensional antiferromagnet of stoichiometric La2NiO4[END_REF][START_REF] Haskel | Stability of the ferromagnetic ground state of La2MnNiO6 against large compressive stress[END_REF]. However, while we cannot exclude frustration as the sole reason for the observed spiral, if we only consider first neighbor interactions, such frustration would arguably drive an antiferromagnetic coupling structure that resembles the reversed helical model (see Fig. 1) since the adjacent NiO 2 planes would favor an antiferromagnetic coupling. Additionally, it is difficult to reconcile the magnetic structure LNO thickness dependence with a magnetic frustration model [START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF]. Emergent magnetic phenomena are often seen in systems that, much like (LSMO) 9 /(LNO) 3 , have 3d-2p ligand holes (negative charge transfer systems [START_REF] Khomskii | Unusual valence, negative charge-transfer gaps and self-doping in transition-metal compounds[END_REF]), including certain cuprates [START_REF] Mizokawa | Origin of the band gap in the negative charge-transfer-energy compound NaCuO2[END_REF], hole-doped La 2 NiO 4 [START_REF] Fabbris | Doping Dependence of Collective Spin and Orbital Excitations in the Spin-1 Quantum Antiferromagnet La2-xSrxNiO4 Observed by X Rays[END_REF][START_REF] Pellegrin | O 1s near-edge xray absorption of La2-xSrxNiO 4+δ : Holes, polarons, and excitons[END_REF], RENiO 3 [START_REF] Bisogni | Ground-state oxygen holes and the metalinsulator transition in the negative charge-transfer rare-earth nickelates[END_REF], and CrO 2 [START_REF] Korotin | CrO2: A Self-Doped Double Exchange Ferromagnet[END_REF]. In fact, it has been argued that doubleexchange and superexchange can act together to create non-collinear canted structures in such systems [START_REF] De Gennes | Effects of Double Exchange in Magnetic Crystals[END_REF][START_REF] Mostovoy | Helicoidal Ordering in Iron Perovskites[END_REF][START_REF] Santos | Delta Doping of Ferromagnetism in Antiferromagnetic Manganite Superlattices[END_REF], but it is not entirely clear how such canting would actually lead to a helix. Finally, we note that the electronic structure of (LNO) 3 closely mimics that of lightly hole-doped La 2 NiO 4 [START_REF] Fabbris | Doping Dependence of Collective Spin and Orbital Excitations in the Spin-1 Quantum Antiferromagnet La2-xSrxNiO4 Observed by X Rays[END_REF][START_REF] Pellegrin | O 1s near-edge xray absorption of La2-xSrxNiO 4+δ : Holes, polarons, and excitons[END_REF][START_REF] Kuiper | Polarization-dependent nickel 2p x-ray-absorption spectra of La2NiO 4+δ[END_REF], which displays stripelike spin density wave instabilities [START_REF] Tranquada | Simultaneous Ordering of Holes and Spins in La2NiO4.125[END_REF][START_REF] Sachan | Charge and spin ordering in La2-xSrxNiO4.00 with x=0.135 and 0.20[END_REF], which in turn are known to foster non-collinear magnetic structures [START_REF] Shraiman | Mobile Vacancies in a Quantum Heisenberg Antiferromagnet[END_REF][START_REF] Oron Zachar | Landau theory of stripe phases in cuprates and nickelates[END_REF]. Therefore, it appears that the (LNO) 3 magnetic helix emerges from the coupling of such SDW instability with the interfacial Ni-Mn interaction.

In conclusion, the presence of an emergent magnetic helical structure in the [La 2/3 Sr 1/3 MnO 3 ] 9 /[LaNiO 3 ] 3 superlattice is demonstrated through XRMR measurements. Such c-axis magnetic helix has not been shown to occur in any other nickelate system. Ni L 3 -edge RIXS and O K-edge XAS experiments show that interfacial charge transfer results in NiO 2 planes with lightly holedoped Ni 3d 8 atoms. We argue that such electronic structure likely drives SDW instabilities that may drive the helical magnetic structure, but further work is required to verify our arguments. Besides theoretical investigations, we believe that the role of octahedral rotations in this system need to be addressed as these are known to relevant to magnetic exchange. Nevertheless, the prospect of engineering spin density waves through interfaces to create novel magnetism is tantalizing, and, in principle, such approach should applicable to the various families of negative charge transfer transition metal oxides [START_REF] Khomskii | Unusual valence, negative charge-transfer gaps and self-doping in transition-metal compounds[END_REF].

H

  FIG.1. Top: Possible magnetic structures of the LaNiO3 layer in (LSMO)9/(LNO)3. Bottom: Experimental geometry used in both XRMR and RIXS measurements. XRMR was measured using circular and linear x-ray polarization with the magnetic field (H) applied along the sample's a-axis. RIXS was measured using linear x-ray polarization with θ and 2θ fixed to 15 • and 85 • , respectively.

Ni L 3

 3 FIG. 2. (a)&(c) Ni L3-edge (853.6 eV) XRMR data using circular and linear polarization, respectively. (b)&(d) MnL3-edge (641 eV) XRMR data using circular and linear polarization respectively. Data from both edges show half-order magnetic peaks (black arrows). This is consistent with a noncollinear magnetic structure between both LNO layers and LSMO layers as shown in Fig.1. (e) Field dependence of the Ni L3 XRMR data. A field of 0.6 T is sufficient to force a ferromagnetic alignment between every layer, which suppresses the half-order peaks, consistent with previous results[START_REF] Hoffman | Oscillatory Noncollinear Magnetism Induced by Interfacial Charge Transfer in Superlattices Composed of Metallic Oxides[END_REF].

FIG. 3 .

 3 FIG. 3. Fits to the XRMR signal at the Ni L2 (871.6 eV) [(a)&(c)] and Mn L3 [(b)&(d)] edges in (LSMO)9/(LNO)3. Different polarization channels CR/CL or σ/π are shown as red/blue points and are overlapped by orange/blue lines representing the fit. Different channels are offset from one another for clarity. Panel (e) displays the XRMR asymmetry[START_REF]The XRMR assymetry is defined here as the difference between right and left polarized XRMR times q 4 z[END_REF]; the helix model is consistent with the presence of peaks at high qz.

FIG. 4 .

 4 FIG. 4. Ni L3-edge RIXS of (LSMO)9/(LNO)3. Data for π and σ polarized incident x-rays are displayed in panels (a)&(b), respectively. Panels (c)&(d) present simulations of the RIXS spectra for π and σ polarization using atomic calculations [36-38].

  2+ -O-Ni 2+ exchange energy is expected to be

	Norm. XAS (arb. units) Circular Dichroism (%)	0.0 0.2 0.4 0.6 0.8 1.0 1.2 0.4 0.0 0.4 0.8 1.2 1.6 2.0	526 528 530 532 534 CR pol. CL pol. 526 528 530 532 Energy (eV) 526 528 530 532 pol. pol. (a) (b) (c) (d)	0.0 0.2 0.4 0.6 0.8 1.0 1.2 4 2 0 2 4 6 8	Norm. XAS (arb. units) Linear Dichroism (%)

FIG. 5. (LSMO)9/(LNO)3 O K-edge XAS data collected at 15 • incident angle. (a)&(b) shows the pre-edge features of the XAS collected with circular and linear x-rays, respectively. The circular and linear dichroism are plotted in panels (c)&(d), respectively. While signal from hybridized Mn-O orbitals dominate the pre-edge features (vertical red dashed line), a signal that is consistent with Ni-O hybridization is seen around 527.4 eV (grey dashed line).

We thank Dr. Daniel Haskel for insightful discussions and Dr. Alessandro Nicolaou for support during the RIXS measurements at the SEXTANTS beamline. This material is based upon work supported by the U.S. Department of Energy, Office of Basic Energy Sciences, Early Career Award Program under Award Number 1047478. Work at Brookhaven National Laboratory is supported by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, under Contract No. de-sc00112704. Work at Argonne is supported by the U.S. Department of Energy, Office of Science, under contract No. DE-AC-02-06CH11357. Part of the research described in this paper was performed at the Canadian Light Source, which is supported by the Canada Foundation for Innovation, Natural Sciences and Engineering Research Council of Canada, the University of Saskatchewan, the Government of Saskatchewan, Western Economic Diversification Canada, the National Research Council Canada, and the Canadian Institutes of Health Research. We acknowledge SOLEIL for provision of synchrotron radiation facilities. S.G.C. acknowledges the support of the Agence Nationale de la Recherche (ANR), under Grant No. ANR-05-NANO-074 (HR-RXRS).


Xiaoran Jiang

Christian Galea

Laurent Guillo

Christine Guillemot

3D Scene Modeling from Dense Video Light Fields

Keywords: Light field video, plenoptic cameras, depth maps, 3D point clouds

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Light field imaging has recently gained in popularity, both at the research and industrial level. Targeted applications include depth estimation, change of viewpoint and view synthesis, augmented reality content capture or post production. Capturing devices range from cameras arrays [START_REF] Wilburn | High performance imaging using large camera arrays[END_REF] to camera gantries in which a single camera moves along a plane and takes captures at regular time intervals, and to plenoptic cameras [START_REF] Ng | Light field photography[END_REF][START_REF] Georgiev | Superresolution with the focused plenoptic camera[END_REF]. Two optical designs have been considered for plenoptic cameras, the so-called plenoptic 1.0 design, called unfocused plenoptic camera, in which the main lens focuses the subject on the lenslet array [START_REF] Ng | Light field photography[END_REF], and the plenoptic 2.0 design [START_REF] Georgiev | Superresolution with the focused plenoptic camera[END_REF], also called focused plenoptic camera, in which the image plane of the main lens is the object plane of the lenslet array.

Considering the simplified 4D representation proposed in [START_REF] Levoy | Light field rendering[END_REF] and [START_REF] Steven J Gortler | The lumigraph[END_REF], a light field can be seen as capturing an array of viewpoints (called sub-aperture images in the case of microlens based capturing devices) of the imaged scene. Camera arrays are naturally designed to capture the set of views, thus offering a high spatial resolution for each view but a low angular resolution (limited set of views), and hence a large baseline. In contrast, plenoptic cameras use an array of microlenses placed in front of the photosensor to separate the light This work has been supported by the EU H2020 Research and Innovation Programme under grant agreement No 694122 (ERC advanced grant CLIM). rays striking each microlens into a small image on the photosensor, and this way capture dense angular information with a small baseline.

In this paper, we used the Raytrix R8 focused video light field camera to capture test video light fields, and we illustrate some possibilities offered by such image modality in terms of scene depth estimation and 3D point clouds scene representation. The acquisition set-up and the captured dataset, that is available for download, are described in Section 2. A method for estimating a disparity (or equivalently depth) map for every viewpoint is presented in Section 3, while most state-ofthe-art methods, often operating in the epipolar plane images, produce a depth map for the central view only [START_REF] Zhang | Robust depth estimation for light field via spinning parallelogram operator[END_REF][START_REF] Johannsen | Occlusion-aware depth estimation using sparse light field coding[END_REF][START_REF] Wanner | Variational light field analysis for disparity estimation and superresolution[END_REF][START_REF] Jeon | Accurate depth map estimation from a lenslet light field camera[END_REF][START_REF] Tao | Depth from combining defocus and correspondence using light-field cameras[END_REF][START_REF] Ting-Chun | Occlusion-aware depth estimation using light-field cameras[END_REF]. For each viewpoint, one can then construct a 3D point cloud using the corresponding estimated depth maps, while the point clouds at each viewpoint may also be aligned with each other, as explained in Section 4.

SCENE CAPTURE WITH AN R8 RAYTRIX VIDEO LIGHT FIELD CAMERA

In order to provide input data to the algorithms to generate disparity maps and 3D point clouds, several scenes have been captured as dense video light fields. The sections below describe the video plenoptic camera we used, the characteristics of the scenes and how the 5 × 5 viewpoints are extracted from the raw data.

The R8 Raytrix plenoptic camera

The video plenoptic camera we used to capture the test scenes is a R8 Raytrix plenoptic video camera [START_REF]Raytrix Products[END_REF] with a C-mount Ricoh lens FL-BC3518-9M. Its focal length is 35mm and its maximum aperture ratio is 1:1.8. The resolution of the sensor is 3840×2160 pixels (Ultra HD). The micro lens array (MLA) contains three kinds of lenses with different focal lengths. The Raytrix API [START_REF]Raytrix Light Field SDK v3.1[END_REF] provides more details about the characteristics. They are listed in the

Captured scenes

Three different scenes have been captured with the R8 Raytrix camera: ChessPieces, Boxer-IrishMan-Gladiator and Chess Pieces-MovingCamera. The R8 Raytrix camera can accurately capture a scene included in a working volume, the size of which depends on parameters such as the focal length of the main lens and the distance between the camera and the scene. Our experiments have shown that this volume has roughly a 10×10×10 cm size when the R8 camera is equipped with a 35mm main lens. For the first two scenes, this volume is centered on a turning table with pieces or characters on it and with a calendar as background. The camera is located 35cm from the scene. For the third scene, the scene is still and is composed of chess pieces with a calendar as background.

The camera moves alongside the scene such that the distance between the camera and the background remains constant, at 60cm. All these videos are composed of 300 frames at a frame rate of 30fps. The first frames for each video are depicted in Table 2.

Extracting views from raw data

The RxLive Raytrix software manages the R8 camera and records the raw data which can be processed to obtain 2D images. We only use it to save the raw data as a sequence file. We used the Raytrix API to access each Raytrix frame in the sequence file in order to extract 25 views as if we had twenty-five virtual cameras (5 rows × 5 columns). Each virtual camera is 4mm away from its neighbours as depicted in Figure 1.

To extract views, we take advantage of the Raytrix API and the function "RX::ApiLF::RxSetPar". Called with the parameter "Epar::VirtCamPinHoleStd ViewOffsetMM g", it allows the specification of an offset of a virtual camera in the plane perpendicular to the z-axis of main lens. The offset has Fig. 1: 25 virtual cameras per frame, one for each extracted view a value of 4mm in both the horizontal and vertical directions, and is measured from the center of the main lens.

DISPARITY ESTIMATION

In order to construct the 3D model of the scene, the depth information must first be estimated, which can be derived from the disparity between the different views. The algorithm used for estimating a disparity map for each light field view proceeds as follows. We first estimate the disparity between the four corner views of the light filed using an optical flow estimator. These disparity maps are then propagated to the other viewpoints using forward warping and any holes arising due to occlusions are filled using a low rank matrix completion method. A Total Variation (TV) regularization is finally applied in the Epipolar Plane Image (EPI) domain of the resulting disparity maps to enforce view consistency.

Optical flow

Captured by a Raytrix lenslet camera, the light field videos in our dataset are of narrow baseline (less than 2 pixels between adjacent views). In this case, the views on the extreme corners arguably contain all color and geometric information, from which the whole light field can be reconstructed. Therefore, we choose to only exploit the corner views to infer disparity maps for every viewpoint.

An optical flow estimator is used to compute disparity between the corner views. In this work, the learning-based optical flow estimator FlowNet 2.0 [START_REF] Ilg | Flownet 2.0: Evolution of optical flow estimation with deep networks[END_REF] is selected for its accuracy and time efficiency. Taking the view L 1,1 as an example, FlowNet 2.0 is performed three times, between the aforementioned view and the view L 1,5 , L 5,1 and L 5,5 . We let D (i,j)→(i ,j ) x and D (i,j)→(i ,j ) y denote the optical flow components from view L i,j to view L i ,j , on axis x and y respectively. A set of 4 estimates is therefore obtained at view L 1,1 :

{D (1,1)→(5,1) x , D (1,1)→(1,5) y , D (1,1)→(5,5) x , D
(1,1)→(5,5) y } (since the light field views are rectified, a scene point moves only horizontally from position (1, 1) to (5, 1), and only vertically from position (1, 1) to (1, 5), thus the two optical flow components D

(1,1)→(1,5) x and D

(1,1)→(5,1) y should equal to zero). Moreover, since our light field is a square grid of 5 × 5 views, and we assume that the horizontal and vertical baseline between views is identical, these estimates are considered to reveal the same disparity information.

As proposed in our recent work [START_REF] Jiang | Depth estimation with occlusion handling from a sparse set of light field views[END_REF], these disparity maps can be fused to a single map within an energy minimization framework, and the resulting maps can be furthermore enhanced by a superpixel-based edge-preserving filtering. However, for the generation of our dataset, which contains 3 light field videos of 300 frames each, each frame being a light field of 5 × 5 views of 1920 × 1080 pixels, this process becomes time consuming. Thus, a simple average of the aforementioned 4 estimates is performed to obtain a single disparity map D1,1 . The same process is performed at the three other corner views. These maps are then normalized by dividing them by the number of angular intervals between corner views such that they represent disparity between adjacent views, assuming a constant baseline.

Propagation and inpainting

The obtained disparity maps Dr , r = (u r , v r ) corresponding to the 4 input corner positions are projected to a novel position s = (u s , v s ) ∈ 1 . . 5 × 1 . . 5 by using the disparity information itself. Forward warping is applied:

Dr s (x , y ) = Dr (x, y), (1)
with x y = x y + u s -u i v s -v i • Dr (x, y). (2)
Dr s denotes the warped disparity map from position r to s. The projected pixel (x , y ) in this destination image may fall onto non integer coordinates, and interpolation should be performed to retrieve values on integer coordinates. It is well known that the resulting interpolated image will have cracks and holes. The former is caused since an integer coordinate may not be the nearest neighbor of any projected coordinates. The latter results from the fact that a part of the scene which is revealed in the projected view can not be seen in the reference view. Therefore, we also define a binary mask M r s , with value 0 at the missing value (cracks + holes) positions, and 1 anywhere else.

At each position s of the light field, we thus obtain 4 pairs of disparity maps, warped from the 4 corners, and their corresponding masks ( Dr s , M r s ). Considering the strong correlation between neighboring viewpoints of a light field, the missing values in the warped maps may be found not only within the 4 maps in the same position, but also within other maps in some neighboring positions. We thus construct the matrix H of 4 × 25 columns (25 views in each light field video frame), each column being a vectorized warped disparity map:

H = vec( Dr1 s1 ) |. .| vec( Dr4 s1 ) |. .| vec( Dr1 s25 ) |. .| vec( Dr4 s25
) . Correspondingly, the mask matrix indicating the position of missing values (due to occlusions) is also defined:

M = vec( M r1 s1 ) |. .| vec( M r4 s1 ) |. .| vec( M r1 s25 ) |. .| vec( M r4 s25
) . Given the small disparity between views, the matrix H is assumed to be of low rank. The missing disparity values, due to occlusions, can therefore be recovered using a low rank matrix completion approach [START_REF] Lin | Linearized alternating direction method with adaptive penalty for low rank representation[END_REF] which searches for a low rank matrix Ĥ such that

min Ĥ rank( Ĥ) s.t. P M ( Ĥ) = P M ( H), (3)
where P M is the sampling operator such that P M (H) i,j is equal to H i,j if Mi,j = 1, and zero otherwise. This inpainting problem is solved efficiently using the Inexact ALM (IALM) method [START_REF] Lin | Linearized alternating direction method with adaptive penalty for low rank representation[END_REF]. The disparity values are well recovered in practice because the occlusion zones at different viewpoints are unlikely to overlap.

To obtain one single disparity map per viewpoint, a simple average is computed: Ds = mean( Dr1 s , ..., Dr4 s ). And finally, in order to enforce angular consistency of the final disparity maps of the whole light field scene, a step of total variational regularization (TV-L1) [START_REF] Chambolle | An introduction to Total Variation for Image Analysis[END_REF] is applied on epipolar plane images of the resulting disparity maps.

3D POINT CLOUDS CONSTRUCTION

Each pixel with coordinates k = (x, y) in a disparity map Ds as obtained with the method described in Section 3 may be projected into a point in 3-D space having coordinates (X k , Y k , Z k ) using an approach similar to that described in [START_REF] Honauer | A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields[END_REF]. Specifically, the 3-D point coordinates can be derived from the view pixel coordinates using the pinhole camera model [START_REF] Hartley | Multiple View Geometry in Computer Vision[END_REF] as follows:

X k = ( x W -1 -1 2 ) • ss • Z k f (4)
Y k = ( y H-1 -1 2 ) • ss • Z k f ( 5
)
where ss is the sensor size of the larger of both dimensions (mm), f is the focal length (mm), x ∈ [0, W -1] and y ∈ [0, H -1] are the column and row indices, respectively, of the pixel in view L i,j being considered, and W = 1920 pixels and H = 1080 pixels are the width and height of L i,j , respectively, such that ss/W and ss/H give the pixel size and W/2 places the origin at the centre of the image. Z k can be similarly derived from the pinhole camera model, accounting for the translation between the cameras and the fact that the disparity is equal to zero at the focal plane (i.e. when Z = f d , where f d is the focus distance (mm)). Thus, Z k may be formally defined as follows:

Z k = b • f • f d • max(H, W ) d k • f d • ss + b • f • max(H, W ) (6)
where b is the baseline (in millimetres, mm).

-Each pixel with coordinates k = (x, y) in a disparity map Ds as obtained with the method described in Section 3 may be projected into a point in 3-D space having coordinates (X k , Y k , Z k ) using an approach similar to that described in [START_REF] Honauer | A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields[END_REF]. Specifically, given a disparity value d k in Ds representing the shift of a pixel in one view to an adjacent view, the depth value Z k may be computed as follows:

Z k = b • f • f d • max(H, W ) d k • f d • ss + b • f • max(H, W ) (7)
where x ∈ [0, W -1] and y ∈ [0, H -1] are the column and row indices, respectively, of the pixel in view L i,j being considered, W = 1920 pixels and H = 1080 pixels are the width and height of L i,j , respectively, b is the baseline (in millimetres, mm), f is the focal length (mm), f d is the focus distance (mm), and ss is the sensor size of the larger of both dimensions (mm). The X k -coordinates may then be obtained as follows:

X k = ( x W -1 -1 2 ) • ss • Z k f (8)
Similarly, the Y k -coordinates are given by:

Y k = ( y H-1 -1 2 ) • ss • Z k f (9)
With the above procedure, the (X k , Y k , Z k ) coordinates of a pixel k = (x, y) given disparity value d k are now available. Repeating this process for all pixels within L i,j yields a set of points that is generally referred to as a point cloud. Point clouds may then be obtained for each of the H LF × W LF views.

At this juncture, points in different views are not aligned such that those corresponding to the same area of an object coincide. Hence, the X k -and Y k -coordinates need to be shifted such that the point clouds from different views are aligned to a common coordinate system. Given a point cloud in view (u, v), where u ∈ [1, W LF ] and v ∈ [1, H LF ] are the column and row indices, respectively, of a view that needs to be aligned with the points in view (u c , v c ), and W LF = H LF = 5 represent the number of views in the horizontal and vertical directions, respectively, then Equations ( 8) and ( 9) may be slightly modified to represent the shift required for the X k -and Y k -coordinates, respectively, as follows:

X k = X k + d k • ss • Z k • (u c -u) f • (W -1) (10)
Y k = Y k + d k • ss • Z k • (v c -v) f • (H -1) (11)

DISCUSSION AND CONCLUSION

While there exist datasets acquired with rigs of video cameras [START_REF] Sabater | Dataset and pipeline for multiview light-field video[END_REF], to the best of our knowledge there do not exist publicly available dense light field videos capturing real scenes.

The dataset that we created with the R8 Raytrix video camera allows us to extract 25 views per frame and to generate disparity maps and 3D point clouds as shown in Figure 2.

The dataset (raw data, extracted views, corresponding disparity maps, and animations demonstrating the variation of the point clouds across the video frames) can be downloaded at http://clim.inria.fr/DataSoftware.html.

The minimal and maximal disparity values are very low as shown in Table 2. These low values are related to the density of the light field and to the distance between virtual cameras. When the distance is larger (e.g. 8mm), artefacts appear in the views of the border as shown in Figure 3. That is why we kept this distance as low as possible. However, the algorithms that we presented show that it is still possible to exploit views with low disparities in order to generate, for instance, disparity maps or 3D point clouds. Moreover, the presented techniques are efficient, such that the disparity maps and point clouds can be obtained quite rapidly. The distortion of the 3D point cloud Figure 2 bottom right is mainly due to the high distance of the camera to the scene which is not compatible with the working volume and the distance required for our R8 Raytrix camera equipped with a 35mm main lens.

In the current stage of works, disparity maps and 3D point clouds are computed frame by frame. The temporal consistency could be better exploited in the future work in order to obtain more accurate 3D scene reconstruction. An improvement in the depth map estimation, particularly in the background of the scene and along the contours of objects that cause the fusion of foreground and background in 3D, would also enable the generation of point clouds with less distortions and artefacts.

Fig. 2 :

 2 Fig. 2: Examples of the light fields in the new dataset, and the corresponding disparity maps and point clouds: the fist row corresponds to the first frame of the central view of each of the three video light fields, the second row to the associated disparity maps, and the third row to the corresponding 3D point clouds, viewed with the MeshLab software program [16].

Fig. 3 :

 3 Fig. 3: View (5,5) of ChessPieces-MovingCamera when the distance between virtual cameras is set to 8mm. Artifacts appear in the top right corner.

Table 1

 1

	below:

Table 1 :

 1 Sensor and MLA characteristics of the R8 Raytrix video camera

Video Name

Min. Max. ChessPieces -1.52 0.45 Boxer-IrishMan-Gladiator -1.50 0.39 ChessPieces-MovingCamera -1.67 0.11

A. C++ SNIPPET TO EXTRACT VIEWS

The following is a snippet of the code to extract views. It assumes that the Raytrix API has been correctly initialized and that a rayHandle referring to an instant of the video (i.e a raySequence) is given to the following method named "ex-tractViewsFromCurrentFrame". The namespace is Rx::ApiLF. x S e t P a r ( EPar : : D e p t h P a t h L T P i x e l S t e p , uLensTypeIdx , 0 . 5 ) ; R x S e t P a r ( EPar : : D e p t h P a t h L T M i n S t d D e v , uLensTypeIdx , 0 . 0 0 ) ; R x S e t P a r ( EPar : : D e p t h P a t h L T M i n C o r , uLensTypeIdx , 0 . 9 ) ; R x S e t P a r ( EPar : : D e p t h P a t h L T M i n P o l y P a r A b s , uLensTypeIdx , 0 . 0 5 ) ; R x S e t P a r ( EPar : : D e p t h P a t h L T M a x P o l y C t r D e l t a , uLensTypeIdx , 0 . 5 ) ; R x S e t P a r ( EPar : : D e p t h P a t h L T P a t c h D i a , uLensTypeIdx , 4U ) ; R x S e t P a r ( EPar : : D e p t h P a t h L T P a t c h S t r i d e , uLensTypeIdx , 1U ) ; } / / Depth Fuse P a r a m e t e r s R x S e t P a r ( EPar : : D e p t h F u s e M e d i a n R a d , 0U ) ; R x S e t P a r ( EPar : : Depth Fuse MeanRad , 0U ) ; R x S e t P a r ( EPar : : D e p t h F u s e M e a n M a x B l a c k P a r t , 0 . 2 ) ; / / Depth F i l l P a r a m e t e r s R x S e t P a r ( EPar : : D e p t h F i l l E n a b l e d , 1U ) ; R x S e t P a r ( EPar : : D e p t h F i l l I t e r C n t , 4U ) ; R x S e t P a r ( EPar : : D e p t h F i l l I t e r S i z e , 1U ) ; R x S e t P a r ( EPar : : D e p t h F i l l C o m p l e t e , 1U ) ; / / Depth F i l l B i l a t e r a l f i l t e r R x S e t P a r ( EPar : : D e p t h F i l l B i l a t e r a l E n a b l e d , 1U ) ; R x S e t P a r ( EPar : : D e p t h F i l l B i l a t e r a l F i l t e r R a d , 5U ) ; R x S e t P a r ( EPar : : D e p t h F i l l B i l a t e r a l E d g e , 0 . 1 ) ; R x S e t P a r ( EPar : : D e p t h F i l l B i l a t e r a l R a n g e , 5 . 0 ) ; RxGetPar ( EPar : : V i r t C a m P i n h o l e S t d V i e w O f f s e t M M g , myadValue ) ; / / g e t t h e h o r i z o n t a l d i s t a n c e i n mm b e t w e e n v i r t u a l c a m e r a s i n t dx = t h i s > m c o n f i g P a r a m e t e r s . getDY ( ) ; / / g e t t h e v e r t i c a l d i s t a n c e i n mm b e t w e e n v i r t u a l c a m e r a s i n t dy = t h i s > m c o n f i g P a r a m e t e r s . getDY ( ) ; i n t x i n f = ( m c o n f i g P a r a m e t e r s . g e t N b C a m e r a s P e r L i n e ( ) 1 ) / 2 * 1 * dx ; i n t y i n f = ( m c o n f i g P a r a m e t e r s . getNbCamerasPerColumn ( )

/ / The r e s u l t o f t h i s o p e r a t i o n i s s t o r e d i n t h e i n t e r n a l image / / C a l c u l a t e t h e raw d e p t h and s u b s e q u e n t l y / / t h e f u s e d d e p t h map i n t h e v i e w camera s p a c e