Edouard Pineau

Sébastien Razakarivony

Thomas Bonald

Seq2VAR: multivariate time series representation with relational neural networks and linear autoregressive model

Keywords: Multivariate time series, Vector linear autoregression, Relational neural networks, Granger causality

Finding understandable and meaningful feature representation of multivariate time series (MTS) is a difficult task, since information is entangled both in temporal and spatial dimensions. In particular, MTS can be seen as the observation of simultaneous causal interactions between dynamical variables. Standard way to model these interactions is the vector linear autoregression (VAR). The parameters of VAR models can be used as MTS feature representation. Yet, VAR cannot generalize on new samples, hence independent VAR models must be trained to represent different MTS. In this paper, we propose to use the inference capacity of neural networks to overpass this limit. We propose to associate a relational neural network to a VAR generative model to form an encoder-decoder of MTS. The model is denoted Seq2VAR for Sequenceto-VAR. We use recent advances in relational neural network to build our MTS encoder by explicitly modeling interactions between variables of MTS samples. We also propose to leverage reparametrization tricks for binomial sampling in neural networks in order to build a sparse version of Seq2VAR and find back the notion of Granger causality defined in sparse VAR models. We illustrate the interest of our approach through experiments on synthetic datasets.

Introduction

Nowadays, more and more data come as multivariate time series (MTS) and finding understandable and meaningful feature representation of observed MTS samples is needed for information mining and downstream machine learning tasks. Among standard MTS representation and modeling [START_REF] Esling | Time-series data mining[END_REF], vector linear autoregression (VAR) [START_REF] Hiro | Vector autoregression and causality: a theoretical overview and simulation study[END_REF] captures dynamical and causal information contained in the data. Yet, VAR does not have representation inference mechanism: one model fits all samples. Consequently, to represent different MTS, independent VAR models must be trained. This operation becomes very expensive when VAR parameters are constrained, for example with sparsity [START_REF] Richard A Davis | Sparse vector autoregressive modeling[END_REF] or symmetry, since there are no closed-form nor simple regularization to solve it. In contrast, efficient and powerful inference is a key advantage of neural networks based representation learning.

Many recent works on neural generative models have been used for representation learning, with particular attention on variational autoencoders (VAE) [START_REF] Diederik | Auto-encoding variational bayes[END_REF][START_REF] Michael Tschannen | Recent advances in autoencoder-based representation learning[END_REF]. The VAE, as a latent variable model, finds the joint distribution between observed data and a set of latent variables. With appropriate assumptions on the variational distribution and regularization, one hopes to uncover and disentangle independent causal sources of variations in the data, with a certain form of interpretability and a good representation power. However, finding such disentangled representation cannot be successfully done without an appropriate inductive bias [START_REF] Locatello | Challenging common assumptions in the unsupervised learning of disentangled representations[END_REF].

In this context, a specialized VAE architecture, called neural relational inference (NRI), has recently been developed for interaction inference in MTS [START_REF] Kipf | Neural relational inference for interacting systems[END_REF]. NRI uses the concept of relational learning [START_REF] Santoro | A simple neural network module for relational reasoning[END_REF] to explicitly capture the variable interactions that explain the data dynamics. The inferred binary interaction graph is used as a variable selection preceding a nonlinear auto-regressive decoding scheme. The objective, with an appropriate setting, is both to help the generative decoder with sparsity and to uncover the real interactions in an unsupervised manner. Nevertheless, this approach suffers from its powerful decoder, that does not necessarily need meaningful latent representation to recover data. The general problem of information vanishing in the presence of a too expressive decoder is classical [START_REF] Ma | Mae: Mutual posterior-divergence regularization for variational autoencoders[END_REF][START_REF] Shen | Improving variational encoder-decoders in dialogue generation[END_REF]: in extreme cases, the inference part becomes meaningless, transforming the VAE into a standard generative auto-regressive model [START_REF] Larochelle | The neural autoregressive distribution estimator[END_REF]. In our experiments, we show that in the NRI case, this problem manifests when the interactions to discover are heterogeneous, i.e., the latent interaction graph is weighted. As a result, the unsupervised interaction discovery does not correspond anymore to true physical reality while the model still decodes properly. Finally, by construction, NRI only covers information contained in first previous time step for prediction part. Therefore, it needs lagged information as explicit input, which mechanically increases the number of parameters to train.

In this paper, we propose an alternative, simple MTS representation learning framework that exploits both VAR generative models and neural network inference capacities. As in [START_REF] Kipf | Neural relational inference for interacting systems[END_REF], we assume that MTS are the observation of a system that can be represented with causal information. Following the classical framework of encoder-decoder, we build the Seq2VAR model, whose architecture presents three main advantages. First, we prevent information vanishing by representing our MTS samples as the parameters of a VAR instead of the input of a neural auto-regressive decoder. Second, we leverage recent advances in neural networks binary sampling [START_REF] Kaiser | Discrete autoencoders for sequence models[END_REF][START_REF] Li | Towards binary-valued gates for robust lstm training[END_REF][START_REF] Louizos | Learning sparse neural networks through l_0 regularization[END_REF] to build explicit sparse representations; this way, non-zero entries of the inferred representation can be interpreted as Granger causalities. Third, we help our model generalizing over the notion of interactions by using a relational inference neural network (RINN) [START_REF] Kipf | Neural relational inference for interacting systems[END_REF] as MTS encoder. These three properties make Seq2VAR able to find meaningful MTS representations.

After presenting Seq2VAR model and assumptions, we perform some experiments to assess its different advantages. We then discuss (1) the interest of using RINN encoder instead of recurrent neural networks (RNN), that are explicitly made for time series modeling, and (2) the relations and differences between Seq2VAR and the NRI model.

Model description

Notations and assumptions

Let X ⊆ R d×T be some finite set of n d-dimensional MTS defined over the time range t = 1, . . . , T . We assume that each MTS X (s) , s = 1, . . . , n is generated by some dynamical system. Specifically, we assume that each MTS follows a K-order linear auto-regressive model, i.e., there exists some tensor A (s) ∈ R K×d×d such that the MTS X (s) = (X

(s) 1 , . . . , X (s) T ) ∈ X satisfies X (s) t = K k=1 A (s) k X (s)
t-k for all t = K + 1, . . . , T . Both the tensor A (s) and the initial values (X

(s) 1 , . . . , X (s)
K ) are unknown and can change from one sample to the other. Some additive observation noise can be added to the samples. In this paper, we assume that the set of tensors {A (s) , s = 1, . . . , n} belongs to A, the set of acceptable dynamical systems. This set A is also unknown.

General problem setup

We place ourselves in the encoder-decoder framework. We use a relational inference neural network (RINN) as MTS encoder [START_REF] Kipf | Neural relational inference for interacting systems[END_REF], which takes a sample X (s) as input and outputs a tensor Â(s) . The decoder is a VAR model parametrized by Â(s) . This tensor is therefore the latent representation of the MTS, with respect to the linear decoding assumption. We denote the encoder by F φ , where φ are a parameter in some finite-dimensional space. Observe that F φ is a mapping from R d×T to R K×d×d . The parameter φ is optimized for the objective function:

n s=1    T t=K+1 X (s) t - K k=1 F φ (X (s) ) k X (s) t-k 2 2 + λΩ F φ (X (s) )    . ( 1
)
Here Ω is a regularizing penalty function and λ a positive coefficient of penalization. These can be used to control the sparsity of the output (number of non-zero entries of the tensor F φ (•)). We call our model Seq2VAR (sequence-to-VAR).

Sparsity in Seq2VAR

Seq2VAR offers the possibility to modify the encoding posterior distribution and the regularization depending on the task of interest. In particular, the output of the encoder F φ may be made sparse [START_REF] Richard A Davis | Sparse vector autoregressive modeling[END_REF][START_REF] Haufe | Sparse causal discovery in multivariate time series[END_REF][START_REF] Yi | Sparse granger causality graphs for human action classification[END_REF]. For such constraint, we use the recent advances in sparse representation learning based on derivations of Gumbel-Softmax trick [START_REF] Jang | Categorical reparameterization with gumbel-softmax[END_REF][START_REF] Chris J Maddison | The concrete distribution: A continuous relaxation of discrete random variables[END_REF].

In theory, true sparse learning rely on L 0 regularization. In practice L 0 norm cannot be directly used as penalty in the objective function since it is nondifferentiable. General approaches uses Ridge, LASSO or group-LASSO regularization [START_REF] Haufe | Sparse causal discovery in multivariate time series[END_REF] as proxy for L 0 regularization. This sparsity becomes implicit and is difficult to control. As an alternative and to obtain explicit sparsity, we propose to use stochastic gates determining for each samples X (s) which entries of F φ (X (s) ) are set to zero [START_REF] Louizos | Learning sparse neural networks through l_0 regularization[END_REF]. The probability of each gate is an additional output of the RINN encoder. Gates are sampled using a continuous relaxation of binomial sampling [START_REF] Jang | Categorical reparameterization with gumbel-softmax[END_REF][START_REF] Chris J Maddison | The concrete distribution: A continuous relaxation of discrete random variables[END_REF], to let the gradient flow. The encoder outputs the probability of opening the gates, denoted P φ (X (s) ). We denote G φ (X (s) ) the binary gates that are sampled with the following derivation of Gumbel-softmax trick proposed in [START_REF] Li | Towards binary-valued gates for robust lstm training[END_REF]:

G φ (X (s) ) = σ P φ (X (s) ) + log U -log(1 -U ) τ , (2)
with σ the sigmoid function, τ the temperature of the relaxation (the higher, the smoother the approximated binary sampling) and U ∼ U(0, 1). The σ, log and + are matrix element-wise operators, U has the same dimensions than P φ (•), with all entries sampled independently. Depending on the task, G φ (•) lives either in R K×d×d or in R 1×d×d . The latter corresponds to variable selection, while the former corresponds to variable and lag selection. The collection of gates forms a binary mask that is directly applied to the dense tensor F φ (•).

We can now control the sparsity of the model with an appropriate penalty function Ω applied to G φ (•) in the following sparse problem:

n s=1    T t=K+1 X (s) t - K k=1 G φ (X (s) ) F φ (X (s) ) k X (s) t-k 2 2 + λΩ G φ (X (s) )    , (3)
where denotes component-wise multiplication.

In [START_REF] Louizos | Learning sparse neural networks through l_0 regularization[END_REF], the authors propose a method for creating sparse neural networks with a similar method, in a Bayesian deep learning setting. They regularize the sparsity with the norm of the probability of the gates. In (3) we generalize this method to representation inference. We use the number of positive gates, i.e., the sum of all the entries of the (almost) binary mask as a proxy for the norm L 0 on encoder output. Thanks to the continuous relaxation of the binary sampling (2), this sum is differentiable with respect to φ and can be used as a penalization in the objective function.

Remark 1. A particular case of sparse approach for Seq2VAR is the pure binary, i.e., by using only G φ (•) as a VAR parameters. This case is studied in the Experiment 3.2. In the experiments, we denote such Seq2VAR model as binary-Seq2VAR.

Symmetry in Seq2VAR

Another constraint that can be easily imposed to Seq2VAR is the symmetry of the tensor Â(s) . Thanks to the inference mechanism, we can force Â(s) to be symmetric by imposing Â(s) = 1/2(F φ (X (s) ) + F φ (X (s) ) T (1,2) ), where T (1, 2) is the transposition of dimensions 1 and 2 of the tensor. This setting can be particularly important in the case where the interactions between variables are physical. We experiment more specifically this constraint in section 3.3, where this model is referred to as symmetric-Seq2VAR. Note that symmetry can be easily coupled with sparsity.

Experiments

Methodology

As a preliminary experimenal work, we illustrate our approach on several synthetic datasets, each with several levels of difficulty to illustrate generalization capacity, causality discovery and representation power. First, in order to assess the generalization capacity of Seq2VAR, we use a test set with intrinsic parameters (causality graphs, interaction intensities) that differ from those of the train set in all experiments. Second, causality discovery is assessed through the F1score between inferred and ground truth causality graphs. When Â(s) is dense, we keep the most significant entries, i.e., those above a certain percentile of all entries in absolute value. This percentile is chosen with a priori knowledge. Third, since the set A is finite, we can assess unsupervisely learned representation with a standard downstream task. We classify the systems with the inferred tensors Â(s) . The classification is a 1-nearest-neighbor on the tensors, for the Frobenius norm. The classification is completed in test set representation. Test set is devided in train and test subset for the classification task.

For each experiment, we compare Seq2VAR to NRI [START_REF] Kipf | Neural relational inference for interacting systems[END_REF] and VAR [START_REF] Hiro | Vector autoregression and causality: a theoretical overview and simulation study[END_REF], from which we respectively inspire for encoder and decoder. We remind that NRI learning relies on variational inference, whose objective function contains a Kullback-Leibler divergence between variational posterior and a certain prior distribution [START_REF] Diederik | Auto-encoding variational bayes[END_REF]. In binary and sparse setting, the prior of each latent graph is the proportion of 1 wanted in the whole latent graph, since all latent entries are independently sampled, by the mean-field assumption of VAEs. We always give NRI the true proportion as prior. For Seq2VAR, eventual usage of sparsity or regularization is specified when existing.

To provide fair comparison benchmarks, the experiments are built such that both VAR and NRI model fit in. Technical details about neural network architectures and parameters are given in Appendix A. We used the Statsmodel python library [START_REF] Seabold | Statsmodels: Econometric and statistical modeling with python[END_REF] for the VAR. The code used for the experiments is available at: https://github.com/edouardpineau/Seq2VAR. Experiments on real datasets, in particular in the context of system health monitoring at Safran, will be the developed as a future work.

Assessing Seq2VAR in binary linear setting

For our first experiment, we aim at [START_REF] Richard A Davis | Sparse vector autoregressive modeling[END_REF] showing that binary sampling approach is relevant for Seq2VAR model and (2) analysing how Seq2VAR behaves in the presence of noise. We place ourselves in a favorable setting for VAR, NRI and Seq2VAR. We generate samples from a stationary 1-order linear autoregressive model, where the linear transition matrix is a permutation matrix. We add several level of additive Gaussian noise to the observations (see Figure 1 for an illustration). We chose d = 10, T = 50, K = 1. Taking #A = 20, we generate 10 permutation matrices for the train set and 10 other permutation matrices for the test set. From each matrix we generate 100 samples with random N (0, 1) initial conditions. We train two versions of the Seq2VAR: dense with no penalty and binary. After training, Seq2VAR has generalized the notion of permutation. We note that the binary version is naturally the only one to find the real permutation matrices. Figure 2 shows the outputs of different versions of Seq2VAR encoder at test time, with different level of observation noise. Fig. 2: Inferred transition matrices for different test samples, using VAR, NRI, Seq2VAR and binary Seq2VAR, compared to ground truth, depending on the level of observation noise: N (0, 0.1) (left), N (0, 0.3) (middle), N (0, 0.5) (right).

We see in Figure 2 an illustration of the problems created by an increase of observation noise variance. While signal-to-noise ratio decreases, disentanglement of the permutations from the noise becomes harder. VAR overestimate the density of the matrix. Optimal mean-square solution (closed form) of VAR decodes noisy signal by mixing noisy signals. On the contrary, Seq2VAR (without regularization) is parsimonious (but not sparse), i.e. it it gathers many entries of Â(s) around zero (see Figure 3). We explain this behavior by two facts. First the inference mechanism of Seq2VAR is shared by all samples and has integrated the noise by seeing numerous noisy examples. Second, the low expressive decoder cannot integrate noise nor deal with complex mixture of noisy signals. This second point explains why Seq2VAR resists better to noise than NRI.

We present in Table 1 the classification accuracy and causality recovery in the presence of different level of observation noise. Seq2VAR approaches outperform VAR and NRI when noise gets stronger. binary-Seq2VAR give also better results, as its assumption fits the data better.

Dataset

Perm. + N (0, 0.1) Perm. + N (0, 0.3) Perm. + N (0, 0.5)

Tasks

Supervised classification

Causality detection

Supervised classification

Causality detection

Supervised classification

Causality detection VAR [START_REF] Hiro | Vector autoregression and causality: a theoretical overview and simulation study[END_REF] 100 ± 0.0 72.2 ± 0.2 96.85 ± 3.8 61.8 ± 4.5 96.5 ± 1.6 52.6 ± 3.9 NRI [START_REF] Kipf | Neural relational inference for interacting systems[END_REF] 100 ± 0.0 95.63 ± 3.1 97.6 ± 3.4 84.3 ± 4.9 97.0 ± 2.1 68.3 ± 3.7 Seq2VAR 100 ± 0.0 97.3 ± 0.3 97.0 ± 4.3 92.5 ± 2.1 97.8 ± 3.9 83.6 ± 2.3 B-Seq2VAR 100 ± 0.0 97.2 ± 0.1 100 ± 0.0 94.6 ± 2.7 97.0 ± 4.3 90.1 ± 2.9

Table 1: Test classification accuracy (%) and causality discovery (F1-score). The standard deviations correspond to the variation in results between different generated datasets (train and test). B-Seq2VAR stands for binary Seq2VAR.

In Figure 4, we see the boxplot representations of the distribution of the L 1 distance between ground truth and inferred causality graph within the test set, for several levels of observation noise. We first see that all methods suffer from noise. When noise is low, both NRI, Seq2VAR and binary-Seq2VAR offers almost perfect graph discover. However, the outliers (red crosses) of NRI spread further than the one of Seq2VAR, which means that NRI fails to find the latent graph not by simply missing some entries but by finding a completely different graph that still fits the decoding requirements. Fig. 4: Quartiles of the distribution of the L 1 distances between true and inferred causality graphs with VAR, NRI, Seq2VAR and binary Seq2VAR respectively with observation noise N (0, 0.1) (left), N (0, 0.3) (middle) and N (0, 0.5) (right). b-Seq2VAR stands for binary Seq2VAR.

Assessing Seq2VAR in physical setting

We now propose to assess the capacity of Seq2VAR to find Granger causality graph hidden in physical data. We use 10-ball-springs system data, consisting of the simultaneous trajectories of 10 identical balls in a 2D space, each ball being connected to others by springs with probability 0.5. The connection network is called interaction graph. System dynamics follows Newton's law of motion, sampled with ∆t = 0.001 and then subsampled at frequency 1/100. The system can be represented as a Granger causality graph [START_REF] Eichler | Causal reasoning in graphical time series models[END_REF] (see Figure 5). Each sample is 49 timesteps long. Note that this experiment is used in NRI paper [START_REF] Kipf | Neural relational inference for interacting systems[END_REF].

For the experiments of this section, we sample 20 different balls-spring binary interaction graphs, 10 for the train set and 10 for the test set. Each of the 20 dynamical systems associated to the 20 graphs is built at random, with balls linked by a spring with probability 0.5. Each graph characterizes a class. We propose two different datasets: one with identical rigidity 1 for all springs (unweighted interaction graph) and one with variable rigidity (weighted interaction graph). For the later, the rigidity of each spring is uniformly sampled in [0.5, 1]. Each binary graph characterizes a class. We use 1000 samples per class. As for permutation MTS (see 3.2) we use different systems for the train set and the test set to challenge the generalization power of Seq2VAR.

We trained respectively a VAR, a dense Seq2VAR, a symmetric dense Seq2VAR and a sparse Seq2VAR. We chose K = 2 for all the experiments. For the later, we impose a slight regularization on the level of sparsity, i.e. on the number of null entries in the matrix. In Equation 3 we use Ω(G φ (X (s) )) = 1 100 G φ (X (s) ) 1 -0.5 and λ is set at 1e -3 , 0.5 being the sparsity prior. Without this regularization, the sparse Seq2VAR generally converges naturally towards Fig. 5: Example of 3-ball-springs system (left) and its Granger causality graph (right). The causality graph represents the first-order dynamical dependencies between balls position at each time t and the positions of their direct neighbors at previous time steps t -1 . . . t -K. For all t = K . . . T and i ∈ {1, 2, 3}, arrows indicates a dynamical dependency between (X

(i) t , Y (i) t ) and (X (j) t-1 , Y (j) t-1 ), . . . , (X (j) t-K , Y (j) t-K ) j∈pa(i)
, with pa(i) the set of parents of node i in the directed graph.

the right proportion of true zeros. The regularization is added for preventing an eventual trivial solution where the mask is a matrix of ones or local minima where only the diagonal paramaters are kept. The only occurrence of this problem happened in the heterogeneous problem.

Table 2 gathers the results. We see that Seq2VAR scores better on both quality measures than the usual VAR approach learned on the test set. If NRI gives very good results of causality detection on homogeneous springs, its causality discovery performance drops when dealing with heterogeneous springs. On the contrary, Seq2VAR gives good results for both homogeneous and heterogeneous rigidity graph. [START_REF] Kipf | Neural relational inference for interacting systems[END_REF]) to obtain better results. See Appendix A for more details.

It is interesting to notice that using an expressive inference network that explicitly model dependencies between variables capitalize on the global informa-tion of the dataset and (1) generalize on new data and (2) overpass the statistical identifiability problem due to subsampling. It is also interesting to notice that sparsity is not optimal in Seq2VAR as it is in NRI. As variable spring rigidity makes the distribution wider and makes the causality less identifiable in both linear (Seq2VAR) and nonlinear (NRI) decoding process, with respect to our experimental setup.

We find in Figure 6 an illustration of the inference capacities of our model.

Discussions

This section proposes discussions around our Seq2VAR approach for MTS representation with VAR matrices and causality graph.

RINN vs. RNN as Seq2VAR encoder

The main assumption when modeling time series data is the autoregressive structure of the observed signal. A generic and expressive autoregressive model is the recurrent neural network (RNN). Hence, in practice we could use RNN to take the MTS as input and do the inference of the tensor A.

Our different attempts with RNN encoder were not able to generalize over the notion of causal interactions, no matter the regularization technique used to avoid overfitting (reduction of network memory capacities, increasing of depth [START_REF] Sak | Long short-term memory recurrent neural network architectures for large scale acoustic modeling[END_REF], dropout [START_REF] Zaremba | Recurrent neural network regularization[END_REF], batch-norm [START_REF] Merity | Regularizing and optimizing lstm language models[END_REF], L 1 -norm and L 2 -norm penalty on weights). We show in Figure 7 an example of train and test performances evolution during training, respectively for permutation and ball-springs experiments with GRU encoder instead of RINN encoder. We can see that a Seq2VAR using RNN as encoder overfits.

We explain this by the fact that RINN explicitly takes as input all the pairs of univariate time series constituting the MTS and outputs a tensor whose entries represent an embedding of each pair. Therefore, this inductive bias incites the network to model one-to-one interactions. We remind that the causality graphs of the test set are different from the causality graphs of the train set. Hence we ask our inference network to generalize over a discontinuous manifold. RINN inference networks and its explicit one-to-one interactions modeling learns well to generalize. Conversely, RNNs implicitly learn the notion of interactions during training, with a vector output that needs to be folded into the right shape. The generalization is more difficult.

Relation with NRI

For the inference learning, the closer existing model is the Neural Relational Inference (NRI) [START_REF] Kipf | Neural relational inference for interacting systems[END_REF], that uses a modified version of the interaction neural networks [START_REF] Santoro | A simple neural network module for relational reasoning[END_REF]. The NRI consists on a graph inference model, set as a VAE with binary latent space. The inferred graph is used as a variable selection procedure for a prediction model. This configuration specifically applies to physical interacting systems MTS (like ball-springs system). Three major differences appear between Seq2VAR and NRI. First the form of the decoder. NRI decoding scheme is a non linear network that takes as input an embedding of the pairwise variable interactions at each time step and output the incremental change to predict next time step from current time step. On the contrary, we propose to leverage the simplicity of a linear autoregressive decoder that is potentially less expressive but do not require additional parameters.

Second, as a consequence of the form of the decoder: the latent representation. We can infer both binary and real latent representation to respectively represent existence and intensity of the causal interactions in the data. The real part is implicit in NRI. Experiment 3.3 shows that NRI does not disentangle existence and intensity of the interactions: when springs are not equally rigid, NRI is perturbed and finds a latent graph that does not correspond to physical reality. Our Seq2VAR, thanks to its continuous part, explicitly disentangles latent causal structure from other information and finds an acceptable causal graph.

Third difference, which is also as a consequence of the decoder: the minimal input information requirement. In fact, the notion of time lag is absent from NRI and lagged information needs to be furnished as input. For example, with the ball-springs systems data, Seq2VAR only needs measures of the location of each ball at each time step while NRI requires both location and velocity. Beyond the minimal input information requirement, the absence of lag in NRI modeling imposes that causality graph remains the same for all lags, like in physical structures.

Finally, these differences materialize with the results of the experiments presented in Section 3. Note that they also procure a significant advantage in term of complexity. We assess this complexity with the number of parameters and computing CPU time. These results are in Table 3.

Conclusions and future works

In this paper, we propose the Seq2VAR model, which learns to represent multivariate time series as Granger causal graph. Seq2VAR a encoder-decoder model consists of a relational neural network for inference and a linear autoregressive for generation. By construction, our model is immune to information vanishing in neural autoregressive framework. It is also capable of generalizing over the notion of interaction and Granger causality, i.e. estimating good VAR representation for unseen samples. The chosen representation is robust to continuous structural

Fig. 1 :

 1 Fig. 1: 10-dimensional permutation MTS with observation noise N (0, 0.3).

Fig. 3 :

 3 Fig. 3: Histogram of the distribution of the entries of inferred transition matrices over a test set, using VAR, NRI, Seq2VAR and binary Seq2VAR, compared to ground truth, depending on the level of observation noise: N (0, 0.1) (left), N (0, 0.5) (right).

Fig. 6 :

 6 Fig. 6: Three examples of inferred transition matrices over a test set, using VAR, NRI, Seq2VAR, Symmetric Seq2VAR, Sparse Seq2VAR and Sparse Symmetric Seq2VAR, compared to ground truth.

Fig. 7 :

 7 Fig. 7: Train and test performances during training of Seq2VAR with GRU encoder instead of RINN encoder. Column (a): MSE of the 1-step VAR prediction. Column (b): F1-score of the inferred causality graph. Top row: permutation dataset. Bottom row: homogeneous 10-ball-springs dataset.

Table 2 :

 2 Test classification accuracy (%) and causality discovery (F1-score). *Hyperparameters different than the one used for homogeneous springs case (from the original paper

Acknowledgments This work is supported by the company Safran through the CIFRE convention 2017/1317.

In further work, we intend to generalize the model to non-linear settings, as well as extending the way sparsity is controlled and constrained.

A Appendix: hyperparameters

For our Seq2VAR, we used a succession of 2-layers perceptrons for our relational encoder, as in NRI [START_REF] Kipf | Neural relational inference for interacting systems[END_REF]. The parameters to chose concern latent dimension for all layers and the temperature parameter τ for the relaxed binary sampling (see Section 2.3). They are presented in Table 4. For NRI, all other parameters are the one of the original paper for the homogeneous springs rigidity except for the dimensionality of the latent space which we set to 64 instead of 256, since in the experimental setup of the original paper, it gives the same results while diminishing computing time and memory needs. For the heterogeneous rigidity, the parameter prediction_steps is set to 5 instead of 10 and τ is set to 0.1. These parameters gave the best average results. In fact, due to the highly expressive form of its decoder, NRI was able to build good predictor with not the good graph. We played with parameters to get more stable and better results. For the experiments, we also tried to change the skip first parameter that is set to False or True in the original paper [START_REF] Kipf | Neural relational inference for interacting systems[END_REF], depending on the dataset studied. It did not change the results of the experiments.

Experiments


Guillaume Lucas
email: guillaume.lucas@renault.com

Florent Colombet
email: florent.colombet@renault.com

Damien Paillot
email: damien.paillot@ensam.eu

Andras Kemeny
email: andras.kemeny@renault.com

Study of latency gap corrections in a dynamic driving simulator

Keywords: Driving simulation, Simulator sickness, latency

Simulator sickness is a well-known side effect of driving simulation which may reduce the passenger well-being and performance due to its various symptoms. The present study investigated the possible effect of the latency gap correction on simulator sickness. Indeed, dynamic simulators can have different delays in their motion and visual systems (the latency gap), which may cause motion sickness. For now twenty-four participants have done two automated driving sessions in a driving simulator, with and without the correction. Simulator sickness has been estimated thanks to the Simulator Sickness Questionnaire (SSQ).

Introduction

The use of driving simulation may be limited by its well-known side effect the simulator sickness [Kol95]. This phenomenon is a specific form of motion sickness, which is quite similar to those that appear in virtual reality or transportation. Motion sickness is related to motion perception and this not only concerns physical motions but also visual motions [Het92]. With varying susceptibility, most people suffer from motion sickness in their own way, except people without organs of balance who have been shown to be insensitive to both physical and visual motion [Rea75]. Motion sickness is mostly considered as a comfort issue in transports; however, in the case of driving simulation, it also affects task performance [Mon70] and limits the duration of the simulations because of its symptoms which can range widely from pallor to vomiting [Ken93]. Although motion sickness has been extensively studied over the past decades, its causes and the mechanisms involved are not fully understood yet and several theories exist [Rea78,Ric91,Tre77]. The most prominent, the sensory conflict and rearrangement, suggests that the sickness occurs when the information delivered by the visual, vestibular and somatosensory senses are not congruent with each other and especially with the sensory patterns which are expected based on the internal models built from previous experiences [Rea78].

In this paper we intend to study a cause of discrepancies between visual and physical motion rendering, observing that there is a latency gap in dynamic driving simulators between the visual and the motion. Indeed motion systems have a delay depending on their mechanical power, mass, design, etc. And the visual system also, depending on the graphic cards and the projectors, etc. The difference between the two creates a latency gap which is supposed to be a cause to simulator sickness, because of the non-coherent sensory stimulation [Het92]. Our hypothesis is, by adding a pure visual delay to compensate the mechanical delay we will increase the motion coherence and then potentially decrease the induced simulator sickness.

Methodology

Measurement of the motion delay

The measurement of the delay of the motion system, on the Renault Ultimate driving simulator, has already been done in a previous study [Fan11]. The apparent response lag of a computer-controlled actuator system is generally composed of two terms: a pure delay and a phase delay. The pure delay corresponds to the time taken by the computer system to transfer an input information into a command for the actuator system (computation time, data buffering, numerical filters,etc.). The phase delay corresponds to the response time of the motion actuators system, and depends on the technology employed for the motion controllers (frequency and parameters of the control loop) and for the actuators themselves (load, power, and damping). These delays have been measured thanks to the software Spymove specially developed for that and an inertial unit. The results (resumed in table 1) were measured at 0,2 Hz and show a delay of 220 ms. 0.2 Hz is the average frequency of use of the simulator, this delay may vary a bit depending on the frequency of use.

Experimental protocol

Simulator setup

The experiment took place at Renault in the Ultimate dynamic simulator with the following main capabilities:

-A motion system with 8 degrees of freedom: a 6 DOF platform and two X-Y rails with 5.2 meters of usable strokes -A full cabin equipped with force feedback systems on the steering wheel and the pedals.

-A cylindrical screen covering 210 • of horizontal field-of-view (FOV) and attached to the cabin (radius of 1.9 m and centered on the driver) -Sound rendering is made inside the cabin.

Scenario

The subjects had to perform two driving sessions, one with the latency correction and the other without. These conditions have been set in a random order, between subjects, to avoid rank effect and habituation. All the driving sessions were separated by at least 18 hours to avoid the accumulation effect on the sickness.

The simulation scenario has been developed on SCANeR Studio 1.8. Subjects were placed in a Renault Espace car, in autonomous mode, on the N104 highway at 110 km/h. The traffic around the autonomous car created some situations forcing the vehicle to move around and do some accelerations and decelerations, like some work on the road, vehicle insertion, etc. This scenario lasted a bit more than 10 minutes and the subjects were asked to sit behind the steering wheel, to look around and pay attention to what happens on the road.

Measurement of the motion sickness

For now, twenty-four subjects have passed the experimentation with the two conditions. The motion sickness has been evaluated with the simulator sickness questionnaire (SSQ) [Ken93] before and after each session. At the end of each session they had to fill out a questionnaire reporting their feelings about the driving session, how did they see the scene moving, etc. And another one at the end to compare the two driving sessions, which one they prefer, etc.

Results and Discussion

For now 24 subjects have completed the experimentation, we don't have yet the results but from the comments of the participants, this correction seems to have a good impact. Some even felt an enhancement on some gaze disturbance they encountered without correction during the strong accelerations.

Here we have 145 ms of latency gap, which is quite big for a simulator, but almost imperceptible for a standard driver though it can create motion sickness after long session of driving. With the correction we have a global delay around 220 ms, which is hard for a standard driver to perceive, especially here in autonomous driving, the drivers were not active on the car comportment, so the global delay is quite transparent for them. Later, we want to do some tests in manual driving, also, to see if this delay isn't too disturbing.

Table 1 : Delay results for the simulator Ultimate

 1 The visual delay of the Renault Ultimate has been measured thanks to a specific tool from AVSimulation for measuring the delays of dynamic driving simulators running with SCANeR. It is composed of an oscilloscope connected to the supervisor computer (which controls the simulation with SCANeR) and to a photodiode placed near the lens of the projector. A specific version of the vehicle dynamics model and the visual module has been used to perform this test.As we saw in the two previous paragraphs the visual and the motion systems have different delays which create à latency gap of 145 ms. The only way to correct it and thus to creates a more synchronized sensory stimulation is to add a 145 ms transport delay on the signals of the visual system. The correction of the latency gap has been made thanks to a Matlab model. This module has been added at the end of the simulation computation and adds a transport delay on the visual signals of 145 ms.

		Pure delay (ms) Phase delay (ms)
	Hexapod	30-35	35
	Rails	15-20	200


Daniel Sadoc Menasché

Kishor Trivedi
email: ktrivedi@duke.edu

Eitan Altman
email: eitan.altman@inria.fr

Rejuvenation and the Age of Information

Two decades after the seminal paper on software aging and rejuvenation appeared in 1995, a new concept and metric referred to as the age of information (AoI) has been gaining attention from practitioners and the research community. In this vision paper, our aim is to show the similarities and differences between software aging and information aging. In particular, modeling frameworks that have been applied to software aging, such as the semi Markov approach can be immediately applied in the realm of age of information. Conversely, we indicate that questions pertaining to sampling costs associated with the age of information can be useful to assess the optimal rejuvenation trigger interval for software systems.

I. INTRODUCTION

The age of information (AoI) is a fundamental concept in networked systems [START_REF] Kosta | Age of information: A new concept, metric, and tool[END_REF], [START_REF] Li | A general model for minimizing age of information at network edge[END_REF], [START_REF] Sun | The ongoing history of age of information[END_REF]. The AoI is the time elapsed since the generation of the last successfully received message containing update information about its source system. In essence, it captures how old is the information available at each node of a network, and has important implications on the rate at which nodes should retrieve content updates. Such updates, in turn, are at the core of most applications running on mobile systems wherein users have constrained resources (e.g., battery and WiFi coverage).

Software rejuvenation has been studied for over two decades in the realm of computer systems and software [START_REF] Avritzer | Estimating the software reliability of smoothly degrading systems[END_REF], [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF]. Although there is a clear relationship between AoI and software aging and rejuvenation, previous research has not leveraged the interplay between models developed for those domains.

Software aging manifests as an increased failure rate or decreased performance in long running software systems. Naturally, the software aging leads to either a software failure or a performance slow down. One approach to deal with software aging is called software rejuvenation, where the software state is quickly refreshed. On the other side, the Age of Information is about information freshness. Connections between software aging and the Age of Information (AoI) are the subject of this vision paper:

1) mathematical methods used for software aging, such as semi Markov processes, are applicable to the analysis of age of information. Such models are discussed in Section III, 2) monitoring and discovery are related to the cost of sampling and of getting information. Fresh data has its price. For software aging and rejuvenation, as well as for age of information, one cares about the cost to get fresh data [START_REF] Hao | Economics of age of information management under network externalities[END_REF]. There is a cost to refresh a software system (and decrease software aging) or to refresh a virtual message (and decrease age of information). In Section IV we study aspects related to monitoring and sampling, 3) a number of software aging types are caused by stale information. We discuss the implications of information aging to software systems in Section V.

In the next section we provide an introduction to the interplay between software aging and age of information mechanisms. The remainder of the paper follows the outline presented in the previous bullets. Then, Section VI presents a case study, Section VII reports related work and Section VIII concludes the paper.

II. INTERPLAY BETWEEN SOFTWARE AGING AND AGE OF

A. Fundamentals of software aging

Software aging in long-running software systems is said to occur when the software system shows a significant increase in the failure rate as its execution progresses. For these systems the software performance may also show partial degradation with the progress of execution.

The software aging mechanisms reported in the software aging literature [START_REF] Grottke | The fundamentals of software aging[END_REF] are known to trigger aging related bugs (ARBs) [START_REF] Avizienis | Basic concepts and taxonomy of dependable and secure computing[END_REF], by one of several causes such as, (1) numerical error accumulation, (2) resource leakage, (3) fragmentation, and, (4) performance degradation due to shared resource (software deadlocks may occur depending on the time to live, or age, of the data lock). Some real case analysis of these software defect mechanisms are provided in [9], [START_REF] Grottke | The fundamentals of software aging[END_REF], [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF], [START_REF] Vaidyanathan | Measurement-based model for estimation of resource exhaustion in operational software systems[END_REF] and reported in Table I, such as, (1) round off accumulation errors in the Patriot missile defense system, (2) file descriptor exhaustion in Linux based software systems, (3) failure to run garbage collection in Java based systems due to memory fragmentation, (4) database deadlocks due to cycles in the request/release graph. Fig. 1. Rejuvenation and AoI (from [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF], [START_REF] Zhao | Software rejuvenation scheduling using accelerated life testing[END_REF], [START_REF] Altman | Forever young: Aging control for hybrid networks[END_REF], [START_REF] Kadota | Minimizing the age of information in wireless networks with stochastic arrivals[END_REF], respectively)

B. Measurements of software aging: static and dynamic thresholds

Software performance indicators are pieces of information that can age as a consequence of software performance degradation. Information aging is intrinsically related to rejuvenation approaches based on static thresholds to detect system performance degradation [START_REF] Avritzer | Performance assurance via software rejuvenation: Monitoring, statistics and algorithms[END_REF], [START_REF] Avritzer | Ensuring stable performance for systems that degrade[END_REF]. Using adaptive thresholds -possibly combined with statistical prediction techniquessolves the problem of aging of static thresholds. Still, the rate at which the thresholds must be rejuvenated should be determined. In [START_REF] Avritzer | Performance assurance via software rejuvenation: Monitoring, statistics and algorithms[END_REF], [START_REF] Avritzer | Ensuring stable performance for systems that degrade[END_REF], aging detection using sampling of response times and software rejuvenation methods are presented and shown to have positive impact on the customer affecting metrics. In [START_REF] Avritzer | Using performance signatures and software rejuvenation for worm mitigation in tactical manets[END_REF], [START_REF] Avritzer | Methods and opportunities for rejuvenation in aging distributed software systems[END_REF], software rejuvenation is shown to be able to extinguish worm epidemics in tactical MANETs.

C. Slow execution after restart

Many software systems require learning, hot caches, and population of internal states to operate at high-performance. For example, in operating systems, after reboots, disk access will be more frequent, because file caching will be operating in the cold state [START_REF] Yamada | Traveling forward in time to newer operating systems using shadowreboot[END_REF]. In this case, the lack of control over the Age of Information will be the source of performance degradation. As additional information is gathered about the system, e.g., as caches get populated with useful data, applications will show improved performance. Therefore, in this case a bootstrapping warm-up is required for optimal performance.

D. Dynamic versus static information

Most of the literature on the age of information assumes that information is static (e.g., news articles) and that its value degrades over time due to aging. Obsolete information should be refreshed. However, one may also consider dynamic information, which is updated locally based on prediction algorithms. Such algorithms are prone to failure due to numerical errors, and the error accumulation depends on the age of information. Therefore, for information that is dynamically updated, the uncertainty about data increases over time both due to numerical errors and due to intrinsic increase in uncertainty as the prediction horizon increases. Errors due to actions taken with outdated information have been broadly analyzed in the software aging literature [START_REF] Grottke | An empirical investigation of fault types in space mission system software[END_REF], and bridge the fields of software aging and age of information.

III. PREVENTIVE MAINTENANCE: MARKOV AND SEMI MARKOV MODELS FOR AGING AND REJUVENATION

A. Preventive maintenance: condition-based versus timebased maintenance

In the literature on preventive maintenance, Markov models are generally used for condition-based (inspection-based or measurement-based) preventive maintenance [START_REF] Chen | Closed-form analytical results for conditionbased maintenance[END_REF] while semi Markov models have been used for time-based preventive maintenance [START_REF]Analysis of periodic preventive maintenance with general system failure distribution[END_REF]. In the former, aging is indicated by a set of states, each indicating a level of degradation. Even if the sojourn time in each state is exponentially distributed, cumulative time to failure will be non-exponential and can have an overall increasing failure rate. If the system is detected to be in a low-degradation state, preventive maintenance is immediately triggered. In the latter, time to failure is considered to be a nonexponentially random variable and thus is not synthesized by a sequence of states. If this distribution as well the mean time to carry out reactive (unscheduled) recovery and the mean to carry out proactive (preventive) recovery are known then the optimal time to trigger rejuvenation can be determined.

B. Observable and non-observable states

Referring to Figure 1(a), which corresponds to Fig. 2 in [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF], Huang et al. [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF] considered the failure probable state S P , as an observable state and thus in the traditional language of preventive maintenance literature, in Figure 2 of [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF] the authors considered condition-based or inspection-based preventive maintenance (rejuvenation in this case). If this is indeed the case then we should trigger rejuvenation immediately from the S P state and not wait for any length of time (as was done in the original paper). We can alternatively consider the transition to the failure state S F from the robust state S 0 via the failure probable state S P as merely an artifice so that the overall time to failure is hypo-exponentially distributed [START_REF] Trivedi | Probability and Statistics with Reliability, Queuing, and Computer Science Applications[END_REF], [START_REF] Trivedi | Reliability and Availability Engineering: Modeling, Analysis, and Applications[END_REF]. Since the hypo-exponential distribution has a failure rate that is increasing with age, we know that, in general, in such a case, preventive maintenance could be useful. In this case, we can consider the state S P as not observable and hence the rejuvenation trigger clock should start on entry to state S 0 . This is traditionally known as time-based rejuvenation wherein the optimal time to trigger rejuvenation can be studied [START_REF] Altman | Rejuvenation and the spread of epidemics in general topologies[END_REF], [START_REF]Analysis of periodic preventive maintenance with general system failure distribution[END_REF], [START_REF] Trivedi | Reliability and Availability Engineering: Modeling, Analysis, and Applications[END_REF].

The time to trigger rejuvenation under a time-based rejuvenation can be deterministic (or generally distributed). This eventually led to a paper published in ISSRE 1995 [START_REF] Garg | Analysis of software rejuvenation using markov regenerative stochastic petri net[END_REF]. Because of the existence of a generally distributed timed transition (rejuvenation trigger) that is concurrent with the transition from S 0 to S P as well as the transition from S P to S F , the resulting stochastic process is not even a semi-Markov process but a Markov regenerative process. Thus, Fig. 1(a) (which corresponds to Figure 2 of [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF]) morphs into a Markov Regenerative Stochastic Petri Net (MRSPN) (see Figure 1 in [START_REF] Garg | Analysis of software rejuvenation using markov regenerative stochastic petri net[END_REF]).

C. CTMC for condition-based maintenance and SMP for timebased maintenance

In order to avoid the complication of working with fully non-Markovian models, we consider a clear separation between the time-based case and condition-based case. In the former we use an SMP that includes a time trigger (Fig. 1(b)) and in the latter we use CTMC where the trigger is based on state identification and not on any time trigger (Fig. 1(a)). This is what has been used in the subsequent papers [START_REF] Zhao | Software rejuvenation scheduling using accelerated life testing[END_REF], [START_REF] Bao | A workload-based analysis of software aging, and rejuvenation[END_REF], [START_REF] Vaidyanathan | A comprehensive model for software rejuvenation[END_REF].

D. Condition-based and time-based rejuvenation for AoI

In the realm of Age of Information (AoI) researchers have also used semi Markov models but in a different fashion, and there has not been a systematic comparison of the Markov and semi Markov approaches -the semi Markov approach has been used in the literature of age of information in the following references [START_REF] Ceran | Average age of information with hybrid arq under a resource constraint[END_REF], [29], [START_REF] Najm | Bits through time[END_REF], [START_REF] Najm | Status updates in a multi-stream m/g/1/1 preemptive queue[END_REF], [START_REF] Najm | Status updates through m/g/1/1 queues with harq[END_REF], [START_REF] Ornee | Sampling for remote estimation through queues: Age of information and beyond[END_REF] and the Markov approach has been used in [START_REF] Kadota | Minimizing the age of information in wireless networks with stochastic arrivals[END_REF], [START_REF] Altman | Forever young: Aging control for hybrid networks[END_REF]. The discussion in the previous sections motivates the following considerations for AoI control (see Table III

E. Markov models for aging control

In Figures 1(c) and 1(d) we illustrate two Markov models that have been used to study time-based AoI control. In those models, the states characterize the age. Markov models simplify the derivation of the optimal control strategy using Markov Decision Processes (MDPs). Alternatively, semi-Markov Decision Processes (SMDPs) can be used to analyze the control problem in the case of general residence time distributions.

In Figures 1(c) and 1(d), the age varies between 0 and 2. In Fig. 1(c) it is assumed that after a successful rejuvenation, the age will always decrease to 0 (saturated source) whereas in Fig. 1(d) a successful rejuvenation may be followed by a decrease of the age from 2 to 1, meaning that the source contained outdated information at the time of the rejuvenation (non saturated source). In both cases, note that there is only one single event of interest, namely the rejuvenation. In Fig. 1(a) and 1(b), in contrast, there are two events of interest: rejuvenation and failure, which are concurrent.

Consider a user that accesses his mobile device and consumes outdated information. If this event is assumed to be concurrent to the rejuvenation event, then we again have two events of interest: information rejuvenation and failure due to access of stale information. Under that framework, the models in Figures 1(a) and 1(b) can be used to study the dynamics of AoI. Conversely, if a software smoothly degrades over time, and the degradation level is solely a function of the time since the last rejuvenation, the models in Figs. 1(c) and 1(d) can be used to analyze software aging and rejuvenation.

In Section VI we provide a case study of an additional type of software aging, in addition to the four already mentioned in Section II, and we show the application of SMP modeling to this case study.

IV. SAMPLING, SOFTWARE REJUVENATION AND AOI

In this section we describe how software rejuvenation relates to Age of Information approaches. In particular, software rejuvenation can be used to mitigate software epidemics [START_REF] Grottke | On the efficiency of sampling and countermeasures to critical-infrastructuretargeted malware campaigns[END_REF], [START_REF] Altman | Rejuvenation and the spread of epidemics in general topologies[END_REF]. We leverage the relationship between rejuvenation and sampling established in [START_REF] Grottke | On the efficiency of sampling and countermeasures to critical-infrastructuretargeted malware campaigns[END_REF], [START_REF] Altman | Rejuvenation and the spread of epidemics in general topologies[END_REF] to indicate its applicability in the field of Age of Information, where sampling also plays a key role [START_REF] Ornee | Sampling for remote estimation through queues: Age of information and beyond[END_REF].

A. Sampling, epidemics and software rejuvenation

In [START_REF] Grottke | On the efficiency of sampling and countermeasures to critical-infrastructuretargeted malware campaigns[END_REF], [START_REF] Altman | Rejuvenation and the spread of epidemics in general topologies[END_REF], a survivability model was introduced with the goal of assessing infection and contagion probability in mission-critical systems that are the target of advanced persistent threats. The attack model used to replicate advanced persistent threats consisted of three attack phases: (1) single node, initial breach, (2) attack propagation to other nodes, and, (3) attack causes suspicious behavior. The goal of the research was to assess the effectiveness of possible countermeasures, such as, quarantine, and software rejuvenation. Three survivability models were developed, corresponding to each attack phase, in the context of advanced persistent threats. They were designed to capture the effect of neighbor nodes on different system metrics related to advanced persistent threats, such as, infection probability, and contagion probability.

An approach related to software rejuvenation was implemented to restore nodes to non-infected states. The authors have found that: (1) malware infections related to advanced persistent threats could be mitigated by using quarantine, software rejuvenation, and, node sampling for infections, (2) sampling plays a key role in software rejuvenation, as it is important to know the system state, to determine when/if to rejuvenate, and, (3) software rejuvenation and quarantine are able to extinguish the network infection, when exogenous infection rate is small.

B. Sampling, epidemics and AoI

In the context of Age of Information, infection probability can be seen as an Age of Information metric that is susceptible to software epidemics. As the epidemic spreads in the network, the opportunity for epidemics mitigation ages, if adequate countermeasures are not promptly activated. Therefore, there is a game theory aspect to worm epidemics that is reflected in the tradeoff between pro-actively activating countermeasures and their impact on software performance, as shown in [START_REF] Grottke | On the efficiency of sampling and countermeasures to critical-infrastructuretargeted malware campaigns[END_REF], [START_REF] Altman | Rejuvenation and the spread of epidemics in general topologies[END_REF].

In the realm of the age of information, epidemics are related to the spread of information, e.g., across users timelines. An example of negative epidemic is fake news propagation in social networks. Rejuvenation, in turn, is one possible countermeasure against fake news. The methods developed in [START_REF] Grottke | On the efficiency of sampling and countermeasures to critical-infrastructuretargeted malware campaigns[END_REF], [START_REF] Altman | Rejuvenation and the spread of epidemics in general topologies[END_REF], together with the timeline model discussed in [START_REF] Giovanidis | Performance analysis of online social platforms[END_REF], set the groundwork to analyze the interplay between rejuvenation and AoI, noting that rejuvenation corresponds to the sharing of a positive post which removes a fake news from the top of the timeline of a user. The age of fake news increases as a function of the rate at which positive posts are shared.

V. SOFTWARE AGING TYPES AND RELATIONSHIP TO AOI

In this section we describe how Age of Information concepts can help derive novel software rejuvenation schemes by iden-tifying the Age of Information metrics that are related to the four software aging types introduced in Section II, namely: (1) numerical error accumulation, (2) resource leakage, (3) fragmentation, and, (4) performance degradation due to a shared resource (software deadlocks may occur depending on the time to live, or age, of the data lock). These were taken from the software aging literature [START_REF] Grottke | The fundamentals of software aging[END_REF]. We expect that by identifying the Age of Information metric new methods that are focused on Age of Information and its rejuvenation will be developed.

A. Numerical error accumulation

In the Patriot missile battery failure example [START_REF] Grottke | The fundamentals of software aging[END_REF], the Age of Information metric is the error in system time computation since boot time. As the system ages, this Age of Information metric increases. The missile failure was related to an error in the system time computation that involved the use of a 24 bit fixed point register truncation, which created a small computer arithmetic error. However, this small error, when multiplied by the large number representing the elapsed time since boot time, created a large enough error that caused the Patriot missile battery to fail its mission of tracking and intercepting an incoming Scud missile. This failure occurred once the Patriot missile battery was up for over 100 hours. As a mitigation procedure, a software rejuvenation could be triggered to reduce the error in system time computation since boot time.

B. Resource leakage

Software aging as a consequence of resource leakage has been surveyed in detail in [START_REF] Cotroneo | A survey of software aging and rejuvenation studies[END_REF]. Specifically, the following resources have been reported in the software aging literature to show leakage: memory, file descriptors, sockets, space consumption in data storage systems, locks, threads and processes. A resource of type X is active if there is at least one process using that resource. The related Age of Information metric for a resource type X is the number of resources of type X that are active in the system at time t.

C. Fragmentation

Memory fragmentation in Java-based systems can impact the performance of its garbage collection engine. Memory fragmentation occurs when the free memory blocks sizes are not large enough to satisfy memory requests. Out-of-memory error is a Java exception that is thrown when memory requests cannot be met. The Age of Information metric for memory fragmentation is the degree of fragmentation. When the degree of fragmentation is outdated, the garbage collection engine may be activated more frequently causing system slow down. Some approaches that could be envisioned for software rejuvenation in this case are related to better tuning of the Java garbage collector [START_REF] Oaks | Java Performance: The Definitive Guide[END_REF]. In addition, approaches for rejuvenating a memory fragmented system are described in [START_REF] Cotroneo | A survey of software aging and rejuvenation studies[END_REF].

D. Performance degradation due to shared resource

The age of information metric related to Performance degradation due to data corruption or deadlocks is waiting time for accessing shared resource X. As the contention for a shared resource increases, the expected waiting time for that resource will show Age of Information as represented by the increased waiting time for the resource. Several approaches for software rejuvenation of shared resource are presented in [START_REF] Cotroneo | A survey of software aging and rejuvenation studies[END_REF].

VI. AGE CONTROL, SCHEDULING AND LAXITY

Next, we illustrate the applicability of the relationship between concepts from software aging and age of information in the realm of scheduling. To that aim, we indicate that the concept of laxity, which is motivated by load balancing schedulers and preventive maintenance, is also helpful for the analysis of age of information (see Table IV).

A. Laxity

In real-time systems, an important metric is the probability a task will meet its deadline, given the task laxity. Laxity is defined as the task slack, or the difference between the deadline time and the total remaining task execution time, if the task is scheduled for immediate execution. In the realm of information system, the deadline associated to a piece of information may correspond to the instant by which it must have been refreshed, as by that time the value of stale information vanishes to zero.

In a time slotted system, if an event of interest (e.g., task completion or information update) does not occur at a given slot, at that slot the age is incremented, and the laxity or the residual lifetime are correspondingly decremented. Therefore, the maximum lifetime (of information) as estimated at a given slot is given by

M oI = AoI + RoI + LoI, (1)
where RoI refers to the expected residual lifetime of information (e.g., time until refresh), LoI refers to the laxity of information, and MoI refers to the maximum lifetime tolerated for that piece of information (see Fig. 2).

A successful rejuvenation of a piece information requires, for instance, Internet access (e.g., through a WiFi network which may be intermittently available). Therefore, the expected time until successful rejuvenation is the mean time to perform a scheduled rejuvenation, e.g., accounting for the uncertainties about network access.

B. An approach for aging detection and rejuvenation control

We next illustrate an approach for aging detection and rejuvenation control. The approach monitors an Age of Information metric (AoI-related metric), as for example laxity, and activates rejuvenation to ensure the system will meet its hard real-time deadlines [START_REF] Avritzer | A load sharing interconnection network for hard real-time systems[END_REF].

Given a piece of information, the following mechanism can be used to control the AoI.

Age of Information Rejuvenation Algorithm 1) For every piece of information i, if LoI(i) < LaxityT hreshold(i) trigger AoI rejuvenation. 2) At every time slot boundary: a) for every piece of information i, increment AoI(i). b) if AoI(i) reached its target value trigger AoI rejuvenation. c) decrement LoI(i) if the expected time to refresh information i is delayed (e.g., due to unforeseen network unavailability). Note that the rejuvenation at step 1) is executed in a preventive fashion, given that conditions are not-favorable and the process of rejuvenation should be triggered immediately. Otherwise, the deadline may not be met. At step 2b), in contrast, rejuvenation is triggered in a reactive fashion, according to a pre-established schedule.

C. Aging and real-time task schedulers

Next, we draw a parallel between the algorithm presented in the previous section and classical algorithms for task scheduling in multi-processor systems. Our aims are to (i) illustrate some results from the multi-processor task scheduling literature that may inspire AoI rejuvenation algorithms and (ii) in some instances, indicate more directly that age of information plays a key role in the problem of task scheduling.

1) General remarks about task scheduling: When a task is not scheduled for immediate execution, the task probability of meeting its deadline will typically decrease as a function of time [START_REF] Avritzer | A load sharing interconnection network for hard real-time systems[END_REF], [START_REF] Van Moorsel | Analysis and algorithms for restart[END_REF], [START_REF]Analysis of restart mechanisms in software systems[END_REF]. The AoI rejuvenation algorithm presented above can be adopted for general task scheduling as follows. Consider a set of tasks, I, which are described by a minimum allowed LaxityT hreshold(i). At every time slot, for each task i, either L(i) or R(i) are decremented (where L and R are the laxity and expected residual life of tasks, respectively). If any L(i) < LaxityT hreshold(i), the AoIrelated rejuvenation procedure is triggered.

AoI-related rejuvenation, in the system model considered in the remainder of this section, consists of activating the load balancing scheduling algorithm to search for a better processor match to meet the tasks laxity requirements. Note that such a rejuvenation implies also a rejuvenation in the age of information (as information about processes will be updated). Nonetheless, it may also imply additional overhead (such as task migration) [START_REF] Avritzer | A load sharing interconnection network for hard real-time systems[END_REF], [START_REF] Van Moorsel | Analysis and algorithms for restart[END_REF], [START_REF]Analysis of restart mechanisms in software systems[END_REF], [START_REF] Vaidyanathan | A comprehensive model for software rejuvenation[END_REF], [START_REF] Eto | Determining the optimal software rejuvenation schedule via semi-markov decision process[END_REF].

2) System model: To make the discussion that follows concrete, we borrow a typical system model used for the analysis of scheduling of tasks in a multi-processor system. The system model considered in this section follows [START_REF] Avritzer | A load sharing interconnection network for hard real-time systems[END_REF] and consists of the following features:

• the system is composed of n fast processors connected by a load balancing fast packet switching network, • the system is synchronous, time slotted, • a load balancing scheduler takes into account number of busy slots in each processor and task laxity, • tasks arrive at local nodes according to a Poisson process,

• tasks cpu execution times are discrete,

• each cpu supports a maximum number of busy slots,

• tasks are scheduled for execution at a local cpu or at a remote cpu using a load balancing scheduling network. The load balance scheduling network used for real-time scheduling employs three stages [START_REF] Avritzer | A load sharing interconnection network for hard real-time systems[END_REF]: (1) a sort network stage to sort task allocation requests and cpu offers using laxity and number of busy slots, (2) a matching network stage to match laxity requests to number of busy slots per cpu, (3) a selfrouting network to deliver task matches to the scheduled cpu.

3) Markov modeling, M k /D/n: In [START_REF] Avritzer | A load sharing interconnection network for hard real-time systems[END_REF] a slotted hardreal time system with n processors was modeled as an M k /D/n Markovian model to obtain lower bounds on the average fraction of tasks that are dropped because they miss the hard real-time deadline. Tasks were modeled with laxity requirements and cpu demands as described in the system model above. The state of the Markov chain was defined as S = (N 0 , N 1 , ..., N n ), where N k is the number of busy slots in processor k at given time instant. The resulting Markov model was solved numerically and shown to have a good accuracy when compared against realistic simulation results.

4) Semi-Markov modeling: In Section III, we introduced the application of SMP to software aging and rejuvenation that includes a time trigger, as presented in [START_REF] Vaidyanathan | A comprehensive model for software rejuvenation[END_REF]. In the hard real-time system considered in this section, the time trigger is represented by the expiration of the time slot and the associated re-computation of the remaining tasks laxities and number of busy slots per processor. The activation of the rejuvenation trigger is modeled by state R as shown in Figure 1(b).

In [START_REF] Eto | Determining the optimal software rejuvenation schedule via semi-markov decision process[END_REF], the authors have formulated an optimization problem by using semi-Markov decision process and have applied it to model a dynamic software rejuvenation policy for a multistage degradation system. They have shown that the proposed control-limit policy is optimal. The approach used was to sample the state of software system, and to determine the best times to initiate the software rejuvenation, taking into account the expected operational cost. In the context of hard-real time system and Age of Information, we expect that the proposed approach in [START_REF] Eto | Determining the optimal software rejuvenation schedule via semi-markov decision process[END_REF] can be used for developing an optimal control limit approach based on sampling of the remaining laxity and using as expected operational cost the probability of the task missing the hard-real time deadline.

5) Blackbox modeling: In [START_REF] Van Moorsel | Analysis and algorithms for restart[END_REF], [START_REF]Analysis of restart mechanisms in software systems[END_REF] an approach for using task restarts was modeled with the objective to create an optimal strategy where the expected task remaining computation time might violate the task laxity requirement. The authors have assumed that tasks completion times were independent between successive attempts and have formulated an analytic model to derive the probability that a given task will meet its hard real-time deadline. We envision that the approach introduced in [39], [START_REF]Analysis of restart mechanisms in software systems[END_REF] could be used in the realm of Age of Information rejuvenation, following an approach similar to the one introduced in this section.

VII. RELATED WORK

The terminology used in the fields of age of information and software aging suggests that there is significant potential synergy between the two. Nonetheless, the only reference to rejuvenation in the literature about age of information (AoI) appears in [START_REF] Li | A general model for minimizing age of information at network edge[END_REF], where the authors refer to Juventas as the ancient Roman goddess for youth and rejuvenation. 1As the age of information concept (which dates back to 2010) is much more novel than that of software aging and rejuvenation (which dates back to the 1994-1995 timeframe [START_REF] Avritzer | Estimating the software reliability of smoothly degrading systems[END_REF], [START_REF] Huang | Software rejuvenation: Analysis, module and applications[END_REF]), it is not surprising that the notion of age of information has not been reported in the literature of software aging [START_REF] Cotroneo | A survey of software aging and rejuvenation studies[END_REF]. The workshop on software aging and rejuvenation is in its 11th edition [START_REF] Wosar | Workshop on software aging and rejuvenation[END_REF], whereas the workshop on age of information is in its 2nd edition [START_REF] Aoi | Age of information workshop[END_REF]. We envision that as the two fields mature, and the two communities interact, the interplay between the two will also become clearer and evolve.

VIII. CONCLUSION

In this paper we have considered the applicability of software aging and rejuvenation research to the Age of Information paradigm. We have presented a brief summary of the interplay between software aging and age of information by considering the several aspects of software aging and rejuvenation research, as for example, the types of aging reported in the literature, measurement of software aging and slow execution after restart. We have also considered dynamic vs. static information for software aging. In each case, we have identified the Age of Information metric that could be used to support new software rejuvenation approaches based on the Age of Information concept.

We have provided an overview of semi-Markov approaches toward aging and rejuvenation and have provided guidance on when to use CTMC, for condition-based approaches, or SMP, for time-based scheduling of rejuvenation or refresh. We have also presented a case study, for hard-real time systems, which introduced an additional type of software aging and rejuvenation based on Age of Information concept, and we have presented an overview on how to transition to Age of Information concepts from the four software aging types considered in this paper.

This work opens up a number of interesting directions for future work. Intentional aging, for instance, may occur both for age of information (to increase click rates and revenue) as well for software aging (to increase the selling of new products). We envision that game theory can be instrumental to study those aspects, which are left as subject for future work.

Fig. 2 .

 2 Fig. 2. Age of Information (AoI) and residual lifetime of information together with laxity produce the maximum tolerated lifetime.

  ): • condition-based AoI control: if the controller receives signals about the state of the network and about the state of the source, a Markov model is adequate to capture the state dynamics; • time-based AoI control: if the controller actions are based on triggers that are dependent on the time that has elapsed since the previous refresh, a semi-Markov model is adequate to capture the age of information.

The corresponding ancient Greek goddess for youth and rejuvenation is called Hebe serving nectar and ambrosia (rejuvenation techniques) to keep gods (important computer systems) forever young.

Acknowledgment: This work was partially supported by CAPES, CNPq, eSulab Solutions, Inc. and FAPERJ, as well as by the joint Brazilian-French team THANES (jointly sponsored by FAPERJ and INRIA).

INFORMATION

In this section we provide a summary of software aging mechanisms described in the literature and describe the interplay between software aging and age of information mechanisms in real software systems.