domain: qfin stat

text:
Loann D Desboulets
email: loann.desboulets@univ-amu.fr

Sparse Manifolds Graphical Modelling with Missing Values: An Application to the Commodity Futures Market

Keywords: Non-parametric, Non-linear Manifolds, Variable Selection, Neural Networks 1

This paper is devoted to practical use of the Manifold Selection method presented in Desboulets (2020). In a first part, I present an application on financial data. The data I use are continuous futures contracts underlying commodities. These are multivariate time series, for the period 1985-2020. Representing correlations in financial data as graphs is a common task, useful in Finance for risk assessment. However, these graphs are often too complex, and involve many small connections. Therefore, the graphs can be simplified using variable selection, to remove these small correlations. Here, I use Manifold Selection to build sparse graphical models. Non-linear manifolds can represent interconnected markets where the major drivers of prices are unobserved.

The results indicate the market is more strongly interconnected when using non-linear manifold selection than when using linear graphical models. I also propose a new method for filling missing values in time series data. I run a simulation and show that the method performs well in case of several consecutive missing values.

Motivations

Linear model are often not flexible enough to depict correlations among random variables.

The recent success of non-parametric approaches shows that more general model are better suited to represent the data. A simple statistical tool for representing these correlations is known as the "Graphical Model". However, a non-parametric approach requires a lot of data.

And it is a common issue when collecting data, that part of them are missing. In this paper, I propose a new method for estimating non-parametric graphical models, in the context of financial markets. I solve the missing values problem by exploiting a non-parametric time series model, proposed by [START_REF] Zimmermann | Modeling large dynamical systems with dynamical consistent neural networks[END_REF], to impute missing values. I compare this approach to others methods, and show that in simulations it has better performance in recovering the true missing values for time series data.

Graphical models are often used in Finance, from modelling interdependences of international markets [START_REF] Wang | Sparse seemingly unrelated regression modelling: Applications in finance and econometrics[END_REF][START_REF] Lafferty | Sparse nonparametric graphical models[END_REF][START_REF] Arbia | Modeling inter-country spatial financial interactions with graphical lasso: An application to sovereign co-risk evaluation[END_REF] to risk management [START_REF] Millington | Robust portfolio risk minimization using the graphical lasso[END_REF]. Taking into account contemporaneous correlations in the basis of optimal portfolio theory, and the graphical model is a simple visual tool in this respect. Here, I am interested in a specific type of assets called "Futures". Futures are standardized financial contracts meant to deliver a pre-specified amount of underlying, and is traded on the stock exchange for a future date. The delivery date can vary from a contract to another, and is referred to as its "maturity". The data I can observe on the market are the time series of prices and volumes for various maturities, where the underlying are various commodities. It is usual not to consider prices but rather their changes, called the returns, to avoid non-stationarity issues. It is very likely that the returns of a single commodity at different maturities are linearly correlated. There is no reason that the price of copper for a delivery in 2 months vary very differently from its price for a delivery in 3 months.

However, the linearity assumption might not hold for correlations between different commodities, or between returns and volumes. Therefore, the question is: are there non-linear correlations between futures returns and volumes of different commodities? I will consider a non-parametric graphical model to answer this. Also, to find the most important ones, I will use a sparse graphical model. A sparse model gets rid of insignificant/small correlations. That is useful in presence of high dimensional data, which is the case here, because non-parametric inference suffers the curse of dimensionality. This problem is then limited with a sparse model, because it lowers the number of parameters to be estimated.

First of all, a graphical model is based on a graph, denoted G, which is composed of edges and vertices. Each vertices correspond to a random variable, while the edges represent the dependences that hold in that distribution. Another information that is often added to the graph, is the strength of the dependences. That can be represented by the thickness of the edge. The thicker the edge, the stronger the correlation. One of the simplest linear graphical model is the Gaussian Model

X ∈ R p ∼ N (µ, Σ).
The graph is given by the inverse linear covariance matrix Σ -1 , also called the precision matrix. This matrix is used as the adjacency matrix of the graph, the vertices represent the variables, and an edge between the variables X i and X j is drawn if Σ -1 i,j is non-zero. Using an empirical estimate of Σ leads to fully connected graphs. Indeed, it is numerically improbable to obtain exact zeros in Σ-1 . That is why a sparse graphical model is required. Estimating a sparse graphical model turns out to be equivalent to solving a variable selection problem [START_REF] Banerjee | Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data[END_REF]. One very well-known solution to variable selection has successfully been applied to sparse graphical modelling is the LASSO [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]. The resulting method is known as Graphical LASSO [START_REF] Friedman | Sparse inverse covariance estimation with the graphical lasso[END_REF]. It operates variable selection on the linear precision matrix. So solving the Graphical LASSO accounts only for linear dependences. A non-parametric extension to the Graphical LASSO is the NonParanormal distribution (hereafter NPN) of [START_REF] Lafferty | Sparse nonparametric graphical models[END_REF]. The authors assume that a nonparametric, but monotonic, transformations f j (X j ) of the data follow the Gaussian Graphical Model.

f (X) = (f 1 (X 1 ), f 2 (X 2 ), • • • , f p (X p )) ∼ N (µ f , Σ f ).
(1)

The Graphical LASSO is then applied to Σ f . The resulting graph is more general than the one from Graphical LASSO. Still, it is not fully non-parametric because it only allows monotonic transformations. It can exist complex dependences that will not be taken into accounts. In their paper, the authors propose a another fully non-parametric approach but the graph is not arbitrary sparse. It has to be acyclic, aka a tree. Despite being very useful in certain cases, in my case it is unlikely that the true correlations form a tree. So I only compare to their first approach, which is NPN. A more general graphical model has to allow for nonmonotonic transformations. For instance, consider non-monotonic transformations of two random variables (f 1 (X 1 ), f 2 (X 2 )). This means that their dependence is a "manifold". The equation of a manifold M is written as a composition of two functions f and g.

X 2 = f -1 2 • f 1 (X 1 ). The term f -1
M :={x ∈ R p |x = f • g(x)} g : R p → R r , f : R r → R p , p > r
(2) Also it does not impose conditional normality. Already from visual inspection, I have been able to observe non-linear manifolds in the data, as illustrated in Figure 1. The figure on the left is a striking example of non-linearity, while the picture on the right shows a dependence that is more complex than a function. These are contemporaneous data, i.e. at the same date. I do not expect to find such strong correlations with cross-correlations on the long run, otherwise there would be many possible arbitrages on the market. The manifold can also be interpreted using a latent structure τ , which represents its coordinates. The data are simply projections of these coordinates into the ambient R p space.

X = f • g(X) ⇔ X = f (τ ) = E[X|τ f (X) = τ ]
where τ is the low dimensional coordinate of greatest length, such that the reconstruction best fit the data:

τ f (X) = sup τ τ : X -f (τ ) F = inf µ X -f (µ) F
And the latent structure represents unobserved variables, correlated to the observed ones, through non-linear functions. From an economic viewpoint, τ is proxy for unobserved variables such as decision-making processes of agents intervening in the market, regime changes, etc... The observed variables are only the prices and purchased volumes of futures contracts.

It is very natural to assume that prices' movements and volumes are determined by common agents' decisions. If these decisions involve some non-linearities, then there will be non-linear manifolds in the data.

Then, there is the missing data issue. et al. (2005). This model produces a complete time series that replicates exactly the prices and volumes at observed values. There also exist several other alternatives to model the data, reviews can be found in [START_REF] Harel | Multiple imputation: review of theory, implementation and software[END_REF] and [START_REF] Murray | Multiple imputation: A review of practical and theoretical findings[END_REF]. Contrary to the existing methods, the DCNN approach takes into account the dynamics of the series and is fully non-parametric. Still, I compare different alternatives on simulated time series and look for the one that best recover the true missing values. The results of these simulations indicates that DCNN performs better than any alternatives.

The rest of the paper is organised as follow. In section 2, I develop two graphical model estimators based on the two variable selection estimators proposed in Desboulets (2020).

In section 3, I present the data and how to impute missing values. I use simulations to compare existing methods and the DCNN approach. In section 4, I present empirical estimates of the graphical models and compare the different estimators. As a result, non-linear graphical models show more inter-connected markets than linear graphical models. The manifold graphs show there is a strong dependence between returns and volumes of different commodities. This finding suggest that these dependences are very non-linear and not well represented as functions but rather as manifolds.

Sparse Graphical Models via Selection on Manifolds

In order to obtain a graph with non-linear manifolds, I use the Local Linear Manifold Selection (LLMS)1 and the Diagonal Auto-Encoders Manifold Selection (DAMS) proposed in [START_REF] Desboulets | Non-parametric Variable Selection on Non-linear Manifolds[END_REF]. I recall their definitions:

LLMS ⇔                                                N i,k (x i ) = x ∈ X : x -x i ≤ x -x i (k) R N i,k = Cor (N i,k ) R N i,k = (VLV ) N i,k V N i,k = V N i,k 1 |(VL 1/2 ) N i,k |>θ ŜN i,k ,θ = j|V N i,k (j, s) = 0, ∀s = j Ψ N i,k ,θ = 1 j∈ ŜN i,k ,θ P θ = 1 n n i=1 Ψ N i,k ,θ θ * = argmax θ (V ar [P θ ]) . DAMS ⇔                                    X = XW s Z (0) ≡ X Z (l) = tanh(Z (l-1) W l ), ∀l ∈ 1, • • • , n h -1 X = Z (n h -1) W n h X = XW s E = X - X W s = argmin W 1 ,...,Wn h ,Ws T race(E E).
As shown by [START_REF] Banerjee | Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data[END_REF], a sparse graph can be obtained via variable selection.

However passing from one to the other depends on the selection operator. A graph is represented by a p × p square matrix (the so-called adjacency matrix), but both LLMS ans DAMS outputs selection probability vectors of length p. The LLMS estimator gives P θ * and the DAMS estimator gives diag(W s ). So I have to modify the output in order to obtain a p × p square matrix. Also, because the graph is undirected, that square matrix has to be symmetric. For LLMS, the modification is simple. In the original algorithm, a sparse correlation matrix is estimated for each neighbourhood. These sparse matrices are reduced to vectors and turned into the boolean form. Here, I just simply skip the reduction part, and directly turn the sparse matrices into the boolean form. So Ψ is no longer a vector, but a square matrix, that is used as the adjacency matrix. Then, I keep the original algorithm and average the results over all neighbourhoods. The LLMS estimator for the graph is therefore:

ĜLLMS → n i=1 Ψ N i,k ,θ .
Turning the DAMS estimator into a graph is more complicated. The selection weight is a diagonal matrix, and so it does not bear any information about pairwise dependences.

A good candidate to get that information, is the symmetric square matrix of all partial derivatives ∂X ∂ X . These are obtained by the chain rule, as a product of the derivatives of each layers. Note that, it is a non-parametric analogue to the precision matrix Σ -1 . Indeed, it measures the strength of the pairwise conditional dependences. But the derivatives are not constants, they are vectors. To summarize them, the simplest is to average. However, it might be that some dependences are non-monotonic. Therefore, the derivative of a nonmonotonic function has positive and negative values. And these values will cancel out if averaged. To avoid that problem, I average the absolute values of the derivatives. In the end, I have a symmetric square matrix that measures pairwise non-parametric correlations.

Still, it does not provide selection. The latter is brought by the vector diag(W s ). To combine both, I would like to multiply them element by element. But their dimensions are not compatible. A simple way to achieve this is turn the selection vector into a matrix as well: W = (diag(W s )diag(W s ) ) 1/2 . W takes on continuous values between 0 and 1. In the original version of DAMS, there is only one output. There is no penalty introduced, that is handled by the network itself. This means that here, it will only produce one graph. To get fair comparison with the other methods, I introduce a threshold to obtain a sequence of graphs. Using a threshold is equivalent to say: "an edge between two variables is drawn if and only if the two selection weights associated are greater than some threshold". The DAMS estimator for the graph is therefore:

ĜDAMS → W1 W>θ n i=1 1 j=n h (1 -tanh(Z (j) ) 2 ) W j
All methods then give sequences of graphs. These sequences vary with a penalty parameter. Up to now, there is little work on an optimal choice for the penalty. Theoretical results were provided by [START_REF] Knight | Asymptotics for lasso-type estimators[END_REF]; [START_REF] Zhao | On model selection consistency of lasso[END_REF]; [START_REF] Bickel | Simultaneous analysis of lasso and dantzig selector[END_REF]. However the problem of estimating it is always ill-posed. Practical solutions were given by [START_REF] Belloni | Square-root lasso: pivotal recovery of sparse signals via conic programming[END_REF] and [START_REF] Chichignoud | A practical scheme and fast algorithm to tune the lasso with optimality guarantees[END_REF], but both are quite computationally intensive.

Yet, I gave an optimal penalty choice for LLMS, which has shown to give good results in simulations. Also, to make fair comparison, it is usual in the literature to choose the graphs so that they have the same number of vertices. What I will do then, is to chose the penalty parameter of other methods, such that their graph has the same sparsity as LLMS. I will also comment the full sequences of graphs.

Empirical Data

Presentation

I focus on commodity market data, and more precisely on Futures Contracts (FC) of the oil and metal markets. These two markets are interesting because they are heavily traded, and directly connected to the real economy. The underlying of each contract is a commodity, used in the industry for a wide variety of goods. Therefore, rational decisions taken by economic agents, can induce patterns in the prices and volumes of FC. Keep in mind that I only look at contemporaneous correlations, there is no predictive power nor any possible arbitrage from these correlations. Already in Figure 1, I can see there are strong non-linear correlations. Nonetheless, having raw futures contracts is not suitable for long term analysis, since they are traded for a short amount of time. For instance, the contract "Copper April 2020" starts to be traded usually 6 months in advance i.e. November 2019, and wont be traded after April 2020 as it ceases to exist. The solution is to use continuous FC. These are chains of individual FC. The FC for a given commodity are combined into long term time series, each with a different maturity, expressed in months. I collect these data from Quandl API, for the period 1985 -2020. There are different kind of assets: major commodities from the Metal Market, major commodities of the Oil Market. On top of that I also include the S&P500 which serves as market index, and the US Treasury Bond (USTBill) as an interest rate index. In total, I have 7618 daily prices and volumes, for different maturities. For some commodities I have up to 6 months ahead contracts. In the end, I have 58 series for constructing a graph.

The prices are not used as is, I prefer to use returns as it is usually done in others studies.

The volumes are also expressed in log, to reduce their range. The series I obtain are very homogeneous in terms of moments, as described in Tables 1 and2. Each series is described by its name, an underscore, and the maturity of the contract in months. They mostly have the same moments, especially the different maturities of a given commodity. From visual inspection, prices follow very similar paths, which explains the similar moments. That supports the idea that the returns at different maturities of a single commodity are greatly and linearly correlated.

The Dynamically Consistent Neural Network approach

The Dynamically Consistent Neural Network of [START_REF] Zimmermann | Modeling large dynamical systems with dynamical consistent neural networks[END_REF] is a statespace model, and was originally designed for forecasting. The main assumption it makes is there exists a deterministic process that has produced the entire series, with no external influences, aka a closed dynamical system. That is very useful for predictions, especially at long-term. The closed system is composed of two different components, the first being the observed data X t , and the second the unobserved data S t . These unobserved vectors play two roles: (1) they constitute the hidden layers of the network that approximates the dynamics, and (2) they form hidden variables that close the system perfectly. There is no a priori on the form of the time dynamics, the model is fully non-parametric. The DCNN is represented as state-space model, with unknown transition functions f (•) and g(•):

S t = g(S t-1 ), X t = f (S t ).
(3)

Here, I use DCNN as a generating process to construct an exact copy of the observed data.

And since the model is based only on an initial state, the reconstructed series are complete series. The missing value problem is therefore solved. However there exist multiple solutions that solve the DCNN. To take this problem into account, I run the DCNN several times and obtain a distribution of possible values. The imputed value in the end is the median of that distribution. The DCNN imputation method is illustrated in Figure 3.2.1.

Fig. 2. Missing values imputation via DCNN

  x 1,t-3 x 2,t-3 x 3,t-3 N A     x 1,t-2 N A x 3,t-2 x 4,t-2     N A x 2,t-1 N A x 4,t-1     x 1,t x 2,t x 3,t N A   X t-3 X t-2 X t-1 X t time Xt-3 Xt-2 Xt-1 Xt S 0 S t-3 S t-2 S t-1 S t g(•) g(•) g(•) f (•) f (•) f (•) f (•)

Performance of filling methods through simulations

I run a simulation to test if DCNN is a valid method for imputation. I will also compare it with more standard methods, such as median-value imputation and previous-value imputation. These are unlikely to achieve what I want, which is to recover the "true" missing values. Previous-value imputation implies static prices from one day to the other. But for some series, there are several missing values in a row. It is unlikely that prices stagnate for several days. I will also compare with 3 more recent methods: Predictive Mean Matching (PMM) of [START_REF] Vink | Predictive mean matching imputation of semicontinuous variables[END_REF], MissForest of [START_REF] Stekhoven | Missforest-non-parametric missing value imputation for mixed-type data[END_REF] and k-Nearest Neighbours of [START_REF] Troyanskaya | Missing value estimation methods for dna microarrays[END_REF]. They are all available on the software R, respectively from packages mice, missForest and impute. These are more complex than median-value and previous-value, they make use of multivariate information to interpolate the data.

I simulate a multivariate time series X t and S t according to equation 3, with f (x) g(x) = Atanh(Bx) where A and B are random square matrices uniformly distributed on [-5, +5].

This parametrization is convenient as it can produce any smooth non-linear functions f and g, but always different at each run. I only keep the series X t , and I remove a certain percentage of the data. These series are standardized to have mean zero and standard deviation one. I chose two sample sizes, 500 and 5000, and p = 50 variables. The ensemble is made of 100 networks. The different levels of overall missing values are 5%, 10% and 50%. This corresponds, in average, to pairwise missing values levels of 40%, 65% and 100% respectively.

That is in line with our data, that has 4.2% overall and 46.4% pairwise missing values. Thus, it supports the Missing At Random hypothesis. The results of the simulations are reported in Figures 3 and4, and in Table 3. On average its MSE is around 0.25. The previous value imputation is certainly the worst, having a MSE over 0.45. Still, the three alternatives are all better than the benchmark.

PMM performs significantly better with more observations, and even with a high number of missing values. MissForest has also a great performance, but when the percentage of missing values is 5% or 10%. It is much less reliable at 50%. The last method, kNN, performs very well. This method seems to be only affected by the percentage of missing values, its performance is independent of sample size. Overall, from these simulations, I prefer to use DCNN for filling missing values on my actual data. Figure 5 shows examples of two reconstructed series. These two series are the ones having the most missing values of all the dataset. The paths exhibit various patterns, visually very consistent with the other data. One should also note that the distribution is very narrow, the Ensemble of DCNN produces very similar paths.

If ever the imputation is in fact inconsistent with the true data, the graph made with original data may differ a lot from the graph made on reconstructed data. Because if the imputation is biased, the imputed data may not lie on a manifold. Keep in mind that DCNN never imposes a manifold constraint, data can have correlations, or not. They simply follow the dynamics.

So, a biased imputation may affect the selection. On the opposite, if the reconstructed data is close to the true values, and that a manifold exist, the two graphs should not differ that much. So I will do both graphs, and compare them. On the returns' side, the oil market (gas oil + crude oil) is strongly correlated for all maturities. Also copper exhibits strong correlations among all its different maturities. Gold

shows the same dependences as copper, in addition with some stronger correlations to the 1 month and the 2 months silver and the 1 month platinum. All remaining dependences between returns all smaller ( something) and concerns all maturities of gold, copper, gas oil, crude oil and silver.

On the volumes' side, there are overall as much correlations as in returns. However, for a given commodity, the dependences among its different maturities are very low. There are stronger links between commodities, compared to the returns. The interdependences between the oil market and 3 months up to 5 months gold quite high ( something). I also observe strong dependences between the oil market and the 2 and 3 months S&P500.

The overall pattern shows dependences in majority for gold, copper, gas oil and crude oil.

For returns these commodities are "self-correlated", there are strong links between the different maturities of a commodity but not much with other commodities. For volumes it is the opposite. The commodities are linked between each others but not much with themselves.Now, keep in mind that we only look at simple linear correlations. There might be dependences involving discontinuities, non-linearities, or maybe outliers.

The objective of a Sparse Graphical Model (SGM) is to reduce the complexity of the graph, by pushing down to zero the less significant dependences.

The results of all 4 estimated graphs are displayed in Figure 7. There is no simple optimal choice for the penalty parameter in Graphical LASSO and NPN. Therefore, there is an infinity of produced graphs for these two methods. However, there is a single graph produced by LLMS. Indeed, an optimal rule was proposed in [START_REF] Desboulets | Non-parametric Variable Selection on Non-linear Manifolds[END_REF] for the choice of the penalty and therefore for the choice of the optimal graph. To make fair comparison, as it is usually done in the literature, I choose the graphs for DAMS, Graphical LASSO and NPN, such that they have the same number of vertices. Based on this, I have chosen the penalty parameter so that all graphs have around 650 edges. The graph of Graphical LASSO is shown in Figure 7(a). As I could have guessed, it resembles the Gaussian graphical model, having removed all low correlations. Only remains self-dependences for returns and dependences between the oil market and the 2 months S&P500 for volumes. There is also still a significant correlation between the volumes of the 2 months S&P500, mid-long term gold (4 to 6 months) and the US T-bill.

The graph of NPN is shown in Figure 7(b). It essentially has the same strong dependences as Graphical LASSO. Still, there are three differences. First, a lot of small dependences in the returns between, gas oil, crude oil, copper, gold, S&P500 and US T-bill. Second, there is no more connections between the oil market and the S&P500 for volumes. It might be due to the fact that these connections are lower than the ones that appeared on the returns' side. Because there is almost the same number of vertices, some connections from Graphical LASSO have been removed. Third, there are small correlations between returns and volumes. Indeed, the volumes of the oil market are connected to the returns of the oil market and short-term gold (1 to 3 months). There are also smaller dependences between the volumes of US T-bill and gold, copper and S&P500. This shows that taking into account non-linearities do modify the estimated graph.

The graph of LLMS is shown in Figure 7(c). At first sight, it seems closer to the graph produced by Graphical LASSO than it is to NPN. Self-correlations among major commodities (gas oil, crude oil, copper and gold) are stronger than Graphical LASSO and NPN. The two main differences w.r.t to the two previous graphs are (1) stronger correlation between returns and volumes of the oil market and (2) a lot of moderate dependences among volumes of gas oil, crude oil, mid-term gold (3 to 5 months), 1 and 2 months S&P500 and US T-bill.

The connections between gold returns and oil market volumes are weaker that it was for NPN, but still non-zero.

The graph of DAMS is shown in Figure 7(d). Contrary to LLMS, this one is closer to NPN than it is to Graphical LASSO. However it has more connections between volumes of the oil market and S&P500 for all maturities. The main difference w.r.t LLMS is the dependences of silver, platinum and gold returns, which appear to be very strong on this graph.

Moving on the full sequences of graphs. In relative terms, the comparison between all four methods would remain the same. When looking at intermediate levels of sparsity, for example 500 or 1000 edges, two method have nearly the same differences. The comments made before would not change that much. For low sparsity, around 200 edges, the graphs all become nearly the same: they exhibit strong correlations for gold, copper, crude oil and gas oil. These correlations are mainly between their own maturities. Only crude oil and gas oil have cross-correlations. The only noticeable difference is w.r.t Graphical LASSO, that never drawn an edge between returns and volumes. Whereas the three other method indicate intermediate to high correlations. This, is a strong indicator that there are non-linear correlations between returns and volumes of many commodities.

The graphs computed on the reconstructed data are also very similar with the graphs computed on the original data, that has missing values. The latter are available in Appendix A. The general appearance of the graphs, for all methods, remains the same. This result is an evidence that the imputation method I used is valid in my setting. However, for all methods, the graphs from original data exhibit slightly stronger correlations. The edges are slightly thicker. Two possible reasons for this. First, the missing value imputation has introduced some noise, and therefore reduced the strength of the correlations coefficients.

Second, having less data points because of missing values has lead to over-fitting, and so the correlations are biased upward. Whatever the reason, the conclusion is that missing values imputation via DCNN seems to be reliable for time series.

Fig. 1 .

 1 Fig. 1. Example of two non-linear manifolds estimated on real data

Fig. 3 .Fig. 4 .

 34 Fig. 3. Comparison of estimated versus true missing values, n = 500

  The plots indicates that DCNN is a unbiased estimator of the missing values. The recovered values lies around the 45 o line. The sample size does not seem to have a great effect on the efficiency of the procedure. However the performance differ a bit w.r.t the percentage of missing values. The more values are missing the less precise DCNN is. But when comparing mean squared errors in the table, DCNN outperforms all other methods. The benchmark for comparison should certainly be the median imputation, as most series are symmetric.

Fig. 5 .

 5 Fig. 5. Examples of reconstructed series via DCNN

Figure 6

 6 Figure6shows the simple linear gaussian graph estimated with empirical covariance matrix. We observe great linear dependences among returns and among volumes, but nearly zero not in between.

Fig

  Fig. 6. Gaussian Graphical Model

Fig. 7 .

 7 Fig. 7. Comparison of estimated graphs with ∼ 650 edges

Fig

  Fig. 8. LLMS estimates for a range of penalty

Fig

  Fig. 9. DAMS estimates for a range of penalty

Fig. 10 .

 10 Fig. 10. Graphical LASSO estimates for a range of penalty

Fig. 11 .

 11 Fig. 11. Nonparanormal estimates for a range of penalty

Fig

  Fig. 12. LLMS estimates for a range of penalty

Fig. 13 .

 13 Fig. 13. DAMS estimates for a range of penalty (original series)

Fig. 14 .

 14 Fig. 14. Graphical LASSO estimates for a range of penalty (original series)

Fig. 15 .

 15 Fig. 15. Nonparanormal estimates for a range of penalty (original series)

Table 1 :

 1 Descriptive Statistics -Continuous Futures Contracts (returns) The overall proportion of missing values in the downloaded data is 4.2%. But the proportion of pairwise missing values is number of days with at least one missing observations

	Returns	Mean Sd	Min	Q1 Median Q3 Max Skewness Kurtosis
	COPPER 1	0.00 0.02 -0.15 -0.01	0.00	0.01 0.16	-0.16	11.18
	COPPER 2	0.00 0.02 -0.16 -0.01	0.00	0.01 0.20	0.21	13.78
	COPPER 3	0.00 0.02 -0.17 -0.01	0.00	0.01 0.18	0.11	13.41
	COPPER 4	0.00 0.02 -0.15 -0.01	0.00	0.01 0.24	0.31	16.72
	GOLD 1	0.00 0.01 -0.16 -0.00	0.00	0.01 0.16	0.14	28.84
	GOLD 2	0.00 0.01 -0.09 -0.00	0.00	0.01 0.10	0.06	12.04
	GOLD 3	0.00 0.01 -0.09 -0.00	0.00	0.01 0.10	-0.05	11.64
	GOLD 4	0.00 0.01 -0.16 -0.00	0.00	0.01 0.16	0.13	26.71
	GOLD 5	0.00 0.01 -0.11 -0.00	0.00	0.01 0.10	-0.03	11.35
	GOLD 6	0.00 0.01 -0.09 -0.00	0.00	0.01 0.08	-0.06	9.99
	SP500 1	0.00 0.01 -0.11 -0.00	0.00	0.01 0.11	-0.42	11.66
	SP500 2	0.00 0.02 -0.15 -0.00	0.00	0.01 0.16	0.08	21.83
	SP500 3	0.00 0.01 -0.18 -0.00	0.00	0.01 0.19	0.05	27.51
	USTBILL 1	0.00 0.01 -0.20 -0.00	0.00	0.00 0.10	-2.45	88.88
	USTBILL 2	0.00 0.01 -0.19 -0.00	0.00	0.00 0.10	-2.14	70.70
	SILVER 1	0.00 0.02 -0.20 -0.01	0.00	0.01 0.18	-0.45	11.10
	SILVER 2	0.00 0.02 -0.20 -0.01	0.00	0.01 0.13	-0.50	8.70
	SILVER 3	0.00 0.02 -0.19 -0.01	0.00	0.01 0.18	-0.41	12.39
	PALLADIUM 1 0.00 0.03 -0.34 -0.01	0.00	0.01 0.37	0.02	27.59
	PLATINUM 1	0.00 0.02 -0.33 -0.01	0.00	0.01 0.22	-0.96	38.01
	CRUDEOIL 1	0.00 0.02 -0.21 -0.01	0.00	0.01 0.26	0.11	11.01
	CRUDEOIL 2	0.00 0.02 -0.12 -0.01	0.00	0.01 0.15	-0.15	6.37
	CRUDEOIL 3	0.00 0.02 -0.12 -0.01	0.00	0.01 0.13	-0.16	6.30
	GASOIL 1	0.00 0.02 -0.17 -0.01	0.00	0.01 0.18	-0.05	9.13
	GASOIL 2	0.00 0.02 -0.20 -0.01	0.00	0.01 0.21	-0.00	9.40
	GASOIL 3	0.00 0.02 -0.12 -0.01	0.00	0.01 0.11	0.04	6.15
	GASOIL 4	0.00 0.02 -0.11 -0.01	0.00	0.01 0.10	-0.02	6.17
	GASOIL 5	0.00 0.02 -0.10 -0.01	0.00	0.01 0.10	-0.02	5.53
	GASOIL 6	0.00 0.02 -0.11 -0.01	0.00	0.01 0.10	-0.02	6.48

values. Also it means one can estimate an unbiased, but less precise, model on the incomplete data, and then use it to impute the missing values. In my case, MAR is a reasonable assumption for two reasons. First, the missing parts of the data are probably due to errors in Quandl itself when it builds the chains of FC. There is no reason to believe this occurs for specific time or asset, it should be random. Second, if you remove 5% of the data at random, you obtain in average 40% of pairwise missing values, this is consistent with the percentage I have. So I can fit a model on incomplete data to impute the missing values.

Table 3

 3

		: Mean Squared Error of reconstructed series
		DCNN Median Previous PMM MissForest kNN
	5% missing
	n = 500	0.023	0.234	0.429	0.301	0.118	0.085
	n = 5000	0.015	0.245	0.491	0.154	0.107	0.081
	10% missing
	n = 500	0.021	0.246	0.464	0.196	0.129	0.144
	n = 5000	0.015	0.251	0.454	0.185	0.113	0.135
	50% missing
	n = 500	0.033	0.246	0.453	0.294	0.239	0.232
	n = 5000	0.029	0.252	0.449	0.163	0.214	0.237

• f 1 is called a function composition. And in the multivariate case, the distribution of the data having these kind of dependences is characterised by what is called

The second version as defined in Algorithm ??, with Random-kNN.

Discussion

I presented here an application of "manifold variable selection" on financial data, using graphical models. The non-linear manifold graph is useful to represent more general correlations among time series. The results reveal more interconnected financial markets.

They especially show many correlations between returns and volumes, that were non-existent in the other types of graphs. The difference among the two manifold selection estimators proposed, essentially lies in which assets are selected. The LLMS estimator highlight correlations on the oil market, while DAMS focuses on correlations between many assets' volumes and returns of the metal market.

I also proposed a non-parametric method for filling missing values in time series data, based on recurrent neural networks. The simulations show that the method performs well when there are several consecutive missing values, which is the case in my data. A state-space model is fitted on observable data, and the model fill in the blanks. The recurrent neural network makes no a priori on the contemporaneous correlation of the data, which would bias the graph otherwise. That makes it an appropriate filling method in my situation. Still, the solutions are not unique and therefore I end up with a distribution of missing values. I simply chose to keep the median value, but alternative methods for handling that uncertainty are possible. That is left for future research.

Appendix A. Supplementary graphs on original series with missing values

Here, the sparse graphs estimated on raw data (without filling the missing values) are reported. These data have 421,519 observations out of 441,844 in total (7618 days times 58 series). That is still a large sample size. Missing values does not prevent from estimating the graphs, but still that means less observations and possibly some bias in the LLMS estimator.

That bias arises because LLMS requires computation of correlation matrices and distances on sub-samples of the data. If these sub-samples have a large proportion of missing values the correlation matrices are known to be biased and so is the selection.

Nonetheless what I observe here, are very similar sparse graphs w.r.t. to the ones estimated on reconstructed data. The analysis of the results in section 4 would be the same on these ones.