PhD Scholar, Shalini Sharma
email: shalinis@iiitd.ac.in

Víctor Elvira

Angshul Majumdar
email: angshul@iiitd.ac.in

Emilie Chouzenoux
email: emilie.chouzenoux@inria.fr

Victor Elvira
email: victor.elvira@ed.ac.uk

Deep State-Space Model for Predicting Cryptocurrency Price

Keywords: Time series analysis, deep state-space models, deep matrix factorization, Kalman filtering, Bayesian smoothing, EM algorithm, , cryptocurrency forecasting, dynamic recurrent network

Our work presents two fundamental contributions. On the application side, we tackle the challenging problem of predicting day-ahead crypto-currency prices.

On the methodological side, a new dynamical modeling approach is proposed.

Our approach keeps the probabilistic formulation of the state-space model, which provides uncertainty quantification on the estimates, and the function approximation ability of deep neural networks. We call the proposed approach the deep state-space model. The experiments are carried out on established cryptocurrencies (obtained from Yahoo Finance). The goal of the work has been to predict the price for the next day. Benchmarking has been done with both state-of-the-art and classical dynamical modeling techniques. Results show that the proposed approach yields the best overall results in terms of accuracy.

Introduction

Investopedia defines crypto-currency as "a digital or virtual currency that is secured by cryptography, which makes it nearly impossible to counterfeit or double-spend" and is built on "decentralized networks based on block-chain technology-a distributed ledger enforced by a disparate network of computers".

A defining feature of crypto-currencies is that they are usually not issued by central banking agencies like the Federal Reserve System in US, Bank of Canada, European Central Bank, or the People's Bank of China; this makes cryptocurrencies (theoretically) immune to government interventions.

The introduction of Bitcoin around 2009 and its meteoric rise led to investors infuse their funds in crypto-currencies. One major reason behind the shift in investment largely owes to the 2008 financial crisis, which subsequently led to the waning of trust in the banking system. The market capitalization of cryptocurrencies rose from less than 10 billion in 2014 to more than 2 trillion in 2021.

However, crypto-currencies are extremely volatile. To give an example, the volatility index of the most stable crypto-currency USD Tether (USDT) has been between 95 and 100 in August 2021, while that of a blue chip corporation like Microsoft (MSFT) has been around 16 in the same period; a highly volatile smallcap stock like Genworth Financial (GNW) in the same period had a volatility index less than 40. Such large volatility makes predicting cryptocurrency prices a more challenging problem than stock forecasting. The reason crypto-currencies are volatile is because they do not have any intrinsic value.

Their prices are mainly dependent on the emotion of investors, and in such a scenario, tweets from influencers can play a major role in swaying their prices; one example of how tweets from a major influencer can drive prices high or low can be seen from (R. [START_REF] Molla | When Elon Musk tweets, crypto prices move[END_REF][START_REF] Soni | Crypto investors lost $748 billion in last seven days as bitcoin, ethereum, dogecoin, others declined[END_REF]. This work addresses the most challenging problem in personal finance today -forecasting -prices. There are a few studies on this subject. Recent study [START_REF] Livieris | An advanced cnn-lstm model for cryptocurrency forecasting[END_REF], [START_REF] Ye | A relationship-aligned transfer learning algorithm for time series forecasting[END_REF] use off-the-shelf deep learning tools for predicting crypto-currency prices. Competitive survey analysis of forecasting crypto-currency using machine learning method can be found [START_REF] Derbentsev | Forecasting of cryptocurrency prices using machine learning[END_REF]. A different branch of study [START_REF] Yasir | Deep-learning-assisted business intelligence model for cryptocurrency forecasting using social media sentiment[END_REF] follows cues from social media for predicting the crypto-currency and gives more insights for estimating future prices [START_REF] Kraaijeveld | The predictive power of public twitter sentiment for forecasting cryptocurrency prices[END_REF]. One of the study uses ARCH-MIDAS framework to identify drivers of Crypto-currency volatility [START_REF] Walther | Exogenous drivers of bitcoin and cryptocurrency volatility-a mixed data sampling approach to forecasting[END_REF]. The recent study uses the predictive power of social signals, specifically user behavior and communication patterns for forecasting prices for cypto-currencies [START_REF] Glenski | Improved forecasting of cryptocurrency price using social signals[END_REF]. The main shortcoming of all the said studies is that they yield point predictions; given the volatility of the cryptocurrency market, the investor needs to know price value and the prediction uncertainty. None of the prior studies can provide that. This is the reason predicting the volatility of cryptocurrencies is a separate branch of research (Ma et al., 2020a;[START_REF] Catania | Predicting the volatility of cryptocurrency time-series[END_REF][START_REF] Kristjanpoller | A hybrid volatility forecasting framework integrating garch, artificial neural network, technical analysis and principal components analysis[END_REF][START_REF] Köchling | Volatility forecasting accuracy for bitcoin[END_REF].

Current research in cryptocurrency forecasting offers a piecemeal solutionone approach for predicting price and a separate one for predicting volatility. This is as good as comparing apples to oranges since the fundamental models and assumptions of the two approaches will be different. The paper propose a single model that yields both the point estimate of cryptocurrency prices as well as the uncertainty around the estimate.

The work is based on the classical state-space model (SSM) for time-series analysis. SSM is defined by two functions: the Markovian state model and the observation model. In the traditional approach, the models are assumed to be known, but in realistic financial forecasting applications this is never the case. This is the reason prior studies (Sharma et al., 2021;Sharma & Majumdar, 2021) proposed to learn the models instead; the two aforementioned papers are similar in principle and only vary in the introduction of an exogenous input in the model. However, these works were based on linear state-evolution and observation models and hence could only model piece-wise linear functions. In this work, the above said restriction is removed by proposing to model the underlying functions by deep learning. This results in the so-called deep SSM. The underlying SSM structure ensures that we can both predict a point estimate as well quantify the uncertainty about it, making it a perfect fit for cryptocurrency forecasting.

The proposed work introduces deep non-negative matrix factorization (deep NMF) models for learning the approximations on operators. Deep NMF [START_REF] De Handschutter | A survey on deep matrix factorizations[END_REF] is equivalent to Deep Rectified Linear Unit (ReLU) networks. There is existing literature [START_REF] Tariyal | Deep dictionary learning[END_REF][START_REF] Mahdizadehaghdam | Deep dictionary learning: A parametric network approach[END_REF]

Related Work

The objective of this work is to predict cryptocurrency prices. Although the problem is new, it is akin to the problem of stock forecasting in particular and financial forecasting in general. There are two approaches to address such timeseries modelling problems. The first approach comprises of state-space-model (SSM) and auto-regressive moving average (ARMA). These methods are usually employed in signal processing and statistical applications (e.g., in statistical ecology [START_REF] Newman | State-space models for ecological time-series data: Practical model-fitting[END_REF]). The second approach is based on machine learning methods, more specifically on recurrent neural network (RNN).

Signal processing techniques are interpretable and yield uncertainty estimates. However, the problem with SSM and ARMA is that their model parameters need to be known. The pros and cons of such assumptions have been studied for the linear [START_REF] Rankin | Kalman filtering approach to market price forecasting[END_REF] and non-linear [START_REF] Andersen | Handbook of Financial Time Series[END_REF] cases. Specification of the underlying models lead to simplistic (and restrictive) models that fail to capture the movement of stock prices. ARMA [START_REF] Rounaghi | Investigation of market efficiency and financial stability between s&p 500 and london stock exchange: monthly and yearly forecasting of time series stock returns using arma model[END_REF] and it variants like Autoregressive Integrated Moving Average (ARIMA) [START_REF] Jarrett | Arima modeling with intervention to forecast and analyze chinese stock prices[END_REF]) (Box Jenkins models in general [START_REF] Dritsaki | Box-Jenkins modeling of Greek stock prices data[END_REF]) have been widely used in financial forecasting. These too require specification of underlying parameters defining the price movement; a higher order model fits the training data but fails to generalize and a lower order model yields poor results both on training and testing data.

RNN on the other hand can learn the underlying function from the training data; thanks to their function approximation ability [START_REF] Hammer | On the approximation capability of recurrent neural networks[END_REF][START_REF] Garzon | Dynamical approximation by recurrent neural networks[END_REF]. They do not need specifying any function parameter, given enough data they can learn the underlying dynamical model. This is the reason they have been more successful in recent years for financial forecasting [START_REF] Baek | Modaugnet: A new forecasting framework for stock market index value with an overfitting prevention lstm module and a prediction lstm module[END_REF][START_REF] Kim | Forecasting stock prices with a feature fusion lstm-cnn model using different representations of the same data[END_REF]. The shortcoming of RNN is that in their vanilla form, they do not yield uncertainty estimates -we have already discussed its importance in the introduction. This is the reason, researchers are concentrating on building RNN models on probabilistic frameworks [START_REF] Rangapuram | Deep state space models for time series forecasting[END_REF]Ma et al., 2020b).

One must note the fundamental difference between our proposal and previous approaches like [START_REF] Rangapuram | Deep state space models for time series forecasting[END_REF]Ma et al., 2020b). In [START_REF] Rangapuram | Deep state space models for time series forecasting[END_REF] the state-evolution of equation of SSM is modelled as a recurrent neural network, but the observation is assumed to be known with further restrictions of linearity and incoherence. Our model embeds deep neural networks in both the state-evolution and observation models, without any restrictions.

One can assume that ours is a more generalized version of [START_REF] Rangapuram | Deep state space models for time series forecasting[END_REF]. The work (Ma et al., 2020b) is in some sense complimentary to [START_REF] Rangapuram | Deep state space models for time series forecasting[END_REF]; they embed a particle filter in an RNN thereby adding a probabilistic flavour to the otherwise deterministic latent states (see more on particle filters in [START_REF] Särkkä | Bayesian Filtering and Smoothing[END_REF][START_REF] Elvira | Adapting the number of particles in sequential monte carlo methods through an online scheme for convergence assessment[END_REF] for more details).

Several prior studies such as [START_REF] Digalakis | ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition[END_REF][START_REF] Sharma | Blind kalman filtering for short-term load forecasting[END_REF] have proposed solutions for the linear SSM when both the state-evolution and observation matrices are unknown. These techniques were able to learn piece-wise linear functions from the data; however they were not able to model arbitrary functions. The work overcomes this limitation by modelling the state-evolution and observation matrices as deep neural networks. Recent papers such as [START_REF] Liang | Why deep neural networks for function approximation?[END_REF][START_REF] Elbrächter | Deep neural network approximation theory[END_REF] are showing how deep neural networks excel over shallow networks in terms of function approximation; the work is rooted on the same.

Proposed Method

This section will discuss the proposed approach (Deep State-Space Model for Predicting Cryptocurrency Price (DeCrypt)) in detail. For convenience the proposed method will be referred by name DeCrypt.

Model Details

The proposed work is based on standard state-space model (SSM). It can be expressed as :

For every k ∈ {1, . . . , K}:

   z k = f (z k-1 ) + g(u k ) + v 1,k , x k = h(z k ) + v 2,k . (1)
The goal is to infer (z k ) 1≤k≤K , a sequence of unknown latent space vector of size N z ≥ 1 given the input (u k ) 1≤k≤K vector of size N y ≥ 1 and observed sequence (x k ) 1≤k≤K of vector of size N x ≥ 1. The work assumes process noises

(v 1,k ) 1≤k≤K , (v 2,k ) 1≤k≤K
to have a Gaussian distribution with zero-mean and covariance matrix Q and R, respectively. The covariance matrices are symmetric definite positive. Here K is the total number of data to be processed (window size in our case).

Traditional solutions to SSM required the functions f (.), g(.) and h(.) to be known. When the functions are linear, prior studies [START_REF] Digalakis | ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition[END_REF][START_REF] Sharma | Blind kalman filtering for short-term load forecasting[END_REF] proposed a solution called blind Kalman filtering; blind since the functions/matrices were assumed to be unknown. Recent extensions introduced a graphical perspective for f (still assumed to be linear), along with suitable sparse priors [START_REF] Elvira | Graphical inference in linear-Gaussian state-space models[END_REF]; [START_REF] Cox | Sparse Bayesian estimation of parameters in linear-gaussian state-space models[END_REF]; [START_REF] Chouzenoux | Graphit: Iterative reweighted l1 algorithm for sparse graph inference in state-space models[END_REF].

This work removes the linearity restriction, by embedding ReLU deep neural networks (DNNs) in place of the functions. Our model thus takes the form:

For every k ∈ {1, . . . , K}:

   z k = T 10 T 11 T 12 z k-1 + T 20 T 21 T 22 u k + v 1,k , x k = D 0 D 1 D 2 z k + v 2,k .
(2) This is a multi-linear Gaussian model; everything except the input (u k ) 1≤k≤K and observed sequence (x k ) 1≤k≤K are unknown. The primary objective is to jointly learn the latent factor matrices T 10 ∈ R Nz×Nz , T 11 ∈ R Nz×Nz , T 12 ∈ R Nz×Nz , three positive-valued linear factors leading to a multi-linear state operator T 10 T 11 T 12 , the control input transition matrices T 20 ∈ R Nz×Nz , T 21 ∈ R Nz×Nz , T 22 ∈ R Nz×Ny , three positive-valued linear factors leading to a multi-linear control operator T 20 T 21 T 22 , and the observation matrices

D 0 ∈ R Nx×Nz , D 1 ∈ R Nz×Nz , D 2 ∈ R Nz×Nz ,

Model Analysis

This section will describe the fundamental characteristics of the proposed DeCrypt approach. We have used the discrete time invariant state-space model with exogenous input. The mathematical fundamentals of the model in Eq. ( 2) and schematic diagram in Fig 1 . The former part of the equation works on the evolution of the hidden state parameters, where the model assumes Markovianity between two consecutive hidden states. The later part defines the relationship between the hidden and observed states. We depart from all prior works in SSM based dynamical modelling in the way we define the functions. Usually a matrix is used when the functions are assumed to be non-linear and an explicit non-linear function otherwise [START_REF] Andrieu | Particle markov chain monte carlo methods[END_REF][START_REF] Chopin | SMC2: an efficient algorithm for sequential analysis of state space models[END_REF][START_REF] Crisan | Nested particle filters for online parameter estimation in discrete-time state-space markov models[END_REF]). Here we model non-linearity by a deep ReLU network; alternately this can be also seen as a deep NMF [START_REF] De Handschutter | A survey on deep matrix factorizations[END_REF]. The reason for using a deep ReLU network is its universal function approximation capbility [START_REF] Liu | Optimal function approximation with relu neural networks[END_REF][START_REF] Daubechies | Nonlinear approximation and (deep) relu networks[END_REF][START_REF] Chen | Efficient approximation of deep relu networks for functions on low dimensional manifolds[END_REF]. It is essential to note here that classical state-space models uses Monte Carlo simulation or Variable Bayes type techniques, doe non-linear SSM. These techniques are complex and usually do not scale well. In contrast, in the proposed method, each layer is modeled and learnt in the form of matrix that can be estimated using an alternating majorization-minimization (AMM) procedure.

When compared to existing literature in machine learning approaches, DNN mostly utilizes in backpropagation for its training [START_REF] Chen | A deep non-negative matrix factorization model for big data representation learning[END_REF][START_REF] Flenner | A deep non-negative matrix factorization neural network[END_REF]. Consequently while modelling dynamical systems, backpropagagtion through time (BPTT) needs to be employed. We are all aware of the pitfalls of BPTT. This is the reason we resort to AMM instead. Unlike BPTT, AMM (under certain conditions) at least guarantees convergence to a local minimum [START_REF] Chouzenoux | A block coordinate variable metric forward-backward algorithm[END_REF]; [START_REF] Jacobson | An expanded theoretical treatment of iteration-dependent majorize-minimize algorithms[END_REF].

Model Inference Algorithm

The inference problem can be viewed as smoothing/filtering problem where we aim to infer probabilistic estimate of the hidden state (z k ) 1≤k≤K . In this work, we have also introduced deep NMF factors (described earlier) which are unknown. We aim to jointly infer both probabilistic distribution on hidden state and deep NMF factors estimation from the data. To estimate the state matrices, control input transition matrices and observation transition matrices, we use Expectation-maximization strategy (for more details (Särkkä, 2013, chap.12) and [START_REF] Shumway | An approach to time series smoothing and forecasting using the EM algorithm[END_REF]. The EM strategy operates in two steps namely E-step where we assume the positive latent factor matrices

φ K (T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 ) = log p(x 1:K |T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 ). (3)
It is important to note that the inference of the i + 1-th EM update is obtained from the estimates from the previous iteration i. We explain the EM algorithm in more detail.

E-step: Kalman/RTS inference

We consider the latent factors

T [i] 10 , T [i] 11 , T [i] 12 , T [i] 20 , T [i] 21 , T [i] 22 , D [i] 0 , D [i] 1 , D [i] 2
to be fixed. The objective of this step is to infer the probabilistic estimation of the state. The initial state takes the form z 0 ∼ N (z 0 , P 0 ) with z0 ∈ R and P 0 defined as definite symmetric positive matrix ∈ R Nz×Nz . The probabilistic estimation is provided by the Kalman filter through predictive distribution :

p(z k |x 1:k , u 1:k ) = N (z k ; zk , P k ). (4)
For every k, the mean zk and the covariance P k are given by the Kalman iterations:

For k = 1, . . . , K:

Predict state:

   z - k = T [i] 10 T [i]
11

T [i] 12 zk-1 + T [i] 20 T [i] 21 T [i] 22 u k , P - k = T [i] 10 T [i] 11 T [i] 12 P k-1 (T [i] 10 T [i] 11 T [i] 12 ) ⊤ + Q. (5)
Update state:

                     y k = x k -D [i] 0 D [i] 1 D [i] 2 z - k , S k = D [i] 0 D [i] 1 D [i] 2 P - k (D i 0 D [i] 1 D [i] 2 ) ⊤ + R, K k = P - k (D [i] 0 D [i] 1 D [i] 2 ) ⊤ S -1 k , z k = z - k + K k y k , P k = P - k -K k S k K ⊤ k . (6)
Hereabove, y k represents the measurement pre-fit residual, S k represents the pre-fit covariance, K k represents Kalman gain, zk represents the updated (a posteriori) state estimate, P k represents the updated (a posteriori) covariance estimate. The backward recursion from the RTS smoother allow to build the smoothing distribution p(z k |x 1:K , u 1:K ). For k = K, . . . , 1

Backward Recursion (Bayesian Smoothing):

                     z - k+1 = T [i] 10 T [i] 11 T [i] 12 zk + T [i] 20 T [i] 21 T [i] 22 u k , P - k+1 = T [i] 10 T [i] 11 T [i] 12 P k (T [i] 10 T [i] 11 T [i] 12 ) ⊤ + Q, G k = P k (T [i] 10 T [i] 11 T [i] 12 ) ⊤ [P - k+1 ] -1 , z s k = z k + G k [z s k+1 -z - k+1 ], P s k = P k + G k [P s k+1 -P - k+1 ]G ⊤ k . (7)
Consequently, for every time step k ∈ {1, . . . , K}, the RTS smoother provides:

p(z k |x 1:K , u 1:K ) = N (z k ; z s k , P s k ). (8)

M-step: Operator update

This step utilizes the estimated state (z k ) following an optimization step to increase the likelihood of the matrix parameters T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 , using the smoothed predictive distribution obtained in the E-step.

φ K (T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 ) ≥ Q(T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 ; Θ [i] ). (9)
Here above, Θ [i] = {Σ [i] , Φ [i] , B [i] , C [i] , ∆ [i] , A [i] , F [i] , I [i] } gathers eight quantities (variables) defined from the outputs of the E-step described in Sec. 3.3.1):

Q(T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 ; Θ [i] ) = + K 2 tr Q -1 (Σ [i] -(T 10 T 11 T 12 ) ⊤ C [i] -A [i] (T 20 T 21 T 22 ) ⊤ -(T 10 T 11 T 12 )(C [i] ) ⊤ +(T 10 T 11 T 12 )Φ [i] (T 10 T 11 T 12 ) ⊤ + (T 10 T 11 T 12 )F [i] (T 20 T 21 T 22 ) ⊤ -(T 20 T 21 T 22 )(A [i] ) ⊤ +(T 20 T 21 T 22 )(F [i] ) ⊤ (T 10 T 11 T 12 ) ⊤ + (T 20 T 21 T 22 )I [i] (T 20 T 21 T 22 ) ⊤ ) + K 2 tr R -1 ∆ [i] -B [i] (D 0 D 1 D 2 ) ⊤ -D 0 D 1 D 2 (B [i] ) ⊤ + D 0 D 1 D 2 Σ [i] (D 0 D 1 D 2 ) ⊤ , (10) with:
                                       Σ [i] = 1 K K k=1 P s k + z s k (z s k ) ⊤ , Φ [i] = 1 K K k=1 P s k-1 + z s k-1 (z s k-1 ) ⊤ , B [i] = 1 K K k=1 x k (z s k ) ⊤ , C [i] = 1 K K k=1 (P s k G ⊤ k-1 + z s k (z s k-1 ) ⊤ ). A [i] = 1 K K k=1 z s k u ⊤ k F [i] = 1 K K k=1 z s k-1 u ⊤ k I [i] = 1 K K k=1 u k u ⊤ k ∆ [i] = 1 K K k=1 x k x ⊤ k (11)
In this step lies the main bottleneck of the deeper extension (compared to the shallow state-space model of (Sharma & Majumdar, 2021)). For the shallow model each of the operators was a single matrix; therefore there update step resulted in a linear inverse problem. Such is not the current case; here the variables are multi-linear in nature. Therefore we do not have the simple (analytic) updates -as was the case for the shallow model. We have to resort to the paradigm of alternating direction method of multipliers (ADMM) [START_REF] Wang | Global convergence of admm in nonconvex nonsmooth optimization[END_REF][START_REF] Nishihara | A general analysis of the convergence of admm[END_REF][START_REF] Lin | On the global linear convergence of the admm with multiblock variables[END_REF] for solving updating the variables from the multi-linear form. In ADMM, the idea is that, one can update one variable assuming the others to be constant and as long as each of the variables have a closed form update, the overall optimization will reach a local minimum.

Based on the ADMM approach the computations each variable under the positivity constraints on the factors

T [i+1] 10 , T [i+1] 11 , T [i+1] 12 , T [i+1] 20 , T [i+1] 21 , T [i+1] 22
and

D [i+1] 0 , D [i+1] 1 , D [i+1] 2
leads to:

(T10) [i+1] = argmax T 10 ≥0 Q(T10, T [i] 11 , T [i] 12 , T [i] 20 , T [i] 21 , T [i] 22 D [i] 0 , D [i] 1 , D [i] 2 ; Θ [i] ) (T11) [i+1] = argmax T 11 ≥0 Q(T [i+1] 10 , T11, T [i] 12 , T [i] 20 , T [i] 21 , T [i] 22 D [i] 0 , D [i] 1 , D [i] 2 ; Θ [i] ) (T12) [i+1] = argmax T 12 ≥0 Q(T [i+1] 10 , T [i+1] 11 , T12, T [i] 20 , T [i] 21 , T [i] 22 D [i] 0 , D [i] 1 , D [i] 2 ; Θ [i] ) (T20) [i+1] = argmax T 20 ≥0 Q(T [i+1] 10 , T [i+1] 11 , T [i+1] 12 , T20, T [i] 21 , T [i] 22 D [i] 0 , D [i] 1 , D [i] 2 ; Θ [i] ) (T21) [i+1] = argmax T 21 ≥0 Q(T [i+1] 10 , T [i+1] 11 , T [i+1] 12 , T [i+1] 20 , T21, T [i] 22 D [i] 0 , D [i] 1 , D [i] 2 ; Θ [i] ) (T22) [i+1] = argmax T 22 ≥0 Q(T [i+1] 10
, T

[i+1] 11

, T

[i+1] 12

, T

[i+1] 20

, T

[i+1] 21

, T22D

[i] 0 , D [i] 1 , D [i] 2 ; Θ [i] ) (D0) [i+1] = argmax D 0 ≥0 Q(T [i+1] 10
, T

[i+1] 11

, T

[i+1] 12

, T20, T

[i+1] 21

, T

[i+1] 22 D0, D

[i] 1 , D [i] 2 ; Θ [i] ) (D1) [i+1] = argmax D 1 ≥0 Q(T [i+1] 10
, T

[i+1] 11

, T

[i+1] 12

, T

[i+1] 20

, T

[i+1] 21

,

T [i] 22 , D [i] 0 , D1, D [i] 2 ; Θ [i] ) (D2) [i+1] = argmax D 2 ≥0 Q(T [i+1] 10
, T

[i+1] 11

, T

[i+1] 12

, T

[i+1] 20

, T

[i+1] 21

, T

[i+1] 22

D [i+1] 0 , D [i+1] 1 , D2; Θ [i] )
The above sub-problems can easily be rewritten as the minimization of convex quadratic functions which can be solved through several solvers. We stick to use simple projected least-squares updates, which is also reminiscent from the literature of deep nonnegative matrix factorization [START_REF] Chen | A deep non-negative matrix factorization model for big data representation learning[END_REF], and the deep ReLu neural networks models [START_REF] Daubechies | Nonlinear approximation and (deep) relu networks[END_REF]. The deep neural network (DNN) based operators are regularized by imposing a positivity constraint on the entries of the estimated matrices, by simply projecting them onto the positive orthant after each update of the M-step (similar to ReLU activation function). This is akin to deep non-negative matrix factorization [START_REF] Trigeorgis | A deep matrix factorization method for learning attribute representations[END_REF][START_REF] Mei | Nonnegative matrix factorization with side information for time series recovery and prediction[END_REF], while keeping the convergence behaviour of the EM algorithm. DNN with ReLU activation is known for its function approximation ability [START_REF] Chen | Efficient approximation of deep relu networks for functions on low dimensional manifolds[END_REF][START_REF] Yarotsky | Optimal approximation of continuous functions by very deep relu networks[END_REF]. This yields the following analytic updates:

T [i+1] 10 = ReLu C [i] (T [i] 12 ) ⊤ (T [i] 11 ) ⊤ -T [i] 20 T [i] 21 T [i] 22 (F [i] ) ⊤ (T [i] 12 ) ⊤ (T [i] 11 ) ⊤ ×(T [i]
11

T [i] 12 Φ [i] (T [i] 12 ) ⊤ (T [i] 11 ) ⊤ )) † T [i+1] 11 = ReLu ((T [i+1] 10
) ⊤ Q -1 (T [i+1] 10
) -1 ) (T i+1 10 ) ⊤ Q -1 C [i] T [i] 12
-T

[i+1] 10

Q -1 T [i] 20 T [i]
21

T [i] 22 (F [i] ) ⊤ T [i] 12 × (T [i] 12 Φ [i] (T [i] 12 ) ⊤ ) † T [i+1] 12 = ReLu ((T [i+1] 11
) ⊤ (T

[i+1] 10

) ⊤ Q -1 T [i+1] 10 T [i+1] 11 ) † × (T [i+1] 11
) ⊤ (T

[i+1] 10

) ⊤ C [i] Q -1 -(T [i+1] 11 ) ⊤ (T [i+1] 10 ) ⊤ (Q -1 T [i] 20 T [i]
21

T [i] 22 (F [i] ) ⊤ )Φ -1 T [i+1] 20 = ReLu (T [i] 22 ) ⊤ (T [i] 21 ) ⊤ -T [i+1] 10 T [i+1] 11 T [i+1] 12
F [i] (T [i] 22 ) ⊤ (T [i] 21 ) ⊤ ×(T [i]
21 T

[i]

22

I [i] (T [i] 21 ) ⊤ (T [i] 22 ) ⊤ )) † T [i+1] 21 = ReLu T [i+1] 20 Q -1 (T [i+1] 20 ) ⊤ † T [i+1] 20 A [i+1] Q -1 (T [i] 22 ) ⊤ -(T [i+1] 20 ) ⊤ Q -1 (T [i+1] 10 )(T [i+1] 11 )(T [i+1] 12
)F [i] (T

[i] 22 ) ⊤ ×(T [i] 22 I [i] (T [i] 22 ) ⊤ ) † T [i+1] 22 = ReLu (T [i+1] 21
) ⊤ (T

[i+1] 20

) ⊤ Q -1 (T [i+1] 20 )(T [i+1] 21 ) × (T [i+1] 21
) ⊤ (T

[i+1] 20

) ⊤ Q -1 A [i] -(T [i+1] 21
) ⊤ (T

[i+1] 20

) ⊤ Q -1 × (T [i+1] 10 ) ⊤ (T [i+1] 11 ) ⊤ (T [i+1] 12 ) ⊤ F [i] (I [i] ) -1 D [i+1] 0 = ReLu B [i] (D [i] 2 ) ⊤ (D [i] 1 ) ⊤ (D [i] 1 D [i] 2 Σ [i] (D [i] 2 ) ⊤ (D [i] 1 ) ⊤ ) † D [i+1] 1 = ReLu ((D [i+1] 0 ) ⊤ R -1 D [i+1] 0 ) † ((D [i+1] 0 ) ⊤ R -1 B [i] (D [i] 2 ) ⊤ ×(D [i] 2 Σ [i] (D [i] 2 ) ⊤ ) † D [i+1] 2 = ReLu ((D [i+1] 1 ) ⊤ (D [i+1] 0 ) ⊤ R -1 D [i+1] 0 D [i+1] 1 ) † (D [i+1] 1 ) ⊤ ×(D [i+1] 0 ) ⊤ R -1 B [i] (Σ [i]] ) -1 . (12)
Hereabove, we use pseudo-inverse operator denoted by (•) † . Each operator is passed over activation function ReLu which stands for, ReLu (•) the rectified linear unit function, that projects each entry of its input to the positive orthant.

Model Summary

The Proposed algorithm is summarized in Alg.1. The algorithm infers the probabilistic estimation of the hidden state (z k ) 1≤k≤K jointly with the estima-tion of latent spaces {T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 } by following the eq. ( 2). The DeCrypt model is ran for i max number of iteration to achieve the stabilisation of the latent spaces.

Algorithm 1. Proposed model inference algorithm.

Inputs. Prior parameters (z 0 , P 0 ) ; model noise covariance matrices Q, R ; set of observations {x k } 1≤k≤K and control input (u k ) 1≤k≤K .

Initialization. Set positive latent factors

{T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 }.
Recursive step. For i = 0, 1, . . . , i max :

(E step) Run the Kalman filter (5)-( 6) and RTS smoother (7) using latent factors

T [i] 10 , T [i] 11 , T [i] 12 , T [i] 20 , T [i] 21 , T [i] 22 , D [i] 0 , D [i] 1 , D [i] 2 .
Calculate Σ [i] , Φ [i] , B [i] , C [i] , ∆ [i] , A [i] , F [i] , I [i] using (11).

(M step) Compute {T [i+1] 10 , T [i+1] 11 , T [i+1] 12 , T [i+1] 20 , T [i+1] 21 , T [i+1] 22 , D [i+1] 0 , D [i+1] 1 , D [i+1] 2 } using (12).
Output. State filtering/smoothing pdfs (4) and (8) along with pointwise estimates of the latent factor from (12).

Time complexity

▷ Training. We describe the time complexity for training DeCrypt in a given window of length τ . The complexity can be understood by delving into Kalman-based approaches [START_REF] Montella | The kalman filter and related algorithms: A literature review[END_REF], which imply complexity analysis to

O(τ N 2.376 z ).
▷ Testing. In testing phase we just perform evaluation of multi-linear equation (Equation 14). This concludes the complexity of O(N x N 2 z ) for each window. This can be further optimized to O(N 2 z ) if performing forecast for just one feature (which looks very similar to the proposed case, ie. forecasting close price.)

Application to cryptocurrency price forecasting

This section discuss in detail how the proposed approach is applied on the very challenging application of predicting next day prices for crypto-currency.

Training

The major drawback of the Expected-Maximization (EM) strategy used in the proposed approach is that it requires reprocessing on the entire sequence to estimate the state, control input, and observation transition operators / matrices. The approach requires imposing explicit prior static assumptions on these parameters for the entire duration of the sequence. Such an assumption may not be an appropriate in practice owing to the volatility of the data; furthermore, processing the entire sequence will be computationally expensive. Owing to the volatility, the parameters should be given the freedom to learn and evolve with time. We thus propose an online implementation based on a simple windowing strategy. Thus we are able to relax the non-volatility assumption on the entire sequence and reduce processing times.

In the said strategy, a window of size τ is slid on the entire dataset. For every time stamp k, the matrices in the multi-linear operators are estimated using the last τ observations contained in the set

X k = {x j } k j=k-τ +1 and U k = {u j } k j=k-τ +1 .
The proposed EM algorithm is iteratively applied on the window to update the state and operators till convergence. Such a strategy reduces the operational cost significantly. Note that the non-volatility assumption is still there, but only on a small window -this is a reasonable assumption. The major challenge is to estimate a reasonable size of τ . A smaller size would be a better approximation for the non-volatility assumption but would lead to overfitting on the multi-linear model. On the other hand a larger size would be less prone to over-fitting but would be computationally costly. We initialize the matrices of the multi-linear model using the warm start strategy. The matrices for the current window are initialized with the final values of the prior window.

Similarly, the mean and covariance are initialized for k -τ + 1 with the past information on the operators from the smoothing process.

Forecasting

As described in detail in Section Model Details, we follow the sliding window strategy. Training each observed window x k along with control-input u k , extracts latent space features and helps in updating the parameters by following EM alternately. Once EM iteration stabilizes ( we have used 50 iterations for EM to converge used in Alg.1), we use the latent space features and learned matrices to estimate the close price for the next timestamp (i.e., the day indexed

as k + τ + 1).    z k = T 10 T 11 T 12 z k-1 + T 20 T 21 T 22 u k + v 1,k , x k = D 0 D 1 D 2 z k + v 2,k ,
where input u k is (u j ) k≤j≤k+τ ∈ R 1 which is computed using the technical indicator SMA(simple moving average) 2 .Simple moving average (SMA) calculates the average of the a fixed range of prices, usually closing price by the number of period in that range.

SM A = c 1 + c 2 + ... + c n n (13)
where c n = closing price of an asset for period n and n= number of total periods

The processing sequence(observed) x k is (x j ) k≤j≤k+τ ∈ R 5 for every k ∈ {0, . . . , K -τ }, where x j [1] is the daily opening price, x j [2] is the daily adjusted close price, x j [3] is the daily high value, x j [4] is the daily low value, and

x j [5] is the daily net asset volume. Running the DeCrypt model on the considered window yields the mean estimate of the five features for the immediate 2 https://www.investopedia.com/terms/s/sma.asp time stamp which can be indexed as k + τ + 1:

x k+α+1 = D 0 D 1 D 2 z - k+α , (14)
The associated covariance matrix is defined as S k+τ for immediate next time stamp indexed as k + τ + 1. /cblueIn particular, DeCrypt is designed to forecast the whole five dimensional vector, but we focus primarily on prediction of single entry of the vector i.e., adjusted closing price of the sequence.

Uncertainty Quantification

The proposed approach is based on a probabilistic framework and hence can provide confidence score/uncertainty quantification associated with each pointwise estimation. The quantification provides predictive distribution of the future observation which is conditioned on previously seen control-input (u k ) and observed sequence (x k ). The probabilistic validation provides informed decision about the (un)certainty associated with model estimation while predicting the future prices of the cryptocurrencies. For each index k, the distribution of the prediction conditioned on x k past observations and control-input u k :

p( x k |x 1:k-1 , u 1:k-1 ) = N x k ; D 0 D 1 D 2 z - k , S k , (15)
where the covariance is defined as

S k = D 0 D 1 D 2 ((T 10 T 11 T 12 )P k-1 (T 10 T 11 T 12 ) ⊤ + Q) + R.
It is important to note that z - k and P k are achieved from Kalman filter, defined in Section E-step in DeCrypt model. The main objective of the proposed approach is to estimate the uncertainty score associated with the prediction given by model for price forecasting. In particular, the main aim is to focus on forecasting the sum of prediction i.e., estimating the first entry x k denoted by x k [0]. The quantification about the prediction can be obtained from first row and column of S k , depicted by S k [0, 0]. We define the (un)certainty score about an increase of the price forecasting value as :

p k = +∞ x k [0] N y; D 0 D 1 D 2 z - k [0], S k [0, 0] dy (16) = 1 -CDF( x k [0]| D 0 D 1 D 2 z - k [0], S k [0, 0]), (17)
where CDF depicts the cumulative distribution function for the multivariate predictive distribution. The equation above quantifies and estimates the probability that forecasting price value will grow in the future time stamp. After estimating p k for every index k, we evaluate cross entropy loss defined as :

log-loss = 1 K K k=1 -(L k [i] log( p k )) , (18)
where ground-truth is depicted as L k ∈ {0, 1} at time k to increase/decrease.

Experiments and Results

This section discuss the experimental results with proposed approach and other state-of-the art methods. The section presents qualitative and quantitative analysis, hence presenting comprehensive study for modeling time series signals.

Dataset description

In this study, we consider a cryptocurrency dataset comprising of ten cryp-

Baseline methods

• N-Beats: Nbeats is a deep neural network structure with forward and backward residual links. It consists of a deep stack of fully connected layers. It does not perform any specific feature engineering or scaling [START_REF] Oreshkin | N-beats: Neural basis expansion analysis for interpretable time series forecasting[END_REF].

• Deep Auto regressive (DeepAR) : DeepAR functions by producing probabilistic forecast on long time series sequences [START_REF] Salinas | Deepar: Probabilistic forecasting with autoregressive recurrent networks[END_REF].

• Temporal Fusion Transformers (TFT): An attention mechanism based recurrent architecture which brings together multi-horizon forecasting without having prior information on how they interact with the target [START_REF] Lim | Temporal fusion transformers for interpretable multi-horizon time series forecasting[END_REF]. • RAO-ANN: The Rao algorithms can be categorised as the metaphor-less optimization techniques which is utilized in optimizing the parameters of ANN in forecasting crypto-currency prices [START_REF] Nayak | Modeling and forecasting cryptocurrency closing prices with rao algorithm-based artificial neural networks: A machine learning approach[END_REF].

• ARIMA: Autoregressive integrated moving average (ARIMA) methodology is used to forecast cryptocurrency prices. The paper identifies the pa-rameter of ARIMA model using partial auto-correlation functions (PACF)

and auto correlation function (ACF). [START_REF] Abu Bakar | Autoregressive integrated moving average (arima) model for forecasting cryptocurrency exchange rate in high volatility environment: A new insight of bitcoin transaction[END_REF] We have compared our proposed approach with above mentioned models.

The ARIMA parameters are set to (p, d, q) = (2, 1, 2) as it was observed to lead to the best practical performance. We modified LSTM from its original version, by removing the softmax layer and instead included a fully-connected layer to obtain a one node output. The Adam optimizer has been used with learning rate of 10 -4 , 200 epochs and batch-size 16 is maintained to minimize the root mean square error. For methods like N-Beats, DeepAR, TFT, CNN-TA, MFNN, RDL, RAO-ANN we stick to their original implementation as described in respective papers mentioned above.

Proposed model Parameter analysis

The proposed approach is non-parametric. It only requires the specification of the window size; we found that τ = 50 yields good results for all cryptocurrencies. The rest of the variables require initialization. These are given below: 10-5 , 10 -1 , 10 -1 ), I : identity matrix

P 0 = σ 2 P I Q = σ 2 Q I R = σ 2 R I (σ Q , σ R , σ P ) = (
z 0 = 0 N z = 5,N y = 1, N x =5
DeCrypt (3 layers): {T 10 T 11 T 12 }, {T 20 T 21 T 22 } is estimated {D 0 , D 1 , D 2 } are estimated.
It is important to note that ignoring the positivity constraints on DeCrypt

(1 layer) would identify with the previous work (Sharma & Majumdar, 2021).

For benchmarking we have used the commons metrics for regression; they are All the said metrics are based on the error between the actual and predicted prices and a lower value implies better result. We have also computed the Pearson correlation coefficient (r) between the predicted and actual prices;

for this metric a higher value implies better result.

We are showing these results for a given window size τ = 50 and number of layers. If we continue to increase the window size, the results start to deteriorate, this is likely due to over-fitting. Increasing the window size does not help either; larger window size fails to capture the volatility of the data and hence the performance falls.

Result Analysis Discussions

In this section we focus on the performance analysis of the proposed approach. The proposed approach (DeCrypt) has been compared with various numerical methods like N-Beats, DeepAR, TFT, CNN-TA, MFNN, RDL, RAO-ANN, ARIMA, RAO-ANN.

Influence of window size and depth

The proposed solution is non-parametric. The only design parameters that need to be fixed are the window size and depth. Therefore, it is very important to choose the optimal window size which finds a good balance between the model complexity (depth) and accuracy. To better understand, we present Table 3: Comparative performance of the proposed approach against different window size for 3-layer architecture. Lower value(↓) is considered the better.

versions. This is because larger window size means more data and hence better generalisation ability. But note that it is a dynamical model. The tacit assumption here is that in the window the size the underlying dynamical function does not change. While this is true for shorter windows, this does not hold for larger ones as the non-stationarity comes into play. This is the reason we find a trade-off between window size and accuracy. We further set this value of window size in upcoming experiments.

Comparison with state-of-the-art methods

The overall performance of the model vis-a-vis the state-of-the-art is shown in Table4. The Score for all the ten crypto-currencies.

and state-of-the-art methods, we present Table 5 which depicts the computational time for forecasting the next day closing price of ten cryptocurrencies. We provide a comprehensive analysis by distinguishing the time required to train and test the methods (on their training and testing time frame as described in sec 5.1) using the walk-forward method described in (Sharma et al., 2021, Section 4.2.1). We conclude that the highest computational time was consumed by LSTM approach. Among the other methods, DeCrypt (1 layer) and RDL method is the fastest while the computation time of DeCrypt (3 layers) is very much comparable to DeepAR and TFT. However, note that the existing algorithms are optimized to take advantage of GPU, the proposed approach does not, it runs only on the CPU. It may be possible to improve the performance in the future through parallelization.

Uncertainty quantification

The advantage of DeCrypt over other baseline approaches is the estimation of (un)certainty quantification associated with each prediction. For cryptocurrencies this measure will be directly proportional to the volatility index. As discussed in section Uncertainty Quantification, it is easy to evaluate (un)certainty of prediction of an increase/decrease of price forecast by calculating the log-loss penalization as explained in eq. 18. To validate the proposed method results on (un)certainty quantification the work also evaluated cryptocurrency volatility index for each cryptocurrency. Cryptocurrecy Volatility Index (CVI) [START_REF] Kim | Vcrix-a volatility index for crypto-currencies[END_REF][START_REF] Woebbeking | Cryptocurrency volatility markets[END_REF]can be defined as a measure of market's expectation of volatility over the near trading terms for a particular asset. Volatility is often described as the and magnitude of changes in prices" and in finance often referred to as risk. Volatility is sometimes associated with the uncertainty of risk related to the amount of changes in security's value. This can be further described as if the security's value can potentially be spread out over a larger range of values, it indicates that the price of the security can change dramatically over a short time period in either direction which is flagged as higher volatility. On the other hand, A lower volatility means that a security's value does not fluctuate dramatically, and tends to be more steady. The mathematical formula to calculate CVI [START_REF] Woebbeking | Cryptocurrency volatility markets[END_REF]:

CVI = √ 365 * 1 N N N =1 ((Closeprice -P riceatN ) 2 )), (19)
Table 7 depicts the calculated log-loss values for each cryptocurrency vis-avis their cryptocurrency volatility index (CVI)4 .Table 7 represents the log loss score for all the layer architecture for DeCrypt and CVI score for DeCrypt (3 layers) and Nbeats. Table 8 represents the CVI scores from state-of-the-art method. A smaller value of the loss should be associated with lower volatility and vice versa. It can be clearly seen that log-loss associated with the Bitcoin, Litecoin, Peercoin, Ripple, NXT, Binance coin is less than one, meaning prices associated with these cryptocurrencies are less volatile. In contrast, Dogecoin, which is highly volatile and has a history of spiked values after a tweet by a major influencer, is more difficult to assess and has a log-loss score of 4.75.

Thus one can see how the proposed algorithm can quantify uncertainty and how this measure is proportionate to the oracle volatility. This is by far the most important result in the paper. This result shows how the proposed approach may be used for practical trading where both the point estimate as well as the uncertainty about the estimate is required for making decisions.

Owing to limitations in space, unfortunately not able to show the convergence of the proposed algorithm or the run-times of different techniques. Although the work have used 50 iterations and its empirically checked that the proposed algorithm converges in about 20-25 iterations. The convergence is monotonic. In terms of speed the proposed method is about 2-4 times faster than LSTM, CNN-TA and MFNN, and is about an order of magnitude faster than TFT, Nbeats and DeepAR. Of the existing methods, only RDL is comparable to the proposed method in terms of speed.

Discussion

Consistently beating in modeling Time series signals has been challenging

for a long time. The proposed three-layer architecture method performed better than the current state-of-the-art method. The proposed method is based on the deep state-space and feedback strategy. As can be seen from in contrast to machine learning models, which use a huge amount of data to learn approximations. We observed that when the time series signals grow, these models suffer from vanishing gradient and exploding gradient problems, which hampers learning model parameters. Due to this, model approximations are not learned efficiently and cannot capture sudden spikes (highs and lows in prices). These models also suffer from over-fitting. The proposed method avoids over-fitting as we move ahead in the sliding window protocol, and previously updated model parameters are used as initialized values for the next window parameters. We have also presented a comprehensive analysis of empirical and statistical performance. When Table 4 and Table 6 are analyzed, it is observed that the proposed method performance is outstanding in 3 out of 4 cryptocurrency presented, and its average score for ten crypto-currency is smallest as compared to other state-of-the-art methods hence we can conclude that more similarity exists between the actual closing prices and predicted closing prices when compared for different Crypto-currencies. The proposed method can be applied to various other prediction applications where its challenging to model unseen volatile data, such as short-term load monitoring, sales and revenue prediction, and predicting hate intensity for social media content.

Acknowledgement

The authors are indebted for the support provided by the Infosys Center

for Artificial Intelligence at Indraprastha Institute of Information Technology-Delhi,(IIIT Delhi).

Conclusion

The current study forecasts the prices of crypto-currencies; this is halfway to the goal. The final objective is to take trading positions (BUY / SELL / HOLD). But that is a very challenging problem where strategies come into play.

Unfortunately successful investors do not reveal their strategy. Pedagogically it is a matured area in traditional financial markets [START_REF] Molinero | Influence decision models: from cooperative game theory to social network analysis[END_REF] where game theory is mainly used in arriving at the decision boundaries; however how effective they are in practice is not known. Currently, a combination of game theory and social network analysis is used for arriving at such decision boundaries [START_REF] Molinero | Influence decision models: from cooperative game theory to social network analysis[END_REF]. In future, the authors would like to see if such cues from stock trading that can be used for maximising returns in crypto-currency trading.

  which establishes its connection with deep dictionary learning; the first paper is a more generalised non-linear version of the later. The work have utilised the potential of deep NMF / ReLU in the proposed work by embedding it into a Gaussian SSM. This allows for modelling non-linearity of the underlying dynamical process. The main difference between the prior shallow model and the proposed deep one is that the proposed work is more generalized version of the former. The shallow model can only approximate piece-wise linear functions; the proposed one, being formulated on ReLU network can approximate arbitrary non-linear and non-smooth functions. The price we pay for the generalisation ability of our approach is the difficulty in training. The shallow model allowed for closed form solutions; the proposed deeper extension does not. Hence, the work resort to alternating majorization-minimization (AMM) approach to solve this. The proposed approach is named as Deep state-space model for Predicting Cryptocurrency Price (DeCrypt). The rest of the paper is organized as follows. Related work in literature is reviewed in Section 2. The proposed model and inference algorithm are explained in Section 3. The application relevance is discussed in Section 4. The experimental results are presented and discussed in Section 5. The acknowledgement is discussed in Section 6. Conclusions and future directions of research are finally given in Section 7.

  three positive-valued linear factors yielding the multi-linear observation model D 0 D 1 D 2 and the sequence (z k ) 1≤k≤K , from observed sequence (x k ) 1≤k≤K and (u k ) 1≤k≤K . 1 The inference problem can be categorised as a blind filtering problem, where the objective is to infer the time series predictions and unknown model parameters from the given input data and observed sequences. As stated earlier, the classical SSM techniques need prior assumptions and information on model parameters. In the proposed model described here, this would imply explicitly setting some prior values to the positive latent factor matrices matrices {T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 } involved in both state, control and observation models. In real-world applications, especially in areas as complex as financial modelling this is never known; this is largely owing to the nonstationarity and volatility of the process. The main objective of the proposed work is to provide a point-wise estimate of the positive latent factor matrices {T 10 , T 11 , T 12 , T 20 , T 21 , T 22 , D 0 , D 1 , D 2 } and obtain a probabilistic estimate of sequence (z k ) 1≤k≤K , given the observed sequence (x k ) 1≤k≤K and control input (u k ) 1≤k≤K . Therefore the work propose to jointly solve for (i) the three deep NMF problems, and (ii) the filtering/smoothing problem.

{T 10 ,Figure 1 :

 101 Figure 1: Schematic Diagram for Proposed model DeCrypt.

  tocurrencies. The dataset is extracted from the Yahoo finance cryptocurrency repository using Yahoo finance API 3 . It consists of active cryptocurrencies ranging from some old and new cryptocurrencies. The data extracted is about eight years (between 01/01/2014 to 01/06/2021) for Litecoin, Namecoin, Dogecoin, Peercoin, Bitcoin, Ripple, NXT. For Gridcoin and Ethereum, we extracted seven years of data (between 01/01/2015 to 01/06/2021), the time of its first release year. The dataset created is divided into train and test datasets. Training data is from the year 2014 to 2018(2017 December). In contrast, testing data is from 2018 to 2021 is used.

•

  Long short term memory(LSTM) : Stacked LSTM with 2-layer architecture is used to forecast the time series sequences. The LSTM used comprises 50 cells and ReLU activation to estimate the predictions.(Elsworth & Güttel, 2020). • Convolutional neural network-Technical analysis(CNN-TA) :1-D Time series is converted to 2-D matrix with technical indicators as rows and timeunits as columns. 2D CNN is used for classification and regression.(Sezer & Ozbayoglu, 2018)• Recurrent dictionary learning (RDL): The RDL approach can be assumed to be a shallow (single layer) version of the proposed work. It can only model piece-wise linear functions.(Sharma et al., 2021) • Multi filter neural network (MFNN) : An end-to-end deep neural network comprising of recurrent neural network and convolutional neural network.[START_REF] Long | Deep learning-based feature engineering for stock price movement prediction[END_REF]

,Figure 2 :

 2 Figure 2: Cryptocurrency price forecasting via different algorithms evaluating test data for (a) Litecoin, (b) Dogecoin, (c) Bitcoin, (d) Gridcoin

  Root mean square error (RMSE), Mean Absolute Percentage Error (MAPE) and Symmetric Mean Absolute Percentage Error (SMAPE).

Figure 3 :

 3 Figure 3: Convergence plots for different layer architecture((a.) 1 Layer, (b.) 2 Layers, (c.) 3 Layer) for (A.) Bitcoin, (B.) Gridcoin, (C.) Litecoin

Figure 4 :

 4 Figure 4: Error bar plot comparing RMSE for DeCrypt against other baseline methods for (a) Litecoin, (b) Dogecoin, (c) Ethereum, (d) Gridcoin.

  Fig. 5.2 the proposed model can predict the sudden spikes in the data compared to other state-of-the-art methods. This is mainly because the proposed method is based on SSM, which uses probabilistic predictive distribution to estimate the future state of the price trajectories. The technique also embeds deep non-negative factors to learn the model parameters. Deep factors helps in updating the model parameters and state-space of unseen signals continuously with time,

Table 1 ,

 1 Table 2, Table 3 which represent the window size analysis in DeCrypt

	(1 layer), DeCrypt (2 layers), DeCrypt (3 layers) architecture respectively. The
	above mentioned tables provides comprehensive analysis on the performance of
	the model on varying window sizes. A comprehensive study has been compiled
	which offers analysis of various metrics like Pearson correlation (r), RMSE (Root
	Mean Square Error), MAE (Mean Absolute Error), and SMAPE (Symmetric
	mean absolute percentage error) for different window sizes τ . After analysing
	the results we conclude that model performance keeps on improving as window

Table 1 :

 1 Comparative performance of the proposed approach against different window size for 1-layer architecture. Lower value(↓) is considered the better.

	Window size(τ )	r	RMSE↓ MAPE(%)↓ SMAPE(%)↓
	10	0.42	0.59	74.1	71.2
	15	0.45	0.54	73.5	70.7
	20	0.49	0.43	71.8	70.2
	25	0.53	0.41	68.5	65.6
	30	0.57	0.39	65.1	64.3
	35	0.59	0.33	58.6	57.8
	40	0.65	0.27	51.5	50.8
	45	0.69	0.21	47.5	39.7
	50	0.76	0.17	31.2	28.7
	55	0.77	0.17	32.3	29.6
	60	0.75	0.15	31.6	27.8
	Table 2: Comparative performance of the proposed approach against different
	window size for 2-layer architecture. Lower value(↓) is considered the better.
	size increases till a stabilization point τ = 50. Ideally one would expect that
	the results would improve as the window size increases; especially for deeper

Table 4 :

 4 Tablepresentsthe average results for ten cryptocurrencies; owing to limitations in space we are not able to show individual results. One Comparative performance of the proposed approach against baseline methods. Lower value(↓) is considered the better. multi-linear state operator achieved from product of three positive-valued linear factors T 10 T 11 T 12 , T 2 is the is the multi-linear control operator achieved from product of three positive-valued linear factors T 20 T 21 T 22 and D is the multilinear observation operator achieved from product of three positive-valued linear factors D 0 D 1 D 2 . It can be clearly seen that the DeCrypt with its three layers Due to space constraints we have provided Avg. Score for t-test for all the ten cryptocurrency for each method in Table6. From the results we conclude that average T-test score is very low for Decrypt (3 layers) method when compared with other state-of-the-art method, hence we conclude that Decrypt (3 layers) performance is very close to ground truth. In contrast when we see individual crypto-currency analysis from table we can see that TFT outperforms all the methods in Gridcoin and DeepAR outperforms Dogecoin.

	can see that the proposed method outperforms the baseline by a considerable
	margin. Fig 5.2 shows the forecast performance of DeCrypt with other base-
	line methods for visual evaluation and Fig 3 shows the convergence plots for
	model parameters for different layer of architecture for DeCrypt where T 1 is the

Table 5 :

 5 Averaged time over 50 random runs for processing the dataset (train(hrs) and test(min)), for the proposed approach and its competitors.To understand the comparison in performance between the proposed method

	Method	Bitcoin Gridcoin Dogecoin Litecoin Avg. Score
	DeCrypt (3 layers)	0.31	0.73	0.82	0.57	0.68
	DeCrypt (2 layers)	0.56	0.85	0.91	0.69	0.72
	DeCrypt (1 layer)	0.58	0.98	0.99	0.72	0.78
	ARIMA	0.90	0.87	0.95		0.83
	LSTM	0.59	1.75	0.97	0.82	0.94
	DeepAR	0.54	0.79	0.81	0.55	0.71
	TFT	0.40	0.70	1.12	0.71	0.73
	Nbeats	0.56	0.68	0.99	0.61	0.84
	CNN-TA	0.87	1.53	2.11	1.19	1.13
	MFNN	0.82	1.30	1.21	0.74	1.37
	RDL	0.58	0.95	0.94	0.68	0.84
	Rao-ANN	0.54	0.70	1.02	1.13	0.94

Table 6 :

 6 Comparison of T-test score for the proposed approach with state-of-

the-art method for (a) Bitcoin, (b) Gridcoin, (c) Dogecoin, (d) Litecoin, (e)Avg.

Table 7 :

 7

	Cryptocurrency LSTM	CNN-	ARIMA Rao-	MFNN DeepAR TFT	RDL
			TA		ANN
	Bitcoin	0.57	0.53	0.46	0.59	0.72	0.52	0.49	0.38
	Dogecoin	2.87	3.27	2.35	3.43	3.87	2.72	2.83	2.57
	Namecoin	1.93	2.12	1.88	1.83	1.96	1.58	1.68	1.73
	Litecoin	1.31	0.56	0.48	0.51	0.67	0.38	0.46	0.41
	Gridcoin	0.99	0.87	0.67	0.74	0.96	0.68	0.75	0.53
	Peercoin	0.81	0.78	0.48	0.69	0.93	0.64	0.71	0.58
	Ripple	0.78	0.83	0.64	0.73	0.88	0.61	0.68	0.54
	NXT 0.57	0.63	0.46	0.53	0.64	0.58	0.54	0.59	0.47
	Ethereum	0.98	0.78	0.43	0.58	0.49	0.52	0.61	0.45
	Binance coin	0.53	0.64	0.41	0.49	0.51	0.45	0.54	0.39

(Un)certainty quantification (log-loss) and Cryptocurrency Volatility Index (CVI) evaluated using DeCrypt (3 layers) and Nbeats

Table 8 :

 8 Cryptocurrency Volatility Index (CVI) evaluated using state-of-the-art method predictions

Throughout the paper, three-terms factorizations is considered , for the sake of readability.The 3-layers modeling and inference methodology has the great advantage of being generic enough to be straightforwardly extended to any number, greater or equals to one, of factors.

https://finance.yahoo.com/cryptocurrencies

https://github.com/dc-aichara/PriceIndices