Amirhossein Sadoghi
email: amirhossein.sadoghi@rennes-sb.com

Jan Vecer
email: vecer@karlin.mff.cuni.cz

Optimal liquidation problem in illiquid markets

Keywords: Finance, Optimal Stopping Problem, Optimal Liquidation Problem, Illiquid Market, Markov-modulated Poisson process, Hawkes processes

In this research, we develop a trading strategy for the optimal liquidation problem of large-order trading, with different market microstructures, in an illiquid market. We formulate the liquidation problem as a discrete-time Markov decision process. In this market, the flow of liquidity events can be viewed as a point process with stochastic intensity.

Based on this fact, we model the price impact as a linear function of a self-exciting dynamic process. Our trading algorithm is designed in such a way that when no favourite orders arrive in the Limit Order Book (LOB), the optimal solution takes offers from the lower levels of the LOB. This solution might contradict conventional optimal execution methods, which only trade with the best available limit orders; however, our findings show that the proposed strategy may reduce final inventory costs by preventing orders not being filled at earlier trading times. Furthermore, the results indicate that an optimal trading strategy is dependent on characteristics of the market microstructure.

Introduction

In an illiquid market, due to the lack of counterparties and uncertainty about the value of assets, there is no guarantee that assets will trade at fair value [START_REF] Ang | Portfolio choice with illiquid assets[END_REF]. Depending on the elasticity of the market, the effect of an increased offer will be compensated by a drop in the price [START_REF] Mrázová | Not so demanding: demand structure and firm behavior[END_REF]. Effectively, the initial price impact is partially temporary and vanishes after the execution of the order (depending on the elasticity of the market).

In a market like this, trading only at best bid and ask prices is not an optimal strategy. The trader faces liquidity restrictions and should behave differently in comparison to an unconstrained market. Conventional optimal execution methods mostly only trade with the best available limit orders. This strategy can increase final inventory costs in an illiquid market.

In this study, we introduce a new algorithm for trading in an illiquid market, picking not only the best available orders but also orders in the deeper level of the order book. Our study is motivated by a real problem faced by traders wanting to liquidate large portfolios in illiquid markets. The optimal solution to this problem is to trade by going deeper into the Limit Order Book (LOB) and reduce the cost of the inventory punishment at the end of the period. The expected revenue of the trader is linked to his/her preferences over a set of limit orders. We explain how the dynamics of price might be affected by the arrival rate of orders. Following this, we show that the information on the arrival rates of limit orders in the order book can be used to compute the price impact. The results of the algorithmic simulations show that the optimal value of the trading rate is dependent on the characteristics of the market microstructure and the dynamics of the incoming orders. In fact, trading in a market with a high probability of the same types of orders arriving (self-exciting property of the arrival rate of orders) is more profitable than other types of markets.

Our paper, in line with studies by [START_REF] Ha | Algorithmic trading for online portfolio selection under limited market liquidity[END_REF]; [START_REF] Ting | Optimal liquidation strategies and their implications[END_REF]; [START_REF] Brunovskỳ | Optimal trade execution under endogenous pressure to liquidate: Theory and numerical solutions[END_REF], considers the portfolio selection and optimal liquidation problems of illiquid assets in markets that have limited liquidity. Illiquid assets are characterised by long times between trades, low turnover, challenges in finding counterparties, indivisibility [START_REF] Henderson | Risk aversion, indivisible timing options, and gambling[END_REF], and assets only being able to be traded infrequently [START_REF] Ang | Portfolio choice with illiquid assets[END_REF]. Assets are generally illiquid, and some liquid assets occasionally become illiquid; consequently, it is necessary to apply a search process to find a suitable counterparty [START_REF] Diamond | Aggregate demand management in search equilibrium[END_REF]. In a systemic way, illiquidity crises can occur when some assets are illiquid. For example, the cause of the 2008-09 global financial problem was rooted in the illiquid financial market [START_REF] Tirole | Illiquidity and all its friends[END_REF]. When insolvent financial institutions must liquidate massive amounts of illiquid assets, they no longer follow their optimal trading strategies. In this situation, financial institutions are forced to liquidate the assets based on available orders as fear of the crisis spreads . Consequently, the illiquid asset price further drops, which can lead to fire sales [START_REF] Diamond | Aggregate demand management in search equilibrium[END_REF]. Our liquidation model is primarily an algorithmic method. These methods apply automatic procedures to make decisions about selling or buying of shares and the submitting of orders. In modern financial markets, algorithmic trading methods are employed to trade illiquid assets. Algorithmic trading represents a large share of market activity and can enhance liquidity supply and fragmentation of order flows [START_REF] Foucault | Competition for order flow and smart order routing systems[END_REF][START_REF] Hendershott | Does algorithmic trading improve liquidity[END_REF] even when financial markets experience systemic events like the flash crash on May 6, 2010 [START_REF] Kirilenko | The flash crash: High-frequency trading in an electronic market[END_REF].

To understand the behaviour of market participants, we need to analyse the stochastic fluctuations in stock prices and the reaction of players in the market. These fluctuations can be explained by a sequence of market equilibria, which are determined by demand and supply. Generally, informed traders prefer to trade mainly during times of liquidity. As opposed to when the market is liquid [START_REF] Collin-Dufresne | Do prices reveal the presence of informed trading[END_REF][START_REF] Kyle | Continuous auctions and insider trading[END_REF], traders are willing to submit small orders when the market is illiquid [START_REF] Philip | Estimating permanent price impact via machine learning[END_REF]. However, traders are not entirely rational, and their trading behaviours are governed to some degree by their beliefs or sentiments [START_REF] Shleifer | The noise trader approach to finance[END_REF]. This disagreement in the market creates an imbalance between supply and demand for limit orders, which causes illiquidity in the market. In this market, assets can be traded only infrequently, contingent on the arrival of randomly occurring liquidity events. Furthermore, the dynamics of order arrival are influenced by the liquidity of assets. Therefore, trading activities have indirect effects on the dynamics of price. We use this fact and model the dynamics of incoming orders and liquidity events with the Hawkes process. This process can capture irregular patterns of asset prices. Following this approach, we measure the price impact of order executions based on a stochastic intensity process using the mutually-and self-exciting properties of the Hawkes process.

In order-driven markets, buy and sell orders arrive at different time points and wait in the LOB to be traded. We study the dynamics of the arrival rate of limit orders and the model distribution of limit orders in a LOB with the help of order statistics. We analytically express the probability of orders ranking, sorted based on price, in the LOB.

Modelling the LOB distribution helps us to forecast order in the hidden levels of the LOB and exploit the order information stored more in-depth in the LOB [START_REF] Biais | An empirical analysis of the limit order book and the order flow in the paris bourse[END_REF]. More precisely, knowing the LOB distribution and expected order quantities in the hidden levels guides the algorithm to select the limit orders of top levels of the LOB (both best and deeper levels of the LOB) to avoid risk execution and lack of offers in the future. It is consistent with the finding of [START_REF] Kavajecz | A specialist's quoted depth and the limit order book[END_REF] which limit order traders maintain a reasonable level of depth around illiquidity events to reduce their exposure to adverse selection losses. Our algorithm mimics informed traders' behaviour that uses depth as a strategic decision factor to adjust trading rate [START_REF] Kavajecz | A specialist's quoted depth and the limit order book[END_REF][START_REF] Goldstein | Eighths, sixteenths, and market depth: changes in tick size and liquidity provision on the nyse[END_REF].

We set up a stochastic control framework to maximize the expected revenue of trading with consideration to liquidity restrictions in trading a large order. We formulate the liquidation problem as a multi-stopping problem with Poisson arrival patterns of orders in a discrete-time model. The optimal liquidation problem can be expressed as an optimal control problem, to determine optimal strategies of trading stock portfolios by minimizing some cost functions. These strategies depend on the state of the market, as well as the price and size of stocks, which are available during the execution time. To find an optimal solution to the liquidation problem, one needs to consider a trade-off between liquidity risk and changing the stock price. The former results in slow order execution, while the latter is caused by exogenous events or a rapid liquidation. An early liquidation causes an unfavourable influence on the stock price, and a late execution has liquidity risk since the stock price can move away from that at the beginning of the period.

There are different ways to measure illiquidity [START_REF] Vayanos | Market liquidity-theory and empirical evidence[END_REF], such as bid-ask spread [START_REF] Glosten | Bid, ask and transaction prices in a specialist market with heterogeneously informed traders[END_REF], market depth [START_REF] Kyle | Continuous auctions and insider trading[END_REF], turnover, and other asset characteristics. In our market model, illiquidity is caused by a disparity between supply and demand, which means that the execution of larger positions requires longer periods of time and cannot be performed in continuous time. Therefore, we formulate the liquidation problem as a continuous-time stochastic control problem. By applying the Markov decision processes (MDP) approach, we decompose the liquidation problem into multiple deterministic optimal control problems. The discrete-time framework is closely related to classical trading algorithms. In most continuous liquidation problem setups, it is assumed that supply and demand are in balance; however, this is only a reasonable assumption in a liquid market.

Through our research, we formulate the liquidation problem associated with the depth function of the LOB. Our algorithm constructs trading boundaries built-in on market depth to determine an optimum strategy. When the stock prices hit one of these trading boundaries, i.e. there is a liquidity event, the algorithm liquids some part of shares by submitting limit orders. Past studies show the link between price impact and market depth and show that there is a linear relationship between an orders flow imbalance and price changes in high-frequency markets. This fact is investigated by [START_REF] Kempf | Market depth and order size[END_REF] who measure the market depth as a surplus demand amount that is needed for jumping one unit price. They show that there is a non-linear relationship between market depth and the price impact of the orders flow.

In our numerical simulation, we compare the performance of our algorithm concerning various market characteristics, as well as price impact functions. In the case where favourite offers do not come, our algorithm will reduce the speed of trading and go deeper into LOB to avoid the not filling of orders and facing an ultimate inventory penalty. This solution might contradict conventional optimal execution methods, which only trade with the best available limit orders. In fact, in illiquid markets, lower price levels of market depth are more attractive than upper levels; these depth levels include orders with significantly larger volumes. Most optimal liquidation methods focus on the best sell and buy price levels since their imbalance can move prices. By causing the lack of available liquidity, we need to take the lower level of market depth into account.

The main contributions of this paper can be summarized as follows. First, we develop a new algorithm to liquidate a large asset in an illiquid market. This algorithm uses trading boundaries built-in on the market depth and adjusts the speed of trading. It helps to avoid not filling of the order by picking up not only the best limit orders from LOB.

Second, we model the arriving pattern of limit orders based on the Poisson cluster process. We then use the same approach to model price impact (Proposition 1) and show how this model can extend to two main classes of price impact functions, i.e. permanent and temporary (Lemma 2). Third, we determine the distribution of a random number of sorted limit orders in LOB (Theorem 1). Forth, we solve the liquidation problem numerically with apply MDP approach (Theorem 2) and prove the uniqueness of the solution (Theorem 3). Fifth, with employing this algorithm, we investigate different trading scenarios in various market characteristics. The results show that a market with a higher level of the self-exciting property of limit order arriving can be more profitable (Table 1).

The rest of the paper is organized as follows. We review the related literature in section 2. Section 3 explains the market model setup and the problem statement and describes the statistical model of the LOB. Section 4 presents the stochastic dynamics of the intensity of the order arrival process, and we turn to model price impact. Section 5 describes the procedure of solution with using discrete-time Markov Process. In section 6, we explain the numerical method for optimal stopping time and simulate with different market microstructures. Section 7 summarizes the results and concludes the paper with further remarks.

Related Literature

Our paper relates to the recent and growing literature on optimal execution problems. The primary research on this subject date back to 1990s, and mainly focus on the discrete-time models, in which optimal strategies are determined as optimal liquidation rates per unit time. [START_REF] Bertsimas | Optimal control of execution costs[END_REF], as a discrete-time model, study the optimal liquidation of a large block of shares with linear permanent price impact in a fixed time horizon. [START_REF] Almgren | Optimal execution of portfolio transactions[END_REF], as one of the most cited optimal execution models, constructs an efficient frontier of execution strategies via mean-variance analysis of expected costs of liquidation and divide the price impact into temporary and/or linear permanent price impact. They use diffusion price processes in a continuous-time trading space. In our study, we model a similar liquidation problem with a restriction on liquidity in continuous-time trading time. We then solve the model in a discrete-time space.

Our study of an optimal stopping problem is a link between two different areas: control theory and market microstructure. In control theory, this problem has been studied as a single or multi-stopping problem in the classical best choice problems by using homogeneous Poisson processes. A single stopping time problem governed by the Poisson process was formulated as the best choice problem in the late 1950s by [START_REF] Lindley | Dynamic programming and decision theory[END_REF]. The optimal k-stopping problem with finite and infinite time horizon was presented with the complete solution by [START_REF] Peskir | Optimal stopping and free-boundary problems[END_REF] in a Bayesian formulation. A recent study by [START_REF] Ciocan | Interpretable optimal stopping[END_REF] develops an interpretable optimal stopping model which helps the decision-maker to determine the connection between the current system state and the policy's action. A similar problem, which has been considered in the market microstructure literature by [START_REF] Garman | Market microstructure[END_REF], who study a trading problem of a market maker who maximizes her profit by assuming order arrival rates depending on the price dynamics governed by the Poisson process. Some other studies explain the unconditional and steady-state distributions of the order book.

The homogeneous Poisson process of sell and buy orders arrival rate was the primary assumption of past studies. [START_REF] Garman | Market microstructure[END_REF] explain conditions necessary and sufficient for the order arrival patterns to be modelled by a homogeneous Poisson processor. In this framework, none of the agents' trading can dominate other agents, or they place a large number of limit orders in a finite time. Nevertheless, most of these assumptions are violated in highfrequency trading. Empirical studies of high-frequency data show that there are significant cross-correlation patterns of the arrival rate of similar limit buy or sell orders and significant autocorrelation in durations of events, (see: [START_REF] Cont | Statistical modeling of high-frequency financial data[END_REF]). [START_REF] Dufour | Time and the price impact of a trade[END_REF] use a framework provided by [START_REF] Hasbrouck | Measuring the information content of stock trades[END_REF] and investigate the time duration between trades. They determine that the trade duration process is correlated with the information about current and past transactions. Some empirical studies like [START_REF] Zheng | Modelling bid and ask prices using constrained hawkes processes: Ergodicity and scaling limit[END_REF] show that the dynamics of the Bid and Ask price can be modelled using Hawkes processes, introduced by [START_REF] Hawkes | Spectra of some self-exciting and mutually exciting point processes[END_REF]. The irregularity properties of high-frequency financial data can be explained by the self-exciting and mutually-exciting properties of the Hawkes processes. We use this fact and model dynamics of incoming limit orders using Hawkes processes. In a similar study, [START_REF] Engle | Trades and quotes: a bivariate point process[END_REF] model trades and quote arrivals with bivariate point processes. Some empirical studies like [START_REF] Bacry | Modelling microstructure noise with mutually exciting point processes[END_REF] show that high frequency data can be modelled using Hawkes processes. Our baseline model only considers the submission of limit orders in a LOB. Therefore, we apply univariate bivariate point processes to model the dynamics of incoming limit orders. Our baseline model can be extended to apply bivariate point processes for submission and cancellation processes. Several studies extend Hawkes processes. [START_REF] Chehrazi | Dynamic credit-collections optimization[END_REF] develop a traceable model based on a controlled self-exciting point processes framework to predict the repayment behaviour of unsecured loans placed in credit portfolios. They also consider the Hawkes processes alongside an extra predictable process that models account-treatment actions. Similarly, [START_REF] Dassios | Efficient simulation of clustering jumps with cir intensity[END_REF] extend Hawkes processes with a diffusion component to estimate default probability and model a portfolio loss process.

In our research, we study the statistical model of the LOB. We determine an analytic expression of the distribution of limit orders in Theorem 1. [START_REF] Garman | Market microstructure[END_REF]; [START_REF] Bayraktar | Queuing theoretic approaches to financial price fluctuations[END_REF] study the fluctuation of the orders arriving in the electronic trading market. Cont et al. (2010) develops a tractable stochastic model of limit order markets to capture the main statistical features of LOBs. They show the short-term price movements could be explained by the information on the current state of the LOB. Their model is based on the dynamics of the best bid and offer queues since the best orders can move the price. This study uses Laplace transforms to analyse the behaviour of the order book with Poisson arriving patterns of buy-and sell-limit orders. Unlike these studies, we express the distribution of LOB, sorted by price, thoroughly based on order statistics methodology. This distribution, which is defined analytically in Theorem 1, can help us to estimate the distribution of a random number of arrival limit orders in a LOB.

Our paper considers a liquidation problem when trader faces liquidity restrictions, and she should adjust her trading strategies. The impact of illiquidity on trading activities has been investigated in some theoretical studies like [START_REF] Kyle | Continuous auctions and insider trading[END_REF][START_REF] Easley | Price, trade size, and information in securities markets[END_REF] as well as some empirical studies like [START_REF] Chordia | Market liquidity and trading activity[END_REF][START_REF] Chordia | Have capital market anomalies attenuated in the recent era of high liquidity and trading activity[END_REF]. The theoretical models mostly focus on mechanisms underlying the trading activities faced with the cost of liquidity constraints, e.g. a theoretical model by [START_REF] Grossman | Liquidity and market structure[END_REF] tries to determine the equilibrium of liquidity level of the market.

Empirical studies mostly concern about changing depth and trading activities due to some significant macroeconomic factors and revolutions in trading technologies.

In studies by [START_REF] Bayraktar | Liquidation in limit order books with controlled intensity[END_REF]; [START_REF] Horst | When to cross the spread? trading in two-sided limit order books[END_REF], the liquidity model of Almgren [START_REF] Almgren | Optimal execution with nonlinear impact functions and trading-enhanced risk[END_REF] has been applied to determine temporary and permanent impacts of trading on fundamental price of asset.

These studies design optimal liquidation strategies in illiquid markets. [START_REF] Bayraktar | Liquidation in limit order books with controlled intensity[END_REF] formulate the liquidation problem as a Hamilton-Jacobi-Bellman equation association with the depth function of the LOB. In our paper, we study a liquidation problem which is formulated similarly to [START_REF] Bayraktar | Liquidation in limit order books with controlled intensity[END_REF]; however, we solve the problem numerically in a discrete space similar to [START_REF] Bäuerle | Mdp algorithms for portfolio optimization problems in pure jump markets[END_REF][START_REF] Huang | Multiconstrained finite-horizon piecewise deterministic markov decision processes with unbounded transition rates[END_REF].

Related Literature on price impact

In the current study, we use a stochastic intensity process to measure the price impact of order executions. Price impact is a traditional topic in conventional market microstructure research (see: e.g. [START_REF] Dufour | Time and the price impact of a trade[END_REF][START_REF] Hasbrouck | Measuring the information content of stock trades[END_REF][START_REF] Kyle | Continuous auctions and insider trading[END_REF]). Modelling of price impact on illiquid markets is not widely studied; we review some different generations of price impact modelling in the literature. An illiquid asset can only be liquidated contingent on the arrival of liquidity events. Accordingly, we cannot consider as a circumstance in which agents trade continuously in time, and there is no price impact [START_REF] Kogan | The price impact and survival of irrational traders[END_REF].

We can loosely classify price impact models based on their effects and how long they influence the dynamics of the prices in the following categories. We then show the link that our price impact model has to these classes.

The first class is a permanent price impact. As a consequence of a significant discrepancy between supply and demand in the market and a spread of bulk trading information, the dynamics of price have stable shifts. [START_REF] Kyle | Continuous auctions and insider trading[END_REF] proposed a basic microstructure model to analyze the price impact. Permanent impact was one the component of price model [START_REF] Bertsimas | Optimal control of execution costs[END_REF][START_REF] Almgren | Optimal execution of portfolio transactions[END_REF]. Empirical researches show that in a liquid market, trading activities cannot alter the dynamics of the price for a long time.

Conversely, this component can be one of the basic building blocks of price impact models in the illiquid market.

The second tier of the price impact model is related to modelling the temporary impacts that just have an effect on the current orders for a short time and not for the entire trading time. This component cannot alter the dynamics of the price for an extended period and just has an impact on the immediate execution of the trades. Some studies like [START_REF] Chen | Analytical results and efficient algorithm for optimal portfolio deleveraging with market impact[END_REF][START_REF] Chen | Optimal deleveraging with nonlinear temporary price impact[END_REF] consider both permanent and temporary price impact in optimal deleveraging problem and other studies like [START_REF] Brunovskỳ | Optimal trade execution under endogenous pressure to liquidate: Theory and numerical solutions[END_REF] leaves out the permanent effect on optimal trade execution problem. The tran-sient impact is the third class of price impact; this component can be significant for a finite period, and it eventually vanishes (see: [START_REF] Gatheral | No-dynamic-arbitrage and market impact[END_REF][START_REF] Alfonsi | Constrained portfolio liquidation in a limit order book model[END_REF]); [START_REF] Predoiu | Optimal execution of a general one-sided limit-order book[END_REF] models price impact by considering this component. The theoretical model by [START_REF] Huberman | Price manipulation and quasi-arbitrage[END_REF] show that permanent price impact can be linear with trade size. This model also shows that with having temporary price impact in a general form, the permanent price impact must be linear.

The last class of the price impact modelling is to control the rate of arrival of limit orders via the trading rate process. [START_REF] Cai | Institutional herding and its price impact: Evidence from the corporate bond market[END_REF] study the price impact of herding in the fixed-income market and show that buy and sell herdings are associated with permanent price impact and transitory price impact. [START_REF] Alfonsi | Dynamic optimal execution in a mixed-market-impact hawkes price model[END_REF] introduce a mixed market impact Poisson model to analyze a temporary shift of dynamics of the rate of order arrival. This model used the advantage of the self-exciting property of the Hawkes process to change the direction of trading in the same or opposite direction of order arrivals. Studies by [START_REF] Bayraktar | Liquidation in limit order books with controlled intensity[END_REF]; [START_REF] Guéant | Optimal portfolio liquidation with limit orders[END_REF] control the intensity of limit orders for the liquidation problem in a risk-neutral and risk-averse model, respectively. [START_REF] Gueant | General intensity shapes in optimal liquidation[END_REF] extend this model to a general form of dynamics of intensity. From a traditional viewpoint on this problem (see: e.g. [START_REF] Alfonsi | Optimal trade execution and absence of price manipulations in limit order book models[END_REF]), the optimal liquidation depends on the existence of a sufficiently large limit order in the LOB, and price impact is a function of the shape and depth of the LOB.

Our modelling of price impact mainly belongs to the third class of price impact models. Essentially, in illiquid markets, the trader faces a liquidity problem, and she should wait for a longer time to execute her orders. Empirical studies show that this market is a plastic (inelastic) market, and limit orders do not arrive in continuous time. The characteristics of this market allow us to model the pattern of the arrival of orders in discrete space and express an impact price function as a consequence of trading on this pattern. In proposition 1, we introduce a new price impact, defined on an impact of trading on the arriving order book. Similar to impact price models in the third class, such as [START_REF] Bayraktar | Liquidation in limit order books with controlled intensity[END_REF]; [START_REF] Guéant | Optimal portfolio liquidation with limit orders[END_REF], our price impact model can control the intensity of limit orders, and resultantly can control/mitigate the impact of price trading. In the lemma 2, we show how this model can be related to the first and second classes of price impact models. Moreover, similar to the [START_REF] Kyle | Continuous auctions and insider trading[END_REF] model, our impact price model is conditional on the state of liquidity in the market.

Market Model Setup

It is assumed that our financial market consists of an illiquid asset and a risk-free asset. The illiquid asset and the risk-free asset can be considered as a risky asset, and as a numeraire with the interest rate r, respectively. The market for the risk-free asset is liquid, that means traders can liquidate a large amount of this security without facing costs of price impact.

In a complete financial market, the price process S t is a stochastic process on a complete filtered probability space (Ω; F ; F; P) where F is the filtration generated by {F t } t∈T . This space is bounded by a maturity time T . The assumption of the finite discrete-time space is contradictory to conventional liquidation problems. However, it is not far away from reality; order arrival patterns of buy and sell orders are not the same. Due to fluctuations in the stock market, especially in a high-frequency environment, orders might not appear regularly in the LOB. Some empirical studies show that, in a short period, the percentage changes of a stock price are not uniformly distributed with the same centrality, but price processes can be internally steady-state processes. Therefore, the model should be solved and interpreted in discrete time and space. We use these facts and propose a model with the rate of incoming buy orders as a Poisson process in a finite discrete-time and space.

Trading Boundaries

Traders often follow trends of price and construct trading boundaries built-in on market depth. They then submit limit orders in touch with an optimum volume when the stock prices hit one of these trading boundaries (liquidity events). Market depth reflects the information related to the prices of buying and sell orders for the price and depends on trade volume and minimum price increment known as tick size [START_REF] Goldstein | Eighths, sixteenths, and market depth: changes in tick size and liquidity provision on the nyse[END_REF]. It is continuously changing and reflects the valuable information about the current orders sitting in the LOB. Knowing this information helps traders to understand hidden patterns of price movements and price impact. Traders can use this potential information and regulate their orders in response to the net order flow. Market depth also gives the overall picture of market conditions and a short term prediction to determine an optimum strategy, e.g. price movement towards selling pressure or buying strength in the short time, especially in illiquid markets. Some platforms apply a number of restrictions for market participants to observe or trade at the best level; we assume all orders are observable and accessible for traders. Market depth can be affected by the transparency of platforms in such a way that some levels of market depth are hidden, and just the latest submitted orders are available. Limiting trading and adjusting the minimum tick size are essential mechanisms to improve the efficiency of the market. As mentioned before, in illiquid markets, lower price levels of market depth include orders with significantly larger volumes. Therefore, to liquidate a large number of shares, we need to take the lower level of market depth into account.

Statistics model of limit order book

The Limit Order Book (LOB) contains information about orders, such as quantity, price, and type of order. Traders use knowledge of the LOB in their trading strategies, see [START_REF] Hasbrouck | Measuring the information content of stock trades[END_REF]. [START_REF] Philip | Estimating permanent price impact via machine learning[END_REF] describes the information included in the configuration of the LOB is a significant element of permanent price impact. The price impact is dependent on the trader's private information as well as their information from the LOB. [START_REF] Philip | Estimating permanent price impact via machine learning[END_REF] provides empirical evidence that the shape of the order book has substantial effects on the dynamics of trade. Trading against the thin (slight) side of LOB has a higher permanent price impact than trading toward the thick (deep) side of the LOB. Cont et al. (2010) shows that this information contains short-term price movements and might change quickly during the trading period. On the one hand, this information can be useful for reducing the complexity of the relation between price fluctuations and LOB dynamics. It also helps to predict different markets' quantities conditional on the current state of the order book. On the other hand, this information might be used by counter-parties to take advantage of price movement and submit or cancel strategic orders. The idea of the dark pool is to reduce the information linkage and adverse price risk.

Underlying our approach is that the dynamics of limit order arrivals follow a Poisson pattern of prices depending on rates of trading. We study the stochastic dynamics of the arrival rate of the limit order by using order statistics methodology.

Distribution of a limit order book

We assume in a financial market, the price processes (S j t ) ( j=1•••d) of limit orders satisfy the following stochastic dynamics for t ∈ [0, T ] is dS j t = S j t (µ t dt + σdW t ). Given a vector of price of buy (sell) orders (S j t ) 1≤ j≤d on the probability space (Ω; F ; P), the orders are sorted into a vector (S (1) t , S (2) t , • • • , S ( j) t ) satisfy the following chain conditions:

(S (1) t ≤ S (2) t , • • • , ≤ S ( j) t ). 10
The vector (S (1) t , S (2) t , • • • , S ( j) t ) is a so-called vector of order statistics of the price processes (S j t ) which is sorted by price.

It is assumed that noisy traders cannot have a strong influence on the market, and no cancellation or strategic cancellation of orders can occur. These assumptions are crucial since [START_REF] Brogaard | Price discovery without trading: Evidence from limit orders[END_REF] show the strong effects of limit order submissions and cancellations on permanent price impacts. In an illiquid market, liquidity events occur infrequently, and the rate of arrival of limit orders is governed by a Poisson process. The liquidity events in the nonoverlapping intervals are independent.

In a small time interval dt, the probability that one limit order arrives and sits in the LOB is:

P(N dt = 1) = λdt 1! e -λdt = λdt + o(dt).
Similarly, the probability that no limit order appears in the LOB in this interval is: P(N dt = 0) = e -λdt = 1 -λdt + o(dt), see Figure 2. In the following Theorem, we determine a distribution of a random number of sorted limit orders in LOB. Theorem 1 (Distribution of the Limit Order Book). Denote by L the number of unexecuted orders at time s, we assume that in the interval (0, t), the number of orders (N t ) is a random variable with the Poisson distribution with mean value λt. Vector {S 1 t , • • • , S N t t } represents the prices of buy (sell) orders with distribution function F on the probability space (Ω; F ; P). If U is the number of unexecuted orders with prices higher than y = max{S 1 t , • • • , S N t t }, then the probability of having k unexecuted orders with prices higher than y is:

P(U = k|N s = L) = e -λ y • λ k y k! ,
where

λ y = λ[1 -F(y)t -ln(e λF(y) - λ L+1 (L + 1)! )].
(The proof is given in the appendix)

With given k = 0 , Equation (3.1) gives the probability of best orders (i.e. none of orders with price (S i t ) i∈{1,•••,N t } has greater price than y), with k = 1 gives the probability of the second best order, and so on.

The number of limit of order book is not constant and it can be changed at each stopping time. In a dynamic framework, the proposed distribution in Theorem 1 can help us to estimate the statistical order distribution of a random number of orders accumulated in the LOB.

Problem statement

The dynamics of the price is described by a right-continuous process (S t ) (t≥0) changing while the times when the book order process meets boundary conditions. Later on, we discuss the impact of the current trade on the underlying price by using a particular approach to model price impact based on exogenous factors as well as characteristics of the stock price processes. We show how to model the dynamics of the intensity of orders arrival corresponding feedback effects of trading and the state of the market during orders execution period.

We consider a problem for an investor holding a large volume (Q 0 ) shares of illiquid assets and a risk-free asset.

The objective of the shareholder is to maximize her profit with subject to liquidity constraints and limited execution time.

The illiquid asset can be considered a risky asset with the following dynamic for t ∈ [0, T ]

dS t = S t (µ t dt + σdW t ),
and a risk-free asset used as a numeraire with the interest rate r with the following dynamic:

dB t = B 0 e rt dt.
Let Ŝ = e -rt S t be a martingale with respect to the measure P on a complete filtered probability space(Ω; F ; P).

The trading strategy of the trader is characterized by a trading rate (γ t ) t∈T . The vector γ contains the information on the amount of trading at each time point t.

The dynamics of the inventory of the investor holding Q 0 shares of an illiquid asset with the trading rate γ as a control process is given with the following counting process:

dq t γ = -γ t dN t γ ,
where the is a fraction of Q 0 shares, assumed to be constant at each stopping time i.e. either we fill whole orders or reject offers. We should note that is a block of shares, each block being of the same volume. Typically, in practice, equals the average (or median) trade size, which can be fulfilled partially. The assumption of might be questionable, but it reduces the problem's complexity from a theoretical point of view. For the sake of simplicity, we have also assumed = 1. Let N t be a counting process, and F t be a cash flow process with dynamics:

dF t γ = Ŝ t γ t dN t γ .
The investor has a finite time to liquidate her risky assets and maximize an expected utility function of her terminal cash account (wealth):

sup γ∈A(Q 0 ) (E[U(F T )]),
where F T is the amount of the cash at the end of time horizon T and U is a utility function to model the effects of illiquidity on preferences of a risk-averse agent. The utility function helps us to measure investor's preferences and the amount of risk they are willing to undertake in the hope of attaining better opportunities. We define A(Q 0 ) as a set of admissible strategies with given initial inventory with Q 0 shares to have nonnegative inventory at all times:

A(Q 0 ) ≡ {γ : γ is a predictable process, and admissible strategy from

Q 0 }
We shall use A ≡ A(Q 0 ) for the set of admissible strategies γ.

The liquidation problem can be formulated as a multi-stopping problem with discrete-time sequences. The trader can liquidate her shares in the m stops

T = {1 ≤ T 1 < • • • < T m ≤ n} with a discrete LOB.
The goal is to maximize the expected gain at each stopping sequence T i (i ≤ m). The trader wants to maximize the expected utility function of her final wealth. The expected utility allows the trader to compare the gain and loss of her trading strategies. Expected utility is comparable to expected revenue which means that the trader prefers a trading strategy that generates the highest expected utility.

The optimum expected revenue of the trade can be modelled as a dynamic optimization problem which is an interaction between price impact and price dynamics. We apply the Bellman's optimality equation in a recursive format to solve the m-stopping problem.

max(Expected Revenue) = max(Immediate Exercise + Continuation).

The trader holds Q 0 number of shares and places a selling order an k = γ (we have assumed = 1) illiquid asset in the market with the trading rate γ in the time horizon T , the performance criteria with the strategy γ is given by:

H γ (t, Q 0 ) = h γ (t, k) + E t,q [H γ (t + 1), Q 0 -k],
where h(t, k) is the depth function (concave and increasing in k; decreasing in t):

h(t, k) = max((S (t,k) -ES (t,k) , 0)) = (S (t,k) -ES (t,k) ) + ,
where:

S (t,k) = n i=n-k+1 S (i) t 1 (i≤k) .
Let ES (t,k) be the expected boundaries, constructed from the set of best limit orders in the LOB by using the distribution of the order book at the stopping time t.

ES (t,k) = n i=n-k+1 P(i ≤ k)S (i) t .
At the stopping time, when one of these expected boundaries (see Figure 1)is hit by the stock price, we execute some proportion of the inventory via limit order at the LOB with the depth function h(t, k) with a boundary condition:

h(T, 0) = h(0, Q 0 ) = 0.
In this regard, the goal of the investor is to maximize her wealth at the end of the period T associated with this dynamic problem; the value function can be defined by:

V(t, q) = sup γ∈A E t,q [H γ (t, q)], t ∈ [0, T ], q ∈ [0, Q 0 ]. (3.1)
Assumption 1. Continuity and concavity of the value function

We assume that the value function V is a continuous and bounded function. It is also strictly concave in q, increasing in t and non-negative. The differentiability condition of the value function is not necessary to be satisfied.

The strict concavity of value function V is an essential assumption that implies the existence of optimal controls and steady-state stability [START_REF] Bäuerle | Discounted stochastic fluid programs[END_REF][START_REF] Bäuerle | Optimal control of piecewise deterministic markov processes with finite time horizon[END_REF]. Then under Assumption 1, the terminal wealth value function of the investor who maximizes her wealth at the end of time period T , with given discounted price process Ŝ , and inventory Q 0 , associated to optimal trading rate γ, can be expressed as the follows:

Lemma 1. Let V(T, Q 0 ) be the continuation value that is obtained from optimal trading of Q 0 shares until the end of period T and the intensity process λ t be defined as a rate of arrival of limit orders. The expected revenue from the execution of limit orders with arrival patterns with Poisson distribution is equal to:

V(T, Q 0 ) = sup γ∈A E t,q [ T 0 e -rT γ t S t λ t dt], (3.2)
where T = T ∧ inf{t ≥ 0 : Q 0q t = 0} is trading time and trading process γ t is a control process, which has influence over the cash process, the inventory process, and the dynamics of prices. The vector γ contains the information on the amount of trading at each time point (γ t ).

(The proof is given in the appendix)

Modeling Stochastic Intensity and price impact

The rate of arrival of limit orders depends on the price and size of orders; the cheaper orders will remain on the LOB for a shorter time. It is empirically shown that the distribution of the price is not constant and depends on the current state of the LOB. The existence of significant autocorrelation of price movement and correlations across time periods rejects the natural assumption of a constant intensity of order arrivals rate (see: e.g. [START_REF] Cont | Statistical modeling of high-frequency financial data[END_REF]). In an illiquid market, bid and ask orders do not arrive consistently, and counterparties do not meet their demands regularly.

These irregular proprieties of a high-frequency environment lead to applying point process to model time series of the bid or ask price movements. [START_REF] Garman | Market microstructure[END_REF] propose models, in which order arrivals are governed by a point process with constant intensity. There are several conditions in previous trading models that do not hold our setting.

First, a small number of traders cannot dominate the market with large-scale orders. The second condition is that the submitting of orders is independent, and we mainly have an assumption of market efficiency. These conditions are essential for having a constant intensity of order arrivals. However, the structure of the market is dynamic, and highfrequency traders dominate over seventy percent of the market; therefore, none of the above conditions can be satisfied.

Hawkes process as a point process was initially applied to model earthquake occurrences. Some empirical studies

show that the Hawkes process can fit with high-frequency data [START_REF] Bacry | Modelling microstructure noise with mutually exciting point processes[END_REF][START_REF] Bacry | Estimation of slowly decreasing hawkes kernels: application to high-frequency order book dynamics[END_REF][START_REF] Chavez-Demoulin | High-frequency financial data modeling using hawkes processes[END_REF]. This process can explain the irregularity properties of data based on the positive and negative jumping behaviour of the asset prices. [START_REF] Cartea | Buy low, sell high: A high frequency trading perspective[END_REF] use the Hawkes process to express the dynamics of market orders and the LOB. This process is a general form of a standard point process. The intensity of this process is conditional on the recent history, increase the rate of arrival of the same type of event (self-exciting property), and it captures the impact of arrivals of orders on other types of orders (mutually-exciting property). In A.1, we explain the basic concept of a point process and the dynamic of the Hawkes process.

Price Impact Model

Estimating and modelling price impacts is a crucial research area in market microstructure literature. It can be expressed as a relationship between trading activities and price movements. Monitoring and controlling the impact of trading are the main parts of algorithm trading. To minimize the market impact, traders split their orders into smaller chunks based on the current liquidity in the LOB. Price impact might be dependent on exogenous factors like trade rate and some endogenous factors, such as liquidity and volatility. It is empirically observed that there is a changing of the volatility of prices on trading activities. [START_REF] Kyle | Continuous auctions and insider trading[END_REF] proposes a simple model for the evolution of market prices and price movement. In his model, noise traders and inform traders submit orders, and a market maker executes the orders. The price is adjusted to a linear relationship between the trade size and a proxy for market liquidity. As a consequence of trading, the price moves permanently, information affects the price for a long time, and price changes are strongly autocorrelated. The most recent literature on market microstructure shows that for liquid markets, price impact cannot be permanent. In highly liquid markets, outstanding shares are small and need less time to be liquidated.

In illiquid markets, the temporary price impact governs the enduring impact, and after a considerable number of trades, the price movement shows a high resilience. Empirical studies show that an elastic market can be converted to a plastic (inelastic) market as a consequence of lacking counterparties, such that high-volume trading has a longlasting impact. In this market, traders face difficulty in finding counterparties at a particular time, and they should wait for a longer time to execute the orders or else cooperate with counterparties. The effect of price impact corresponding to the liquidation strategy can be significantly large for substantial risk-averse traders who liquidate shares at a fast rate.

For this study, we use a stochastic intensity process to measure the price impact. This model is based on a counting process using the mutual-exciting property of the Hawkes process. In illiquid markets, the imbalance between supply and demand causes illiquidity, and the execution of a larger position needs a longer time and effect on the arrival of the new orders, or to stay longer in the LOB. Also, we explained the coming pattern of orders governed by a stochastic point process. Therefore, the intensity of order arrival is influenced by the arrival time of orders, the order value, and indirectly, the price will be affected by trading.

Generally, the supply/demand of financial securities is not elastic [START_REF] Obizhaeva | Optimal trading strategy and supply/demand dynamics[END_REF]. According to fundamental concepts of economics, in equilibrium, the relationship between supply and demand determines the price that traders are willing to take for positions during a specified period. If the market is illiquid, trading a large number of shares produces a large price fluctuation [START_REF] Mrázová | Not so demanding: demand structure and firm behavior[END_REF]. High-frequency traders Often execute trades in the course of the order book imbalance. As a result of this shock, the equilibrium of supply and demand could be changed, and the price must rise/decline to reach a new equilibrium (see: Figure 10). The price movement occurs when a change in demand is caused by a change in a number of market orders. This change in the price equilibrium is sensitive to shifts in both prices, and quantity demanded, which is called the "elasticity of demand" [START_REF] Mrázová | Not so demanding: demand structure and firm behavior[END_REF].

We model this impact with close form solutions for the dynamics of price impact. Let function f (γ t ) be a general form of the impact market, conditional on the state of liquidity of the market. Similar to [START_REF] Kyle | Continuous auctions and insider trading[END_REF] model, we use parameter α, as inverse liquidity of the market, to measure price impact. Function Γ(ts) is the decay of price impact function, which is independent of the state of the market. This function can be shown by the exponential or power-law decay of price impact. [START_REF] Gabaix | Power laws in economics: An introduction[END_REF] shows power-law functions can explain several patterns in the stock market, including price impact.

Different types of price impact and decay functions have been used in the algorithm trading and market microstructure literature. We define two well-known impact functions: exponential and power-law functions, linked to the exponential decay function. The parameter α is a proxy for the inverse of the market liquidity:

• Power law impact function:

f (γ) = γ α , α > 0.
• Exponential impact function:

f (γ) = exp(αγ), α > 0.
Gatheral ( 2010) examines the dynamic-arbitrage for different combinations of the market impact function and decay function to not admit price manipulation strategies. [START_REF] Alfonsi | Optimal trade execution and absence of price manipulations in limit order book models[END_REF], proposition 2) proves the convex and non-constant exponential price impact, leads to the strictly positive definite property, to avoid arbitrage implication, which is already stated in [START_REF] Gatheral | No-dynamic-arbitrage and market impact[END_REF]. Nonlinear impact function that is not a related state of the market is questionable; however, the above functions defined in an illiquid market depend on the state of the market.

We model the dynamics of the order arriving intensity with the Hawkes process as a point process. This model can capture irregular properties of the high-frequency data. The strength of incentive to generate the same event is presented with parameter σ. With choosing the price impact function f as a function of a trading rate γ t and the parameter κ as the exponent of the decay of market impact, we can model a liquidation effect on the trade arrival dynamics. This model can be shown by a particular form of the Hawkes process with the following SDE:

dλ t = ( f (γ t ) -κλ t )dt + σdN t . (4.1)
In the following proposition, we model the impact of trading on the rate of order arriving based on the Hawkes process:

Proposition 1. Price Impact Model Let f be a price impact function and a function of a trading rate γ t . σ and κ represent the magnitude of self-exciting and the exponent of the decay of market impact, respectively. The solution of the SDE is expressed by:

λ t = t 0 ( f (γ s )Γ(t -s))ds + σ t 0 e -κ(t-s) dN s , (4.2)
where the function Γ is the decay of impact.

(The proof is given in the appendix)

Lemma 2. The impact stochastic intensity equation: (A.12) is a general function of price impact. It can then measure an instantaneous price impact in the short term, and a permanent price impact on the long run.

(The proof is given in the appendix)

Solving Model by Discrete-Time MDP

The liquidation problem (3.2) is a continuous-time stochastic control problem. [START_REF] Bayraktar | Liquidation in limit order books with controlled intensity[END_REF] formulate this problem as a Hamilton-Jacobi-Bellman equation, and provided viscosity solutions as closed-form solutions associated with the LOB model and its depth function. The classical stochastic control approach solves nonlinear partial differential equations, and it is necessary the differentiability of the value function to be satisfied.

In contrast, a discrete-time Markov Decision Processes (MDP) approach provides a set of optimal policies, condition on the differentiability of the value function. [START_REF] Bäuerle | Mdp algorithms for portfolio optimization problems in pure jump markets[END_REF] use this approach and by applying dynamic programming principles, prove the existence and uniqueness of the solution. This liquidation problem is a mathematical abstraction of real problems in which an investor should decide on several stopping times to gain certain revenue at each stage. The investor has a finite period to liquidate a position and maximize the total revenue at the end of the period. Therefore, in this problem with a finite number of sub-periods, a mapping function should be applied to compute optimal policies through the limited number of steps of the dynamic programming algorithm. She must find an equilibrium to minimize the cost of the present extensive trading against the future abandon risk where the overall cost is not predictable. This problem can be formulated as a deterministic or stochastic optimal control problem with Markov or semi-Markov decision property under different setups. [START_REF] Bertsekas | Stochastic optimal control: The discrete time case[END_REF] distinguish between the stochastic optimal control problems from its deterministic form regarding available information. In a deterministic optimal control problem, we can specify a set of states and policy as a control process in advance. Thereby, a succeeding state is the function of the present state and its control process. On the other hand, in the stochastic control problem, controlling the succeeding state of the system leads to evaluate unforeseen states; therefore, control variables that are no longer appropriate or have ceased to exist.

In this paper, we use a Piecewise-Deterministic Markov Decision Model (PDMD) to decompose the liquidation (problem: 3.2) as a continuous-time stochastic control problem into discrete-time problems. Alternatively, we can apply a method by [START_REF] Mamer | Successive approximations for finite horizon, semi-markov decision processes with application to asset liquidation[END_REF], which is successive approximations for semi-Markov decision processes [START_REF] Huang | Finite horizon semi-markov decision processes with application to maintenance systems[END_REF] or the pathwise optimization method [START_REF] Desai | Pathwise optimization for optimal stopping problems[END_REF].

Solution by PDMP

Piecewise-Deterministic Markov Process (PDMP), introduced by [START_REF] Davis | Piecewise-deterministic markov processes: A general class of non-diffusion stochastic models[END_REF], is now applied mainly in various areas such as natural science, engineering, optimal control, and finance. The PDMP is a member of the cádlág Markov Process family; it is a non-diffusion stochastic dynamic model, with a deterministic motion that is punctuated by a random jump process.

Definition 1. Piecewise Deterministic Markov Process

A piecewise-deterministic Markov process (PDMP) is a cádlág Markov process with deterministic emotion controlled by the random jump at jump time.

Three measurement quantities characterize this process. The first feature is the transition measure k, which selects the post-jump location. The other quantities are deterministic flow motion φ between jumps and the intensity of the random jump Λ t , defined as Borel measures on the Borel sets of the state space E, and the control action space A.

In the state space E, set (t, x) donates the desired process value at the jump time point t. In an embedded Markov chain as a discrete-time Markov chain, the state of the PDMP process can be defined as a set of components of the

continuous trajectory Z k = (T k , X T k ) k=1,•••,
Z t =           
φ(X t ), for t < T, X t , for t = T.

(5.1)

Figure 3: Iterative Procedure of PDMP Figure 3 shows the iterative procedure of the PDMP: starting point of the PDMP is X 0 = Z 0 , then X 0 follows the flow φ until T 1 to determine first jump location X 1 = φ(X 0 ). The stochastic Kernel K(.|φ(X 1 ), .) selects the next location of the post-jump Z 1 = K(.|φ(X 1 ), .). Latter, similar to the first jump, X t follows the flow φ until T 2 so X 2 = φ(Z 1 ). In next step is the selection of a location of the post-jump Z 2 = K(.|φ(Z 1 ), .) by using the stochastic Kernel K(.|φ(Z 1 ), .).

This iterative procedure will be continued until it hits a boundary of the state space.

As mentioned earlier , the process of the illiquid asset for satisfies the stochastic difierential equation:

dS t = S t (µ t dt + σdW t ), t ∈ [0, T ]
In this market, the price processes can have jumps that could be occurred at random time points and followed by a Poisson process. We denote by T = {0 := T 0 < T 1 • • • < T m < n} the set of jump time points of the Poisson process.

The price processes for the time between two jumps i.e. t ∈ [T i , T i+1 ) for i ≤ m is given by

S t = S T n (exp(µ t - σ 2 2 )(t -T i ) + σW t ),
and at time of a jump, the price process is:

S T i -S T i+1 = S T i+1 J i ,
where J i is an independent and identically distributed random variable. The price process S t is a so-called piecewisedeterministic Markov process (PDMP). We note that since J i > -1 the price process stays positive.

Markov Process Components

General speaking, Markov process models, which are not stationary in our setup, include a set of the following terms:

• Let A be an action space includes the action a t which denotes the quantity of shares to be liquidated at time t.

a t = γ t , t ∈ [0, • • • , T ]
where γ t is an admissible strategy which satisfies the condition γ t ≤ q to have a non-negative inventory.

• With a given action set A, the set E is defined as a state-space, that contains the state x of inventory after applying action a. The process state X t denotes the amount of not liquidated shares at jump time t.

X t = Q 0 - t 0 q s dN s = Q 0 - t 0 γ s dN s.
• Let K be a stochastic transition kernel from E × A, as a set of all state-action, to a set of states E. We measure the probability of the next state and action with transition kernel K(.|x t , a t ) based on the transition law at Markov decision model.

• The reward function R represents the expected gains as a result of applying the strategy γ, of each state at the jump point time t:

R(t, x, γ) = t 0 γ s S s dN s = t 0 γ s S s λ s ds (5.2) = t 0 U γ (X s )dN s , (5.3)
where the function U : (0, ∞) → R + is an increasing concave function, which represents the preference over a set of limit orders or satisfaction of the trader from offers in the market and X s is the process state X t represents the amount of not liquidated shares at jump time t.

• The deterministic flow motion φ measures the movement of the inventory of investor between two jumps with a given the strategy η:

φ η (X t ) = t 0 η s dN s.
(5.4)

To be more precise on how we can solve the liquidation problem with PDMD, we explain the formal expression of deterministic optimal control problems, which is well documented in [START_REF] Bertsekas | Stochastic optimal control: The discrete time case[END_REF]. In a formal deterministic optimal control problem, x i presents the state of the system at stage i and function c i is a corresponding control at that stage. System equation x i+1 = f (x i , c i ) is the generating function of next state x i+1 from current state x i and its control c i . Function g(x i , c i (x i )) is the rewarding function of state i a associate with function c i .

The total expected revenue after N decision steps is defined by:

J C (x 0 ) = E t,x [ N i=0 g C (x i , c i (x i ))].
(5.5)

The set C contains all control functions (c i ) i={0,•••,N} , i.e. the expected total revenue is set of states and corresponding a sequence of Markovian decision controls.

Let Π = {π 1 , π 2 , • • • , π N } be the sequence of all Markovian decision controls π i . These decision controls are corresponding to Markovian policy for predictable admissible process γ defined in the set Ψ. Each Markovian policy is a function of the jump time component (T k ) and the post jump location (Z T k ).

In each stage, the reward obtained as a result of sequential Markovian decisions after the jump by applying the strategy

π i is: R(Z i , π i (Z i )) = t 0 U π (φ s (X i ))dN s .
(5.6)

The total expected revenue after N steps is defined by:

Ψ π (t, x) = E t,x [ N i=0 R(Z i , π i (Z i )).
(5.7)

In the following theory, we explain how the above equation is mathematically equivalent to our main problem: (3.2).

Theorem 2. Suppose policy set

Π = {π 1 , π 2 , • • • , π N } is a Markovian policy set and Z = {Z 1 , Z 2 , • • • , Z N } is a set of
post jump location of PMDP and R is the reward function equation: (5.3). Then the value function is the expected reward of the PDMP under the Markovian policy Π at time point t and in the state x:

V(t, q) = sup π∈Π Ψ π (t, x), t ∈ T, q ∈ [0, Q 0 ]
where

Ψ π (t, x) = E t,x [ N i=0 R(Z i , π i (Z i ))].
(The proof is given in the appendix)

Uniqueness of solution

We started to model the optimal liquidation problem as a stochastic control model Equation (3.1) and used the piecewise-deterministic Markov process to find an equivalent deterministic model for this problem. We also proved this problem could be constructed as a summation of the sequence of state processes and corresponding control processes in the set Π. The formulation has been given in the Equation (3.1) and is more consistent with the dynamic programming principle. [START_REF] Bertsekas | Stochastic optimal control: The discrete time case[END_REF] define the universal measurable mapping T to map Equation (3.1) from ( 5.3) as follows:

T π (Ψ)(x) = H[x, π, Ψ]. (5.8)
Therefore the operator T π n can be decomposed to

T π 0 • T π 1 • T π 2 • • • T π n-1 .
From above definitions, we have:

Ψ π n = T π n (r) = T π 0 T π 1 • • • T π n-k Ψ π n-k .
The mapping T π is an universal measurable mapping, let T π 0 = 0 and k ≤ n, (see: [START_REF] Bertsekas | Stochastic optimal control: The discrete time case[END_REF](chapter 1) we have:

Ψ π k = T π 0 T π 1 • • • T π k-1 Ψ π 0 .
In the following theorem, we prove the uniqueness of the solution by applying a piecewise-deterministic Markov process model.

Theorem 3. Uniqueness of the solution Let V(t, q) be a revenue function, and the optimal solution of the liquidation problem is a concave and decreasing function in q and increasing in t. Then the solution of the liquidation problem by applying a PMDP model is converging to a unique solution.

(The proof is given in the appendix)

Numerical method and Simulation

In this section, we apply a simulation method to assist the performance of our model under various market microstructures' characteristics. The trader will decide to take a number of offers in the LOB at given prices. The optimum trading rate is dependent on the dynamics of orders' arrival as well as time to maturity. We approximate the value function with a quantization method.

Approximation of the value function

As we explained before, the solution of the value function V of the optimal liquidation problem is obtained by the summing rewards of sequential Markovian decisions with corresponding the Markovian policies π and a set of post jump process of the PMDP:

V(t, Q 0 ) = sup γ∈A E t,q [V(t + 1, Q 0 -q) + h(t, q)] = sup γ∈A E t,q [ ∞ i=0 R(Z i , π i (Z i ))1 T i <T + h(t, q)1 T i ≥T ]
We approximate the value function V with the function V, such that |V -V| Lp is minimized for the Lp norm. To approximate the continuous state space by a discrete space, we use a technique that is called Quantization method. [START_REF] Bally | A quantization algorithm for solving multidimensional discrete-time optimal stopping problems[END_REF][START_REF] Bally | A quantization tree method for pricing and hedging multidimensional american options[END_REF]

V(t, Q 0 ) = max γ∈A E t,x [ ∞ i=0 R( Ẑi , π i ( Ẑi ))1 Ti <T + h(t, q)1 Ti ≥T ]
De Saporta et al. ( 2010) estimate the error and the convergence rate of the approximated value function with Lipschitz assumption of local characteristics of PMDP, and showed it is bounded by the constant rate of quantization error Qe.

|V(t, Q 0 ) -V(t, Q 0 )|≤ Qe (6.1)

Simulation

Conditional expectation of value function can be computed with some numerical methods in finite dimensional space, such as regression method [START_REF] Longstaff | Valuing american options by simulation: A simple least-squares approach[END_REF][START_REF] Tsitsiklis | Regression methods for pricing complex american-style options[END_REF][START_REF] Carriere | Valuation of the early-exercise price for derivative securities using simulations and splines[END_REF] or on Malliavin calculus (as in [START_REF] Cont | Change of variable formulas for non-anticipative functionals on path space[END_REF]). [START_REF] Bally | A quantization algorithm for solving multidimensional discrete-time optimal stopping problems[END_REF] propose a quantization method to approximate the state space of problem; from each time step T k , a sate function Ẑ can be projected to the grid Υ k := {ẑ T i k } 1≤i≤N (see: Figure 4)

ẐT k = 1≤i≤N ẑT i k 1 T i k ∈B k i , (6.2)
where B k i is a Borel partition of R d (see: [START_REF] Bally | A quantization algorithm for solving multidimensional discrete-time optimal stopping problems[END_REF]).

As previously stated, at each stage, the reward obtained for each stage of sequential Markovian decision after a jump by applying strategy π i is:

Vk = R( ẐT k , π( ẐT k )) (6.3)
By applying dynamic programming principle for n fixed grids Υ 0≤k≤n , Vk satisfies the backward dynamic programming condition:

Vn = R( ẐT n , π( ẐT n )) (6.4) Vk = max(R( ẐT k , π( ẐT k ), E( Vk+1 | ẐT k )) (6.5)
Figure 4: Quantization of state space (Z i , T i )

We have used a particular SDE form of the Hawkes process of the intensity of the rate of order arrivals to express the impact of order execution on the market: (6.6) where f (γ t ) is a function of trade rate γ t , σ explained the strength of the incentive to generate the same event , and κ is the exponent of the decay of market impact.

dλ t = ( f (γ t ) -κλ t )dt + σdN t ,
The Hawkes process can capture both exogenous impacts and endogenous influence of past events to measure the probability of occurrences of events. This intensity, with the condition (σ > 0 ), can capture the contribution of past events which amplifies the chance of an occurrence of the same type of events (self-exciting property of the Hawkes process). By using this endogenous mechanism, (while σ < 0), the model can explain the significant aspect of the strategic function of market participants known as a "market manipulation" (see: e.g. [START_REF] Cartea | Buy low, sell high: A high frequency trading perspective[END_REF]). It is often used by traders to submit or cancel orders strategically to detect hidden liquidity or to manipulate markets.

Suppose trading activity with reducing possibility occurrence has an adverse influence on the arrival of orders. In that case, the jump process has an adverse impact on its intensity and makes imbalance in supply and demand of the market exponentially. This damping factor can be measured with Hawkes model the magnitude of self-exciting and the strength of the incentive while it is negative (self-damping property of the Hawkes process).

Result of Simulation

In this part, we present the numerical solution of the optimal liquidation problem. To study the characteristics of the value function and the level of inventory associated with the control variable γ as the rate of trading, we compute some numerical examples using different scenarios from empirical studies. Our simulation is an abstract of real problems. We consider a trader who wants to liquidate Q 0 shares of a risky asset within a short time and fixed time horizon T . In an illiquid market, she expects a longer time to liquidate the whole position. Her goal is to minimize the implicit and explicit cost of trading and keep the low-level inventory by controlling the trading rate.

We implement our model in the discrete state space. We choose time steps small enough to increase the chance of catching orders, and larger than the usual tick time to make sure that quotes are not outside of the market bid-ask spreads. We assess the performance of our model by quantizing the value function at a fixed position within the space and time of the mesh refinement.

To illustrate how our model behaves, we study the inventory level associated with the optimal trading rate control for both types of price impact Hawkes models: self-damping and self-exciting properties. Concerning our simulation scenarios, we choose values for the parameters of the model:

• The parameter σ as the magnitude of self-exciting

• The parameter κ as the exponent of the decay of market impact

• The size of the bid-ask spread is used as a proxy to measure illiquidity These parameters can be estimated from real market data.

Plotting the trading boundaries (Figure 5) shows that the optimal trading level depends on the price process, the remainder of the inventory, and time to maturity.

Due to market conditions and asset characteristics, an agent with a higher degree of risk aversion cares more about the execution risk and price fluctuations. She starts trading with available orders at a deeper level of the LOB to avoid risk execution and lack of offers in future. She splits the original order into smaller slices to mitigate price impact.

Figure 5: Inventory level of a trader with given the initial inventory Q 0 and time to maturity T Table 1 summarizes results of simulations by our model, including the level of inventory and its corresponding optimal trading rate for different scenarios of implementations of strategies. In the case of coming of not favourite offers, the algorithm reduces the speed of trading (panel I: self-damping property) and waited for a longer time to find better matching counterparties. In unstable market conditions, are indicated with the higher level of self-damping, the trader should pay for final inventory to liquidate the whole position of initial shares. At this point, the best strategy is to accept offers in the LOB to avoid never to face severity penalties at the end of period. If estimated parameters of markets might show that the higher chance of same types of orders' occurrences (panel II: self-exciting property), the algorithm reduces the trading rate in the hope of getting better offers. The second column shows the related entries of the value function as a result of the implicit and explicit cost of trading. The results demonstrate that the second type of market characteristics with a higher level of self-exciting property can be more profitable. It is worthwhile to compare our algorithm with some benchmarks, such as the conventional algorithm that trades only the best LOB orders. Table 1 (panel III: Conventional method) presents the results of the conventional algorithm simulations. The conventional optimal execution methods only trade with the best available limit orders in the LOB.

We thus replicated the simulation experiments with both market microstructure conditions: self-exciting and selfdamping properties . Our results showed that the trader faces more costs at the end of the period and fewer profits than our algorithm in both scenarios. The conventional algorithm also considers orders at the top level of the LOB, and is less flexible when the market is illiquid. Having a look at the graphs (Figure 8) showing the optimal trading rate of different scenarios, one can conclude that there are more fluctuations in trading if the probability of arrival new offers is reduced.

Our numerical results also show that the proportion of the gain of the liquidation model considerably depends on the specification of the price impact function. It indicates that market conditions have an effect on the final inventory level and causes a substantial dropping in the final wealth of the trader. An important aspect of the optimal strategies, which we have developed, is to take into account the execution risk in an illiquid market, i.e. inability to liquid shares at the given time. The central assumption of the majority of limit order models is to trade at the best bid and ask prices (see: [START_REF] Cao | Order placement strategies in a pure limit order book market[END_REF]). We allow the trading procedure to go to more in-depth into the LOB to avoid not filling the order and face last-minute inventory penalties.

Discussion and Further works

In this study, we proposed an analytical solution to the optimal liquidation problem, taking a dynamic approach and building numerical boundaries of multi-stopping problems in an illiquid market. We simulated optimal splitting order models according to the existing liquidity in the order book with different parameters and price impact models.

We used the PDMD to decompose the liquidation problem into discrete period problems, and applied Markov decision rules to obtain the solution. We examined the uniqueness and practical existence of the optimal solution. We showed that the percentage gain of the liquidation model depends on the market conditions and specifications of the price where λ 0 is a deterministic long run "base" intensity, which is assumed to be constant and α represents the magnitude of self-exciting.

The function f (t) expresses the impact of the past events on the current intensity process, and the parameter σ explains the magnitude of self-exciting and the strength of an incentive to generate the same event, see Figure 9. We define one-dimensional (1D) Hawkes processes as follows:

Definition 6. Hawkes Processes :

Hawkes (1971) defines Hawkes Processes as a linear self-exciting process with an exponential kernel function f (t) = αe -βt which is parameterised by constants α, β and β > 0. Simple one-dimensional (1D) Hawkes processes can be expressed:

λ t = λ 0 + t i <t 0 αe -β(t-t i ) (A.8) = λ 0 + t 0 αe -β(t-s) dN s , (A.9)
where β is the exponent of decay which represents the process of lessening an amount by a constant discount rate over a period.

Generally, the arrival pattern in a system can be modelled by Hawkes processes as a non-Markovian extension of the Poisson process. Each arrival in the system instantly changes the arrival intensity by α; then over time, these arrivals' impacts decay at rate β. In the self-exciting point process framework, this damping factor's magnitude indicates the Hawkes processes' self-exciting and self-damping properties. The former property corresponds to the positive value of β, while the latter corresponds to the negative value of β. An alternative choice for the kernel function f (t) is a power-law function. As a result of converting the integrated intensity into independent components via exponentially distributed variables, we can apply most of the analytical methods to analyze these statistical random variables. One can calibrate the 36 parameters of the Hawkes process with parametric estimation methods like maximum likelihood estimation [START_REF] Ozaki | Maximum likelihood estimation of hawkes' self-exciting point processes[END_REF] or with non-parametric estimators like the Expectation-Maximization (EM) algorithm [START_REF] Bacry | Non-parametric kernel estimation for symmetric hawkes processes. application to high frequency financial data[END_REF]. The goodness of the fit of these models also can be examined with conventional statistical tests. in a finite interval of length t. We consider the order with the maximum price of N (0,t) number of limit orders. Let F N (0,t) (y) be the distribution of y = max{S 1 t , • • • , S N (0,t) t }. where F(y) is the distribution of the price process and L is the number of unexecuted orders up to time point s. Where k = 0, this distribution function gives the probability of the first best order, k = 1 is the second best order, etc.

Indeed

Proof of Lemma 1. It is assumed that the limit orders arrival rate is a point process with intensity rate λ t . At each stopping time, we liquid q t shares of an illiquid asset with dynamics dq t γ = -γ t at price Ŝ = e -rt S t . The trading rate γ t is a control process. From Equation 3.1 we have:

V(T, Q 0 ) = sup γ∈A E t,q [H(T, Q 0 )] = sup γ∈A E t,q [ n i=1
e Proof of Proposition 1. Following SDE represents the impact of trading on the dynamics of the rate of orders' arrival:

dλ t = ( f (γ t ) -κλ t )dt + σdN t .
To prove, we can move the first term of SDE to the left side, then multiply it by e κt (see: [START_REF] Norberg | Vasiček beyond the normal[END_REF])), or we can define an initial guess for the solution of above SDE as follows: We defined the liquidation problem as a finite time investing problem on a limited time horizon T . λ ∞ represents the long-run trading impact on the intensity of order arrivals rate, we can think of it as a permanent price impact as "base" intensity part of stochastic intensity. It is a linear function of the trading rate to avoid dynamic arbitrage [START_REF] Gatheral | No-dynamic-arbitrage and market impact[END_REF]. We can express the permanently effected stochastic intensity by:

λ t = λ 0 + σ t 0 e -κ(
λ Perm t = λ ∞ + σ t 0
e -κ(t-s) dN s . (A.12)

Instantaneous market impacts can be measured from the small interval of trading, and the difference between the pre-trade and post-trade price movements: (t-s) ds ≈ e (-κt+αγ ε ) = λ ε .

We can simply define the instantaneously affected stochastic intensity by:

λ Inst t = λ ε + σ t 0
e -κ(t-s) dN s . (A.13)

Proof of Theorem 2. By using the information on the jump location from Z = (Z 1 , Z 2 , • • • , Z N ) as a set of post jump of PMDP , we have:

V π (t, q) = E t,q [ T 0 e -rt γ t s t λt dt] (t ∈ T)

refer to Equation :(5.3)

= E t,x [ T 0 U π (X T )dN t ] = E t,x [ N i=0 [ T i+1 ∧T T i U π (φ(X T i ))dN t ]
(Z i define as

Z i = [T i , X T i ]) = E t,x [ N i=0 E t [ T i+1 ∧T T i U π (φ(X T i ))dN t |Z i ]] = E t,x [ N i=0 R(Z i , π i (Z i ))]
= Ψ π (t, x).

We define a set Π = {π 1 , π 2 , • • • , π N } as a sequence of all Markovian decision controls π i corresponding to the Markovian policy for the predictable admissible process γ included in the set Ψ. We can then decompose this optimal control problem into the piecewise-deterministic Markov process:

V(t, q) = sup π∈Π Ψ(t, x).

Proof of Theorem 3. In the Theorem 2, we have shown that

V(t, q) = sup γ H γ (t, q) = sup π∈Π Ψ π (t, x),
with defining the sequence of Π = {π 0 , π 2 , • • • , π n } as set of Markovian policies [START_REF] Bertsekas | Stochastic optimal control: The discrete time case[END_REF], we have:

Ψ π n = lim n→+∞ T π 0 • T π 1 • T π 2 • • • T π n-1 Ψ π 0 (x 0 ) = lim n→+∞ (T π n ) n (Ψ π 0 (x 0 )) = T n (Ψ π n (x 0 )).
Therefore under the same condition, the optimal solution is defined as:

Ψ(x) = sup π∈Π (Ψ π (x)).
Equally Ψ(x) = T (Ψ(x)).

Figure 1 :

 1 Figure 1: Schematic model of trading boundaries built on market depth with Poisson arrival patterns

Figure 2 :

 2 Figure 2: Orders' arrival governed by Poisson process

  selects the next location of the jump given the current information on state and action. Each Markovian policy is a function of the jump time component (T k ) and the post jump location (Z T k ) with the following condition:

  develop quantization methods to compute the approximation of a value function of the optimal stochastic control. De Saporta et al. (2010) explain the implication of the numerical solution to PMDP, such that transmission function cannot be computed explicitly from local characteristics of PMDP. De Saporta et al. (2010) express a numerical solution for Embedded Markov chain, while the only source of randomness is a set of the post jump process (T n , Z n ). By quantization of Z n = (X T n ; T n ), we can transfer the conditional expectations into finite sums, and least upper bound of the value function (sup) into its maximum value (max) in the discretized space of [0, T ]. We define V as an approximation of the value function as follows:

Figure 6 :

 6 Figure 6: Trading rate of liquidation problem with constant order size

Figure 8 :

 8 Figure 8: Trading rate of liquidation Problem with different specification

Figure 9 :

 9 Figure 9: Hawkes Processes with 31 events

Figure 10 :

 10 Figure 10: Changes in price equilibrium as a result of increasing and decreasing of demands

F

  N (0,t) (y) = P[(S 1 ≤ y) ∩ (S 2 ≤ y)∩, • • • , S N ≤ y)] = F(y)N (0,t) The generating function with distribution function F(S t ) isG t (F(y)) = E[F(y) N (0,t) ] G t+dt (F(y)) = E[F(y) N (0,t+dt) ] = E[F(y) N (0,t) +N (t,t+dt) ] = E[F(y) N (0,t) ]E[F(y) N (t,dt) ] = G t (F(y))(1 -λdt + λdtF(y)) G t+dt (F(y)) -G t (F(y)) dt = -λ(1 -F(y))G t (F(y)) d dt (G t (F(y)) = -λ(1 -F(y))G t (F(y)) d dt ln(G t (F(y)) = -λ(1 -F(y))Solving for F(y) givesG t (F(y)) = e -λt(1-F(y)) (A.10)With assumption of a L number of unexecuted orders at the time point s, (s < t), the generating function for the time interval (0, s) is: with remainder:G s (F(y)) = e -λ (e λ(F(y)) -f (c)λ L+1 (L + 1)! ) (A.11) For c ∈ [0, 1], f (c) = F(y) L+1 ≈ e cλ .For the sake of simplicity, it is assumed that c = 0.G t (F(y)) = G s+(t-s) = G s (F(y))G t-s (F(y))From Equations (A.10) and (A.11), the generating function of time interval ((ts), t):G t-s (F(y)) = e -λ(1-F x (y))e -λ ln(e λF(y)-λ L+1(L+1)! ) = e -λ[(1-F x (y))-ln(e λF(y) -λ L+1 (L+1)! )]Consider the number of orders in the coming stopping time is a random variable with Poisson distribution and mean value λt. From the above generating function, we can define the probability that no order arrival in time interval (0, t) is said to have a Poisson distribution greater than y: P(U = k|N (0,s) = L) = e -λ y • λ k y k! λ y = λ[(1 -F(y)tln(e λF(y) -λ L+1 (L + 1)! )]

  n where T k is an increasing sequence of the jump time component and X t and Z t are a jump location component and a post jump location component, respectively:(Z t = X t if t = t k ).This process starts at the state x t , and jumps with the Poisson rate process Λ t (fixed or time-dependent) to the next state or hits the boundary of the state space. Stochastic Kernel K(.|x t , a t ) by measuring the transmission probability

Table 1 :

 1 Summary of the result of simulation the level of inventory and its corresponding optimal trading rate under different market conditions

	Panel I

  -rT i γ t S t 1 (T

i ≤T ) ] (when n → ∞)(t ∈ T) = sup γ∈A E t,q [ T 0 e -rt γ t S t dN t ] = sup γ∈A E t,q [ T 0 e -rt γ t S t λ t dt]

  t-s) dN s , 0 Verify by Itô's lemma on e κt λ t e κt λ t = D(0) + e κt t 0 f (τ s )e -κ(t-s) ds + e κt σ κe κt λ t dt + e κt dλ t = f (γ t )e κt dt + σe κt dN tκλ t dt + dλ t = f (γ t )dt + σdN t dλ t = ( f (γ t ) -κλ t )dt + σdN t .Proof of Lemma 2. From Equation (A.12) we have:

				t
				e -κ(t-s) dN s
				0
	t		t
	=	f (γ s )e κs ds + σ	e κs dN s
	0		0
	where
		t
	λ 0 = e -κt D(0) +	f (γ s )Γ(t -s)ds.
		0

λ

impact function. In contrast to most limit order models for liquidating a market, which only trade at the best bid and ask prices, our model allows the trading to go deeper into LOBs to avoid situations in which orders are not filled until the last time periods.

We believe that a possible area for further research would be to study the optimal liquidation problem for multiple assets [START_REF] Tsoukalas | Dynamic portfolio execution[END_REF] across trading venues (see: e.g. [START_REF] Barclay | Competition among trading venues: Information and trading on electronic communications networks[END_REF], known as the optimal splitting problem, by considering different trader types (see, e.g., [START_REF] Brogaard | Price discovery without trading: Evidence from limit orders[END_REF] in lit or dark markets (lit or dark pools) (see, e.g, [START_REF] Comerton-Forde | Dark trading and price discovery[END_REF]. In this framework, a trader might generate profits by pushing up prices at the traditional venue and parallel trading in the multi-platform. We could apply the same modelling approach to the arrival flow of orders and solve the problem using the interpretable optimal stopping modelling approach developed by [START_REF] Ciocan | Interpretable optimal stopping[END_REF].

Acknowledgments

Jan Vecer's work was partially supported by Grant Agency of the Czech Republic under grant 21-19311S.

A. Appendix

A.1. Hawkes process

The concept of a point process is fundamental to the stochastic process. Before we explain the dynamic of the Hawkes process, we state the following formal definition of a point process, a counting process, and an intensity process. For more explanation about point process and intensity process, we refer to [START_REF] Giesecke | Exact simulation of point processes with stochastic intensities[END_REF].

Definition 2. Point Process : Let t i (i ∈ N) be a sequence of non-negative random variables, which is measurable on the probability space (Ω; F ; P), in such a way that ∀i ∈ N, t i < t i+1 , is defined as a point process on R.

Definition 3. Counting Process :

The right-continuous process N t = i∈N 1 t i ≤t ,with given a point process t i (i ∈ N), is a so-called counting process if it measures the number of discrete events up and including the time point t.

Definition 4. Intensity Process :

With given N t as a point process adapted to a filtration F , the intensity process λ as a left-continuous process is defined by:

To be more precise, a intensity process λ t is determined by a counting process N t with the following probabilities:

Homogeneous Poisson process is a so-called intensity process that is independent of the probability of the occurrence in the small interval ∆t and a filtration F t .

Definition 5. Linear self-exciting process :

A general form of a linear self-exciting process can be expressed:

which is a subset of the Banach space, so we can apply the Banach fixed point theorem and show that V is an unique fixed point of the operator T on the set Π.