Łukasz Bartosik
email: lbartosik@ichf.edu.pl

Janusz Stafiej

Dung Di Caprio

Bartosik Lukasz

Stafiej Janusz

Di Caprio

Corrosion-passivation processes in a cellular automata based simulation study

Keywords: Anodization, Cellular Automata, Parallel programing, Aluminum oxide growth

published or not. The documents may come

Introduction

It is known for almost 100 years that several metals can be coated by the passive layer of oxides formed in an electrochemical process called anodization.

It has been used extensively to increase corrosion resistance and aesthetical outlook of the surface. Anodization is a rather simple procedure that can be applied to a variety of metal surfaces e. g. titanium [1], aluminum [2] and hafnium [3]. In the process the surface between the metal and a proper electrolyte solution is polarized by an external voltage producing metal anodic oxidation and related phenomena. The peculiarity of structures that evolve on anodized aluminum surface was first noticed in 1953. Keller, et. al. [4] showed with transition electron microscopy that the alumina layer displayed hexagonally ordered pores.

For a considerable period of time this phenomenon attracted little attention.

A renewed interest comes from its potential nanotechnological applications as scaffolds for other nanostructure synthesis [5]. Masuda et. al. [6] determined the experimental procedures for the preparation of regular nanoporous anodic aluminum oxide layers. Then other advancements in the problem started appearing one after another as evidenced by the works of Li [7], Jessensky [8] and Masuda [9]. A major obstacle to obtain nicely organized pore structures is the initial state of the layer. It is fairly disorganized due to defects and initial surface roughness. It needs a thorough pre-treatment of the surface by polishing, chemical and electrochemical etching etc. In this way the so called two step anodization is recommended by Masuda [10]. It consists of two steps. The first step reduces the number of defects and prepares the first network of pores. The layer is then washed out but it leaves an imprint of the pore organization on the metal surface. Then the second anodization produces a better quality organized pore network grown on top of the imprints of the previous one.

The theoretical work on the phenomenon is clearly delayed and unable to describe all the aspects of the process. There are two most considered approaches to base a theoretical description on: Field Assisted Dissolution(FAD) [11] and Field Assisted Flow(FAF) [12]. Sheintuch and Smagina [13] consider the problem of stability for the Field Assisted Flow model by Singh, et. al [14] and arrive at the Damped Kuramoto Sivashinsky (DKS) type equation. The model does predict a hexagonally ordered porous layer. But there are two weak points in the approach. The first point is that their analysis applies to the initial stage of the pore growth and cannot guarantee their sustained growth. The second point is that their two dimensional model can be inapplicable in three dimensions. Indeed, Gomez and Paris [15] explore the regimes of the Damped Kuramoto Sivashinsky equation and conclude that despite stable regimes of the DKS equation in a 2D space they find only chaotic regimes in three dimensions.

Stable regimes for the 3D version of the equation need not exist and more research is needed to clarify this issue. In view of the above doubts on viability of 3D FAF model in this work we focus on a 3D Field Assisted Dissolution model.

In this work we design a parallel cellular automata model to simulate the 3D Field Assisted Dissolution model. We explain the rules and states of the model as well as explain our parallelization strategy in the Methods section. We test the strategy on a simple problem of diffusion between emitting and absorbing wall. With the positive test results we pass to simulations of anodization using the verified strategies. We compare our results to theoretical predictions of the Field Assisted Dissolution model

Methods

Cellular automata model

Our model is based on a probabilistic, three dimensional, asynchronous cellular automaton. We consider periodic boundary conditions along the two axes parallel to the initial surface while fixed boundary conditions are applied along the third axis. We use the 3D cubic lattice with the Moore neighborhood for a given cell. In our update scheme each lattice site is given the same chance for an update in one step of system evolution. Then one neighbor is selected from the 3D Moore neighborhood of the updated site. Each neighbor has a uniform probability of 1/26 to be selected as 26 is the coordination number for the Moore neighborhood in 3D. Let us note that the pair of sites thus selected has a double chance for selection since the neighbor and the site in another instance may change the roles. The pair thus selected is subject to an update and the rules of updates are symmetrical in that they ignore the distinction between the site and the neighbor. The pair as a whole is updated according to the formal two body reaction scheme depending on the states of the two sites. of our model. These are: M the state corresponding to the metal, OX -to the oxide, EF -to the walker representation for electric field in the oxide that will be detailed further, A -to anion rich sites in the oxide S -to solvent. Let us note that the modeled oxide layer is represented by cells in one of the three states: OX, EF and A. We call them oxide-like states. All of these cells are considered to be filled with oxide and two of them

M + OX = M + EF (1)
should be therefore read as: An M particle in site A reacts with an OX particle in site B yielding an M particle in site A and an EF particle in site B. In the above reaction, we omit an explicit notation for the sites A and B, as in the course of the simulation they correspond to a random pair of neighbouring sites chosen with the algorithm detailed thereafter.

the particle at A comes before that at B:

M + OX = M + EF (2)
has a different final effect than the reaction:

M + OX = EF + M (3)
The difference comes from the fact that in reaction 2 the M particle stays at the site A while in reaction 3 it moves from site A to site B. As mentioned above neighbouring sites A,B are equivalent to B,A. Thus reaction

OX + M = EF + M (4)
is equivalent to reaction 2 and need not be duplicated.

The rules we postulate are as follows:

M + S = OX + OX (5)
EF + S = S + S

EF + S = A + S (7)

M + A = OX + OX (8)
M + OX = M + EF (9)
EF + OX = OX + EF (10)

EF + A = A + EF (11)
S + OX = OX + S (12)
S + A = A + S (13)
These 9 rules can be divided into three groups: reaction-like rules (5 to 9), diffusion-like rules (10 and 11) and surface reorganization (12 and 13). Let us rephrase you mean emphasize ? that states OX, EF and A are all oxide-like states, containing a portion of oxide. In the following paragraphs we explain are present within the oxide layer. These rules remind diffusion but there is a difference with the diffusion rules mentioned previously. The decision if a swap of particles occurs is based on two factors: the neighborhoods of the locations chosen for the swap and a predefined probability, P BOND , called "unbonding"

probability. If the solvent site has more oxide neighbors then the oxide or anion site a swap occurs. In the opposite case a swap may still occur with a probability given by the power law:

P = P N BOND ( 14
)
where P BOND is the "unbonding" probability and N is the difference of the number of neighbors between the sites. Additionally if an oxide site has no oxide neighbors it dissolves into a solvent. The relative frequency of reactionlike, diffusion-like and surface reorganization events can be set at the beginning of the simulations.

Modeling the effect of electric field

The aspect of this work worth pointing out is the way the presence of electric field is accounted for. We share many general features with the model of anodization developed by Parkhutik and Shershulsky [16]. To keep it simple we treat the metal-oxide and oxide solvent surfaces as equipotential and thus avoid hypotheses about eventual double layers that appear in a microscopic description for metal-layer and layer-solvent interface:

φ Ox\S = 0 ( 15
)
φ M \Ox = U M \Ox ( 16
)
where φ Ox\S is the potential at the oxide-solvent interface, φ M \Ox is the potential at the metal-oxide interface and U M \Ox is the voltage applied to the system.

To find the electric field we use the Poisson equation:

∇ 2 φ = ρ f ϵ (17)
where ρ f is the free charges density and ϵ is the electric permittivity of the layer.

We neglect the free charges contribution so ρ f = 0 and the Poisson equation is reduced to a Laplace equation

∇ 2 φ = 0 (18)
The Laplace equation is also encountered in a problem of diffusion defined as:

c interface1 = 0 ( 19
)
c interface2 = c 0 (20)
where c interfacen is the concentration of diffusing particles at particular interfaces and c 0 is a fixed concentration. The Fick's diffusion equation is as follows:

∂c ∂t = D∇ 2 c (21)
In steady state ∂c ∂t = 0 and the equation simplifies to a Laplace equation:

∇ 2 c = 0 (22)
We therefore conclude that one can solve the Poisson equation in this case via methods used to solve the steady-state diffusion equation. It is well established that the steady-state diffusion equation can be solved using the Random Walk [17]. In the case of anodization no steady-state is reached, and both the metal/oxide and oxide/solvent interface move with time. We assume that a quasi-stationary state is present in the system and hence ∂c ∂t ≈ 0. We justify our assumption on the basis that the rate of diffusion is at least two orders of magnitude greater than the rate of interface movement. This stems from the fact that one diffusion step is taken per time step. And typically for the parameters used in our simulation, the interfaces move overall approximately 150 lattice sites in about 40000 time-steps. With other parameters our assumption need not hold as the walker distribution may not reach the stationary form. With these stipulations in mind we decide to employ the Random Walk approach to model the electric field in our system via the diffusion of EF walkers. We can then follow the rearrangement of the electric field distribution which conditions the reactions at the interfaces in terms of their spatial evolution.

Parallelization

In order to shorten the time required for the simulations we use of the ca- After a given substep the global memory of the card is updated and another substep commences. We have verified that in some cases the code using shared memory is 5 times faster than that based on global memory updates only. With a predefined data collection frequency the content of global GPU memory is copied from the device onto the CPU and output to files. It is mainly in the CPU that simulation data are processed to extract the shapes of the interfaces, Fourier transforms of cross-sections, distributions of particles. This can be done with conventional tools so that CUDA resources are not engaged in these tasks.

Results and discussion

All simulations are performed (to avoid using twice results/results) on the Tesla C2070 card. The initial results of parallelization yield a significant increase in simulation speed compared to sequential code and we decided to work exclusively with the parallel code. A comparative study of the code efficiency certainly interesting in other context is beyond our scope.

Diffusion

First we verify if our approach to diffusion and parallelization yields reasonable physical results. We simulate a system composed of a single type of particles in a three dimensional lattice. In this system the bottom layer sites are filled with diffusing particles. The particles disappear when they arrive / arriving at the top layer on the opposite side of the box. The analytic solution of the problem of steady state diffusion in this geometry is well known. According to it the particles form a linear distribution along the direction perpendicular to the sides. We test for any possible effect of spurious correlations that may arise in our scheme of update in the parallel algorithm. The linearity of the distribution of the diffusing particles is the indication that our system is free from such effects. To confirm our findings we calculate the diffusion coefficient resulting from our simulations and compare it to the theoretical diffusion coefficient that we should obtain for a three dimensional cellular automaton using the Moore neighborhood. The value of the theoretical diffusion coefficient is obtained using the following calculation:

< R 2 >= 2D × d × T ( 23
)
where < R 2 > is the mean square displacement, D is the homogeneous diffusion coefficient, T is time and d is the dimensionality. In our case d = 3.

The mean square displacement calculated for the Moore neighborhood is given as follows. There are a total of 26 neighbors to a site, 6 of them are at square We compare this theoretical value to values obtained in our simulation via the Fick equation:

J = -D × ∆c ∆x (26)
The flux J is obtained by counting the particle creation and disappearance events at opposite boundaries of the system along the Z axis. We assume that the steady state is reached when number of annihilations approaches the number of creations and both are equal within the relatively small fluctuation. Then we average the distribution for a period of simulation time steps. This average particle distribution over the simulation time period after reaching the steady state is plotted and a linear fit of the result is performed via standard methods.

As seen in figure 3 the R 2 parameter for the linear regression is over 0.9999 which signifies that the distribution plot is a straight line indeed. Otherwise we would be forced to admit that the spurious effects prevent the program from reproducing well established diffusion behavior. The average numbers of particles created and destroyed in a specific time interval gives us an independent estimate of the diffusion coefficient. It is readily obtained from the common value of these numbers divided by the absolute value of the slope of the curve and additionally divided by 2 due to the symmetry of the used procedures with every pair of sites having two chances to undergo a diffusion step. Our simulation yields the diffusion coefficient D = 0.34602, in the range of 0.1% from the expected result of 0.346154.

The simplest Random Walk numerical experiments consider a single particle displaced at random to neighboring sites. Our parallel approach converges with these experiments well in the case of low concentrations of moving particles.

Then the probability of two mobile species meeting and one interfering with another's movement is low. The movement of each particle may be considered independent. This assumption need not hold in regions of high concentrations of moving particles and one may not rule out an effect on the diffusion process.

However, lack of deviation from linearity even in regions of high concentrations (close to 1 particle per site) is a good indication that if such an effect exist it is negligible. We conclude that our procedure to simulate diffusion does not contain any significant error leading to obviously unrealistic results and can be considered for steady-state diffusion even at high concentrations.

Ordered structures

In this section we present the results of simulations using our approach.

All of the simulations were conducted in systems of 100 by 100 by 200 sites using constant simulation parameters throughout each run. Regardless of the parameters all simulations concluded within 30000 to 35000 time steps. Figure To address this problem we present a cross-section taken at half oxide layer 300 height and a Fourier transform of this cross-section in figure 5. The cross-section confirms the existence of developed well separated pores. The Fourier transform indicates a hexagonal symmetry. It is noteworthy to acknowledge the neighborhood selected for the cellular automaton is the Moore neighborhood, which has a quadratic symmetry. We thus conclude that the observed hexagonal pattern does not originate from our choice of the neighborhood for the automaton. We further calculate the Fourier transform modulus with respect to the wave vector modulus to find a characteristic length between the pores. The results are presented in figure 6. The peak in figure 6 corresponds to the characteristic length present in the system. In figure 7 we present the distribution of oxide-like states (oxide, electric field and anion) along the Z-axis of the system. This distribution can be described as having two peaks and a plateau between them. The peak corresponding to low Z-axis values corresponds to pore bottoms, where more oxide material is present. The plateau is the region where a stable porous layer exists. The second peak nearing the "top" of the system is the result of the simulations starting conditions. Starting the simulation from a planar geometry between the metal and solvent layers creates an overflow of oxide material at the initial stage of the simulation (passivation of the active metal rule).

This material then aggregates due to surface tension and stays relatively stable throughout the simulation.

Two step anodization

In order to test our simulation against a real world experiment we decide to simulate two step anodization. The expected results is a better organized porous structure with an unchanged characteristic length and shape. We use a metal-oxide interface obtained in a source simulation as the starting interface for the second step of anodization. This is meant to model the washing off In figure 8, we present a side by side comparison of simulations of single step and two step anodizations. Both cross-sections are very similar between the simulations and both show similar porous structure. It is impossible to tell if an improvement in organization has been achieved just from these results. To highlight the differences between the layers we compare the Fourier transforms of the above cross sections in figure 9 and the comparison of Fourier transform moduli in figure 10 . The Fourier transforms show that the characteristic length of the system has not changed between the simulations. However, a hexagonal pattern is much more apparent in the two step simulation transform. Figure 10 shows that the location of the peak in Fourier transform modulus is unchanged, meaning that the characteristic length of the system is the same in both simulations. These result signify that the layer obtained from a previously obtained interface is indeed more organized compared to the original layer. Last we compare the particle profiles in figure 11. The overall shape of the profiles is similar, however two differences require an explanation. The first difference is the slight shift of the first peak (corresponding to pore bottoms) towards lower Z-axis value. The second is different shape of the second peak near the "top" of the system. Both of 12 were embedded into the structure. This fact means that some particles of solvent of the loaded system were initially lower along the Z-axis than analogous particles in the initial system. The change of the shape of the "top" peak can be explained much in the same manner. The early stage oxide structures were different in both cases producing a different outcome globally. Despite these differences the overall shape of the distribution is similar in both simulations and the concentrations of oxide-like particles along the Z-axis remains unchanged between the simulations. This corresponds to both structures displaying the same porosity.

Structure diagram

We present our data from one hundred simulations in figure 12. The aim of these simulations was to investigate the influence of two parameters: "unbonding" probability and oxide dissolution chance in high electric field. Simulations show a variety of results. These result are in good agreement with theoretical predictions of Stanton and Golovin [19]. There is a clearly visible region of non-porous structures. The impact of both investigated parameters is visible, "unbonding" probability seems to have a particularly significant effect. In figure 13, we present some non-porous structure which we have obtained. The existence of such structures highlights the possibility of using our model or its derivative to simulate organized structures other than porous layers.

Conclusions

We have designed and implemented a three dimensional asynchronous cellular automaton for the purpose of simulating anodization of aluminum. To that end we employed the computational power of GPU on Tesla Nvidia cards.

We introduce to the parallelization scheme features related to CUDA resources such as on board random number generation and fast access shared block memory. The algorithm is tested on a simple three dimensional diffusion problem.

Results of this test are consistent with predictions and we believe that our algorithm does not cause unphysical behavior to emerge. Results of our simulations match other theoretical models on a qualitative level and we were able to obtain an ordered hexagonally symmetric porous layer. Moreover we have reproduced the qualitative effect of performing two step anodization which results in an improvement in ordering of the structure if the anodization is performed on a pre-treated metal surface.

  EF and A have an additional function. States EF should be interpreted as OX electrostatic field carriers such that the average flux of these particles in a stationary state yields the magnitude of electric field. States A are considered as anion (typically superoxide O 2-or hydroxide OH -ions) rich OX oxide and the anions are dragged by the electric field towards the metal anode. It is convenient often to think of states as of particles. Our update rules apply to two particles at two neighboring sites A and B where at update they can undergo a binary reaction: P article1(position A) + P article2(position B) = P article3(position A) + P article4(position B) which can be selected with a given probability relative to the obvious alternative reaction channel, in which nothing is changed and leaving the selected pair of sites in their actual state: P article1(position A) + P article2(position B) = P article1(position A) + P article2(position B) For example the following reaction

100 2 . 3 . 4 .

 234 how we apply the rules in actual coding. Four of the five reaction-like rules can be viewed as descriptions of the well known electrochemical processes taking place during anodization. These are:1. Passivation of the active metal (rule 5) Dissolution of the oxide in high electric field (rule 6) Dissociation of the solvent and anion incorporation into the oxide layer in high electric field (rule 7) Oxidation of the active metal by the incorporated anions (rule 8)The final reaction-like rule creates electric field walkers that we use to model the presence of electric field in the oxide layer. A detailed explanation of how we account for the electric field presence is given in the next subsection. All of the reaction-like rules have a fixed probability during simulations. Additionally the probabilities of rules 6 and 7 sum to 1.The diffusion-like rules govern the Random Walk of electric field walkers and the movement of anions inside the oxide layer. The Random Walk is modeled by two appropriate sites swapping their positions. These rules differ from reaction-like rules in the sense that they conserve the species involved leading to their spatial redistribution. For the most part we set their probability to 1 if the correct particles encounter one another. However in all other aspects they are identical to reaction-like rules and are parallelized in the same way. In our model we assume that anion movement is mostly caused by the strong electric field present in the oxide layer. During anodization the electric field arises from the onset of the voltage between the metal and the solution. The electric field is strongest at the thinnest metal-solution separation by the oxide layer and becomes weaker as the oxide layers gets thicker. Rule 11 represents this behavior well. It creates the drift of anions in the direction opposite to the gradient of the electric field walker concentration. The concentration gradient of the electric field walker is balanced in quasistationary conditions by the electric walker flux and then both can be thought to mimic the presence of electric field. Rules 12 and 13 govern surface reorganization. The goal is to introduce a form of surface tension to prevent excessive ramification of the obtained structures. Oxide particles are held in place by bonding to other oxide particles. Let us recall that oxide-like means oxide, anion and electric field states as they all

Figure 1 :

 1 Figure 1: A schematic 2D representation of a pore forming in the model. Arrows connect a chosen site to its selected nearest neighbor. States are depicted by different colors.

  pabilities of GPU. Employing CUDA Tesla cards requires however running a parallelized program. The abbreviation CUDA stands for Compute Unified Device Architecture[18]. We base our approach to parallelization on the blocksynchronous method presented by Olga Bandman [? ? ]. The updates according to each rule-type group should need to occur with uniform probabilities for every site in the lattice. Let us recall that a kernel is a CPU launched procedure executed on GPU controlling multiple threads running in parallel bunches. Thus we organize our parallel computing in three separate kernels: reaction-like rule kernel, diffusion-like rule kernel and surface reorganization kernel. In our mind the kernel launch of one of the three types constitutes a substep in the single time step of the simulation. The number of substeps to make a step comes from the following organization of the calculation. We divide the grid of sites

Figure 2 :

 2 Figure 2: The parallelization scheme for two adjacent 5 × 5 blocks on a 2D example. The red and green squares represent respectively the active site and its selected neighbor with their respective neighborhoods hatched right and left and in both directions for common sites. The 5 × 5 blocks guarantee that no overlap takes place between neighborhoods from two adjacent blocks.

distance 1 , 6 × 1 + 12 × 2

 161122 these points are the centers of the cubes walls, 12 are at a square distance 2 each of them at the center of cube edges and the corners of the cubes (8 points) are at a square distance 3. The mean square displacement is therefore: -50,30x+ 9980 R 2 > 0.9999

Figure 3 :

 3 Figure 3: Number of particles distribution profile.

Figure 4 :

 4 Figure 4: A side view of a simulated oxide layer. Ox\S interface is shown.

Figure 5 :Figure 6 :

 56 Figure 5: A cross-section of a organized porous layer (left) and the Fourier transform of the cross-section(right). In yellow the Ox\S interface, the oxide in between pores is not shown.

Figure 7 :

 7 Figure 7: Oxide layer profile

Figure 8 :

 8 Figure 8: Comparison of cross-section of source (left) and loaded (right) simulation.

Figure 9 :Figure 10 :

 910 Figure 9: Comparison of Fourier transforms of cross-sections of source (left) and loaded (right) simulation.

Figure 11 :

 11 Figure 11: Particle profile comparison

Figure 12 :

 12 Figure 12: Structure diagram.

contains a side view of the oxide-solvent interface in one of our systems. A porous structure is clearly visible, however not much can be said about how organized it is.

Acknowledgements

The work of L. B. was realized within the International PhD Projects Programme of the Foundation for Polish Science, cofinanced from European Regional Development Fund within Innovative Economy Operational Programme "Grants for innovation" . The work has also benefited from Polish NCN grant no. N N204 139038.


Student Member, IEEE Yiqun Liu

Fellow, IEEE Marc Riviere

Fellow, IEEE Thomas Guionnet

Fellow, IEEE Aline Roumy

Fellow, IEEE Christine Guillemot

TRANSACTIONS ON IMAGE PROCESSING 1 CNN-based Prediction of Partition Path for VVC Fast Inter Partitioning Using Motion Fields

Keywords: VVC, multi-scale motion vector field, VTM, QTMT, inter partitioning acceleration, U-Net, multi-branch CNN, multi-class classification

The Versatile Video Coding (VVC) standard has been recently finalized by the Joint Video Exploration Team (JVET). Compared to the High Efficiency Video Coding (HEVC) standard, VVC offers about 50% compression efficiency gain, in terms of Bjontegaard Delta-Rate (BD-rate), at the cost of a 10-fold increase in encoding complexity. In this paper, we propose a method based on Convolutional Neural Network (CNN) to speed up the inter partitioning process in VVC. Firstly, a novel representation for the quadtree with nested multi-type tree (QTMT) partition is introduced, derived from the partition path. Secondly, we develop a U-Net-based CNN taking a multi-scale motion vector field as input at the Coding Tree Unit (CTU) level. The purpose of CNN inference is to predict the optimal partition path during the Rate-Distortion Optimization (RDO) process. To achieve this, we divide CTU into grids and predict the Quaternary Tree (QT) depth and Multi-type Tree (MT) split decisions for each cell of the grid. Thirdly, an efficient partition pruning algorithm is introduced to employ the CNN predictions at each partitioning level to skip RDO evaluations of unnecessary partition paths. Finally, an adaptive threshold selection scheme is designed, making the trade-off between complexity and efficiency scalable. Experiments show that the proposed method can achieve acceleration ranging from 16.5% to 60.2% under the RandomAccess Group Of Picture 32 (RAGOP32) configuration with a reasonable efficiency drop ranging from 0.44% to 4.59% in terms of BD-rate, which surpasses other state-of-the-art solutions. Additionally, our method stands out as one of the lightest approaches in the field, which ensures its applicability to other encoders.

I. INTRODUCTION

A CCORDING to [1], global internet traffic has increased substantially, primarily due to the growing video usage, which now accounts for 65% of internet traffic. In addition, the rapid development of Ultra-High Definition (UHD) and Virtual Reality (VR) makes it critical to design more efficient video compression codecs. For this purpose, the latest video coding standard VVC has been finalized in 2020. In comparison to its predecessor, HEVC, its efficiency of inter coding is boosted by about 50% in terms of BD-rate at the cost of 10 times higher complexity [START_REF] Bross | Overview of the versatile video coding (vvc) standard and its applications[END_REF]. The substantial complexity of VVC impedes Manuscript received October 10, 2023 Yiqun Liu, Aline Roumy and Christine Guillemot are with the IN-RIA (Institut National de Recherche en Informatique et en Automatique) Rennes Bretagne Atlantique, Rennes, France (e-mail: yiqun.liu@irisa.fr; aline.roumy@inria.fr; christine.Guillemot@inria.fr).

Yiqun Liu, Thomas Guionnet and Marc Riviere are with the ATEME company Rennes Bretagne Atlantique, Rennes, France (e-mail: y.liu@ateme.com; m.riviere@ateme.com; t.guionnet@ateme.com). its direct implementation in real-time applications such as TV broadcasting. Apart from multiple newly added inter coding tools [START_REF] Wang | Adaptive motion vector resolution scheme for enhanced video coding[END_REF]- [START_REF] Alshin | Bi-directional optical flow for improving motion compensation[END_REF], a novel partition structure introduced in VVC, called QTMT [START_REF] Huang | Block partitioning structure in the vvc standard[END_REF], is the main contributor to this complexity surge. In particular, it has been observed in [START_REF] Tissier | Complexity reduction opportunities in the future vvc intra encoder[END_REF] that the VVC Test Model (VTM) encoder, which is an implementation of the VVC codec, dedicates 97% of its encoding time to searching for the optimal partition. Consequently, fast partitioning methods emerge as the most promising approaches to speed up the whole VVC encoding process.

A. Partitioning Acceleration for VVC 1) Fast Intra Partitioning Methods

Numerous works achieve an important acceleration of intraframe partitioning in the All-Intra (AI) encoding configuration. These approaches fall primarily into two categories: heuristicbased methods and machine learning-based methods. Some heuristic-based methods are built upon pixel-wise statistics, such as gradients [START_REF] Fan | A fast qtmt partition decision strategy for vvc intra prediction[END_REF]- [START_REF] Chen | Fast qtmt partition decision algorithm in vvc intra coding based on variance and gradient[END_REF] and variances [START_REF] Fan | A fast qtmt partition decision strategy for vvc intra prediction[END_REF][START_REF] Chen | Fast qtmt partition decision algorithm in vvc intra coding based on variance and gradient[END_REF]. To simplify the partitioning process, other heuristic methods reuse some data generated during the encoding process, such as Rate-Distortion cost (RD-cost) of Coding Unit (CU) encoding [START_REF] Lei | Look-ahead prediction based coding unit size pruning for vvc intra coding[END_REF], coding tool decisions [START_REF] Saldanha | Fast partitioning decision scheme for versatile video coding intra-frame prediction[END_REF], best split type, and the intra mode of sub-CUs [START_REF] Fu | Fast cu partitioning algorithm for h.266/vvc intra-frame coding[END_REF].

Machine learning-based methods utilize CNN or Decision Tree (DT) models to expedite intra partitioning. In [START_REF] Galpin | Cnn-based driving of block partitioning for intra slices encoding[END_REF]- [START_REF] Wu | Hg-fcn: Hierarchical grid fully convolutional network for fast vvc intra coding[END_REF], a CNN model is trained to predict the split boundaries inside CTU partitions. In [START_REF] Feng | Partition map prediction for fast block partitioning in vvc intra-frame coding[END_REF], Feng et al. propose a fast partitioning method by predicting a QT depth map, multiple MT depth maps, and multiple MT direction maps with CNN. Regarding the DT-based approach, various Light Gradient Boosting Machine (LGBM) classifiers are separately trained for different CU sizes to predict the possible splits, as demonstrated in [START_REF] Saldanha | Configurable fast block partitioning for vvc intra coding using light gradient boosting machine[END_REF].

2) Fast Inter Partitioning Methods

Fewer contributions of fast inter partitioning methods have been proposed for VVC. Due to the fact that the inter coding consists of predicting pixels of current frame depending on previously encoded reference frames, encoding errors resulting from the use of fast coding methods are propagated and accumulated between frames. Therefore, the acceleration of inter partitioning is a more challenging task compared to that of intra partitioning. Nevertheless, the acceleration of interframe coding is key to speeding up the overall encoding process, especially in RandomAccess (RA) and Low-Delay (LD) configurations. These configurations are employed more widely than the AI configuration in scenarios such as broadcasting and streaming.

Generally, fast partitioning approaches aim to reduce the search space of potential partitions. Therefore, accurately predicting the subset of partitions is of crucial importance. Heuristic methods proposed for fast intra partitioning of VVC [START_REF] Fan | A fast qtmt partition decision strategy for vvc intra prediction[END_REF][START_REF] Fu | Fast cu partitioning algorithm for h.266/vvc intra-frame coding[END_REF] heavily depend on handcrafted features to determine whether to check a partition. These methods are fast and simple to implement but lack accuracy for two reasons. Firstly, the features are computed locally on the CU and/or sub-CUs, which fails to provide a synthesized view of the entire CTU. Secondly, these features, including variances, gradients, and coding information, are low dimensional and do not adequately capture the complexity of CTU.

One approach to improve the accuracy of partition prediction involves increasing the dimension of the extracted features. This is the case with the methods based on Random Forest (RF) [START_REF] Amestoy | Tunable vvc frame partitioning based on lightweight machine learning[END_REF] or DT [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF], which use over 20 features from a given CU and its sub-CUs. As a result, decisions made by these methods remain confined to the local context of CU, without considering the entirety of CTU. Rather than relying on local information, a more effective selection of subsets of partitions should be based on global features computed on the entire CTU. This can be accomplished through the utilization of CNN-based methods.

Several approaches [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF]- [START_REF] Liu | Lightweight cnn-based vvc inter partitioning acceleration[END_REF] use CNN to partially accelerate the partition search process. For instance, in [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF], Pan et al. propose a multi-branch CNN to perform a binary classification of the "Partition" or "Non-partition" at the CU level. In [START_REF] Yeo | CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction[END_REF], the split type at the CTU level is predicted, whereas the partitions of its sub-CUs are not determined. Liu et al. in [START_REF] Liu | Lightweight cnn-based vvc inter partitioning acceleration[END_REF] employ a CNN to estimate an 8x8 grid map of QT depth, which is used to discard a portion of the MT splits. These methods cover only a part of the partition search space, while the partition search is conducted exhaustively on the remainder. These methods could be referred as partial partitioning acceleration methods by CNN.

A complete partitioning acceleration of inter coding by CNN is proposed in [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF]. A vector containing probabilities of the existence of split boundaries in the partition is predicted similarly to [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning scheme for vvc intra encoders[END_REF]. This method is fast in the sense that a single vector is computed for each CTU. Nevertheless, it is observed in [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning scheme for vvc intra encoders[END_REF], that the predictions are more accurate at higher levels of the partitioning tree. Hence, they propose improving the decisions by adding 16 trained DTs to process the CNN output, introducing additional complexity to the method.

B. Proposed Method

In the MT partitioning, both binary and ternary (with sub-CUs of two different sizes) splits are available. Consequently, CUs at a specific depth in the tree do not correspond to the same size and shape, introducing dependence between the MT splits along the partition path. This dependence partly explains the decrease in partition prediction accuracy as the depth of the partitioning tree increases, as observed in recent studies [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF][START_REF] Tissier | Machine learning based efficient qt-mtt partitioning scheme for vvc intra encoders[END_REF] presented in the previous section.

More precisely, since the size and shape of a CU depend not only on its depth in the tree but also on consecutive MT splits, depth alone is insufficient for defining a partition. Therefore, we propose making decisions on MT partitioning in a hierarchical manner, considering their dependence on the partition path. We also introduce a one-shot approach for QT partitioning which precedes MT partitioning, since there is a one-to-one correspondence between the QT depth and the CU size at that particular depth.

Hence, our overall proposition involves predicting the partition path, which includes a one-shot prediction for the QT partitioning, followed by a hierarchical prediction for the MT partitioning. Additionally, to further improve the accuracy of partition prediction, we suggest basing the partition decision not only on pixel values and residual values but also on motion vector fields, as these fields exhibit a strong correlation with partitioning [START_REF] Amestoy | Tunable vvc frame partitioning based on lightweight machine learning[END_REF].

Our two main contributions are as follows:

• We propose a novel partition-path-based representation of the QTMT partition at the CTU level as a map of QT depth plus three maps of MT split well adapted to the sophisticated partitioning scheme in VVC.

• We design a U-Net-based CNN model taking multi-scale fields of motion vectors as input to effectively predict QT depth map as well as split decisions at different MT levels.

We also have other contributions such as:

• We build MVF-Inter 1 , a large scale dataset for inter QTMT partition of VVC, which could facilitate the research in this field.

• We propose a fine tuned loss function for this complex multi-branch multi-class classification problem.

• We develop a fast partition scheme effectively exploiting the prediction of a CNN model in a way that the most possible splits are determined at each partition level.

• We design a specific threshold-based selection approach to match with the partition scheme, which allows us to realize a large range of trade-offs between complexity and compression efficiency.

The remainder of this paper is organized as follows. In Section II, we provide an overview of QTMT partitioning scheme in VVC, including the concept of the partition path. The motivation and detailed description for our proposed representation of the QTMT partition are presented in Section III. In Section IV, the structure of the proposed CNN model is illustrated. We give a detailed description of the partitioning acceleration algorithm in Section V. The loss function of CNN and the dataset generation process are described in Section VI. In section VII, the evaluation of the prediction accuracy of our CNN model is carried out. Furthermore, we compare our result with the state-of-the-art of RF-based methods and CNNbased methods, respectively. The complexity analysis of our method is also included in this section. Finally Section VIII concludes the paper. Our source code is available at https: //github.com/Simon123123/MSMVF VVC TIP2023.git.

II. OVERVIEW OF PARTITIONING STRUCTURE IN VVC A. QTMT Partitioning

The partitioning of VVC is done in a top-to-bottom manner. Starting from CTU, the encoder applies possible split types recursively on CU at each level of the partitioning tree, in order to find the partition which best exploits spatial and temporal redundancy. The QTMT partitioning scheme in VVC consists in splitting the CU using either a QT split or a MT split. For the MT split, four split types are introduced: Horizontal Binary Tree (HBT) split, Vertical Binary Tree (VBT) split, Horizontal Ternary Tree (HTT) split and Vertical Ternary Tree (VTT) as illustrated in Figure 1. Based on the QTMT partitioning scheme, dimensions of the final encoded CU could range from 128×128 to 4×4 [START_REF] Huang | Block partitioning structure in the vvc standard[END_REF] including squared and rectangular CUs of 32 different sizes in the RA configuration with the CTU size of 128x128. Figure 2, QTMT can achieve fine partitioning adapted to the local frame texture. A worth-noting characteristic of QTMT partitioning is that the available split types depend on CU sizes. Figure 3 presents the number of available split types including No Split (NS) per CU size for luma samples. The number of split options varies from 1 to 6 for different sizes of CU, making the partition acceleration at the CU level more complicated. Except for the CU size restrictions on available split types, VVC implements various shortcuts [START_REF] Wieckowski | Fast partitioning decision strategies for the upcoming versatile video coding (vvc) standard[END_REF], including speed-up rules based on content gradients and QT search restrictions estimated on neighboring CUs, etc.

The partitioning is executed at CTU level with dual tree allowing separated partitioning tree for luma and chroma. Our algorithm only accelerates the luma partitioning search using luma samples, and the resulting luma partitioning tree is then applied to the chroma components.

B. Partition Path

In this work, we introduce the concept of partition path to depict the partition of a CU. The partition path refers to the sequence of splits applied to obtain a CU during the partitioning. In the RDO process of partitioning, numerous partition paths included in the partition search space are checked and the optimal one leading to the final partition is selected. Figure 4 illustrates a simplified tree representation showing all possible partition paths checked for a CTU. The red arrows indicate the selected partition path with the lowest RD-cost [START_REF] Sullivan | Rate-distortion optimization for video compression[END_REF], while the blue arrows represent other paths that have been tested by RDO, but were not selected.

Specifically, within the QTMT partition, it is important to note that QT splits are prohibited for the child nodes of a MT split. Consequently, the search for optimal partition path in VVC can be conceptualized as a sequential two-step decisionmaking process, comprising a sequence of QT splits followed by a series of MT splits.

III. NOVEL REPRESENTATION OF QTMT PARTITION

Based on the QTMT partitioning structure and the partition path of VVC presented in the previous section, we introduce in this section a novel representation of QTMT partition by partition path. In III-A, we explain the motivation for this new representation. The partition path representation is illustrated in III-B.

A. Motivation

Previous partition representations at CTU level have typically used binary vectors to depict split boundaries. In [START_REF] Galpin | Cnn-based driving of block partitioning for intra slices encoding[END_REF][START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF][START_REF] Tissier | Machine learning based efficient qt-mtt partitioning scheme for vvc intra encoders[END_REF], the authors intend to predict the split boundary of each 4x4 sub-block in CTU. Lately, Wu et al. improve this representation in [START_REF] Wu | Hg-fcn: Hierarchical grid fully convolutional network for fast vvc intra coding[END_REF] by proposing hierarchical boundaries. This adaptation is designed to better align with the QTMT partition pattern. In this work, binary labels for split boundaries of varying lengths are predicted. Collectively, these methods provide a geometric representation of the partition.

The limitations of the geometric representation mainly lie in two aspects. Firstly, it is an implicit representation of the partitioning process, requiring conversions from boundary vectors to split decisions. In the case of [START_REF] Galpin | Cnn-based driving of block partitioning for intra slices encoding[END_REF], conversions are carried out by computing the average probability at the location of the specific split. [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF] and [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning scheme for vvc intra encoders[END_REF] convert boundary vectors to split decisions by DT models separately trained for different CU sizes. Secondly, different partition paths could be deduced from a particular partition presented in a geometric way. For example, as demonstrated in Figure 5, the partition defined by the split boundaries can lead to three distinct partition paths. These partition paths correspond to different coding performances and are individually tested in the RDO process. This multiplicity of partition paths of the geometric representation limits the acceleration potential of the method. To address the above limitations, we introduced a novel representation based on the partition path. Our representation comprises the QT depth map and the MT split maps. Firstly, the split decisions at each depth can be directly deduced from either the QT depth map or the split map. This eliminates the need for decision trees, reducing method overhead, and simplifying implementation. Secondly, it corresponds to a unique partition path, maximizing the potential for complexity reduction.

B. QT Depth Map and MT Split Maps

Considering that the maximum number of QT splits and MT splits is typically set to 4 and 3 in VTM, any partition can be effectively described by a QT depth map (i.e. QTdepthMap) along with three MT split maps (i.e. MTsplitMap) in sequence. Each element within QTdepthMap and MTsplitMap corresponds to an 8x8 and 4x4 area, which aligns with the dimensions of the smallest sub-CUs for the QT split and the MT split in VTM.

A detailed example of our partition representation is shown in Figure 6. To keep it simple and without loss of generality, we represent this example for a CTU size of 64×64. In this figure, (a) shows an instance of QTMT partition with its corresponding tree representation shown in (b). (c)-(f) illustrate the QTdepthMap and MTsplitMaps generated from this partition. Given that the CTU size in this example is 64x64, the sizes of QTdepthMap and MTsplitMap are 8x8 and 16x16, respectively. The QTdepthMap in (c) consists of QT depth values ranging from 0 to 4, while each element in MTsplitMap in (d)-(f) represents the split decision among five options: NS, HBT, VBT, HTT and VTT. This representation depicts a distinct partition path for every CU within the partition. To provide an example, consider the CU highlighted in the green circle in Figure 6. Its partition path can be expressed as three QT splits (QT depth 3), followed by a HBT split and two NS decisions.

IV. CNN-BASED PREDICTION OF PARTITION PATH

Predicting the optimal partition is equivalent to predicting the optimal partition path. In VTM, the size of CTU is set to 128x128 by default, consequently yielding QTdepthMap and MTsplitMap dimensions of 16x16 and 32x32, respectively. The representation of partition path can be predicted by a multi-branch CNN, where one branch infers the QTdepthMap of regression values with dimension 16x16x1, while the other three branches produce the MTsplitMap. Each element of MT-splitMap is classified into one of five classes, corresponding to five split types, resulting in three MT outputs with dimensions of 32x32x5. We have handled the classification of MT splits as an image segmentation problem based on 4x4 sub-blocks. Accordingly, we adopted the classical U-Net structure [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] to design our CNN model to address this segmentation-like task.

In this section, the U-Net structure is briefly introduced. Then we present the structure of the proposed CNN in Section IV-B. Afterwards, we list its input features and explain the reasons for choosing them in Section IV-C.

A. U-Net

The U-Net structure is derived from Fully Convolutional Network (FCN) [START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF]. It consists of an encoder part which is composed of a sequence of convolutional layers plus maxpooling layers. Then this part is followed by a decoder part in which the maxpooling layers are replaced by upsampling layers. In addition, skip connections concatenate feature maps from the encoder and decoder with the same dimension. The U-Net and its variations have been widely applied to image segmentation tasks.

B. MS-MVF-CNN Structure

The CNN structure proposed in this paper, named Multi-Scale Motion Vector Field CNN (MS-MVF-CNN), is depicted in Figure 7. The proposed CNN has 7 inputs and 4 outputs. After two convolutional layers with stride, the tensor of Input 1 is downsampled to dimension 32x32x8 and then concatenated with the Input 2. The merged input is then fed to the U-Net feature extractor demonstrated in Figure 8. Regarding the design of this module, we are referring to the classical structure of U-Net depicted in [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]. Specifically, we concatenate the upsampled feature map in the decoder part of U-Net, the feature map copied from the encoder part with the motion vector field of the same scale. At the decoding part, the feature map is gradually expanded and merged with normalized motion field of 2x2x6, 4x4x6, 8x8x6, 16x16x6 and 32x32x6. As a result, the U-Net feature extractor outputs a feature map of dimension 32x32x8, combining pixels features with motion estimation features.

Since the split at each level depends on previous splits, we employ a hierarchical multi-branch prediction mechanism. QTdepthMap is predicted after shrinking the features extracted from U-Net by four convolutional layers. For MT branches, we designed the MT branch module presented in Figure 8. Two inputs of this module are the extracted features of U-Net and outputs from previous partition levels. We utilize the asymmetric kernel structure to process the extracted features. This structure is originally proposed by [START_REF] Chen | Learned fast hevc intra coding[END_REF] in HEVC to pay attention to near-horizontal and near-vertical textures for predicting split decision of intra coding by CNN. We adopt this structure to exploit the horizontal and vertical information contained in Multi-Scale Motion Vector Field (MS-MVF). The MT branch module contains branches of kernel size MxN, LxL, and NxM. The values of (M, N, L) are set as [START_REF] Alshin | Bi-directional optical flow for improving motion compensation[END_REF][START_REF] Tissier | Complexity reduction opportunities in the future vvc intra encoder[END_REF][START_REF] Cui | Gradient-based early termination of cu partition in vvc intra coding[END_REF] for branch MT0, [START_REF] Wang | Adaptive motion vector resolution scheme for enhanced video coding[END_REF][START_REF] Alshin | Bi-directional optical flow for improving motion compensation[END_REF][START_REF] Tissier | Complexity reduction opportunities in the future vvc intra encoder[END_REF] for branch MT1, and (1, 3, 3) for branch MT2. On deeper MT levels, splits are made on smaller CUs. Thus, smaller kernel sizes are applied to extract finer features. After the asymmetric kernels, the feature map is then concatenated with outputs from previous levels. In the end, the merged feature maps are given to two residual blocks [START_REF] He | Deep residual learning for image recognition[END_REF] before yielding classification results of MT branches. No activation is applied to the fully connected output layer of the QT depth branch. The output layer of the MT branch is with softmax activation.

C. Input Features

This network structure takes three different types of input. The involved inputs are presented below:

1) Original and Residual CTU

In Figure 7, Input 1, with dimensions of 128x128x2, is created by merging the original CTU with the residual CTU. The original luma pixels carry the texture details of the CTU, while the residual CTU is generated through motion compensation of the original CTU based on the nearest frame.

Several studies [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF][START_REF] Yeo | CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction[END_REF][START_REF] Wang | Fast qtbt partitioning decision for interframe coding with convolution neural network[END_REF]] have adopted a method in which both the original CTU values and the residual of CTU are fed to a CNN. Combining the original and residual values as input allows CNN to assess the similarity between current CTU and reference CTU. This combined input offers features that reflect the temporal correlation between frames which is a crucial factor in inter partition prediction.

2) QP and Temporal ID

The Input 2, as illustrated in Figure 7, has dimensions of 32×32×2, consisting of two separate 32x32 matrices. These matrices are assigned specific values: one holds the Quantization Parameter (QP) value, while the other contains the temporal identifier. This temporal identifier in VVC, similar to its usage in HEVC, signifies a picture's position within a hierarchical temporal prediction structure, controlling temporal scalability [START_REF] Wang | The highlevel syntax of the versatile video coding (vvc) standard[END_REF]. We specifically utilize the QP value and temporal identifier as input features since inter partitioning depends on them. In essence, a higher temporal layer identifier or a lower QP value tends to result in finer partitions, as outlined in [START_REF] Yeo | CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction[END_REF]. Instead of developing separate models for each parameter instance, our approach focuses on training a model with adaptability to varying values of QP and temporal identifier.

3) Multi-Scale Motion Vector Field

In this paper, we have introduced a CNN model based on a novel input feature called MS-MVF. Our MS-MVF at five scales is presented as Input 3-7 in Figure 8. To compute MS-MVF, we divide the 128x128 CTU into multiple scale subblocks ranging from 4x4 pixels to 64x64 pixels, and perform motion estimation on these sub-blocks. Each motion vector of sub-block comprises a vertical and horizontal motion value, along with the associated Sum of Absolute Differences (SAD) cost value as the third element. By concatenating elements pointing to reference frame of L0 with those of L1, each subblock corresponds to 6 elements in the motion vector field. For example, the motion vector field input for 8x8-pixel scale has dimensions of 16x16x6.

A significant challenge in inter partition prediction is the large motion search space, which spans up to 6 regions of 384x384 pixels across different reference frames in the RAGOP32 configuration. State-of-the-art methods typically employ motion fields or pixels from reference frames as input features for machine learning models. Notably, in [START_REF] Amestoy | Tunable vvc frame partitioning based on lightweight machine learning[END_REF] and [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF], a crucial feature used is the motion field, which comprises motion vectors calculated for each 4x4 sub-block referring to the nearest frame. As mentioned in [START_REF] Amestoy | Tunable vvc frame partitioning based on lightweight machine learning[END_REF], this motion field is strongly correlated with the optimal partition. In a different approach, Tissier et al. in [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF] opt to utilize two reference CTUs in the nearest frames. The choice of using MS-MVF as the CNN input, instead of motion fields and reference pixels, is based on the following reasons. First, the MS-MVF contains crucial motion information for the current CTU, which is essential for both inter prediction and inter partitioning. This information can be interpreted more effectively by the CNN model compared to using reference pixels as CNN input. Second, the multi-scale nature of MS-MVF aligns well with the multi-level structure of U-Net and can leverage this structure effectively. Essentially, MS-MVF represents motion features at various resolutions, allowing for the combination with features extracted from CTU pixels at the same resolution scale.

To demonstrate the effectiveness of our MS-MVF input, we conducted an experiment involving the training of two CNN models. The only distinction between these models is their input: the first model, PIX-CNN, takes the pixels of two reference CTUs as input, while the second model, MVF-CNN, utilizes our proposed MS-MVF as input. Both models share the same architecture as in Figure 7. The training dataset comprises 250k samples randomly selected from the RAGOP32 encoding of 200 sequences with a resolution of 540p from [START_REF] Ma | Bvi-dvc: A training database for deep video compression[END_REF]. Performance evaluations in Figure 9 are based on Class C sequences of Common Test Condition (CTC). The results consistently show that MVF-CNN outperforms PIX-CNN at all four data points, which justifies the advantages of using the MS-MVF input over pixel input. Based on our evaluation conducted on the first 64 frames of all CTC sequences using the RAGOP32 configuration, the computation of MS-MVF for each CTU consumes, on average, a mere 0.52% of the encoding time in VTM10. Importantly, the generation of MS-MVF introduces only minimal encoding overhead, making it a task that can be readily preprocessed or parallelized.

V. PROPOSED CNN-BASED ACCELERATION METHOD

After the prediction by our trained CNN model, we obtain one QTdepthMap and three MTsplitMaps per CTU. The predicted QTdepthMap is composed of floating-point values. The predicted MTsplitMaps comprise probabilities of five split types for each 4x4 sub-block within the CTU. In this section, we elucidate the post-processing of the CNN prediction, with the aim of achieving a wide range of acceleration-loss tradeoff. end for 14: end if Decision errors at low partitioning depth can result in large loss of BD-rate. Based on the predictions of our CNN, selecting the best single partition path, equivalent to choosing the best split at each MT level, will not be optimal or scalable. Our approach involves generating candidate lists at each level, which means that multiple partition paths are chosen for the RDO test. This approach of creating candidate lists at various levels is designed to achieve satisfying trade-off between acceleration and coding loss while assuring the scalabilty of method.

The acceleration algorithm is precisely described in Algorithm 1 and Figure 10. We introduce two parameters Thm and QTskip to regulate the acceleration-loss trade-off. Specifically, Thm is the threshold for the split probability. QTskip represents whether we should accelerate RDO of QT splits or not. Increasing the Thm value and setting QTskip to true will lead to greater acceleration at the cost of increased coding loss.

Regarding the algorithm applied at the CU level, Algorithm. 1 is first executed. This algorithm produces two outputs: the SkipMT variable and the CandSplit list, both of which are subsequently utilized in the flowchart in Figure 10. To start with, the mean QTdepth pred of current CU is calculated based on the corresponding area in Quad Tree depth map (QTdepthMap). If the rounded QTdepth pred is larger than the QT depth of the CU and QT split is feasible, the current CU should be split by QT. Consequently, all MT splits are excluded from the CandSplit list and SkipMT is set to true. Otherwise, the mean probability of each available split is computed on the corresponding MTsplitMap in a similar way to that of the QTdepth. Then CandSplit is filled by splits with Proba sp larger than the threshold Thm. In this case, the value assigned to SkipMT is False.

In the flowchart of Figure 10, if the SkipMT is true after the execution of Algorithm 1, we directly check the CandSplit. In this scenario, the encoder conducts RDO of CU and splits CU with QT because CandSplit contains only NS and QT. If SkipMT is false, then we will verify if NS is the only choice in CandList. If this is the case, we will add the MT split with the highest probability to the list. Next, if QT split is not allowed for CU due to CU shape or shortcuts, we go directly to the check of CandSplit. If the QT split is feasible, we refer to QTskip to determine whether to add QT to the CandList or not. Setting the QTskip to true signifies that we will always check QT if possible. This is for rectifying the potential error of predicting a QTdepth pred value smaller than the actual ground truth value. However, it comes at the expense of sacrificing some acceleration. Finally, we execute RDO on CU and partition it by split types in the CandSplit list. The partition search then repeats for the next CU, and the algorithm described above is applied anew.

Our inter partitioning acceleration method is designed on top of the partitioning algorithm of VTM which performs a nearly exhaustive search on possible partition paths of a CTU, except that it incorporates a handful of handcrafted conditional shortcuts as mentioned in Section II. Therefore, this work can be considered as a CNN-based shortcut to reduce the search space of partition paths.

VI. TRAINING OF MS-MVF-CNN

To effectively train our CNN model, we have designed a hybrid loss function and created a large-scale dataset named MVF-Inter 1 . First, we will explain how this loss function is determined in Section VI-A. Then Section VI-B describes training details and the generation of dataset.

A. Loss Function

The outputs of MS-MVF-CNN contain one regression output as well as three classification outputs. Therefore, a hybrid loss function is developed in our case. We choose the category (1) Here, we have n q = 256, n b = 3, n m = 1024 and n s = 5, representing the number of elements in QTdepthMap, the number of MT branches, the number of elements in MTsplitMap, and the number of split types, respectively. In this equation, d i denotes the ground-truth QT depth value, while di represents the predicted QT depth value. Additionally, ŷb,i,s is used to denote the predicted probability of split type s for the i-th element of the MT decision map at the b-th MT branch. Similarly, y b,i,s signifies the ground-truth label for the same case. Notably, we introduce a parameter a, which falls within the range [0, 1], in Equation 1 to fine-tune the relative weights of the regression loss and classification loss.

The split types are distributed unbalancedly at different MT depths as illustrated in Figure 11. To counteract this imbalance, we introduce class weights for split type s on MT branch b, denoted as w b,s . The definition of these weights is as follows:

w b,s = λ s p b,s=ns p b,s (2)
where p b,s represents the percentage of split type s within MT branch b. For each branch b, p b,s=ns p b,s can be interpreted as the inverse percentage of the split type s normalized by the inverse percentage of the NS split. In [START_REF] Huang | Block partitioning structure in the vvc standard[END_REF], a series of tests were performed to evaluate the coding gain and increase of complexity associated with the Binary Tree (BT) and Ternarytype Tree (TT) splits individually as demonstrated in Table I. When comparing Setting 1 and Setting 2 to the anchor configuration, it's observed that Setting 1 and Setting 2 exhibit similar BD-rate gains, but the encoding time in Setting 2 is twice that of Setting 1. These tests suggest that BT split performs better in terms of the trade-off between complexity and coding gain compared to TT split. Thus, placing greater importance on the prediction of the BT split can result in a better acceleration-loss trade-off. To achieve this, the ratio between the proportion of NS and proportion of split s is computed for MT branch b. The class weight w b,s in Equation 2is formulated as the product of this ratio and λ s which is another weight added to prioritize the split type s.

After fine-tuning the model, we find that the best performance is achieved with a value of 0.8 for a and λ s set to 2 for BT splits and 1 for TT splits and NS.

B. Dataset Generation and Training Details

Constructing a large scale inter partition dataset is more challenging than that of intra partition because the former needs to encode a substantial number of video sequences, while the latter could be done by encoding images. To the best of our knowledge, there exists no prior work focused on developing an inter partition dataset.

Our MVF-Inter 1 dataset involved the encoding of 800 sequences from [START_REF] Ma | Bvi-dvc: A training database for deep video compression[END_REF] and an additional 28 sequences of 600 frames in 720p resolution extracted from [START_REF] Wang | Youtube ugc dataset for video compression research[END_REF]. Sequences of [START_REF] Ma | Bvi-dvc: A training database for deep video compression[END_REF] cover resolutions of 240p, 540p, 1080p, and 4k, with 200 videos of 64 frames for each resolution. We have encoded all these videos with the VTM10 [START_REF] Chen | Algorithm description for versatile video coding and test model 10 (vtm 10)[END_REF] encoder in the RAGOP32 configuration with QP 22, 27, 32, and 37. We randomly selected a total of 820k CTU partition samples, equally distributed per resolution and QP, with 120k samples reserved as a validation set.

Each sample of our dataset contains the following components of each CTU: pixel values, residual values, motion vector fields at five scales, QP value, temporal ID value, QTdepthMap with depths ranging from 0 to 4, and MTsplitMaps for MT0, MT1, and MT2. MTsplitMap labels are encoded as VTT (0), VBT (1), NS (2), HBT (3), and HTT [START_REF] Li | An efficient four-parameter affine motion model for video coding[END_REF].

In terms of training details, we employed the Adam optimizer [START_REF] Adam ; Author=kingma | A method for stochastic optimization[END_REF] to train the model. The initial learning rate was set to 1e -3 and was exponentially decreased by 3% every 5 epochs. The batch size set for training is 400.

VII. EXPERIMENTAL RESULTS AND ANALYSES

In this section, we present the results of our experiments and provide an in-depth analysis of the results. To begin, in Section VII-A, we assess the precision of the prediction of our CNN model. Subsequently, comparisons with the RF and CNN based approaches are made in Section VII-B. Finally, the complexity analysis of our framework is carried out in Section VII-C.

A. Prediction Accuracy Evaluation

At the CU level, our algorithm can be broken down into two decisions: the decision of SkipMT and the decision of CandSplit list. To evaluate the precision of decisions based on our model's output, we have performed the encoding where both the ground truth partitioning and the CNN output were collected. The analysis is done on the first 64 frames of all CTC sequences excluding class D with QP 22, 27, 32, 37. The accuracy of these decisions presented in Table II and Figure 12 are calculated by averaging four QPs and various test sequences.

There is no need to make a SkipMT decision on QT depth 4 since the partitioning is forced to proceed to MT splits with the maximum of QT depth reached. The accuracy of SkipMT decision is independently measured on QT depth from 0 to 3. If the current CU requires further splitting of QT and SkipMT is equal to False, then this decision of SkipMT is classified as False Negative (FN). The proportion of True Positives (TP), FN, True Negatives (TN), False Positives (FP) and their corresponding Precision (Prec) and Recall (Rec) are shown in Table II. Precision, recall, and F1 score are calculated as follows:

P recision QTdepth = T P QTdepth T P QTdepth + F P QTdepth ( 3
)
Recall QTdepth = T P QTdepth T P QTdepth + F N QTdepth (4)

F 1score QTdepth = 2 P recision QTdepth Recall QTdepth P recision QTdepth + Recall QTdepth (5)
Generally, our model exhibits strong performance at QT depths ranging from 0 to 2, as depicted in Table II. Both precision and F1 score decrease as QT depth increases. At QT depth 3, the precision and F1 score drop to 25% and 40%, respectively, suggesting that the SkipMT decision at this level is less reliable. These observations could be explained by two reasons:

First of all, the scale of decision-making diminishes as the QT depth increases. More explicitly, the SkipMT decision at QT depth 0 is made at the CTU scale by computing the mean of 256 values from the QTdepthMap. Nevertheless, the decision at QT depth 3 relies only on 4 values from the QTdepthMap within the 16x16 CU. Consequently, decisions at smaller scales are less resilient to incorrectly predicted QTdepthMap values, resulting in lower overall accuracy at higher QT depths.

Secondly, decisions at higher QT depths are noticeably more imbalanced than those at lower QT depths. Positive cases of ground truth at QT depth 3 represent only 0.02% , while the proportion of positive cases is 49.65% at QT depth 0. In conclusion, the model is trained in such a way that it tends to make negative SkipMT decision at larger QT depths. This explains the decline in precision as the QT depth increases. In Figure 12, the accuracy of the CandSplit list decision is determined by whether the list contains the ground truth split at the MT level. We calculate and draw separate accuracy curves for MT0, MT1 and MT2 separately by varying the threshold Thm. As Thm increases, the size of the CandSplit list decreases, leading to decreasing precision. Once Thm reaches a certain value, the accuracy stabilizes because CandList is constant, containing only the MT split type with the highest probability and NS. It's worth noting that the minimum accuracy of the MT increases with the MT depth. This is mainly due to the fact that NS is more frequent at larger MT depths, as illustrated in the pie chart in Figure 11. Since our CandSplit list consistently includes NS, the accuracy tends to be relatively higher at larger MT depths.

In general, our model achieves a satisfactory F1 score for QT depths 0, 1 and 2 regarding the SkipMT decision. As for the CandSplit list decision, our algorithm maintains an accuracy exceeding 65% while adjusting the value of Thm at various MT levels. These performance evaluations justifies the high accuracy of the decisions made by our method during the partition search process in VVC. The proposed method has been implemented in the VTM10.0 encoder using the Frugally deep library [START_REF] Hermann | Frugally-Deep[END_REF] for CPU-based inference in real time. To showcase the effectiveness of our method in the latest version of VTM, we conducted experiments using VTM21, as represented by the black curve in Figure 13. Encodings of CTC sequences are performed on a Linux machine with Intel Xeon E5-2697 v4 in a single-threaded manner. These experiments were conducted on the first 64 frames of CTC sequences with the RAGOP32 configuration on four QPs values of 22, 27, 32, 37. Two metrics were used to assess the performance: BD-rate [START_REF] Bjontegaard | Calculation of average PSNR differences between rdcurves[END_REF] and Time Saving (TS). The formula for computing TS is provided in Equation 6. Here, T Test denotes the encoding time of the proposed method, while T VTM represents the encoding time of the original VTM10 under the same conditions. The average BD-rate loss and Time Saving (TS) are computed as the arithmetic mean and geometric mean, respectively, on four QPs values over CTC sequences as defined in [START_REF] Boyce | Jvet common test conditions and software reference configurations[END_REF]. In addition, sequences of class D are excluded when computing the overall average performance.

T S = 1 4 q∈{22,27,32,37} T V T M (q) -T Test (q) T V T M (q) (6)
The acceleration performances obtained from the state-ofthe-art RF-based methods could not be directly compared with our performance. There are two main reasons for this. First of all, the results of [START_REF] Amestoy | Tunable vvc frame partitioning based on lightweight machine learning[END_REF] and [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF] are based on VTM5.0 and VTM8.0, respectively. The differences of encoder complexity among various VTM versions are not negligible as highlighted in [START_REF] Sullivan | Versatile Video Coding (VVC) Delivers: Coding Efficiency and Beyond[END_REF], which makes it less valid to directly compare our performances with theirs. Secondly, the training dataset was generated from a subset of CTC sequences, and the results were not obtained from the entire CTC. This approach results in possible overfitting and reduces the credibility of their results. As a result, comparing our results obtained on the entire CTC with their results is not fair.

[20] is an extended and specialized work for VVC based on [START_REF] Amestoy | Tunable vvc frame partitioning based on lightweight machine learning[END_REF]. We have reproduced the result of [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF] in VTM10 to perform an unbiased comparison between our method and RF-based method in [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF]. First of all, we created a non-CTC dataset for training. Table III presents details on the composition of sequences for the dataset. For the 720p resolution, sequences are selected from [START_REF] Wang | Youtube ugc dataset for video compression research[END_REF] and sequences for other resolutions are from [START_REF] Ma | Bvi-dvc: A training database for deep video compression[END_REF]. In the end, we generated a large dataset with 3.7e 7 samples for the training of 17 Hor/Ver classifiers as well as 2.5e 6 samples for the training of 4 QT/MTT classifiers. After generating the dataset, we trained, pruned and integrated the RF classifiers in VTM10.0. This was done in a manner consistent with the original article, including the implementation of the early termination rule for TT 2 . We reproduce the result of the medium and fast speed preset of [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF] in VTM10. It should be noted that the maximum MT depth is limited to 2 for the fast preset. We plot the curve of BD-rate loss and TS of our method by gradually adjusting the threshold Thm and QTskip to build six settings. The curves obtained are shown in Figure 13. For example, the label (T, 0.125) signifies that in this particular setting, QTskip and Thm are assigned the values True and 0.125, respectively. Our method can achieve scalable acceleration varying from 16.5% to 60.2% with BD-rate loss ranging from 0.44% to 4.59%. Comparing with the fast preset, the setting (T, 0.175) produces the same acceleration with a 0.84% lower BD-rate loss. Similarly, the setting (T, 0) reaches the same BD-rate loss while providing a 17% higher speed-up compared to the medium preset. In summary, our method generally outperforms the state-of-the-art RF-based method. It is worth mentioning that the results in VTM21 are obtained by implementing our CNN model, which was originally trained on VTM10. Consequently, it is expected to exhibit reduced performance compared to the results in VTM10. Nonetheless, our method remains applicable and effective in the latest version of VTM.

Regarding CNN-based approaches, we compared our method with [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF] and [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF] in Table IV. The VTM version of [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF] is VTM6. Thus we reimplement our method and integrate our model trained on VTM10 into VTM6 for a fair comparison within the same context. In Table IV, the reimplementation in VTM6 labeled as (T, 0, VTM6) reaches a slightly larger acceleration with only one-third of BD-rate loss compared to [START_REF] Pan | A cnn-based fast inter coding method for vvc[END_REF]. For [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF], their VTM version is the same as ours, allowing for direct comparisons. Encoding with Thm = 0.125 yields a 40. 6% reduction in the encoding time, which is similar to the acceleration achieved by the C2 configuration in [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF], but with only half of its BD-rate loss. Furthermore, our method with Thm set to 0.2 outperformed their C3 configuration, achieving a 0.52% lower BD-rate loss at the same level of acceleration. In conclusion, our method consistently outperforms all stateof-the-art methods. Ours (VTM10) Ours (VTM21) Reproduction of [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF] (VTM10) Fig. 13: Comparison of performances between proposed method and reproduction of [START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF].

It is important to note that the level of acceleration can vary depending on different sequence classes (e.g. resolution), which is consistent with other CNN-based methods. As discussed in [START_REF] Huang | Block partitioning structure in the vvc standard[END_REF], CTUs that exceed the picture boundary are called partial CTUs. These partial CTUs require a different partition search scheme compared to regular CTUs. Consequently, the encoding of partial CTUs are not accelerated since the CNNbased approaches are not applicable to them. Generally, the proportion of the frame region occupied by partial CTUs is larger for lower resolutions, resulting in less acceleration when fast partitioning approaches are used on smaller resolutions. This could partially explain the limited acceleration observed in class D which was excluded from the overall performance calculation. More specifically, our method tends to perform better on higher resolutions (e.g. class A and class B) while achieving less acceleration than state-of-the-art methods on lower resolutions (e.g. class C, class D and class E). Investigating and improving this aspect could be a focus of future work.

C. Complexity Analysis

Machine learning-based fast partitioning methods may not be suitable for alternative implementations of the same codec. For example, VVenc [START_REF] Wieckowski | Vvenc: An open and optimized vvc encoder implementation[END_REF] is a fast implementation of VVC. In the All Intra configuration, VTM10.0 is reported to be 27 times more complex compared to VVenc with fast preset, as mentioned in [START_REF] Taabane | Machine learning based fast qtmtt partitioning strategy for vvenc encoder in intra coding[END_REF]. The overall complexity of the CNNbased method presented in [START_REF] Feng | Partition map prediction for fast block partitioning in vvc intra-frame coding[END_REF] accounts for only 2.34% of the encoding time of the VTM10 encoder. However, when this method is implemented in VVenc without any adjustments, its overhead increases to about 67% of the encoding time with the fast preset, which means that this method is not directly applicable to VVenc. Consequently, it is crucial to develop a lightweight method to ensure its applicability across different implementations. Furthermore, lightweight methods do not As a result, we conducted a complexity analysis of our method to compare it with the state of the art. The overhead of a machine learning-based method typically consists of three components: preprocessing time, inference time, and postprocessing time. The post-processing of our method is integrated into the VVC partitioning process and introduces minimal overhead to the encoding process. However, preprocessing is necessary to compute the MS-MVF as model input. Table V provides the complexity of the preprocessing and the inference of CNN related to the encoding of the anchor VTM10. The last column corresponds to the geometric average of complexity for sequences from class A to E (including class D). Based on experimental results, the CNN inference time on a CPU accounts for only 0.60% of the total encoding time. Our approach consumes only 1.21% of the total encoding time, underscoring its lightweight nature.

Another important metric for evaluating the complexity of the model is its floating point operations (FLOPs). Our model has a FLOPs value of 1.12e 6 . In comparison, the FLOPs of the model in [START_REF] Li | Deepqtmt: A deep learning approach for fast qtmt-based cu partition of intra-mode vvc[END_REF] is approximately 1.1e 9 [START_REF] Wu | Hg-fcn: Hierarchical grid fully convolutional network for fast vvc intra coding[END_REF]. [START_REF] Wu | Hg-fcn: Hierarchical grid fully convolutional network for fast vvc intra coding[END_REF] employs a pruned ResNet-18 as the backbone with 9e 7 FLOPs, and [START_REF] Tissier | Machine learning based efficient qt-mtt partitioning for vvc inter coding[END_REF] utilizes the pretrained MobileNetV2 with 3.14e 8 FLOPs. Our model is hundreds of times lighter than these methods.

The lightweight nature of our proposed approach facilitates its adaptation to faster encoders.

VIII. CONCLUSION

In this study, we propose a machine learning-based method to accelerate VVC inter partitioning. Our method leverages a novel representation of the QTMT partition structure based on partition path, consisting of QTdepthMap and MTsplitMaps. Our work is structured as follows. Firstly, we have built a large scale inter partition dataset. Secondly, a novel Unetbased model that takes MS-MVF as input is trained to predict the partition paths of CTU. Thirdly, we develop a scalable acceleration algorithm based on thresholds to utilize the output of the model. Finally, we speed up the VTM10 encoder under RAGOP32 configuration by 16.5%∼60.2% with BD-rate loss of 0.44%∼4.59%. This performance surpasses state-of-theart methods in terms of coding efficiency and complexity trade-off. Notably, our method is among the most lightweight methods in the field, making it possible to adapt our approach to faster codecs.

For future work, we intend to investigate how video resolution influences partitioning acceleration, aiming to boost the speed-up of our method on lower resolutions. Furthermore, there is still acceleration potential lying in the selection of inter coding modes at the CU level, as discussed in [START_REF] Liu | Statistical analysis of inter coding in vvc test model (vtm)[END_REF]. An extension of our approach could be the incorporation of fast inter coding mode selection algorithm into our method to further accelerate the inter coding process.

Fig. 1 :

 1 Fig. 1: Different split types in VVC.

Fig. 2 :

 2 Fig.2: Example of a partitioned frame in VVC.[START_REF] Fan | A fast qtmt partition decision strategy for vvc intra prediction[END_REF]

Fig. 3 :

 3 Fig. 3: Number of split types per CU in VVC for Luma.

Fig. 4 :

 4 Fig. 4: Example of partition paths.

Fig. 5 :

 5 Fig. 5: Possible partition paths for a final partition given by split boundaries

Fig. 6 :

 6 Fig. 6: Example of QTMT partition, tree representation, QTdepthMap and MTsplitMaps of CTU size of 64x64

Fig. 7 :

 7 Fig. 7: Multi-Scale Motion Vector Field CNN. The vector above Resblock and Conv2D represents (kernel size, number of filters, stride).

Fig. 8 :

 8 Fig. 8: U-Net feature extractor and MT branch module

Fig. 9 :

 9 Fig. 9: Comparison of performances between PIX-CNN and MVF-CNN.

Fig. 10 :

 10 Fig. 10: Flowchart of acceleration algorithm

Fig. 11 :

 11 Fig. 11: Distribution of split types for MT0, MT1, MT2

Fig. 12 :

 12 Fig. 12: Curves of accuracy and Thm for MT0, MT1, MT2

  Compute the average QTdepth pred based on Size CU , Pos CU and QTdepthMap 2: if round(QTdepth pred ) > QTdepth cur and QT is possible for current CU then

	Algorithm 1 MT splits early skipping
	Input:
	QTdepthMap; MTsplitMap; Thm; QTdepth cur ,
	CU; Size CU ; Pos CU
	Output:
	SkipMT: Boolean to decide whether to skip MT split
	types or not.
	CandSplit: Candidate list of splits for RDO check
	1: 3:	SkipMT = True
	4:	CandSplit = {NS, QT}
	5: else
	6:	SkipMT = False
	7:	CandSplit = {NS}
	8:	for sp in {BTH, BTV, TTH, TTV} do
	9:	Compute average Proba sp based on Size CU , Pos CU
	and MTsplitMap
	10:	if Proba sp > Thm then
	11:	CandSplit append split sp
	12:	end if
	13:

TABLE I :

 I Settings of split type in VTM9 under RA[START_REF] Huang | Block partitioning structure in the vvc standard[END_REF]

		BT split	TT split	BD-rate	Encoding Time
	Anchor Setting	X	X	-	-
	Setting 1		X	-8.26%	337%
	Setting 2	X		-10.22%	732%

TABLE II :

 II Table of confusion for SkipMt (Unit: %)

		TP	FN	TN	FP	Prec	Rec	F1score
	QT depth 0	41.84	7.81	45.83	4.52	90.3	84.3	87.2
	QT depth 1	19.53	0.58	72.57	7.32	72.7	97.1	83.1
	QT depth 2	2.69	0.08	94.67	2.57	51.1	97.1	67.0
	QT depth 3	0.02	0	99.92	0.06	25.0	100.0	40.0

TABLE III :

 III Breakdown of sequences used to train RFs of[START_REF] Kulupana | Fast versatile video coding using specialised decision trees[END_REF]

				Resolution
	Number of	240p	480p	720p	1080p	4k
	videos	50	13	10	10	5

TABLE IV :

 IV Performance of the proposed method in comparison with reference CNN-based methods (Unit: %)

	Class	Sequence	Pan [21] (VTM6) BD-rate TS	Tissier [15] (C2) BD-rate TS	Tissier [15] (C3) BD-rate TS	Ours (T, 0, VTM6) BD-rate TS	Ours (T, 0.125, VTM10) BD-rate TS	Ours (T, 0.2, VTM10) BD-rate TS
		Tango2	4.03	38.56	-	-	-	-	1.35	32.3	1.84	43.7	3.08	57.5
		FoodMarket4	1.74	46.12	-	-	-	-	0.75	29.4	0.85	55.1	1.13	53.6
	A (4k)	Campfire CatRobot1 DaylightRoad2	3.17 6.45 5.63	38.23 36.84 35.47	---	---	---	---	1.49 1.31 1.57	40.7 36.5 39.4	1.83 1.45 2.00	48.5 42.6 45.6	3.22 2.67 3.94	63.2 57.6 57.9
		ParkRunning3	2.10	26.45	-	-	-	-	0.99	42.6	0.98	45.9	1.93	59.8
		Average	3.85	36.46	1.84	47.7	3.06	59.7	1.25	37.0	1.49	45.3	2.66	58.4
		MarketPlace	4.33	33.64	-	-	-	-	0.99	37.6	1.48	46.3	2.78	57.7
	B (1080p)	RitualDance Cactus BasketballDrive BQTerrace	3.55 5.72 3.30 1.90	34.17 29.36 37.28 20.21	----	----	----	----	1.75 1.05 1.34 0.99	39.9 37.8 39.6 32.6	1.91 1.30 1.95 1.18	49.4 44.8 49.7 39.8	3.91 2.45 3.65 2.23	61.8 58.3 63.4 52.2
		Average	3.76	30.27	2.21	46.5	3.09	58.2	1.22	37.5	1.56	46.1	3.00	58.9
		BasketballDrill	2.29	29.23	-	-	-	-	1.04	26.9	1.08	30.3	2.60	39.7
	C (480p)	BQMall PartyScene RaceHorses Average	2.69 2.22 3.02 2.56	27.48 20.80 26.39 25.77	---3.20	---43.1	---3.79	---53.8	1.20 0.78 0.96 0.99	29.1 31.5 32.8 30.1	1.18 0.86 1.09 1.05	32.2 33.3 34.6 32.6	2.71 2.25 2.94 2.63	39.7 43.3 45.6 42.1
		BasketballPass	1.85	26.97	-	-	-	-	0.76	19.0	0.85	22.2	1.72	25.1
	D (240p)	BQSquare BlowingBubbles RaceHorses Average	1.61 3.03 2.92 2.35	14.86 22.15 24.20 21.53	---3.02	---36.8	---3.26	---45.2	0.50 0.33 1.04 0.66	17.6 17.2 22.8 19.2	0.54 0.45 0.85 0.67	19.6 18.9 24.4 21.0	1.38 1.12 2.11 1.58	22.5 23.4 31.2 25.6
	E (720p)	FourPeople Johnny KristenAndSara Average	2.31 3.53 2.58 2.81	33.77 35.22 36.50 35.15	---1.45	---38.7	---2.2	---49.6	0.95 0.93 1.00 0.96	29.5 22.1 24.3 25.4	0.90 1.13 1.11 1.04	34.3 27.6 30.3 30.8	1.65 2.01 1.73 1.79	41.2 32.8 36.4 36.9
		Total average	3.18	30.63	2.33	43.4	3.12	54.3	1.14	33.8	1.34	40.6	2.60	52.2
	require parallel execution, enhancing the cost-effectiveness of
	such solutions.

TABLE V :

 V Overhead of our method (Unit: %)

		240p	480p	720p	1080p	4k	Average
	CNN	0.23	0.37	0.99	0.90	0.84	0.60
	Preprocess	0.24	0.41	1.15	0.81	0.86	0.62

The code and dataset of reproduction is available at https://github.com/ Simon123123/vtm10 fast dt inter partition pcs2021.git

dataset MVF-Inter is available at https://1drv.ms/