Angélica Atehortúa

Eduardo Romero
email: edromero@unal.edu.co

Mireille Garreau

Characterization of motion patterns by a spatio-temporal saliency descriptor in cardiac cine MRI

Keywords: Multi-scale representation . . . Different scales . . Saliency map Saliency Heart motion, cardiac cine magnetic resonance imaging, spa-tio-temporal saliency, cardiac motion patterns

diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

A B S T R A C T

Background and objective: Abnormalities of the heart motion reveal the presence of a disease. However, a quantitative interpretation of the motion is still a challenge due to the complex dynamics of the heart. This work proposes a quantitative characterization of regional cardiac motion patterns in cine magnetic resonance imaging (MRI) by a novel spatio-temporal saliency descriptor. Method: The strategy starts by dividing the cardiac sequence into a progression of scales which are in due turn mapped to a feature space of regional orientation changes, mimicking the multi-resolution decomposition of oriented primitive changes of visual systems. These changes are estimated as the difference between a particular time and the rest of the sequence. This decomposition is then temporarily and regionally integrated for a particular orientation and then for the set of different orientations. A final spatio-temporal 4D saliency map is obtained as the summation of the previously integrated information for the available scales. The saliency dispersion of this map was computed in standard cardiac locations as a measure of the regional motion pattern and was applied to discriminate control and hypertrophic cardiomyopathy (HCM) subjects during the diastolic phase. Results: Salient motion patterns were estimated from an experimental set, which consisted of 3D sequences acquired by MRI from 108 subjects (33 control, 35 HCM, 20 dilated cardiomyopathy (DCM), and 20 myocardial infarction (MINF) from heterogeneous datasets). HCM and control subjects were classified by an SVM that learned the salient motion patterns estimated from the presented strategy, by achieving a 94% AUC. In addition, statistical differences (test t-student, p < 0.05) were found among groups of disease in the septal and anterior ventricular segments at both the ED and ES, with salient motion characteristics aligned with existing knowledge on the diseases. Conclusions: Regional wall motion abnormality in the apical, anterior, basal, and inferior segments was associated with the saliency dispersion in HCM, DCM, and MINF compared to healthy controls during the systolic and diastolic phases. This saliency analysis may be used to detect subtle changes in heart function.

Introduction

Clinical context and motivation

Characterization of heart diseases is not an easy task because of the several compensatory mechanisms that may hinder the effect exclusively caused by the disease [START_REF] Mann | Mechanisms and models in heart failure: the biomechanical model and beyond[END_REF]). Pathologies like hypertrophic cardiomyopathy (HCM) are asymptomatic and present an impaired regional myocardial function at the diastole and systole, with preserved or slightly increased ejection fraction [START_REF] Marian | Hypertrophic cardiomyopathy: genetics, pathogenesis, clinical manifestations, diagnosis, and therapy[END_REF]; [START_REF] Weissler-Snir | Prevention of sudden death in hypertrophic cardiomyopathy: bridging the gaps in knowledge[END_REF]). This scenario of few or no changes at all complicates treatment personalization, above all if one considers how the cardiac structure is transforming to maintain the hemodynamic physiology [START_REF] Elliott | ESC Guidelines on diagnosis and management of hypertrophic cardiomyopathy: The Task Force for the Diagnosis and Management of Hypertrophic Cardiomyopathy of the European Society of Cardiology (ESC)[END_REF]; [START_REF] Weissler-Snir | Prevention of sudden death in hypertrophic cardiomyopathy: bridging the gaps in knowledge[END_REF]). In this context, new cardiac characterization strategies are required to understand and individualize disease adaptation mechanisms [START_REF] Editorial | Taking personalized medicine to heart[END_REF]). This disease knowledge may facilitate an earlier HCM detection, being useful to reduce the risk of sudden cardiac death in about 25% of HCM patients [START_REF] Marian | Hypertrophic cardiomyopathy: genetics, pathogenesis, clinical manifestations, diagnosis, and therapy[END_REF]; [START_REF] Weissler-Snir | Prevention of sudden death in hypertrophic cardiomyopathy: bridging the gaps in knowledge[END_REF]), as well as, to improve the poor prognosis in other complex and asymptomatic diseases.

By directly analyzing the temporal dynamics of the heart, subtle and compensatory changes during diastolic and/or systolic functions may be detected [START_REF] Mazein | Systems medicine disease maps: communitydriven comprehensive representation of disease mechanisms[END_REF]; [START_REF] Buckberg | Cardiac mechanics revisited[END_REF]; [START_REF] Omar | Left ventricular twist and torsion[END_REF]). They may reveal abnormal movement patterns which can be associated with actual disease mechanisms. In this context, quantitative temporal measurements can follow the cardiac function, despite the variable acquisition protocols, pathology conditions, and complex physiology. However, these challenging scenarios require analysis methods with high resilience to different types of noise for extracting relevant information.

Technical context

Recent machine learning strategies have accurately predicted the patient's risk of heart failure by tracking the motion in cine MRI [START_REF] Bello | Deep-learning cardiac motion analysis for human survival prediction[END_REF]; [START_REF] Krittanawong | Deep learning for cardiovascular medicine: a practical primer[END_REF]; [START_REF] Leiner | Machine learning in cardiovascular magnetic resonance: basic concepts and applications[END_REF][START_REF] Sarmiento | Cardiac disease prediction from spatio-temporal motion patterns in cine-mri[END_REF]). Deep learning models have been used to predict outcomes from dense phenotypic data in heart time-resolved segmentation [START_REF] Bello | Deep-learning cardiac motion analysis for human survival prediction[END_REF]). In these works, the motion has been tracked using a B-spline image registration between consecutive 3D heart segmentations to estimate a motion field, which is represented by the displacement vector at each voxel and at each instant of the cardiac cycle. Despite these results, deep learning strategies are dependent on the dataset size, the image quality, the available annotations, and the standardization. Unlike these approaches, the strategy herein presented, based on the human visual attention model, extracts relevant information in complex scenarios with less dependence on the number or quality of data, as demonstrated by proving the robustness of this method to different types of noise, namely Rician, speckle and Gaussian and different configurations of the Gaussian noise. Likewise, the method performed well when classifying different databases, showing relative independence of the number of cases to train.

Visual attention models have been used to extract meaningful image content in very noisy conditions. Current computational models of saliency, inspired by the visual system, select information by applying a bottom-up scheme, i.e, a location is highlighted if it is different from its surroundings and similar to other regions in the image. These saliency models have been broadly applied to medical images in problems related with anomaly detection [START_REF] Zhao | Intensity and compactness enabled saliency estimation for leakage detection in diabetic and malarial retinopathy[END_REF]; Dubost et al. ( 2019)), region of interest detection [START_REF] Iakovidis | Detecting and locating gastrointestinal anomalies using deep learning and iterative cluster unification[END_REF]; [START_REF] Huang | Vp-nets: Efficient automatic localization of key brain structures in 3 D fetal neurosonography[END_REF]), registration [START_REF] Nguyen | Multiple kernel point set registration[END_REF]) and segmentation [START_REF] Mahapatra | Visual saliency-based active learning for prostate magnetic resonance imaging segmentation[END_REF]). Lately, deep learning strategies have been introduced to highlight salient regions in the brain, breast, and dermoscopy, images [START_REF] Zhang | Attention residual learning for skin lesion classification[END_REF][START_REF] Zhang | Hierarchical convolutional neural networks for segmentation of breast tumors in mri with application to radiogenomics[END_REF]; [START_REF] Ding | A deep learning model to predict a diagnosis of alzheimer disease by using 18f-fdg pet of the brain[END_REF]) and in the heart to detect artifacts of capture [START_REF] Oksuz | Automatic cnn-based detection of cardiac MR motion artefacts using k-space data augmentation and curriculum learning[END_REF]). All these approaches exclusively lie in spatial information. The temporal component of the cardiac function is a crucial feature not only to find out hidden characteristic movement patterns but to interpret the disease mechanisms.

Proposed approach

In this paper, a new multi-scale region-based spatiotemporal saliency map descriptor is presented, named as STSalmap (Spatio-Temporal SALiency MAP). This descriptor is inspired by the visual attention system, aiming to extract relevant temporal information in any dynamic scene. The main contributions of this work are summarized as follows:

• The formulation of a novel spatio-temporal saliency description set upon a set of simple techniques which requires little parameter tuning and data for training, i.e., multi-resolution analysis by oriented Gaussian kernels that integrates temporal information and 3D multiscale representations, making this model more robust to capturing noise, variability of the vendor and the specific pathology.

• A spatio-temporal saliency framework devised to characterize subtle changes of complex cardiac pathologies, capable of distinguishing between healthy and HCM subjects, with comparatively lower false-positive rates than other tested methods.

• An interpretable indicator that provides summarized information of the cardiac function that can complement the traditional cardiac indexes like the ejection fraction, which has been reported to show normal values in some cases of heart failure or hypertrophic cardiomyopathy cases.

• A model evaluated against several datasets, specifically four different collections of cases, demonstrating the generalizability of the proposed method.

This paper is organized as follows: a background on spatiotemporal saliency methods is presented in Section 2. Section 3 introduces the motion perception model, which has inspired the proposed spatio-temporal saliency descriptor. Section 4 introduces the spatio-temporal descriptor. Experiments and results are provided in Section 5. Finally, discussion and conclusion about the proposed strategy are presented in Sections 6 and 7 respectively.

Related spatio-temporal saliency methods

This section presents different spatio-temporal saliency methods in video applications. Few methods have estimated saliency in video sequences, organized into three groups: i) approaches using optical flow to extract motion information, ii) methods based on learning a saliency model, and iii) other strategies.

i) Optical flow approaches. The resulting flow, combined with low-level features such as edges, intensity and color, serves to regionally perform intra-frame differences. These approaches, performed by regions known as superpixels, obtain a saliency as a probability map of the intra-frame difference, modulated by background information [START_REF] Wang | Saliency-aware video object segmentation[END_REF]; [START_REF] Chen | Scom: Spatiotemporal constrained optimization for salient object detection[END_REF]; [START_REF] Liu | Superpixel-based spatiotemporal saliency detection[END_REF][START_REF] Liu | Saliency detection for unconstrained videos using superpixel-level graph and spatiotemporal propagation[END_REF]; [START_REF] Wang | Consistent video saliency using local gradient flow optimization and global refinement[END_REF]). The main limitation of these strategies is their dependence on the regularization equation that governs the amount of allowed motion to track and when an object location changes abruptly, optical flow algorithms would generally not perform so well.

ii) Deep learning to model saliency. [START_REF] Tang | Multi-scale spatiotemporal conv-lstm network for video saliency detection[END_REF] trained a multi-scale spatio-temporal convolutional network to predict saliency by mixing coarse labels with fine labels of saliency information. [START_REF] Bak | Spatio-temporal saliency networks for dynamic saliency prediction[END_REF] built a spatio-temporal net by learning spatial correlations between adjacent frames, while [START_REF] Jiang | Deepvs: A deep learning based video saliency prediction approach[END_REF] used a convolutional neural network to predict intra-frame saliency, using a kind of objectness estimation and motion subnets. A great limitation with these strategies is that they require lots of manually annotated data, a real problem when dealing with cardiac MRI. In addition, these methods tend to overfit and overparameterize the classification model, leading to false-positive results and also overestimating the image saliency. Other recent state-of-the-art methods [START_REF] Sun | Sg-fcn: A motion and memory-based deep learning model for video saliency detection[END_REF]; [START_REF] Faisal | Epo-net: Exploiting geometric constraints on dense trajectories for motion saliency[END_REF][START_REF] Patil | Motion saliency based generative adversarial network for underwater moving object segmentation[END_REF]; [START_REF] Maczyta | Cnn-based temporal detection of motion saliency in videos[END_REF]) use convolutional neural networks to learn the motion saliency in nature images yet. These methods use a lot of training data to learn its variability and combine other classical strategies as optical flow to extract relevant information. Data availability is a major limitation for applying these methods in medical images, which in addition are much more complex than natural images.

iii) Other strategies. [START_REF] Kim | Spatiotemporal saliency detection for video sequences based on random walk with restart[END_REF] modeled motion as a Markov process in natural images, using a graph with transition probabilities among the graph nodes and each node being a state of the process. A temporal saliency distribution is obtained as a steady-state distribution of a random walker running over such a graph. Spatial features of intensity, color, and compactness are used to estimate the transition between states. However, this strategy is based on assumptions that are impossible to fulfill in data with high variability, absence of color, and subtle differences between objects.

The proposed method uses neither optical flow nor saliency learning, but it integrates temporal information, unlike most saliency methods in medical imaging. Spatiotemporal saliency models may simply correlate subtle morphological abnormalities during the cardiac cycle by capturing the variability of motion of these regions. In contrast to the state-of-the-art spatio-temporal saliency methods, designed for 2D natural images, the proposed descriptor uses the third dimension and obtains a better representation of the heart movement. Given the 3D shape nature of the cardiac structures, global and compact information may be estimated even in fuzzy slices like the apex. In addition, the proposed descriptor follows a motion perception model, and therefore the spatio-temporal salient information is captured even in noisy environments. Likewise, a minimal number of the model parameters has to be tuned, and then no training or huge quantities of data are needed to capture the high structure and dynamics variability of the heart. This motion perception model is described in the following section 3.

Introducing the motion perception model

The visual system reacts to changes or discontinuities and captures the salient spatio-temporal information by estimating several features or primitives with coherent motion during a sequence, even in noisy conditions [START_REF] Johansson | Visual perception of biological motion and a model for its analysis[END_REF]). Motion processing is carried out at two different levels: the primary visual cortex and the middle temporal (MT) area [START_REF] Treue | Attentional modulation of visual motion processing in cortical areas mt and mst[END_REF]; [START_REF] Albright | Visual motion perception[END_REF]). At the former, the first processing consists in locally detecting oriented spatio-temporal discontinuities of low-level primitives, namely contrast, texture, and edges. Relevant primitives are small regions moving with different orientations and velocities. A global velocity estimation is then constructed at the latter area [START_REF] Albright | Visual motion perception[END_REF]), the MT area, which integrates the output of the first area and defines a preferred direction of motion which serves as a global reference of the moving primitives. Different primitives may be coherently perceived from the built-up reference [START_REF] Schrater | Mechanisms of visual motion detection[END_REF]).

The proposed STSalmap descriptor aims to mimic this dynamic by estimating the coherent motion of relevant features in a sequence. For doing so, every sequence frame is firstly decomposed in multiple scales which are divided into small irregular regions, each containing oriented responses from low-level features. The motion of these irregular regions is estimated at any time t by superimposing the partition at time t to the rest of the sequence. Afterward, the sum of the squared differences between the frame at time t and the other frames is computed at each region, obtaining an oriented regional motion. Since the energy of these motions are uniformly distributed over all orientations, these oriented motions are then integrated, approximating a regional speed pattern, i.e., the local motion corresponds to the spatio-temporal locations where energy is concentrated [START_REF] Schrater | Mechanisms of visual motion detection[END_REF]; [START_REF] Adelson | Spatiotemporal energy models for the perception of motion[END_REF]). Finally, these results are averaged in the distinct scales to remove uncorrelated noise and preserve the most relevant information, even in short or large displacements. This information represented the coherent or global motion of moving features at any instant of time t of the sequence, being it correlated with psychological studies reported by [START_REF] Schrater | Mechanisms of visual motion detection[END_REF] and [START_REF] Bradley | Velocity computation in the primate visual system[END_REF]. This global motion was here defined as the motion saliency map. This section briefly introduces a spatio-temporal saliency descriptor to characterize cardiac movement patterns. The workflow is shown in Fig. 1. The proposed STSalmap descriptor starts by mapping the original image to a multi-scale space, as described in subsection 4.2.2, to generate a perscale-coarse spatial saliency (subsection 4.2.3). Likewise, a region-based approach splits each of the different scales into a set of disjoint regions, as presented in subsection 4.2.4. Afterward, a spatio-temporal saliency map is regionally estimated as an average of the saliency at each of the scales, as further explained in subsection 4.2.5. Finally, regional left ventricular (LV) wall segmentations at end of the diastole and the end of the systole are superimposed to the obtained saliency and movement patterns are characterized by computing a dispersion measure of the saliency at each of the myocardium segments.

Method

Overview

4D cardiac MRI

Spatio-temporal saliency map descriptor (STSalmap)

Problem formulation

Let us denote an image I ⊂ R 3 , being (x, y, z) the spatial coordinates of I. A temporal sequence is defined as the set T of times t , T = {t = 1, t = 2, . . . , t = N}, where N is the number of frames. Hence, an image at time t ∈ T is defined as I(t ) and the STSalmap descriptor maps I(t ) to its corresponding salient image S(t ) by quantifying regional spatio-temporal changes of oriented features during the sequence T.

Multi-scale representation

A multi-scale analysis is applied to extract the most relevant information preserved through the scales. Each image I(t ) from the temporal sequence T is decomposed into several s scales, I(t , s), by applying a Gaussian pyramidal decomposition [START_REF] Burt | The laplacian pyramid as a compact image code[END_REF]). This pyramid is constructed by iteratively smoothing and down-sampling/up-sampling (reduction/expansion direction) the image. Smoothing is carried out for each voxel of the image as a weighted average of its neighboring voxels. The weights are computed by the kernel w, defined as w [START_REF] Burt | The laplacian pyramid as a compact image code[END_REF]), being a the parameter that controls the shape of this kernel. In this work, a = 0.375, for which w is approximated to a Gaussian distribution. Then, this smoothed image is dyadically down-sampled/up-sampled along each direction, without modifying the length of the third dimension of the image. This scale factor has been commonly used in image analysis trying to remove non-correlated noise and to preserve the scale invariance law [START_REF] Florack | Scale and the differential structure of images[END_REF]). In this work, the multi-scale representation is set to three coarser levels, the original scale, and a single finer level.

= [ 1 4 -a 2 , 1 4 , a, 1 4 , 1 4 -a 2 ]

Extraction of oriented features

Visual perception of moving structures is a quite robust mechanism even under very noisy conditions, e.g., fuzzy edges, superposition, occlusion, among others. In particular, most imaging techniques of the cardiac dynamics are contaminated by different types of noise, being inherent to the properties of each physical principle of the image. Gaussian, Poisson, and Rician correspond to typical noises found in medical imaging. Movement patterns are then difficult to characterize, especially for cardiac problems because the heart seems to move slower or less than it moves [START_REF] Massanes | Motion perception in medical imaging[END_REF]). The main purpose of this investigation is to find out relevant moving features and their basic relationships. The first step is to convert low-level primitives of the image, such as orientation, contrast, and intensities, into relevant information by applying an oriented filter [START_REF] Freeman | The design and use of steerable filters[END_REF]), which mimics the basic directional decomposition achieved by the retina neurons in the first phase of the perception phenomenon [START_REF] Mather | Interactions between motion and form processing in the human visual system[END_REF]). This filter is applied to every image I(t , s) at time t of the temporal sequence T and at each of the scales s, obtaining a salient orientation F(t , s, θ) at arbitrary θ direction. This oriented filter is an orientation-selective convolution kernel G of the oriented first derivative of a 2D Gaussian function f (x, y), being the 2D Gaussian function defined as:

f (x, y) = 1 2πσ 2 exp -(x 2 + y 2 ) 2σ 2 (1)
Where the first x derivative of this Gaussian f (x, y) rotated by an angle 0°about the origin defined as

G 0°= ∂ f (x, y) ∂x = - x σ 2 exp( -(x 2 +y 2 ) 2σ 2 ) σ 2π (2)
And the first y derivative of that same function, rotated 90°, corresponds to

G 90°= ∂ f (x, y) ∂y = - y σ 2 exp( -(x 2 +y 2 ) 2σ 2 ) σ 2π (3)
Being G 0 and G 90 the basis filter. Since convolution ( ) is a linear operator, the image I(t , s) can be synthesized at an arbitrary orientation by performing linear combinations of the image I(t , s) with the basis filter as follows:

R 0°= G 0° I(t , s) (4) R 90°= G 90° I(u, t , s) (5)
The oriented filter (G 0°a nd G 90°) is applied to the 3D data by moving the first derivative Gaussian 2D kernel slice per slice. Finally, the oriented response in projection functions cos θ and sin θ of the features in the image I(t , s), at the arbitrary θ direction is

F(t , s, θ) = cos (θ)R 0°+ sin (θ)R 90°( 6)
The intensity of the original input image is filtered by the orientation filter whose output F(t , s, θ) is normalized per volume by using min-max algorithm. Notice how this approximation corresponds to an actual continuous representation in discrete terms. In this work, following orientations were used Θ ={0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°}. The σ parameter of this filter (Eq 1) was set to 3 after an exhaustive exploration, described in Section results 5. The same σ value was used in all scales in the multi-scale approach of the STSalmap descriptor since this parameter is invariant scale. Unlike Gabor filters, widely used, these oriented filters are computationally efficient since any arbitrary orientation can be obtained as a linear combination of a few filter responses [START_REF] Huang | Motion estimation method using a 3d steerable filter[END_REF]).

Region-based approach

Motion is estimated by computing regional spatiotemporal changes of low level features at arbitrary directions. Localized changes are herein estimated by quantifying the L2-norm of pairwise spatio-temporal differences of oriented features, computed for each region of the scene. This regional information correlates with objects moving during a sequence and is computed within regions defined by a supervoxel partition (SLIC) [START_REF] Achanta | Slic superpixels compared to state-of-the-art superpixel methods[END_REF]) which matches region boundaries (supervoxels) with edges of structures. For doing so, SLIC algorithm uses a 4-dimensional (4D) space to represent a region I(t , s) as (l , x , y , z ), being l the intensity and (x , y , z ) the region spatial position. Given L, a desired number of regions, the image I(t , s) is spatially divided in a grid of L similarly-sized regions, each of them with a center (l i , x i , y i , z i ), and i = 1, . . . , L. Every region is then iteratively modified by assigning a voxel to the closest region center and updating the region center location. At each iteration, the gradient of every 3 × 3 × 3 neighborhood is computed and the center of each region is set to the spatial position of the smallest gradient, avoiding the center coinciding with an edge. The closest region center i for each voxel j is then determined by a metrics D that combines intensity and spatial distances as follows: 7)

d i nt ensi t y = (l j -l i ) 2 (
d spat i al = (x j -x i ) 2 + (y j -y i ) 2 + (z j -z i ) 2 (8) D = d i nt ensi t y m 2 + d spat i al Q 2 (9)
where Q is the initial distance between two adjacent region centers and m is a compactness parameter, a scalar value that controls the shape of the region using both intensity similarity and spatial proximity. This parameter is herein regionally adapted by forcing the region center to be the mean of all voxels belonging to the supervoxel, up to 10 iterations, since this number has demonstrated to be enough to find regions with minimum intra-region variance [START_REF] Achanta | Slic superpixels compared to state-of-the-art superpixel methods[END_REF]). However, the SLIC method may mismatch region boundaries and structures by an unsuitable number of regions L. Hence, this L parameter is adaptively estimated for each image I(t , s) aiming to match an initial regular tessellation with regions (supervoxels) of size ω × ω × ω. In this paper, ω = 7 is set as suggested by Kervrann et al. [START_REF] Kervrann | Optimal spatial adaptation for patch-based image denoising[END_REF]. The number of regions L(t , s) at scale s and time t is here then defined as:

L(t , s) = Round Di m(I(t , s)) ω + med i an(Di m(I(t , s)))
(10) being Dim(I(t , s)) a vector whose elements are the corresponding spatial dimensions of the image I(t , s) and Round the L(t , s) value to the nearest integer. Once this L number of regions is obtained, the supervoxel partition is performed for each scale of the sequence I(t , s), as previously mentioned, to deform the initial tessellation to irregular regions whose edges remain well aligned with structural boundaries. Each obtained region r (t , s), r ∈ R 3 , from the image I(t , s) represents the spatial coordinates of the supervoxel, contained in the spatial domain of the image at time t and scale s. The set of regions for each image is then stored in a grid L G (t , s) to be used in the next step.

Spatio-temporal saliency map

The non-linear dynamic of the cardiac motion describes a series of smooth and abrupt changes during the same cardiac cycle, different at global and local levels, i.e., parts of the heart move more, and others remain static during different phases of the cycle. The proposed motion saliency model captures local and global changes of the heart during the cardiac sequence as follows.

Local salient motion.. The local saliency of a region r ∈ I(t , s) is estimated along the sequence of images I(T, s) by locally detecting oriented spatio-temporal discontinues in each direction θ. For doing so, I(T, s) is filtered as described in subsection 4.2.3, obtaining a set of oriented features F(T, s, θ). The grid L G (t , s) at time t and scale s is superimposed to the whole sequence F(T, s, θ), and using this grid as spatial support, the differences between the features F(t , s, θ) at time t and the rest of the sequence frames T is computed. Finally these differences are summed up to obtain:

E (r, t , s, θ) = N τ=1 F(r, t , s, θ) -F(r, τ, s, θ) 2 (11)
where N is the number of images of the sequence. Equation 11 describes the summation of the spatio-temporal changes (map of changes) with respect to the time t of the sequence. This regional oriented motion (Eq 11), after integration along the different directions θ, yields the regional speed pattern of the salient features S(r, t , s), at time t and scale s:

S(r, t , s) = θ∈Θ E (r, t , s, θ) (12)
Global salient motion. The regional speed pattern is simply lower when the region visual information is uniformly distributed and much higher when it is concentrated in specific orientation bands, imitating thereby what has been reported in psychophysical experiments of the human visual system [START_REF] Schrater | Mechanisms of visual motion detection[END_REF]). This speed pattern is computed for every region r ∈ L G (t , s), obtaining a saliency map S(t , s) for each scale. Afterward, each of these scaled saliency maps is interpolated Ŝ(t , s) to the spatial dimensions of the original image, using the nearest-neighbor algorithm, and averaged to obtain the global motion saliency, i.e., a master spatio-temporal saliency map S(t ):

S(t ) = 1 5 5 s=1 Ŝ(t , s) (13)
This average is a low-pass filter that further removes uncorrelated noise and preserves the most relevant information in all scales [START_REF] Allard | Reducing luminance intensity can improve motion perception in noise[END_REF]).

Characterization of movement patterns in cine MRI

Abnormal movements of segments of the left ventricle (LV) have been reported during the cardiac cycle in cases of cardiac failure [START_REF] Rickers | Utility of cardiac magnetic resonance imaging in the diagnosis of hypertrophic cardiomyopathy[END_REF]; [START_REF] Mishiro | Regional left ventricular myocardial contraction abnormalities and asynchrony in patients with hypertrophic cardiomyopathy evaluated by magnetic resonance spatial modulation of magnetization myocardial tagging[END_REF]; [START_REF] Sanchez-Martinez | Machine learning analysis of left ventricular function to characterize heart failure with preserved ejection fraction[END_REF]; [START_REF] Garg | Association of concentric left ventricular hypertrophy with subsequent change in left ventricular end-diastolic volume: the dallas heart study[END_REF]). For this reason, motion characterization of the left ventricle is herein carried out at the level of LV segments, as follows.

Anatomical segments of the left ventricular wall

Currently, the LV function is more precisely estimated by a regional examination of a set of anatomic segments of the LV wall, as recommended by the American Heart Association [START_REF] Cerqueira | Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association[END_REF]). Usually, this analysis is manually carried out per individual, a delicate time-consuming task that may introduce error by the high intra-and intervariability heart shapes. An automatic LV wall segmentation is herein proposed, in which the wall is segmented into four segments: the first three, septal, anterior, and inferior are those parts nearby the septum, lung, and liver respectively, and the fourth lateral segment is the free wall (as observed in Fig. 2). Likewise, in the plane orthogonal to the cardiac long axis, the LV is divided into four circular sections: basal, midcavity, apical, and the apex section at the end of the ventricular chamber, composed of those slices not showing any LV cavity at all. Figure 2 illustrates how the LV wall in the short axis slice Υ is automatically parcelled, as long as both right (RV) and left ventricles are segmented. For each slice, the strategy consists in tracing two lines (in blue), which must pass by the LV centroid c = (x c , y c ) and connect opposite sides of the border of the slice Υ. These two lines, Γ α and Γ β in Fig. 2, must in addition cross the closest point of the external surface of the right ventricle (ρ 1 and ρ 2 in Fig. 2). An exhaustive search at every clockwise degree sets the two lines defined by x j = j cos(λ) + x c and y j = j si n(λ) + y c with j = 0, 1, 2, . . . , k, being k the border partition in degrees and λ a particular direction. As shown in Fig. 2 these two lines define the different wall segments, i.e., septal, anterior, lateral, and inferior. Likewise, the basal, mid, and apical sections along the long axis are obtained by simply dividing the LV cavity into thirds, while the fourth section, the apex, corresponds to those slices without any LV cavity.

Movement quantification in cine MRI

Once the LV myocardium segments are obtained, salient information may be quantified. Intuitively, high values of saliency are associated with more motion, i.e., the more a segment constantly moves, the greater its saliency is. This analysis can be useful in specific disorders like hypertrophic cardiomyopathy, where the non-homogeneous muscle growth may be correlated with a scattered saliency by the presence of both rapid and slow-motion patterns of the whole myocardial wall during the cardiac cycle [START_REF] Kramer | Regional heterogeneity of function in hypertrophic cardiomyopathy[END_REF]; [START_REF] Marian | Hypertrophic cardiomyopathy: genetics, pathogenesis, clinical manifestations, diagnosis, and therapy[END_REF]).

In this context, a measure of the saliency dispersion, the coefficient of variation CV, quantifies the segment movement. The higher this value, the greater the motion variability. This coefficient is dimensionless and scale-invariant, useful to describe data variation with respect to its mean, i.e., this approach facilitates comparison between different subjects by removing the influence of the heart size. This coefficient is defined as:

CV seg = σ seg µ seg , seg = 1, . . . , N seg
being σ seg and µ seg m the standard deviation and mean of the saliency computed in the segment seg . This coefficient is computed for all segments, N seg , in which the left ventricular wall was divided.

Results

Data

The STSalmap descriptor was applied to short-axis cine MRI images from four public available cardiac image datasets. These datasets are: Automated Cardiac Diagnosis Challenge, ACDC [START_REF] Bernard | Deep learning techniques for automatic MRI cardiac multistructures segmentation and diagnosis: Is the problem solved[END_REF]). Eighty cine MRI cases were used from this database. Images were acquired, over six years, with two MRI scanners of 1.5 T (Siemens Area) and 3.0 T (Siemens Trio Tim) (Siemens Medical Solutions), with spatial and temporal resolutions of 1.5 ± 0.2 mm 2 /pi xel and 27.1 ± 6.1 frames per cycle, respectively and a slice thickness of 9.3±1.7mm [START_REF] Bernard | Deep learning techniques for automatic MRI cardiac multistructures segmentation and diagnosis: Is the problem solved[END_REF]). These cases were divided into four subgroups of 20 individuals:

• Control.

• Previous myocardial infarction (MINF) with and an ejection fraction of the LV (LVEF) lower than 40% and myocardial segments with abnormal contraction.

• Hypertrophic cardiomyopathy (HCM) with an LV cardiac mass higher than 110 g /m 2 , several myocardial segments with a thickness higher than 15 mm in diastole and a normal LVEF

• Dilated cardiomyopathy (DCM) with diastolic left ventricular volume > 100 mL/m 2 and an LVEF lower than 40%.

Cardiac MR Left Ventricle Segmentation Challenge MICCAI 2009, MI09 [START_REF] Radau | Evaluation framework for algorithms segmenting short axis cardiac MRI[END_REF]). Twelve cases were acquired with an MRI scanner of 1.5 T GE Signa MR at spatial resolution and slice thickness of 1.4 ± 0.1 mm 2 /pi xel and 8.9 ± 1.0 mm respectively, and temporal resolution of 20 cardiac frames over the heart cycle. The used data comprises two subgroups: three control cases with no hypertrophy but LVEF > 55% and nine HCM cases with normal LVEF and a ratio of LV mass over body surface area higher than 83 g /m 2 .

Cardiac MRI dataset from York University, YORK (Andreopoulos and Tsotsos ( 2008)). Seven cases (one control and six HCM) were acquired with an MRI scanner of 1.5 T GE Genesis Signa MR at spatial resolution and slice thickness of 1.4±0.2mm 2 and 8.9±1.5mm, respectively, a temporal resolution of 20 frames per cardiac cycle. All the cases were under the age of 18.

Motion tracking challenge from STACOM 2011, MI11 [START_REF] Tobon-Gomez | Benchmarking framework for myocardial tracking and deformation algorithms: An open access database[END_REF]). Nine control cases (aged 28 ± 5 years) were used from this dataset. Images were acquired with an MRI scan of 3 T Philips Achieva System, with spatial and temporal resolutions of 1.2±0.1mm 2 pi xel and 23.4±8.3 frames per cycle, respectively, and a slice thickness of 8 mm.

Each dataset provides manual delineations of the left ventricle (LV) myocardium and right ventricle cavity (RV) at the end of the diastole (ED) and the end of the systole (ES).

Evaluation

The present method associates the characterized movement patterns with control and pathological subjects from heterogeneous datasets of cine MRI images. Such association was twofold demonstrated: the former consisted in discriminating control and hypertrophic (HCM) subjects with an SVM (Gaussian Radial Basis Function) classifier (Subsection 5.4) and the latter, in finding out significant differences of regional movement patterns between control and subjects diagnosed with HCM, dilated cardiomyopathy and myocardial infarction (Subsection 5.5).

Setting parameters

Only two parameters required some tuning. Firstly, the standard deviation σ of the oriented filter (Equations 2 and 3) which was set to 3 after exhaustive exploration (σ = {0.2, 0.5, 3, 5, 8, 10}). Secondly, since the image is divided into a number of regions L in the region-approach step (Eq. 10 of subsection 4.2.4), such number was computed by the adaptive strategy already described in the same section, with initial region size of ω = 7 after exploring with ω = {7, 11} and number of regions set to L = {100, 1000, 2000}.

Movement patterns of control and HCM subjects

This part of the evaluation was addressed to show that the motion characterization separates control and HCM patients in a classification task.

Experimental setup

Features. For each subject, saliency was computed at the ED using only volumes from the diastolic phase to evaluate the diastolic function as an indicator of disease. The left ventricle (LV), at the ED, was partitioned into four anatomical segments, as explained in subsection 4.3.1. A dispersion saliency measure was computed in each anatomical segment at the ED (movement patterns as described in subsection 4.3.2). As result, each subject was characterized by four features representing the saliency dispersion at different segments in the diastolic phase.

Classification. The experimental group from the whole set of heterogeneous databases consisted of 33 control subjects and 35 HCM subjects. A support vector machine (SVM) classifier with radial basis function (RBF) kernel was used to separate control and HCM subjects. Parameters of this kernel were automatically determined by a grid search with 3-fold crossvalidation. A Z-score normalization was applied to features of each training and testing data. The classification was performed in two experiments:

Experiment 1: The SVM classifier was trained and tested using several K-fold cross-validation setups, with K = {3, 5, 7} and using all datasets.

Experiment 2: The SVM classifier was trained only with the ACDC dataset (40 subjects, 20 control, and 20 HCM), and the other datasets were used for testing.

Performance evaluation. Performance was quantified by different metrics, namely accuracy, sensitivity, specificity, precision, area under the curve (AUC) and F-measure. F-measure was defined as 2 * ((precision × recall)/(precision + recall));.

Baselines. Since the proposed strategy estimates a spatiotemporal saliency, the same protocol of experimentation was applied to the methods presented by [START_REF] Liu | Superpixel-based spatiotemporal saliency detection[END_REF] (Liu), [START_REF] Wang | Consistent video saliency using local gradient flow optimization and global refinement[END_REF] (Wang) and [START_REF] Kim | Spatiotemporal saliency detection for video sequences based on random walk with restart[END_REF] (Kim). These three methods were here selected for two reasons: they estimate spatio-temporal saliency and their source codes were available. The first two approaches used optical flow to estimate saliency, while Kim et al. modeled motion as a Markov process.

Figure 3 illustrates the saliency obtained at the ED in the basal slice from a control subject by several spatio-temporal saliency methods. Note how Liu and Wang methods highlighted the ventricular cavities as the more salient structures, but practically no local motion information is observed. Some regional motion is captured by both Kim's and the proposed method. Nevertheless, the structures highlighted by Kim are blurry and practically non-interpretable. In contrast, the present method nicely delineates the cardiac structures and it is easy to observe, as expected, how the motion pattern is localized in the free wall of the left ventricle and lateral wall of the right ventricle.

Classification results

Experiment 1: K-fold cross-validation setups. Three setups, with K = {3, 5, 7}, were used to assess the aptness of this characterization to separate control subjects from HCM subjects. Results for each K-fold are shown in Table 1. Note how the STSalmap descriptor outperformed baseline methods for each of the metrics and most K-fold groups. Kim obtained similar F-measure values (in about 0.71), but overall their other metrics show lower values with higher standard deviations. Liu reported an average F-measure of 0.44 and Wang's F-measure values were not included since subjects were all classified as HCM in all folds. These results look consistent with what has been shown in Fig. 3, i.e., Liu's estimated motion saliency looks parceled and with no relation with ventricular structures while the other methods are concentrated within the cardiac cavities but Kim's method also highlights motion from breathing movements within the thoracic box. In addition, significant statistical differences were observed when applying a test t-student between the presented approach and each of the state-of-the-art methods: Liu (p-value<0.05), . Figure 5 illustrates the short-axis view at basal and midlevels (in rows) of four different subjects at the ED and their respective STSalmap (color map), superimposed to each of the corresponding slices. Saliency in these images highlights myocardium zones with higher motion in red and areas with little movement in blue. Illustration of these sections is or-ganized, the control in the first two columns, and the HCM in third and fourth columns Fig. 5. These subjects are selected from different databases, demonstrating the method independence of the particular acquisition protocol or the anatomic cardiac level. As previously mentioned, saliency provides quantifiable heart motion information, and interestingly, the first two columns in Fig. 5 illustrate how control subjects show redder LV movement patterns, i.e, motion is more important and homogeneously observed for the whole ventricular structure along with the diastolic phase, in contrast to HCM subjects who appear to concentrate the ventricular movement in the lateral wall (red), while the septum and inferior segments show little motion (blue areas), a finding widely reported in the literature in case of HCM patients [START_REF] Rickers | Utility of cardiac magnetic resonance imaging in the diagnosis of hypertrophic cardiomyopathy[END_REF]; [START_REF] Mishiro | Regional left ventricular myocardial contraction abnormalities and asynchrony in patients with hypertrophic cardiomyopathy evaluated by magnetic resonance spatial modulation of magnetization myocardial tagging[END_REF]).

Regional and local motion patterns for control and pathological subjects

Figure 5 illustrates the short-axis view at basal and midlevels (in rows) of four different subjects at the ED and their respective STSalmap (color map), superimposed to each of the corresponding slices. Significant differences, between control and HCM groups, were established by a t-test of the saliency dispersion (α = 0.05, p-value < 0.05) for the LV segments. Two tests were here performed: the former compared the saliency dispersion at the four anatomical LV segments (septal, anterior, lateral, and inferior) of the whole ventricular volume, and the latter compared the local saliency dispersion at different ventricular levels, specifically, basal, mid and apical sections, aiming to show the aptness of the STSalmap to describe local motion patterns. For doing so, in this second test, a different ventricular partition was obtained by dividing in half the septal and lateral segments of the basal and mid sections. This partition was then composed of sixteen segments, the usual four of each of the three ventricular levels, and the new subdivisions at basal and mid-levels, that is to say, anterior-septal, inferior-septal, anterior-lateral, and inferior-lateral, following the AHA protocol [START_REF] Cerqueira | Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association[END_REF].)

Regional motion patterns in control and HCM

Saliency dispersion, defined in Eq. 4.3.2, was computed for each of four LV segments (septal, anterior, inferior, and lateral), at the ES and ED, for 20 control and 20 HCM subjects from the ACDC dataset. Saliency at each time was estimated by using only frames of each phase, and average and standard deviation were computed for each anatomical segment. Figure 6 illustrates the saliency dispersion average (lines) and standard deviation (shaded area) for control (in blue) and HCM (in red). Overall, control subjects exhibit a lower and more stable dispersion along the LV myocardium wall than HCM subjects, during both cycle phases. Significant differences of the dispersion values between control and HCM were found for septal (p-value < 0.05) and anterior (p-value < 0.05) segments at both the ES and ED. This higher dispersion in septal and anterior segments of HCM subjects might be attributed to the compensatory and dissimilar hypertrophy re- action in these patients which produces complex motion patterns during diastolic and systolic phases. In contrast, control and HCM subjects exhibit more similar dispersion patterns at the inferior and lateral segments at both phases, yet differences were also found to be significant (p-value < 0.05), but in general smaller than those observed in the other two segments. Notice also how the dispersion trend for both phases, in the control group, shows a slight decrease at the anterior, inferior, and lateral segments, while the septal dispersion is conserved. This pattern is also observed in the HCM group, even though it shows higher values, but the septal dispersion increases at the ES. This particular value may suggest hypertrophy is more concentrated in septal and anterior areas, the reason why the contractility is usually impaired in these segments, a well-known sign of the hypertrophic cardiomyopathy [START_REF] Rickers | Utility of cardiac magnetic resonance imaging in the diagnosis of hypertrophic cardiomyopathy[END_REF]; [START_REF] Mishiro | Regional left ventricular myocardial contraction abnormalities and asynchrony in patients with hypertrophic cardiomyopathy evaluated by magnetic resonance spatial modulation of magnetization myocardial tagging[END_REF]; [START_REF] Betocchi | Regional left ventricular mechanics in hypertrophic cardiomyopathy[END_REF]; Garg et al. ( 2017)).

The saliency as a metrics of cardiac impairment

Regional motion patterns are here mapped to a feature space with eight dimensions (saliency dispersion of the four segments at the ED and ES). This space is complex by nature and it is not straightforward how to compare multidimensional points and how motion information may be distributed. In such space, the Mahalanobis distance determines a particular grouping, control, and HCM, which may assume other forms than spherical. This distance was normalized by the maximum distance to provide a reliable comparison between both groups. Figure 7 illustrates a 2D projection of such space. For doing so, any subject (multidimensional point) is plotted as follows, the x coordinate corresponds to the Mahalanobis distance of the point to the control group and the y coordinate stands for the Mahalanobis distance to the HCM group. Therefore, the higher the distance to the control group, the higher the case probability of having HCM. Overall, HCM subjects (red triangles) are concentrated in an HCM zone (red area), where the distance to the HCM group is small and to the control group is vari- able. Likewise, control subjects were concentrated in the blue zone, where the distance to the control group is small. Only one control subject obtained a small distance to the HCM group. This subject shows little deformation in the septal segment when visually inspected. Interestingly, this analysis locates individuals with a high probability of belonging to the correct class. This analysis quantifies the degree of motion impairment and eventually could be used as a potential risk indicator of HCM. Distance to Control group Distance to HCM group 1 Fig. 7: 2D projection of the feature space, x coordinate corresponds to the Mahalanobis distance of any multidimensional point to the control group while the y coordinate represents the Mahalanobis distance to the HCM group. The golden line stands for the division between HCM and control subjects (provided in the dataset). Transition zone represents a classification error probability of about 30% for each group.

Localized motion patterns in control, HCM, DCM and MINF

A more local analysis was also performed by dividing the LV into sections (basal, mid, and apical) and the LV wall in anatomical segments (septal, inferior, lateral, and anterior). In basal and mid sections, the septal segment was divided into anteroseptal and inferoseptal, and the lateral segment was divided into anterolateral and inferolateral. All these divisions were automatically obtained as described in Subsection 4.3.1 by following the AHA model [START_REF] Cerqueira | Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association[END_REF]). The saliency dispersion was computed for each of the anatomical segments and sections at the ED and ES for each of 20 control, 20 HCM, 20 DCM, and 20 MINF subjects from the ACDC dataset. The average of the saliency dispersion for each segment and section at each time was computed to illustrate a global pattern (Fig. 8). Note how control subjects show a more homogeneous motion along the LV wall at the ED. At the end of the systole, a significant difference between control and MINF groups was found for the mid/apical anterior segments (test t-student, p-value < 0.05) and mid/apical inferior segments (test t-student, p-value < 0.05). These differences correlate with the impaired rotation in the LV apical/basal sections reported in MINF subjects [START_REF] Park | Different impacts of acute myocardial infarction on left ventricular apical and basal rotation[END_REF]). DCM also showed significant differences with control subjects at the ES in the mid/apical anterior segments (test t-student, p-value < 0.05) and the basal/mid-inferior and all apical segments (test t-student, p-value < 0.05), results that might be related with impaired apical rotation, as reported in DCM [START_REF] Kanzaki | Impaired systolic torsion in dilated cardiomyopathy: reversal of apical rotation at mid-systole characterized with magnetic resonance tagging method[END_REF]). As visualized in Fig. 6, control and HCM subjects presented a slight homogeneous motion in the lateral wall at both phases. While the septal and anterior segments in HCM evidenced a mix of movements. At the ED, these differences were also noted, with slight differences concerning the ES.

Note how multiple t-tests were performed per segment, and separately per cardiac phase: control group vs MINF group, control group vs HCM group, and control group vs DCM group. We obtained 48 t-tests per cardiac phase, at the ED and ES. Since p-value <0.05 was established in the conclusions, finding significant differences between groups per segment for each cardiac phase, we computed the Bonferroni correction to see the significance levels corrected for multiple testing. For doing so, the Bonferroni correction was estimated as 0.05/48=0.001. At the ES, we found significant differences in all segments except for 1st, 2nd, 5th, 6th, 9th, and 14th (anterior and lateral segments in the basal region), in which similar motion patterns were observed in all groups. While at the ED, the DCM group obtained no significant difference in the 4th and 5th segments. HYP obtained similar motion patterns in segments 5th, 6th, 10th, 11th, 12th, and 14th segments (infero and antero lateral segments in basal section and apical region). Interestingly, no significant differences were reported for the first segments in the MINF group: 2nd,3rd,5th,6th,8th,9th,and 10th,i.e., in all segments from the basal region.

These findings show the STSalmap descriptor can extract relevant information of the systolic and diastolic function (movement patterns) at any measurable location. It can be correlated with pathological findings. This analysis can be carried out per patient to detect local abnormal movement.

Method performance on noisy images

In addition, the presented saliency descriptor was robust to noise when Rician, speckle, and Gaussian noise were separately added to the same cases classified in experiment 2 of Section 5.4. Specifically, speckle noise with mean 0 and variance 0.05 was added to the images, while the Rician noise was configured with mean 0 and 5 Gaussian noise levels. These images were re-normalized by using the min-max algorithm. Results using the two first types of noise did not exhibit differences with the original images (data not shown). A more exhaustive analysis was performed for the Gaussian noise, a more typical MRI noise, by testing different parameter configurations (mean and standard deviation). This analysis is observed in Table 3. The SVM model was trained with the ACDC dataset, contaminated with the Gaussian noise, and tested in the other four datasets. Observe how the descriptor maintains comparable accuracy when the mean and standard deviation are set to 0.3 and relative deterioration is observed when increasing these values, the descriptor discriminates up to 50% of the data in a very noisy environment.

Time-performance analysis

All experiments were performed in a computer with 2.9 GHz CPU and 16 GB of RAM, using a simple Matlab implementation with no training. The average of the volume dimensions was of 241.8 ± 25.3 × 255.2 ± 19.4 × 10.8 ± 2.5. The estimated STSalmap saliency was computed in 3D, unlike the state-of-the-art methods which compute this in 2D. The time in seconds (s) to obtain a salient volume by the LV segments were obtained following the AHA protocol [START_REF] Cerqueira | Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association[END_REF]). Segments one to six are: anterior, anteroseptal, inferoseptal, inferior, inferolateral and anterolateral at the basal section. Segments seven to twelve correspond to the same previous segments but at the mid section while segments thirteen to sixteen are the anterior, septal, inferior, and lateral at the apical section. The 17 t h segment is the apex, but this information was not included. Four groups are compared: Control, hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), and myocardial infarction (MINF). (2015). The saliency descriptor code could be available by contacting the corresponding author directly and under the conditions of citing this article in any scientific publication and, it will be published in https://github.com/amatehortual/MSMS.

Discussion

This work introduced a novel spatio-temporal saliency descriptor to characterize and quantify motion cardiac patterns independently of the acquisition protocol. This saliency descriptor describes and summarizes relevant temporal information of the cardiac function, no matter the particular angle or resolution of the captured video. The underlying idea of this model is to mimic the visual motion perception and extract relevant features by decomposing the scene into regions at multiple scales and detecting local-oriented motion associated with spatio-temporal changes. After these estimations are integrated at the multiple orientations and scales, a coherent regional motion sets a local saliency. This descriptor highlights a relative motion history of different structures which is correlated with normal and abnormal cardiac patterns. This work presents, in addition, a statistical analysis of the ventricular differences in different cardiac entities. Finally, this descriptor was applied to cine MRI to associate local-regional motion with control and pathological patterns, correlating them in terms of well-known anatomic findings.

Several works have characterized myocardial motion patterns to discriminate controls and pathological subjects in cine MRI from homogenous datasets. Among them, techniques such as optical flow, free-form deformation, block matching, and deep learning were used to extract velocity features [START_REF] Peressutti | A framework for combining a motion atlas with non-motion information to learn clinically useful biomarkers: Application to cardiac resynchronisation therapy response prediction[END_REF] 2020). Other methods segmented the ventricles in the mid-slice of cine MRI to track the geometric features (thickness, radius) over time [START_REF] Zheng | Explainable cardiac pathology classification on cine mri with motion characterization by semisupervised learning of apparent flow[END_REF]. However, these related works present some disadvantages. These methods are prone the error yet, the noise accentuates the particular pathology since adaptation mechanisms increase the variability of motion to both, the cardiac and nearby thoracic structures. In another way, impaired apical and basal rotations, specific to myocardial infarction and diastolic cardiomyopathy [START_REF] Park | Different impacts of acute myocardial infarction on left ventricular apical and basal rotation[END_REF]; [START_REF] Kanzaki | Impaired systolic torsion in dilated cardiomyopathy: reversal of apical rotation at mid-systole characterized with magnetic resonance tagging method[END_REF]), would be impossible to characterize with these methods since the analysis was focused in the mid-slice of cine MRI. These methods require correcting large affine and nonlinear transformations associated with respiratory motion. These works in general require still to prove their aptness to generalize their results in heterogeneous databases and to cope with the large range of cardiac anatomic deformations and the particular acquisition protocols.

Unlike the mentioned approaches, the presented spatiotemporal saliency captures motion trends and, as herein demonstrated, they can be associated with several pathological patterns. The estimated saliency stands for a quantified motion, i.e., slow and rapid movements, evidence of the myocardial stage when contracting, relaxing, twisting, untwisting, lengthening, and shortening over time. Because some segments of the myocardium are more affected by the disease, the saliency dispersion was computed for each of the standardized left ventricular segments. A high inter-andintra saliency variability in the segments reflected an abnormal motion pattern, well correlated with disease physiopathology, for instance, the reduced apex motion in patients with infarction. The proposed descriptor captured non-homogeneous movement in the septal and anterior regions, at the end of the diastole and the end of the systole, signs of hypertrophic cardiomyopathy [START_REF] Marian | Hypertrophic cardiomyopathy: genetics, pathogenesis, clinical manifestations, diagnosis, and therapy[END_REF]). Likewise, the high dispersion visualized in the apical-anterior, mid and apical-inferior segments in subjects with dilated cardiomyopathy are well correlated with the impaired torsional dynamics of the left ventricle, a finding widely acknowledged as a marker of the disease severity [START_REF] Popescu | Left ventricular remodelling and torsional dynamics in dilated cardiomyopathy: reversed apical rotation as a marker of disease severity[END_REF]). These findings evidence that saliency can quantify systolic and diastolic dysfunction at local and regional levels, a definite method plus in terms of clinical interpretability. Moreover, the presented descriptor demonstrated to be generalizable since it dealt with several heterogeneous datasets. Furthermore, only two parameters required adjustments, namely the patch size (7 × 7 × 7) and σ (herein set to 3), an attractive matter of this method because it decreases the dependence on data, as it may be the case of other approaches like deep learning, whose robustness is limited by the distribution and size of the training set.

Several limitations of the presented method may be: i) The sum for the set of orientations from 0°to 315°might remove the main motion direction, which could be useful in motionrelated tasks. However, the herein obtained saliency estimates a quantity about the motion history and the relative temporal homogeneity within a region along the time. This characterization is well-correlated with known described disease findings. ii) Comparison with regional cardiac motion descriptors such as regional displacement, velocity, strain, or was not possible since this information was not provided by the available dataset. However, strain and motion tracking methods could also be used for identifying wall motion abnormalities [START_REF] Puyol-Antón | Fully automated myocardial strain estimation from cine mri using convolutional neural networks[END_REF]; [START_REF] Vo | Mri-derived myocardial strain measures in normal subjects[END_REF]. iii) Comparison with state-of-the-art motion saliency methods can be considered unfair since they were not optimally parameterized. However, their source codes were closed and only executable codes were available. iv) Motion patterns characterized by the proposed method can potentially pro-vide new insights of the physio-pathology to devise risk indicators adapted to the myocardial dysfunction. The fact of following up any instant of the cardiac cycle is an advantageous side effect that can be used to design local/global indicators of the temporal cardiac function. disease severity [START_REF] Popescu | Left ventricular remodelling and torsional dynamics in dilated cardiomyopathy: reversed apical rotation as a marker of disease severity[END_REF]). These findings evidence that saliency can quantify systolic and diastolic dysfunction at local and regional levels, a definite method plus in terms of clinical interpretability. Moreover, the presented descriptor demonstrated to be generalizable since it dealt with several heterogeneous datasets. Furthermore, only two parameters required adjustments, namely the patch size (7 × 7 × 7) and σ (herein set to 3), an attractive matter of this method because it decreases the dependence on data, as it may be the case of other approaches like deep learning, whose robustness is limited by the distribution and size of the training set.

Despite these limitations, results evidenced that the proposed framework was able to quantify regional motion patterns associated with cardiac function, an impossible task to reach just by visual inspection of the data. The proposed framework may potentially help to detect and quantify subtle changes in the myocardial function at any time of the cardiac cycle. This study may complement current strategies of cardiac function assessment and can help to find out myocardial dysfunction with more individualized indicators of the cardiac output [START_REF] Weissler-Snir | Prevention of sudden death in hypertrophic cardiomyopathy: bridging the gaps in knowledge[END_REF]. Likewise, the proposed investigation can be applied to both right and left ventricles, achieving a saliency of the whole sequence. An interventricular analysis may provide a description more detailed of cardiac dynamics, and a quantitative progression of heart disease.

Conclusion

The presented method demonstrated robustness and stability at extracting relevant information in short axis cine-MRI images from heterogeneous datasets with images acquired from different equipment and diseases. This saliency information is demonstrated to be useful at highlighting and quantifying abnormal movement patterns during the diastolic and systolic functions associated with hypertrophic cardiomyopathy. Results demonstrate this method correlates with abnormal patterns of the disease at local and regional cardiac levels. In addition, several medical applications could benefit from this descriptor to improve disease understanding [START_REF] Jan | Modern imaging techniques in cardiomyopathies[END_REF]), specifically assessing myocardium function, extracting information for gait and eye motion analysis, and designing adapted indicators for risk management of cardiac or brain complications in angiography, among others. Future research will focus on the characterization of cardiac pathologies in a larger dataset of cine MRI images.

Conflicts of Interest

None.

Fig. 2 :

 2 Fig.2: LV wall is divided by the lines Γ. RV and LV cavities and LV myocardium are illustrated in red, light pink, and dark pink, respectively.

Fig. 3 :

 3 Fig. 3: Spatio-temporal saliency methods. The original image corresponds to the short axis MRI from a control subject at the ED in the basal slice.

  Fig. 5: Spatio-temporal saliency (color map) superimposed with the short axis cine MRI images of different subjects from different datasets at the ED.

Fig. 6 :

 6 Fig. 6: Mean and standard deviation of dispersion values for each of four segments of the LV ventricular wall and the control (in blue) and HCM (in red) subjects from the ACDC dataset at the ED and ES.

Fig. 8 :

 8 Fig.8: Bull's eye plot[START_REF] Bakos | Combined preoperative information using a bullseye plot from speckle tracking echocardiography, cardiac ct scan, and mri scan: targeted left ventricular lead implantation in patients receiving cardiac resynchronization therapy[END_REF] of the saliency dispersion average in different anatomical segments and sections, at the ED and ES (ACDC dataset). LV segments were obtained following the AHA protocol[START_REF] Cerqueira | Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the cardiac imaging committee of the council on clinical cardiology of the american heart association[END_REF]). Segments one to six are: anterior, anteroseptal, inferoseptal, inferior, inferolateral and anterolateral at the basal section. Segments seven to twelve correspond to the same previous segments but at the mid section while segments thirteen to sixteen are the anterior, septal, inferior, and lateral at the apical section. The 17 t h segment is the apex, but this information was not included. Four groups are compared: Control, hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), and myocardial infarction (MINF).

  ; Puyol-Antón et al. (2017); Puyol-Antón et al. (2019); Bello et al. (2019); Piras et al. (2017); Morales et al. (2019); Terpstra et al. (

Table 1 :

 1 Mean (standard deviation) for each of the K-folds used to classify data.

	Method	k	Accuracy	AUC	F-measure
		3 0.74 (0.08) 0.84 (0.04) 0.72 (0.08)
	STSalmap	5 0.72 (0.10) 0.82 (0.09) 0.71 (0.13)
		7 0.74 (0.11) 0.82 (0.08) 0.71 (0.15)
		3 0.51 (0.08)	0.55 (0.11)	0.47 (0.12)
	Liu	5 0.49 (0.11)	0.54 (0.15)	0.46 (0.14)
		7 0.42 (0.13)	0.43 (0.19)	0.40 (0.14)
		3 0.51 (0.01)	0.50 (0.00)	-
	Wang	5 0.52 (0.02)	0.50 (0.00)	-
		7 0.52 (0.03)	0.50 (0.00)	-
		3 0.72 (0.08)	0.80 (0.08)	0.72 (0.09)
	Kim	5 0.72 (0.12)	0.80 (0.12) 0.71 (0.13)
		7 0.72 (0.14)	0.80 (0.14) 0.71 (0.15)

Table 2 :

 2 Classification performance of the test group, 28 subjects (13 control subjects and 15 subjects with hypertrophic cardiomyopathy). Baseline methods:[START_REF] Liu | Superpixel-based spatiotemporal saliency detection[END_REF],[START_REF] Wang | Consistent video saliency using local gradient flow optimization and global refinement[END_REF] and[START_REF] Kim | Spatiotemporal saliency detection for video sequences based on random walk with restart[END_REF].

	Measure	STSalmap Liu Wang Kim
	Accuracy	0.89	0.36	0.57	0.61
	Sensitivity	0.92	0.38	0.54	0.62
	Specificity	0.87	0.33	0.60	0.60
	Precision	0.86	0.33	0.54	0.57
		AUC	0.94	0.36	0.59	0.71
	F-measure	0.89	0.36	0.54	0.59
		1
	True Positive rate	0.5
		0
		0	0.2 0.4 0.6 0.8	1
			False Positive rate
		Our method	Liu	Wang	Kim
	Fig. 4: ROC curve of the 28 testing. Blue, red, yellow and brown lines corre-1 spond to the STSalmap, Liu, Wang and Kim, respectively.

Table 3 :

 3 Accuracy for each Gaussian noise configuration (mean and standard deviation) added to data for the classification task.[START_REF] Liu | Superpixel-based spatiotemporal saliency detection[END_REF]), 45.30 s Wang et al. (2015) and 81.93 s Kim et al.

		standard deviation
	mean	0.3	0.5	0.7	1
	0.3	0.85 0.65 0.68 0.65
	0.5	0.73 0.55 0.58 0.50
	0.7	0.53 0.50 0.60 0.68
	1	0.68 0.63 0.63 0.70
	STSalmap descriptor was about 12.12 s, while obtained times
	from other methods to compute all the 2D slices of a whole
	volume were 51.31 s

Acknowledgments

This work was supported by Colciencias-Colombia, Grant No. 647 (2015 call for National PhD studies), and Région Bretagne in the framework of the Investissement d'Avenir Program through Labex CAMI (ANR-11-LABX-0004).