Tewodros Weldebirhan

Stéphanie Bricq

Fabrice Meriaudeau

Using Polynomial Loss and Uncertainty Information for Robust Left Atrial and Scar Quantification and Segmentation

Keywords: Cardiac MRI, Late Gadolinium Enhancement MRI, Left Atrium, Scar quantification, Segmentation, Deep learning, PolyLoss, Uncertainty

Automatic and accurate segmentation of the left atrial (LA) cavity and scar can be helpful for the diagnosis and prognosis of patients with atrial fibrillation. However, automating the segmentation can be difficult due to the poor image quality, variable LA shapes, and small discrete regions of LA scars. In this paper, we proposed a fully-automatic method to segment LA cavity and scar from Late Gadolinium Enhancement (LGE) MRIs. For the loss functions, we propose two different losses for each task. To enhance the segmentation of LA cavity from the multicenter dataset, we present a hybrid loss that leverages Dice loss with a polynomial version of cross-entropy loss (PolyCE). We also utilize different data augmentations that include histogram matching to increase the variety of the dataset. For the more difficult LA scar segmentation, we propose a loss function that uses uncertainty information to improve the uncertain and inaccurate scar segmentation results. We evaluate the proposed method on the Left Atrial and Scar Quantification and Segmentation (LAScarQS 2022) Challenge dataset. It achieves a Dice score of 0.8897 and a Hausdorff distance (HD) of 16.91mm for LA cavity and a Dice score of 0.6406 and sensitivity of 0.5853 for LA scar. From the results, we notice that for LA scar segmentation, which has small and irregular shapes, the proposed loss that utilizes the uncertainty estimates generated by the scar yields the best result compared to the other loss functions. For the multi-center LA cavity segmentation, we observe that combining the region-based Dice loss with the pixelwise PolyCE can achieve a good result by enhancing the segmentation result in terms of both Dice score and HD. Furthermore, using moderate-level data augmentation with histogram matching improves the model's generalization capability. Our proposed method won the Left Atrial and Scar Quantification and Segmentation (LAScarQS 2022) Challenge.

Introduction

Atrial fibrillation (AF) is an irregular and often very rapid heart rhythm (arrhythmia). During atrial fibrillation, the heart's upper chambers (the atria) beat irregularly and out of synchronization with the heart's lower chambers (the ventricles). AF increases the risk of stroke, heart failure, and other heart-related complications [START_REF] Clinic | Atrial fibrillation -symptoms and causes[END_REF]. One of the most commonly used techniques to treat AF patients is radio-frequency catheter ablation using the pulmonary vein (PV) isolation [START_REF] Wilber | Comparison of antiarrhythmic drug therapy and radiofrequency catheter ablation in patients with paroxysmal atrial fibrillation: a randomized controlled trial[END_REF].

Late Gadolinium Enhancement (LGE), sometimes called delayed-enhancement MRI, is a gold standard imaging technique to visualize and quantify the left atrial (LA) scars. In a clinical routine, human experts generally segment the LA anatomy and LA scars manually. Manual segmentation is time-consuming and suffers from intra-and inter-observer variability. This problem can be addressed by automating the segmentation. However, automatic segmentation of LA anatomy and LA scars from LGE MRI is still challenging due to poor image quality, variable LA shapes, thin LA walls, and small isolated regions of the LA scars [START_REF] Li | Atrialjsqnet: A new framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information[END_REF].

Few studies have been proposed to segment LA cavity from LGE MR images. [START_REF] Gao | Segmentation of the endocardial wall of the left atrium using local regionbased active contours and statistical shape learning[END_REF] [START_REF] Gao | Segmentation of the endocardial wall of the left atrium using local regionbased active contours and statistical shape learning[END_REF] and [START_REF] Zhu | Automatic segmentation of the left atrium from mr images via variational region growing with a moments-based shape prior[END_REF] [START_REF] Zhu | Automatic segmentation of the left atrium from mr images via variational region growing with a moments-based shape prior[END_REF] utilized region-based activecontour and variational region growing with shape prior respectively to segment LA cavity. [START_REF] Tao | Fully automatic segmentation of left atrium and pulmonary veins in late gadolinium-enhanced mri: Towards objective atrial scar assessment[END_REF] [START_REF] Tao | Fully automatic segmentation of left atrium and pulmonary veins in late gadolinium-enhanced mri: Towards objective atrial scar assessment[END_REF] used atlas-based methods leveraging auxiliary images with better anatomical information to help the LA cavity segmentation from LGE MRI [START_REF] Li | Medical image analysis on left atrial lge mri for atrial fibrillation studies: A review[END_REF]. However, accurately segmenting LA cavity using these conventional methods depend on additional information such as shape prior or auxiliary images [START_REF] Li | Medical image analysis on left atrial lge mri for atrial fibrillation studies: A review[END_REF]. Recently, deep learning-based algorithms have been successfully applied to segment LA cavity from LGE MRI. [START_REF] Vesal | Dilated convolutions in neural networks for left atrial segmentation in 3d gadolinium enhanced-mri[END_REF] [START_REF] Vesal | Dilated convolutions in neural networks for left atrial segmentation in 3d gadolinium enhanced-mri[END_REF] proposed a 3D U-Net with dilated convolutions at the bottleneck of the network and residual connections between the encoder blocks to incorporate local and global information. Chen et al. (2018) [START_REF] Chen | Multi-task learning for left atrial segmentation on ge-mri[END_REF] adopted multi-task learning to perform both LA cavity segmentation and pre/post ablation classification. Other works [START_REF] Jamart | Two-stage 2d cnn for automatic atrial segmentation from lge-mris[END_REF][START_REF] Xia | Automatic 3d atrial segmentation from ge-mris using volumetric fully convolutional networks[END_REF][START_REF] Yang | Combating uncertainty with novel losses for automatic left atrium segmentation[END_REF] utilized a two-stage cascaded segmentation framework to first locate the region of interest (ROI) that covers the atrial cavity, then used a second network to segment LA cavity from the cropped ROI. The main problem with these cascaded approaches is that they can be time-and resource-intensive.

Recently, semi-automatic and fully-automatic deep learning based methods have been widely used to segment scar [START_REF] Arega | Automatic myocardial scar segmentation from multisequence cardiac mri using fully convolutional densenet with inception and squeezeexcitation module[END_REF][START_REF] Moccia | Development and testing of a deep learning-based strategy for scar segmentation on cmr-lge images[END_REF][START_REF] Zabihollahy | Myocardial scar segmentation from magnetic resonance images using convolutional neural network[END_REF]. For LA scar segmentation, some studies proposed to use non-deep learning based methods such as thresholding [START_REF] Oakes | Detection and quantification of left atrial structural remodeling using delayed enhancement mri in patients with atrial fibrillation[END_REF][START_REF] Ravanelli | A novel skeleton based quantification and 3-d volumetric visualization of left atrium fibrosis using late gadolinium enhancement magnetic resonance imaging[END_REF], clustering [START_REF] Perry | Automatic classification of scar tissue in late gadolinium enhancement cardiac mri for the assessment of left-atrial wall injury after radiofrequency ablation[END_REF], deformable and graph-based methods [START_REF] Karim | A method to standardize quantification of left atrial scar from delayedenhancement mr images[END_REF][START_REF] Karim | Evaluation of current algorithms for segmentation of scar tissue from late gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge[END_REF]. Although these conventional methods have shown encouraging results, they rely on initial manual segmentation of the LA cavity. Deep learning methods have been presented to automatically segment LA scar from LGE MRIs. Li et al. [START_REF] Li | Atrial scar quantification via multi-scale cnn in the graph-cuts framework[END_REF] proposed to use graph-cuts with multi-scale CNNs to automatically segment LA scar. Other works utilized multi-task learning to jointly segment LA cavity and scar [START_REF] Li | Atrialjsqnet: A new framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information[END_REF][START_REF] Yang | Simultaneous left atrium anatomy and scar segmentations via deep learning in multiview information with attention[END_REF].

In this paper, we proposed a fully automatic deep learning based method that leverages a polynomial loss and an uncertainty based loss to segment LA cavity from multi-center LGE MRIs and LA scar from single-center LGE MRIs, respectively. To increase the variety of the dataset, we also employ various data augmentation techniques, including histogram matching. We evaluated our method on Left Atrial and Scar Quantification and Segmentation (LAScarQS 2022) Challenge dataset. The proposed losses achieve the best result compared to other losses in both the multi-center LA cavity segmentation and the highly imbalanced LA scar segmentation. In addition, the employed data augmentation techniques improve the model's generalization on the LA cavity segmentation from multi-center images.

Dataset

The Left Atrial and Scar Quantification and Segmentation Challenge (LAS-carQS 2022)1 consists of 200 LGE MRIs acquired in a real clinical environment from patients suffering Atrial fibrillation (AF). All the LGE MRIs were collected from three different clinical centers. The images from the first center (University of Utah) were acquired using Siemens Avanto 1.5T or Vario 3T. The voxel resolution of the images was 1.25 × 1.25 × 2.5 mm. The LGE MRIs from the second center (Beth Israel Deaconess Medical Center) were acquired with Philips Achieva 1.5T. The spatial resolution of the images was 1.4 × 1.4 × 1.4 mm. Similar to the second center, the images from the third center (King's College London) were acquired with a Philips Achieva 1.5T. The spatial resolution of the LGE MRI scan was 1.3 × 1.3 × 4.0 mm. The challenge has two tasks. The first one focuses on left atrial blood pool segmentation from multi-center LGE MRIs. The second task focuses on segmentation of left atrial scar [START_REF] Li | Atrialgeneral: Domain generalization for left atrial segmentation of multi-center lge mris[END_REF][START_REF] Li | Atrialjsqnet: A new framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information[END_REF][START_REF] Li | Medical image analysis on left atrial lge mri for atrial fibrillation studies: A review[END_REF]. We declare that the segmentation method implemented for participation in the LAScarQS 2022 challenge has not used any pre-trained models nor extra MRI datasets other than those given by the organizers.

Methods

Network Architecture

For both LA cavity and LA scar segmentation, we employed a 3D segmentation network. The network architecture is based on 3D nnU-Net framework [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF]. As demonstrated in Fig. 1, we altered the standard nnU-Net network architecture by adding Dropout at the network's middle layers [START_REF] Arega | Using mri-specific data augmentation to enhance the segmentation of right ventricle in multi-disease, multicenter and multi-view cardiac mri[END_REF] to lessen overfitting and improve generalization. The U-Net's encoder and decoder consist of 10 convolutional layers where each convolution is followed by instance normalization and Leaky ReLU (negative slope of 0.01) activation function. The kernel size of the convolution is 3 × 3 × 3. During pre-processing, we resampled all the volumes to 0.625mm × 0.625mm × 1.0mm and 0.625mm × 0.625mm × 2.5mm for LA cavity segmentation and LA scar segmentation respectively (the median voxel spacing of the training cases). The intensity of every volume was normalized to have zero-mean and unit-variance.

Loss functions

Recently, [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF] [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF] proposed PolyLoss, a new loss function that expresses the commonly used loss functions such as cross-entropy (Eq. 1) and focal loss (Eq. 3) as a linear combination of polynomial functions. Using Taylor expansion, cross-entropy can be represented as sum of polynomial bases (1 -p) j , as shown in Eq. 2, where p is the prediction probability of the target class [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF]. By dropping the higher-order polynomials and adding terms that perturb the polynomial coefficients, they came up with a simplified version of the polynomial loss called Poly-1. This loss function modifies the cross-entropy by only adding one hyper-parameter (ϵ) [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF], as can be seen in Eq. 2. PolyLoss has shown good performance on computer vision tasks by outperforming cross-entropy and focal losses [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF].

Inspired by [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF], in this paper, we proposed a loss function that uses Dice loss with PolyLoss (Eq. 5) for a LA cavity and scar segmentation. Dice loss is a region based loss that directly optimizes the Dice coefficient metric as shown in Eq. 4. We hypothesized that by combining the region based Dice loss with the polynomial version of the cross-entropy (PolyCE) (Eq. 5) can improve the segmentation of LA cavity from multi-center LGE MRIs.

L CE = -log(p) = ∞ n=1 1 n (1 -p) n , (1)
L P olyCE = L CE + ϵ(1 -p), (2)
L F ocal = -(1 -p) γ log(p), (3)
L Dice = 1 - 2|Y ∩ G| |Y | + |G| , (4)
L DiceP olyCE = L Dice + L P olyCE , (5)
where Y and G represent the predicted and manual segmentation maps, respectively.

Arega et al. (2021) [START_REF] Arega | Leveraging uncertainty estimates to improve segmentation performance in cardiac mr[END_REF] proposed a segmentation model that generates uncertainty estimates (sample variance) during training using the Monte-Carlodropout Bayesian method and utilizes the uncertainty information to enhance the segmentation results by incorporating it into the segmentation loss function [START_REF] Arega | Leveraging uncertainty estimates to improve segmentation performance in cardiac mr[END_REF]. During training, the model is sampled N times, and the mean of these samples is used as the final segmentation. The sample variance (uncertainty) (σ i ) is computed as a variance of the N Monte-Carlo prediction samples of each pixel i [START_REF] Arega | Leveraging uncertainty estimates to improve segmentation performance in cardiac mr[END_REF]. Since sample variance is a pixel-wise uncertainty measure, to determine the image-level uncertainty the mean of the pixel-wise uncertainty values is computed as shown in Eq. 7, where I is the total number of pixels of the image [START_REF] Arega | Leveraging uncertainty estimates to improve segmentation performance in cardiac mr[END_REF]. This image-level uncertainty is considered as uncertainty loss. Then, it is added to a segmentation loss with a hyper-parameter value alpha (α) that controls the contribution of the uncertainty loss to the total loss as shown in Eq. 8. They have shown that uncertainty information can be advantageous, particularly to improve the segmentation of semantically and visually challenging pathologies such as scars which generate higher epistemic uncertainty [START_REF] Arega | Leveraging uncertainty estimates to improve segmentation performance in cardiac mr[END_REF].

In this paper, we proposed to adopt the uncertainty loss in combination with the hybrid loss of Dice and Focal (DiceFocal) loss [START_REF] Zhu | Anatomynet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy[END_REF] (Eq. 6) for LA scar segmentation. We hypothesized that by fusing DiceFocal loss, which has shown good performance on highly imbalanced dataset [START_REF] Ma | Loss odyssey in medical image segmentation[END_REF], with the uncertainty loss (Uncertainty DiceFocal Loss) (Eq. 8) can enhance the segmentation of the more challenging LA scar segmentation.

L Seg(DiceF ocal) = L Dice + L F ocal , (6)
L U ncertainty = 1 I i (σ 2 i ), (7)
L T otal(U ncertaintyDiceF ocalLoss) = L Seg(DiceF ocal) + α × L U ncertainty , (8)

Data Augmentations

We applied a variety of data augmentations to improve the generalization and robustness of the models in the multi-center dataset, including intensity-based data augmentation, spatial data augmentation, and histogram matching augmentation [START_REF] Arega | Using mri-specific data augmentation to enhance the segmentation of right ventricle in multi-disease, multicenter and multi-view cardiac mri[END_REF][START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF]. Histogram matching is the transformation of an image so that the histogram of a source image matches the histogram of a reference image [START_REF] Gorelick | Histogram matching[END_REF]. Mathematically, it is the process of altering one image so that the cumulative distribution function (CDF) of values in each band corresponds to the CDF of bands in another image.

In the LAScarQS 2022 challenge, there was no specific information about each training image regarding the clinical center from which they came from. In this work, we used histogram matching by taking random training images and matching them to a selected low performing training images which had the worst performance in terms of Dice. The matched images were then added to the training dataset to enhance the generalizability of the model on LA cavity segmentation from multi-Center LGE MRIs.

Training

The segmentation models were trained for 1000 epochs in a 5-fold cross-validation scheme. Stochastic gradient descent (SGD) with Nesterov momentum (µ = 0.99) with an initial learning rate of 0.01 was used to optimize the network's weights. The learning rate was decayed using the "poly" learning rate policy [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF]. We employed a mini-batch size of 2. We used a value of 1 for epsilon (ϵ) in PolyLoss (Eq. 2) and a value of 2 for gamma (γ) in Focal loss (Eq. 3). For the uncertainty loss, the weighting factor (α) (in Eq. 8) is empirically selected to be 2.0. For histogram matching, we utilized Simple ITK's python library [START_REF] Lowekamp | The design of simpleitk[END_REF]. The training was done on NVIDIA GPUs using Pytorch deep learning framework based on nnU-Net implementation [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF].

Results and Discussion

To evaluate LA cavity segmentation, Dice coefficient, average surface distance (ASD) and Hausdorff distance (HD) metrics were used. For LA scar segmentation and quantification, accuracy, specificity, sensitivity, Dice coefficient of the scar, and generalized Dice score of the cavity and scar were used [START_REF] Li | Atrialjsqnet: A new framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information[END_REF]. All the comparisons were done on the validation set provided by the challenge. The validation dataset for LA scar segmentation consists of 10 cases from center 1, the same center as the training dataset. For LA cavity segmentation which focuses on a multi-center problem, the validation dataset contains 10 cases from center 1, the same center as the training dataset, and 10 cases from center 2.

The baseline method is the standard nnU-Net network [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF] with Dropout layers added at the middle layers of the segmentation network as mentioned in Section 3.1. It uses light data augmentation that includes rotation, scaling, Gaussian blur and noise. In terms of the loss function, the baseline method employs a hybrid loss of Dice loss with cross-entropy Loss (DiceCE).

Regarding the data augmentation, we separated the experiments into light data augmentation (baseline), moderate data augmentation and histogram matching augmentation. The same network architecture was used during the comparison. The moderate data augmentation uses elastic deformation, rotation, scaling, mirroring, additive brightness, Gaussian noise and blurring. For histogram matching (HM) augmentation, the matched images were added to the training dataset as mentioned in Section 3.3.

Comparing the data augmentation experiments' performance in Table 1, it can be observed that moderate data augmentation improved the segmentation performance from 17.1836mm to 16.8721mm in terms of HD. However, it yielded a bit worse result in both Dice and ASD compared the baseline (light data augmentation). Similarly, the histogram matching-based data augmentation significantly decreased the HD from 17.1836mm to 16.6851mm. However, its performance was slightly lower in terms of Dice score and ASD.

Comparing the performance of the loss functions on the segmentation of LA cavity, the proposed loss outperformed the other loss functions as shown in Table 1. The baseline (DiceCE loss) yielded a Dice score of 0.8884, ASD of 1.74629 and HD of 17.18363 mm whereas DiceFocal loss achieved a Dice score of 0.8885, ASD of 1.7474 and HD of 17.2035 mm. Using only the polynomial version of cross-entropy loss (PolyCE) enhanced the segmentation result mainly in terms of Dice score and HD. When PolyCE is combined with Dice loss, the segmentation result of LA cavity was improved further from 0.8884 to 0.8897, from 1.7463 to 1.7203, and from 17.1836 to 16.9067 mm in terms of Dice score, ASD and HD respectively compared to the baseline which uses DiceCE loss.

Table 2 shows the comparison of the different loss functions on LA scar segmentation. Compared to LA cavity segmentation, it has imbalanced classes because the scar is very small compared to the cavity. Due to this, we have compared the proposed loss not only to the baseline but also to other loss functions which are commonly used for imbalanced segmentation. For example, hybrid loss functions such as DiceFocal loss [START_REF] Zhu | Anatomynet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy[END_REF], and DiceTopK loss [START_REF] Brugnara | Automated volumetric assessment with artificial neural networks might enable a more accurate assessment of disease burden in patients with multiple sclerosis[END_REF] which combines Dice loss with Focal loss and TopK loss respectively to mitigate class imbalance [START_REF] Ma | Loss odyssey in medical image segmentation[END_REF]. In the comparison, we have also included Focal loss [START_REF] Lin | Focal loss for dense object detection[END_REF], a loss function that was designed to deal with foreground-background class imbalance by focusing more on the hard examples. As shown in Table 2, the baseline, which combines Dice loss with crossentropy loss (DiceCE) [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF], yielded an accuracy of 0.7764, sensitivity of 0.5529, Dice score of 0.6258 and generalized Dice score 0.9187 for scar segmentation. The DicePolyCE loss enhanced the performance of baseline as it increased the accuracy, sensitivity, Dice and generalized Dice of scar by 22%, 1%, 0.5%, 0.01%, respectively. DiceTopK loss [START_REF] Brugnara | Automated volumetric assessment with artificial neural networks might enable a more accurate assessment of disease burden in patients with multiple sclerosis[END_REF] achieved an accuracy of 0.7751, sensitivity of 0.5503, Dice score of 0.6222 and generalized Dice score 0.9183 which is lower than the baseline. Using only Focal loss [START_REF] Lin | Focal loss for dense object detection[END_REF] achieved the worst result as can be seen in Table 2. The other commonly used loss function for an imbalanced dataset that is DiceFocal loss [START_REF] Zhu | Anatomynet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy[END_REF] yielded much better result compared to the baseline, DicePolyCE and DiceTopK loss with an accuracy of 0.9999, sensitivity of 0.5749, Dice score of 0.6363 and generalized Dice score 0.9199. The proposed loss, where uncertainty loss is combined with DiceFocal loss achieved the best result outperforming the other loss functions. In terms of specificity, all the loss functions achieved a similar score of 0.9999.

From the results, we observed that a compound loss that utilizes Dice loss with the polynomial version of cross-entropy loss (DicePolyCE) consistently improves the performance of the most common compound loss that combines Dice loss with cross-entropy loss. The performance enhancement was in both the mildly imbalanced LA cavity segmentation and the highly imbalanced LA scar segmentation. This shows the robustness of the proposed loss in LA cavity and scar segmentation.

In LA scar segmentation, the second proposed loss function which utilizes uncertainty information outperformed the commonly used loss functions for highly imbalanced segmentation such as DiceTopK [START_REF] Brugnara | Automated volumetric assessment with artificial neural networks might enable a more accurate assessment of disease burden in patients with multiple sclerosis[END_REF] and Focal loss [START_REF] Lin | Focal loss for dense object detection[END_REF] functions [START_REF] Ma | Loss odyssey in medical image segmentation[END_REF], DiceFocal loss [START_REF] Zhu | Anatomynet: Deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy[END_REF]. This confirms the importance of incorporating uncertainty information as part of the learning process to enhance particularly the segmentation of pathologies with irregular structures like scars. In terms of data augmentation, the experiments were mainly focused on the multi-center LA cavity segmentation. From the results, we can say that using moderate data augmentation and histogram matching can enhance the model's generalization as it improved the segmentation result, particularly in terms of HD compared to the light data augmentation.

Conclusion

In this paper, we proposed a fully automatic deep learning method that utilizes a novel hybrid loss function that combines Dice loss with a polynomial version of cross-entropy loss to segment LA cavity from multi-center LGE MRIs and an uncertainty-based loss function to segment scar from single-center LGE MRIs. We also employed various data augmentation techniques, which include histogram matching, to increase the size and variety of the training dataset. In the experiments, we have compared the proposed loss function with the commonly used losses in the multi-center LA cavity segmentation and in the highly imbalanced LA scar segmentation. We observe that the proposed losses yield the best result outperforming the other losses in both LA cavity and scar segmentation. From the results, we can say that using the polynomial version of cross-entropy in combination with Dice loss can be a better alternative loss function for anatomical segmentation such as LA cavity. For segmentation such as LA scar, which generates high epistemic uncertainty due to its small and complex structure, utilizing a loss function that incorporates uncertainty information can be useful for robust segmentation. Additionally, applying moderate-level data augmentation with histogram matching can improve the results and increase the model's generalization capability.

Fig. 1 .

 1 Fig. 1. Overview of the network architecture.

Table 1 .

 1 Comparison of LA cavity segmentation performance using various data augmentations and compound loss functions on validation set (n = 20) of the challenge. Dice: Dice score, ASD: average surface distance, HD: Hausdorff distance. The bold values are the best.

	Method	Dice ASD HD (mm)
	Baseline	0.8884 1.7463 17.1836
	Moderate DataAug	0.8868 1.7755 16.8721
	HM DataAug	0.8867 1.7536 16.6851
	DiceFocal	0.8885 1.7474 17.2035
	OnlyPolyCE	0.8893 1.7413 17.0053
	Proposed (DicePolyCE) 0.8897 1.7203 16.9067

Table 2 .

 2 Comparison of LA scar segmentation performance using different compound loss functions on validation set (n = 10) of the challenge. GDice: generalized Dice score of cavity and scar. The bold values are the best.

	Method	Accuracy Specificity Sensitivity Dice GDice
	Baseline	0.7764	0.9999	0.5529	0.6258 0.9187
	DiceTopK Loss	0.7751	0.9999	0.5503	0.6222 0.9183
	DiceFocal Loss	0.9999	0.9999	0.5749	0.6363 0.9199
	Focal Loss	0.9999	0.9999	0.5095	0.6047 0.9139
	DicePolyCE Loss	0.9999	0.9999	0.5605	0.6301 0.9187
	Proposed (Uncertainty+ DiceFocal Loss)	0.9999	0.9999	0.5853	0.6406 0.9205

https://zmic.fudan.edu.cn/lascarqs22

Acknowledgements

This work was supported by the French National Research Agency (ANR), with reference ANR-19-CE45-0001-01-ACCECIT. Calculations were performed using HPC resources from DNUM CCUB (Centre de Calcul de l'Université de Bourgogne) and from GENCI-IDRIS (Grant 2022-AD011013506). We also thank the Mesocentre of Franche-Comté for the computing facilities.


Tewodros Weldebirhan

Stéphanie Bricq

Fabrice Meriaudeau

Automatic Quality Assessment of Cardiac MR Images with Motion Artefacts using Multi-task Learning and K-Space Motion Artefact Augmentation

Keywords: Cardiac MRI, Multi-task Learning, Quality Control, Aleatoric Uncertainty, Segmentation, Deep Learning, Motion Artefact

The movement of patients and respiratory motion during MRI acquisition produce image artefacts that reduce the image quality and its diagnostic value. Quality assessment of the images is essential to minimize segmentation errors and avoid wrong clinical decisions in the downstream tasks. In this paper, we propose automatic multi-task learning (MTL) based classification model to detect cardiac MR images with different levels of motion artefact. We also develop an automatic segmentation model that leverages k-space based motion artefact augmentation (MAA) and a novel compound loss that utilizes Dice loss with a polynomial version of cross-entropy loss (PolyLoss) to robustly segment cardiac structures from cardiac MRIs with respiratory motion artefacts. We evaluate the proposed method on Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion 2022) challenge dataset. For the detection task, the multi-task learning based model that simultaneously learns both image artefact prediction and breath-hold type prediction achieved significantly better results compared to the single-task model, showing the benefits of MTL. In addition, we utilized test-time augmentation (TTA) to enhance the classification accuracy and study aleatoric uncertainty of the images. Using TTA further improved the classification result as it achieved an accuracy of 0.65 and Cohen's kappa of 0.413. From the estimated aleatoric uncertainty, we observe that images with higher aleatoric uncertainty are more difficult to classify than the ones with lower uncertainty. For the segmentation task, the k-space based MAA enhanced the segmentation accuracy of the baseline model.

From the results, we also observe that using a hybrid loss of Dice and PolyLoss can be advantageous to robustly segment cardiac MRIs with motion artefact, leading to a mean Dice of 0.9204, 0.8315, and 0.8906 and mean HD95 of 8.09 mm, 3.60 mm and 6.07 mm for LV, MYO and RV respectively on the official validation set. On the test set, the proposed segmentation method was ranked in second place in the segmentation task of CMRxMotion 2022 challenge.

Introduction

Cardiovascular diseases (CVDs) are the number one cause of death globally. More people die annually from CVDs than from any other causes [START_REF] Organization | Cardiovascular diseases (cvds)[END_REF]. Advanced medical imaging techniques are employed in clinical practice for the diagnosis and prognosis of cardiac diseases. Cardiac magnetic resonance (CMR) is a set of magnetic resonance imaging used to provide anatomical and functional information about the heart.

Deep learning-based methods have recently achieved promising results in cardiac structures (left ventricular blood pool, myocardium, and right ventricular blood pool) segmentation from CMR images [START_REF] Bernard | Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved[END_REF][START_REF] Campello | Multi-centre, multi-vendor and multidisease cardiac segmentation: The m&ms challenge[END_REF]. As part of the ACDC challenge [START_REF] Bernard | Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved[END_REF], Baumgartner et al. [START_REF] Baumgartner | An exploration of 2d and 3d deep learning techniques for cardiac mr image segmentation[END_REF] used a 2D U-Net with a cross-entropy loss to segment the cardiac structures from CMR images. Isensee et al. [START_REF] Isensee | Automatic cardiac disease assessment on cine-mri via time-series segmentation and domain specific features[END_REF] implemented an ensemble of 2D and 3D U-Net segmentation models with Dice loss, and Khened et al. [START_REF] Khened | Densely connected fully convolutional network for short-axis cardiac cine mr image segmentation and heart diagnosis using random forest[END_REF] proposed 2D Dense U-Net with inception module to segment the cardiac structures. Li et al. [START_REF] Li | Apcp-net: Aggregated parallel cross-scale pyramid network for cmr segmentation[END_REF] utilized a two-stage FCNs method that first localizes the heart region as a region of interest (ROI) and then segments the left ventricular blood pool, myocardium and right ventricular blood pool from the localized region of the CMR image. In addition, [START_REF] Full | Studying robustness of semantic segmentation under domain shift in cardiac mri[END_REF][START_REF] Ma | Histogram matching augmentation for domain adaptation with application to multi-centre, multi-vendor and multi-disease cardiac image segmentation[END_REF] utilized data augmentation-based solutions, including histogram matching, contrast modification, and image synthesis to tackle domain shift or distribution shift in CMR images segmentation as part of M&Ms challenge [START_REF] Campello | Multi-centre, multi-vendor and multidisease cardiac segmentation: The m&ms challenge[END_REF]. However, in clinical settings, the CMR images are prone to motion artefacts due to respiratory motion or patient movement during CMR acquisition. This degrades the CMR image quality, which can challenge the diagnostic value of the image and can lead to incorrect analysis.

To detect and reduce motion artefact in Cardiac MR images, Lorch et al. (2017) [START_REF] Lorch | Automated detection of motion artefacts in mr imaging using decision forests[END_REF] introduced a random forest-based method that used box, line, histogram, and texture features as inputs. However, only artificially created motion artifacts were used to evaluate their approach. Oksuz et al. (2018) [START_REF] Öksüz | Deep learning using k-space based data augmentation for automated cardiac mr motion artefact detection[END_REF] proposed a deep learning based method to detect cardiac MR motion artefacts. They utilized 3D spatio-temporal CNNs to identify motion artefacts in 2D+time short axis CMR sequences. To increase their training dataset size, they employed kspace based data augmentation. [START_REF] Lyu | Cine cardiac mri motion artifact reduction using a recurrent neural network[END_REF] [START_REF] Lyu | Cine cardiac mri motion artifact reduction using a recurrent neural network[END_REF] utilized a recurrent generative adversarial network for cardiac MR motion artefact reduction. [START_REF] Oksuz | Deep learning-based detection and correction of cardiac mr motion artefacts during reconstruction for high-quality segmentation[END_REF] [START_REF] Oksuz | Deep learning-based detection and correction of cardiac mr motion artefacts during reconstruction for high-quality segmentation[END_REF] presented a deep learning framework that jointly detects, corrects, and segments CMR images with motion artefacts.

In this paper, we proposed a fully automatic classification network to detect cardiac MRIs with motion artefact. The cardiac images were acquired using different breath-hold instructions. Noticing that there might be a contextual similarity between predicting the level of motion artefact and breath-hold type (as some breath-hold types may cause more motion artefact than others), we propose a multi-task learning (MTL) based classification network to learn better features for the main task and reduce overfitting through shared representations.

To robustly segment cardiac structures from cardiac MRIs with respiratory motion artefact, we propose a fully automatic deep learning segmentation model that benefits from k-space based motion artefact data augmentation to increase training appearance variability and improve the generalization. We evaluated our method on Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion 2022) challenge dataset. For the classification task, the MTL-based method achieved better results compared to the single-task method, showing the advantage of sharing features between related tasks. For the segmentation task, the motion artefact augmentation and compound loss of Dice and polynomial loss enhanced the segmentation performance of the baseline method.

Dataset

The Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CM-RxMotion 2022)1 consists of 360 cine CMR images which have extreme cases with different levels of respiratory motions. The images were acquired using Siemens 3T MRI scanner (MAGNETOM Vida). To acquire the images, scan parameters with a spatial resolution of 2.0 × 2.0 mm 2 , slice thickness of 8.0mm, and slice gap 4.0mm were used. In this challenge, the standard of procedure (SOP) mandates the 45 healthy volunteers to follow specific breath-holds guidelines. Each of the volunteers undergoes a 4-stage scan over the course of a single visit: 1) adhere to the breath-hold instructions; 2) halve the breath-hold period; 3) breathe freely; 4) breathe intensively. For each scan, short-axis CMR acquired at End-Systolic (ES) and End-Diastolic (ED) time frames are provided. From the total 360 (45 volunteers × 4 scans × 2 frames) short-axis CMR images, 160 of them were used for training, 40 for validation, and 160 for test.

The challenge has two tasks. The first one focuses on the quality assessment of the CMR images under respiratory motion artefacts. The quality of the images is labeled by radiologists using a standard 5-point Likert scale. Based on these scores, three levels of motion artefacts were defined: mild motion artefacts (label 1), intermediate motion artefacts (label 2) and severe motion artefacts (label 3). The second task focuses on the segmentation of cardiac structures from CMR images. All images with diagnostic quality are segmented by an experienced radiologist, including contours for left ventricular blood pool (LV), myocardium (MYO) and right ventricular (RV) blood pool.

Methods

Cardiac Image Quality Classification

Multi-task learning aims to solve multiple tasks simultaneously by taking advantage of the commonalities and differences across these tasks. Compared to training the models separately, it can lead to improved regularization, learning efficiency, and prediction accuracy for the task-specific models [START_REF] Caruana | Multitask learning[END_REF][START_REF] Chen | Multi-task learning for left atrial segmentation on ge-mri[END_REF]. To leverage the advantages of multi-task learning, we proposed a multi-task deep learning based classification network to classify the cardiac image quality based on their respiratory motion artefact level. The proposed method is designed to simultaneously conduct two classification tasks: image quality prediction (main task) and patient's breath-hold type prediction (auxiliary task). The main task has three classes: mild motion artefacts (class 1), intermediate motion artefacts (class 2) and severe motion artefacts (class 3). The auxiliary task focuses on predicting the type of breath-hold the patient followed during the CMR acquisition. It has four classes: adhere to the breath-hold instructions (class 1), halve the breathhold period (class 2), breathe freely (class 3), and breathe intensively (class 4). The loss function of the multi-task model is computed as follows:

L T otal = L M ainT ask + βL AuxT ask (1)
where L M ainT ask is a weighted categorical cross-entropy loss for the main classification task, L AuxT ask is a categorical cross-entropy loss for the auxiliary task and β is a hyper-parameter value that controls the contribution of L AuxT ask to the total loss (Eq. 1).

For the network architecture, we modified the 3D ResNet-182 architecture by adding a second classification branch, as shown in Fig. 1. More specifically, after the global average pooling of ResNet-18, we added a fully connected layer (512) with ReLU activation then, it is followed by two classification branches for the two tasks. During pre-processing, all the volumes were resampled to 0.664mm × 0.664mm × 9.60mm and the intensity of every volume was normalized to have zero-mean and unit-variance. In order to improve the robustness of the model, we utilized data augmentation such as random cropping (with size of (300, 300, 8)) , random rotation (degrees = (-15, 15) with probability of 0.5), random flipping (horizontal and vertical flipping with probability of 0.5) and random scaling (range (0.9, 1.2) with probability of 0.5). To enhance the results further and study the data-dependent (aleatoric) uncertainty [START_REF] Ayhan | Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks[END_REF][START_REF] Wang | Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks[END_REF], we employed test-time augmentation (TTA). For TTA, we used M different types of data augmentations during testing. The augmentations utilized for TTA include horizontal flipping, vertical flipping, random rotation (degrees = (-10, 10)) and random scaling (range = (0.75, 1.25)). Then the mean prediction of the M augmented images is used as the final prediction, and the variance or entropy of these predictions is considered as the uncertainty measure.

Cardiac Segmentation

For the Cardiac MRI segmentation, we employed a 3D segmentation network which is based on 3D nnU-Net framework [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF]. The U-Net's encoder and decoder consist of 12 convolutional layers where each convolution is followed by instance normalization and Leaky ReLU (negative slope of 0.01) activation function.

Dice loss (L Dice ) is a region-based loss that directly optimizes the Dice coefficient metric. PolyLoss [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF] redesigns loss functions as a linear combination [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF] came up with a simplified version of the polynomial loss, which is called Poly-1. As can be observed in Eq. 2, this loss function modifies the cross-entropy (L CE ) by just adding one hyper-parameter (ϵ) [START_REF] Leng | Polyloss: A polynomial expansion perspective of classification loss functions[END_REF], where p is the prediction probability of the target class. In this work, we utilized a hybrid loss of Dice loss and a Polynomial version of cross-entropy loss (L DiceP olyCE ) to segment the heart structures from cardiac MRIs, as shown Eq. 3.

L P olyCE = L CE + ϵ(1 -p) (2)
L DiceP olyCE = L Dice + L P olyCE (3)
To enhance the robustness of the segmentation model in the cardiac MRI dataset with motion artefact, we utilized k-space motion artefact data augmentation in addition to intensity-based data augmentation and spatial data augmentation. We applied brightness, contrast, gamma, Gaussian noise, and blur as an intensity-based augmentation. Elastic deformation, random rotation, random scaling, random flipping, and low resolution simulation are employed as spatial augmentations. During MR image acquisition, signals from the MRI scanner are temporarily stored in the k-space. The inverse Fourier transform of the k-space is then computed to provide the MR image. To generate k-space motion artefact from the cardiac MRIs, the k-space is filled with random rigidly modified versions of the original image. Then the inverse transform of the compound k-space is computed [START_REF] Arega | Using mri-specific data augmentation to enhance the segmentation of right ventricle in multi-disease, multicenter and multi-view cardiac mri[END_REF][START_REF] Pérez-García | Torchio: A python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning[END_REF]. The k-space motion artefact augmentations were generated using TorchIO library3 . We applied the k-space motion artefact on randomly selected good-quality cardiac MR training images. The generated images were then added to the training dataset to increase the variety of the training images.

Training

The segmentation models were trained for 1000 epochs in a five-fold crossvalidation scheme. The weights of the network were optimized using Stochastic gradient descent (SGD) with Nesterov momentum (µ = 0.99) with an initial learning rate of 0.01 [START_REF] Isensee | nnU-Net: a selfconfiguring method for deep learning-based biomedical image segmentation[END_REF]. The learning rate was decayed using the "poly" learning rate policy. The mini-batch size was 2 for the segmentation model. For the classification task, the dataset was shuffled and randomly split into 65% training, and 35% validation. The models were trained for 300 epochs with an ADAM optimizer and a mini-batch size of 32. We used a value of 1 for epsilon (ϵ) in the PolyLoss (Eq. 2). For the multi-task loss, the weighting factor (β) (in Eq. 1) is empirically chosen to be 0.4. To compute the data-dependent uncertainty, we used a value of 5 for M (number of data augmentations). All the training was done on NVIDIA Tesla V100 GPUs using Pytorch deep learning framework.

Results and Discussion

For the evaluation of segmentation results, Dice coefficient and Hausdorff distance 95% percentile (HD95) metrics are employed. To assess the classification results, accuracy and Cohen's kappa metrics are utilized.

As can be seen in Table 1, the proposed multi-task network yielded a significantly better result compared to the single-task network. The performance increase was almost by 10% and 15% in terms of accuracy and Cohen's kappa, respectively. This shows that simultaneously learning both tasks can help the network learn better representations for the main task. Applying TTA to the multi-task model further improved the results, as shown in Table 1. When we ensemble the results of the models which use different loss functions with the results of single-task and multi-task models, it yielded an accuracy of 0.7 and Cohen's kappa of 0.507.

TTA can also be used to estimate heteroscedastic aleatoric uncertainty of images [START_REF] Ayhan | Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks[END_REF]. Comparing the aleatoric uncertainties on our own validation split (n = 56) in Fig. 2, one can observe that the input-dependent uncertainty (variance) of the incorrect predictions (0.073) is higher than the correct predictions (0.060). This indicates that images with higher aleatoric uncertainty are harder for the model to classify than images with lower aleatoric uncertainty. For the segmentation task, the baseline method is the default nnUNet network. It uses light data augmentation and a compound loss that combines Dice loss with cross-entropy Loss (DiceCE). We compared the performance of the proposed DicePolyCE loss with two common segmentation losses: DiceCE loss (baseline) and DiceFocal loss [START_REF] Ma | Loss odyssey in medical image segmentation[END_REF] (a hybrid loss that combines Dice loss with Focal loss). Regarding data augmentations, we divided the experiments into light data augmentation (baseline), moderate motion artefact augmentation (moderate MAA) and heavy motion artefact augmentation (heavy MAA). The light data augmentation uses elastic deformation (α : (0., 200.), σ : (9., 13.), random rotation (-30, +30 degrees), scaling (0.85, 1.25), mirroring, brightness (µ : 0, σ : 0.1), and gamma (0.7, 1.5). For moderate MAA, we added ten images with artificial motion artefact (number of transforms = 2). For heavy MAA, we increased the number of images to 30 and the number of simulated movements to 6 (number of transforms = 6), where larger values generate more distorted images. Some visual examples of motion artefact augmented images are shown in Fig. 3.

Comparing the performance of the data augmentation types, using a moderate MAA achieved the best result in terms of Dice and HD95 of the cardiac structures compared to the baseline and heavy MAA, as can be seen from Table 2. This shows the advantage of adding a moderate number of artificially generated motion artefacts into the training images to improve the model's generalizability on a dataset with respiratory motion artefacts. While using heavy MAA, which generates unrealistic motion artefacts that are more distorted from the original training images, leads to poor segmentation performance. Regarding the performance of the different hybrid losses, DiceFocal loss increased the segmentation accuracy of the baseline (DiceCE). Utilizing the proposed Dice-PolyCE loss outperformed the other two loss functions by achieving Dice scores of 0.9204, 0.8315, and 0.8906 and HD95 of 8.09 mm, 3.60 mm and 6.07 mm for LV, MYO and RV respectively (Table 2). This shows the robustness of the proposed loss that combines region-based Dice loss with a polynomial version of cross-entropy on cardiac MR segmentation with motion artefacts. Since the challenge allows the usage of an external public dataset, we added ACDC dataset [START_REF] Bernard | Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved[END_REF] to CMRxMotion dataset to diversify the input images and improve the generalization of the segmentation model. For the ACDC dataset, as a preprocessing step, we resampled the images to the median voxel spacing of CMRxMotion and applied moderate MAA. As shown in Table 2, the addition of ACDC (CMRx+ACDC) improved the result of the baseline. When we changed the loss function from DiceCE to the proposed loss (DicePolyCE), it enhanced the segmentation performance further and achieved the best result with Dice of 0.9236, 0.8341, and 0.8967 and HD95 (in mm) of 8.31, 3.46 and 5.86 for LV, MYO and RV respectively.

Conclusion

In this paper, we proposed fully automatic deep learning methods to detect motion artefact levels and segment cardiac structures from cardiac MRIs with respiratory motion artefact. For the classification task, the proposed network simultaneously predicts both the motion artefact level (main task) and breathhold type to leverage the benefits of multi-task learning. From the results, we observed that training a multi-task network encourages the network to extract useful features for both tasks and results in better prediction performance. The proposed method greatly increased the classification result in terms of accuracy and Cohen's kappa compared to the single-task model. Applying test-time augmentation also helped the model to improve the classification result further and to estimate the aleatoric uncertainty of the data. In the segmentation task, we showed that moderate k-space based motion artefact augmentation and hybrid loss of Dice and polynomial loss can enhance the segmentation performance and improve the model's generalization capability on cardiac MRIs with motion artefacts.

Acknowledgements

This work was supported by the French National Research Agency (ANR), with reference ANR-19-CE45-0001-01-ACCECIT. Calculations were performed using HPC resources from DNUM CCUB (Centre de Calcul de l'Université de Bourgogne) and from GENCI-IDRIS (Grant 2022-AD011013506). We also thank the Mesocentre of Franche-Comté for the computing facilities.

Fig. 1 .

 1 Fig. 1. Overview of the multi-task classification network

Table 1 .Fig. 2 .

 12 Fig. 2. Comparing the aleatoric uncertainty of correctly and incorrectly predicted images on our local validation split

Fig. 3 .

 3 Fig. 3. Visual appearance of a) original CMR image b) moderate motion artefact augmented CMR image c) heavy motion artefact augmented CMR image

Table 2 .

 2 Comparison of cardiac MR segmentation performances using various data augmentations and compound loss functions on validation set (n = 40) of the challenge. Dice: Dice score, HD95: Hausdorff distance 95% percentile in mm, moderate MAA: moderate motion artefact augmentation, heavy MAA: heavy motion artefact augmentation. The bold values are the best.Asterisk * indicates that the dataset used is a mixture of CMRxMotion and ACDC datasets.

	Method	Dice LV Dice MYO Dice RV	HD95 LV	HD95 MYO	HD95 RV
	Baseline	0.9147	0.8298	0.8886	8.64	3.74	6.11
	Moderate MAA	0.9168	0.8306	0.8918	8.49	3.70	6.04
	Heavy MAA	0.9141	0.8298	0.8927	8.45	3.69	5.83
	DiceFocal Loss	0.9181	0.8303	0.8927	8.24	3.67	5.62
	DicePolyCE Loss	0.9204	0.8315	0.8906	8.09 3.60	6.07
	Baseline + ACDC * (CMRx+ACDC)	0.9196	0.8325	0.8961	8.39	3.64	5.57
	CMRx+ACDC * with DicePolyCE Loss	0.9236 0.8341	0.8967 8.31	3.46 5.86

http://cmr.miccai.cloud/

https://github.com/Project-MONAI

https://github.com/fepegar/torchio