domain: info sdv

text:
Salomé Papereux

Ludovic Leconte

Cesar Augusto Valades-Cruz

Tianyan Liu

Julien Dumont

Zhixing Chen

Jean Salamero

Charles Kervrann

Anaïs Badoual
email: anais.badoual@inria.fr

DeepCristae, a CNN for the restoration of mitochondria cristae in live microscopy images

Keywords: Convolution neural network, image restoration, mitochondria cristae, fluorescence microscopy, super resolution, live imaging

Mitochondria play an essential role in the life cycle of eukaryotic cells. However, we still don't know how their ultrastructure, like the cristae of the inner membrane, dynamically evolves to regulate these fundamental functions, in response to external conditions or during interaction with other cell components. Although high-resolution fluorescent microscopy coupled with recently developed innovative probes can reveal this structural organization, their long-term, fast and live 3D imaging remains challenging. To address this problem, we have developed a convolutional neural network (CNN), called DeepCristae, to restore mitochondrial cristae in low spatial resolution microscopy images. Our CNN is trained from 2D STED images using a novel loss specifically designed for cristae restoration. Random sampling centered on mitochondrial areas was also developed to improve training efficiency. Quantitative assessments were carried out using metrics we derived to give a meaningful measure of cristae restoration. Depending on the conditions of use indicated, DeepCristae works well on broad microscopy modalities (STED, Live-SR, AiryScan and LLSM). It is ultimately applied in the context of mitochondrial network dynamics during interaction with endo/lysosomes membranes.

Introduction

The study of certain pathologies has shown the importance of mitochondria, which above all, ensure ATP production within cells and are central in many biological functions (e.g., metabolic pathways, ion homeostasis, apoptosis, autophagy, epigenetics…) [START_REF] Bock | Mitochondria as multifaceted regulators of cell death[END_REF][START_REF] Stimpfel | New Challenge: Mitochondrial Epigenetics?[END_REF] . Mitochondrial energetic adaptations to environmental constraints encompass a plethora of processes that maintain cell survival. An alteration of these processes generally leads to serious diseases such as cancer, neurodegenerative and cardiovascular disorders [START_REF] Chakrabarty | Mitochondria in health and disease[END_REF] . Although much attention has been paid to the role of mitochondria, the precise niche the organelle plays in cell life and death still remains unclear. The lack of in-depth knowledge about the ultrastructural evolution of mitochondria in live cells, under normal and stressful conditions, might be one of the blind spots. In particular, the cristae formed by the inner membrane of mitochondria that concentrate ATP production in a defined area, their dynamic behavior, sublocation or density have been poorly related to the various functionalities or dynamic processes (e.g., fusion, fission) that mitochondria undergo. The challenge we address, lies in imaging mitochondria cristae, which measure between 30 and 50 nm wide [START_REF] Stephan | Live-cell STED nanoscopy of mitochondrial cristae[END_REF] , at a high spatial and temporal resolution so that their structural dynamics and interactions can be accurately studied over time for several dozens of milliseconds to a few seconds. However, this is starting to be possible with the recent development of high-resolution imaging approaches [START_REF] Jakobs | Light Microscopy of Mitochondria at the Nanoscale[END_REF] .

Stimulated emission depletion (STED) microscopy, which allows for sub-diffraction resolution (xy: 30-50 nm), is one of the very few techniques [START_REF] Ishigaki | STED super-resolution imaging of mitochondria labeled with TMRM in living cells[END_REF][START_REF] Yang | Cyclooctatetraene-conjugated cyanine mitochondrial probes minimize phototoxicity in fluorescence and nanoscopic imaging[END_REF] able to decipher dynamics of mitochondria cristae in live cells [START_REF] Stephan | Live-cell STED nanoscopy of mitochondrial cristae[END_REF] . However, their observation in 3D and in fast time is limited by the acquisition frame rate capacity (1 plane ≈1 to 10 s). In addition, depletion STED, which is the principle that achieves nanoscopic resolution, induces local heat by high illumination intensity [START_REF] Vicidomini | STED super-resolved microscopy[END_REF] to which mitochondria are known to be particularly sensitive [START_REF] Yang | Mitochondrial dynamics quantitatively revealed by STED nanoscopy with an enhanced squaraine variant probe[END_REF][START_REF] Kilian | Assessing photodamage in live-cell STED microscopy[END_REF] . This can affect their overall physiology and potentially lead to apoptosis and mitophagy. A number of new fluorescent probes that are more photostable with less saturation intensity and that allow cristae decoration, have been developed the very last years [START_REF] Yang | Cyclooctatetraene-conjugated cyanine mitochondrial probes minimize phototoxicity in fluorescence and nanoscopic imaging[END_REF][START_REF] Yang | Mitochondrial dynamics quantitatively revealed by STED nanoscopy with an enhanced squaraine variant probe[END_REF][START_REF] Liu | Multi-color live-cell STED nanoscopy of mitochondria with a gentle inner membrane stain[END_REF][START_REF] Wang | A photostable fluorescent marker for the superresolution live imaging of the dynamic structure of the mitochondrial cristae[END_REF] . Yet, the application of a dark recovery step (≈30 s) after STED imaging is still necessary, again at the expense of temporal resolution. This could be improved by applying a partial STED depletion protocol, leading to an intermediate quality resolution (xy≈100 nm) [START_REF] Zhang | Low-Power Two-Color Stimulated Emission Depletion Microscopy for Live Cell Imaging[END_REF] , but insufficient to spatially resolve mitochondria cristae and not solving the frame rate limitation (4-5 s in average).

In this context, one solution to study the dynamics of mitochondria cristae is to collect as much temporal information with minimal phototoxicity using an appropriate microscope, and then restore the spatial dimension using computational methods. Indeed, the development of image restoration algorithms has become increasingly popular in recent years with the need for nanoscale analysis [START_REF] Guo | Rapid image deconvolution and multiview fusion for optical microscopy[END_REF][START_REF] Koho | Fourier ring correlation simplifies image restoration in fluorescence microscopy[END_REF][START_REF] Huang | Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy[END_REF][START_REF] Zhao | Sparse deconvolution improves the resolution of live-cell superresolution fluorescence microscopy[END_REF][START_REF] Arigovindan | High-resolution restoration of 3D structures from widefield images with extreme low signal-to-noise-ratio[END_REF][START_REF] Weigert | Content-aware image restoration: pushing the limits of fluorescence microscopy[END_REF][START_REF] Jin | Deep learning enables structured illumination microscopy with low light levels and enhanced speed[END_REF][START_REF] Qiao | Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes[END_REF][START_REF] Li | Incorporating the image formation process into deep learning improves network performance[END_REF] . At the heart of fluorescence microscopy have been actively developed denoising algorithms [START_REF] Boulanger | Patch-Based Nonlocal Functional for Denoising Fluorescence Microscopy Image Sequences[END_REF][START_REF] Kempen | Comparing maximum likelihood estimation and constrained Tikhonov-Miller restoration[END_REF][START_REF] Van Der Voort | Restoration of confocal images for quantitative image analysis[END_REF][START_REF] Van Kempen | A quantitative comparison of image restoration methods for confocal microscopy[END_REF][START_REF] Buades | A non-local algorithm for image denoising[END_REF][START_REF] Dabov | Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering[END_REF] , dedicated to images corrupted by a mixed Poisson-Gaussian noise, as well as deconvolution algorithms [START_REF] Richardson | Bayesian-based iterative method of image restoration[END_REF][START_REF] Lucy | An iterative technique for the rectification of observed distributions[END_REF][START_REF] Gonzalez | Digital Image Processing[END_REF] , designed to remove the blur induced by the limited aperture of the microscope objective. Some methods combine the two approaches [START_REF] Prigent | SPITFIR(e): a supermaneuverable algorithm for fast denoising and deconvolution of 3D fluorescence microscopy images and videos[END_REF] . However, these conventional restoration methods usually rely on general assumptions, such as the nature and level of noise and spatial regularity, which hampers their effectiveness on the diversity of structures and level of degradation in microscopy images. Over the years, the literature on image restoration has evolved considerably due to deep learning and the rapid growth of convolutional neural networks (CNNs). These methods have the advantage of making assumptions based on image content, resulting in state-of-the-art performance in denoising [START_REF] Krull | Noise2Void -Learning denoising from single noisy images[END_REF][START_REF] Batson | Noise2self: Blind denoising by self-supervision[END_REF] and deblurring [START_REF] Guo | Rapid image deconvolution and multiview fusion for optical microscopy[END_REF][START_REF] Weigert | Content-aware image restoration: pushing the limits of fluorescence microscopy[END_REF][START_REF] Li | Incorporating the image formation process into deep learning improves network performance[END_REF][START_REF] Ledig | Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[END_REF][START_REF] Zhang | Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising[END_REF][START_REF] Zhang | Image Super-Resolution Using Very Deep Residual Channel Attention Networks[END_REF][START_REF] Chen | Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes[END_REF] fluorescence microscopy images. However, these methods have two major drawbacks. First, these CNNs often require a training step based on a large ground truth dataset that is generally not available in microscopy. Second, they focus on restoring the entire image, while sometimes little information is worth restoring within it, especially in the dark background. This is the case with mitochondria cristae, which have a sparse number of pixels in the image compared to the background. Therefore, CNNs that have been previously applied to mitochondria microscopy images [START_REF] Qiao | Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes[END_REF][START_REF] Li | Incorporating the image formation process into deep learning improves network performance[END_REF][START_REF] Chen | Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes[END_REF][START_REF] Chen | Self-supervised denoising for structured illumination microscopy enables long-term super-resolution live-cell imaging[END_REF] provide good global restoration of the background and mitochondria but fail to accurately restore fine details as cristae, especially in very low spatial resolution images. To circumvent this, new conventional methods have been proposed to enhance resolution and suppress artifacts in high-resolution techniques, including Hessian-SIM [START_REF] Zhao | Sparse deconvolution improves the resolution of live-cell superresolution fluorescence microscopy[END_REF] . However, as explained above, the denoising results are limited when dealing with low signal-to-noise ratio images and Hessian deconvolution assumes that the unknown image is smooth in some sense and sparse. A hybrid solution has been proposed in TDV-SIM [START_REF] Wang | Hybrid reconstruction of the physical model with the deep learning that improves structured illumination microscopy[END_REF] , which combines the strengths of conventional physical modelbased algorithms with deep learning-based algorithms. Another hybrid solution, rdLSIM [START_REF] Qiao | Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes[END_REF] , incorporates the deterministic physical model of specific microscopy into network training and inference. Nevertheless, the effectiveness of these methods, along with conventional restoration algorithms, relies on the careful selection of optimal parameters or on prior knowledge of illumination patterns, respectively.

Instead of developing an additional generic image restoration method that may not satisfactorily enhance certain sparse but informative pixels in the image, we present DeepCristae, a CNN specifically developed to restore mitochondrial cristae in low spatial resolution microscopy images. DeepCristae was applied to several microscopy modalities and different biological scenarios capturing live mitochondria at high speed with low illumination and thus low phototoxicity. DeepCristae allows long-term/fast dynamic observation of cristae behavior and organization. The main challenge was to handle the low number of cristae pixels compared to the background in the acquired images. Therefore, the main contributions of this work are 1) the design of a new training loss dedicated to the restoration of specific pixels of interest, 2) a random patch sampling focusing on areas of mitochondria, and 3) the building of metrics for objective assessment of cristae restoration.

Results

Overview of DeepCristae

DeepCristae aims to restore mitochondrial cristae in intermediate to low spatial resolution microscopy images (Fig. 1). DeepCristae mainly consists of a U-Net trained on a dedicated dataset built from real high-resolution 2D STED images (Fig. 1 and Supplementary Table 1) and using a novel training loss we specifically designed for cristae restoration (Methods, Eq. ( 1)). Although the term is not fully appropriate, for simplification we refer to this dataset as "synthetic" 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 . A pipeline for random patch sampling focusing on regions of mitochondria in the acquired data was also developed (Supplementary Fig. 1). DeepCristae image restoration network was implemented in Python (TensorFlow version 2.11) and is freely available as an open-source software (see code availability). DeepCristae is also integrated into BioImageIT [START_REF] Prigent | BioImageIT: Open-source framework for integration of image data management with analysis[END_REF] , an open-source platform with existing software for microscopy.

DeepCristae quantitatively outperforms state-of-the-art algorithms on the synthetic dataset 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠

Our method was quantitatively compared to existing both conventional and deep learning algorithms for image restoration. It includes denoising methods (ND-SAFIR [START_REF] Boulanger | Patch-Based Nonlocal Functional for Denoising Fluorescence Microscopy Image Sequences[END_REF] , 2D median filter and Noise2Void [START_REF] Krull | Noise2Void -Learning denoising from single noisy images[END_REF] ), deconvolution algorithms (Richardson-Lucy [START_REF] Richardson | Bayesian-based iterative method of image restoration[END_REF][START_REF] Lucy | An iterative technique for the rectification of observed distributions[END_REF] and Wiener 31 ), and approaches combining both, such as SPITFIR(e) [START_REF] Prigent | SPITFIR(e): a supermaneuverable algorithm for fast denoising and deconvolution of 3D fluorescence microscopy images and videos[END_REF] , CARE [START_REF] Weigert | Content-aware image restoration: pushing the limits of fluorescence microscopy[END_REF] , RCAN [START_REF] Zhang | Image Super-Resolution Using Very Deep Residual Channel Attention Networks[END_REF] and SRResNet [START_REF] Ledig | Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[END_REF] (details are in Supplementary Note 2.2.2). All methods were applied to 9 test images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 (Supplementary Note 1.2.1). Note that in this dataset, the ground truth images were obtained by deconvolving the high-resolution 2D STED images with a Richardson-Lucy (RL) algorithm to enhance mitochondria cristae. Iterative RL algorithm is suitable for signal-dependent noise removal. Nevertheless, after several iterations it may create unwanted artifacts such as a "night sky" pattern in the image. To generate ground truth images from the input 2D STED images, restored intermediate images with little "sky night" pattern, less blurred and noisy than the input images, were carefully selected. All deep learning methods were trained from the same patches extracted from 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 . First, to evaluate the performance of the different methods, we used current metrics, namely NRMSE, PSNR and SSIM (Supplementary Note 2.1). However, these measures are relevant to the image as a whole, but insufficient in the context of mitochondrial cristae restoration. Indeed, the images contain only few pixels of cristae and thus have too little impact in those metrics unlike the many background pixels. To overcome this issue, we encouraged the evaluation metrics to focus exclusively on mitochondria pixels. To solve this problem, evaluation metrics are focused exclusively on mitochondrial pixels. We call these mitochondrial metrics 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 , 𝑃𝑃𝑁𝑁𝑁𝑁𝑁𝑁 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 and 𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 . To go one step beyond and accurately assess cristae restoration, we also introduced the cristae metrics 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 , 𝑃𝑃𝑁𝑁𝑁𝑁𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 and 𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 . These metrics are computed over mitochondria cristae pixels only, obtained from manual annotations (Supplementary Note 2.1). Each competing algorithm was evaluated over nine test images, for the nine aforementioned metrics (Fig. 2b). For all measurements focusing on mitochondria or cristae, DeepCristae ranks first, and is either first or second otherwise. Conventional methods behave worse than deep learning approaches, CARE appearing DeepCristae's most competitive method. However, as confirmed by the values of the metrics 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 , 𝑃𝑃𝑁𝑁𝑁𝑁𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 and 𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 , CARE restores mitochondria cristae with less sharpness, especially for mitochondria with low contrast or in a noisy background (Fig. 2a, CARE white arrows). It should be noted that if we had considered conventional NRMSE, PSNR and SSIM measurements, we would not have been able to demonstrate the benefits of restoring more informative pixels. In terms of visual assessment, SRResNet and RCAN amplify the background noise, resulting in less accurate restoration of cristae and unrealistic reconstructed structures in the background or in mitochondria (Fig. 2a). DeepCristae removes noise background while restoring most of the cristae details.

Second, a test image of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 was restored with DeepCristae (Fig. 3a). We selected four regions of interest (ROIs) to better appreciate the restoration results (Fig. 3b-e). For each ROI, a comparison of normalized intensity profiles between the input image, DeepCristae restored image and the ground truth image is performed. It shows that DeepCristae restores spatial information by revealing mitochondria cristae while improving signal to noise ratio. The line profiles of DeepCristae are sometimes smoother than the ones of the ground truth, especially in areas where the image background is noisier (Fig. 3d,e). This is due to a residual "night sky" effect in the ground truth that does not appear with DeepCristae. This surprising result can be explained in two ways. First, we reinject information from the input into the network at the end and second, our new Similarity Component Prioritization (SCoP) loss (see Methods) especially focuses on the mitochondria and cristae pixels where there are the fewest artifacts with RL.

Robustness and stability of DeepCristae.

We have shown that DeepCristae performs well on 2D STED images and outperforms stateof-the-art algorithms. However, it is important to verify the reliability of DeepCristae more widely. DeepCristae has been trained on a dedicated dataset acquired with specific microscope settings and mitochondria properties (e.g., fluorescence markers, width in pixels of the mitochondria in the images). Any change in these settings is expected to alter the quality of the restoration results. To evaluate the influence of changes in these parameters on the results, three experiments were performed. Our model was trained from the training images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 depicting mitochondria of width 15.64 ± 4.04 pixels on average. We started by studying the quality of the predictions as a function of the mitochondria width in pixels in the input images. To that end, the 9 test images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 were rescaled 11 times in order to contain mitochondria of specific widths (in pixels) on average. It thus results on 11 test datasets on which our trained DeepCristae model was applied (Supplementary Fig. 2a,b) and the metrics (Supplementary Note 2.1) were computed. The evolution of the metrics as a function of average mitochondrial width shows that the closer you get to the training parameters (i.e. an average width of 15.64 pixels), the better the quality of the restorations. In fact, if the mitochondria are too small, few cristae are restored and the mitochondria are thin. On the contrary, if the size is too large, DeepCristae tends to create artifacts looking like cristae patterns. Next, our model was trained on images obtained with specific parameters that mimic microscope settings. Real images are assumed to be corrupted by mixed Poisson-Gaussian noise (with standard deviation 𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 = 4) and the point spread function of the microscope is approximated by an isotropic Gaussian function of standard deviation 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 = 3.25 pixels. We performed two experiments, similar to the one described above, to examine the assessment of DeepCristae image restoration as a function of 𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 and 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 (Supplementary Fig. 3). For both experiments, visual results show that the quality of the restoration decreases as 𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 and 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 increase (Supplementary Fig. 3b,d). The higher the 𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 or 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 values are, the blurriest the mitochondria's boundaries and their cristae. This is also confirmed by the evolution of the metrics as a function of 𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 (Supplementary Fig. 3a) and by the evolution of 𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 and 𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑐𝑐𝑐𝑐𝑚𝑚𝑠𝑠𝑠𝑠𝑐𝑐𝑐𝑐 as a function of 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 (Supplementary Fig. 3c). Surprisingly, the evolution of the PSNR and NRMSE as a function of 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 have a bell-shape with a maximum and a minimum, respectively, for values of 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 close to 3.25 pixels. We thus recommend using DeepCristae on microscopy images with blur and noise levels at worst equal to our training conditions (𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 = 4 and 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 = 3.25 pixels). Beyond this, the quality of the restoration can drastically decrease, especially for blur, and for too high values of these parameters DeepCristae could reveal hallucinated cristae shapes.

Moreover, it is important to guarantee that under well-controlled conditions of use (mitochondria width, image blur and noise), DeepCristae is hallucination-free and stable. By stable we mean that different training leads to consistent predictions. To demonstrate that these requirements have been met, we performed three experiments. First, we trained 10 DeepCristae neural networks with different training data, each generated with our patch generation method applied to the 24 training images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 (Fig. 4g-l). For each training, the resulting model was applied to the 9 test images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 and the aforementioned metrics were computed. The average metrics obtained over the 10 trainings are close to the ones obtained with our model and the standard deviations are very low, showing consistency between predictions (Fig. 4l). By visually analyzing the predictions, the color map of the standard deviation (Fig. 4g) as well as looking at normalized intensity line profiles along mitochondria (Fig. 4h-k), we observe that the 10 trainings agree overall on the presence or absence of cristae but diverge in their intensity and their precise boundaries. In this experiment, all networks were initialized with the same weights, confirming that our patch sampling method is robust and leads to homogeneous learning. A second similar experiment was carried out. Ten trainings were performed from one dataset but with 10 different weights' initializations (Supplementary Fig. 2c-h). This experiment indicates that the same dataset leads to homogeneous learning, meaning that the randomness of initialization does not play a key role in the learning process. Finally, to qualitatively assess the performance of DeepCristae on real data, we acquired 5 pairs of real 2D STED images. Each pair contains one low-resolution (LR) and one high-resolution (HR) STED image, acquired as quickly as possible (~30 s), to minimize the displacements and deformations of mitochondria between the two acquisitions (Supplementary Note 1.2.1). The HR STED images were deconvolved using the RL algorithm to enhance mitochondria cristae and are considered as "ground truths" (GTs). The LR STED images were resized to have an average mitochondrial width of 15.64 pixels (391 nm), in line with the conclusions drawn above, and were then given as input to DeepCristae. The obtained predictions were qualitatively compared to the GTs to control their consistency. Four ROIs, from two of the five pairs of real STED images and corresponding predictions, were selected in regions where small mitochondrial displacements were observed to better appreciate the restoration (Fig. 4a-f). For each ROI, a comparison of normalized intensity profiles between the input LR image, DeepCristae restored image and the ground truth image is performed. A consistency between the cristae restored by DeepCristae and the ones present in the ground truths is observed overall, and no meaningful "hallucination" is observed.

DeepCristae enables to restore 2D+time STED images

While 2D STED nanoscopy enables to resolve mitochondria cristae and was here helpful to develop DeepCristae, live STED acquisition encompasses a number of hurdles. It includes relatively long-time frames between images, even when a photostable probe was used (Fig. 5a,b), limiting the temporal overview of the mitochondrial dynamics in the same plane. Moreover, STED imaging may rapidly induce photo-bleaching, which makes ultrastructural details progressively dimmed. More problematic, repeated STED imaging rapidly induces morphological deterioration of live mitochondria, illustrated by their swelling in the latest time points (Fig. 5a,b). This swelling effect was quantified here for HR STED by measuring the lateral widths of 7 distinct mitochondria over the 10 time points (Fig. 5d) in the image series (Fig. 5a,b). The swelling appears between the 5th and the 7th frames.

In order to improve the frame rate of 2D+time STED imaging while limiting the photodamages on mitochondria, one may adjust the STED imaging protocols (Supplementary Note 1). Accordingly, we built another live STED dataset, first to give an indication on how long, with STED acquisition in lower depletion conditions, we can image before the mitochondria are damaged and second, to control the efficiency of DeepCristae restoration over time. In what follows, these low-resolution (LR) STED images were referred to as Fast STED images. This goes at the depend of xy resolution (Fig. 5c, left bottom triangles in image series) for both the lateral width of mitochondria and the cristae width (Fig. 5d, Fast STED and Fig. 5e, RAW, respectively). Applying DeepCristae restoration on these latest series clearly revealed cristae morphology (Fig. 5c right top triangle in image series). As expected, LR (or Fast) 2D+time STED images show little changes in mitochondria lateral widths in time, in contrast to HR STED (early and late time points in Fig. 5d) but a degraded resolution in the cristae widths (from a mean (𝜇𝜇) of ~90nm in HR STED to ~120 nm in Fast STED with 𝜎𝜎 = ± 47 nm). Applying DeepCristae allows recovery of a resolution lower than 100 nm and drastically reduces the variability of the measurement. The mean crista-to-crista distance, measured as peak-to-peak intervals (Fig. 5f), widely depends on the cristae density along the mitochondria network. Here, in RPE1 cell, it varies from 50 nm to more than 500 nm in early time points in HR STED (𝜇𝜇 =319 nm, 𝜎𝜎 = ± 247 nm) while the heterogeneity increases in late time points (from 130 nm up to 1.6 µm), consistently with the observable swelling of the mitochondria. In Fast STED, the cristae intervals measurements were non-significant. However, after DeepCristae restoration the mean crista-to-crista distance was estimated at ~142 nm (𝜎𝜎 = ± 46 nm) (Fig. 5f,g). Differences in these crista-to-crista measurements with similar studies on HeLa or Cos7 cells for instance, will be further discussed. DeepCristae restored the individual cristae at 81 nm of resolution (𝜎𝜎 = ± 9 nm) at FWHM (Full Width at Half Maximum), as compared to the approximately 50 nm obtained in other studies [START_REF] Stephan | Live-cell STED nanoscopy of mitochondrial cristae[END_REF] . DeepCristae provides a useful way to improve live STED nanoscopy by improving the resolution and decreasing the frame rates (3 to 6s versus 13s), yet with no observable photodamage as illustrated here by measuring the swelling of mitochondria.

DeepCristae restore 3D+time images of mitochondria cristae by using intermediate high-resolution and diffraction limited microscopy

STED nanoscopy is not the only high-resolution microscopy adapted to resolve internal mitochondria ultrastructures in live cells. Indeed, a number of works using adaptation of SIM approaches have been published over the last few years [START_REF] Huang | Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy[END_REF][START_REF] Opstad | Multi-color imaging of submitochondrial structures in living cells using structured illumination microscopy[END_REF] , some combined with conventional deep learning methods [START_REF] Jin | Deep learning enables structured illumination microscopy with low light levels and enhanced speed[END_REF][START_REF] Li | Incorporating the image formation process into deep learning improves network performance[END_REF][START_REF] Chen | Self-supervised denoising for structured illumination microscopy enables long-term super-resolution live-cell imaging[END_REF] . Yet, the best compromise between Fast and 3D imaging still remains an issue. We next investigated the performance of DeepCristae prediction on intermediate HR microscopies chosen for their 3D optical slicing performance. Spinning disk confocal equipped with a Live-SR module (or SDSRM for Spinning Disk Super Resolution Microscopy) is one of those well-disseminated systems equivalent to SIM. It improves the xy resolution by a factor of ~2 (~120-130 nm at 488 nm, ~140 nm at 561 nm) [START_REF] Hayashi | Ultrafast superresolution fluorescence imaging with spinning disk confocal microscope optics[END_REF] while giving access to the depth (z-axis) of the sample and the live imaging of mitochondria (time t) without severe photo-bleaching and phototoxicity. The use of the Live-SR is therefore motivated here by both the study of these four dimensions and the ability of our model to correctly perform cristae reconstruction via multiple microscope imaging modalities. DeepCristae efficiently revealed cristae organization in single 2D Live-SR images acquired within 30 ms (Fig. 6a, upper images) and thus in 3D (Fig. 6a, lower images, MIP on 14 planes, with a stack time ~800 ms), giving access to the overall mitochondria network in the live cell at a fast rate. In this respect, it outperforms HR STED imaging and even Fast (LR) 2D-STED imaging after DeepCristae restoration (compared to Fig. 5c). Cristae width comparative estimation (Fig. 6b) shows the improvement in resolution obtained after DeepCristae restoration on single plane Live-SR images (Raw =149 ± 64 nm when measurable; DeepCristae =87 ± 11 nm). These results are close to the expected widths of circumvoluted cristae tubules (50 to 100 nm) obtained by other methods derived from SIM imaging [START_REF] Li | Incorporating the image formation process into deep learning improves network performance[END_REF] . We then tested DeepCristae restoration on LLSM [START_REF] Chen | Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution[END_REF] imaging which is not an HR microscopy by itself (at least in the dithered mode) but gives the best compromise in terms of fast 3D acquisition with minimal photon dose illumination and consequently low photo-damage of the mitochondria over time. Surprisingly, although with intrinsic limited and non-isotropic resolutions (PSF xy=300 nm and z=600-700 nm in our system) and a particular geometrical acquisition mode, cristae were however detectable in some mitochondria after realignment and a Richardson Lucy (RL) deconvolution. The resulting images were here considered as "Raw" data (Fig. 6c, 2D upper panel, left image). Applying DeepCristae to them (Fig 6c ,2D upper panel, right image and composite zoomed area for comparison) improves the cristae resolution (Raw = 339 ± 248 nm, when measurable; DeepCristae= 94 ± 15 nm) and strongly reduces the variance of paired measurements (Fig. 6d). One of the obvious advantages of LLSM over confocal imaging is to allow continuity between single image planes over large stacks coupled to an extended depth of focus, as illustrated here by the 3D rendering as an oblique projection (Fig. 6c,3D). Moreover, LLSM is particularly adapted to long range/high frequency imaging on whole living cells, which, coupled to low photon dose illumination, makes it one of the best imaging systems, if not the best, for the highly light-sensitive organelles that are the mitochondria. Applying DeepCristae adds information on cristae ultrastructural organization in the whole mitochondria network of the cell. Finally, Fast 3D Live-SR and LLSM time series (Fig 6e,f) were treated for DeepCristae restoration. Cristae ultrastructural features can be observed, while the mitochondrion network undergoes well known dynamic modifications such as fusion or fission processes (Fig. 6e, f, panels of composite zoomed area in both time series; left "RAW '' and right "DeepCristae"; Movies S1 and S2). Images are of better quality after restoration of Live-SR compared to LLSM images. However, it should be noted the gain in acquisition parameters for the latter in these experiments, with 75 slices per stack and a double channel stack time=1.3 s versus 14 planes per stack and double channel stack time= 3.9 s for Live-SR. DeepCristae restoration was also tested with an AiryScan 5 LSM 980. It provided similar improvements, although for a 15 planes stack time of about 30s and with more artifacts appearing after DeepCristae, the nature of which most probably lies in the way the reconstruction of the AiryScan images was carried out from the values determined automatically by the commercial software (Supplementary Fig. 5).

DeepCristae restoration allows to decipher mitochondria cristae morphodynamics during inter organelles interactions

The most documented membrane-membrane interactions involving mitochondria are the endoplasmic reticulum (ER)-mitochondria contacts, whose functions have been continuously expanded since the 1990s [START_REF] Lang | ER-mitochondria contact sites in yeast: beyond the myths of ERMES[END_REF][START_REF] Vance | Phospholipid synthesis in a membrane fraction associated with mitochondria[END_REF] . In addition to the ER, mitochondria contact vacuoles/lysosomes, peroxisomes, lipid droplets, endosomes, the Golgi, the plasma membrane (PM) and melanosomes [START_REF] Lackner | The Expanding and Unexpected Functions of Mitochondria Contact Sites[END_REF] . The number of these interactions as well as their duration drastically vary from one type to the other, as they depend on the respective membrane surface of the specific organelles within the cell and their contact time [START_REF] Valm | Applying systems-level spectral imaging and analysis to reveal the organelle interactome[END_REF] . Their detection may thus require fast and/or long-range 3D imaging. As already mentioned, even high-resolution approaches which are well adapted to decipher ultrastructural features of mitochondria such as cristae, generally fail to capture their dynamic evolution in the 3D space of the whole cell at multiple time scales. This can be critical, if one wants to study interorganelle membrane interactions and their effects. We next initiate the investigation of endosome/lysosome-mitochondria dynamic interactions by addressing specifically the ultrastructural behavior of the cristae during these contacts. This was done by imaging multiple 3D+time double fluorescence series in Live-SR (represented as a single stack MIP in Fig. 7a, top left) or LLSM (represented as a single stack MIP in Fig. 7c, top left), where the membranes of the endo-lysosomal pathway were continuously labeled (Supplementary Note 1.1). DeepCristae restoration was applied on both MIP datasets (respectively, Fig. 7 a,c, bottom left). A number of mitochondria dynamic events correlated with endosomal structure behaviors were captured. Only a few of them are here extracted as thumbnail time series (Fig. 7b,d) of zoomed area (colored insets in Fig. 7a,c) from the Live-SR and LLSM acquisitions, respectively. Among others, the formation of endo-lysosomes contacts sites with mitochondria (Fig. 7b, blue and green), positioning of endo-lysosomes relative to fission sites of mitochondria (Fig. 7b, blue and red), very long confinement of endo-lysosomes within the mitochondria network (Fig. 7b, orange) and image series of endo-lysosomes appearing to pull a small mitochondria from one to another elongated tubules of mitochondria (Fig. 7b,red). DeepCristae restoration on the space-time localization of these events can also be evaluated dynamically (Movie S3). Similar events are followed with LLSM, such as the fission of mitochondria at a contact site with an endo/lysosome vesicle (Fig. 7d, orange) and long confinement of an endo/lysosome vesicle within the mitochondria network (Fig. 7d, green). The main advantage of the LLS modality (fast frame rate, low photon illumination of the sample coupled to whole cell 3D acquisition) is the improvement of the time resolution of the data series (or long-range acquisition). Consequently, fast events involving endo-lysosome contacts with mitochondria are easier to capture and these dynamics are precisely deciphered. For instance, one may extract first (Fig. 7c, blue inset), probably a fusion process (Fig. 7d, from time point 105 to time point 119; ∆𝑡𝑡 = 13 𝑠𝑠), and second a fission process (Fig. 7d, from time point 192 to time point 198; ∆𝑡𝑡 = 6 𝑠𝑠; Movie S4). At each time point, DeepCristae-restored mitochondria and denoised/deconvoluted endo-lysosomes double-labeled images (Supplementary Note 1) are paired to non-treated images (right and left panel, respectively, of thumbnails time series in Fig. 7b,d). While cristae resolution in LLSM does not reach that obtained with Live-SR, DeepCristae restoration brings values closer together (Fig. 6b,d). In all situations and for both intermediate HR (Live-SR) and diffracted limited (LLSM) imaging modalities, DeepCristae restoration provides ultrastructural information on the positioning, density, and dynamics of mitochondria cristae. We then wanted to quantitatively assess how the dynamic architecture of the mitochondria internal membrane during endo/lysosomesmitochondria interaction could be revealed with DeepCristae. We focused on the fission process. To do so, we first selected 21 distinct 3D+time image series from the Live-SR datasets, in which mitochondria fission was monitored. Intensity line plots were measured along mitochondria on some time points framing the fission event. This was done on both DeepCristae-restored and unrestored individual time points in a "blind" manner, meaning without looking in the second channel depicting the location of endo-lysosomes (Supplementary Fig. 6a). Measurements of "peak to peak" intervals between cristae, were only possible in the DeepCristae restored images and show an increased density after fission occurs (Supplementary Fig. 6b, dark circles). Interestingly, by overlaying the second channel in a second step, 60% (13 over 20) of these selected time series showed proximity if not direct contact at the site where mitochondria fission is observed (Supplementary Fig. 6b, red circles; 6c for statistics). While still preliminary and not deciphering the exact nature of the endosomal compartments involved (i.e., PMDR labels the overall endo-lysosomal pathway), this illustrates how DeepCristae would represent an asset to quantitatively study the dynamic architecture of the mitochondria internal membrane during diverse dynamic processes or in particular physiological or constrained conditions.

Discussion

Mitochondrial membrane architecture is essential for the many functions of mitochondria. In particular, mitochondria cristae are the main site of energy production and are dynamic ultrastructures remodeling upon various cellular stimuli and natural processes (apoptosis 1 ; aging [START_REF] Amorim | Mitochondrial and metabolic dysfunction in ageing and age-related diseases[END_REF] ). Therefore, understanding the structure and dynamics of cristae is vital for comprehending mitochondrial function and its implications in cellular physiology and diseases. High-resolution microscopy coupled with robust mitochondrial probes [START_REF] Yang | Cyclooctatetraene-conjugated cyanine mitochondrial probes minimize phototoxicity in fluorescence and nanoscopic imaging[END_REF][START_REF] Yang | Mitochondrial dynamics quantitatively revealed by STED nanoscopy with an enhanced squaraine variant probe[END_REF] are key recent developments that started to reveal the fine details of mitochondrial cristae structure and organization, overcoming the limitations of conventional microscopy. However, imaging at high spatial and temporal resolution remains a challenge.

DeepCristae exploits the power of deep learning to reveal cristae in images taken with low photon illumination, enabling clearer visualization and analysis of mitochondria cristae in living cells without interfering with the natural behavior of mitochondria. While it has been trained on a dedicated dataset that was created from real high-resolution 2D STED images, we have shown that it operates for a wide range of optical resolutions, from diffraction-limited to intermediate high-resolution microscopy, providing researchers with a powerful tool to study cristae dynamics without compromising their structural integrity or functionality.

While there are other deep learning approaches available for revealing cristae ultrastructure [START_REF] Qiao | Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes[END_REF][START_REF] Li | Incorporating the image formation process into deep learning improves network performance[END_REF][START_REF] Chen | Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes[END_REF][START_REF] Chen | Self-supervised denoising for structured illumination microscopy enables long-term super-resolution live-cell imaging[END_REF] , DeepCristae offers unprecedented advantages. First, thanks to a welldefined training loss dedicated to the restoration of mitochondria signals, it outperforms stateof-the-art methods. Secondly, it not only makes it possible to visualize and restore cristae dynamics in 2D STED nanoscopy with minimal illumination and without damaging mitochondria but more importantly, it extends these capabilities to other high-resolution imaging techniques such as Live-SR, SIM and AiryScan, more suited to such 3D dynamics. Finally, DeepCristae can be applied to advanced microscopy techniques such as LLSM, enabling fast and long-duration 3D+time acquisitions within the diffraction-limited range. This versatility makes DeepCristae a unique and valuable solution for studying cristae dynamics across a range of spatial and temporal scales.

Overall, our results show that fluorescence microscopy combined with DeepCristae enables long-term/fast dynamic observation of cristae behavior and organization with high quality. To illustrate the contribution of our approach to biological phenomena that are likely to involve the functional structure of mitochondria, we have chosen to focus on inter-organelle interactions and their consequences. While mitochondria-associated ER membranes, the biochemical composition of the contact sites and diverse physiological and disease-related functions have been extensively studied over the decades [START_REF] Vance | MAM (mitochondria-associated membranes) in mammalian cells: Lipids and beyond[END_REF][START_REF] Van Vliet | Mitochondria-Associated Membranes and ER Stress[END_REF] , it is increasingly recognized that other organelle contacts have a vital role in diverse cellular functions [START_REF] Rowland | ER Contact Sites Define the Position and Timing of Endosome Fission[END_REF] . More recently, there has been growing interest in quantifying other membrane interactions with mitochondria and their cell distribution in space and time [START_REF] Valm | Applying systems-level spectral imaging and analysis to reveal the organelle interactome[END_REF] , in particular within the endo-lysosomal pathway and their contribution to the fission/fusion process of the mitochondria network [START_REF] Kleele | Distinct fission signatures predict mitochondrial degradation or biogenesis[END_REF] . Here, while confirming the coincidence of contacts between the endo-lysosomal membrane and mitochondria, we enlightened the change of cristae density during fission (Supplementary Fig. 6). This density as well as complex cristae arrangements depends on cell types and metabolic activities [START_REF] Stephan | Live-cell STED nanoscopy of mitochondrial cristae[END_REF][START_REF] Stoldt | Spatial orchestration of mitochondrial translation and OXPHOS complex assembly[END_REF] , not talking of obvious modifications induced by environmental conditions. Until now, to provide a dynamic view of individuals and groups of cristae required 3D nanoscopy or linear SIM [START_REF] Huang | Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy[END_REF] , which are not always compatible with the time frame required to capture the event of interest. In this respect, DeepCristae might be an asset to compare the cristae dynamics in different cell types and in these various conditions. However, as with any image restoration method, scientists may be concerned by the robustness of DeepCristae to accurately restore mitochondria cristae and not hallucinate them. This is why we investigated the stability and limits of our method. First, we consider the RL algorithm in our proof-of-concept as it may be considered as the baseline algorithm for image deconvolution in fluorescence microscopy. The behavior of this algorithm is well established, including the "night sky" artifacts when iterated. The GTs shown in the manuscript may be improved if we consider more sophisticated deconvolution algorithms. We also worked out different conditions of use to be respected to guarantee good quality and truthfulness of the results. It is important to feed DeepCristae with images containing mitochondria whose average width in pixels is close to the one seen during the training. Concerning the microscope settings, it is better to ensure that the level of noise and blurring in the input images is equivalent to or better than the one present in the training data (which was quite high in our training). Under these conditions of use, across all our experiments and through different microscopy modalities, no hallucination was observed: a consistency between line profiles along mitochondria between raw and restored data was always observed.

Like cytoskeletal elements, the mitochondrial ultrastructure is a key element for comparing the performance of new super-resolution microscopy techniques. In terms of applications, DeepCristae makes it possible to track the evolution of mitochondrial cristae morphology over time, during interactions with other membrane components of the cell, or under extracellular conditions that mimic various pathological or stress situations. Second, as our goal is to learn how to restore LR images to better reveal the mitochondria cristae, we paired the LR STED images 𝑆𝑆 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝐿𝐿𝐿𝐿 with their restored counterpart 𝑆𝑆 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝐺𝐺𝐺𝐺 . These ground truths are obtained by deconvolving the real HR images 𝑆𝑆 𝐻𝐻𝐻𝐻 with the Richardson-Lucy algorithm [START_REF] Richardson | Bayesian-based iterative method of image restoration[END_REF][START_REF] Lucy | An iterative technique for the rectification of observed distributions[END_REF] , enhancing the mitochondria cristae. Other deconvolution algorithms were tried, such as SPITFIR(e) [START_REF] Prigent | SPITFIR(e): a supermaneuverable algorithm for fast denoising and deconvolution of 3D fluorescence microscopy images and videos[END_REF] or Wiener [START_REF] Gonzalez | Digital Image Processing[END_REF] , but the results obtained after training were not as good.

Finally, the dataset 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 is made of 24 training images and 9 test images. To further increase the size of the training set, data augmentation and patch sampling (later described in Methods) are performed on the pair of LR images 𝑆𝑆 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝐿𝐿𝐿𝐿 and ground truths 𝑆𝑆 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝐺𝐺𝐺𝐺 . The dataset is first augmented by applying three different rotations to the images (90°, 180° and 270°). Then, a shrink transform, and horizontal and vertical flips are successively applied to 25% of the augmented dataset, randomly selected. The final training set is made of 1824 patches of size 128×128 pixels, whose 20% are used for the validation set (summary in Supplementary Table 1).

Network architecture. We used the network proposed by Weigert et al. [START_REF] Weigert | Content-aware image restoration: pushing the limits of fluorescence microscopy[END_REF] as the backbone of the CNN architecture, itself built upon the U-Net [START_REF] Ronneberger | Convolutional Networks for Biomedical Image Segmentation[END_REF] . It has a contracting path and an expansive path, each one consisting of 3 sequential downsampling and upsampling blocks, respectively. Each block of the first path is skip-connected to the associated one of the expansive paths. The contracting path consists of two successive 3×3 convolutions, each followed by a Rectified Linear Unit (ReLU), and a 2×2 max pooling operation with stride 2 for downsampling. Every depth in the expansive path consists of a 2×2 up-sampling of the feature map, concatenated with the corresponding feature map from the contracting path, followed by two 3×3 convolutions with a ReLU activation function. At the final layer, one 1×1 convolution is used. The output results from an additive assembly between the input of the neural network and the last layer's output. The network (Fig. 1f) outputs the same size restored images.

Design of the training loss.

We present our new loss, the Similarity Component Prioritization (SCoP) loss, that has been designed to better restore mitochondria cristae. Most losses and metrics used to train networks or to evaluate the quality of restorations compute the score on the whole image, giving the same weight to any pixel. For example, the MAE computes the mean absolute error between the prediction and the target image, while the SSIM, despite not basing the calculation on pixels-to-pixels difference, computes the similarity among all the pixels of both images. Instead, our purpose is to focus on informative pixels corresponding to target structures in images. Indeed, the dark and noisy background occupies most of fluorescence images, which alters the learning. To overcome this issue, we introduce SCoP, a novel loss which adaptively prioritizes the restoration of mitochondria pixels.

Our SCoP loss is built upon the structural dissimilarity (DSSIM) measure. Consider (𝑝𝑝, 𝑗𝑗) the spatial coordinates of a given pixel and a patch of size (𝑁𝑁, 𝑁𝑁). The loss formula between a target image 𝑦𝑦 and its prediction 𝑦𝑦 � is given by

𝑁𝑁𝑆𝑆𝑆𝑆𝑃𝑃(𝑦𝑦, 𝑦𝑦 �) = 1 𝑁𝑁 2 � � � 1 -𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑠𝑠,𝑠𝑠 � 𝑚𝑚𝑐𝑐𝑚𝑚 (𝑝𝑝, 𝑗𝑗) 2 � 𝛾𝛾 𝑖𝑖,𝑗𝑗 𝑁𝑁 𝑗𝑗=1 𝑁𝑁 𝑚𝑚=1 , (1)
Where 𝑁𝑁𝑁𝑁𝑆𝑆𝑁𝑁 𝑠𝑠,𝑠𝑠 � 𝑚𝑚𝑐𝑐𝑚𝑚 is the map of the local structural similarity (SSIM) values for corresponding pixels between the images 𝑦𝑦 and 𝑦𝑦 �. Each SSIM value ranges in [-1,1], where -1 (1, respectively) testifies of a bad (very good similarity, respectively) between 𝑦𝑦(𝑝𝑝, 𝑗𝑗) and 𝑦𝑦 �(𝑝𝑝, 𝑗𝑗).

The parameter 𝛾𝛾 𝑚𝑚,𝑗𝑗 prioritizes the restoration of specific regions of interest. In our case, we chose 𝛾𝛾 𝑚𝑚,𝑗𝑗 = 1 if the pixel (𝑝𝑝, 𝑗𝑗) belongs to a mitochondrion, 4 otherwise. In this way, we encourage the network to focus on restoring mitochondria pixels and reduce the influence of a poorly restored background on the loss. Determining whether a pixel belongs to the background or to a mitochondrion can be performed automatically (using our method described in Methods "Image patch sampling for the training step -Thresholding") or manually by using any binary segmentation provided by the user.

Data normalization.

Our training images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 have different ranges of intensity values. To homogenize them, we normalized the input data and their corresponding ground truth to a common distribution of intensity values with the percentile normalizer. This normalization also has the advantage to exclude outliers, which are very frequent in microscopy imaging due to noise and luminance. The percentile normalization of an image 𝑆𝑆 is defined as

𝑆𝑆 𝑠𝑠𝑚𝑚𝑐𝑐𝑚𝑚 = 𝑆𝑆 -𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝(𝑆𝑆, 𝑝𝑝 𝑏𝑏𝑚𝑚𝑙𝑙 ) 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝�𝑆𝑆, 𝑝𝑝 ℎ𝑚𝑚𝑖𝑖ℎ � -𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝(𝑆𝑆, 𝑝𝑝 𝑏𝑏𝑚𝑚𝑙𝑙 ) , (2)
where 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝(𝑆𝑆, 𝑝𝑝) is the p-th percentile of 𝑆𝑆. We used 𝑝𝑝 𝑏𝑏𝑚𝑚𝑙𝑙 = 2 and 𝑝𝑝 ℎ𝑚𝑚𝑖𝑖ℎ = 99.8. This step is also performed during the inference step on any input data. � patches of size 128×128. As our images contain more background pixels than mitochondria pixels, grid or simple random patch sampling will end in too many empty patches. This can degrade the training of our model. Instead, we perform a random sampling focusing on the regions of interest, the mitochondria. Our pipeline (Supplementary Fig. 1) is described as follows.

Image

• Anscombe transform. To detect the areas of interest, we need to enhance the mitochondria signal with respect to the noise. To do this, we first remove the Poisson-Gaussian noise in STED images. This is achieved by applying an Anscombe transform, which enables to stabilize noise variance and to approximately convert Poisson-Gaussian noise into white Gaussian noise with a constant variance. The Anscombe transform of an image 𝑆𝑆 is given by

𝑆𝑆 𝐴𝐴𝑠𝑠𝑠𝑠𝑐𝑐 (𝑝𝑝, 𝑗𝑗) = 2 � 3 8 + 𝑆𝑆(𝑝𝑝, 𝑗𝑗), ∀(𝑝𝑝, 𝑗𝑗) ∈ 𝛺𝛺 . (3)
• Z-score. Then, we compute the Z-score map defined as

𝑍𝑍(𝑝𝑝, 𝑗𝑗) = 𝑆𝑆 𝐴𝐴𝑠𝑠𝑠𝑠𝑐𝑐 (𝑝𝑝, 𝑗𝑗) -𝜇𝜇̂∈ 𝜎𝜎 � ∈ , ∀(𝑝𝑝, 𝑗𝑗) ∈ 𝛺𝛺 , (4)
where 𝜇𝜇̂𝜖𝜖 and 𝜎𝜎 � 𝜖𝜖 are the estimated mean and standard deviation of the Gaussian noise 𝜖𝜖, respectively. Since most of the pixels in 𝑆𝑆 belong to the background, we consider 𝜇𝜇̂𝜖𝜖 =

  patch sampling for the training step. Our model is trained on the dataset 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 containing 24 images (96 after data augmentation) of different sizes. In order to homogenize and increase the training dataset, we performed patch sampling. We sampled each input training image 𝑆𝑆 ∈ 𝑁𝑁 𝑊𝑊×𝐿𝐿 , defined over the grid 𝛺𝛺 of size 𝑊𝑊 × 𝐿𝐿, within 𝑁𝑁 𝐼𝐼 = �

FiguresFigure 1 .

 1 Figures

Figure 2 .

 2 Figure 2. DeepCristae outperforms state-of-the-art methods for restoring mitochondria cristae in low-resolution 2D STED images. (a) The image grid displays restoration results of 3 test images from the dataset 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 obtained with DeepCristae and three competitive deep learning methods: RCAN 37 , SRResNet 35 and CARE 19 . GT: ground truth. Pixel size: 25 nm. Scale bar: 1 μm. White arrowheads indicate mitochondria with low contrast or in a noisy background restored by CARE; to be compared with DeepCristae column (b) Quantitative comparison of DeepCristae with conventional (Richardson-Lucy (RL)[START_REF] Richardson | Bayesian-based iterative method of image restoration[END_REF][START_REF] Lucy | An iterative technique for the rectification of observed distributions[END_REF] , Wiener[START_REF] Gonzalez | Digital Image Processing[END_REF] , SPITFIR(e)[START_REF] Prigent | SPITFIR(e): a supermaneuverable algorithm for fast denoising and deconvolution of 3D fluorescence microscopy images and videos[END_REF] , ND-SAFIR[START_REF] Boulanger | Patch-Based Nonlocal Functional for Denoising Fluorescence Microscopy Image Sequences[END_REF] , 2D median filter (Median 2D) and deep learning (Noise2Void 33 , CARE[START_REF] Weigert | Content-aware image restoration: pushing the limits of fluorescence microscopy[END_REF] , RCAN[START_REF] Zhang | Image Super-Resolution Using Very Deep Residual Channel Attention Networks[END_REF] and SRResNet[START_REF] Ledig | Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[END_REF] ) image restoration methods. Metrics were computed on the 9 test images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 . Note that all deep learning methods were trained using the same patches extracted from the training images of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 . Parameters used for conventional methods are indicated in Supplementary Note 2.2.

Figure 3 .

 3 Figure 3. DeepCristae reveals mitochondria cristae from low resolution (LR) 2D STED images. RPE1 cells were labeled with PKMITO-Orange prior to high resolution (HR) 2D STED imaging. (a) Degradation of the HR image with a Gaussian filter (𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 = 3.25 pixels) and additive Poisson-Gaussian noise (𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 = 4.0). The resulting image is part of the test dataset of 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 . Pixel size: 25 nm. Scale bar: 3 μm. (b-e) Zoom in on the colored insets depicted in (a). Top, from left to right: thumbnails of the zoomed-in area on the LR image, the image restored by DeepCristae and the ground truth (GT) image, respectively. The GT was obtained by deconvolving the HR raw image with the RL algorithm to enhance mitochondria cristae. Bottom: comparison of normalized intensity line profiles along a mitochondrion between the three thumbnails. The yellow line indicated in the corresponding colored inset in (a) serves to identify the fluorescence profile. Note that GT shows undesired high-intensity pixels in the background due to noise amplification by the RL algorithm. Therefore, DeepCristae line profiles are sometimes smoother than the one of the ground truth, especially in (d) and (e).

Figure 4 .

 4 Figure 4. Reliability of image restoration DeepCristae. First experiment (a-f): assessment of the reliability of DeepCristae on real data by controlling the consistency between DeepCristae restoration and a real "ground truth". RPE1 cells were labeled with PKMITO-Orange prior to low resolution (LR) and to high resolution (HR) 2D STED imaging, acquired successively. (a, d) Two LR STED images (Inputs) were given as input to DeepCristae for restoration and qualitatively compared to the corresponding HR STED images. Notes that the HR data were deconvolved using the Richardson-Lucy algorithm to enhance mitochondria cristae and are considered here as "ground truths" (GTs). Pixel size: 25 nm. Scale bar: 1 μm. (b, c) and (e,f) Zoom in on the colored insets depicted in (a) and (d), respectively. Top, from left to right: thumbnails of the zoomed-in area on the LR image, the image restored by DeepCristae and the ground truth (GT) image, respectively. Bottom:

Figure 5 .

 5 Figure 5. DeepCristae reveals mitochondria cristae from low resolution 2D live STED. (a-c) 2D live STED imaging of RPE1 cells labeled with PKMITO-Orange. (a, b) Time series of 10 images (Δt~13s) using 2D high-resolution (HR) STED (Supplementary Note 1: Materials and Methods). Pixel size is 50 x 50 nm in (a) and 25 x 25 nm in (b). Early and late time points are shown in a, the 4 first time points are shown in b. Phototoxic damage as illustrated by the swelling of the mitochondria. (c) Time series of 2D Fast STED, improving the time delay between time points (Δt~5.9 s) and reducing mitochondrial damage. Each thumbnail is diagonally divided for raw 2D Fast (LR) STED images (bottom-left part) and after DeepCristae treatment (top-right). To match the training mitochondria settings (see Results "Reliability and

Figure 6 .

 6 Figure 6. DeepCristae restoration enhances cristae width resolution in 3D and 3D+live imaging. (a) 2D plane (top) and 3D MIP (Maximum Intensity Projection of 14 planes) (bottom) of cell labeled with PKMITO-Orange, acquired with a SD microscope with a Live-SR module, before (left) and after DeepCristae (right). Thumbnails are zoomed areas corresponding to the insets (red and blue) and are composites of raw and DeepCristae images. Color scale bar indicates the bottom to top position of mitochondria (bottom right). (b) Cristae widths were

Figure 7 .

 7 Figure 7. DeepCristae reveals 3D+time cristae morphology during endo/lysosome mitochondria interactions. RPE1 cells incubated 4 hours with Cell Mask PM Deep Red (red) were labeled with PKMITO-Orange (green) for the last 15 minutes. (a) A Maximum Intensity Projection (20 planes; Stack time=1.86s/channel, time point t1 over 60) image acquired with Live-SR microscopy is shown before (top) and after (bottom) DeepCristae restoration of the

  To match the training mitochondria settings (see Results "Reliability and limits of image restoration by DeepCristae"), a rescaling of 1.87 is applied to the LR image before DeepCristae inference. Scale Bars in a-c are: 2μm. (d) Lateral widths of 7 mitochondria measured at each time point (series of 10 time points) in HR STED and in Fast (LR) STED, before and after DeepCristae (line profiles as depicted as arrow heads in a t8 and b t2). Line profiles were fitted to a Gaussian model and FWHM was measured, as indicated in Supplementary Note 1: Materials and Methods. Data are expressed as mean ± SD. (e) Cristae widths measured as in d for 30 cristae on 20 mitochondria (from 4 distinct image series) in early and late time points of HR STED and 31 cristae in Fast (LR) STED, before (Raw; two measures above the ordinate scale) and after DeepCristae (line profiles as illustrated in a t8, b t4 and c t10). Data are expressed as mean ± SD. Student t-test: ** (p-val=0.0082), ***(p-val=0.0003). (f) Distances between two cristae (Cristae intervals) measured in a peak-to-peak intensity from plot profiles (RAW Fast STED was not possible). Early: N=56 from 2 series; Late: N=80 in 2 series; DeepCristae: N=60 in 3 series, distributed in all time points. Data are expressed as mean ± SD. F-test: ****(p-val<0.0001). (g) Table with complete statistics including significance using Student-and Fisher-tests; ns= non-significant.

Acknowledgements

This work was supported by the France-BioImaging Infrastructure (French National Research Agency, ANR-10-INBS-04-07, "Investments for the future") and the Labex Cell(n)Scale (ANR-11-LABX-0038) as part of the Idex PSL (ANR-10-IDEX-0001-02). We acknowledge the Cell and Tissue Imaging (PICT IBiSA, Institut Curie) and the IMACHEM (Collège de France) platforms, also members of the national infrastructure France-BioImaging (ANR-10-INBS-04-01) for access to and maintaining the spinning-disk, Airyscan and STED microscopes. We also wish to thank M. Maurin from Inserm U932 for his help in a preliminary study on STED image acquisition.

Code availability

DeepCristae source code used in this publication is open-source and published under the BSD 3-Clause "Original" or "Old" License. Source code will be available shortly through GitHub.

Methods

In this section, we present the main features of DeepCristae. We first present the dataset we created from real 2D STED images to train the network. Then, we overview our network architecture and present the novel learning loss function, which prioritizes the restoration of specific pixels. We finally detail the image patch-sampling steps, which is a crucial stage in the model pipeline.

Generation of the 2D STED dataset -𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 . As mitochondria are living organelles, mostly organized as a quite fast-moving network in RPE1 cells (Supplementary Note 1.1), the acquisition of a pair of high and low-resolution images at the exact same time point is impossible. To train and quantitatively validate DeepCristae, we thus created a dataset, called 𝐷𝐷 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 , from 33 acquired high resolution (HR) 2D STED images (25 x 25 nm) that we denote 𝑆𝑆 𝐻𝐻𝐻𝐻 (Fig. 1 a-d). More information on the acquisition of the images 𝑆𝑆 𝐻𝐻𝐻𝐻 are available in Supplementary Note 1.2.1.

First, we degraded the images 𝑆𝑆 𝐻𝐻𝐻𝐻 to obtain low resolution (LR) images of mitochondria, denoted 𝑆𝑆 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝐿𝐿𝐿𝐿 , that will serve as input to the neural network. To that end, we first applied a Gaussian filter of standard deviation 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 = 3.25 pixels to the images 𝑆𝑆 𝐻𝐻𝐻𝐻 in order to approximate the blurring effect due to the point spread function of the microscope. Then, we added a Poisson-Gaussian noise (𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 = 4.0 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑠𝑠), consistently with noises observed in real STED images. The parameters 𝜎𝜎 𝑏𝑏𝑏𝑏𝑏𝑏𝑐𝑐 and 𝜎𝜎 𝑠𝑠𝑚𝑚𝑚𝑚𝑠𝑠𝑐𝑐 were set to create pertinent input data that mimic real LR STED images (Supplementary Note 1.2.1). 𝑚𝑚𝑝𝑝𝑚𝑚𝑝𝑝𝑚𝑚𝑚𝑚�{𝑆𝑆(𝑝𝑝, 𝑗𝑗)} (𝑚𝑚,𝑗𝑗)𝜖𝜖𝜖𝜖 �. For 𝜎𝜎 � 𝜖𝜖 , we use a robust estimator derived from the Median Absolute Deviation (MAD) such that 𝜎𝜎 � 𝜖𝜖 = 1.4826 ⋅ 𝑚𝑚𝑝𝑝𝑚𝑚𝑝𝑝𝑚𝑚𝑚𝑚�{ |𝑝𝑝(𝑝𝑝, 𝑗𝑗)|} (𝑚𝑚,𝑗𝑗)𝜖𝜖𝜖𝜖 �, where 𝑝𝑝(𝑝𝑝, 𝑗𝑗) = 2𝐼𝐼 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 (𝑚𝑚,𝑗𝑗) -𝐼𝐼 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 (𝑚𝑚+1,𝑗𝑗)-𝐼𝐼 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 (𝑚𝑚,𝑗𝑗+1)

√6

, ∀(𝑝𝑝, 𝑗𝑗)𝜖𝜖𝛺𝛺, are the pseudo-residuals. In fact, under the hypothesis of having a white Gaussian noise and that the noise-free image is piecewise smooth in a local neighborhood, we have that 𝜎𝜎 � 𝜖𝜖 2 = 𝑁𝑁[𝑝𝑝 2 (𝑝𝑝, 𝑗𝑗)].

• Thresholding. The higher the Z-score in Eq. ( 4), the higher the pixel value is above the mean of the measured noise and therefore the pixel (𝑝𝑝, 𝑗𝑗) is considered as a pixel of interest. We apply a threshold c, in a way that any pixel (𝑝𝑝, 𝑗𝑗)𝜖𝜖 𝛺𝛺 such that 𝑍𝑍(𝑝𝑝, 𝑗𝑗) > 𝑝𝑝 is considered as a mitochondria pixel. We denote this set as 𝛺𝛺 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 . The threshold is automatically adapted for each training image. Starting from a fixed high value of 30, while 𝛺𝛺 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 does not contain a minimum of 10% mitochondrial information (i.e., #𝛺𝛺 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 < 10% #𝛺𝛺, where #𝛺𝛺 and #𝛺𝛺 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 denote the number of pixels in the sets 𝛺𝛺 and 𝛺𝛺 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 , respectively), we subtract 5 from the threshold value. This creates a binary mask on which we apply a median to remove the surrounding noise. This automatic procedure avoids cumbersome manual annotations. Note that this mask can also be used to compute the parameter 𝛾𝛾 𝑚𝑚,𝑗𝑗 in our loss (see Eq. ( 1)).

• ROIs selection. From 𝛺𝛺 𝑚𝑚𝑚𝑚𝑠𝑠𝑚𝑚 , we randomly choose 𝑁𝑁 𝐼𝐼 different pixels to be the center of ROIs of size 128×128 pixels. Thus, the more pixels of mitochondria a ROI contains, the more likely it is to be chosen. The following conditions have to be respected: i) the ROI centers should not belong to the borders of the image; ii) to avoid redundancy, a minimum distance of 60 pixels is established between each pairwise ROI center. The resulting ROIs are finally used to create the patches from the normalized training data (see above "Data normalization").

Network evaluation. In addition to a quantitative comparison to the state-of-the-art methods and experiments to show the reliability of our method (Results), an ablation study was also performed (Supplementary Fig. 4; Supplementary Note 2.3) to highlight the individual contribution of key components of our method. More details about the evaluation metrics and the implementation details of DeepCristae are also given in Supplementary Note 2.1 and Supplementary Note 2.2.1, respectively.

Other Methods and Materials

Cell culture and biological materials, fluorescence labeling, all used microscopy techniques, image acquisition protocols and quantitative measurements are detailed in the Supplementary Note 1. PKMITO dyes are commercially available at Spirochrome (Stein-am-Rhein, Switzerland) and Genvivo Biotech (Nanjing, China)

Data availability

Competing interests

Z.C. is an inventor of the patent on the mitochondria dye described in this work. All other authors have no competing interests to declare.