Oussama Ennafii
email: oussama.ennnafii@ign.fr

Arnaud Le Bris

Florent Lafarge

Clément Mallet

A learning approach to evaluate the quality of 3D city models

Keywords: 3D urban modeling, buildings, quality assessment, taxonomy, classification, error detection, geometry, aerial imagery, Very High Spatial Resolution, Digital Surface Model

The automatic generation of 3D building models from geospatial data is now a standard procedure.

An abundant literature covers the last two decades and several softwares are now available. However, urban areas are very complex environments. Inevitably, practitioners still have to visually assess, at city-scale, the correctness of these models and detect frequent reconstruction errors. Such a process relies on experts, and is highly time-consuming with approximately two hours/km 2 per expert. This work proposes an approach for automatically evaluating the quality of 3D building models.

Potential errors are compiled in a novel hierarchical and versatile taxonomy. This allows, for the first time, to disentangle fidelity and modeling errors, whatever the level of details of the modeled buildings. The quality of models is predicted using the geometric properties of buildings and, when available, Very High Resolution images and Digital Surface Models. A baseline of handcrafted, yet generic, features is fed into a Random Forest classifier. Both multi-class and multi-label cases are considered: due to the interdependence between classes of errors, it is possible to retrieve all errors at the same time while simply predicting correct and erroneous buildings. The proposed framework was tested on three distinct urban areas in France with more than 3,000 buildings. 80%-99% F-score values are attained for the most frequent errors. For scalability purposes, the impact of the urban area composition on the error prediction was also studied, in terms of transferability, generalization, and representativeness of the classifiers. It shown the necessity of multimodal remote sensing data and mixing training samples from various cities to ensure a stability of the detection ratios, even with very limited training set sizes.

Introduction

Context and objectives

3D urban models have a wide range of applications. They can be used for consumer purposes (video games or tourism) as much as they can be vital in more critical domains with significant societal challenges (e.g., disaster control, run-off water or microclimate simulation, urban planning or security operations preparation) [START_REF] Musialski | A survey of urban reconstruction[END_REF][START_REF] Biljecki | Applications of 3D city models: state of the art review[END_REF]. Therefore, automatic urban reconstruction from geospatial imagery (spatial/airborne sensors) focuses efforts on both scientific research and industrial activities. 3D city modeling has therefore been deeply explored in the photogrammetric, GIS, computer vision, and computer graphics literature with an emphasis on compactness, full automation, robustness to acquisition constraints, scalability, and inevitably, quality [START_REF] Müller | Procedural modeling of buildings[END_REF][START_REF] Over | Generating web-based 3D city models from OpenStreetMap: The current situation in Germany[END_REF][START_REF] Vanegas | Building reconstruction using Manhattan-world grammars[END_REF][START_REF] Lafarge | Creating large-scale city models from 3D-point clouds: a robust approach with hybrid representation[END_REF][START_REF] Poli | 3d modeling of large urban areas with stereo VHR satellite imagery: lessons learned[END_REF][START_REF] Stoter | Three-dimensional modeling with national coverage: case of The Netherlands[END_REF][START_REF] Zhou | Complete residential urban area reconstruction from dense aerial lidar point clouds[END_REF][START_REF] Cabezas | Semantically-aware aerial reconstruction from multimodal data[END_REF][START_REF] Monszpart | Rapter: Rebuilding man-made scenes with regular arrangements of planes[END_REF][START_REF] Kelly | Bigsur: Large-scale structured urban reconstruction[END_REF][START_REF] Nguatem | Modeling urban scenes from pointclouds[END_REF]. However, the problem remains partly unsolved [START_REF] Sester | Theme issue "Quality, scale and analysis aspects of urban city models[END_REF][START_REF] Musialski | A survey of urban reconstruction[END_REF][START_REF] Rottensteiner | Results of the ISPRS benchmark on urban object detection and 3D building reconstruction[END_REF]. In fact, besides the seamless nature of reconstituted models, current algorithms lack of generalization capacity. They cannot handle the high heterogeneity of urban landscapes. As such, for operational purposes, human intervention is needed either in interaction within the reconstruction pipeline or as a post-processing refinement and correction step. The latter is highly tedious: it consists in an individual visual inspection of buildings [START_REF] Musialski | A survey of urban reconstruction[END_REF]. Consequently, automatizing the last step remains, for all stakeholders (from researchers up to end-users), a critical step, especially in a production environment. It has been barely investigated in the literature. This paper addresses this issue by expanding earlier work [START_REF] Ennafii | The necessary yet complex evaluation of 3D city models : a semantic approach[END_REF].

Qualifying 3D building models

Our work focuses on assessing polyhedral structured 3D models, representing building architectures [START_REF] Haala | An update on automatic 3D building reconstruction[END_REF]. These models result from a given urban reconstruction method, that is unknown from our evaluation pipeline. We discard triangle meshes that are standardly generated from multiview images or point clouds with state-of-the-art surface reconstruction methods. Here, the studied objects are, by design, more compact but less faithful to input data. In counterpart, they hold more semantic information: each polygonal facet typically corresponds to a façade, a roof, or any other architecturally atomic feature of a building. 3D modeling algorithms traditionally build a trade-off between representation compactness and fidelity to the input data (meshes or 3D points). Depending on its spatial accuracy, the urban setting, and the targeted application, the reconstituted result achieves a certain Level of Detail (LoD) [START_REF] Kolbe | CityGML: Interoperable access to 3D city models[END_REF]. A LoD-1 model is a simple building extrusion (flat roof) [START_REF] Ledoux | Topologically consistent 3d city models obtained by extrusion[END_REF][START_REF] Biljecki | Generating 3D city models without elevation data[END_REF]. A LoD-2 model considers geometric simplification of buildings, ignoring superstructures, such as dormer windows and chimneys [START_REF] Taillandier | Automatic buildings reconstruction from aerial images: a generic Bayesian framework[END_REF]. These are taken into account in LoD-3 [START_REF] Brédif | 3D building reconstruction with parametric roof superstructures[END_REF]. The LoD rational is still open for debate (Biljecki et al., 2016b). Nevertheless, in this paper, we will follow the LoD categorization introduced above, which is also standard in the computer vision and graphics literature.

A large body of papers has addressed the 3D building modeling issue and subsequently tried to find the trade-off between fidelity and compactness [START_REF] Dick | Modelling and interpretation of architecture from several images[END_REF][START_REF] Zebedin | Fusion of feature-and area-based information for urban buildings modeling from aerial imagery[END_REF][START_REF] Lafarge | Structural approach for building reconstruction from a single DSM[END_REF][START_REF] Verdie | LoD generation for urban scenes[END_REF]. Conversely, few works investigate the issue of assessing the quality of the derived models, especially out of a given reconstruction pipeline [START_REF] Schuster | A new approach towards quantitative quality evaluation of 3d building models[END_REF].

Usually, quality assessment is based on visual inspection [START_REF] Durupt | Automatic building reconstruction from a Digital Elevation Model and cadastral data: an operational approach[END_REF][START_REF] Macay Moreia | From DSM To 3D building models: a quantitative evaluation[END_REF], geometric fidelity metrics [START_REF] Kaartinen | Accuracy of 3d city models: EuroSDR comparison[END_REF], or by extending standard 2D object detection criteria [START_REF] Karantzalos | Large-scale building reconstruction through information fusion and 3-d priors[END_REF], without any semantic dimension. Only one benchmark dataset has addressed the issue [START_REF] Rottensteiner | Results of the ISPRS benchmark on urban object detection and 3D building reconstruction[END_REF]. It remains focused on very few areas and a geometric comparison with manually extracted roof structures [START_REF] Li | Manhattan-world urban reconstruction from point clouds[END_REF][START_REF] Nan | Polyfit: Polygonal surface reconstruction from point clouds[END_REF][START_REF] Nguatem | Modeling urban scenes from pointclouds[END_REF]. Consequently, it cannot be easily extended.

Similar conclusions can be drawn for indoor reconstruction [START_REF] Tran | Geometric comparison and quality evaluation of 3d models of indoor environments[END_REF].

Positioning and contributions

The current situation motivates the need for a well suited quality assessment paradigm. Since the building models display strong structural properties, an unconstrained evaluation based on data fidelity metrics, as in [START_REF] Berger | A benchmark for surface reconstruction[END_REF], is too general. The evaluation should also ignore format issues or geometric consistencies as proposed in [START_REF] Ledoux | val3dity: validation of 3D GIS primitives according to the international standards[END_REF]. Although being a serious issue and clean 3D models are usually not the norm (Biljecki et al., 2016a;[START_REF] Hu | Tetrahedral meshing in the wild[END_REF], we rule out, at this stage, these cases for simplicity. Instead, we target a semantic evaluation in which building semantics is taken into account through the detection and categorization of modeling errors at the facet level for each 3D building. The framework is independent from the LoD and the modeling method. The standard criteria used in the reconstruction process (e.g., L 1 norm between the model and a DSM) will not be taken into account, as they are usually chosen as minimization targets in the modeling procedure. Thus, we define an evaluation framework that can be used for:

• Building model correction: for the automatic or interactive [START_REF] Kowdle | Active learning for piecewise planar 3d reconstruction[END_REF] refinement of building models using the detected errors.

• Change detection: modeling errors can straightforwardly stem from changes, which frequently occur in urban environments [START_REF] Taneja | Geometric change detection in urban environments using images[END_REF]. Conversely, changes can be implicitly detected from other defects [START_REF] Tran | Geometric comparison and quality evaluation of 3d models of indoor environments[END_REF].

• Reconstruction method selection: evaluating models from various reconstruction algorithms can allow assessing which method(s) is(are) the most adapted for a specific LoD and building type.

• Crowd-sourcing evaluation [START_REF] Kovashka | Crowdsourcing in computer vision[END_REF]: categorizing user behaviors during crowd-sourced modeling and vandalism detection process [START_REF] Neis | Towards automatic vandalism detection in OpenStreetMap[END_REF].

This work proposes an adaptable and flexible framework indifferent to input urban scenes and reconstruction methods. For that purpose, our contributions are three-fold:

• A new taxonomy of errors, hierarchical, adapted to all LoDs, and independent from input models;

• A supervised classification formulation of the evaluation problem which predicts all errors affecting the building model;

• A multimodal baseline of features that are extracted from the model itself as well as from Very High Resolution external data (optical images and height data).

Section 2 introduces the problem of the evaluation of 3D building models and discusses existing methods. Section 3 details the proposed approach, while data and experiments conducted over three urban areas are presented in Section 4. A more comprehensive set of experiments studying the scalability of the proposed method is reported in Section 5. The same experiments are conducted at other semantic levels and recorded in Section 6. Main conclusions are drawn in Section 7.

Related Work

Quality assessment methods can be classified according to two main criteria: reference data and output type.

Reference data types

Existing methods rely on two types of reference data.

Manually plotted ground truth data with very high spatial accuracy: these models can be obtained either from field measurements [START_REF] Dick | Modelling and interpretation of architecture from several images[END_REF][START_REF] Kaartinen | Accuracy of 3d city models: EuroSDR comparison[END_REF] with the highest possible precision (σ(error) ≈ 0.05 m), or using stereo-plotting techniques [START_REF] Jaynes | Recognition and reconstruction of buildings from multiple aerial images[END_REF][START_REF] Kaartinen | Accuracy of 3d city models: EuroSDR comparison[END_REF][START_REF] Zebedin | Fusion of feature-and area-based information for urban buildings modeling from aerial imagery[END_REF][START_REF] Zeng | A multicriteria evaluation method for 3D building reconstruction[END_REF]. Generally, the criterion is the root mean squared error on the height values. Such a strategy does not scale well, does not straightforwardly bring semantics, and requires a 3D matching procedure (overlapping ratio between surfaces, minimal roof areas, integration of superstructures) that can be complex in dense urban environments.

Raw remote sensing data: models can either be compared to the source that allowed the generation of the models or remote sensing data of superior geometric accuracy: LiDAR point clouds, height maps (i.e., DSMs) [START_REF] Akca | Quality assessment of 3D building data[END_REF][START_REF] Lafarge | Creating large-scale city models from 3D-point clouds: a robust approach with hybrid representation[END_REF][START_REF] Li | Manhattan-world urban reconstruction from point clouds[END_REF][START_REF] Zhu | Large scale urban scene modeling from mvs meshes[END_REF] or multi-view Very High Resolution images as in [START_REF] Boudet | A supervised classification approach towards quality self-diagnosis of 3D building models using digital aerial imagery[END_REF][START_REF] Michelin | Quality evaluation of 3D city building models with automatic error diagnosis[END_REF].

Despite the fact such strategy better samples the area of interest, it may not always be helpful. On one hand, they have been exploited by the modeling methods and such comparisons are often the basis for their fidelity criterion. On the other hand, additional remote sensing data is not easy to obtain, especially at large scales under operational constraints.

Evaluation outputs

The quality assessment methods can produce two kinds of outputs: geometric fidelity metrics and labels of errors.

Geometric fidelity metrics summarize the quality at the building level. These criteria are computed at different levels: average precision of specific points of interest (corners or edge points, [START_REF] Vögtle | On the quality of object classification and automated building modeling based on laser scanning data[END_REF][START_REF] Kaartinen | Accuracy of 3d city models: EuroSDR comparison[END_REF], surface dissimilarity [START_REF] Jaynes | Recognition and reconstruction of buildings from multiple aerial images[END_REF][START_REF] Dick | Modelling and interpretation of architecture from several images[END_REF][START_REF] Kaartinen | Accuracy of 3d city models: EuroSDR comparison[END_REF][START_REF] Zebedin | Fusion of feature-and area-based information for urban buildings modeling from aerial imagery[END_REF][START_REF] Lafarge | Creating large-scale city models from 3D-point clouds: a robust approach with hybrid representation[END_REF][START_REF] Zeng | A multicriteria evaluation method for 3D building reconstruction[END_REF][START_REF] Li | Manhattan-world urban reconstruction from point clouds[END_REF][START_REF] Nan | Polyfit: Polygonal surface reconstruction from point clouds[END_REF], average mean absolute distance [START_REF] Duan | Towards large-scale city reconstruction from satellites[END_REF][START_REF] Zeng | Neural procedural reconstruction for residential buildings[END_REF], tensor analysis of residuals [START_REF] You | A Quality Prediction Method for Building Model Reconstruction Using LiDAR Data and Topographic Maps[END_REF] or volume discrepancy to reference data [START_REF] Jaynes | Recognition and reconstruction of buildings from multiple aerial images[END_REF][START_REF] Zeng | A multicriteria evaluation method for 3D building reconstruction[END_REF][START_REF] Nguatem | Modeling urban scenes from pointclouds[END_REF]. Evaluation can also be performed according to compactness, which is complementary to fidelity metrics: number of faces/vertices in the model [START_REF] Lafarge | Creating large-scale city models from 3D-point clouds: a robust approach with hybrid representation[END_REF]Zhang and Zhang, 2018). For both cases, the obtained outputs have the drawback of being too general for the special case of urban structured models. Far from surface reconstruction evaluation [START_REF] Berger | A benchmark for surface reconstruction[END_REF], it is preferred that a diagnosis pinpoints specific types of errors that can be easily corrected with specific procedures [START_REF] Elberink | Quality analysis on 3d building models reconstructed from airborne laser scanning data[END_REF].

Semantic errors identify topological and geometric errors that affect building models. One example of such defects is the traffic light paradigm ("correct", "acceptable/generalized", and "incorrect") [START_REF] Boudet | A supervised classification approach towards quality self-diagnosis of 3D building models using digital aerial imagery[END_REF]. However, these errors depend on the definition of the end-user oriented nomenclature and a specific "generalization" level at which models are rejected. In addition, this taxonomy does not help in localizing the model shortcomings. Another solution is to look at the issue at hand through the used reconstruction algorithm perspective. For instance, defects are discriminated in [START_REF] Michelin | Quality evaluation of 3D city building models with automatic error diagnosis[END_REF], between footprint errors ("erroneous outline", "inexisting building", "missing inner court" and "imprecise footprint"), intrinsic reconstruction errors ("over-segmentation", "under segmentation", "inexact roof", and "Z translation"), and "vegetation occlusion" errors or are considered only for roof topology as in [START_REF] Xiong | A graph edit dictionary for correcting errors in roof topology graphs reconstructed from point clouds[END_REF] ("Missing Node", "False Node", "Missing Edge", and "False Edge"). In most of these methods, the evaluation is cast as a supervised classification process: the predicted classes are defects listed in an established taxonomy. Features used for this classification are extracted from very high spatial resolution (VHR, 0.1 m to 0.5 m) images and DSMs, like 3D segments or texture correlation score comparisons. In spite of their semantic contribution in quality evaluation, such taxonomies are prone to overfitting to specific urban scenes/ modeling algorithms or require the computation of complex features on This work aims to propose a new quality evaluation paradigm that detects and describes semantic errors that affect 3D building models. Two important characteristics must be taken into account.

First, the definition of the semantic errors should not vary from one urban scene to another and guaranty independence to the underlying 3D reconstruction method. Secondly, the scalability of the method should be addressed in order to ensure the ability to correctly classify unseen areas and to define the minimal amount of data required. This is all the more necessary in case of limited training sets in order to avoid overfitting to a specific problem and environment.

Problem formulation

We start by establishing a novel hierarchical error taxonomy. It is parameterizable1 and agnostic towards reconstructed models. Independence from the modeling method and the urban scenes is mandatory for generalization and transferability capacities. Depending on the evaluation objectives, we deduce error labels that can pinpoint defects altering the models. Their presence is predicted using a supervised classifier, trained with manually annotated data.

The quality assessment pipeline is constructed in order to be modular. Building models are represented by intrinsic geometric features extracted from the model facet graph. If available, the classifier can also be fed with remote sensing data: depth features based on the comparison of the model altimetry and a DSM or image information with spectral or textural features available in satellite, aerial or street view optical images.

Error taxonomy

In order to build a generic and flexible taxonomy, we rely on two criteria for error compilation:

the building model LoD and the error semantic level, named henceforth finesse (cf. Figure 2). Different degrees of finesse describe, from coarse to fine, the specificity of defects. Errors with maximal finesse are called atomic errors. Multiple atomic errors can affect the same building. For instance, topological defects induce, almost always, geometrical ones. In practice, only independently coexisting atomic defects are reported. The idea is to provide the most relevant information to be able to correct a model. Atomic errors can thus be intuitively correlated to independent actions to be chosen by an operator or an algorithm so as to correct the model.

The general framework

The main idea of error hierarchization is to enable modularity in the taxonomy, and thus achieve a strong flexibility towards input urban scenes and desired error precision. A general layout is first drawn, followed by a more detailed error description.

At a first level, model qualifiability is studied. In fact, aside from formatting issues or geometric inconsistencies [START_REF] Ledoux | val3dity: validation of 3D GIS primitives according to the international standards[END_REF], other reasons make building models unqualifiable. For instance, buildings can be occluded by vegetation and thus cannot be assessed with most of the remote sensing data sources. Generally speaking, input models can be impaired by some pathological cases that are outside our evaluation framework. In consequence, qualifiable models are distinguished here from unqualifiable buildings. This first level corresponds to a finesse equal to 0. At the finesse level 1, we predict the correctness of all qualifiable buildings. It is the lowest semantization level at which the evaluation of a model is expressed. Then, a model is either valid or erroneous. Most state-of-the-art evaluation methods address this level.

Model errors are grouped into three families depending on the underlying LoD. The first family of errors "Building Errors" affects the building in its entirety. It corresponds to an accuracy evaluation at LoD-0 (footprint errors) ∪ LoD-1 (height/geometric error). At the next LoD-2, the family "Facet

Errors" gathers defects that can alter façade or roof fidelity. The last error family, "Superstructure

Errors", describes errors that involve superstructures modeled at LoD-3. Only the first two families are studied in this paper.

Each family contains atomic errors of maximal finesse equal to 3. Although they can co-occur in the same building model and across different families, these errors are semantically independent.

They represent specific topological or geometric defects. Topological errors translate inaccurate structural modeling, while geometric defects raise positioning infidelity.

At evaluation time, three parameters play a role in determining which error labels to consider.

The first is the evaluation Level of Detail (eLoD). Every reconstruction method targets a certain set of LoDs. In consequence, when assessing a reconstruction, a LoD must be specified. At a given eLoD, all error families involving higher orders will be ignored. Depending on the target of the qualification process, a finesse level might be preferred. This second evaluation parameter specifies the appropriate semantic level at which errors will be reported. The last one is error However, it is not the case at the atomic errors level since they are independent.

exclusivity. It conveys family error hierarchy. Errors of a given LoD= l family are prioritized over ones with higher LoD> l.

Application to the geospatial overhead case

This paper tackles the aerial reconstruction case where the objective is to reconstruct large urban scenes using Very High Resolution geospatial images or, if available, lidar point clouds. The framework is general enough to encompass both orthorectified images and oblique ones. In this paper, we only used orthorectified images. In an ideal scenario, using oriented images is better for edge verification (as already shown in [START_REF] Michelin | Quality evaluation of 3D city building models with automatic error diagnosis[END_REF]) as orthoimages are a byproduct of earlier ones. However, in practice, oblique imagery would give rise to other issues, especially, registration problems. Hereaafter, 3D buildings are evaluated. The atomic errors are (Figures 2 and3):

Building errors family:

• Under segmentation (BUS): two or more buildings are modeled as one. In Figure 3.i.a, two distinct buildings were identified as one building, eventhough they can be visually distinguished.

• Over-segmentation (BOS): one building is subdivided into two or more buildings. This is the opposite of the previous situation. Figure 3.i.b shows a single building that, when modelled, was subdivided into three parts.

• Imprecise borders (BIB): at least one building footprint border is incorrectly located. A sample is shown in Figure 3.i.c.

• Inaccurate topology (BIT): the building footprint suffers from topological defects as missing inner courts or wrong primitive fitting (for instance, a circular footprint approximated by a polygon). In Figure 3.i.d, we illustrate how the footprint morphology can be erroneous. This error, as the earlier ones, result either from defective building identification process, or from an outdated cadastral map.

• Imprecise geometry (BIG): inaccurate building geometric estimation. In case eLoD > LoD-0 ∪ LoD-1, this error is not reported as it becomes redundant with below delineated errors.

Facet errors family:

• Under segmentation (FUS): two or more facets are modeled as one, as illustrated in Figure 3.ii.a.

• Over-segmentation (FOS): one facet is subdivided into two or more facets. Refer to Figure 3.ii.b for an example.

• Imprecise borders (FIB): at least one facet border is incorrectly located. As en example, Figure 3.ii.c shows that the central edge that links the two main roof sides does not correspond to the one on the image position.

• Inaccurate topology (FIT): the facet suffers from topological defects such as wrong primitive fitting (for example, a dome approximated by planar polygons). In Figure 3.ii.d), we can observe how two cylindrical towers were reconstructed as a rectangular parallelepiped.

• Imprecise geometry (FIG): inaccurate facet geometric estimation :e.g., wrong height or inaccurate slope. The latter is depicted in Figure 3.ii.e2 . All these errors stem either from the modeling approach, or from the poor spatial resolution of the input data (DSM or point cloud).

These errors can be related to state-of-the-art labels. For instance, "Missing Node" (resp.

"False Node", "Missing Edge" and "False Edge") in [START_REF] Xiong | A graph edit dictionary for correcting errors in roof topology graphs reconstructed from point clouds[END_REF] correspond to, or are included in, the topological atomic errors from the Facet Errors family: FUS (resp. FOS, FIT, and FIT). The difference is that we distinguish flaws that can affect superstructure facets (LoD-3) from the ones that impair building facets (LoD-2). The taxonomy developed by [START_REF] Michelin | Quality evaluation of 3D city building models with automatic error diagnosis[END_REF], on the other hand, is closer to ours. In fact, while footprint errors is reshuffled into Building Errors as BIB ("erroneous outline" and "imprecise footprint") and BIT ("missing inner court") , intrinsic reconstruction errors ("over-segmentation", "under segmentation", "inexact roof" and "Z translation") can be re-adapted into both family errors. Finally, "vegetation occlusion" and " non existing" are gathered into the unqualifiable label at finesse level 0. [START_REF] Boudet | A supervised classification approach towards quality self-diagnosis of 3D building models using digital aerial imagery[END_REF], however, studied the acceptability of a model in a whole. Their taxonomy cannot directly fit with our labels.

The acceptability dimension can be incorporated into our framework by attributing an confidence score to each error: for example, a prediction probability.

Feature baseline

In order to detect such specific labels while guarantying a certain flexibility towards reference data, multiple modalities are necessary. The structure of the 3D model can be directly used to extract geometrical features. Dense depth information can be added, through for instance a DSM, so as to help detecting geometric defects that can be hardly discriminated otherwise (as in Figure 3.ii.(e)), in particular for the outer part of buildings. VHR optical images bring additional information (high frequencies and texture) that is particularly suited for inner defect detection.

Since there is no comparable work that studies the previously defined errors, we propose a baseline for each modality. Attributes are kept simple so as to be used in most situations relying on generally available data. We avoid computing and comparing 3D lines [START_REF] Michelin | Quality evaluation of 3D city building models with automatic error diagnosis[END_REF], correlation scores [START_REF] Boudet | A supervised classification approach towards quality self-diagnosis of 3D building models using digital aerial imagery[END_REF] or any Structure-from-Motion (SfM) based metric [START_REF] Kowdle | Active learning for piecewise planar 3d reconstruction[END_REF]. In addition of being very costly, these features are methodologically redundant with the 3D modeling techniques: they are vulnerable to the same defects. Conversely, evaluation metrics used in the 3D building reconstruction literature (e.g., Root Mean Square Error (RMSE)) are too weak for such a complex task.

Geometric features

The model facet set is denoted by F. ∀(f, g) ∈ F × F f ∼ g correspond to facets f and g being adjacent: i.e., they share a common edge. As the roof topology graph in [START_REF] Verma | 3d building detection and modeling from aerial lidar data[END_REF], the input building model M can be seen as a facet (dual) graph:

M F, E {f, g} ⊂ F : f ∼ g . (1) f g d(f ) A (f ) C (f ) ||G (f ) -G (g)|| 2 arccos( n(f ). n(g))
S = S p hist : l → histogram(l, p), ( 2
)
with p standing for histogram parameters. A simpler option could be:

S = S synth : l → max(l) min(l) l median(l) σ(l) , (3)
where l (resp. σ(l)) represents the mean (resp. the standard deviation) over a tuple l.

These features are designed for general topological errors. For instance, over-segmentation may result in small facet areas and small angles between their normals. Conversely, an undersegmented facet would have a large area. Later on, the importance of these features will be discussed in details based on experimental results.

Each building M can consequently be characterized by a geometric feature vector that accounts for its geometric characteristics:

v geometric (M) =              S d(f ) f ∈F S A (f ) f ∈F S C (f ) f ∈F S ||G (f ) -G (g)|| (f,g)∈E S arccos( n(f ), n(g)) (f,g)∈E              . ( 4
)
Additionally to individual facet statistics, regularity is taken into account by looking into adjacent graph nodes as in [START_REF] Zhou | 2.5 D building modeling by discovering global regularities[END_REF]. Such features express a limited part of structural information. Taking this type of information into account would implicate graph comparisons which are not genuinely simple tasks to achieve. Since our objective is to build a baseline, this approach

has not yet been considered.

Height-based features

For this modality, raw depth information is provided by a DSM as a 2D height grid: dsm ∈ R w×h3 . It must have been produced around the same time of the 3D reconstruction so as to avoid temporal discrepancies. It is compared to the model height [START_REF] Brédif | 3D building reconstruction with parametric roof superstructures[END_REF][START_REF] Zebedin | Fusion of feature-and area-based information for urban buildings modeling from aerial imagery[END_REF]. The latter is inferred from its facets plane equations. It is then rasterized into the image:

alt ∈ R w×h at the same spatial resolution as dsm. Their difference generates a discrepancy map (Figure 1.c). A baseline approach is proposed relying on the statistics of pixel values computed using the S functions (Figure 5).

v height (M) = S dsm -alt .

(5) Equation 5 summarizes how building height-based features are computed. Different from a root mean square metric [START_REF] Lafarge | Creating large-scale city models from 3D-point clouds: a robust approach with hybrid representation[END_REF][START_REF] Poullis | A Framework for Automatic Modeling from Point Cloud Data[END_REF], the histogram captures the discrepancy distribution, which is particularly helpful in detecting undersegmentation defects or geometric imprecision. However, as for the previous geometric attributes, the grid structure of information coming from the model is lost. Errors cannot be spatialized and linked to a specific facet.

Image-based features

We aim to benefit from the high frequencies in Very High Spatial Resolution optical images.

Building edges correspond to sharp discontinuities in images [START_REF] Ortner | Building outline extraction from Digital Elevation Models using marked point processes[END_REF]. We verify this by comparing these edges to local gradients. We start by projecting building models on the orthorectified image I (Figure 6.a). For each facet, we isolate an edge s (Figure 6.b). In an ideal setting, gradients computed at pixels g that intersect s need to be almost be collinear with its normal n(s).

In consequence, applying the same statistics functions S, we compute the distribution of the cosine similarity between the local gradient and the normal all along that s: Once the distribution is computed over a segment, it is stacked over all facet edges to define the distribution over projected facets. In the case of histograms S p hist with the same parameters (and thus the same bins), it is equivalent to summing out the previous vectors D S p hist (s, I) over edges s from the projection q(f ) of the facet f . In order to take into account the variability of segment dimensions, this sum is normalized by segment lengths.

D S (s, I) S ∇I(g) • n(s) ∇I(g)
D S p hist (f, I) s∈q(f ) s • D S p hist (s, I). ( 7
)
The same can be done over all facets of a building M (Equation 8). The weights are added in order to take into account the geometry heterogeneity. The gradient to normal comparison is similar to the 3D data fitting term formulated in [START_REF] Li | Manhattan-world urban reconstruction from point clouds[END_REF]. Once again, the model structure is partially lost when simply summing histograms over all segments.

v image (M) = D S p hist (M, I) f ∈F A (q(f )) • D S p hist (f, I). ( 8
)
These image-based attributes are helpful for precision error detection. As example, facet imprecise borders can be detected as local gradients direction will be expected to differ greatly from the inaccurate edge. It can also be detrimental in under-segmentation detection as colors can change considerably from one facet or one building to another inducing an gradient orthogonal to edge normals.

Classification process

Two sources of flexibility are taken into account. First, the parametric nature of the taxonomy leads to a varying set of label. Secondly, the classifier should be able to handle the heterogeneity of the feature vector and must adapt to different input vectors types and sizes.

Classification problems.

We first define two terms used afterwards. In a multi-class classification problem, each instance has only one label that takes only one value amongst multiple ones (two in the case of a binary problem). The multi-label problem decides, for multiple labels, the most probable state: present (+1) or absent (-1). Both the classification problem nature and the label set are determined by the three previously defined taxonomy parameters (Table 1). Finesse = 1 level corresponds to the standard binary classification problem: ' 'Valid" or "Erroneous". At finesse = 2, the eLoD can then take two values in the aerial reconstruction case: LoD-1 or LoD-2. If set at LoD-1, it is a binary classification problem: "Valid" or "Building error". For LoD-2, if the exclusivity is on, it will be a multi-class problem: "Valid", "Building error" or "Facet errors". If set off, it becomes a multi-label one with the same labels. At finesse = 3 level, if the exclusivity is on, it is a 2-stage classification problem. In the first stage, a multi-class task predicts the error family, after which a second multi-label problem decides between the predicted error family children. If the exclusivity is off, it turns into 1-stage multi-label problem that predicts the existence of each atomic error corresponding to the chosen eLoD.

Classifier choice

The highly modular nature of the framework with multimodal features involving many parameters restricts the choice of classifiers. Random Forest classifiers [START_REF] Breiman | Random forests[END_REF][START_REF] Criminisi | Decision forests for computer vision and medical image analysis[END_REF] are selected. They can manage a large number of features with different dynamics and coming from multiple modalities. Relying on their bagging property, a high number of trees

(1, 000 elements) is necessary to cover most of the feature space, while a limited tree depth (4) helps avoiding overfitting during training. It adapts to any of our classification paradigm: multi-class or multi-label. In the latter case, a one-vs-all approach is adopted in addition so as to address each label separately.

Results

Dataset

3D models from three different cities of France are selected in order to assess the performance of our framework: Elancourt, Nantes, and the XIII th district of Paris (Paris-13) (Figure 6.i).

Elancourt is a small city exhibiting a high diversity of building types: residential areas (hipped roof buildings), and industrial districts (larger buildings with flat roofs). Nantes represents a denser urban setting with lower building diversity. In Paris-13, high towers, with flat roof, coexist with

Haussmann style buildings that typically exhibit highly fragmented roofs. The Elancourt (resp.

Nantes and Paris-13) scene contains 2,009 (resp. 748 and 478) annotated building models. The DSM and orthorectified image resolution is 6 cm while it is 10 cm for the two other areas.

3D models were generated using the algorithm described in [START_REF] Durupt | Automatic building reconstruction from a Digital Elevation Model and cadastral data: an operational approach[END_REF], out of existing building footprints and aerial VHR multi-view DSMs. The modeling algorithm simulates possible roof structures with facets satisfying some geometric constraints. The best configuration is selected using a scoring system on the extrapolated roofs. Finally, vertical building façades connect the optimal roof to the building footprint. These models have a LoD-2 level. This method is adapted to roof types of low complexity and favors symmetrical models (residential areas). It has been selected to ensure a varying error rate for the three areas of interest, especially since models were generated with partly erroneous cadastral maps. 3,235 buildings in total are considered. They were annotated according to the atomic errors list provided by our taxonomy. Figure 6.ii reports modeling errors statistics over the annotated buildings.

Experimental set-up

Four feature configurations were tested: "geometric features" (Geom.) only, "geometric and height features"(Geom. ∪ Hei.), "geometric and image features"(Geom. ∪ Im.) as well as "geometric, height and image features"(All.). Each feature modality generates a 20 dimension vector. The DSMs and orthorectified images used to derive height and image features have the same spatial resolution as the reconstruction input data. Labels are extracted from a non exclusive and eLoD = LoD-2 taxonomy. All finesse levels were tested. The overall accuracy is not interesting due to the highly unbalanced label distribution. We prefer reporting recall (Rec) and precision (P rec) ratios. Recall expresses, from a number of samples of a given class, the proportion that was rightfully detected as such. Precision indicates how much samples, amongst the detected ones, were, in truth, part of the studied class [START_REF] Powers | Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation[END_REF]. We also summarize these two ratios with their harmonic mean, the F-score.

Feature analysis

We assess the added value of each modality. Various feature configurations are studied. They are compared with a baseline consisting in predicting the errors using only the RMSE, which is the standard measure in most of 3D reconstruction methods. We conclude the analysis by studying the feature importance per training zone. All experiments are conducted performing a 10-fold cross validation to avoid overfitting/underfitting issues.

4 Geometrically inconsistent 3D models were filtered out in a preprocessing (nadir projection) step. This fraction corresponds only to the occluded (partially or completly) buildings that could not be qualified.

5 Valid means the absence of errors for a specified building

RMSE predictive capacity

We train the classifier on Elancourt with a one dimensional feature vector RM SE. Mean test results are shown in Table 2 We can conclude that the RMSE is not able to detect our errors. We can distinguish two clusters: high recall and low precision and overall accuracy (BOS, FOS and FIG) and low recall and precision (BUS, BIB, BIT, FUS, FIB and FIT). The first group consists of the most numerous errors (Figure 6.ii). This explains how the classifier assigns to almost all samples the positive class: we end up with a high ratio of false positives (false alarms) and hence a high recall ratio but coupled with a weak precision and overall accuracy. The inverse happens with the rest of the errors as we obtain a high percentage of false negative.

Feature ablation study

We tested the different feature configurations, at finesse level 3 and in all urban zones. Mean precision and recall test results are reported in Table 3. F-scores are averaged across all feature configurations and represented in Figure 8. We can first conclude that geometric features alone are generally sufficient. It is the best alternative for topological error detection as shown for BOS, FOS, FUS, FIT and BIT in Table 3. This is confirmed also by the low variance observed in Figure 8.(a). An exception is noticed with BUS in Elancourt, where height-based features allow an increase of around 9% in recall without, practically, any loss in precision. Similar behaviour is noticed for Nantes and Paris-13 with image-based features (+20% in recall). The first case can be explained by the discrepancy in height that can be observed between under-segmented buildings. The second is made clear by the difference in roof colors, in dense uniform settings (Figure 3.i.a). This helps identifying different instances of buildings.

Figure 8 shows all "Building errors" family labels are better detected for Elancourt. It is also the case of FOS and FIT. A certain monotony can be noticed, at the exception of BOS. Better resultas are obtained for Paris-13 than for Nantes, while having around half the number of models to train on. This means that BOS cannot be easily learnt in Nantes. It is coherent with the fact that the dataset represents a part of the dense downtown of the city. The same monotony is observed, this time in reverse, with the rest of "Facet errors" defects. Paris-13 is much better with less training samples. For geometric defects (FIG and FIB), Nantes is comparable to Paris-13, but, with FUS, it is way much worse. This may result from the highly heterogeneous aspect of this dataset that encompasses high tower buildings with a densely populated city district. FInally, well represented errors are more easily detected than the less frequent ones, especially the rare ones like FIT in Nantes and Paris-13.

Feature importance

Random forest classifiers can easily infer feature importance at training time. These were here computed and aggregated by modality in all urban scenes (Figure 9). At first, we observe how much individual attributes are important before being gathered. For geometric features, all attributes are equally important. However, concerning image-and height- based features, only a few are relevant (higher feature importance ratio). Indeed, these few attributes correspond to the highest and lowest values of the histograms. As described earlier, image and height features consist of a histogram of distances between the model and the real measured signals: vector cosine similarity, for the first, and the L 2 norm for the last. It is clear that the presence of errors would result in saturating the high values in the histogram, while an absence of defects would imply a big number of low values. This intuitively explains the observed phenomenon.

In a second time, we notice that no modality is more important than the others, contrarily to what was observed in Table 3. In fact, for most atomic errors, test results using geometric features are comparable to those obtained with more modalities. However, during training, all modalities are relevant (∼1/3 in Figure 9). This explains why all configurations are kept for subsequent analysis.

Scalability analysis

It is established that the scene composition can affect greatly model defect detection. This fact motivates studying training the classifier and testing prediction on different scenes. The goal is to prove the resilience of the prediction to unseen urban scenes. As the annotation process require a lot of effort, this trait is crucial to guarantee the scalability of this method. Different configurations are possible (Figure 10). In a first experiment, we train on one urban scene and test on another one (transferability of the classifier model). In a second configuration, the classifier is trained on two scenes and tested on the last one: the goal is to investigate generalization. The last experiment targets the representativity of a single 3-area dataset by trying multiple train-test splits. We will see how "Building errors" depend on the training scene, in contrast to "Facet errors".

The latter will prove to be more transferable and generalizable than the first one. We will also discuss how every modalities play a role in error prediction. Image-based features will demonstrate to be the most valuable compared to height-based ones. Eventually, we will review each atomic error prediction sensitivity provided the training set.

Transferability study

In this configuration, we test how transferable are the learned classifiers from one urban scene to another. We train on a zone Z i and test on another one Z j . We will denote each transferability experiment by the couple (Z i , Z j ) or by Z i → Z j . Six transferability couples are possible. F-scores are shown, per label, and per experiment, in Figure 11.

First, a coherence analysis is performed. We compare the results of the transferability experiments to the ablation results with the same training scene (for a given area Z i in all couples (Z i , Z j ) ∀j =i , differences between Figure 11 and Table 3/Figure 8). Secondly, we investigate how an urban scene composition helps predicting defects in an unseen one. This is called the projectivity comparison. For a given test scene Z j in couples (Z i , Z j ) ∀i =j , we compare results from Figure 11 with Table 3

Generalization study

We try to find out how omitting one urban zone from the training dataset affects the test results

on that same area. An other way to look at it is, from an operational point of view, to find out how much learning on a union of many urban scenes is helpful when testing on an unseen one. We also We compare these experiments with the ablation study on the same area (cf. Table 4). We analyse result along the same criterea as the transferability study.

We start again with a comparison depending on error families. Out of the 11 possibilities for the "Building errors" family, 8 yield worse results. For the "Facet" family, 6 out of 13 comparisons exhibit the same trend. This is worst than the transferability comparisons in ratio. This results from the fact that fusing more datasets, that are not tailored for a specific error detection, does not Similarly to the previous study, image and height modalities play a major role in error detection.

Image-based features are crucial for FIB, BIB, FUS and BUS detection (Table 4). Height-based attributes, however, induce a larger impovement in predicting FIG and BIT, while geometric ones are relegated to playing a minor role. Otherwise, a curiosity can be noticed: only when fusing all modalities together, in Paris-13 and Nantes, does predictions improve for BOS.

We also confirm the observations about the best urban scene for error prediction training. In this case, the best zone should always give the worst scores. It is mostly the case with all atomic errors, with the exception of BIT. This outlier can be explained by the resemblance of the Paris-13 3D model to Nantes (which was established to be the best) samples. Indeed, for most labels, Nantes and Paris-13 reach the same scores. However, the discrepancy in F-scores proves the added value for each dataset.

Representativeness study

The objective is to find out, after merging all training samples from all datasets, what is the minimal amount of data that can guaranty stable predictions. We can, thereafter, understand how much learning on one scene typology can affect results compared to a mixed training set. Figure 13 depicts F-score as a function of training ratios (between 20-70%) and atomic errors.

We note the high stability of the F-score. This indicates that having a small heterogeneous dataset is not detrimental to the learning capacity and can be even the most suitable solution. ensured with a limited training set. No standard logarithmic behaviour can be found at the studied scales. 20% of the full label set is sufficient so as to retrieve results with a performance similar to the initial ablation study. The best results are observed for BOS, BUS, and FUS. These errors are topological defects of building roof facets which require a high diversity of training samples for their detection. More sophisticated features are however still required to help predicting less frequent and more semantic labels.

Finesse study

In this section, we reproduce the experimental settings described in Sections 4 and 5. This time, the finesse level is fixed at 2. The goal is to find out how good, transferable and stable are the model quality predictions at the semantic level of error families (i.e., "Building errors" vs. "Facet errors").

Error family detection

We start by the ablation study. Table 5 reveals that inserting more remote sensing modalities do not change the prediction results dramatically. This is perfectly illustrated by the low variance in Figure 14 a for the three areas of interest. These results are in line with the conclusions at the finesse level 3. In the higher finesse level, only BUS, from all atomic errors, was highly impacted by a change in the feature configuration. This may explain the observed low variability. We also note the prevalence of FOS errors (between 60% and 85%) and FIG (between 70% and 95%) in the "Facet errors" family. This added to the fact that they are, in a large capacity, easily detected individually (>90%, in F-score, for the first and 80% for the second, see Figure 8b) helps understanding why the F-score reaches at least 90% for this family (Figure 14a). As with finesse level 3 experiments, "Facet errors" yields higher prediction scores than on "Building errors". Indeed, we can see a smaller discrepancy between F-scores on different scenes for "Facet errors"' (below 5%) than for "Building errors" (15%).

The transferability study (Figure 14b) compares the F-scores with the ablation study provided in Figure 14a. Out of all 12 possible comparisons, only 2 exhibit a decrease in error discrimination.

Both affect the "Building errors" family when trained on Nantes. "Facet errors", on the other hand, confirms, its transferability and stability (less than 5% of discrepancy between the two extremal values). For this reason, we skip the generalization study, all together, at this section.

The representativeness study conducted for the finesse level 2 results in the F-scores that are illustrated in Figure 14c. Family detection scores are very stable across all different tested split ratios. Moreover, in contrast to atomic errors results (cf. Figure 13), F-scores do not vary by more than 1% in mean and standard deviation. This proves that at finesse level 2, error family prediction is evened out independent of different split ratios, as opposed to higher order errors.

Again, it benefits from the higher heterogeneity of the training set with multiple areas.

Detection of erroneous models

Now, we work at finesse level 1, first on feature ablation. Since valid samples are very rare in our case, it is expected that it will be very difficult to detect these instances. In consequence, in Table 6, we choose to report correctly "Valid" buildings instead of computing the precision score in percentage.

At this level, even more that the error family semantic degree, feature configurations have virtually no impact on test results: Elancourt was the only exception when image features are added to geometric ones. Furthermore, we confirm expectations as, at most, only 1 out of 57 (resp.

0 out of 55 and 3 out of 21) valid instances are detected for Elancourt (resp. Nantes and Paris-13).

As a consequence, we do not report the rest of previously conducted experiments for this finesse level. Indeed, it is senseless to compare detection transferability, generalization or representativeness if we hardly detect them at all on the same training scene.

Conclusion

A learning framework was proposed to semantically evaluate the quality of 3D models of buildings. For that purpose, errors were hierarchically organized into a novel flexible taxonomy. It aims to handle the large diversity of urban environments and varying requirements stemming from endusers (geometric accuracy and level of details). Based on the desired LoD, exclusivity and semantic level, an error collection is considered. Model quality is then predicted using a supervised Random Forest classifier. Each model provides intrinsic geometrical characteristics that are compiled in a handcrafted feature vector. Remote sensing modalities can be introduced. This helps better describing building models and detecting errors. Attributes can indeed be extracted by comparing the 3D model with optical images or depth data at the spatial resolution at least similar to the input 3D model. Experiments shows it helps detecting hard cases both for geometrical and topological errors.

This new framework was applied to the case of aerial urban reconstruction, where features are extracted from Very High Resolution airborne images and a DSM. A fully annotated dataset con-taining 3,235 aerial reconstructed building models with high diversity and from three distinct areas was used to test our method. It was associated with multimodal RGB optical and Digital Surface Model features. Although being mitigated over under-represented errors, results are satisfactory in the well balanced cases. More importantly, we proved that the urban scene composition affects greatly error detection. In fact, some predictions scores are not only stable, when training on a different urban scene, they even outperform when learning on the same scene. Additionally, we reported how, for a heterogeneous training dataset, the size of the training set have, practically no effect, as test score stay stable for all errors. This demonstrates that the proposed framework can be easily scaled with the right choice of training samples with little manually generated data.

This exactly answers to the raised problematic, contrarily to the present state-of-the-art literature.

As a next step, more structure-aware features (based on graph comparison, for instance) could be proposed [START_REF] Boguslawski | Modelling and analysing 3d buildings with a primal/dual data structure[END_REF]) so as to be applied on a richer and more diverse dataset (potentially involving data augmentation) under a deep-based framework.

Figure 1 :

 1 Figure 1: Our semantic evaluation framework for 3D building models (a). Semantic errors affecting the building are predicted using a supervised classifier and handcrafted features. In addition to the input model topological structure (b), features are extracted from Very High resolution overhead data. It can be based on a comparison with the Digital Surface Model (DSM) (c). Optical images can also be used through, for instance, local gradient extraction (d). Several errors can be detected at the same time, in a hierarchical manner (e). Fidelity errors correspond to geometrical imprecision as shown in red. On the other hand, modeling errors denote morphological inconsistancies with the real object.

Figure 2 :

 2 Figure 2: The proposed taxonomy structure. In our case of very high resolution overhead image modeling, only two family errors are depicted. At finesse level 2, hierarchization is possible: the exclusivity parameter can thus act.

  BOS (c) BIB (d) BIT i. Building errors family samples.

  (a) FUS (b) FOS (c) FIB (d) FIT (e) FIG ii. Facet errors family samples.

Figure 3 :

 3 Figure 3: Illustration of various errors of our taxonomy. One can see that geometric, spectral and height information are required for an accurate detection of all kinds of errors.

  Figure 4: Computed geometric attributes represented on the dual graph, for facets f and g. The green vector groups the node (facet) attributes while the blue one shows the edge features.

Figure 5 :

 5 Figure 5: Histogram height-based features computed from the DSM residuals. (a) DSMs. (b) Height maps extracted from the 3D model. (c) Difference between (a) and (b). The difference is transformed into a vector using a histogram (d).

Figure 6 :

 6 Figure 6: Illustration of how features are derived from optical images. Model facets (each represented by a specific color) are projected onto the aerial image (a). (b) local gradients (in purple), on intersecting pixels (in green), are compared to the edge (in red) normal (in black).

  Selection of two areas of interest in the three datasets. Statistics: in (a) Unqualifiable ∪ finesse = 2 statistics are represented, while (b) illustrates "Building errors" and (c) "Facet errors".

Figure 7 .

 7 Figure 7. Statistics per urban scene and error type. Almost similar situations can be noticed.

  Figure 8. Mean F-score and standard deviation for the feature ablation study.

Figure 9 .

 9 Figure 9. Modality importance computed by stacking single feature importances retrieved from the Random Forest classifier. The first (resp. second and third) column represents Elancourt (resp. Nantes and Paris-13).

  Figure 10. A graph representing possible experiments: arrow origins represent training scenes while test ones are depicted as targets. Zi, i = 1, 2, 3 represent the urban zones. All these nodes are assembled in one, meaning that all urban scenes were aggregated in on train/test node. The numbers indicate in which section each experiment is analyzed.

  Figure 12.

  Figure 13. Mean F-score and standard deviation for the representativeness experiments depending on the training set size.

Figure 14

 14 Figure 14. F-score mean and standard deviation for the feature ablation study outcomes per zone for finesse level 2. (a) corresponds to the ablation study, (b) to the transferability experiments and (c) to the representativity setting.

Table 1

 1

	: The summary
	of all possible classifi-
	cation problem types.
	children(error) lists the
	children of error from
	the taxonomy tree (Fig-
	ure 2).

Table 3 .

 3 Feature ablation study preformed on the three areas at finesse level 3. Test results are expressed in percentage. All atomic errors are considered over all possible configurations.

					Elancourt
		Geom.	Geom. ∪ Hei.	Geom. ∪ Im.	All
		Rec	P rec	Rec	P rec	Rec	P rec	Rec	P rec
	BOS	93.96	76.15	91.43	77.76	91.51	76.08	90.83	76.14
	BUS	32.98	76.47	41.86	75.57	40.38	71.00	39.32	71.81
	BIB	12.32	67.57	12.81	68.42	16.26	67.35	16.75	68.0
	BIT	25.25	92.59	20.20	90.91	20.20	95.24	11.11	91.67
	FOS	98.91	99.07	98.91	99.30	98.99	98.84	98.91	98.84
	FUS	1.90	54.55	0.63	66.67	1.61	50	1.27	66.67
	FIB	9.17	87.5	0	-	8.30	82.61	7.42	100
	FIT	6.67	100	8.73	95.24	3.33	100	3.33	100
	FIG	80.54	73.14	80.45	72.62	78.69	72.12	79.02	71.82
					Nantes
		Geom.	Geom. ∪ Hei.	Geom. ∪ Im.	All
		Rec	P rec	Rec	P rec	Rec	P rec	Rec	P rec
	BOS	38.14	61.67	36.43	60.23	36.77	62.21	34.71	60.48
	BUS	7.35	62.5	7.35	55.56	29.41	66.67	26.47	64.29
	BIB	0	-	0	-	1.01	50.0	1.01	50.0
	BIT	1.77	22.22	3.54	44.44	0	0	2.65	50.0
	FOS	98.54	98.13	98.54	98.13	98.33	97.92	98.12	97.91
	FUS	27.62	55.24	27.62	59.18	24.76	54.74	23.33	53.85
	FIB	37.80	62.0	36.59	63.16	49.39	60.90	46.39	60.90
	FIT	0	-	0	-	0	-	0	-
	FIG	86.32	78.09	86.77	78.02	84.53	78.71	83.86	78.08
					Paris-13
		Geom.	Geom. ∪ Hei.	Geom. ∪ Im.	All
		Rec	P rec	Rec	P rec	Rec	P rec	Rec	P rec
	BOS	45.54	65.25	46.53	68.61	50.0	68.24	46.53	70.15
	BUS	6.35	66.67	7.94	71.43	22.22	77.78	7.94	62.5
	BIB	0	-	0	-	0	0	0	-
	BIT	2.63	50.0	0	-	1.32	50.0	0	0
	FOS	97.19	97.19	97.19	97.19	97.59	98.38	97.19	97.19
	FUS	85.09	75.0	84.36	74.12	85.09	74.52	84.36	74.12
	FIB	53.47	62.10	51.39	61.67	53.47	63.11	52.78	61.79
	FIT	0	-	0	-	0	-	0	-
	FIG	97.65	84.62	98.96	84.79	97.65	84.62	98.96	84.79

  /Figure8. Analysis is provided in Table4. In both settings, if a feature type appears, it means it is, by a large margin, the most decisive one. A color scheme was devised to encode the amplitude of change. All various feature configurations are tested these experiments. If a modality stands out, in terms of the F-score, it is mentioned in the corresponding cell in Table4.To summarize the comparisons, error family wise, out of 22 "Building errors" possible projectiv-topology errors (FIT and BIT). Paris-13 represents also a dense downtown scene but with even more diverse building types. This is instrumental to achieve transferability for BUS and BIB. Conversely, Elancourt offers more heterogeneity on the LoD-2 level. As a consequence, it is the best training zone for FUS, FIB and FIG. Finally, as one can obviously suspect, FOS learning is evenly transferable, as it is well detected when training on any scene.

Table 5

 5

	. Feature
	ablation study on the
	three datasets for the
	finesse = 2 case.
	Results are expressed
	in percentage.

Parameters determine the considered errors

There is a problem of slope. The model corresponds to a flat roof whereas in reality the slope is ca. 25 • . The error could only be shown if we provided the height residual. However, for the sake of homogeneity, we only provided orthoimages as background. It motivates also the need for a height-based modality.

w (resp. h) is the grid width (resp. height)

ity comparisons, 14 yield worse results. This proves how hard it is, for this error family, to transfer learned classifiers. It is, however, the contrary for the "Facet errors". Only 8 out of 27 projectivity errors are worse than training on the same test area. Elancourt → Paris-13 Im.

Im.

Nantes → Paris-13 Geom.

Hei.

Nantes → Elancourt Geom.

Paris-13 → Nantes Geom.

Paris-13 → Elancourt Geom.

Projectivity

Elancourt → Nantes Im.

Elancourt → Paris-13 Im.

Im.

Nantes As mentioned earlier, additional modalities play a important role in prediction accuracy. We start with image-based attributes. In some cases, they were pivotal in obtaining better results

for geometric errors (FIB, BIB), as well as for topological ones (FUS, FIT). These features have a significant coherence power when trained over Elancourt (FIB and FUS), and projects very well to other scenes (FIB, FUS, BIB and FIT, Table 4). On the other hand, as expected, geometric features alone are best for topological errors, when trained on dense areas, especially BIT (Table 4). Finally, although stiking out for FIG in a minor capacity (cf. Table 4), height-based features proved to be less transferable. In fact, adding height-based features leads, in most cases, to a small decrease in accuracy (≈ 2%) for atomic errors. All these previous findings further justify why we did not leave out any modality, as they are more frequently critical for transferability than in the ablation study (Table 3).

An analysis can also be drawn for atomic errors with respect to the best training scene. We can see that for BOS, training on a dense urban scene like Nantes, is the best solution, as for


Benoît Pilmis

Alexandre Alanio

Olivier Lortholary

Fanny Lanternier
email: fanny.lanternier@aphp.fr

Benoit Pilmis

Recent advances in the understanding and management of mucormycosis

Keywords: Mucormycosis, Reverse Halo sign, PCR, Isavuconazole, Liposomal amphotericin B

Introduction

Mucormycoses are life-threatening fungal infections mostly occurring in hematology, solid organ transplant, or diabetic patients, it may also affect immunocompetent patients following a trauma or burn [START_REF] Roden | Epidemiology and outcome of zygomycosis: a review of 929 reported cases[END_REF] . Nosocomial or community outbreaks have been described [START_REF] Alanio | Investigating Clinical Issues by Genotyping of Medically Important Fungi: Why and How?[END_REF] . Mucormycosis is characterized by host tissue infarction and necrosis resulting from vasculature invasion by hyphae starting with a specific interaction with endothelial cells. Most common clinical presentations are rhino-orbito-cerebral and pulmonary. Multicenter and single-center studies have reported an increasing incidence probably due to an increase in the at-risk population and improved diagnostic tools [START_REF] Bitar | Increasing incidence of zygomycosis (mucormycosis), France, 1997-2006[END_REF][START_REF] Guinea | Increasing incidence of mucormycosis in a large Spanish hospital from 2007 to 2015: Epidemiology and microbiological characterization of the isolates[END_REF] . In a French study, mucormycosis incidence increased by 7.3% per year, especially in patients with neutropenia [START_REF] Bitar | Population-based analysis of invasive fungal infections, France, 2001-2010[END_REF] . These infections are difficult to manage for several reasons [START_REF] Ribes | Zygomycetes in human disease[END_REF] . Firstly, diagnosis is difficult because of clinico-radiological similarities with invasive aspergillosis and historical lack of diagnostic tools. However, new tools in serum and tissue as well as the recognition of highly suggestive radiological signs recently modified diagnostic possibilities. Secondly, treatment is an emergency and combines surgery, which is frequently required owing to the angioinvasive and necrotic character of infection [START_REF] Katragkou | Why is mucormycosis more difficult to cure than more common mycoses?[END_REF] , and antifungal treatment. Primary in vitro resistance to several antifungal drugs limits therapeutic options [START_REF] Dannaoui | Antifungal resistance in mucorales[END_REF] . However, recent data enlarge the antifungal armamentarium with the US Food and Drug Administration's and European Medicines Agency's approval of the new triazole isavuconazole. However, comparative clinical data are lacking, and the respective places of polyenes and different azoles need to be discussed.

Moving towards improved mucormycosis understanding

Human mucormycoses are caused by a wide range of pathogenic species. Mucormycosis location is linked to the mucorales species; Rhizopus arrhizus (Figure 1) is present in 85% of rhino-cerebral forms, compared with only 17% of non-rhinocerebral forms in the French RetroZygo study [START_REF] Lanternier | A global analysis of mucormycosis in France: the RetroZygo Study (2005-2007)[END_REF] . This finding could be explained by virulence differences between Mucorales species. In experimental studies, ketoacidosis has been found to predispose mice to Rhizopus spp. but not Lichtheimia spp. infection [START_REF] Ibrahim | Pathogenesis of mucormycosis[END_REF][START_REF] Spellberg | Novel perspectives on mucormycosis: pathophysiology, presentation, and management[END_REF] . In parallel, corticosteroid treatment enhanced the susceptibility of mice to lung infections caused by Lichtheimia corymbifera or Lichtheimia ramosa [START_REF] Ibrahim | Pathogenesis of mucormycosis[END_REF][START_REF] Spellberg | Novel perspectives on mucormycosis: pathophysiology, presentation, and management[END_REF] .

Mucormycosis' clinical presentation is also related to underlying conditions. Rhino-cerebral mucormycosis is the most common form in patients with diabetes mellitus 1 , while pulmonary mucormycosis occurs most often in patients with hematological malignancies [START_REF] Tedder | Pulmonary mucormycosis: results of medical and surgical therapy[END_REF] . Radiological findings in patients with pulmonary mucormycosis are also related to immunological status [START_REF] Nam | Pulmonary mucormycosis: serial morphologic changes on computed tomography correlate with clinical and pathologic findings[END_REF] . Although unusual, lately there have been diagnoses made in the gastrointestinal system. The stomach is more frequently involved, and then the colon. Symptoms are abdominal pain and gastro-intestinal bleeding [START_REF] Dioverti | Gastrointestinal mucormycosis in immunocompromised hosts[END_REF] . Diagnosis is suspected on endoscopic findings with necrotic lesions that can lead to perforation and peritonitis [START_REF] Singh | Invasive gastrointestinal zygomycosis in a liver transplant recipient: case report and review of zygomycosis in solid-organ transplant recipients[END_REF] .

Mucorales can gain entry to a susceptible host through inhalation, ingestion of contaminated food, or abraded skin. These routes result in rhino-orbito-cerebral, pulmonary, gastrointestinal, or cutaneous/wound infections. One of the characteristic features of mucormycosis is its angioinvasive property, resulting in vascular thromboses and ultimately tissue necrosis. Ketoacidosis and deferoxamine are known to predispose to mucormycosis, revealing the importance of hyperglycemia, iron, and acidifying ketone bodies in mucorales virulence. Angioinvasion was reported to be related to the interaction between a spore-coating protein family (CotH) on Rhizopus spp. surface and endothelium glucose regulator protein 78 (GRP78) expressed at the surface of endothelial cells. This interaction triggers host cell injury and subsequent fungus hematogenous dissemination [START_REF] Baldin | Molecular mechanisms of mucormycosis-The bitter and the sweet[END_REF] . Elevated levels of serum glucose, iron, and ketone bodies increase fungal growth and induce the expression of GRP78 and CotH, resulting in increased ability of Rhizopus to invade host tissues and explaining the susceptibility of diabetic and deferoxaminetreated patients to mucormycosis. However, it should be noted that the majority of studies on virulence and the association between ketoacidosis and the occurrence of mucormycosis have been conducted with Rhizopus species [START_REF] Spellberg | Mucormycosis pathogenesis: Beyond Rhizopus[END_REF] . Decreased numbers and impaired function of monocytes and neutrophils are important mucormycosis risk factors, since they are known to inhibit Mucorales spore germination. This includes patients with hematological disorders, AIDS, or liver cirrhosis, those who have undergone solid organ transplant, and those being treated with high-dose steroids [START_REF] Xhaard | Mucormycosis after allogeneic haematopoietic stem cell transplantation: a French Multicentre Cohort Study (2003-2008)[END_REF][START_REF] Pana | Invasive mucormycosis in children: an epidemiologic study in European and non-European countries based on two registries[END_REF] . Finally, victims of natural disaster are also at risk [START_REF] Benedict | Invasive fungal infections after natural disasters[END_REF] owing to wounds contaminated with water, soil, or debris [START_REF] Ivers | Infectious diseases of severe weather-related and floodrelated natural disasters[END_REF] , such as after the 2004 Indian Ocean tsunami [START_REF] Andresen | Multifocal cutaneous mucormycosis complicating polymicrobial wound infections in a tsunami survivor from Sri Lanka[END_REF] or after the 2011 Missouri tornado [START_REF] Fanfair | Necrotizing cutaneous mucormycosis after a tornado in Joplin, Missouri, in 2011[END_REF] .

Computed tomography

The most common radiological pattern of lung mucormycosis on initial computed tomography (CT) scan is a halo sign and then nodule or mass [START_REF] Nam | Pulmonary mucormycosis: serial morphologic changes on computed tomography correlate with clinical and pathologic findings[END_REF][START_REF] Legouge | The reversed halo sign: pathognomonic pattern of pulmonary mucormycosis in leukemic patients with neutropenia?[END_REF] . However, when studied very early and on serial follow-up, sequential morphologic changes could be observed as (i) reversed halo sign (Figure 2) followed by (ii) consolidation or nodule or mass with halo sign and, finally, (iii) central necrosis and air-crescent sign. For pulmonary mucormycosis, a recent study showed that there was a significant increase in the prevalence of reversed halo sign in neutropenic (79%) and non-neutropenic (31%) patients (P <0.05) [START_REF] Bourcier | Prevalence of the reversed halo sign in neutropenic patients compared with non-neutropenic patients: Data from a single-centre study involving 27 patients with pulmonary mucormycosis (2003-2016)[END_REF] .

Major steps in mucormycosis: towards earlier diagnosis (Figure 3)

Mucormycosis diagnosis is challenging, as it is associated with high mortality, especially in hematological patients. Early distinction from invasive aspergillosis is of utmost importance, as antifungal treatment may differ, whereas underlying conditions and clinical presentation are often similar. Until recently, mucormycosis diagnostic tools were based on limited basic microbiology and frequently led to diagnosis delay. Unlike invasive aspergillosis, the detection of circulating antigen such as galactomannan and β-D-1,3-glucan provides no help for mucormycosis diagnosis. Therefore, samples from the infection site are highly required to diagnose mucormycosis based on the microscopic detection of typical hyphae or on a positive culture [START_REF] Garcia-Hermoso | Agents of Systemic and Subcutaneous Mucormycosis and Entomophthoromycosis[END_REF] . Recently, the development of molecular biology tools has allowed the non-invasive diagnosis of mucormycosis. Million et al. designed a quantitative multiplex polymerase chain reaction (qPCR)-based 18S rRNA targeting Mucor/Rhizopus, Lichtheimia, and Rhizomucor. This PCR assay was evaluated with the aim to detect Mucorales DNA early in the course of the infection in the blood (serum) [START_REF] Millon | Early diagnosis and monitoring of mucormycosis by detection of circulating DNA in serum: retrospective analysis of 44 cases collected through the French Surveillance Network of Invasive Fungal Infections (RESSIF)[END_REF] . The authors were able to detect Mucorales DNA in serum samples from 90% of patients up to three days before mucormycosis diagnosis [START_REF] Millon | Early diagnosis and monitoring of mucormycosis by detection of circulating DNA in serum: retrospective analysis of 44 cases collected through the French Surveillance Network of Invasive Fungal Infections (RESSIF)[END_REF] . Negative serum PCR was also associated with better outcome as compared to patients with persistently positive PCR. Furthermore, a study among severely ill burn patients found that circulating Mucorales DNA was detected 11 (4.5-15) days before standard diagnosis for invasive wound mucormycosis [START_REF] Legrand | Detection of Circulating Mucorales DNA in Critically Ill Burn Patients: Preliminary Report of a Screening Strategy for Early Diagnosis and Treatment[END_REF] . Other studies have evaluated the use of real-time PCR targeting Mucorales on tissue or respiratory samples in patients with hematological malignancy suffering from proven and probable mucormycosis [START_REF] Alanio | Molecular identification of Mucorales in human tissues: contribution of PCR electrospray-ionization mass spectrometry[END_REF][START_REF] Springer | A Comparison of Aspergillus and Mucorales PCR Testing of Different Bronchoalveolar Lavage Fluid Fractions from Patients with Suspected Invasive Pulmonary Fungal Disease[END_REF][START_REF] Springer | Clinical evaluation of a Mucoralesspecific real-time PCR assay in tissue and serum samples[END_REF][START_REF] Zaman | Molecular diagnosis of rhino-orbitocerebral mucormycosis from fresh tissue samples[END_REF] .

Consequently, it is currently necessary in patients with hematological malignancies to include the value of reverse halo sign on CT combined with serum qPCR targeting Mucorales in the early diagnosis of pulmonary mucormycosis [START_REF] Caillot | Is It Time to Include CT "Reverse Halo Sign" and qPCR Targeting Mucorales in Serum to EORTC-MSG Criteria for the Diagnosis of Pulmonary Mucormycosis in Leukemia Patients?[END_REF] .

Several antifungals now available for mucormycosis treatment

Mucormycoses: an indication of emergency surgery Current guidelines recommend antifungal treatment, surgical debridement, and correction of risk factors [START_REF] Tissot | ECIL-6 guidelines for the treatment of invasive candidiasis, aspergillosis and mucormycosis in leukemia and hematopoietic stem cell transplant patients[END_REF][START_REF] Cornely | ESCMID and ECMM joint clinical guidelines for the diagnosis and management of mucormycosis 2013[END_REF] . Surgical debridement has to be extensive, involving all necrotic areas for rhino-oculo-cerebral infection, and repeated surgical procedures are recommended to achieve local control and improve outcome [START_REF] Vironneau | Local control of rhino-orbito-cerebral mucormycosis dramatically impacts survival[END_REF] . For pulmonary mucormycosis, the indication and timing of surgical management outside emergency care (hemoptysis) is still unclear [START_REF] Chretien | Emergency and elective pulmonary surgical resection in haematological patients with invasive fungal infections: a report of 50 cases in a single centre[END_REF] . In a European series of 230 patients, surgical treatment reduced mortality by 79% [START_REF] Skiada | Zygomycosis in Europe: analysis of 230 cases accrued by the registry of the European Confederation of Medical Mycology (ECMM) Working Group on Zygomycosis between 2005 and 2007[END_REF] , leading to discuss surgery when feasible for any localization, however mandatory for rhino-cerebro-oculo-cerebral and posttraumatic mucormycosis [START_REF] Vironneau | Local control of rhino-orbito-cerebral mucormycosis dramatically impacts survival[END_REF][START_REF] Lelievre | Posttraumatic mucormycosis: a nationwide study in France and review of the literature[END_REF] .

Antifungal treatment

Amphotericin B (Amb) and its lipid formulations and posaconazole were the only antifungal drugs available with in vitro activity against mucorales [START_REF] Sabatelli | In vitro activities of posaconazole, fluconazole, itraconazole, voriconazole, and amphotericin B against a large collection of clinically important molds and yeasts[END_REF][START_REF] Almyroudis | In vitro susceptibilities of 217 clinical isolates of zygomycetes to conventional and new antifungal agents[END_REF] . The antifungal armamentarium recently enlarged with the development of isavuconazole. The first-line recommended antifungal agent is liposomal Amb (L-Amb) or Amb lipid complex (ABLC) [START_REF] Cornely | ESCMID and ECMM joint clinical guidelines for the diagnosis and management of mucormycosis 2013[END_REF] . Studies in mice proposed that the efficacy of L-Amb and ABLC was dependent on the dose given and that 10 mg/kg yielded the best outcomes. A prospective French phase II multicenter study (AmBizygo trial) evaluated the efficacy and tolerance of high-dose (10 mg/kg/day) L-Amb in association with surgery when recommended for the treatment of 34 mucormycosis cases. A favorable response was seen in 45% of patients at week 12. However, serum creatinine doubled in 40% of patients, but in 63% of cases, once treatment had ended, creatinine levels normalized after three months [START_REF] Lanternier | Prospective pilot study of high-dose (10 mg/kg/day) liposomal amphotericin B (L-AMB) for the initial treatment of mucormycosis[END_REF] . According to this study, ECMM/ESCMID and ECIL-6 guidelines recommend the use of L-Amb with a daily dosage of at least 5 mg/kg/day for mucormycosis [START_REF] Tissot | ECIL-6 guidelines for the treatment of invasive candidiasis, aspergillosis and mucormycosis in leukemia and hematopoietic stem cell transplant patients[END_REF][START_REF] Cornely | ESCMID and ECMM joint clinical guidelines for the diagnosis and management of mucormycosis 2013[END_REF] , and dosages at 10 mg/kg/day are strongly supported by ECMM/ ESCMID for cerebral infections [START_REF] Cornely | ESCMID and ECMM joint clinical guidelines for the diagnosis and management of mucormycosis 2013[END_REF] . Moreover, because of better diffusion, L-Amb should be favored in central nervous system infections.

The duration of the first-line antifungal treatment is still a matter of debate and should be determined on an individual basis and adjusted based on the underlying condition. Some authors proposed a lipid Amb treatment for at least three weeks, and, when there is clinical and radiological improvement, a consolidation by posaconazole can be started [START_REF] Kontoyiannis | How I treat mucormycosis[END_REF] . However, it could possibly be guided by negative PCR and therefore shortened for some patients.

Isavuconazonium sulfate is a water-soluble pro-drug, which is quickly hydrolyzed to the triazole isavuconazole after oral or intravenous administration. Isavuconazole has high oral bioavailability, linear pharmacokinetics, and a broad antifungal spectrum. The in vitro activity of isavuconazole minimum inhibitory concentration (MIC) ranges were 0.125 to 4 mg/L across L. corymbifera, L. ramosa, Rhizomucor pusillus, Rhizomucor microspores, and R. arrhizus but somewhat higher against Mucor circinelloides (1 to 16 mg/L). The MICs were in general one-to three-fold higher than those for posaconazole [START_REF] Lamoth | Changing Epidemiology of Invasive Mold Infections in Patients Receiving Azole Prophylaxis[END_REF] . In the recently published Vital study, 21 patients were treated with isavuconazole as first-line treatment; 42-day response rate was only 14% and week 12 response was 10% (compared to 45% in the AmBizygo study) with 43% deaths [START_REF] Marty | Isavuconazole treatment for mucormycosis: a single-arm open-label trial and case-control analysis[END_REF] . The results of this study found a mortality rate at day 42 comparable to that observed in the AmBizygo study [START_REF] Lanternier | Prospective pilot study of high-dose (10 mg/kg/day) liposomal amphotericin B (L-AMB) for the initial treatment of mucormycosis[END_REF] . In the VITAL study, isavuconazole was well tolerated and toxic effects were an uncommon cause of discontinuation. The place of isavuconazole has not yet been specified in the most recent guidelines [START_REF] Tissot | ECIL-6 guidelines for the treatment of invasive candidiasis, aspergillosis and mucormycosis in leukemia and hematopoietic stem cell transplant patients[END_REF] . Finally, a costeffectiveness study demonstrated the positive economic impact of the use of isavuconazole compared to Amb in the treatment of mucormycosis [START_REF] Bagshaw | The cost of treating mucormycosis with isavuconazole compared with standard therapy in the UK[END_REF] .

Posaconazole has been shown to have in vitro and in vivo activity against mucorales, but there are no data for the use of first-line posaconazole therapy. Posaconazole, therefore, finds its place in the therapeutic armamentarium for prophylaxis or consolidation after induction treatment with L-Amb. No study on the efficacy of posaconazole intravenous or tablet formulations in mucormycosis treatment were conducted. Finally, mucormycosis cases have been reported in patients undergoing posaconazole prophylaxis despite satisfactory serum concentrations [START_REF] Kang | Fatal Breakthrough Mucormycosis in an Acute Myelogenous Leukemia Patient while on Posaconazole Prophylaxis[END_REF] .

However, it seems important to note that there are no current validated MIC breakpoints for any of the available antifungals and thus the determination of susceptibility categories is not possible for the agents of mucormycosis.

Immunostimulating drugs

A case report has recently reported the benefit of a treatment with the checkpoint inhibitor nivolumab and interferon-Υ for an immunocompetent patient with extensive abdominal mucormycosis unresponsive to conventional therapy [START_REF] Grimaldi | Nivolumab plus interferon-γ in the treatment of intractable mucormycosis[END_REF] .

Conclusion

Mucormycosis is a life-threatening fungal infection characterized by host tissue infarction and necrosis that occurs mostly in immunocompromised patients and is associated with an increasing incidence and mortality despite the availability of therapeutic tools. Determining whether the patient has invasive aspergillosis or mucormycosis could be challenging at the bedside. In this context, new tools of molecular biology have been developed to obtain earlier diagnosis and start optimal medico-surgical treatment. Comparative studies are needed to better optimize induction and consolidation treatment.

Grant information

The author(s) declared that no grants were involved in supporting this work.

2 07

 2 Sep 2018, (F1000 Faculty Rev):1429 (doi: First published: 7 ) 10.12688/f1000research.15081.1 07 Sep 2018, (F1000 Faculty Rev):1429 (doi: Latest published: 7 ) 10.12688/f1000research.15081.1 v1

Figure 1 .

 1 Figure 1. Typical features of Rhizopus arrhizus. (a) Sporangiophore branching and rhizoids (stereomicroscope); (b) grey-brownish colony on malt 2% medium; (c) melanized sporangium and sporangiospores.

Figure 3 .

 3 Figure 3. Mucormycosis: from suspicion to treatment. PCR, polymerase chain reaction.

Figure 2 .

 2 Figure 2. Inversed halo sign. Inversed halo sign.

F1000Research 2018, 7(F1000 Faculty Rev):1429 Last updated: 07 SEP 2018

Acknowledgements

Thanks to Dea Garcia-Hermoso for reading the manuscript and providing strain photographs.

She received research grant from Gilead Sciences. OL has been a consultant for Gilead Sciences, a speakers' bureau for Pfeizer, Novartis, Astellas, Merck, and Gilead Sciences, and has received research grants from Gilead Sciences. AA has received travel grant from MSD, Gilead and Astella and honoraria from Pathoquest and Gilead outside the submitted work. BP has no competing interests to declare. The author(s) declared that no grants were involved in supporting this work.

Open Peer Review

Current Referee Status: Editorial Note on the Review Process are commissioned from members of the prestigious and are edited as a F1000 Faculty Reviews F1000 Faculty service to readers. In order to make these reviews as comprehensive and accessible as possible, the referees provide input before publication and only the final, revised version is published. The referees who approved the final version are listed with their names and affiliations but without their reports on earlier versions (any comments will already have been addressed in the published version).

The referees who approved this article are: Version 1

The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias You can publish traditional articles, null/negative results, case reports, data notes and more

The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com Department of Microbiology, All India Institute of Medical Sciences, Delhi, India

Immaculata Xess

No competing interests were disclosed. Competing Interests: