Guillaume Cerutti
email: guillaume.cerutti@liris.cnrs.fr

Laure Tougne
email: laure.tougne@liris.cnrs.fr

Julien Mille
email: julien.mille@liris.cnrs.fr

Antoine Vacavant
email: antoine.vacavant@iut.u-clermont1.fr

Didier Coquin
email: didier.coquin@univ-savoie.fr

Understanding Leaves in Natural Images -A Model-Based Approach for Tree Species Identification

Keywords: plant recognition, tree leaf, image segmentation, natural background, active contours, geometric features, classification

published or not. The documents may come

L'archive ouverte

Foreword

Advances in technology and prevalence of urban settlement undoubtedly contributed to widen a gap between nature and us that was inconceivable a couple of generations ago. Along the triumphant march of alleged progress, we carelessly dropped the knowledge of many plants, trees and herbs that used to be found in our most immediate environment. Knowing how to recognize a plant, and what uses it may provide, is now the mere privilege of a handful of specialists. Today however, with the growing awareness that nature's diversity ought to be treasured, the need to regain some touch with this knowledge never felt so present.

Books are available to the neophyte, but when it comes to precise species identification, it is easy to get lost in the bushy floras, without the minimal theoretical background. However, with the rise of mobile applications that have the potential, yet scarecely exploited, to place powerful resources in everyone's pocket, the access to such knowledge can be made much more intuitive and flexible for anyone who feels the need. Recently, attempts in this direction have come to light with great success, being with user-based (TreeId 2 , Fleurs en Poche 3 ) or automatic recognition (LeafSnap4 ) on white background images.

Leaves are the most obvious and widespread choice for tree species recognition, even though the botanical classification was not built upon their properties. They can be found almost all year long, are easy to photograph, and their shapes present well studied specificities that make the identification, if not trivial, possible. Our goal with the Folia application is then to build a system for leaf shape analysis that processes, unlike what has been done to date, pictures in a natural environment. With the aim of being an educational tool, it relies on high-level geometric criteria inspired by those used by botanists, that make a semantic interpretation possible, to classify a leaf into a list of species.

In this paper, we propose an attempt for such a system. Other works tackling similar challenges are presented in Section 2, and a global view of our system is presented in Section 3. Section 4 introduces the color model we use for segmentation, and Section 5 expounds the leaf model estimation, and the subsequent active contour segmentation algorithm. Section 6 details the leaf-specific contour descriptors we introduce. The classification method we use and its results are shown in Section 7 and perspectives of our work are discussed in Section 8.

Related Works

Leaf Recognition

Plant recognition and leaf image retrieval have recently been topics of interest for many works in image processing, though most of them limit the problem to leaves on a plain background. Challenges for the community have even been organized such as the ImageCLEF Plant Identification Task [START_REF] Goëau | The CLEF 2011 Plant Images Classification Task[END_REF] since 2011. Some authors [START_REF] Belhumeur | Searching the World's Herbaria: A System for Visual Identication of Plant Species[END_REF] also pursued the goal of conceiving a mobile guide with the LeafSnap application, combining established shape descriptors (Inner-Distance Shape Context) and classification methods to perform remarkably well on white-background images. Our goal however is to deal with the kind of complex background images one can take out in the nature.

Many works similarly rely on statistical features such as moments [START_REF] Wang | Classification of Plant Leaf Images with Complicated Background[END_REF], histogram of gradients or local interest points [START_REF] Nilsback | Delving into the Whorl of Flower Segmentation[END_REF][START_REF] Goëau | Participation of INRIA & Pl@ntNet to ImageCLEF 2011 Plant Images Classification Task[END_REF] or generic shape descriptors such as the Curvature-Scale Space [START_REF] Mokhtarian | Matching Shapes with Self-Intersections: Application to Leaf Classification[END_REF]. Such descriptors were not designed to take into account the nature of the object, even if they fit quite well with its specificities.

A commonly used geometrical descriptor for leaf image retrieval is the Centroid-Contour Distance (CCD) curve [START_REF] Wang | Leaf Image Retrieval with Shape Features[END_REF][START_REF] Teng | Leaf Segmentation, Its 3D Position Estimation and Leaf Classification from a Few Images with Very Close Viewpoints[END_REF] though it can be applied to any type of object. On the other hand, some morphological features explicitly computed on the shape of the object to model its natural properties have also been used [START_REF] Saitoh | Automatic Recognition of Wild Flowers[END_REF][START_REF] Du | Leaf shape based plant species recognition[END_REF][START_REF] Goëau | Participation of INRIA & Pl@ntNet to ImageCLEF 2011 Plant Images Classification Task[END_REF]. Even more dedicated methods have been designed, basing their recognition on an explicit representation of the leaf contour and of its teeth [START_REF] Im | A Hierarchical Method of Recognizing Plant Species by Leaf Shapes[END_REF]. Most of these descriptors are used on white background images where robustness to noise is not an issue since retrieving the exact contour is something trivial.

Plant Segmentation

There are still some authors addressing the hard task of isolating a natural object from a natural environment. Luminosity conditions, complexity of the environment, and presence of very similar objects around the one of interest make it a very difficult problem where generic unconstrained segmentation methods are very likely to fail, as shows Figure 1.

Considering plants, flowers are obviously an easier target to segment, as their color generally clearly stands out. Introducing a geometric shape model [START_REF] Nilsback | Delving into the Whorl of Flower Segmentation[END_REF] or putting constraints on the image acquisition [START_REF] Saitoh | Automatic Recognition of Blooming Flowers[END_REF] seems however necessary to perform well.

The case of leaves appears even more challenging, and the use of complex methods, such as 3D reconstruction to take depth into account [START_REF] Teng | Leaf Segmentation, Its 3D Position Estimation and Leaf Classification from a Few Images with Very Close Viewpoints[END_REF], or multiple step segmentation processes [START_REF] Wang | Classification of Plant Leaf Images with Complicated Background[END_REF], is inevitable to overcome the difficulties. The use of deformable templates to include as much prior knowledge as possible has been used in the restricted case of weed leaves [START_REF] Manh | Weed leaf image segmentation by deformable templates[END_REF] but is a very convincing way to deal with occlusions, overlaps and other problems posed by natural images. Our work follows the same idea, making use of active models to isolate a shape on which we have some notion beforehand.

Active Contour Models

The first formulation of such models goes back to active contours, or snakes [START_REF] Kass | Snakes: Active contour models[END_REF], introduced as a way to solve edge detection problems. They rely on the minimization of an energy functional to adjust splines to the edges in the image. This energy classically presents a two term formulation, summing an internal energy term that constrains the properties (regularity, smoothness) of the contour, and an external or image energy term whose goal is to make sure that the contour corresponds to the content of the image, based on the gradient intensity.

The evolution of the contour can be based on other image characteristics, color consistency is a possible choice when objects are not well defined by gradient [START_REF] Chan | Active Contours Without Edges[END_REF] using a level-set formulation for the contour [START_REF] Osher | Fronts Propagating with Curvature Dependent Speed: Algorithms Based on Hamilton-Jacobi Formulations[END_REF]. Texture information has also been used [START_REF] Unal | Information-Theoretic Active Polygons for Unsupervised Texture Segmentation[END_REF], the contour being represented in this case by a polygon.

But it can be very beneficial, when some prior information about the shape of the target object is available, to include this knowledge in the evolution of the contour. A deformable template modeling complex objects [START_REF] Yuille | Feature Extraction from Faces Using Deformable Templates[END_REF] is a solution that additionally presents the advantage of compressing the representation of the contour by the use of parameters. When objects have a well-determined shape, a strongly constrained template, such as active shape models [START_REF] Cootes | Active Shape Models-Their Training and Application[END_REF], learned compositions of deformable triangulated polygons [START_REF] Felzenszwalb | Representation and Detection of Deformable Shapes[END_REF] or level-set active contours with shape priors [? ], is a powerful solution even when the object is occluded. With such methods including prior knowledge, shapes need to be known precisely and not to vary, and they consequently lack the necessary flexibility and expressiveness to capture the diversity of leaf shapes.

In the following section, we give an overview of the proposed method for leaf segmentation and recognition which is based on active contour models.

Leaf Recognition Overview

Framework

To cope with the hardly solvable challenge of segmenting and identifying leaves in complex images, we decided to limit our focus to simple and palmately lobed leaves only, reducing the scope to some 80% of the nearly 200 European native broad-leaved tree species. Our method is designed to process photographs of leaves in a natural background, taken by mobile cameras, but it is still able to perform on white or plain background images. In any case, we assume that the leaf is roughly centered and vertically-oriented in the image, so that we start from an area where we are certain to find the leaf. This assumption is anyway legitimate when we consider our final objective of offering a mobile application in which the user is guided for the picture taking as illustrates Figure 2. The algorithms we propose have to run on the mobile processor and consequently fit its requirements in terms of space and execution time.

Botany as a Pattern for Recognition

Tree species recognition using leaf features is a well studied problem, when we look at it from a botanist's view. If the expert will identify the tree at a glance, a well established identification process exists to guide the untrained amateur, through the use of dichotomous keys. The diligent user will be asked to look for several significant traits over the leaf (size, global shape, number of lobes, venation, basal and apical shapes, type of margin,. . . ) following the path towards the species.

These characteristics are described using a dedicated terminology referring generally to canonical, ideal shapes. They are not the exact shapes that are found in nature but rather a human interpretation focusing only on the relevant details. For instance the presence or absence of teeth around the apex does not change the denomination of the apical shape though it certainly creates different shapes. The different features are considered independently from one another, an easy abstraction for the human eye. There is consequently a part of subjectivity and vagueness in the use of those botanical terms, but they constitute a determining clue as to where to look at and what to look for.

Knowing in advance the whole set of morphological features one has to search in a leaf opens the possibility of modelling shapes beforehand accordingly. The introduction of specific shape models from the very beginning of the process has the advantage of including prior knowledge, acting like a constraint to ensure that the resulting shapes will correspond to what can be found in the nature. It is also a way of putting semantics on what is extracted in the image, since the models are constructed at the same level of abstraction as the botanist's interpretation.

The method we propose to identify leaves is a transposition of this idea. It relies on several models to represent the different botanical criteria defining the shape of a leaf, starting with the global shape, estimated first to guide the segmentation. Each of them focuses on one single element of interest and is designed to avoid taking the other ones into account. This decomposition of shape features into independent characteristics, later to be combined together, fits with the challenges of leaf recognition, where the intra-class variability for a given species may be high for some botanical features whereas other ones will be very discriminating. This specificity is displayed in Figure 3. This is the reason why it is important to decorrelate features as they are extracted in order to reduce the amount of noise that would prejudice the classification.

Process Overview

As depicted in Figure 4, the identification process can be divided into four main phases. The first one consists in building a representation of what will make the leaf identifiable in the image. We chose to base this representation on color only and estimate a model of the color of the leaf providing information on what may or may not be part of the leaf. The segmentation method directly relies on this representation. Its first step is based on a highly variable polygonal model representing the global shape of the leaf. The estimation of this model is performed using the color representation, and later used to retrieve the actual leaf contour. This final segmentation step can be seen as a deformation of the rigid polygonal shape into a contour adjusting to teeth on the leaf margin. The choice of an active contour starting from the polygonal contour and evolving towards the margin is a natural one. The model capturing the global shape acts as a strong shape constraint, which avoids common shortcomings of unsupervised segmentation methods shown in Figure 1. To describe the shape obtained after the segmentation, and capture the discriminating features of a species, we model successively the characteristics of the margin, base and apex, after interpreting the final contour using the Curvature-Scale Space. To describe those features we introduce dedicated representations, that rely on existing appropriate shape descriptions to capture the interesting morphological properties. The different and independent leaf features are used to build species models, and combined in a simple manner to measure their very contribution, without the help of a sophisticated classification algorithm. In the following, we successively detail each of these steps.

Leaf Color Representation

To guide the first phase of the segmentation, we made the decision to rely on the color information. The gradient information is a useful tool, but its efficiency in our case is reduced by the venation that often creates strong edges inside the leaf, and by the fact that, since we start by a polygonal approximation of the leaf's shape, it will hardly be possible to fit exactly the actual edges of the leaf. Texture is another alternative, but most leaves present large-scale texture elements which are consequently detected with less precision, being moreover different over the surface of the leaf.

Building an a priori color model that would be always valid, independently of the species, lighting and season is not an achievable goal. It is therefore essential to build a custom color model for each and every leaf we want to process. In our case, we have a good idea of the leaf's location in the first place, ensuring that the central area of the image will contain only leaf pixels. Starting from this initial area, it is possible to build a representation of the leaf's color. The method we present here differs from our previous approach [START_REF] Cerutti | A Parametric Active Polygon for Leaf Segmentation and Shape Estimation[END_REF] where a 2-Gaussian model was used. That view proved to lack accuracy, and our color representation now tries to estimate both a global color model, that separates leaf pixels from background pixels, and a local mean color model, that focuses on the leaf of interest, and to combine them in a relevant way using belief functions.

Global Linear Regression Model

The L*a*b* colorspace is a perceptually intuitive alternative to the regular RGB color representation, and separates well the chrominance information from the luminance. We observed the repartition of leaf pixels in various colorspaces (L*a*b*, RGB, HSL) and it appeared that the color distribution generally forms a straight continuous cylinder in L*a*b, whereas it appears appreciably more curved in the other spaces.

One plausible way to model a leaf's color is then to estimate the main direction of this seeming cylinder. It is important to consider that leaves often contain reflections or shadows which present colors that might not be found in an initial leaf model. Our observations however show that these colors are generally consistent with the colors in a smaller initial area, and that the L*a*b* cylinder presents roughly the same axis for the whole leaf and for a limited part of the leaf. It is in any case essential to have a model with the ability to extrapolate colors outside the initial area, and estimate the whole cylinder and not only an ellipsoid. Computing a linear regression over the L*a*b* pixels and using it to predict colors is then a good way to achieve this generalization. Figure 5 shows how the L*a*b* colorspace is best suited for linear regression, as the prediction line stays almost the same if we consider a small part of the leaf (containing a wide enough luminosity range) or the whole leaf, whereas this line tends to move in the other spaces.

Concretely, given an initial region Ω 0 , we apply a standard linear regression to the pixels (L x , a x , b x ) x∈Ω 0 where L is considered as the explanatory variable for both variables a and b. We obtain a color prediction given the luminosity â(L) = a 0 + a 1 L and b(L) = b 0 + b 1 L based on which it is possible to compute a dissimilarity map (as shown in Figure 7) by calculating the distance of every pixel in the image to the color model:

d LinReg (x) = (L x , a x , b x ) -L x , â(L x ), b(L x )
changes, which makes it in particular unable to discriminate the leaf of interest from other leaves of similar color in the background.

Local Adaptive Mean Color

Color inside a leaf may vary. Reflections, venation, shadows or spots may create brutal changes, but the main variation along the surface of the leaf is caused by orientation and illumination differences and is a progressive change. A global color model may be able to acknowledge all these varying colors as leaf colors, but it fails to capture the fact that the leaf may present a specific color only in a given area of the image. And if this color is found in another part of the image, it should not necessarily be considered as a leaf color there. A rigid global model is then not enough, and it is also necessary to have a model that adapts to progressive local color variations.

We build such a model by exploring the image, trying to answer the question "what would be the most probable color in this part of the image if the considered pixel was a leaf pixel?" We start from the same initial region as previously, in which we consider this local expected mean Lx , āx , bx to be known, taking the mean color of a neighbourhood around each pixel. Then, the idea is to propagate, and update if necessary, this mean through the image. As long as the color variation between the pixel and the local expected mean is limited, we consider that the leaf's color is locally changing, and that this change has to be taken into account in the color we expect to find further on. If the change is too brutal, it is certainly not a natural variation of the leaf's color, and we should not change the expected color for the following pixels. This method relies then on an adaptivity parameter α that defines how fast the mean will adapt to changes, and a tolerance threshold θ that represents the maximal color distance (between the pixel color and the propagated mean) up to which a pixel will be taken into account to update the mean during its propagation. The pixels are examined once, starting by the center, considering first the current closest pixel in terms of color. This propagation and adaptation process is illustrated by Figure 6.

When considering a new pixel x, if the pixel color is close enough to the expected mean propagated from a previous neighbour, the mean we propagate to its unexplored neighbours is a weighted sum of the mean and the color ; otherwise it is the same mean. The neighbours that are not yet to be considered are then placed inside a queue, according to their distance to the propagated mean. In other terms, if we call N 8 (x) the 8 neighbours of the pixel x :

∀y ∈ N 8 (x), Ly , āy , by =              α (L x , a x , b x ) + (1 -α) Lx , āx , bx , if (L x , a x , b x ) -Lx , āx , bx 2 < θ Lx , āx , bx , otherwise (1)
Estimating the adaptive expected mean in every pixel of the image makes it possible to compute a complementary dissimilarity map, by calculating the distance of each pixel to the local mean:

d Adaptive (x) = (L x , a x , b x ) -Lx , āx , bx 2
The exploration method however presents the risk of finding a flaw, a path of progressive color transition between the leaf and the background, that may ultimately lead to consider a large part of the background as close. A good tolerance/adaptivity compromise allows to minimize this risk, but it is an inherent property of the method to consider as leaf color any color reachable by a gradual color path starting from the inside of the leaf.

Evidence-Based Combination

Both previous approaches constitute accurate ways to model the color of the leaf, but have their drawbacks. To keep only the best of each, it is essential to bear in mind that they do not measure the same thing in the image. It is important to take this into account to combine them effectively, as a simple, flat combination as the mean or the minimum would include both their advantages and disadvantages, and therefore not be sufficient.

The combination of these two representations could be performed using a probabilistic method such as Bayesian inference (as it is done in [START_REF] Cheng | Component Optimization for Image Understanding: A Bayesian Approach[END_REF]) but we chose to rely on Dempster-Shafer theory of evidence [START_REF] Dempster | A Generalization of Bayesian Inference[END_REF][START_REF] Shafer | A Mathematical Theory of Evidence[END_REF] that is a framework generalizing probability theory. It has notably the advantage to reason with uncertainty, and to explicitly model the absence of knowledge when the sources do not provide tangible information.

As a matter of fact, a pixel appearing close in the global linear regression approach is generally a leaf pixel (not necessarily from the one of interest) whereas a distant pixel might also belong to the leaf but result being far from the prediction line if it is outside the learning luminosity range. This is what typically happens with reflections when not enough bright pixels belong to the initial region. On the other hand, a pixel being close in the local approach only means that a progressive path exists between the leaf and it, which reciprocally implies that a distant pixel will most certainly not belong to the leaf since no path could be found. Intuitively, we see that the global approach puts confidence on what belongs to the leaf whereas the local approach tends more to distrust what lies outside the leaf.

Viewing it as a binary problem for each pixel, we define two opposite basic belief assignments that will be combined using Dempster's rule. In the following, we consider the problem for one given pixel x. The linear regression distance puts mass on the "leaf" hypothesis {L}, and the adaptive distance will conversely be used only to place belief mass on the "non-leaf" hypothesis { L}, the missing mass being in any case assigned to the whole set {L, L}, representing uncertainty.

The combination of these two mass distributions using Dempster's rule gives a resulting couple of belief and plausibility of the pixel x being a leaf pixel, that are used to compute a distance-like term d Belie f (x) and produce a dissimilarity map. More details about the exact computation of the distance can be found in Appendix A. Figure 7 shows an example of such a map, and illustrates the behaviour of our combination compared to more basic possibilities. A more thorough justification of this evidence-based approach in terms of segmentation performance is provided in Appendix B.

A Model for Leaf Segmentation

The definition of a leaf, despite seeming intuitively easy to comprehend, covers actually a wide diversity of objects, which are surprisingly different in terms of shape, ranging from pine needles to spread-out bipinnate leaves. Such variety in shape makes it almost impossible to come up with a single well-defined model that could be applied to any possible leaf, even though they all develop with the same goal and result from a common growing process.

However, to facilitate the segmentation of leaves in complex images, it is necessary to integrate as much general shape information as possible, and exploit prior knowledge to constraint the resulting shapes. We propose to use a general leaf model to guide the segmentation, in an accurate and flexible way, without leaving too much adaptivity to non-leaf shapes. This is a way to ensure the coherence of the obtained shapes with both the content of the image and the natural possibilities.

Modeling the Leaf Object

A polygonal model based on the notion of global leaf shape in botany and covering the whole set of possible shapes for simple leaves was first introduced [START_REF] Cerutti | A Parametric Active Polygon for Leaf Segmentation and Shape Estimation[END_REF] and extended to the trickier case of palmately lobed leaves [START_REF] Cerutti | Guiding Active Contours for Tree Leaf Segmentation and Identification[END_REF]. This model relies on a compact set of numerical parameters, accounting for geometric characteristics, to recreate the approximated shape of a leaf, as shown in Figure 8. It needs only two points B and T , representing the base and top, and the values for the various parameters we recall here:

• w, the relative maximal width • p, the relative position where this width is reached • α T , the opening angle at the apex • α B , the opening angle at the base These values can directly be used as descriptors of the global shape of the leaf, which is a discriminant feature used by botanists to identify leaves [START_REF] Coste | Flore descriptive et illustrée de la France de la Corse et des contrées limitrophes[END_REF]. The model's parameters are additionally significant enough to constitute a first step towards a semantic interpretation, while keeping the precision and flexibility of a numerical description. The idea is then to use this model with a deformable template approach, where the elementary deformations are simply the variations of the parameters.

(a) (b) (c) (d) (e) (f) (a) (b) (c) (d) (e) (f)

Parametric Active Polygon

This color dissimilarity d Belie f is then the main force that will guide the evolution of the polygonal model. The deformable model approach relies on an energy functional that is minimized by the model during its evolution. At each step, it will select a deformation that leads to the greatest decrease of the energy until it remains stable.

Energy Definition

The energy is classically split into an image, or external, term and an internal term. Here, the external energy is directly based on the color distance. Since the goal is to gather pixels that are close to the leaf's color model, the value that needs to be minimized is the overall color dissimilarity of the pixels inside the polygonal region Π defined by a model built on the two points B and T and the set of parameters P. It just has to be balanced by a balloon force d max to prevent the optimal model from being an empty one.

E ext (B, T, P) = x∈Π(B,T,P) d Belie f (x) -d max (2)
The value d max pushes the region to grow, but can also be seen as a threshold for which a pixel will be costly to add into the polygon.

The internal energy can be considered to be largely implicit, contained in the rules of the polygon's construction. But given only its definition, there is still a risk that the resulting shape does not look like a leaf at all. To ensure that consistency, we add a constraint of the values of the parameters. Using K-means on the parameters from handtuned models on white background leaf images, the centroids µ k,i k=1..K = ({ wk , pk , ᾱT k , ᾱBk }) k=1..K of 10 classes of archetypal shapes appear, and the corresponding standard deviations σ k,i k=1..K can be computed to account for the width of each class. To be of a leaf-consistent shape, the model has to be close to at least one of these shape classes in the parameter space . We subsequently define an internal energy term expressing this constraint, based on the normalized Euclidean distance to the classes of a model built with parameters (P i ) = {w, p, α T , α B } :

E int (P) = min k=1..K        i P i -µ k,i 2 σ 2 k,i        (3)
The final expression of the energy to be minimized is a combination of these two energy, with a coefficient ω int weighting the internal term, which sums up the problem to finding the optimal region Π * defined by points B * , T * and parameters P * , such as:

(B * , T * , P * ) = argmin (B,T,P) (E ext (B, T, P) + ω int E int (P)) (4)

Model Evolution

Starting from an initial position at the center of the image, the model will undergo a sequence of elementary deformations until it converges. In our case, the optimization is performed over the model's parameters and points, making the number of deformations to be considered at each step very limited. We look at the energy resulting from the two points B and T moving in one of the four directions, and from each parameter varying positively or negatively. The deformation giving the lowest energy is selected, accordingly modifying the model. To avoid local minima, a heuristic close to simulated annealing is employed, with a temperature variable making possible to choose positive energy variations, but also conditioning the amplitude of the deformations. This temperature is slowly decreasing and regularly raised to a high value, until the model remains stable between two temperature rises.

All parameters are optimized simultaneously following this process, except for the number of lobes, that has to be treated differently given the drastic changes in shape it involves. To retrieve the accurate number of lobes, the model is initialized with a clearly excessive number of lobes (greater than the maximal number of lobes that can be found in nature) and redundant lobes are eliminated at the end of each temperature cycle (Figure 9). Lobes in excess tend to group inside the actual lobes of the leaf, and by computing an overlap factor it is possible to detect which lobes are unnecessary and suppress them [START_REF] Cerutti | Guiding Active Contours for Tree Leaf Segmentation and Identification[END_REF].

Guided Active Contour

The model Π * = Π (B * , T * , P * ) obtained after this evolution constitutes both a rough segmentation and a generally pertinent representation of the leaf's global shape. But to have a more detailed look at the particular morphological characteristics of the leaf, it is necessary to refine this segmentation and to obtain a contour corresponding precisely to the actual leaf margin, where most botanical differential features appear.

The interesting point is that we do not start from nothing, but we can use the polygonal model to initialize an active contour and to guide its evolution. The active contour implementation we use differs from the original model [START_REF] Kass | Snakes: Active contour models[END_REF] since it uses an implicit representation based on the binary level-set framework [START_REF] Shi | A Real-Time Algorithm for the Approximation of Level-Set-Based Curve Evolution[END_REF], an approximation for the original level-set evolution [START_REF] Malladi | Shape Modeling with Front Propagation: a Level Set Approach, Pattern Analysis and Machine Intelligence[END_REF]. The constraints on the shape takes the form of an additional term in the energy functional minimized by the contour, that is then defined, for a contour Γ, by:

E(Γ) = ω L E Lea f (Γ) + ω Π E S hape (Γ, Π * ) + ω ∇ E Gradient (Γ) + ω S E S mooth (Γ) -ω Balloon ( 5
)
Instead of having an external energy term based on color consistency, or distance to a mean, we decided to reuse the dissimilarity map from the previous step, considering we have already an efficient measure of how well a pixel should fit in the leaf, in terms of color. The E Lea f term is then simply the sum of the dissimilarity inside the region.

To use the polygonal model as a shape constraint, we decided to force the contour to remain close to the polygon Π * . The E S hape energy term ensures this guidance by summing a distance function, based on the distance of a pixel to an extended version Π * of the polygon, over the contour [START_REF] Cerutti | Guiding Active Contours for Tree Leaf Segmentation and Identification[END_REF]. The Figure 10 shows an example of the distance to such an extended polygon, as well as the evolution of the contour constrained by this shape. This constraint controls the overflow that would occur when a boundary between the leaf of interest and a neighbouring leaf is neither clear in terms of color nor gradient. The extension performed on the polygon, at the base and at the top, gives more liberty to the contour in these areas where it is crucial to find the actual margin to analyze the local shape. However, all these energies are hard to harmonize into a coherent whole, independently of the image. To avoid conflicting interactions, we chose to have the coefficient of the leaf dissimilarity energy α initially set to a very low value, so that the behaviour of the contour preeminently relies on the gradient in the first steps of its evolution. This coefficient is then gradually raised to its final value, so that the final contour minimizes always the same energy. This choice makes it possible for the contour to reach high gradients corresponding to the leaf margin even in color distant areas, and to keep sticking to this boundaries even when the leaf term reaches its normal value.

Examples of the whole segmentation process over various natural background leaf images can be found in Appendix B.

Capturing the Local Morphology

To describe the segmented leaves and make their identification possible, we want to represent their different botanical properties through independent sets of descriptors. Considering that the global shape is characterized by the parameters of the polygonal model, we have to look at a smaller scale and to focus on the interest areas that are the base and the apex of the leaf, and the potential teeth one may find on the margin. We propose separate descriptions for these three local leaf shapes, using curvature to locate and describe teeth, and introducing deformable models to approximate basal and apical shapes.

Teeth and Pits on the Margin

The places we want to locate to examine local shapes are generally easy to spot in terms of geometry, as they stand out clearly when giving a look at the contour's curvature. The main problem of this intuitive definition is that the level of detail where the apex of the leaf or smaller teeth appear is never the same, and looking at the contour with a fixed scale is consequently not sufficient.

Curvature Scale Space Transform

A richer representation of the contour that avoids this obstacle is the Curvature-Scale Space (CSS), that has already been used in the context of shape-based leaf recognition [START_REF] Mokhtarian | Matching Shapes with Self-Intersections: Application to Leaf Classification[END_REF]. It piles up curvature measures at each point of the contour over successive scales, summing up the information into a map where concavities and convexities clearly appear, as well as the relative scale until which they persist. For shape matching applications, the CSS contours, namely the curves formed by zero crossings of the curvature, or more exactly their maximal scale points, are generally used as a shape representation, losing some information on the actual curvature value.

We used a curvature measure based on the estimated derivatives of the contour points, but whereas the original method estimates the same curvature on gaussian smoothings of the contour with growing kernels, we chose to keep the same contour but to estimate the derivatives in each point on a growing neighbourhood. Our scale variable represents then the size of the considered neighbourhood. The CSS consists of the kind of matrix displayed in Figure 11, containing curvature values K for a given scale S and at a given point (x; y) defined by its normalized arc length u: K(u, S ) =

x S (u) • y S (u)x S (u) • y S (u)

x S (u) 2 + y S (u) 2 3/2

x S (u)

= 1 S x(u + S 2 ) -x(u - S 2 ) y S (u) = 1 S y(u + S 2 ) -y(u - S 2 ) x S (u) = 1 S 2 x(u + S 2 ) -2x(u) + x(u - S 2 ) y S (u) = 1 S 2 y(u + S 2 ) -2y(u) + y(u - S 2 ) (6)
The CSS transform is a very powerful description of a contour but is too informative to be used as a descriptor and to build class models by averaging several of them. As we wish to locate and characterize precisely curvature-defined elements on the contour, it was a more judicious choice to look, not at zero-curvature points, but on the contrary at the maxima and minima of curvature. The fact that these points visually stand out in the CSS image, and can be matched obviously with convex or concave structures on the contour, argues at least in favour of this choice.

Detection of Dominant Points

To locate these points, we actually want to detect dominant points on the leaf contour and to find the scale of the concavity or convexity they correspond to. This is a well studied problem [START_REF] Teh | On the Detection of Dominant Points on Digital Curves[END_REF][START_REF] Pei | The detection of dominant points on digital curves by scale-space filtering[END_REF] and the method we use is largely inspired of existing works. The same steps are repeated at each scale, starting with S = 4, and the set of dominant points at each scale D(S ) is initialized with all the contour points. The process is then the following: The result we obtain is a map of the located dominant points at each scale, as presented in Figure 12. This representation reveals the teeth and pits of the contour in the form of chains of dominant points. Given the particularities of our method, dominant points may shift between scales, leading to an imprecision on the exact location of the point on the contour, but giving some potentially useful information on the orientation of the corresponding concave or convex structure.

Relevant scale selection

For the sake of interpretation, the information we want to retrieve is the visible curvature and size of every located structure on the contour. Each chain that appears on the dominant points map corresponds indeed to a convex or a concave part, and its actual size, as a human eye would interpret it, is the scale until which it persists, that is to say the scale of the chain's end point.

To keep only these terminal points, we scan the dominant points starting from the highest scale. When a dominant point of arc-length u is found at scale S , all the dominant points of the same curvature sign are suppressed at lower scales, in a neighbourhood of size S .

∀S < S , D(S ) ← D(S )\{u , |u -u| < S 2 , sign(K(u, S )) = sign(K(u , S ))}
This way, we ensure that a single structure is not counted twice, and small structures which are included in larger ones are merged into one. In the case of large-scale elements, we consider then that only the largest appearing structure is relevant, and that smaller concave or convex parts it includes are part of it and can be discarded. The only exception concern the leaf's base and apex, which are generally strongly marked convex parts, and would encompass all the smaller teeth that should be kept and analysed.

Understanding the leaf contour

Prior to this processing, the contour needs to be divided into areas corresponding to the tip, to the base, to the potential lobe tips, and finally to the rest of the margin, so that the most salient features do not absorb the rest of the information. Here we can advantageously rely on the polygonal shape model to generate this partition, that would otherwise be a much more complicated task. Supposing that the axes of the polygonal model are accurate, we can use them to mark off contour points that are in one of these areas of interest. To achieve this labelling, as shown in Figure 13, we only keep the intersection of the contour with a fixed angular sector, built around the corresponding axis, issuing from the opposite point (base for the tips, and main tip for the base) and with a minimal distance to this point.

The interest of this partition is twofold. First, it is now possible to locate accurately the base point B and the tip point T of the leaf as the most salient dominant points in the corresponding area. And secondly, outside of these areas, we have, for each side of the leaf, located the teeth and the pits of the margin S D(S ), at their maximal scale S , allegedly their actual size. To complete this size information, we estimate the actual curvature K(u, S ) of each dominant point, as the average curvature of the dominant points belonging to the same chain. This computation is done considering scales from the highest to the lowest, while suppressing the points at lower scales :

∀S , ∀u ∈ D(S ), K(u, S ) = 1 S S s=1 max u ∈D(s) |u -u|< S 2 K(u , s) (7)
Finally, what we produce is a semantically rich representation of the leaf contour, where the base and tip points are precisely located, and where teeth are detected and characterized independently in terms of size and curvature. This representation is displayed in Figure 14.

A descriptor for the leaf margin

Even if this representation of the contour is full of information and could be used to compare two leaves, it still does not come in very handy if we want to average several of them for classification purposes. It is necessary to sum up this information into a condensed descriptor that will make it possible to capture the properties of a species, and allow a semantic interpretation of its specificities.

Margin shapes in Botany

To describe the shape of the leaf margin, botanists use a terminology that refers both to the properties of teeth taken separately and to their repartition over the margin. Some of these terms are shown in Figure 15. For instance a "doubly dentate" leaf implies two levels of teeth with different sizes and frequencies, with bigger teeth divided into smaller sub-teeth. The words themselves are vague enough so that leaves with teeth that look rather different may fall inside the same term. The margin shape constitutes however a very discriminant criterion for species identification that generally presents less variability than the global shape, with the exception of some pathological species. Our descriptor will have to capture the differences regarding size, sharpness, frequency and variability of the teeth implied in the botanical terms, but also to benefit from the use of numerical values to be more precise and discriminate leaves of different aspect that would be called the same.

Curvature-Scale Histogram

The descriptor we propose to represent leaf margin shapes is simply a square two dimensional histogram summing up the properties of the detected teeth and pits over scale and curvature dimensions. Since every detected dominant point u ∈ S D(S ) is characterized by its maximal scale S and average curvature K(u, S ), it will contribute at the location ( K(u, s), S ) of the histogram. To simplify the notations, we now write ((K i ; S i )) i=1..N = S {( K(u, S ); S ), u ∈ D(S )} to enumerate the dominant points.

We want to count all the vertices of the contour, and count them once, to have a valid statistics of what proportion of the contour is part of a concavity or convexity of given properties. Consequently, the weight that is put in the histogram for one structure has to be proportional to its scale S i , since, given their definition, detected structures account for S i vertices. For smoothing purpose, we chose to place in the histogram a Gaussian G centered on (K i , S i ) and whose total weight is equal to S i . If we set σ s (S i ) = S i 4 and σ k = 4 the values of the histogram can then be expressed in function of ((K i ; S i )) i=1..N :

H CS (k, s) = n i=1 G σ s (S i ),σ k ((k; s) -(K i ; S i )) where G σ s ,σ k (K; S ) = 1 2π exp       -( S 2 2σ 2 s + K 2 2σ 2 k )       (8) (a) (b) (c) (d)
Figure 16: Curvature Scale Histograms for the 4 leaves of Figure 14 : curvature on the vertical axis, scale on the horizontal axis ; convexities in orange, concavities in blue

In this way, we cumulate the structures found along the margin. We build two distinct histograms for the left margin and the right margin, as they are detected when locating the different areas of the leaf. Then we compute their geometric mean to keep only teeth that are present on both sides, and reduce the influence of a possibly bad segmentation. This resulting histogram is normalized back to the sum of the weights of the two previous histograms. The total weight of the histogram at this step is then equal to the number of vertices detected as part of a convexity or concavity along both sides. The missing weight, i.e. the number of vertices of either side that have not yet been counted, because they do not belong to any concavity or convexity, is added as such in the bin H(0, 0) that represents then the proportion of the margin that is smooth. The histogram is then normalized again so that the weight is the same for every leaf.

The final histograms are presented in Figure 16 where we can see how an entire margin in (a) will produce an empty histogram (where all the weight is actually in one single bin) and how the different levels of teeth in (b) and (c) appear in the histogram.

This descriptor represents well the characteristics of the teeth of a leaf, and is generally able to reflect the levels of dentition even when there are several. However, it has the disadvantage of compressing the information and losing trace of the spatial repartition of teeth along the margin. It is still a very dedicated tool that captures with accuracy what needs to be observed on a leaf margin.

Local Deformable Curves for Tip and Base

Concerning the tip and the base of the leaf, other regions of great interest for identification, a single curvaturescale measure would not be enough (especially at the base where the hypothetical presence of the petiole makes this measure irrelevant) and a finer way to characterize the local shape of the contour around these points is a necessity.

Similarly to our approach for the global shape, we decided to use a deformable model approach, defining a simple model able to re-create the various shapes that can be found in the botanical literature. And again, we expect the construction parameters, which will be optimized during the model's evolution, to be significant and representative measures for the resulting shape.

Parametric Bézier Curves

The models for the base and for the tip are basically defined in the same way. If we take for instance the tip, the model is built on the tip point T that has been located based on curvature (6.1.4). We consider then the contour around this point as two curves that join at the tip, making a pointy, blunt or even squared shape, with either convex or concave edges. The contour can then be modelled by two simple Bézier curves, that are expressive enough to form these shapes, the two sides being not necessarily symmetrical, but of equal length.

The set of parameters we use to build these two curves, or more precisely the control points they are built upon, are the following :

• α, the global angle of the model • o, the orientation angle relatively to the leaf axis • (α t1 ) l,r , for each side, the tangent angle at the tip point • (α t2 ) l,r , for each side, the tangent angle at the end point • (δ t1 ) l,r , for each side, the distance of the 1st control point from the tip • (δ t2 ) l,r , for each side, the distance of the 2nd control point from the end The model's construction is illustrated in Figure 17. The angle α defines two end points T l and T r as the opposite vertices of an isosceles triangle. The apex of the triangle is the fixed tip point T , the length of the two equal sides is set constant and the main axis differs from the leaf axis (found with the polygonal model) of an angle o. Then for each side apart, the tangent lines at the tip and at the corresponding end point are defined by angles α t1 and α t2 , a positive angle placing the tangent towards the outside in every location. The distance parameters δ t1 and δ t2 define the position of the two control points (t 1 ) l,r and (t 2 ) l,r on their respective tangent lines. And the 4 points T , t 1,l , t 2,l and T l (resp. T , t 1,r , t 2,r and T r ) make it possible to build the Bézier curve.

Fitting the curves

The main goal is to obtain curves that are as close as possible to the leaf contour. Consequently the model's evolution has to be driven by an energy based on the distance to this contour. The distance transform of the final contour Γ gives a distance map indicating for each pixel x in the image its distance d(x, Γ) to the leaf contour. The distance of the two end points to the contour are considered with a greater weight because these points need to be actual contour points for the tangent parameters to be relevant. If we consider the model Ψ formed by the two curves built with parameters

P = {α, o, (α ti,k ) i=1,2 k=l,r , (δ ti,k ) i=1,2 k=l,r
}, the external energy term is then defined as :

E ext (P) = x∈Ψ( T ,P) (d(x, Γ)) + η (d(T l , Γ) + d(T r , Γ)) (9)
The curves are free to evolve, and may take asymmetrical shapes, but it is important to limit their expressiveness to plausible shapes and avoid aberrant behaviour. This is why we added soft symmetry constraints, since leaves are mostly symmetrical objects, in the form of an internal energy, as a weighted sum of the differences between parameters for the left curve and the right curve :

E int (P) = 2 i=1 w α |α ti,l -α ti,r | + w δ |δ ti,l -δ ti,r | (10)
And combining those two energies, the evolution process is strictly the same as described for the polygonal model (5.2.2), selecting at each step the variation of one parameter that creates the greatest decrease in the energy. The model is initialized with an angle α set to the corresponding angle parameter α * T or α * B in the polygonal model, and all the other angle parameters set to 0 so that the curves are initially straight. An example of such evolution is displayed in Figure 18. The presence of a petiole, the stalk linking the leaf to the stem, is a source of trouble when considering the shape of the base. One of the main problems is the uncertainty of its presence, some species having very short to absent petiole (sessile leaves), or the petiole being too different in terms of color to eventually be part of the contour. However, when it is there, it will appear as a major convexity in the base area, which generally leads to placing the base point B (around which the base model is built) at the end of the petiole. The estimated "base shape" in this case would be irrelevant, since it would represent more the shape of the petiole.

To overcome this problem, the base point is made moveable in the base model, along one direction only : the model axis. To include it in the deformation process, we added a shift parameter s that defines the position of the actual leaf base B0 on the axis defined by the detected base B and the orientation o. The control points are then built with B0 as base point (the end points being still built from B) and the shift, though being meaningless for shape description, is optimized along with the other parameters. To make sure that the base will move, the curves are pushed to remain convex (which is generally the case if we consider botanical base shapes) by adding a component in the internal energy that pushes the angles to remain positive : As shown in Figure 19, even when starting with an erroneous base point, the model will be able to fit the actual leaf margin, with the base moving up along the petiole. This way, the angles obtained at the end of the evolution have a real significance, which would not be the case if the base remained in its first location.

Êint (P) = E int (P) -γ i=1,2 k=l,r min(α ti,k , 0) (11)
The descriptors introduced in this section use very explicit representations of the contour, and can be expected to capture the specificities of leaves at a high level. It could be interesting to connect them with a semantic representation, such as the terms used in botany. This perspective holds furthermore a great potential for educational and interactive purposes.

Identfication Results

Datasets

The database used to test our algorithms is a subset of the Pl@ntLeaves [START_REF] Goëau | The CLEF 2011 Plant Images Classification Task[END_REF] database, keeping only 3603 leaf images (out of 5436) on white, plain or natural backgrounds of the 50 species (out of 71) with non-compound leaves. This database is divided into two sets, Train and Test, containing leaves from different individuals.

Classification Method

To assess the relevance of our descriptors, and compare their performance with more usual generic shape descriptors, we decided to use a basic classification method, assuming that results with a simple approach measure accurately the significance of features. The descriptors are separated into independent sets, since we consider they represent uncorrelated features of the leaf's morphology. Samples are first classified using only one set of descriptors, and results of such classifications are to be fused together to obtain a final class.

The descriptors we extracted on all the leaves in the database can be distinguished between those we introduced : And some usual shape descriptors to perform comparisons :

• Central moments m i, j [CM] • Hu moments (h i ) [HM] • Centroid contour distance curve (∆(u)) [CCD]
The moments have interesting invariance properties that make them popular general shape descriptors [START_REF] Wang | Classification of Plant Leaf Images with Complicated Background[END_REF][START_REF] Mokhtarian | Matching Shapes with Self-Intersections: Application to Leaf Classification[END_REF]. We extract the central moments of degrees 2 and 3, along with the 7 Hu moments over the segmentation results. The CCD curve is also commonly used [START_REF] Wang | Leaf Image Retrieval with Shape Features[END_REF][START_REF] Teng | Leaf Segmentation, Its 3D Position Estimation and Leaf Classification from a Few Images with Very Close Viewpoints[END_REF] and the one we compute is a function of the normalized arc-length which starts at the detected tip point of the leaf, making comparisons easier, and allowing to compute means without shifting and resampling.

The database formed by all the parameters from all the images is first centered and normalized, so that every parameter has its global mean set to 0 and its standard deviation set to 1. Assuming that each parameter simply follows a Gaussian distribution, we compute class centroids and standard deviations with examples from the Train dataset. We chose to use the number of lobes n L from the polygonal model as a first filter, computing actually, for each species, one class model per plausible number of lobes. For each value of n L up to 3, and for each species S we build a class Φ S,n L = µ S,n L ,i , σ S,n L ,i if at least 10% of all the species examples have this value.

It is important to notice that we extracted two different types of descriptors here: those for which each parameter taken independently conveys significance ( P Π,i , P T,i , P B,i , M i, j , (H i )) and those which need to be considered as a whole ((H CS (k, s)), (D(u))). Considering that one single bin of a histogram provides interpretable information does not make sense, and the same goes for the distance to the center of one given contour point.

For the first type, a class can be seen as an hyper-ellipsoid in the parameter space, whose axes are given by a covariance matrix. Again for simplicity purposes, we assume here that this matrix is always diagonal, making the abusive hypothesis that the descriptors are not correlated. On the other hand, for the other descriptors, speaking of a parameter space seems inappropriate, and it is easier to see a class as an envelope inside which the curves, or the histogram peaks, have to fit. To classify a new example, we will simply have to compare its descriptors with the centroids of the classes sharing the same number of lobes n L . For each one of the 6 sets of parameters, we compute a distance term. The distance for the first type of descriptors is an estimated Euclidean distance to the surface of the class ellipsoid in the parameter space. Its expression, for a set of descriptors P, is given by:

D(P, Φ S,n L ) = P -µ S,n L 2 max 1 - 1 P -µ S,n L M , 0 (12)
More details on what is represented by this distance are to be found in Appendix C. In this equation P -µ S,n L M is the Mahalanobis distance, which is simply in our diagonal case a normalized Euclidean distance:

P -µ S,n L M = i P i -µ S,n L ,i 2 σ 2 S,n L ,i (13)
For the other descriptors, the class model can be seen as thick curve, or surface in the case of the histogram, to which we want to compare another regular curve or surface. A measure that is used to compare two CCD curves (∆ 1 (u)) and (∆ 2 (u)) of same length L, but not necessarily aligned, is:

D(∆ 1 , ∆ 2 ) = min k=0..L-1            L-1 u=0 (∆ 1 (u) -∆ 2 (u + k mod L)) 2            (14)
In our case, since the curves are aligned, there is no need to compute L different sums, and we compute the distance of a curve (∆(u)) u=0..L-1 to a mean thick curve centered at each point on µ S,n L (u) and of width 2σ S,n L (u). This curve can be written ( ∆S,n L (u)) u=0..L-1 = [µ S,n L (u) -σ S,n L (u); µ S,n L (u) + σ S,n L (u)] u=0..L-1 and the distance is then given by :

D(∆, Φ S,n L ) = L-1 u=0 min δ∈ ∆S,n L (u) (∆(u) -δ) 2 = L-1 u=0 max |∆(u) -µ S,n L (u)| -σ S,n L (u), 0 2 (15)
We use the same idea to compute a χ 2 -like distance between a new histogram and the thick histogram surface representing the class:

D(H CS , Φ S,n L ) = k s max |H(k, s) -µ S,n L (k, s)| -σ S,n L (k, s), 0 2 H(k, s) + µ S,n L (k, s) + σ S,n L (k, s) (16)
Each one of the 6 distance terms corresponding to the 6 sets of descriptors is then weighted differently after having learned the classes, dividing it by the average distance of the correct class over the training base, which weights it accordingly to its significance. The final distance we use for classification is simply the sum of these weighted terms, and we finally produce a list of species ordered by their distance.

Performance Estimation

We compared the classification results obtained using different sets of descriptors. First, we used only the descriptors introduced in our work [MP+B/T +HCS]. These three sets were designed to represent the properties of a leaf both globally and locally, and it is no surprise that they give better results than generic shape descriptors alone [CM+HM+CCD]. To be fair, we considered that the curvature-scale histogram, based on curvature, captures details that these descriptors can not. This means that the actually relevant comparisons are to be made between [CM+HM+CCD] and [MP+B/T], and also between [CM+HM+CCD+HCS] and [MP+B/T+HCS]. Finally, we want to measure the information we miss with our three sets of dedicated descriptors by comparing the classification scores obtained with [MP+B/T+HCS] and with [MP+B/T+HCS+CM+HM+CCD].

We computed classification rates for the different types of images in the Pl@ntLeaves Database (Scan, Pseudoscan and Photograph) and measured the presence of the true species among the top k answers (k going from 1 to 10) returned by the classification algorithm. These measures were obtained on the Train database, using a cross validation approach: class models were learned on a random two thirds of the database and scores estimated on the remaining third.

Figure 20 presents these results for the different sets of parameters. Obviously our descriptors [MP+B/T+HCS] perform better than generic shape descriptors alone [CM+HM+CCD] (dark blue and orange curves) since they take more aspects into account, such as the curvature information. What appears here is first the great improvement in performance induced by the use of [HCS] in the classification, when we look at the gap between the curves for [CM+HM+CCD] and [CM+HM+CCD+HCS] (orange and red curves) and between [MP+B/T] and [MP+B/T+HCS] (light blue and dark blue curves). It also seems that, as far as the general shape is concerned, our dedicated descriptors [MP+B/T] are sufficient to outperform the joint moments and CCD [CM+HM+CCD] (light blue and orange curves). However, when all the descriptors are taken into account, the performance seems better, yet slightly, which tends to prove that some of the shape characteristics are not captured by our descriptors. A more objective performance measure is given by the official score computation tool for the Pl@ntLeaves Test database, that considers the classification to be correct only if the true species is the first (top-1) answer. The scores we present for our classification in the first line of Table 1 are to be compared with the best fully-automatic participation submitted to ImageCLEF 2011 [START_REF] Goëau | Participation of INRIA & Pl@ntNet to ImageCLEF 2011 Plant Images Classification Task[END_REF] whose results are recalled on the second line, as well as our former participation and the maximal reachable scores given our limitations (no compound species, pictures of one single leaf).

Looking Further

The system presented in this article constitutes a solid starting structure for the recognition of tree species from natural leaf images. The use of scarcely employed curvature-based features proves to be an essential asset for identification, as well as intuitive leaf shape characteristics. Everything relies however on the robustness of the segmentation part which is obviously the toughest task in a complex natural environment. The methods we presented deal with a good number of obstacles one can encounter in such images, but are not yet the perfect key to unlock this problem. A survey of other existing image models and segmentation techniques is engaged to assess their effectiveness in our context.

We aim at recognizing most of native European tree species, which sum up to nearly 150, but the models, upon which our recognition is built, need extension and refining to fit more complicated leaf shapes. The case of compound leaves is a challenging problem that will be at the heart of our further developments. With the number of potential classes increasing, it will become also a necessity to reduce the scope of the search, by including geographical information linked to the GPS system that now exists in every smartphone. Knowing in advance which species are likely to be found in the geographical area where the user stands may be a decisive step towards a truly reliable identification.

The implementation of our current algorithms on mobile devices through the Folia application is a way to put them into concrete use, with the hope of making it accessible for users who are not necessarily specialists. The performance in terms of processing time is also a concern, as the execution that now lasts a few seconds on a computer, is generally 10 times longer on a smartphone. The introduction of high-level botany-inspired descriptors makes it also conceivable to develop an educational interface, introducing a semantic level displayed to the user. It is of course a great plus to explain in intelligible terms what led the system to its decision. And it would also be beneficial in terms of identification, where interaction and correction would be possible if the words presented to the user do not fit what lies in front of him.

We hope now to have laid the foundations for a new tree identification application, covering the diversity of European species, and destined to anyone with interest in plants but without the otherwise compulsory botanical background. The goal is still to open a little door to this field, as a way to transmit recognition skills that are today a rare knowledge, and to satisfy in the same time the everlasting desire to understand nature.

We consider then, as an approximation, that the actual probability that the pixel is a leaf pixel, which is somewhere between bel LR⊕AM (L) and pl LR⊕AM (L) to be right in the middle, and compute the dissimilarity term following this assumption. For normalization purposes, this term is only proportional to the probability that the pixel is not a leaf pixel

d Belie f (x) ∝ 1 - bel LR⊕AM (L) + pl LR⊕AM (L) 2 ∝ 1 -m LR⊕AM {L} - 1 2 m LR⊕AM {L, L} (A.7)
This dissimilarity, by definition of the rule of combination, and given that the basic belief assignments are affine functions, will generally be a rational function of degree 2 of d LR and d AM that takes into account more information than a regular linear combination, or a minimal value. This equation leads to:

y * = µ + x -µ (x -µ) T Σ -1 (x -µ) (C.6)
And finally:

x -Φ = x -y * 2 = x -µ - x -µ (x -µ) T Σ -1 (x -µ) 2 = (x -µ)        1 - 1 (x -µ) T Σ -1 (x -µ)        2 = 1 - 1 x -µ M x -µ 2 (C.7)
When x is inside E(Φ), the distance (x -µ) T Σ -1 (x -µ) ≤ 1, leading the distance to be negative. If we want x -Φ to be equal to 0 inside the ellipsoid, the distance can be written as :

x -Φ = x -µ 2 max 1 - 1 x -µ M , 0 (C.8)
If we simplify our initial problem to the case where the covariance matrix is diagonal, with variances σ 2 i i=1..N , the ellipsoid is aligned on the axes, and the Mahalanobis distance becomes a normalized Euclidean distance. The expression for the distance to its surface becomes then:

x -Φ =            N i=1 (x i -µ i ) 2            max             1 - 1 N i=1 (x i -µ i ) 2 σ 2 i , 0             (C.9)
This distance is proportional to the Euclidean distance between a point and the centroid, avoiding risky distortions of the parameter space, and still gives an advantage to highly variable classes. However points that are very distant from the centroid in the first place remain far from the surface, which makes sure that they can be recognized as closer to other centroids, even with smaller variances.

Figure 1 :

 1 Figure 1: Segmentation issues for leaves on natural background without prior knowledge; results obtained with unconstrained active contours

Figure 2 :

 2 Figure 2: Example of the picture taking interface for the mobile application, on simple 2(a) and on a palmately lobed 2(b) leaves

Figure 3 :

 3 Figure 3: Variability of leaf shapes within a single species : Populus alba L.

Figure 4 :

 4 Figure 4: Overview of the tree leaf identification system

Figure 5 :

 5 Figure 5: Comparison between linear regressions of color distributions in L*a*b*, RGB and HSL for a leaf image with reflections, on an initial region (5(a)) and on the whole leaf (5(b))

Figure 6 :

 6 Figure 6: Building a local adaptive mean color : image (a), initial region (b), propagation (c), (d) and final estimated local mean (e)

Figure 7 :

 7 Figure 7: Combination of distance maps of a leaf image (a) obtained from linear regression (b) and local mean (c) color models : using Dempster-Shafer theory (d) compared to the mean distance (e) and the minimum (f)

Figure 9 :

 9 Figure 9: Evolution of the parametric active polygon using the distance map, and redundant lobe suppression

Figure 10 :

 10 Figure 10: Guided active contour results: polygonal model (left) initial contour (center left) guiding shape function (center right) and final contour (right)

1 . 2 , 3 .

 123 Extract points for which the curvature value is a local optimum : D(S ) ← D(S ) ∩ {u, ∀u /|u -u| < S |K(u, S )| ≥ |K(u , S )|} 2. Suppress points the curvature intensity of which is below a threshold k min : D(S ) ← D(S ) ∩ {u, |K(u, S )| > k min } Suppress points that can not be traced to a dominant point at the previous scale (if S > 4): D(S ) ← D(S ) ∩ {u, ∃u /|u -u| < S 2 , u ∈ D(S -1)} 4. Keep only the median point of potential remaining groups of size ≤ S : if u ∈ D(S ): g u = {u , |u -u| < S 2 , u ∈ D(S )} D(S ) ← (D(S )\g u ) ∪ {median(g u )}

Figure 12 :

 12 Figure 12: Traced dominant points over scale (c) and their curvature, obtained from the curvature-scale space transform (b) of a leaf contour (a)

Figure 13 :

 13 Figure 13: Labelling contour parts corresponding to leaf areas

Figure 14 :

 14 Figure 14: Various leaf contours with detected base, tip, teeth and pits ; tip area in red, base area in dark blue ; convexities in orange, concavities in blue, brightness representing curvature intensity, extent representing scale.

Figure 15 :

 15 Figure 15: Examples of leaf margin shapes : Entire, Denticulate, Dentate, Sinuate, Lobate. Images taken from[START_REF] Coste | Flore descriptive et illustrée de la France de la Corse et des contrées limitrophes[END_REF]

Figure 17 :

 17 Figure 17: Construction of the two Bézier curves defining the tip model

Figure 18 :

 18 Figure 18: Evolution of the tip model, based on the distance transform of the leaf contour

Figure 19 :

 19 Figure 19: Evolution of the base model, with moving base point

Figure 20 :

 20 Figure 20: Classification rates on the Pl@ntLeaves Database on scan (a) pseudoscan (b) and photograph (c) images

Figure B. 22 :Figure B. 23 :

 2223 Figure B.22: Overlap curves for segmentation with and without polygonal approximation and constraint

•

  Parameters of the polygonal model P Π,i [MP] • Parameters of the base and tip models P B,i and P T,i [B/T] • Curvature-scale histogram (H CS (k, s)) [HCS]

Table 1 :

 1 Official scores on the Pl@ntLeaves Test database

		Scan	Pseudoscan	Photograph	Mean
	[MP+B/T+HCS]	0.581	0.538	0.374	0.498
	INRIA Imedia 2011	0.685	0.464	0.197	0.449
	LIRIS 2011	0.546	0.513	0.251	0.437
	Maximal Scores	0.824	0.835	0.716	0.792

http://leafsnap.com : developed by researchers from Columbia University, the University of Maryland, and the Smithsonian Institution[START_REF] Belhumeur | Searching the World's Herbaria: A System for Visual Identication of Plant Species[END_REF]

This color representation has the double-edged properties of generalization, making it possible for leaf pixels that were not present in the initial area to be considered as close, but risking in the same time to include non-leaf pixels that would fit the model. It remains a global view of the leaf's color that does not consider local distribution

1 https://itunes.apple.com/app/folia/id547650203 $ This work has been supported by the French National Agency for Research with the reference ANR-10-CORD-005 (ReVeS project).

Appendix A. Evidence-based Dissimilarity Computation

We consider the problem of determining the actual dissimilarity of a pixel x to the leaf color model, using an evidence-based combination of its distance d LR = d LinReg (x) to the global model and its distance d AM = d Adaptive (x) to the local adaptive mean.

Considering the binary problem of knowing whether the pixel x is a leaf pixel (L) or not ( L), we place ourselves in a universe X = {L, L}. In the following we will work in the power set P(X) = {L}, { L}, {L, L} .

The first step is then to define two basic belief assignments m LR and m AM over P(X). As explained in 4.3, d AM is only used to put belief mass on the L hypothesis whereas d LR does on L. The functions we use to assign masses are affine functions of the distances that have to remain between 0 and 1. The coefficients λ 0 , λ 1 , α 0 and α 1 have been set experimentally, but could also be have been learned.

Dempster's rule of combination provides a way to compute the joint mass distribution m LR⊕AM = m LR ⊕ m AM , resulting from the combination of evidence from both sources. It relies on a measure K of the conflict between the two distributions:

The joint distribution defines a couple of belief and plausibility of being a leaf pixel bel(L) and pl(L), the definition of which is given by :

Appendix B. Segmentation Results

To measure the performance of our segmentation algorithm, we performed measurements on 226 manually segmented natural background images. The value we measure is an overlap factor ρ between the final region we obtain after the evolution of the guided active contour Ω, and the manual ground truth Ω T .

Though it is not the most accurate measure of how well the algorithm will perform afterwards, it is still a less tolerant measure than, for instance, the Dice index or other similar measures, and has the advantage of being scale invariant and always between 0 and 1.

For comparison purposes, we estimated this overlap factor for segmentation produced with various dissimilarity maps : with the linear regression alone [LR], with the adaptive local mean [AM], with the mean of those two, with their minimum, and finally with the evidence-based combination. The results of this comparison are presented in Figure B.21 as cumulated histograms of the overlap, representing for each overlap value, the percentage of the images for which this value is reached. Such curves are obviously better as they tend to the upper-left corner. This underlines the benefit of a combination of the global and local approaches, and also the interest of introducing evidence in this combination process. We also compared our results with the segmentation obtained with the same dissimilarity map (evidence-based combination) but without the first step of computing the polygonal model. The region is initialized as an initial polygon, but then it simply tries to evolve and minimize the energy in which the weight ω Π corresponding to the shape constraint is set 0 (there is anyway no shape to constrain it in the first place). The resulting overlap curves are shown in Figure B. [START_REF] Cheng | Component Optimization for Image Understanding: A Bayesian Approach[END_REF] where it appears that, if the polygonal constraint seem to lower performance for very easy leaves (overlap over 0.95) it has a significant influence to produce good results on more complicated images.

To illustrate the procedure of our segmentation algorithm,

Appendix C. A distance for classification

We consider a class Φ in a N-dimensional parameter space to be represented by its centroid (µ i ) i=1..N and its covariance matrix Σ. Given a parameter vector (x i ) i=1..N , we want to compute its distance to the centroid while taking into account the variability of the class. A standard measure would be the Mahalanobis distance:

This distance tends however to stretch or contract the parameter space in different ways for the different classes, which is a disadvantage in our case where the species may have very different variabilities over different parameters. The risk is that a point which is very close (in terms of Euclidean distance) from a centroid may appear closer from a more distant one if the class it represents has a lot of variability. Very variable classes tend to swallow up more restricted ones, that will always appear more distant.

To avoid this distortion, we want to always compute distances with the same scale, as does the Euclidean distance. To include the width of each class in the computation, we represent a class by the hyper-ellipsoid centered at µ and with axes and dimensions determined by the covariance matrix Σ. This volume is then defined as:

If the point x lies inside this ellipsoid, we can consider to be within the class Φ, and then we should have ∀x ∈ E(Φ), x -Φ = 0. Outside of it, we want to estimate the distance of x to the surface of E(Φ) given by: S (Φ) = y, (y -µ) T Σ -1 (y -µ) = 1 (C.3) The distance we chose to compute is the distance from x to the point y * of S (Φ) that is in the same direction as x from the centroid µ.


Guillaume Cerutti

Laure Tougne

Julien Mille

Antoine Vacavant

Didier Coquin

A MODEL-BASED APPROACH FOR COMPOUND LEAVES UNDERSTANDING AND IDENTIFICATION

Keywords: plant recognition, compound leaf, deformable templates, image segmentation, active contours, classification

In this paper, we propose a specific method for the identification of compound-leaved tree species, with the aim of integrating it in an educational smartphone application. Our work is based on dedicated shape models for compound leaves, designed to estimate the number and shape of leaflets. A deformable template approach is used to fit these models and produce a high-level interpretation of the image content. The resulting models are later used for the segmentation of leaves in both plain and natural background images, by the use of multiple region-based active contours. Combined with other botany-inspired descriptors accounting for the morphological properties of the leaves, we propose a classification method that makes a semantic interpretation possible. Results are presented over a set of more than 1000 images from 17 European tree species, and an integration in the existing mobile application Folia 1 is considered.

INTRODUCTION

When considering a tree identification mobile application, leaves are an obvious choice for recognition. They can be found all year long, are plane enough to be easily photographed, and show properties that make the identification possible. However, leaves are natural objects whose morphological diversity makes modelling complicated, not to say impossible, and distinguishing between leaf types is necessary. This work concerns only compound leaves, for which a single leaf is actually divided in many leaflets.

The proposed method aims at modelling explicitly the disposition, the global shape and the local specificities of the leaflets in order to classify photographs of leaves in a natural environment into a list of species. The use of high-level, independent descriptors offers the possibility of an explanatory process, putting semantic concepts on what is recognized, which may be of great interest for the user. Figure 1 gives an insight of the proposed process.

This work has been supported by the French National Agency for Fig. 1. Overview of the compound leaf recognition process Section 2 presents other works addressing close topics. The specific models used to represent and segment compound leaves are introduced in Section 3. Section 4 expounds the descriptors and classification we use, as well as some results, and perspectives of future work are given in Section 5.

RELATED WORK

Plant recognition is a topic of interest in image processing, mostly in the context of leaf image retrieval. Some authors [START_REF] Belhumeur | Searching the world's herbaria: A system for visual identification of plant species[END_REF] even share our goal of conceiving a mobile application with great success2 , though being designed essentially for white-background images. The problem of segmenting leaves from a natural environment appears indeed to be much more challenging and is addressed by few other authors, using very complex methods [START_REF] Teng | Leaf segmentation, its 3d position estimation and leaf classification from a few images with very close viewpoints[END_REF][START_REF] Wang | Classification of plant leaf images with complicated background[END_REF].

Segmentation of natural objects is a context where the introduction of prior shape knowledge could be very beneficial. Deformable models, which can be traced back to active contours [START_REF] Kass | Snakes: Active contour models[END_REF], are a popular way to achieve this. Deformable templates modeling complex objects [START_REF] Yuille | Feature extraction from faces using deformable templates[END_REF], strongly constrained templates, such as active shape models [START_REF] Cootes | Active shape models-their training and application[END_REF] or level-set active contours with shape priors [START_REF] Cremers | Diffusion snakes: introducing statistical shape knowledge into the mumford-shah functional[END_REF] constitute ways of including prior knowledge in the segmentation. However, their shapes generally lack the necessary flexibility and expressiveness to capture the diversity of leaf shapes.

Concerning leaf shape description, many works apply established shape descriptors such as the Inner-Distance Shape Context [START_REF] Belhumeur | Searching the world's herbaria: A system for visual identification of plant species[END_REF], moments [START_REF] Wang | Classification of plant leaf images with complicated background[END_REF], Centroid-Contour Distance curves [START_REF] Teng | Leaf segmentation, its 3d position estimation and leaf classification from a few images with very close viewpoints[END_REF] or Curvature-Scale Space transform [START_REF] Mokhtarian | Matching shapes with self-intersections: Application to leaf classification[END_REF]. Such descriptors were not designed to take into account the nature of the object, even if they fit quite well with its specificities. On the other hand, some explicit geometric descriptions of the leaf morphology have been proposed [START_REF] Arora | A plant identification system using shape and morphological features on segmented leaflets[END_REF][START_REF] Caballero | Plant species identification using leaf image retrieval[END_REF].

DEFORMABLE COMPOUND LEAF MODELS

Similarly to what we have done in the case of simple leaves [START_REF] Cerutti | Guiding active contours for tree leaf segmentation and identification[END_REF], the segmentation method we propose relies on the prior evaluation of a flexible leaf model, designed to cover the variety of leaf shapes. Such a model constitutes a way of providing a first segmentation as well as a description of the leaf's global shape that can later be used for recognition.

Deformable Compound Leaf Model

A first model that tries to estimate the number and disposition of the leaflets was introduced in [START_REF] Cerutti | Reves participation -tree species classification using random forests and botanical features[END_REF] and was modified here to achieve better robustness. It is crucial to point out that very often, the number of leaflets is not the number of connected components one would find in the segmented image, given that the overlap between leaflets is a constant risk. Estimating the actual number location of those leaflets beforehand would therefore be a guarantee that the resulting shapes will be correctly described.

This model makes assumptions about the axial symmetry of the leaves, and the regularity of the leaflets in size and orientation, that are not strictly speaking always true, but are satisfying for a first approximation. As shown in Figure 2, the model represents leaflets by a variable number n L of pairs of circles (C 2l , C 2l+1 ) l=1..n L symmetrically positioned on either side of a curved axis defined by two points T (for the top leaflet C 1 ) and B (for the base of the petiole) and a curvature parameter k. The additional parameters used to build the model are the following :

• (p l ) l=1..n L ,

the position of pairs of circles on the axis

• d, the distance of all the circles to the axis • r, the radius of all the circles The estimation of the optimal model M on the actual image is performed through the minimization of an energy function by successive variations of the parameters. This energy term is based on a color dissimilarity map estimated beforehand (Figure 3(b)) that accounts for each pixel's likelihood of being part of the leaf, given only its color. It is In the context of compound leaves, we model the color by a single Gaussian (µ, Σ) in the L*a*b* colorspace, which is a perceptually more accurate representation than RGB. The dissimilarity of a pixel p to this color model is then simply given by the Mahalanobis distance d(p, µ, Σ) with respect to the computed Gaussian parameters.

The energy function the model minimizes during its evolution is simply the sum of the dissimilarity over the region defined by the model, minus a maximal dissimilarity that acts as a balloon force:

E(M ) = p∈ 2n L +1 i=1 Ci d(p, µ, Σ) -d max (1)
The number of leaflets n L has to be optimized separately, since the changes it produces in the shape of the model are too important to consider it in a gradient-descent like approach. To overcome this difficulty, the model is initialized with an excessive number of leaflets. Following a process close to simulated annealing, a temperature variable is slowly decreasing through the evolution, and brutally raised in regular cycles. At the end of each of those cycles, circles that have grouped in actual leaflets are simply suppressed, a decision made comparing the distance between the centers of two consecutive pairs of circles and the radius r. This way, unlike what was done in [START_REF] Cerutti | Reves participation -tree species classification using random forests and botanical features[END_REF] where the number of leaflets was approximated a posteriori, the convergence of the model ideally shows one single circle per leaflet (Figure 3(c)).

Deformable Joint Polygonal Leaflet Models

To capture the global shape of the leaflets, we rely on the polygonal leaf model introduced in [START_REF] Cerutti | A parametric active polygon for leaf segmentation and shape estimation[END_REF], making the assumption that all the leaflets share the same shape. This is generally true (and species are described in the literature by the shape of their leaflets anyway) even if little exceptions (appreciably different shapes for the top leaflets) might cause some problems.

Consequently we propose a novel joint approach where we place one model Π i for each of the 2n L + 1 leaflets obtained after the evaluation of the leaf model, and constrain them to have the same shape parameters. Only the points defining the base and apex of each leaflet vary independently. This new model {Π i } 2n L +1 i=1 evolves the same way as the previous one, minimizing the same energy function, but with no suppression of overlapping leaflets. Constraints are added throughout the evolution, on the shape parameters so that leaflets keep leaf-like shapes, and on the points so that pairs of leaflets remain locally symmetrical, under the form of an internal energy term.

The process of fitting a model to a single leaflet, which may overlap with its neighbours is risky, but the fact that all the leaflet models are evaluated simultaneously ensures that they self-constrain (Figure 3(d)) The parameters of the optimal models

{Π * i } 2n L +1 i=1
we obtain are therefore more robust than if they were computed independently on each leaflet.

Multiple Active Contour Segmentation

To obtain different interpretable contours corresponding to each leaflet, the natural choice is to deform the polygonal shapes resulting from the previous step towards the actual contours. Once again, it is very interesting to perform this step in a joint fashion, so that the contours we are evaluating act as a constraint on each other. The contour of each leaflet is represented by a regionbased active contour model, using an extension of the levelset framework to the case of multiple regions [START_REF] Brox | Level set segmentation with multiple regions[END_REF] and an implicit definition approximating the original level-set evolution [START_REF] Shi | A real-time algorithm for the approximation of level-set-based curve evolution[END_REF]. This model have the limitation that a pixel can only be part of one region, so that the contours do not interpenetrate.

The 2n L + 1 regions {Ω i } 2n L +1 i=1 evolve simultaneously, minimizing the energy functional :

E({Ω i } 2n L +1 i=1 ) = 2n L +1 i=1 ω L E Leaf (Ω i ) + ω Π E Shape (Ω i , Π * i ) + ω ∇ E Gradient (Ω i ) + ω S E Smooth (Ω i ) -ω Balloon (2)
The energy is composed of an external term based on the same color dissimilarity as in 3.1 and 3.2 and on the gra-dient, and an internal term containing a shape constraint to remain close to the corresponding polygonal model Π * i and a smoothness term. After evolution, the result is a set of independent contours (Figure 3(e)) that can be studied to extract local properties.

SPECIES CLASSIFICATION & RESULTS

Shape Descriptors

The descriptors we use to learn and classify compound leaves are directly inspired from botany and aim at capturing in a decorrelated way the various morphological specificities of the leaf. This decision falls within the choice of an explanatory recognition process, where the justifications leading to the classification should be displayed to the user of the application using high-level semantic concepts. The introduced descriptors consist of:

• Parameters of the compound leaf model (3.1)

• Parameters of the polygonal leaflet model (3.2)

• Averaged parameters of base and apex models ( [START_REF] Cerutti | Guiding active contours for tree leaf segmentation and identification[END_REF])

• Averaged CSS-based contour parameters ( [START_REF] Cerutti | Reves participation -tree species classification using random forests and botanical features[END_REF])

The features describing the base, apex and margin shapes, are extracted and averaged over a subset of the leaflet contours. The idea is to discard the overlapping leaflets, for which the contour is unlikely to be accurate. To perform this selection, we compute an overlap score for every pair of leaflets, and only those which have a low score with all the others are kept. This step ensures that the extracted parameters will represent actual leaflet shapes and not erratic shapes resulting from an uncertain border between two overlapping leaflets.

Learning and Classification

We tested our methods on a subset of the Pl@ntLeaves II Dataset [START_REF] Goëau | The imageclef 2012 plant image identification task[END_REF] considering only species with compound leaves. The resulting set consists of 1040 images spread over 17 different species, the images being of three types : scans, scan-like (photographs on a light plain background) and photographs.

The database formed by all the parameters for all the examples is first normalized and centered. The classification algorithm is rather naive, as it simply computes for a given species S a class model Φ S per species, keeping only the mean µ S,p and standard deviation σ S,p of each parameter over all the examples for the species.

To classify a new example, we compute the distance of its parameter vectors P to all the class models. Rather than using the Euclidean distance that does not account at all for intraclass variability, or the Mahalanobis distance that distorts the parameter space and penalizes classes with low variability, we estimate the distance to the surface of the ellipsoid defined by the means and standard deviations in the parameter space. This distance is given by:

D(P, Φ S ) = P -µ S 2 max 1 - 1 P -µ S M , 0 (3)
In this equation P -µ S,n L M is the Mahalanobis distance, which is simply in our diagonal case a normalized Euclidean distance. These distances are computed separately for the 4 sets of descriptors presented in 4.1, which are meant to represent independent leaf features. Each of those distance terms is then weighted accordingly to its significance, estimated by observing the average distance to the correct class over the training base. The sum of the weighted distances is used to produce a ranked list of species, the top five of which are presented to the user as a result.

Results & Evaluation

We measured our results in terms of correct classification rate, not only for the first species of the resulting list, but also for the top k answers, k ranging from 1 to 5. This is a way to evaluate the performance relatively to the application of our method in a pertinent way. The classification rates are 61% on scan images, 60% on scan-like images and 43% on photographs. One interesting fact is that the presence of the correct species in the list presented to user climbs up to 95% in the case of plain background images, and 86% for natural environment photographs. A more detailed view of these results can be seen in Figure 4. In addition to this performance measure, we tried to evaluate the robustness of our approach by comparing the number of leaflets we estimate with the actual number on the leaves. This estimation is performed along the evolution of the compound leaf model and is of crucial importance for the following of the process. It also has a visual interest for the user witnessing its evolution, and it is beneficial from this point of view that this phase sticks with the actual visible content of the image.

The 1040 images were manually labelled with their number of leaflets, and the estimated number compared with this ground truth. Our method deals with overlapping leaflets, which is basically impossible to do without introducing a kind of model. For the sake of comparison, we also performed a concurrent, non model-based, estimation of this number, by thresholding the color dissimilarity map using Otsu's method, cleaning the binary image with some mathematical morphology operations, and counting the connected components.

We measured the performance of both approaches by computing the mean squared error (MSE), in a global fashion first, and also for each true number of leaflets. When the thresholding based method has a MSE of 28,67, our estimation reaches 13,11. The error by number of leaflets presented in Figure 5 allows a finer analysis and shows that our approach has a significant advantage when the number of leaflets and the subsequent probability of overlap becomes high. Fig. 5. Mean Squared Error of the leaflet estimation for our method (-) and the Otsu based method (-)

CONCLUSIONS & PERSPECTIVES

The approach presented in this paper introduces a modelbased method to solve a complex problem of image understanding, providing a high-level interpretation of the image content. The description of compound leaves it produces appears to be more accurate than what can be obtained with less dedicated approaches, through the use on prior botanical knowledge, and constitutes a great improvement compared to our primary results.

The performance in terms of species identification is very satisfying, and the fact that high-level representations are used throughout the process makes an explanatory recognition process using botanical semantics possible. The introduction of this method in the existing mobile application Folia is engaged, and will constitute a nice extension towards a more general tree identification helping tool.

Fig. 2 .

 2 Fig. 2. Construction of the compound leaf model

Fig. 3 .

 3 Fig. 3. Example of model-fitting and segmentation of compound leaves

Fig. 4 .

 4 Fig.4. Classification scores for scan (-), pseudoscan (-) and photograph (-) images compared to our primary results[START_REF] Cerutti | Reves participation -tree species classification using random forests and botanical features[END_REF] (dotted lines)

https://itunes.apple.com/app/folia/id547650203

http://leafsnap.com : developed by researchers from Columbia University, the University of Maryland, and the Smithsonian Institution