John Chaussard
email: chaussard@math.univ-paris13.fr

Laurent Noël
email: laurent.noel@esiee.fr

Venceslas Biri
email: v.biri@esiee.fr

Michel Couprie
email: michel.couprie@esiee.fr

A 3D curvilinear skeletonization algorithm with application to path tracing

We present a novel 3D curvilinear skeletonization algorithm which produces filtered skeletons without needing any user input, thanks to a new parallel algorithm based on the cubical complex framework. These skeletons are used in a modified path tracing algorithm in order to produce less noisy images in less time than the classical approach.

Introduction

Path tracing algorithms [START_REF] Kajiya | The Rendering Equation[END_REF][START_REF] Lafortune | Bi-directional path tracing[END_REF] are able to render photorealistic images from a scene description by simulating light propagation. The photorealistic aspect is achieved by simulating diffusion of light: elements of the scene which are illuminated by a light source can cast part of the light they receive. Consequently, parts of the scene which are not directly illuminated by any light source actually receive light, thus giving a realistic effect of light diffusion to the resulting image.

A path tracing algorithm sends rays from the camera, and let them bounce around the scene until they encounter a directly illuminated zone. When a ray hits a point of any surface, it is reemitted in a random direction. Once it reaches a directly illuminated area, all luminance gathered through its journey in the scene is averaged in order to set one of the image pixels, a process repeated for each pixel of the output image. To reduce the noise resulting from this stochastic process, the algorithm actually sends many rays for each pixel of the image: the more rays, the longer the algorithm will take to produce an image, but the more realistic the result will look. A path tracing can be seen as a random walk process through the scene.

In order to avoid infinite bouncing of rays in the scene, a limit is set for the number of bounces a ray can do. If a ray does not reach any directly illuminated area after a given number of bounces, its contribution is considered null. Such lost ray is a waste of computation time, and minimizing this loss would speed up the algorithm (see Our proposition is to reduce both the time taken by a path tracing algorithms and the noise in the output images by guiding rays towards the scene's light sources (see Fig 1.b), thus reducing the number of rays "lost" during the computation. To do this, we perform some precomputation on the void space of the scene (the medium where the rays of light propagate) in order to have, later in the algorithm, a clue on the direction a ray should bounce in order to meet an illuminated area. Unfortunately, such precomputation may take time and become more expensive than the original algorithm.

To keep the precomputation fast, we perform it only on a subset of points of the voids of the scene. Since this precomputation relies on paths and visibility tests, we want this set of points to be representative of both the topology and the geometry of the original scene. For example, it should possess branches propagating in the elongated parts of the scene. A filtered curvilinear skeleton of the scene's void meets all the required properties for our subset of points: it is a thin subset of points, with the same topology and retaining the main geometrical information of the scene. Note that in this application, the filtering of the skeleton (elimination of spurious branches) need to be both efficient and completely automatic, for one cannot ask a user to set parameters for each scene.

The purpose of this paper is twofold: we first introduce a new curvilinear skeletonization algorithm (Sec. 3) in the cubical complex framework (Sec. 2), producing filtered skeletons without needing any user input (Sec. 4). Then, we expose a new method, based on such skeletons, to enhance the results (see Fig 1 .c andd) and the performance of the path tracing algorithm (Sec. 5).

In the 3D voxel framework, objects are made of voxels. In the 3D cubical complex framework, objects are made of cubes, squares, lines and vertices. Let Z be the set of integers, we consider the family of sets F 1 0 and F 1 1 , such that

F 1 0 = {{a} | a ∈ Z} and F 1 1 = {{a, a + 1} | a ∈ Z}. Any subset f of Z n such that f is the Cartesian product of m elements of F 1
1 and (nm) elements of F 1 0 is called a face or an m-face of Z n , m is the dimension of f , we write dim(f ) = m. A 0-face is called a vertex, a 1-face is an edge, a 2-face is a square, and a 3-face is a cube.

Given m ∈ {0, . . . , n}, we denote by F n m the set composed of all m-faces in Z n . We denote by F n the set composed of all m-faces in Z n :

F n = m∈[0;n] F n m . Let f ∈ F n . We set f = {g ∈ F n |g ⊆ f }, and f * = f \ {f }. Any element of f (resp. f * ) is a face of f (resp. a proper face of f ). The closure of a set of faces X is the set X = ∪{ f |f ∈ X}.
Definition 1. A finite set X of faces in F n is a complex if X = X, and we write X F n . Any subset Y of X which is also a complex is a subcomplex of X, and we write Y X.

A face f ∈ X is a facet of X if f is not a proper face of any face of X. The dimension of X is dim(X) = max{dim(f ) | f ∈ X}. If dim(X) = d, then we say that X is a d-complex.
Traditionally, a binary image is a finite subset of Z n (called voxel image when n = 3). To transpose such an image S to the cubical complex framework, we associate to each element of S ⊆ Z n an n-face of F n . Let x = (x 1 , ..., x n ) ∈ S, we define the n-face Ψ (x) = {x 1 , x 1 + 1} × . . . × {x n , x n + 1}. We can extend the map Ψ to sets: Ψ (S) = {Ψ (x)|x ∈ S}. Given a set S ⊂ Z n , we associate to it the cubical complex Ψ (S).

The collapse operation is the basic operation for performing homotopic thinning of a complex, and consists of removing free pairs of faces: Definition 2. Let X F n , and let f, g be two faces of X. The face g is free for X, and the pair (f, g) is a free pair for X if f is the only face of X which strictly contains g.

It can be easily seen that if (f, g) is a free pair for a complex X, then f is a facet of X and dim(g) = dim(f ) -1.

Definition 3. Let X

F n , and let (f, g) be a free pair for X. The complex X \ {f, g} is an elementary collapse of X.

Let Y F n . The complex X collapses onto Y if there exists a sequence of complexes (X 0 , ..., X ℓ ) of F n such that X = X 0 , Y = X ℓ and for all i ∈ {1, . . . , ℓ}, X i is an elementary collapse of X i-1 . We also say, in this case, that Y is a collapse of X.

A parallel directional thinning based on cubical complex

In the cubical complex framework, parallel removal of free pairs can be easily achieved when following simple rules that we give now. First, we need to define the direction and the orientation of a free face. Let (f, g) be a free pair for X F n : we have dim(g) = dim(f ) -1, and it can be seen that g = f ∩ f ′ , where f ′ is the translate of f by one of the 2n vectors of Z n which have all their coordinates equal to 0 except one, which is either equal to +1 or -1. Let v be this vector, and c its non-null coordinate. We define Dir (f, g), called the direction of the free pair (f, g), as the index of c in v. The orientation of the free pair (f, g) is defined as Orient(f, g) = 1 if c = 1, and Orient(f, g) = 0 else. Now, we give a property of collapse (previously proven in [START_REF] Chaussard | Surface thinning in 3d cubical complexes[END_REF]) which brings a necessary and sufficient condition for removing two free pairs of faces in parallel from a complex, while preserving topology. Proposition 4. Let X F n , and let (f, g) and (k, ℓ) be two distinct free pairs for X. The complex X collapses onto X \ {f, g, k, ℓ} if and only if f = k.

From Prop. 4, the following corollary is immediate.

Corollary 5. Let X F n , and let (f 1 , g 1 ) . . . (f m , g m ) be m distinct free pairs for X such that, for all a, b ∈ {1, . . . , m} (with a = b), f a = f b . The complex X collapses onto X \ {f 1 , g 1 . . . f m , g m }.
Considering two distinct free pairs (f, g) and (i, j) for X F n such that Dir (f, g) = Dir (i, j) and Orient(f, g) = Orient(i, j), we have f = i. From this observation and Cor. 5, we deduce the following property. Corollary 6. Let X F n , and let (f 1 , g 1 ) . . . (f m , g m ) be m distinct free pairs for X having all the same direction and the same orientation. The complex X collapses onto X \ {f 1 , g 1 . . . f m , g m }.

Intuitively, we want our thinning algorithm to remove free faces of a complex "layer by layer" and to avoid having unequal thinning of the input complex. Therefore, we want each execution of the algorithm to remove free faces located on the border of the input complex. We define Border (X) as the set all faces belonging to a free pair for X. We now introduce Alg. 1, a directional parallel thinning algorithm.

On a single execution of the main loop of Alg. 1, only faces located on the border of the complex are removed (l. 7). Thanks to corollary 6, we can remove faces with same direction and orientation in parallel (l. 8), while guaranteeing topology preservation. Figure 2 depicts the first steps of the algorithm.

Different definitions of orientation and direction can be given, each corresponding to a different order of free faces removal in the complex and leading to different results. Algorithm 1 can be implemented to run in linear time complexity (proportionally to the number of faces in the complex). Indeed, checking if a face is free or not may be easily done in constant time and when a free pair (f, g) is removed from the input complex, it is sufficient to scan the faces contained in f in order to find new free faces.

Algorithm 1: P arDirCollapse(X, W, ℓ) Data: A cubical complex X F n , a subcomplex W X which represents faces of X which should not be removed, and ℓ ∈ N, the number of layers of free faces which should be removed from X Result: A cubical complex 1 while there exists free faces in X \ W and ℓ > 0 do 2 L = Border (X);

3 for t = 1 → n do 4 for s = 0 → 1 do 5 for d = n → 1 do 6 E = {(f, g) free for X | g / ∈ W, Dir (f, g) = t, Orient(f, g) = s, dim(f ) = d}; 7 G = {(f, g) ∈ E | f ∈ L and g ∈ L}; 8 X = X \ G; 9 l = l -1; 10 return X;

Aspect preservation during thinning

As previously said, our goal is to use the skeleton of the voids of a scene in order to guide light rays from the camera of the scene towards the light. Moreover, this skeleton needs to capture the main geometrical features of the original scene. For example, if the input is a corridor (the void of the corridor), then the skeleton should be a line following the main direction of the corridor.

Generally, two strategies are possible to achieve this goal: find, during the skeletonization process, points whose neighbourhood configuration seems interesting and keep them in the result [START_REF] Couprie | Discrete bisector function and euclidean skeleton in 2d and 3d[END_REF] [7] [START_REF] Chaussard | Robust skeletonization using the discrete lambda-medial axis[END_REF], or choose, before skeletonization, interesting points of the object which should remain untouched, based on a function on these points and a filtering parameter [2] [12].

Algorithm 1 does not necessarily preserve geometrical features of the input object in the resulting skeleton (for example, the skeleton of a corridor could be reduced to a single vertex). In the following, we introduce a new method in the cubical complex framework, requiring no user input, for obtaining a curvilinear skeleton yielding satisfactory geometrical properties. Our method is based on the two previously listed strategies: it finds, during thinning, elements with a specific neighbourhood configuration, and uses a function on these elements to decide whether to preserve them, or not, in the result.

The lifespan of a face

In the following, we define additional functions in the cubical complex, related to the thinning process (Alg. 1), which are essential for the filtering step of the skeletonization. The first one we present is the death date of a face. Definition 7. Let f ∈ X F n . The death date of f in X, denoted by Death X (f ), is the smallest integer δ such that f / ∈ P arDirCollapse(X, ∅, δ).

Intuitively, the death date of a face indicates how many layers of free faces should be removed from a complex X, using Alg. 1, before removing completely the face from X. We now define the birth date of a face:

Definition 8. Let f ∈ X F n . The birth date of f in X, denoted by Birth X (f ), is the smallest integer b such that either f is a facet of P arDirCollapse(X, ∅, b), or f / ∈ P arDirCollapse(X, ∅, b).
The birth date indicates how many layers of free faces must be removed from X with Alg.1 before transforming f into a facet of X (we consider that a face "lives" when it is a facet). Finally, we define the lifespan of a face :

Definition 9. Let f ∈ X F n . The lifespan of f in X is the integer Lifespan X (f ) = +∞ if Death X (f ) = +∞ Death X (f ) -Birth X (f ) otherwise
These three values depend on the order of direction and orientation chosen for Alg. 1.

The lifespan of a face f of X indicates how many "rounds" this face "survives" as a facet in X, when removing free pairs with Alg. 1, and is a good indicator of how important a face can be in an object. Typically, the higher the lifespan is, and the more representative of an object's geometrical feature the face is. The lifespan, sometimes called saliency, was used in [START_REF] Liu | A simple and robust thinning algorithm on cell complexes[END_REF] (with the name "medial persistence") in order to propose a thinning algorithm in cubical complexes based on two parameters.

Distance map, opening function and decenterness map

In addition to the lifespan of a face, the proposed homotopic thinning method uses information on distance between faces in order to decide if a face should be kept safe from deletion. We define hereafter various notions based on distances in the voxel framework.

We set d 1 (x, y) as the L1 distance between x and y (Manhattan distance). Let S ⊂ Z n , we set S c = Z n \ S, and for all x ∈ Z n , the map D 1 (S) : Z n → N is such that D 1 (S)(x) = min y∈S c d 1 (x, y). The maximal 1-ball in S centered on x is the set MB 1 S (x) = {y ∈ Z n |d 1 (x, y) < D 1 (S)(x)}. We set, for all x ∈ S, the map Ω 1 (S) : Z n → N such that Ω 1 (S)(x) = Given S ⊂ Z n , the value of Ω 1 (S)(x) of every x ∈ S can be naively computed by performing successive morphological dilations of values of the map D 1 (S). A linear algorithm for computing the map Ω 1 (S) (with regard to the size of the input image) was proposed in [START_REF] Chaussard | Topological tools for discrete shape analysis[END_REF], and will be explored further in details in a future paper.

Finally, we define the decenterness map:

Definition 10. Given S ⊂ Z n , the decenterness map of S is the map DC 1 (S) = Ω 1 (S) -D 1 (S).

An example of these maps is shown on Fig. 3.

In order to extend all these previous maps defined in Z n to the cubical complex framework, we use the map Ψ -1 , inverse of the bijective map Ψ : Z n → F n n defined in Sec. 2. It is used to project any n-face of F n into Z n . This map induces a map from P(F n n ) to P(Z n ), that we also denote by

Ψ -1 . Given Y ⊂ F n , we set S = Ψ -1 (Y ∩F n n ). We define the map D cc 1 (Y ) : F n → N as follows: for all f ∈ F n , D cc 1 (Y )(f ) = D 1 (S)(Ψ -1 (f )) if f is an n-face max f ∈(ĝ * ∩F n n ) D cc 1 (Y )(g) otherwise Informally, if f is a 3-face, then D cc 1 (Y )(f )
is the length of the shortest 1-path between the voxel "corresponding" to f and the set of voxels corresponding to Y . In the same way, we define Ω cc 1 (Y ) and DC cc 1 (Y ).

Parameter-free filtered thinning

As previously said, we add edges to the set W of Alg. 1 in order to retain, in the resulting curvilinear skeleton, important edges from the original object. Given Algorithm 2: CurvilinearSkeleton(X)

Data: A cubical complex X F 3 Result: A cubical complex Y F 3 1 W = {f ∈ X|Lifespan X (f ) > DC cc 1 (X)(f ) + BirthX (f ) -D cc 1 (X)(f )
and dim(f ) = 1} ; 2 return P arDirCollapse(X, W, +∞); a cubical complex X, if an edge of X has a high decenterness value for X, then it is probably located too close to the border of X and does not represent an interesting geometrical feature to preserve. On the other hand, if an edge has a high lifespan for X, then it means it was not removed quickly, after becoming a facet, by the thinning algorithm and might represent some precious geometrical information on the original object. An idea would be to keep, during thinning, all edges whose lifespan is superior to the decenterness value. Unfortunately, this strategy produces skeletons with many spurious branches in surfacic areas of the original object.

We can identify surfacic areas of a complex as zones where squares have a high lifespan. Therefore, in order to avoid spurious branches in surfacic areas, we need to make it harder for edges to be preserved in these zones. It can be achieved by deciding that an edge will be kept safe from deletion by the thinning algorithm if its lifespan is superior to the decenterness value plus the lifespan of squares "around" this edge. This leads us to proposing Alg. 2.

In order to understand what was realised on line 1 of Alg. 2, we might point out that the birth date of an edge corresponds to the highest death date of the squares containing this edge. Moreover, the map D cc 1 (X) gives, for all 3-faces of X, their death date (as the thinning algorithm naturally follows this map to eliminate cubes from a 3-complex). Therefore, for an edge f of X, D cc 1 (X)(f ) informs us on the highest death date of cubes containing f , also equal to the highest birth date of squares containing f . In conclusion, Birth X (f )-D cc 1 (X)(f ) is an approximation of the lifespan of the squares containing f .

Although the output of Alg.2 may contain 2-faces, the algorithm is said to be a curvilinear skeletonization algorithm because it only adds 1-faces (edges) in the inhibitor set W (used for the visual aspect preservation step) (on line 1 of Alg. 2). For the same reason, we say that the output of Alg. 2 is a curvilinear skeleton. In the scenes studied in Sec. 5, the outputs of Alg. 2 were one dimensional complexes.

Results

Algorithm 2 allows to obtain a filtered curvilinear skeleton from a three dimensional complex. The results presented in Fig. 4 show that the skeletons contain the main geometrical information from the input shapes, and no spurious branch.

Application to path tracing

In this section, we explain how the curvilinear skeleton of the voids of the scene can enhance the path tracing algorithm. We start with some basic elements related to path tracing and then present our modified path tracing algorithm.

The path tracing

Let O be the origin of the camera in the scene. For each pixel P of the final image, let x be the nearest intersection point of the ray (O, --→ OP = -Θ) with an element of the scene. To obtain the luminosity, we must solve the rendering equation [START_REF] Kajiya | The Rendering Equation[END_REF] which is a recursive integral equation, the integrand containing the radiance function that we must compute:

L(x → Θ) = L e (x → Θ) + Φ∈Ωx f s (x, Θ ↔ Φ)L(r(x, Φ) → -Φ)dω Φ
where L is the radiance (a radiometric quantity that represents luminosity) from point x toward direction Θ and L e is the emitted radiance (non null only on light sources). The point r(x, Φ) is the nearest visible point from x in the direction Φ of the hemisphere Ω x . The f s function expresses how much luminosity is exchanged, on point x, between an incoming ray Φ and the outgoing ray Θ.

The previous equation expresses an intuitive idea: the reflected radiance from x towards the camera is the result of computing all incoming luminosity on x scaled by a factor which depends on the material on x and the angle of the ray going from the camera to x. The most common method used to estimate the integral, denoted by L r (x → Θ), is the Monte-Carlo integration which provides the following:

L r (x → Θ) = f s (x, Θ ↔ Φ)L(r(x, Φ) → -Φ) p(Φ)
The function p is a probability density function (pdf) that is used to sample Φ. This estimator is unbiased, meaning that the expected value E[ L r (x → Θ) ] is equal to L r (x → Θ). The variance of the estimator expresses its quality and depends on the chosen pdf p. The best choice is a pdf that matches the shape of the function to integrate (ie. gives high density to samples that have high values for the function and low density to samples that have low values). The strategy of choosing an adapted pdf is called importance sampling [START_REF] Kajiya | The Rendering Equation[END_REF] and is used in global illumination to improve the convergence speed of the algorithms.

The algorithm computes L as follow: it chooses one random direction Φ based on the pdf p and applies the estimator by calling L recursively to compute L r (shoots a new ray in the direction Φ, computes the new hit point with the scene, and computes the new radiance at this point). Finally, it returns the sum of the emitted radiance L e and the result for L r . The recursion stops when either a maximal number of bounces or a directly illuminated area have been reached.

This computation is done multiple times for each pixel. We average the results and get an estimation of the mean radiance passing through the pixel and heading towards the camera. A bad pdf would lead in picking directions that do not reach the light before the end of the recursion, and produce results with a lot of noise in the final image. In the next part, we explain how to produce a pdf based on the curvilinear skeleton.

Skeleton based importance sampling

As stated in the introduction, a curvilinear skeleton of the void (with some preprocessing performed on it) gives us information on which directions the light comes from. Given these directions, we can build a efficient pdf p skel and guide our rays by sampling the hemispheres with p skel . The integrand of L r is:

f s (x, Θ ↔ Φ)L(r(x, Φ) → -Φ).
The most common strategy used to sample Ω x is to use the function f s because it is an input of the algorithm. The term L, representing the distribution of light in the scene, is unknown. Our method gives a way to sample L.

Construction of the importance points

The skeleton of the voids of the scene is computed using Alg. 2 (as shown on Fig. 5 on the right) and is converted into a graph (the nodes are the 0-faces and the edges are the 1-faces). We then compute a set of importance points, which will be used to sample Ω x in the path tracing algorithm. To each node n of the skeleton, one importance point imp n is computed. Intuitively, the importance point associated to n is the direction to follow in order to find a light source.

Let L be the light source of the scene and n L the nearest node of the skeleton that is visible by L. For each node, we compute the shortest path to n L along the skeleton. To do so, we use the Dijkstra algorithm and weight the edges of the skeleton depending on a visibility criteria: the weight of an edge e is 1 if e is visible from n L and 10 else. It results that illuminated paths will be shorter than paths located in dark areas. Let n be a node of the skeleton and V n the set of visible nodes from n along the shortest path toward n L . The importance point imp n associated to n is the barycenter of V n . An example is shown on Fig. 5 on the left. The algorithm can be extended to multiple light sources by taking into account, when computing the importance point of a skeleton node, only the closest light source.

Sampling according to L Given a point x on the scene and a direction Θ, we want to compute L(x → Θ) and then sample the hemisphere Ω x . We look for the nearest skeleton node n to x and its importance point imp n . We sample the hemisphere with a power-cosine pdf centered on ----→ x imp n :

p skel (Φ) = s+1 2π * cos s α with α the angle between ----→ x imp n and Φ, s being a parameter called skeleton strength. The higher s is, the closer to ----→

x imp n we sample.

Results and discussion

Some results are presented on Fig. 6 and Fig. 1 where we can see that our method produces less noisy images compared to the regular path tracing. We present, in Fig 7, the time taken by each method to produce an image of same quality (the quality is measured by the mean square error (MSE) with a reference image) in different scenes: our method is the fastest one.

Conclusion

We presented in this article a new skeletonization algorithm that both preserves geometrical features of objects and produces a pure curvilinear skeleton. These two properties allow us to improve the path tracing algorithm in guiding the rays towards the main illuminated area of the scene. Our algorithm is faster and produce less noise than the classical path tracing method.

Fig. 1 .

 1 Fig. 1. Note how rays are guided toward the illuminated area thanks to our skeleton based algorithm (image b), compared to classical path tracing (image a). Images c and d point out the noise reduction between the classic path tracing (c) and our method (d).

  Fig 1.a).

Fig. 2 .

 2 Fig. 2. Four first iterations of Alg. 1 running on the left-most shape.

Fig. 3 .

 3 Fig. 3. Examples of opening and decenterness map -From left to right: a shape S ⊂ Z 2 (in gray), D1(S), Ω1(S) and DC1(S) (low values have dark colour).

max x∈MB 1 S

 1 (y) D 1 (S)(y): this value indicates the radius of a largest maximal 1-ball contained in S and containing x. If x ∈ S c , we set Ω 1 (S)(x) = 0. The map Ω 1 (S) is known as the opening function of S based on the 1-distance (also called the granulometry function)[START_REF] Matheron | Eléments pour une Théorie des Milieux Poreux[END_REF]: it allows to compute efficiently results of morphological openings by balls of various radius, and gives information on the local thickness of an object.

Fig. 4 .

 4 Fig. 4. Results of algorithm 2 for two shapes: a hand (left) and a statue (right). In each pair, the rightmost image represents the skeleton.

Fig. 5 .

 5 Fig. 5. On the left, we illustrate the importance point impn associated to a node n. The dotted black line shows the closest node to n not included in Vn. On the right, an example of the curvilinear skeleton (in red) obtained in the Sponza scene.

Fig. 6 . 2570 Fig. 7 .

 625707 Fig. 6. Same scene rendered using the classical path tracing (image a) or our method (image b). Images c and d are both details of respectively images a and b

⋆ This work has been partially supported by the "ANR-2010-BLAN-0205 KIDICO" project


Chesner Désir

Caroline Petitjean

Laurent Heutte

Mathieu Salaün

Luc Thiberville

Classification of endomicroscopic images of the lung based on random subwindows and extra-trees

Keywords: Medical imaging, confocal fluorescence microscopy, image classification, extra-trees, reject rule

Recently, the in-vivo imaging of pulmonary alveoli was made possible thanks to confocal microscopy. For these images, we wish to aid the clinician by developing a computeraided diagnosis system, able to discriminate between healthy and pathological subjects. The lack of expertise currently available on these images has first led us to choose a generic approach, based on pixel-value description of randomly extracted subwindows and decision tree ensemble for classification (extra-trees). In order to deal with the great complexity of our images, we adapt this method by introducing a texture-based description of the subwindows, based on Local Binary Patterns (LBP). We show through our experimental protocol that this adaptation is a promising way to classify FCFM images. In addition, we introduce a rejection mechanism on the classifier output to prevent non detection errors.

I. INTRODUCTION

Whereas the conduction airways can be explored during bronchoscopy, the alveolar region is currently investigated only in vitro, using invasive techniques such as open lung biopsies. A new endoscopic technique, called Fibered Confocal Fluorescence Microscopy1 (FCFM), has recently been developed that enables the real-time visualisation of the alveoli in vivo [START_REF] Thiberville | In vivo imaging of the bronchial wall microstructure using fibered confocal fluorescence microscopy[END_REF]. The technique is based on the principle of fluorescence confocal microscopy, where the microscope objective is replaced by a fiberoptic miniprobe, made of thousands of fiber cores. The miniprobe can be introduced into the 2 mm working channel of a flexible bronchoscope to produce in vivo endomicroscopic imaging of the human respiratory tract. This promising technique, whose applications for lung exploration are currently under investigation, could replace lung biopsy in the future and might prove to be helpful in a large variety of diseases, including interstitial lung diseases [START_REF] Thiberville | Human in vivo fluorescence microimaging of the alveolar ducts and sacs during bronchoscopy[END_REF].

The system produces 420x420 pixel images (Fig. 1 and2), with a circular 600 µm diameter field of view, an approximate spatial resolution of 1µm per pixel and a depth of focus of the probe of 0-50 µm. In smokers, FCFM images are quite different notably because of the presence of macrophages (cells which digest cellular debris inside the alveoli), which are made visible because of tobacco-tar induced fluorescence (they appear as white spots 2). When images are acquired on C. Désir, C. Petitjean, L. Heutte, M. Salaün and L. Thiberville are with the LITIS EA 4108, Université de Rouen, 76801 Saint-Etienne-du-Rouvray, France. M. Salaün and L. Thiberville are also with the Clinique Pneumologique, Rouen University Hospital, 76031 Rouen, France.

e-mail: caroline.petitjean@univ-rouen.fr healthy subjects, the alveolar structure appears as a network of (almost) continuous lines, but it can be altered by distal lung pathologies. As shown in Fig. 1 and2, FCFM images exhibit some complexity; there is a large intraclass and relatively small interclass variability in the images. The possible confusion originates mostly from the elastin fiber framework which is not always clearly visible and from the noise present in the images.

Interpreting the lung FCFM images represents several levels of difficulties for the clinician, the first being to be able to differentiate normal images from real pathological images. In order to solve this problem, we have recorded in-vivo FCFM images from non-smoking and smoking healthy volunteers, that are considered by definition as "normal". We also recorded images from patients suffering from several diagnosed diffuse interstitial -or parenchymal -lung diseases, obtained from a lung segment that appeared abnormal on the chest CT Scan. Alveolar images from this latter group have been selected by the clinicians for being visually different from the normal images and considered abnormal. Because normality criteria based on the confocal image by itself are currently not known, we chose an objective definition for normality as "any alveolar confocal image taken from an healthy subject". Our aim is to provide the clinician with a computer aided-diagnosis (CAD) tool, so as to help him to analyze these new images. For instance, such a strategy may help to the automated recognition of the area to biopsy during endoscopy.

We describe in this paper such a CAD tool, able to classify FCFM images as healthy or pathological. A state-of-the-art efficient classification technique called extra-trees [START_REF] Geurts | Extremely randomized trees[END_REF], which has proved its efficiency on various image classification tasks [START_REF] Marée | Biomedical image classification with random subwindows and decision trees[END_REF], is implemented. This classification method is combined with a feature extraction technique based on the random extraction of subwindows from the image . The extraction of subwindows artificially increases the training sample size and allows to take into account more local characteristics. In order to capture the complex nature of FCFM images, we introduce high-level descriptors to describe the subwindows. The aim is to benefit from the association of the informative nature of texture description and the high precision and computational efficiency of the extra-trees.

A brief visual analysis of FCFM images reveals that in pathological cases, the disorganization of the alveolar meshing is characterized by the numerous irregularities and the tangle of the fibered structures, in all possible directions. The distributions of local texture patterns in healthy and pathological images are thus expected to be different and could be adequately described by rotation and scale invariant texture descriptors, such as the Local Binary Patterns (LBP) operator. Their ability to characterize the spatial configuration of local image texture and to recognize certain local binary texture patterns will help to apprehend the complex, textured nature of FCFM images. This new approach to characterize the images, that combines texture-based description with the random extraction of subwindows, is the first part of our contribution. It is shown to have encouraging results on FCFM images. The second point of our contribution concerns the introduction of a rejection mechanism on the classifier output that allows to improve the practical use of the system. We indeed show how such a mechanism can help reducing the non detection rate of the system, thus increasing its reliability.

The remainder of the paper is organized as follows: we review in Section II related works in the field of image classification and introduce our approach. The main steps of the FCFM image classification system are described in Section III. We present in Section IV the experimental protocol and the results. Finally we conclude in Section V by drawing some future works on this challenging task of classification.

II. RELATED WORKS IN IMAGE CLASSIFICATION

A classification system usually includes a feature extraction step and a classification step.

A. Feature extraction

The aim of the feature extraction step is to feed the classifier with relevant features that must be chosen to maximize interclass variance and minimize intraclass variance. The choice of a suitable feature space is often problem-dependent and features are therefore usually adapted to the image content. For instance, blood cell images can be described by frequency analysis, using Fourier-Mellin transform [START_REF] Dahmen | Automatic classification of red blood cells using gaussian mixture densities[END_REF]; protein subcellular images can be described using Haralick statistical parameters on cooccurrence matrices, Zernike moment features and features derived from morphological image processing [START_REF] Murphy | Robust numerical features for description and classification of subcellular location patterns in fluorescence microscope images[END_REF]. An alternative approach in feature extraction consists in using unspecific features such as raw pixel values or generic texture descriptors, such as the LBP operator [START_REF] Ojala | Gray scale and rotation invariant texture classification with local binary patterns[END_REF] for example, which is particularly suitable for image description and has been widely used for different image classification tasks [START_REF] Heikkilä | Description of interest regions with local binary patterns[END_REF]. In order to locally characterize the image, subwindows can also be extracted and described with a feature vector. The sampling of subwindows may be either deterministic or random, this latter solution having proved its efficiency for image characterization in various image classification tasks, when a lot of subwindows are sampled [START_REF] Nowak | Sampling strategies for bag-offeatures image classification[END_REF]. Random selection also allows to bypass costly steps such as keypoint detection or exhaustive window sliding on all pixels. Such a sampling method is very efficient computationally and remains generic since it does not require to design a dedicated keypoint detection for a specific application.

B. Classification

In the past decades, extensive work has been done in the machine learning and pattern recognition communities to design efficient classifiers. Recently, ensemble methods such as bagging [START_REF] Breiman | Bagging predictors[END_REF] or boosting [START_REF] Freund | Experiments with a new boosting algorithm[END_REF] have been of main interest especially in medical imaging [START_REF] Zhou | Lung cancer cell identification based on artificial neural network ensembles[END_REF]. Among these methods, ensembles of decision trees, such as random forests [START_REF] Breiman | Random forests[END_REF], have proved their efficiency over single classifiers on various classification tasks [START_REF] Dietterich | Ensemble methods in machine learning[END_REF].

Random Forests (RF) are a family of classifier ensemble methods that use randomization to produce a diverse pool of individual classifiers. It can be defined as a generic principle of classifier combination that uses L tree-structured base classifiers {h(x, Θ i ),i =1, ...L} where h is a tree-based classifier, Θ i a family of i.i.d random vectors, and x an input feature vector. The particularity of this kind of ensemble is that each decision tree is built from a random vector of parameters. RF have been shown to be particularly competitive with SVM and boosting, one of the most efficient learning principles [START_REF] Breiman | Random forests[END_REF], [START_REF] Freund | Experiments with a new boosting algorithm[END_REF]. Other randomization mechanisms may be exploited in RF: bagging (bootstrap-aggregating) for training each individual tree in the ensemble on a different replica of the training set (a bootstrap), obtained by subsampling the training set at random with replacement; and random feature selection, that selects a random subspace at each node of the tree. The Extremely Randomized Decision Trees, or extra-trees [START_REF] Geurts | Extremely randomized trees[END_REF], are based on another randomization process. At each node of an extra-tree, splitting rules are randomly drawn and the rule that performs best according to a score computation is then chosen to be associated to that node. This allows to drastically reduce the complexity of the induction process, increase the speed of training, and weaken the correlation between the induced decision trees. The whole available learning set is provided to each individual decision tree, i.e. there is no bagging.

C. Proposed approach

We have chosen the extra-trees due to their interesting statistical and computational properties. Considering the biasvariance tradeoff, extra-trees are expected to better reduce the variance than classical randomized methods such as the Forest-RI approach [START_REF] Breiman | Random forests[END_REF] as the randomization process is stronger and to minimize the bias as it uses the whole available learning set for the induction of each tree, contrary to classical approaches such as bagging [START_REF] Breiman | Random forests[END_REF]. The computational load of the algorithm is also reduced due to the simplicity of the node splitting procedure as it does not search for a local optimal cut-point.

The extra-trees approach, in combination with an original feature extraction technique, has proved its efficiency on various image classification tasks and compares well with state of the art classification systems [START_REF] Marée | Biomedical image classification with random subwindows and decision trees[END_REF]. In these works, the authors propose to extract a huge number of subwindows from the images of the learning set at random locations and with random sizes, and to label each of them with the class label of the original image. This new learning set, i.e. the set of subwindows, is used to train the extra-trees. The classification of a test image follows the same principle: subwindows are extracted from the image and a majority vote on the decisions assigned to the subwindows is used to decide for the class of the image. We propose to use this generic framework to classify FCFM images with fast extra-trees. The approach in [START_REF] Marée | Biomedical image classification with random subwindows and decision trees[END_REF] used raw pixel description of the subwindows. In our study, we first test this low-level based description for FCFM images but we suggest that a higher level descriptor, namely LBP, can be efficiently used to describe subwindows without loss of genericity. Additionally, we show that the classification of FCFM images can be made more reliable by taking benefit from the posterior probability assigned to an image. Indeed, by introducing a threshold on posterior probabilities, we give our system the ability to detect (i.e. reject) images that are likely to be false negative and thus to classify FCFM images with no error.

III. OUR CLASSIFICATION APPROACH

Our classification system is composed of three main steps (Fig. 3):

1) random extraction of N w subwindows of different sizes and locations from the image; 2) characterization of each subwindow by means of a feature vector that is classified using extra-trees;

3) combination of probabilities output by extra-trees on each extracted subwindow through a majority vote to either assign a class to the input image, or to reject it.

A. Feature extraction in FCFM images

In the original work presented in [START_REF] Marée | Biomedical image classification with random subwindows and decision trees[END_REF], the authors have proposed a low-level description of the subwindows, claiming that it makes their approach more generic. Indeed, the subwindows extracted from the images are chosen at random locations with random sizes; then they are resized with bilinear interpolation to the empirical resolution 16x16 pixels, in order to obtain a fixed size feature vector. Downsampling to 16x16 implies a loss of details, which is thought to be compensated for with a large number (N w = 100) of subwindows extracted in the image. This genericity has been of particular interest for our application since no a priori knowledge was available on FCFM images. However, we suggest to go further in the description of the subwindows by investigating the use of a classical texture descriptor, namely the LBP operator, in order to feed the system with more relevant features. The LBP [START_REF] Ojala | Gray scale and rotation invariant texture classification with local binary patterns[END_REF] consists of computing the distribution of binary patterns. These patterns are obtained by thresholding the neighbouring pixels compared to the central pixel, for each pixel of the subwindow, and coding them with powers of 2. The circular neighbourhood can be characterized with a number P of pixels and a radius R. Each pixel c is assigned the following number:

b(c)= P -1 p=0 s(g p -g c ).2 p
where g c is the gray value of the center pixel, g p defines the gray value of neighbor p and s(x) is the thresholding function defined as:

s(x)= 1 if x ≥ 0 0 otherwise.
The final characterization is the distribution of b values over the whole subwindow. Combining the responses of LBP operators computed with different values for P and R can also be of interest. This multiresolution approach has been chosen for our study, using standard values (P =8, R =1) and (P =16, R =2 ). Since the LBP operator yields P +2 values [START_REF] Ojala | Gray scale and rotation invariant texture classification with local binary patterns[END_REF], each subwindow is characterized by a 28-dimensional feature vector.

B. Classification based on extra-trees

In the classification step we use an ensemble of Extremely Randomized Trees [START_REF] Geurts | Extremely randomized trees[END_REF] induced on the generated learning set of subwindows. Each tree is induced by performing at each node the following steps: (i) randomly pick K different features from the whole feature vector, (ii) compute one random split per feature and (iii) select the one that maximizes a score measure based on a normalized version of the Shannon information gain [START_REF] Wehenkel | Automatic learning techniques in power systems[END_REF]. K acts as a parameter that enables to control the degree of randomization for inducting the tree: choosing a high value for K offers the opportunity to filter irrelevant variables and thus makes the tree classification process more reliable while introducing more correlation between trees in the forest; on the contrary, a low value for K will provide more randomized trees and thus decrease the correlation within the forest. Yet the classification process will take benefit from the ensemble of trees.

Performance of extra-trees also depend on the adjustement of two parameters, L, the number of trees in the ensemble, and n min , the minimum sample size for splitting a node. n min controls the strength of averaging output noise; L, the number of trees in the ensemble, determines the strength of the variance reduction of the forest. As usual, these parameters can be adapted to the problem specificities either manually or in an automatic way (e.g. by cross-validation).

However, it has been shown in [START_REF] Geurts | Extremely randomized trees[END_REF] that the default values of the two main parameters, i.e. K and n min , are near-optimal for a broad range of typical classification problems. The choice of these default values is discussed in section IV-B.

C. Combination of output probabilities and proposed rejection mechanism

The probabilities output by extra-trees on each extracted subwindow are combined through a majority vote to assign, to the input image to classify, a class, "healthy" (denoted ω H ) or "pathological" (denoted ω P ): let x i be the feature vector describing a subwindow, l H i the number of trees assigning the class ω H to x i ; the probability of the subwindow to be assigned the "healthy" class is:

p(ω H |x i )= l H i L
where L is the number of trees in the forest. The posterior probability of ω H assigned to the image I is then obtained by averaging over all N w subwindows:

p(ω H |I)= 1 N w Nw i=1 p(ω H |x i )
where N w is the number of subwindows extracted from the image. The same reasoning applies to the computation of probability of the pathological class.

The fraction of trees that voted for a particular class can be used to handle the reliability of the extra-trees decision. Indeed, reliability of automatic pattern recognition systems is a critical issue in many practical applications. A rejection threshold is commonly used to reach a trade-off between error and reject probabilities. From the practical point of view, it offers the possibility of refusing to assign the examined pattern to any class, possibly prompting for a further investigation by another system or by a human supervisor [START_REF] Foggia | On rejecting unreliably classified patterns[END_REF]. In our medical context, providing our FCFM image classification system with a reject option would enable to filter the doubtful (low confidence) images that could further be handled either by the clinician who will make his/her own diagnosis or by an additional, more accurate and usually more costly, classification system. The combination of probabilities output by the extra-trees offers a means to introduce a rejection mechanism.

Indeed, the probabilities of each class (healthy or pathological) enable to measure the level of confidence associated to the classification. For instance, a test image that has been classified by the extra-trees with a probability of 0.9 means that a great number of subwindows have been assigned a high probability of being labeled healthy or pathological: such a high level of agreement between subwindows, knowing their random and independent process of extraction, suggests that the class has been assigned to the image with a high level of confidence. On the contrary, a probability near 0.5 should be considered as a low level of confidence decision and suggests that the decision associated to the image should be rejected in order to avoid a non detection or false alarm error. Consequently, thresholding the probability assigned to an image should enable to minimize the non detection error rate of the system. Note that the proposed rejection mechanism could also be controlled another way. For example, instead of determining the posterior probability of the image by averaging the probabilities assigned to the extracted subwindows, one could filter first subwindows whose predictions are "uncertain", i.e. remove from the final vote subwindows whose probability is under a given threshold, in order to keep only strong decisions, which boils down to voting on highly confident subwindows [START_REF] Marée | A generic approach for image classification based on decision tree ensembles and local subwindows[END_REF].

IV. EXPERIMENTS

A. Image databases and subwindow extraction

In-vivo FCFM images from non-smoking and smoking healthy volunteers, that are considered by definition as "normal", have been recorded. We also recorded images from patients suffering from several diagnosed diffuse intersitialor parenchymal -lung diseases, obtained from a lung segment that appeared abnormal on the chest CT Scan. Due to the great difference between images acquired on smoker and non-smoker subjects, our experiments have been conducted separately on those two groups. The non-smoker (NS) image database and the smoker (S) image database are composed of respectively 133 and 93 images, assigned to two different classes: "healthy" (31 for NS and 60 for S) and "pathological" (102 for NS and 33 for S). In order to assess the generalization error of the system, a 10-fold cross validation process is used. The principle is to partition the complete image dataset into 10 complementary subsets. The training is performed on 9 subsets and testing is done on the remaining subset to assess classification performance. To reduce variability, 10 rounds of cross-validation are performed using the 10 different partitions, and the classification results are averaged over the rounds. In order to deal with the imbalanced datasets, an identical number of subwindows (namely 80000, i.e. 40000 per class) is extracted from the images present in the 9 folds, during the learning phase [START_REF] Marée | Biomedical image classification with random subwindows and decision trees[END_REF]. The 80000 subwindow extraction process is thus renewed for each different training partition. During the test phase, N w = 100 subwindows are randomly extracted from each test image of the remaining set.

B. Extra-trees parameterization

As mentioned previously, performance of extra-trees depend on the adjustement of three parameters, namely K, n min and L. The tree depth is implicitly fixed by n min , which is the minimum number of samples required for splitting a node: the smaller the value of n min , the deeper the tree. In our experiments, we have chosen n min =2(as for normal tree classifier) [START_REF] Geurts | Extremely randomized trees[END_REF]. Regarding parameter L (number of trees in the forest), it is well-known that for randomized ensembles the classification error is a monotically decreasing function of L: the higher the value of L, the better the accuracy. Nevertheless, choosing an appropriate value for L is not so critical: given the computational efficiency of the tree growing algorithm, a high value for L may be chosen. In this study, an ensemble of 30 trees has been found to be large enough and that convergence is reached well before L =30(typically between 15 and 20 trees could be enough). Parameter K, the number of variables randomly selected at each node during the tree growing, determines the strength of the variable selection process and is usually set to √ M where M is the dimension of the feature vector [START_REF] Breiman | Random forests[END_REF]. This default setting is the best compromise for a wide range of classification problems [START_REF] Bernard | Influence of hyperparameters on random forest accuracy[END_REF], [START_REF]Forest-RK: A new random forest induction method[END_REF], [START_REF] Geurts | Extremely randomized trees[END_REF]. Let us recall that M = 256 for the raw pixel feature space since each subwindow has been resized to 16x16 pixels whereas M =28for the LBP operator. Note that the resizing operation of the subwindow is performed only for the raw pixel space.

C. Experimental results

Raw pixel vs. LBP characterization:

Results obtained on the FCFM image database for the previously defined feature spaces (raw pixel values and LBP) are compared. Mean classification rates and standard deviations obtained over the 10 folds are provided in Table I. For the FCFM image database, we can observe in Table I that the classification rates obtained with raw pixel values are rather low, underlining the difficulty for the classifier to cope with such a low level description of FCFM images. Though it has been shown that this classification approach is efficient [START_REF] Marée | Biomedical image classification with random subwindows and decision trees[END_REF], describing the subwindows with raw pixel values does not seem to be adequate for our application. The image content requires a higher level of description as confirmed by the higher classification rates obtained with LBP descriptors. LBP descriptors perform better on smoker images than on non-smoker images, whereas raw pixels perform worse, as can be noticed also on ROC curves2 (Fig. 5). This phenomenon could be related to the presence of several additional corners in smoker images, because of the macrophages: they are clearly visible on healthy images and somehow hidden when the alveolar structure is disorganized because of a pathology.

Random-sized subwindows vs. constrained and small-sized subwindows: One advantage of extracting random subwindows is that no additional hyperparameter value (such as subwindow size) has to be set. The other advantage is that random subwindows are able to catch both large and small patterns in the image and that their combination brings additional information to discriminate between classes. To gain more insight on this issue, we have investigated the effect of constraining vs. not constraining the size of subwindows. Yet it is well-known that small local patches can give good results for texture classification tasks [START_REF] Varma | A statistical approach to material classification using image patch exemplars[END_REF]. We have thus compared the extraction of constrained and small-sized subwindows vs. unconstrained, random-sized subwindows, using the same protocol as described in Section IV-A, with an LBP description of each subwindow. Table II shows that the extraction of random subwindows provides the best results, compared to small-size subwindows. This can be explained by the fact that the fibered structure of FCFM images is randomly tangled and differs from textured images, that exhibit periodical patterns. The periodicity property is important to explain why small patches contain sufficient information for successful discrimination [START_REF] Varma | A statistical approach to material classification using image patch exemplars[END_REF]. Unconstrained random-sized subwindows thus seem to be the best solution for FCFM images.

Reduction of non detection with a rejection mechanism: For assessing the ability of our method to minimize the non detection rate, we show in Fig. 6 the distribution of the confidence values associated to the decision of the forest, i.e. when the class with highest posterior probability is retained (no rejection rule is applied). These distributions are drawn for the two groups, smoker and non smoker, on the test images over the 10 folds.

As shown in Fig. 6, the posterior probabilities assigned by the classifier to the correctly classified (either pathological or healthy) test images vary from 0.51 up to 0.98 for the non smoker group and from 0.53 up to 0.99 for the smoker group (red circles). More interestingly, without rejection rule, the posterior probabilities assigned by the classifier to the incorrectly classified test images (either pathological or healthy) range from 0.53 up to 0.89 for the non smoker group and from 0.52 up to 0.61 for the smoker group (blue crosses). This means that if a reject threshold on posterior probabilities (e.g. 0.89 for the non smoker group, 0.61 for the smoker group) is properly adjusted, accepting only decisions above this threshold will make the decision process more reliable, as errors will be eliminated. False alarm and non detection rates will then be equal to zero. The rejection threshold will thus be useful to filter doubtful images that could further be handled either by the clinician who will therefore focus only on these images, or by an additional but more costly classification system. This additional handling of doubtful images would allow to make a decision on, in particular, rejected pathological images that would not have been detected otherwise. An alternative to an automatic rejection process of low confidence images could also consist in providing the clinician with the system decision, i.e. the probability associated to the healthy or pathological class, along with the rejection threshold, so as to warn him of a possible error risk.

From Table III, one can see that the application of such a rejection rule with our approach (RS+ET) on the non smoker group turns into rejecting 36.1% of the test images (i.e. 48/133) leading to a correct classification rate of 63.9% (i.e. 85/133) with no error. Of course, this null error rate can only be achieved at the expense of rejecting some test images that would have been correctly classified otherwise (i.e. 38/48) but this is the price worth paying for eliminating non detection errors. Similarly, for the smoker group, applying such a rejection rule on the posterior probabilities enables to classify with no error 94.6% of the test images (i.e. 88/93) while rejecting 5.4% (i.e. 5/93). A null detection rate for the smoker group can thus be achieved at the expense of rejecting 2 correctly classified test images. All in all, introducing a rejection mechanism enables to correctly classify 76.6% of the test images of the two groups with no error (neither false alarms nor non detections) whereas accepting, without reject, all decisions of the automatic system would lead to a 5.9% error rate.

Random subwindows vs. global image descriptor: In order to assess the interest of introducing a random subwindow description of the image, we compared our random subwindow based approach (RS+ET) to the image-based approach (I+ET) that relies on an LBP-based description of the image, without random subwindow extraction. Results with the I+ET approach have been obtained using a leave-one-out protocol and are reported in Table III. As classification rates of RS+ET and I+ET are close, it is not possible to conclude on the superiority of the RS+ET approach. Nonetheless, the RSbased approach is more efficient for rejection: less images have to be rejected to reach a null error rate, compared to the I+ET approach. This advantage of the RS-based approach might originate from the redundancy (by extracting numerous overlapping subwindows) and diversity (by means of random locations and sizes) induced by the subwindow extraction process. These properties lead to better estimated posterior probabilities, that enable to better control the rejection of doubtful images.

It is worth noticing that the generalization of the rejection threshold on other datasets of FCFM images holds of course if and only if enough training data are used to estimate an accurate threshold and if the data, on which the rejection threshold will be applied in practical use, share the same statistical properties as the training data. At present, we do not have at our disposal enough data to make the evaluation of the rejection threshold more robust but we have shown that our method offers a means to go one step further in making the system more reliable by minimizing non detection errors. Gathering more labeled images is one of our future works. V. C ONCLUSION In this paper, we have presented a classification system, based on the analysis of FCFM images representing the alveolar structure in vivo. This system allows to distinguish healthy cases from pathological ones for computer-aided diagnosis. The proposed classification approach is based on a local characterization of the images and an extra-tree classification system. One major advantage of the approach is that no critical parameter has to be tuned as default settings for extra-trees and unconstrained sizes for extracted subwindows are used, thus maximising the genericity and autonomy of the method. The methodological contribution relies on (i) the extraction of higher level descriptors on random subwindows to describe FCFM images and (ii) a rejection mechanism based on the thresholding of the probabilities output by the classifier, in order to equate non detection to zero. We showed that rawpixel values description does not allow to account for the complexity of the images, and that the introduction of a more relevant texture descriptor, namely the LBP, allowed to achieve higher classification rates. A study on the level of confidence associated to the decision of the forest has also been conducted that showed that our system is rather accurate for classifying FCFM images and highly reliable as it is possible to handle a reject rule based on probability thresholding to reach a null non-detection rate.

Future works include the validation of these results on larger databases of FCFM images, in order to make the rejection process more robust for practical computer-aided diagnosis.

The development of a multi-class classification system able to discriminate different kinds of pathologies (fibrosis, sclerosis, sarcoidosis, abestosis) would also be of interest for the clinician. In terms of applications, we plan to evaluate our method on bigger and different databases of images.

Fig. 1 .

 1 Fig. 1. FCFM images of non-smoking (a) healthy and (b) pathological subjects, showing the alveolar structure. Note how the meshing is tangled and altered for pathological cases (b).

Fig. 2 .

 2 Fig. 2. FCFM images of smoking (a) healthy and (b) pathological subjects, showing the alveolar structure. White spots are macrophages, visible here due to tobacco-tar induced fluorescence. Note how the meshing is tangled and altered for pathological cases (b).

Fig. 4 .

 4 Fig. 4. Subwindows extracted from an FCFM image and their associated distributions of LBP patterns.

Fig. 3 .

 3 Fig. 3. Our classification approach based on random subwindows extraction and extra-trees. Nw is the number of extracted subwindows from image I, F the forest of L extra-trees and ω H the Healthy class.

Fig. 5 .

 5 Fig. 5. ROC curves with random subwindows + LBP + ET

TABLE II COMPARISON

 II IN CLASSIFICATION RATES (MEAN ± STDEV) BETWEEN SMALL SUBWINDOWS AND RANDOM SUBWINDOWS EXTRACTION.

		Non-smoker	Smoker
	4x4 subwindows	76.1±12.4%	90.0±9.7%
	8x8 subwindows	78.4±13.4%	92.2±9.1%
	16x16 subwindows	82.3±10.2%	94.4±5.8%
	Random subwindows	92.3 ± 8.8%	96.6 ± 5.4%

  Distribution of the probabilities associated to correctly and incorrectly classified images. "label" denotes the ground truth associated to each image.

		Correctly classified image I				Reject threshold
									(NDR = 0)
	0.5	0.55	0.6	0.65	0.7	0.75	0.8	0.85	0.9	0.95	1
										p(ω = label | I i )
	0.5	0.55	Reject threshold 0.6 0.65	0.7 (a) Nonタsmoker group 0.75 0.8	0.85	0.9	0.95 p(ω ≠ label | I i )	1
			(NDR = 0)
	0.5	0.55	0.6	0.65	0.7	0.75	0.8	0.85	0.9	0.95	1
										p(ω = label | I i )
	0.5	0.55	0.6	0.65	0.7	0.75 (b) Smoker group 0.8	0.85	0.9	0.95 p(ω ≠ label | I i ) 1

i Incorrectly classified image I i Fig. 6.

TABLE III MEAN

 III CLASSIFICATION RATES OF THE RANDOM SUBWINDOW BASED APPROACH (RS+ET) VS. THE IMAGE-BASED APPROACH (I+ET)

				Correct	Error	Reject
	NS	RS+ET I+ET	without reject with reject without reject with reject	92.3% 63.9% 92.1% 0%	7.7% 0% 7.9% 0%	0% 36.1% 0% 100%
	S	RS+ET I+ET	without reject with reject without reject with reject	96.6% 94.6% 95.7% 88.8%	3.4% 0% 4.3% 0%	0% 5.4% 0% 11.2%

FCFM is also refered to as probe-based Confocal Laser Endomicroscopy (pCLE), especially in gastrointestinal imaging.

A single ROC curve can be computed with a 10-fold protocol by merging all test folds into one large single test set, and sorting the instances by their assigned score[START_REF] Flach | Tutorial on the many faces of ROC analysis in machine learning[END_REF]. A ROC curve can then be plotted using a generation algorithm such as[START_REF] Fawcett | An introduction to ROC analysis[END_REF].

ACKNOWLEDGEMENTS

The authors would like to thank the Ligue Contre le Cancer for supporting Chesner Désir. They also would like to thank the anonymous reviewers whose comments helped to improve the manuscript.


Emmanuel Vincent
email: emmanuel.vincent@inria.fr

Jon Barker
email: j.barker@dcs.shef.ac.uk

Shinji Watanabe

Jonathan Le Roux

Francesco Nesta

Marco Matassoni

THE SECOND 'CHIME' SPEECH SEPARATION AND RECOGNITION CHALLENGE: DATASETS, TASKS AND BASELINES

Keywords:

Distant-microphone automatic speech recognition (ASR) remains a challenging goal in everyday environments involving multiple background sources and reverberation. This paper is intended to be a reference on the 2nd 'CHiME' Challenge, an initiative designed to analyze and evaluate the performance of ASR systems in a real-world domestic environment. Two separate tracks have been proposed: a small-vocabulary task with small speaker movements and a medium-vocabulary task without speaker movements. We discuss the rationale for the challenge and provide a detailed description of the datasets, tasks and baseline performance results for each track.

INTRODUCTION

Despite tremendous progress in close-microphone ASR for broadcast news, telephone speech or meeting speech, robust distant-microphone ASR in everyday environments remains a challenging goal. In parallel to research in the Speech and Language (SL) community, new techniques have emerged in the Audio and Acoustic Signal Processing (AASP) and Machine Learning for Signal Processing (MLSP) communities which are currently changing the face of robust ASR [START_REF]Blind speech separation[END_REF][START_REF] Wölfel | Distant Speech Recognition[END_REF][START_REF]Robust Speech Recognition of Uncertain or Missing Data -Theory and Applications[END_REF].

The 1st 'CHiME' Challenge [START_REF] Barker | The PASCAL CHiME Speech Separation and Recognition Challenge[END_REF] held in 2011 was the first concerted evaluation of ASR systems in a real-world domestic environment involving both reverberation and highly dynamic background noise made up of multiple sound sources. It differentiated itself from past noise-robust ASR challenges [START_REF] Pearce | The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions[END_REF][START_REF] Mcdonough | To separate speech!: A system for recognizing simultaneous speech[END_REF][START_REF] Himawan | Microphone array beamforming approach to blind speech separation[END_REF][START_REF] Cooke | Monaural Speech Separation and Recognition Challenge[END_REF] by considering more realistic noise conditions and from concurrent source separation challenges [START_REF] Vincent | The Signal Separation Evaluation Campaign (2007-2010): Achievements and remaining challenges[END_REF] by assessing the results in terms of ASR. Thirteen systems were submitted [START_REF] Delcroix | Speech recognition in the presence of highly nonstationary noise based on spatial, spectral and temporal speech/noise modeling combined with dynamic variance adaptation[END_REF][START_REF] Gemmeke | Exemplar-based speech enhancement and its application to noise-robust automatic speech recognition[END_REF][START_REF] Hurmalainen | Exemplar-based recognition of speech in highly variable noise[END_REF][START_REF] Kallasjoki | Mask estimation and sparse imputation for missing data speech recognition in multisource reverberant environments[END_REF][START_REF] Kim | Zero-crossingbased channel attentive weighting of cepstral features for robust speech recognition: The ETRI 2011 CHiME challenge system[END_REF][START_REF] Koldovský | CHiME data separation based on target signal cancellation and noise masking[END_REF][START_REF] Kolossa | CHiME Challenge: Approaches to robustness using beamforming and uncertainty-of-observation techniques[END_REF][START_REF] Ma | Recent advances in fragment-based speech recognition in reverberant multisource environments[END_REF][START_REF] Maas | A two-channel acoustic front-end for robust automatic speech recognition in noisy and reverberant environments[END_REF][START_REF] Nesta | Robust automatic speech recognition through on-line semi blind source extraction[END_REF][START_REF] Ozerov | Using the FASST source separation toolbox for noise robust speech recognition[END_REF][START_REF] Vipperla | Robust speech recognition in multi-source noise environments using convolutive non-negative matrix factorization[END_REF][START_REF] Weninger | The Munich 2011 CHiME challenge contribution: NMF-BLSTM speech enhancement and recognition for reverberated multisource environments[END_REF] which cover a wide range of signal enhancement and robust acoustic modeling techniques. The absolute keyword accuracy achieved by the best system was only 3% below that of a human listener. Further analysis showed that multicondition training and spatial enhancement are the most effective single strategies but that the resulting performance im-* Now at Conexant Systems, Newport Beach, CA, USA.

provements are not additive and that careful combination of these and other strategies is needed for further improvement.

In order to maximize scientific insight, a number of simplifications were brought to the task so as to keep it tractable and ensure a diversity of submissions. A key decision was to focus on the realism of the noise background while employing an unrealistically simple target speech signal. Surveyed for their opinion, the challenge entrants highlighted three main additional dimensions of difficulty to be considered in future challenges: variability of speaker location, vocabulary size and speech naturalness. Indeed, ASR systems can be surprisingly sensitive to speaker location and it is well known that systems optimized for small vocabulary read speech often fail to scale to larger vocabulary spontaneous speech.

This paper is intended as a reference on the ongoing 2nd 'CHiME' Challenge supported by the IEEE AASP, MLSP and SL Technical Committees. We extend the difficulty of the 1st Challenge in the first two dimensions above, such that the target speech conditions become closer to those in [START_REF] Mcdonough | To separate speech!: A system for recognizing simultaneous speech[END_REF][START_REF] Himawan | Microphone array beamforming approach to blind speech separation[END_REF] but a realistic multisource noise background is retained as opposed to a single interfering speaker. In order to avoid too large an increase in difficulty, two separate tracks have been proposed: a small vocabulary task with small speaker movements and a medium vocabulary task without speaker movements.

The structure of the rest of the paper is as follows. In Section 2, we detail the creation of the datasets and define the tasks to be addressed. In Section 3, we describe the baseline recognizers provided together with the data and report their performance when trained either from clean, reverberated or noisy data. We conclude in Section 4.

DATASETS AND TASKS

The configuration considered by the 2nd 'CHiME' Challenge is that of speech from a single target speaker being binaurally recorded in a domestic environment. Three datasets are provided for each task: a training set, a development set and a test set. Following [START_REF] Barker | The PASCAL CHiME Speech Separation and Recognition Challenge[END_REF][START_REF] Vincent | The Signal Separation Evaluation Campaign (2007-2010): Achievements and remaining challenges[END_REF], these data were generated by convolving clean speech signals with binaural room impulse responses (BRIRs) and mixing them with noise backgrounds.

Noise and BRIR recordings

The BRIRs and the noise backgrounds were recorded in the same domestic living room using two ear microphones built into a B&K head and torso simulator (HATS) placed at a fixed position [START_REF] Barker | The PASCAL CHiME Speech Separation and Recognition Challenge[END_REF]. About 14 h of noise backgrounds were collected in chunks of 0.5 to 1.5 h over a period of several days. These include the major sources of noise in a typical family home: concurrent speakers, TV, game console, footsteps, and distant noise from outside or from the kitchen. The BRIRs were measured via the usual sine sweep method for 121 different positions covering a horizontal square grid of 20 cm side centered on the position 2 m directly in front of the HATS, with a grid step of 2 cm. A fixed gain was applied to the estimated BRIRs so that the level after convolution approximately matched that of a human speaker at a natural conversational level.

Track 1: small vocabulary

As in the 1st Challenge, the small vocabulary task relies on the Grid speech corpus [START_REF] Cooke | An audio-visual corpus for speech perception and automatic speech recognition[END_REF]. The target utterances are 6-word sequences read by 34 speakers of the form <command:4> <color:4><prepos.:4><letter:25><digit:10><adverb:4>, where the numbers in brackets indicate the number of choices per word. The task is to recognize the letter and digit tokens. Success is measured by the keyword recognition rate, that is the percentage of correctly recognized tokens.

The temporal placement of the utterances within the noise background was controlled in order to produce mixtures at 6 different ranges of signal-to-noise ratio (SNR): -6, -3, 0, 3, 6 and 9 dB1 . This was achieved by randomly scanning the background recordings and picking a time interval in the desired SNR range for each utterance. In comparison to conventional robust ASR evaluations [START_REF] Pearce | The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions[END_REF] which operate by rescaling the speech and noise signals, this mixing procedure is ecologically more valid, although it does not yet account for the Lombard effect as in [START_REF] Mcdonough | To separate speech!: A system for recognizing simultaneous speech[END_REF][START_REF] Himawan | Microphone array beamforming approach to blind speech separation[END_REF]. The backgrounds at 9 dB are dominated by quasi-stationary ambient sources, while those at -6 dB typically involve nonstationary, sudden sound events.

In order to make the task more realistic, the clean utterances were convolved with time-varying BRIRs mimicking small head movements within the aforementioned horizontal square grid. The parameterization of the movements was kept simple in order to allow analysis of the results as a function of the movement amplitude and speed: the target speaker was assumed to be static at the beginning of each utterance, then to move once, and finally to be static again. The movements follow a straight left-right line at fixed front-back distance from the HATS and each movement covers a distance of at most 5 cm at a speed of at most 15 cm/s. These movements were implemented by interpolating the set of recorded BRIRs in the same way as the Roomsimove toolbox 2 : for each front-back distance, a finer left-right grid of 2.5 mm step was designed and, for each point of this grid, the corresponding BRIRs were estimated by linear interpolation of the BRIRs recorded on the coarser 2 cm grid; each time sample of the clean speech signal was then convolved with the BRIRs associated with the point of the finer grid that is closest to the source position at that instant. This operation is an approximation of the true timevarying BRIRs. In order to validate this approximation, we conducted a simulation using non-binaural RIRs for simplicity. We generated the RIRs at each point of the finer grid using the source image method [START_REF] Allen | Image method for efficiently simulating small-room acoustics[END_REF] assuming similar room geometry and reverberation time and we computed the worst-case modeling error achieved by linearly interpolating the filters over the whole grid, as measured in terms of SNR after convolution with a speech signal. We compared the results to the alternative interpolation procedure that consists of computing the FFT of the RIRs, unrolling phase according to the time delay of arrival, linearly interpolating phase and magnitude, and computing the inverse FFT in a way similar to [START_REF] Hacıhabiboglu | Headrelated transfer function filter interpolation by root displacement[END_REF]. Although phase interpolation yields perfect interpolation in the case of pure delay filters, the results in Fig. 1 show that linear interpolation performs better in the case of reverberant RIRs and that it achieves a worst-case SNR of 19 dB for a coarse grid step of 2 cm. Therefore, the modeling noise is at least 10 dB lower than the background noise, which is a reasonable approximation given that it is hardly feasible in practice to record RIRs on a finer grid.

We generated 600 noisy test utterances and 600 noisy development utterances at each of the 6 SNRs, where the same utterances are used for all SNRs and they do not overlap with each other within each dataset. We also provided a training set containing 500 utterances of each of the 34 Grid talkers in clean, reverberated and noisy conditions. All data were sampled at 16 kHz and are freely available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 license both as isolated utterances and as embedded utterances including 5 s or more background before and after.

2 http://www.irisa.fr/metiss/members/evincent/Roomsimove.zip

Track 2: medium vocabulary

The medium vocabulary task relies on the Wall Street Journal (WSJ0) 5k vocabulary read speech corpus [START_REF] Garofalo | CSR-I (WSJ0) Complete[END_REF] also used in [START_REF] Pearce | The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions[END_REF][START_REF] Mcdonough | To separate speech!: A system for recognizing simultaneous speech[END_REF][START_REF] Himawan | Microphone array beamforming approach to blind speech separation[END_REF]. The task is to recognize all words. Success is measured in terms of word error rate (WER), that is the number of word substitutions, insertions and deletions as a fraction of the number of target words.

The data were mixed in the same way as the 1st 'CHiME' Challenge, i.e., clean speech utterances were convolved with the BRIRs corresponding to the fixed position directly in front of the HATS. Different recording conditions were used for the training, development and test datasets with the door being open/closed and the curtains being drawn/undrawn. The temporal placement of the utterances was controlled in a similar way as above in order to produce mixtures at the same range of SNRs. Since WSJ0 features longer utterances, the SNR for a given utterance was defined as the median value of the segmental SNR computed over segments of 200 ms. Also, due to the increased amount of data, nonoverlapping temporal placement of the utterances in the development and test sets was no longer feasible hence limited signal rescaling and overlap were allowed when necessary.

The development set includes 409 noisy utterances from 10 speakers, forming the "no verbal punctuation" (NVP) part of the WSJ0 speaker-independent 5k vocabulary development set. The test set comprises 330 noisy utterances from 12 other speakers, forming the Nov92 ARPA WSJ evaluation set. The training set includes 7138 reverberated utterances from 83 speakers forming the WSJ0 SI-84 training set. Both the development and the test utterances are released at each of the 6 SNRs while a noisy training set is provided by mixing each utterance at one random SNR, uniformly distributed in the defined range. All the noisy utterances are provided both in isolated and in embedded form. All data were sampled at 16 kHz and are available under agreement with the Linguistic Data Consortium (LDC).

Instructions

A set of instructions was provided in order to keep the task as close to an application scenario as possible, avoid unvoluntary overfitting and allow systems to be broadly comparable. The systems are allowed to exploit knowledge of the temporal placement of the utterances, of the surrounding background, of the speaker identity (for task 1) or of the speaker movements (also for task 1). However, they cannot exploit the SNR labels in the test set, the fact that the same utterances are used at each SNR, the fact that the same noise backgrounds are used in the development and test sets, the fact that the same utterances are used within the clean, reverberated and noisy training sets 3 , the fact that the BRIRs are identical between different test utterances (for task 2) or the fact that the noise signals in the test utterances may temporally overlap (also for task 2). All parameters should be tuned on the provided training and development sets using the provided language models and the system should be run only once on the test set. Besides these rules, entrants are left entirely free in the development of their system, so as not to artificially disadvantage one research community over another.

BASELINES

For each of the two tracks, a baseline ASR system based on HTK [START_REF] Young | The HTK Book, version 3.4[END_REF] was made available so as to lower the entry bar for researchers outside the SL community and demonstrate the performance achievable with neither signal enhancement nor advanced robust acoustic modeling techniques. This system includes both training and decoding scripts. Trained acoustic models were provided for clean, reverberated and noisy data. An alternative baseline ASR system based on Kaldi was made available for Track 2 and is separately described in [START_REF] Tachioka | Effectiveness of discriminative training and feature transformation for reverberated and noisy speech[END_REF].

Acoustic features

The speech waveforms are parameterized into a sequence of standard 39-dimensional Mel-frequency cepstral coefficient (MFCC) vectors: 12 Mel-cepstral coefficients processed by cepstral mean normalization (CMN), plus logarithmic frame energy and delta and acceleration coefficients (MFCC E D A Z). The MFCCs are extracted from 25 ms time frames with a step size of 10 ms. Prior to feature extraction, the input binaural signals are downmixed to mono by averaging the two channels together. Note that, although this downmixing operation leads to a small degradation of WER (1.5% on average) for Track 2 compared to the front end in [START_REF] Vertanen | Baseline WSJ acoustic models for HTK and Sphinx: Training recipes and recognition experiments[END_REF], we decided to use it in order to allow comparison of the results across tracks and with the 1st 'CHiME' Challenge.

Small vocabulary recognizer

The baseline system for Track 1 is identical to that of the 1st 'CHiME' Challenge. Each of the 51 words in the Grid vocabulary is modeled with a left-to-right HMM with 2 states per phoneme. The emission probability for each state is represented as a Gaussian mixture model (GMM) with 7 components with diagonal covariance. The language model is fixed according to the Grid syntax.

Training proceeds in two stages: a speaker-independent model is first trained from a flat start using the full 17,000utterance training set with HCompV, HERest and HHEd; a speaker-dependent model is then derived for each of the 34 speakers by applying further Baum-Welch iterations on the 500 utterances belonging to that speaker using HERest. Exact Viterbi decoding is performed using HVite.

Training -6 dB -3 dB 0 dB 3 dB 6 dB 9 dB Clean 10.58 11.17

Medium vocabulary recognizer

The baseline system for Track 2 follows the recipe in [START_REF] Vertanen | Baseline WSJ acoustic models for HTK and Sphinx: Training recipes and recognition experiments[END_REF].

The number of phonemes is 41: 39 phones plus 1 silence (sil) and 1 short pause (sp) model. The output distributions of sp and sil have their parameters tied. The number of clustered triphone HMM states is 1860 and is relatively smaller than the conventional setup (more than 2000 states). Each HMM has three output states with a left-to-right topology with self-loops and no skip. Each HMM state is represented by a GMM with 8 components for phoneme-based HMMs and 16 for silencebased HMMs. The standard WSJ 5K non-verbalized closed bigram language model is considered. The provided training scripts only re-estimate the HMM-GMM parameters from a clean speech acoustic model, and do not change the model topology for simplicity. Decoding is performed using HVite with a pruning threshold.

Parameter tuning

We did not fine tune the features (0th MFCC vs. log-energy, other features than MFCCs), the acoustic model topology (triphone HMM clustering and number of GMM components) and the search parameters (language model weight, insertion penalty), as the optimal tuning is highly dependent on the enhancement technique used. Other LVCSR decoders could also be considered. Readers interested in providing suggestions or advice in this regard are welcome to contact us.

Baseline results

Tables 1 and2 report the performance of the baseline systems trained on clean, reverberated or noisy data as a function of the SNR. The best results are achieved by training on noisy data for all SNRs except the 9 dB SNR condition in Track 1.

These figures must be compared to a keyword accuracy of 97.25% and 95.58% and to a WER of 7.49% and 18.40% when decoding the clean or reverberated utterances underlying the test set using clean or reverberated acoustic models respectively. While reverberation alone increases the error rate by a factor of 1.6 to 2. by a factor of 2.3 to 11 depending on the SNR and it is therefore the main issue to be solved by the challenge entrants.

Comparison with the baseline results of the 1st 'CHiME' Challenge in Table 3 shows the impact of speaker movements and vocabulary size. Speaker movements decrease the keyword accuracy by 4% on average with clean training, but they increase it by 2% with reverberated training due to the averaging effect they induce on the spectral differences between training and test. Larger vocabulary size increases the error rate by a factor of 1.3 to 3.2 with reverberated training depending on the SNR. Of course, the impact of speaker movements and vocabulary size may be different on more advanced systems and this is what the Challenge will seek to determine.

CONCLUSION

The series of 'CHiME' Challenges pursues the endeavor of evaluating robust ASR systems in real-world environments involving multisource background noise. The 2nd edition has increased the difficulty along two axes: small speaker movements and vocabulary size. Precise instructions have been provided to allow comparison of systems and maximize scientific insight. The submitted systems and the results will be unveiled at the 2nd 'CHiME' Workshop.

ACKNOWLEDGMENT

We would like to thank the IEEE AASP, MLSP and SL Technical Committees for supporting this challenge, and Conexant Systems, Audience, Adobe Systems, Google and MERL for sponsoring the 2nd 'CHiME' Workshop.

Fig. 1 .

 1 Fig. 1. Comparison of linear vs. phase interpolation for the simulation of time-varying RIRs.

Table 1 .

 1 Baseline test set keyword accuracy (%) for Track 1.

		13.33 17.75 21.17 24.42
	Reverb	32.17 38.33 52.08 62.67 76.08 83.83
	Noisy	49.33 58.67 67.50 75.08 78.83 82.92
	Training -6 dB -3 dB 0 dB 3 dB 6 dB 9 dB
	Clean	93.56 93.05 91.67 89.05 84.79 79.21
	Reverb	87.97 83.19 78.05 71.87 65.23 55.91
	Noisy	70.43 63.09 58.42 51.06 45.32 41.73

Table 2 .

 2 Baseline test set WER (%) for Track 2.

Table 3 .

 3 Baseline test set keyword accuracy (%) for the 1st 'CHiME' Challenge.

	5, background noise further increases it

In order to better match the perceived SNR, the SNRs were computed from high-pass filtered versions of the signals with a cutoff frequency of 80 Hz. Each SNR must be understood as a distribution of values with a standard deviation on the order of ±1 dB.

Note that this forbids so-called "stereo data" approaches, which assume the availability of synchronised clean and noisy data.