Seong-Gyun Jeong

Yuliya Tarabalka

Nicolas Nisse

Josiane Zerubia

Progressive Tree-like Curvilinear Structure Reconstruction with Structured Ranking Learning and Graph Algorithm

We propose a novel tree-like curvilinear structure reconstruction algorithm based on supervised learning and graph theory. In this work we analyze image patches to obtain the local major orientations and the rankings that correspond to the curvilinear structure. To extract local curvilinear features, we compute oriented gradient information using steerable filters. We then employ Structured Support Vector Machine for ordinal regression of the input image patches, where the ordering is determined by shape similarity to latent curvilinear structure. Finally, we progressively reconstruct the curvilinear structure by looking for geodesic paths connecting remote vertices in the graph built on the structured output rankings. Experimental results show that the proposed algorithm faithfully provides topological features of the curvilinear structures using minimal pixels for various datasets.

Introduction

Many computer vision algorithms have been proposed

to analyze tree-like curvilinear structures (also called line networks), such as blood vessels for retinal images [START_REF] Frangi | Multiscale vessel enhancement filtering[END_REF][START_REF] Staal | Ridge based vessel segmentation in color images of the retina[END_REF], filament structure of biological images [START_REF] Peng | Automatic 3D neuron tracing using all-path pruning[END_REF][START_REF] Wang | Novel 4D open-curve active contour and curve completion approach for automated tree structure extraction[END_REF], road network for remote sensing [START_REF] Hu | Road network extraction and intersection detection from aerial images by tracking road footprints[END_REF][START_REF] Lacoste | Point processes for unsupervised line network extraction in remote sensing[END_REF][START_REF] Valero | Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images[END_REF], and defects on materials [START_REF] Chambon | Road crack extraction with adapted filtering and Markov model-based segmentation[END_REF][START_REF] Iyer | A robust approach for automatic detection and segmentation of cracks in underground pipeline images[END_REF]. Although human can intuitively perceive the curvilinear structures in very different images and application scenarios, most curvilinear structure detection methods work only for one specific application. The difficulties arise because the geometry of the line networks shows various shapes. Also, there is often insignificant contrast between a pixel belonging to a curvilinear structure and the background texture. Thus, the information from an individual pixel fails to interpret the such topology. To represent an arbitrary shape of the curvilinear structure, segmentation algorithms [START_REF] Becker | Supervised feature learning for curvilinear structure segmentation[END_REF][START_REF] Frangi | Multiscale vessel enhancement filtering[END_REF][START_REF] Law | Three dimensional curvilinear structure detection using optimally oriented flux[END_REF] evaluate linearity scores, which correspond to likelihood of pixels given the curvilinear structure. Then, a threshold value is set to discard pixels showing Figure 1. Compared with (b) the segmentation [START_REF] Becker | Supervised feature learning for curvilinear structure segmentation[END_REF] and (c) the centerline detection [START_REF] Sironi | Multiscale centerline detection by learning a scale-space distance transform[END_REF] methods, (d) the proposed algorithm represents well topological features of the curvilinear structure. In this example, road network is partially occluded by trees or cars. Setting a threshold value for the local measure often fails to detect the underlying curvilinear structure because this process neglects correlated information of the reconstructed curvilinear structure.

Although the centerline is able to quantify scale (width) of curvilinear structure, it inaccurately classifies pixels around intersections. In this work we learn structured output ranking of the image patches that contain a part of curvilinear structures. The proposed graph-based representation uses the smallest pixels but also preserves the topological information.

low scores. This process implicitly ignores correlated information of the pixels on the curvilinear structure, so that it yields discontinuities on the reconstruction results. Centerline detection algorithm [START_REF] Sironi | Multiscale centerline detection by learning a scale-space distance transform[END_REF] is able to encode the width of the curvilinear structure; however, it is weak for localizing junctions of the curvilinear structure. On the other hand, graph-based models [START_REF] González | Delineating trees in noisy 2D images and 3D image-stacks[END_REF][START_REF] Peng | Automatic 3D neuron tracing using all-path pruning[END_REF][START_REF] Wang | Novel 4D open-curve active contour and curve completion approach for automated tree structure extraction[END_REF][START_REF] Zhao | Automated reconstruction of neuronal morphology based on local geometrical and global structural models[END_REF] reconstruct the curvilinear structure with geodesic paths in a sparse graph. For an efficient computation, these algorithms usually exploit a subset of pixels which correspond to local maxima of the linearity score. Contrary to the previous methods, our approach initially infers the curvilinear structure based on the structured ranking scores, which are related to the spatial patterns of the latent curvilinear structure. We then search for the longest geodesic path on the graph and iteratively add fine branches to rebuild the curvilinear structure with minimal pixels. Figure 1 compares the results obtained with curvilinear structure segmentation [START_REF] Becker | Supervised feature learning for curvilinear structure segmentation[END_REF], centerline detection [START_REF] Sironi | Multiscale centerline detection by learning a scale-space distance transform[END_REF], and the proposed method.

In this work we aim to learn structured rankings of the input image patches which may contain a part of curvilinear structures. In particular, we assign a high ranking score for the image patch if it shows plausible spatial pattern to the latent curvilinear structures. Since the curvilinear structures are arbitrarily oriented within in an image patch, we measure the local orientation of the image patch and rotate it with respect to the baseline orientation (align horizontally). We then explore geodesic paths in the graph built upon the structured ranking score map. The proposed algorithm can provide the topological importance level of the curvilinear structure. Our experiments demonstrate that the proposed algorithm localizes the curvilinear structure with a high accuracy compared to the state-of-the-art methods.

Related work

Geometrically speaking, curvilinear structures are rotatable and elongated shape. Image gradient information [START_REF] Frangi | Multiscale vessel enhancement filtering[END_REF][START_REF] Freeman | The design and use of steerable filters[END_REF][START_REF] Jacob | Design of steerable filters for feature detection using Canny-like criteria[END_REF][START_REF] Law | Three dimensional curvilinear structure detection using optimally oriented flux[END_REF][START_REF] Perona | Deformable kernels for early vision[END_REF] is useful to estimate local orientation and to describe the shape of the curvilinear structure. The local image features are insufficient to discern curvilinear structures from undesirable high-frequency components due to the lack of geometric interpretation. To take the structural information into account, graphical models [START_REF] González | Delineating trees in noisy 2D images and 3D image-stacks[END_REF][START_REF] Türetken | Reconstructing loopy curvilinear structures using integer programming[END_REF] formulate an energy optimization problem with spatial constraints in a local configuration. Stochastic models [START_REF] Jeong | Marked point process model for curvilinear structures extraction[END_REF][START_REF] Lacoste | Point processes for unsupervised line network extraction in remote sensing[END_REF] also specify a distribution of the line objects given image data with pairwise interaction terms. A sampling technique [START_REF] Green | Reversible jump Markov chain Monte Carlo computation and Bayesian model determination[END_REF] is employed to maximize the probability density. However, the geometric constraints are heuristically designed, and the number of constraints are increased to describe complex shaped line networks. Recently, machine learning algorithms have been involved to detect curvilinear structures latent in various types of images. Becker et al. [START_REF] Becker | Supervised feature learning for curvilinear structure segmentation[END_REF] applied a boosting algorithm to obtain an optimal set of convolution filter banks. A regression model is proposed to detect centerlines by learning the scale (width) of the tubular structures with the non-maximum suppression technique [START_REF] Sironi | Multiscale centerline detection by learning a scale-space distance transform[END_REF].

Structured learning system has been employed in image segmentation models based on random fields [START_REF] Bertelli | Kernelized structural SVM learning for supervised object segmentation[END_REF][START_REF] Kim | Image segmentation using higher-order correlation clustering[END_REF][START_REF] Lucchi | Learning for structured prediction using approximate subgradient descent with working sets[END_REF][START_REF] Szummer | Learning CRF using graph cuts[END_REF]. More specifically, Structured Support Vector Machine (SSVM) [START_REF] Tsochantaridis | Large margin methods for structured and interdependent output variables[END_REF] is used to predict model parameters for inference of the structured information between input image space and output label space. Exploring all possible combinations of the labels in the output space is computationally intractable. Thus, the random field models define the pairwise relationship in a neighborhood system to enforce the labeling consistency. While such prior models based on random fields are successful to describe convex shaped objects, it is inefficient to detect thin and elongated shaped The main contributions of this work are summarized as follows:

• We propose an orientation-aware curvilinear feature descriptor for the curvilinear structure inference;

• We learn a ranking function which can infer spatial patterns of the curvilinear structure within image patches; and

Overview

In this section, we define notations and provide an overview of the proposed algorithm (see Figure 2). Assume that an image I contains a curvilinear structure. We denote the latent curvilinear structure ⌦ : I 7 ! {0, 1} for any pixel x of the image I:

⌦(x) = ⇢ 1 if x is on the curvilinear structure, 0 otherwise. ( 1
)
This function is also related to a ground truth map, which is manually labeled, for the machine learning framework and for the performance evaluation. We compute a curvilinear feature map : I 7 ! R that represents oriented gradient information (see Section 3.1). Since information embedded in a single pixel is limited to infer the latent spatial patterns, we exploit image patches to compute input feature vectors. Let P x be a patch of the feature map values within p M ⇥ p M size of square window centered at

x, i.e., P x = { (x 0 ) | kx x 0 k 1  p M 2 }.
Using a rotation matrix R ✓ defined on Euclidean image space, we can rotate a patch with respect to the given orientation ✓ such as

P x,✓ = { (x 0 ) | kR | ✓ (x x 0 )k 1  p M
2 }. We use graph theory for shape simplification of the complex curvilinear structure. For the pixels showing higher rankings, we build an undirected and weighted graph G = (V, E). Then, we look for the longest geodesic path which corresponds to the coarset curvilinear structure in the image. We iteratively reconstruct the curvilinear by collecting paths that connect the remotest vertices in the graph. Consequently, the proposed algorithm can represent the different levels of detail in the latent curvilinear structure using the minimum number of pixels.

Curvilinear Feature

This section is devoted to compute the curvilinear feature descriptor that is used for the inputs of the learning system. We perceive the latent curvilinear structure based on inconsistency of background textures and its geometric characteristics. In other words, a sequence of pixels corresponding to the curvilinear structure has different intensity values compared to its surroundings, and shows thin and elongated shape. Thus, we compute multi-direction and multi-scale image gradients to detect locally oriented image features.

Curvilinear feature extraction

We obtain oriented gradient maps using a set of steerable filters [START_REF] Freeman | The design and use of steerable filters[END_REF][START_REF] Jacob | Design of steerable filters for feature detection using Canny-like criteria[END_REF][START_REF] Perona | Deformable kernels for early vision[END_REF]. Before applying the convolution operations, we normalize the training and test images to remove the effects of various illumination factors:

Ĩ = 1 1 + e I E[I] max(I) min(I) , (2)
where E[I] is the sample mean of the image. The steerable filters are created by the second order derivative of isotropic 2D Gaussian kernels. Let f ✓, be a steerable filter that accentuates image gradient magnitude for direction ✓ at scale . To take into account varying orientations and widths of the curvilinear structure, a feature map is combined by multiple filtering responses:

= 1 |⇥| X ✓ max {f ✓, ⇤ Ĩ}, (3)
where |⇥| denote the total number of orientations. We first find the maximum filtering responses of the scale spaces, and then average for all directional responses. In this work we consider 8 orientations and 3 scales.

Local orientation estimation

In the previous subsection, we evaluate the presence of curvilinear structure from the amount of gradient magnitudes in image patches. Complex shaped curvilinear structure also consists of various local orientations.

Assume that B 2 {0, 1} M be a basis spatial pattern of the curvilinear structure for the baseline orientation ( ✓ = 0 ). We manually design B as a simple line shape template which highlights the middle of image patch (see Figure 3). If an image patch P

x contains a part of curvilinear structure, pixels on the curvilinear structure are sequentially accentuated towards a particular direction ✓. Recall that the geometric properties of the curvilinear structure, it is rotatable and symmetric. We can rotate image patch to be aligned with the basis spatial pattern.

More specfically, we estimate the local orientation shown in image patches using statistical difference of the areas determined by B. For a rotated image patch P

x,✓ , we compute normalized histograms of the pixels which are labeled as curvilinear structure p

x,✓ = hist (P x,✓ | B) and its counterpart q

x,✓ = hist (P x,✓ | ¬B), where hist (•) denotes histgram of the given distribution with 32 bins. Similar to [START_REF] Arbeáez | Contour detection and hierarchical image segmentation[END_REF], we employ 2 test to compute statistical difference of two distributions:

✓ = argmax ✓ 2 (p x,✓ , q x,✓ ) , (4) 2
(p, q) = 1 2 X i (p(i) g(i)) 2 p(i) + g(i) . (5)
Figure 3 compares statistics of the image patches with respect to different orientations. If an image patch contains curvilinear structure, 2 test score shows uni-modal distribution which shows a peak at the dominant orientation.

Learning

We regulate the orientation and the shape of the input patches with respect to the basis spatial pattern B. In this section, we aim to learn a function that predicts structured output rankings of the input image patches. Thus, the output rankings indirectly infer the spatial patterns of the curvilinear structure for image patch.

Let vec (•) be an operator to convert an image patch into a column vector. For a pixel x i , the input feature vector is defined as z i = vec (P xi,✓i | B) 2 R N and y i 2 R + denote the corresponding rank value. For a feature vector, we exploit subsampled pixels on the linear template to avoid data imbalance. Thus, the dimension of feature vector N is small than the total number of pixels within the patch, i.e., N  M .

For the setup of machine learning, a training dataset D = {(z i , y i )} K i=1 consists of the K input-and-output pairs. Let {z 1 , . . . , z K } 2 Z be an unordered list of the input feature vectors and {y 1 , . . . , y K } 2 Y be the corresponding classes. The input feature vector z is built from the curvilinear feature map . Our goal is to learn a ranking function h(z) which assigns a global ordering (ranking) of feature vectors: h(z i ) > h(z j ) , y i > y j . The inner product of the model parameter and a feature vector w | z is used to predict ranking score of the given image patch.

In our work Structured SVM framework [START_REF] Joachims | Training linear SVMs in linear time[END_REF][START_REF] Mittal | Taxonomic multi-class prediction and person layout using efficient structured ranking[END_REF] is employed to exploit the structure and dependencies within the output space Y. To encode the structure of the output space, a loss value of each input feature vector is defined: h(z i ) > h(z j ) , i < j . The loss value evaluates the quality of the learning system. Intuitively, the input image patches containing curvilinear structure are comparable to the basis spatial pattern. Hence, as an image patch is similar to the basis spatial pattern, we should put it on a higher ranking. Specifically, we compute the loss value as the overlapping ratio between image patch from the groundtruth map ⌦ and the basis B:

i = 1 |⌦ xi,✓i \ B| |⌦ xi,✓i [ B| , (6)
where ⌦ xi,✓i denotes a rotated image patch referring to groundtruth map ⌦. The objective function of Structured SVM with a single slack variable ⇠ is given by:

min w,⇠ 0 1 2 w | w + C⇠ s.t. 1 |N| P (i,j)2N c ij w | (z i z j ) 1 |N| P (i,j)2N c ij ⇠ j i , 8(i, j) 2 N, 8c ij 2 {0, 1} (7)
where c ij is the indicator variable to reduce the number constraint in linear complexity O(K):

c ij = ⇢ 1 if i < j and w | z i w | z j < 1, 0 otherwise. ( 8
)
Cutting plain algorithm [START_REF] Tsochantaridis | Large margin methods for structured and interdependent output variables[END_REF] is employed to solve [START_REF] Frangi | Multiscale vessel enhancement filtering[END_REF]. For the details about the Structured SVM optimization, we refer the reader to [START_REF] Joachims | Training linear SVMs in linear time[END_REF][START_REF] Mittal | Taxonomic multi-class prediction and person layout using efficient structured ranking[END_REF][START_REF] Tsochantaridis | Large margin methods for structured and interdependent output variables[END_REF]. In the following section, we plot the initial segmentation map from the structured rankings scores. We also exploit this ranking scores for the shape simplification based on graph traversal algorithm.

Curvilinear Structure Reconstruction

To represent the latent curvilinear structure, various local measures have been proposed to detect irregular shaped curvilinear structure. The score of the local measure is regarded as a likelihood probability whether a pixel is on the latent curvilinear structure. Most of the previous works represent the curvilinear structure as a binary map based on the linearity scores of the pixels. This approach easily misinterpret the topological features. In this section, we propose a novel structured score map based on the output ranking scores and the basis spatial pattern. We also develop a graph-based model which is able to organize the topological features of the structure in different levels of detail. Figure 4 shows step-by-step processing results to reconstruct the latent curvilinear structure.

Structured score map

We have obtained a score function h(z; w) that evaluates the compatibility between the input features and the underlying curvilinear structure based on the structured output rankings. From the training dataset, we pre-compute the proportion ⇢ of pixels being part of the curvilinear structure. Note that the value ⇢ maximizes F

1 test scores at the groundtruth maps in the training dataset. During the test phase, we retain the output ranking according to the output ranking scores from the highest to the ⇢|I|-th rank. As shown in Figure 4 (c), a binary map can be disconnected.

To avoid unwilling breakpoints, we composite the structured score map ⇧ : I 7 ! R using the binary map, induced by ranking scores w | z, and the basis spatial pattern B. Recall that we initially estimate the ranking score using the shape similarity between oriented image patch of ground Compute the longest geodesic path T in G 0 using [START_REF] Corneil | Diameter determination on restricted graph families[END_REF] 7:

longest path length |T | 8:
if longest path length< ˆthen w(u, v) 0, 8{u, v} 2 T , Update all edge weights on the path T as 0 for the given subgraph G 0 13: end while truth and the basis spatial pattern in [START_REF] Corneil | Diameter determination on restricted graph families[END_REF]. Thus, it can be regarded as the inverse mapping from output ranking score to the latent curvilinear structure. Let b = vec (B) be the column vector version of the binary mask B and Q xi,✓i be the patch of structured score map ⇧ centered at x i with ✓ i . We minimize the following cost function with respect to

⇡ i = vec (Q xi,✓i ): J = min X xi2I 0 kb (w | z i )⇡ i k 2 2 . ( 9
)
This is a least square problem, so that the solution is found at points which satisfies @J @⇡ = 0. To reconstruct the structured segmentation map, we synthesize obtained patches Q xi,✓i = vec 1 ( ⇡i ) at the subsampled grid points on image x i 2 I 0 , where ⇡i = (w | z i ) 1 b. We refer the readers to [START_REF] Kwatra | Texture optimization for example-based synthesis[END_REF] for the implementation details of the related texture synthesis technique.

Progressive curvilinear path reconstruction

We consider the graph G = (V, E) where V is the set of pixels. Two pixels u and v are connected by the edge if and only if ku vk  p 2. Moreover, we assign a weight for each edge {u, v} 2 E as w(u, v) = ku vk ⇧(u)+⇧(v)

2

. A path P in the graph G is a sequence of distinct vertices such that consecutive vertices are adjacent. The length of P is the sum of the weights of its edges, and the distance dist (u, v) between two vertices u and v is the minimum length of a path from u to v. The eccentricity ecc (v) denotes the maximum distance from the vertex v to a vertex

u 2 V , i.e., ecc (v) = max u2V dist (u, v). The diameter diam (G) of G equals max v2V ecc (v), i.e.
, it is the maximum distance between two vertices in G.

To simplify the latent curvilinear structure, we look for long geodesic paths in the subgraph G 0 of G induced by the pixels with structured score map ⇧. More precisely, our algorithm computes a diameter of G 0 , i.e., a shortest path T with length diam (G 0

). This path T is added in the simplified curvilinear structure, then the path T is contracted into a single (virtual) vertex. For the implementation, the weight of all edges of T become 0. We repeat this process till the diameter of the subgraph is larger than pre-defined path length ˆ. The entire procedure is summarized in Algorithm 1.

The time-consuming part of the proposed algorithm is the computation of a diameter of G 0 . Rather than computing all pair distances (which requires a linear number of application of Dijkstra's algorithm), we use an efficient heuristic algorithm called 2-sweep algorithm [START_REF] Corneil | Diameter determination on restricted graph families[END_REF]. The 2-sweep algorithm randomly picks a vertex t in G 0 , then performs Dijkstra's algorithm from t to find a vertex u at the maximum distance from t, i.e., dist (u, t) = ecc (t). Then, it computes (using Dijkstra's algorithm) a path from u to a vertex v at the maximum distance from u. The length of the second path (from u to v) is a good estimation of diam (G 0

). Note that this algorithm is able to compute the exact diameter if the graph has tree structure [START_REF] Borassi | Fast diameter and radius BFSbased computation in (weakly connected) real-world graphs: With an application to the six degrees of separation games[END_REF]. Topologically speaking, most of latent curvilinear structures are very close to trees [START_REF] González | Delineating trees in noisy 2D images and 3D image-stacks[END_REF][START_REF] Türetken | Automated reconstruction of dendritic and axonal tress by global optimization with geometric priors[END_REF]. Therefore, the 2-sweep algorithm is well adapted to reconstruct the tree-like curvilinear structures. For a better understanding, we schematically explain the intermediate steps of the proposed curvilinear structure simplification algorithm in Figure 5.

Experimental results

In this section, we first discuss the parameters of the proposed algorithm and datasets. We then compare the quantitative and qualitative results of the proposed algorithm and those of competing models proposed by [START_REF] Frangi | Multiscale vessel enhancement filtering[END_REF], [START_REF] Law | Three dimensional curvilinear structure detection using optimally oriented flux[END_REF], [START_REF] Becker | Supervised feature learning for curvilinear structure segmentation[END_REF], and [START_REF] Sironi | Multiscale centerline detection by learning a scale-space distance transform[END_REF].

Parameters and Datasets

The proposed algorithm requires few parameters to compute the curvilinear feature descriptor . We

Evaluations

The proposed algorithm progressively reconstructs the curvilinear structure by adding a long path on the subgraph. Figure 6 shows the intermediate steps of the proposed curvilinear structure simplification algorithm for DRIVE dataset. Unlike the previous models, the proposed algorithm is able to show different levels of detail for the latent curvilinear structure. Such information to visualize shape complexity of the curvilinear structure cannot be retrieved by setting a threshold. In practice, a few number of iterations is required to converge the algorithm and each step to find a long path takes less than milliseconds for the computation. For the experiments, we use a PC with a 2.9 GHz CPU (4 cores) and 8 GB RAM. Moreover, we visually compare the performance of the proposed algorithm with the competing algorithms. Figure 7 shows the results from DRIVE, RecA, Aerial, and Cracks dataset, respectively. The proposed algorithm is the most suitable to show the topological information of the latent curvilinear structures. For the quantitative evaluation, we provide F 1 scores of the proposed algorithm and the state-of-the-art models in Table . 1. The measure of true positive is sensitive for the misalignment; therefore, we consider surrounding pixels of the detection results as the true positive if a predicted point is falling into the ground truth with a small radius (equivalent to its thickness parameter ⌧ ) similarly to [START_REF] Sironi | Multiscale centerline detection by learning a scale-space distance transform[END_REF]. We also provide the average proportion of pixels to represent the curvilinear structures. It shows that the proposed algorithm efficiently draws the curvilinear structures using smaller number of pixels than the other algorithms.

The proposed algorithm achieved the best F 1 scores for all datasets except the DIRVE dataset. It is because the proposed algorithm use the fixed thickness ⌧ to describe linear structure in the binary mask B; however, the images consisting of DRIVE dataset exhibit varied thicknesses of blood vessels and many junction points. In other words, we obtain a model parameter regarding to the manually designed binary pattern B. To overcome this drawback, we plan to exploit generative binary patterns based on the training images in the future, which is possible in our framework.

The topology of the latent curvilinear structure is varied while we compare it within the same dataset. Especially, Cracks dataset consists of difficult images due to the rough surfaced background textures. The normalization operation (2) is necessary to remove irregular illumination factors on the datasets. Manual segmentation (ground truth) contains many errors around boundaries and minutiae components. Also, the number of training data employed in this work is relatively small due to the difficulties of making the accurate annotations. It is remarkable that the proposed algorithm shows good performance for all datasets using the imperfect training dataset.

Conclusions

This paper proposed a curvilinear structure reconstruction algorithm based on the ranking learning system and graph theory. The output rankings of the image patches corresponded to the plausibility of the latent curvilinear structure. Using an optimal number of pixels, the proposed algorithm provided different levels of detail during reconstruction of the curvilinear structure. More precisely, we learned a ranking function based on Structured SVM with the proposed orientation-aware curvilinear feature descriptor. In this paper, we developed a novel graphical model that infers the curvilinear structure according to the topological importance. The proposed algorithm looked for remote vertices on the subgraph which is induced from the output rankings. Across the various types of datasets, our model showed good performances to reconstruct the latent curvilinear structure with a smaller number of pixels comparing to the state-of-the-art algorithms.

Figure 2 .

 2 Figure 2. Overview of the proposed algorithm

2 Figure 3 .

 23 Figure 3. In this figure, we compare the statistics of pixel values in the rotated image patches with respect to the given binary mask B 2 {0, 1} M . Assume that B encodes a basis spatial pattern of the curvilinear structure toward the baseline orientation ( ✓ = 0 ), where darker gray color refers to 1. To estimate local orientation of the input image patch, we rotate it and find the orientation which maximizes the statistical difference of pair distributions. If the input image patch does not contain curvilinear structure (upper row), there is no meaningful statistical difference for any orientation.

Figure 4 .Algorithm 1 4 :

 414 Figure 4. For the given image (a), we compute (b) the structured output ranking with Structured SVM. We retain binary map (c) from the highest to ⇢|I|-th rankings. However, it could be broken. We exploit the basis spatial pattern B to reconstruct (d) the structured score map. Graph-based representation provides a tool to analyze the topological features of the curvilinear structures. (e) The coarsest curvilinear structure evolves to define (f) branches.

Figure 5 .

 5 Figure 5. Toy example of the proposed curvilinear structure inference algorithm: (a) input image contains a curvilinear structure which is denoted by gray color; (b) subgraph G 0 is induced from the structured segmentation map; (c) and (d) show the intermediate processes of the 2-sweep algorithm starting from vertex t to find a diameter of the subgraph; (e) we assign 0 weight for all edges on the path; and (f) we repeat the process and add branches if the path length is larger than pre-defined length ˆ.

  use 8-different orientations in this work: ⇥ = {0 , 22.5 , 45 , 57.5 , 90 , 112.5 , 135 , 157.5 }. The size of steerable filters is fixed to 21 ⇥ 21 pixels. The scale factors 2 and the minimum path length ˆare adaptively selected for each dataset to obtain the best performances. The binary spatial pattern B is based on the physical attribute of the dataset in that thickness ⌧ of curvilinear structure shows various depending on the dataset. For Structured SVM training, we sample 2000 image patches on each dataset. All patch sizes are fixed to M = 33 2 . C which controls the relative importance of slack variables is set to 0.1 for all datasets. We choose the set of parameters for the SSVM training via 3fold cross validation [17] which maximizes the average F 1 score [22] of the training set. We test our curvilinear structure model on the following public datasets: • DRIVE [27]: The dataset consists of 40 retina scan images with manual segmentation by ophthalmologists to evaluate the blood vessel segmentation algorithms. We use 20 images for the training and 20 images for the test, respectively. The path length ˆis set to 40. The scale factors and thickness are set to 2 = {2, 4, 8} and ⌧ = 5, respectively. • RecA [14]: We collect electron microscopic images of RecA proteins on DNA which contain filament structure. We use 4 training images and 4 test images. We use ˆ= 30, 2 = {4, 8, 12}, and ⌧ = 5. • Aerial [26]: The dataset contains 14 remote sensing images of road networks. We select 7 images for the training and 7 images for the test, respectively. We use ˆ= 80, 2 = {4, 8, 12}, and ⌧ = 9. • Cracks [5]: Images of the dataset correspond to road cracks on the asphalt surfaces. We use 6 images to train and test the algorithms on different 6 images. For this dataset, we set to ˆ= 30, 2 = {2, 4, 8}, and ⌧ = 3, respectively.

Figure 6 .

 6 Figure 6. Intermediate steps of the curvilinear structure reconstruction for a retina image. We iteratively reconstruct the curvilinear structure according to topological importance orders. As the iteration goes on, detail structures (layer) appear.

Figure 7 .

 7 Figure 7. Illustration of the curvilinear structure reconstruction results on DRIVE, RecA, Road, and Crack dataset (top to bottom). We compare the results of Frangi et al.'s [7], Law & Chung's [20], Becker et al.'s [2], Sironi et al.'s [26], and the proposed algorithm (left to right).

Table 1 .

 1 Comparison of the quantitative performances over the datasets. We provide results with various performance measures: (a) F1 scores, and (b) average proportion ⇢ of the pixels being a part of curvilinear structure. Boldfaced numbers are used to show the best score in each test.

		Frangi et al. [7]	DRIVE RecA Aerial Cracks 0.33 0.33 0.32 0.056
		Law & Chung [20]	0.43	0.21	0.25	0.085
	(a) F	Sironi et al. [26] Proposed (Graph) Frangi et al. [7]	0.50 0.55 0.36 37.09	0.45 0.50 0.59 16.14 22.19 0.53 0.55 0.59	0.23 0.27 0.38 28.64
	Law & Chung [20] (b) ⇢(%) Becker et al. [2] Sironi et al. [26] Proposed (Graph)	19.60 12.67 5.41 0.17	29.82 29.51 9.39 9.76 5.47 0.83 0.57 0.35	33.00 32.13 17.12 1.07

1

Becker et al.

[START_REF] Becker | Supervised feature learning for curvilinear structure segmentation[END_REF]