Elia Moscoso Thompson

Silvia Biasotti

Andrea Giachetti

Claudio Tortorici

Naoufel Werghi

Ahmad Shaker Obeid

Stefano Berretti

Hoang-Phuc Nguyen-Dinh

Minh-Quan Le

Hai-Dang Nguyen

Ahmad Shaker Obeid

Minh-Triet Tran

Leonardo Gigli

Santiago Velasco-Forero

Beatriz Marcotegui

Ivan Sipiran

Benjamin Bustos

Ioannis Romanelis

Vlassis Fotis

Gerasimos Arvanitis

Konstantinos Moustakas

Ekpo Otu

Reyer Zwiggelaar

David Hunter

Yonghuai Liu

Yoko Arteaga

Ramamoorthy Luxman

SHREC'20 track: Retrieval of digital surfaces with similar geometric reliefs

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Figure 1: A visual representation of the challenge proposed in this contest. A query model Q with a bark-like relief impressed on its surface is selected. In the ideal case, models with a bark-like relief are retrieved before than models with different reliefs, independently of the global geometry of the models. The "check" and "cross" marks highlight models that are relevant or non-relevant to the query.

Geometric reliefs are a significant component for the local characterization of a surface, which are independent of its overall shape and spatial embedding.

Being able to characterize different repeated relief patterns on a surface is a key issue for several tasks, such as the analysis and detection of molding marks, composite materials and ornamental decorations on an object surface. The characterization of this local surface property is an open problem that is gaining more and more interest over the years.

Several methods have been introduced for the characterization of local, repeated, geometric variations on a surface, showing this is a vivid research field.

In the set of methods that face this problem, we distinguish two main strategies: i) to (fully or partially) project a 3D model into an image or a set of images and then apply a texture image retrieval method; ii) to extend the image texture characterization directly to 3D model representations. Examples of methods that face this problem as an image texture retrieval problem have been proposed for the classification of trees based on their bark reliefs [START_REF] Othmani | Single tree species classification from terrestrial laser scanning data for forest inventory[END_REF] or the classification of engraved rock artifacts based on their height-fields [START_REF] Zeppelzauer | Interactive 3D segmentation of rock-art by enhanced depth maps and gradient preserving regularization[END_REF]. In this trend, the combination of the SIFT descriptor with the Fisher Vector, which gave very good performances for image texture retrieval [START_REF] Cimpoi | Describing textures in the wild[END_REF], has shown very good performances also for the retrieval of relief patterns [START_REF] Giachetti | Effective characterization of relief patterns[END_REF]: in this case, representative surface images were obtained by projecting the mean curvature of the neighborhood of the center of the model. Methods that directly use 3D representations are generally designed for triangle meshes or point clouds, because these representations allow a precise and locally adaptive representation of the surface that is less accessible with grids (e.g., voxels). This class of methods includes the numerous extensions of the the Local Binary Pattern (LBP) [START_REF] Ojala | A comparative study of texture measures with classification based on featured distributions[END_REF] proposed in recent years. The first of these extensions was the meshLBP [START_REF] Werghi | Local binary patterns on triangular meshes: Concept and applications[END_REF][START_REF] Werghi | The mesh-LBP: A framework for extracting local binary patterns from discrete manifolds[END_REF], followed by the edgeLBP [START_REF] Thompson | Description and retrieval of geometric patterns on surface meshes using an edge-based lbp approach[END_REF][START_REF] Thompson | Edge-based LBP Description of Surfaces with Colorimetric Patterns[END_REF][START_REF] Thompson | Retrieving color patterns on surface meshes using edgelbp descriptors[END_REF][START_REF] Biasotti | Retrieval of surfaces with similar relief patterns: Shrec'17 track[END_REF][START_REF] Moscoso Thompson | Retrieval of gray patterns depicted on 3d models[END_REF] and the mpLBP [START_REF] Moscoso Thompson | mpLBP: An Extension of the Local Binary Pattern to Surfaces based on an Efficient Coding of the Point Neighbours[END_REF][START_REF] Moscoso Thompson | mplbp: A point-based representation for surface pattern description[END_REF]. Besides the different strategies to encode the neighbour of a vertex, the main idea behind these LBP-based 3D characterizations is to replace the gray-scale value in the pixels of an image with geometric or colorimetric properties (e.g., curvatures or color channels) defined on the faces or the vertices of the model. Recently, also the multi-scale properties of the Laplacian operator have been used in [START_REF] Othmani | 3d geometric salient patterns analysis on 3d meshes[END_REF] to obtain a scale-aware surface description. In this case, the parts of interest are obtained by analyzing the difference between a surface and its counterpart obtained by smoothing.

Based on the increasing number of methods for 3D pattern retrieval made available in recent years, we think it is now important to understand how much existing methods are suitable to address realistic applications. The aim of this SHREC 2020 track is to provide a new benchmark for geometric pattern retrieval and to evaluate methods for assessing the similarity between two objects, only on the basis of the local, geometric variations of their surfaces, without considering their global shape. Our new collection of 3D models is characterized by different classes of reliefs on the models surface. A visual representation of the task addressed in this contest is shown in Figure 1.

These reliefs represent different kinds of materials, like bark wood or rocks, and structures, like bricks. The peculiarity of the models proposed in this contest is that a realistic geometric pattern (derived from real texture images) is applied to a number of base models, some of those have a non-trivial topology (with handles, tunnels, boundaries, etc.).

The remainder of the paper is organized as follows. Section 2 briefly overviews existing datasets and benchmarks that address the geometric pattern retrieval or strictly related tasks. Section 3 describes the 3D models used in this challenge and details how they have been generated from a base model and a set of real textures. Section 4 details the eight methods submitted to this contest, while Section 5 introduces the methodology and the measures used to evaluate the different runs. Section 6 presents the settings of the runs submitted to this contest and their retrieval and classification performances. Finally, discussions and concluding remarks are in Section 7.

Related benchmarks

The interest for geometric pattern analysis has been borrowed from image texture analysis, which is a typical problem of Computer Vision. To the best of our knowledge, the first dataset explicitly delivered for 3D texture analysis was the "MIT CSAIL Textured Models Database" [START_REF]MIT CSAIL Textured Models Database[END_REF]. During years, several factors have concurred to the increase of collections of 3D models equipped with textures; for instance, the improvement of the spatial data acquisition systems that also allow the representation of the surface details; the increase of applications interested in the comparison of 3D models on the basis of their texture or material and, even, the success of benchmarks and methods for image texture retrieval [START_REF] Cimpoi | Describing textures in the wild[END_REF].

Without aiming to list all the existing general purpose data collections that contain some model equipped with a 3D texture (e.g., Skechfab [START_REF]Publish & find 3D models online[END_REF] or Turbosquid [START_REF]Turbosquid[END_REF]), we focus on benchmarks for similarity evaluation that provides also a ground truth and a number of evaluation measures. Several previous SHape REtrieval Contest (SHREC) tracks are somehow related to our challenge. The first SHREC track that partially faced the problem of local surface characterization is the SHREC'13 track on retrieval and classification of 3D textured models [START_REF] Cerri | SHREC'13 Track: Retrieval on Textured 3D Models[END_REF], extended the SHREC'14 track [START_REF] Biasotti | Retrieval and classification on textured 3d models[END_REF] with the same task but a larger dataset. A complete analysis of the methods tested on those contests was published in [START_REF] Biasotti | Retrieval and classification methods for textured 3D models: A comparative study[END_REF]. Differently from this contest that only focuses on local, geometric surface variations, there, the task was to group models based on their overall shape and their colorimetric texture. In other words, models that were globally similar but with different textures were less similar than those with the same shape and texture. While in the SHREC'13 and SHREC'14 tracks, texture analysis was colorimetric and only marginal, here it is geometric and the only aspect that drives the similarity among models.

The interest on geometric reliefs shaped into the SHREC'17 track on the retrieval of reliefs [START_REF] Biasotti | Retrieval of surfaces with similar relief patterns: Shrec'17 track[END_REF]. There, fabrics with different patterns were acquired with photogrammetry and used to create a benchmark for the pattern retrieval task.

That benchmark entirely focused on the local characterization of surfaces based on patterns and it is currently used as the reference benchmark by many of the works on this topic. The high number of subscribers to that track, but the quite limited number of effective runs submitted to the track revealed the high interest in the subject and the difficulty in facing that task. Aside from some highlights, the methods submitted to the original contest showed quite limited performances, later on the research on this topic progressed, several methods have been proposed and successfully tackled such a benchmark. On a similar note, the SHREC'18 track on gray patterns [START_REF] Moscoso Thompson | Retrieval of gray patterns depicted on 3d models[END_REF] proposed a retrieval task on a dataset of models characterized by gray-scale patterns. Interestingly, all the participants proposed feature-vector based methods.

It is also worth mentioning the SHREC'18 track on geometric pattern recognition [START_REF] Biasotti | Recognition of Geometric Patterns Over 3D Models[END_REF] that differs from the previous benchmarks on 3D pattern retrieval because the participants were asked to locate a query relief sample in a set of 3D models. The challenge was to recognize if a type of geometric pattern is contained or not in another model and, eventually, to identify it on the model. The challenge launched in that task is still an open problem and that track report can be considered as a position paper on 3D pattern recognition.

Based on the progresses made in recent years, it is now important to analyse how much the performance of the various approaches has improved compared to the methods presented in the SHREC'17 track [START_REF] Biasotti | Retrieval of surfaces with similar relief patterns: Shrec'17 track[END_REF], while bearing in mind that the more general issue of 3D pattern recognition presented in the SHREC'18 track [START_REF] Biasotti | Recognition of Geometric Patterns Over 3D Models[END_REF] is still open. Figure 2: An example of the transformation process from texture to height map. On the left, the original textures are shown. On the right, the final height-map obtained with the process explained in Section 3. This process can end with a binary image (just black and white, as in the example at the Top) or a gray-scale one (like that at the Bottom).

The dataset

The dataset proposed for this challenge consists of 220 triangulated surfaces.

Each one of them is characterized by one of 11 different geometric reliefs.

To create the models, we selected the 20 base models already used in [START_REF] Moscoso Thompson | Retrieval of gray patterns depicted on 3d models[END_REF].

These models represent pots, goblets and mugs. The surfaces of these models and may present a boundary, depending on the object represented. Then, a set of 11 textures is selected from the free dataset of textures available online from the site Texture Haven [START_REF]Texture Haven[END_REF] that contains a set of natural, high quality texture images made from scanned maps. Most of these textures represent real bricks, floors, roofs surfaces and rock or wood materials.

Given the nature of the textures selected, on the one hand, models of buildings or their agglomerates would be the most realistic; on the other hand, we think that in 3D pattern retrieval the most challenging issue is to deal with free-form models, possibly with more complex bendings and non trivial topology. Given the heterogeneity of the textures selected and the geometry of the base models, methods that perform well in this contest have a high chance of being equally valid in other contexts, with little to no changes.

We transform each texture in height values suitable to create a geometric relief by converting each texture into a gray-scale image. The brightness and the contrast values of each image were tuned for each image, based on the values that better enhance the details of the respective color texture. The obtained height field map is applied to the models: initially, the texture is projected onto the target model. Depending on the surface bending, this procedure deforms the texture. To limit this effect, each model is fixed by hand, in particular, in correspondence of significant distortions and parts of the surface with complex geometry (like tight handles). Finally, we rise the vertices of the triangle mesh based on the gray-scale value of the previously processed image along the normal vectors of the models. The same process is repeated for all the textures. A couple of examples of the conversion of a texture into a height map are depicted in Figure 2.

Finally, the models are slightly smoothed to minimize the perturbations in the color derived from the gray-scale conversion of the textures and the models are sampled with 50000 vertices. Base models, height fields and examples of the final 3D models are shown in Figure 3.

The Ground Truth

The challenge proposed in this contest is to group the models only according to the geometric reliefs impressed on them, rather than their shape. In other words, a perfect score is obtained if a method is able to define 11 groups of 20 models each, each group with the models characterized by one of the 11 different geometric reliefs.

The participants and the proposed methods

Seven groups subscribed to this track. All of them submitted at least one method; one group submitted two methods; overall, eight methods and twenty runs were submitted to evaluation. The participants are anonymous for review for and their proposed method(s) are summarized in the following. While the APPF extraction and APPFD algorithms are described in detail here, the reader is referred to the literature in [START_REF] Otu | Nonrigid 3d shape retrieval with happs: A novel hybrid augmented point pair signature[END_REF] for more details on the other stages. Finally, the Fisher Kernel approach to local descriptor aggregation with Gaussian Mixture Model (GMM) [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF][START_REF] Simonyan | Fisher vector faces in the wild[END_REF] is applied to the local APPFD to derive a single signature, APPFD-FK, for each 3D shape. Figure 5 shows the processing pipeline of the APPFD-FK algorithm. The three main steps of the algorithm are outlined in the following:

1. Augmented Point Pair Feature Descriptor (APPFD): The APPFD is derived by three sub-steps: i) For each LSP extracting four-dimensional local Point-Pair Feature (PPF), f 1 = (α, β, γ, δ) as in [START_REF] Wahl | Surflet-pair-relation histograms: a statistical 3d-shape representation for rapid classification[END_REF], ii) Augmenting f 1 to a six-dimensional feature -the Augmented PPF, using additional two-dimensional local angular feature, f 2 = (θ, φ), depicted in Figure 4, and iii) Discretizing the six-dimensional augmented feature f 3 = (α, β, γ, δ, θ, φ) into one or multi-dimensional histograms to yield the final local APPFD. Firstly, extracting PPF involves two sets of oriented points, p i , p j = [(p i , n i ), (p j , n j )], used to encode the underlying surface geometry for their patch on a 3D surface. For every possible combination q of p i , p j in LSP (i.e. r-neighbourhood of p ki ), where p i is the source point w.r.t. the constraint in (1) holding TRUE, where i = j, then a transformation independent Darboux frame, D f = U, V, W is defined as:

U = n i , V = U × ((p j -p i )/δ), W = U × V . |n i • (p j -p i )| ≤ |n j • (p j -p i )| (1)
Alternatively, p j becomes the source point (i.e. point with the larger angle between its associated normal and the line connecting the two points)

if the constraint in (1) is FALSE, and the variables in (1) are reversed.

f 1 (p i , p j ) = (α, β, γ, δ) is then derived for the source point as follows:

α = arctan(W • n j , U • n j ), α ∈ - π 2 , π 2 (2)
β = V • n j , β ∈ (-1, 1) (3)
γ = U • p j -p i p j -p i , γ ∈ (-1, 1) (4)
δ = p j -p i . (5)
Secondly, f 2 (p i , p j ) = (θ, φ) is extracted for every possible combination of point-pair, p i , p j in the LSP, because f 1 is not robust enough to capture the entire geometric information for a given surface region or LSP. In addition, the PPF approach opens up possibilities for additional feature space. Therefore, as illustrated in Figure 4, θ is geometrically the angle of the projection of the vector, -→ S onto the unit vector -→ V 1 , and φ is the angle of the projection of the vector -→ S onto the unit vector -→ V 2 , where

-→ V 1 = p i -p c , -→ V 2 = p i -l, and - → S = p i -p j , with p c = 1 ni ni i=1 p ki (i.e.
LSP centroid), and l = (p j -p c ), the vector location of p ki w.r.t. its LSP.

Note that p i , p j , p c , and l are all points in R 3 space, although l is a vector.

Basically, α, β, γ are the angular variations between (n i , n j ), while δ is the spatial distance between p i and p j . In Euclidean geometry, each of the projections φ and θ can be interpreted as an angle between two vec-

tors. For example ∠ 1 - → S , -→ V 1 and ∠ 2 - → S ,
-→ V 2 are equivalent to θ and φ respectively. These angles are derived by taking the scalar products of

( - → S • -→ V 1 ) for ∠ 1 , and ( - → S • -→ V 2 )
for ∠ 2 about a point p i in a given LSP. Mathematically, scalar products defined in this manner are homogeneous (i.e.

invariant) under scaling [START_REF]Dot Product[END_REF] and rotation [START_REF] Mathworld | Dot Product[END_REF]. For this reason, our twodimensional local geometric features, θ, φ are rotation and scale invariant for 3D shapes under rigid and non-rigid affine transformations. Moreover, notice that since geometric information are embodied by these variations and projections, the global shape of the 3D shape is not considered at all.

Lastly, for each LSP or keypoint, p ki with q combinations, q(q -1)/2 six-dimensional APPF:

f 3 = (f 1 + f 2 ) is obtained thus: f 3 (p i , p j ) = (f 1 (p i , p j ), f 2 (p i , p j )) = (α, β, γ, δ, φ, θ
) and descritized into histograms to yield APPFD. In computing APPFD for this task, 4500 points and their normals, (P, N ) were sampled from each 3D surface, K keypoints, {p ki , i = 1 : K} were selected and for each p ki , a LSP, {P i , i = 1 : K} and their corresponding normals, {N i , i = 1 : K} were computed. Points in P i are within the specified radius, r = 0.30 -0.40 around p ki .

For our first and second experimental runs (APPFD-FK-run1 and APPFD-FK-run2) a one-dimensional [0, 1] normalized histogram with bins = 35 is used to represent each of the feature-dimension in APPF, concatenated to yield a final 6 times 35 = 210-dimensional local descriptor for each LSP or keypoint. In our third experimental run (APPFD-FK-run3) all six-dimensional feature in APPF are discretized into a multi-dimensional histogram with bins = 5 in each feature-dimension, flattened and normalized to give 5 6 = 15625-dimensional local descriptor for each LSP or keypoint.

2. Keypoint APPFD Aggregation with Fisher Vector (FV) and Gaussian Mixture Model(GMM): Inspired by the work in [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF][START_REF] Simonyan | Fisher vector faces in the wild[END_REF], the final stage of our novel APPFD-FK algorithm consists of computing a global FV for each input 3D shape given their keypoint APPFDs. The FV computation relies on training a GMM, as a generative probabilistic model, with the keypoint APPFDs for all database shapes. The GMM is trained with 10 Gaussians, using diagonal covariances for all experimental runs. Using the trained GMM and local keypoint APPFDs for a given 3D shape, a final global FV which is L 2 and power-normalized (so it has unit length) is computed with the help of [START_REF]Github -Fisher Vector[END_REF]. Then, for local APPFD with 210 and 15625 dimensions, FVs with 4210 and 312510 dimensions, respectively are returned, which represent a single 3D shape. However through experimental findings, applying linear dimensionality reduction (in our case principal component analysis, PCA) to either of the 4210 or 312510 dimensional FVs remaining 99% of their information reduces them to 162 or 186 respectively, and still yield close matching results.

3. Shape Similarity Measurement: Overall, the L 2 or cosine distance metric between FVs is expected to give a good approximation of the similarity between shapes in the sc20-relief-rc dataset. The cosine metric in ( 6) is adopted, instead. This group submitted two different methods, with three runs each. Since the two methods share the pre-processing steps, we describe both methods in this Section. As the goal of the track is to retrieve 3D models based only on the relief of their surfaces and not the shape of the 3D models, the authors do not exploit the 3D mesh directly but take the 2D screenshots of the 3D models.

cos(FV 1 , FV 2 ) = FV 1 • FV 2 FV 1 FV 2 = n i=1 FV 1i FV 2i n i=1 (FV 1i ) 2 n i=1 (FV 2i ) 2 (6)
Best view among multiple 2D screenshots is selected by searching the maximum inscribed rectangle.

Orientation Histogram (OH)

1. Uprighting and rendering a 3D object: The first step is to upright the 3D object by transforming the object into a new 3D coordinate system so that the object stands vertically across the y-axis for the ease of rendering. That could be done by finding the eigenvector of all the vertices of the object and then choose the normalized version of it (called vector j ) to be the O y axis of the new system. The two remaining axes O x and O z are chosen randomly, satisfying that all the three vectors are unit vectors and pairwise orthogonal. The origin of the new system is the centroid of the object. Moving the camera around the O y axis, many 2D images of the object are sampled. Among these, the image having the most relief patterns is selected. As plain images would have fewer points at which the gradient vectors equal to zero and vice versa, the Sobel Filter [START_REF] Kanopoulos | Design of an image edge detection filter using the sobel operator[END_REF] is used to calculate the gradients of an object's rendered images. The image with the most non-zero gradient vectors is set to be the one representative of this object. An overview of this step is shown in Figure 6.

this step, each 3D object has one representing a 2D square image.

3. Feature extraction: The goal of this step is to represent every image after the second step as feature vectors with the length of N. Such vector is obtained with the method of counting the "gradient histogram" of an image. Specifically, in each image, first, Sobel Filter [START_REF] Kanopoulos | Design of an image edge detection filter using the sobel operator[END_REF] is used to find the gradient vectors of every point and derive their modules and their angles with the O x axis. Second, a histogram with the number of bins being N, ranging from -π/2 to π/2, is computed on the frequency of the calculated angles. Every angle is counted with the weight of its corresponding vector's module instead of one as usual. Furthermore, the weight of a sample is distributed to the two nearest bins with a suitable ratio instead of just one. This histogram could describe the direction and size of the relief on an image. The histogram is then normalized by making the sum across N bins be 1 and translating the histogram so that the highest bin is the first bin (ranged from -π/2 to π/2 + π/n). Every histogram is saved as a 1D-array called the feature vector of the image.

Creation of the Dis-similarity Matrix:

The distance between pairs of 3D objects is calculated on their feature vectors using suitable metrics, such as L1 distance, L2 distance, chi-square distance, cosine-distance, etc.. The original distance matrix is then created by calculating the distance between every pair of vectors. Authors aim to further exploit the visual relationships of an object x and its neighbors with another object y. Therefore, the authors use the Average Query Expansion (AQE) [START_REF] Azad | Query expansion techniques for information retrieval: a survey[END_REF] to modify the original distance matrix (see Figure 7. Let R(x) be the list of the nearest neighbors (in the ascending order) of the object x. The modified distance between object x and object y is defined as follows:

dist AQE (x, y) = α × dist(x, y) + 1 -α k × k i=1 dist(R(x) i , y)
where dist is the original distance, dist AQE is the modified distance matrix, k and α are hyperparameters.

An overview of these last two steps is shown in Figure 8. The runs submitted to the track differ for the number of bins and the metrics for the feature vectors, their settings are described in Section 6.1.

Deep Feature Ensemble (DFE)

This method shares the first two steps (i.e., the model pre-processing steps) with the method described in Section 4.2.1. The third and fourth steps are described in the following. the authors propose using these pre-trained models to extract features of each pattern. Many high-performance models such as ResNet [START_REF] He | Deep residual learning for image recognition[END_REF],

DenseNet [START_REF] Huang | Convolutional networks with dense connectivity[END_REF], VGG [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF], and Efficient-Net [START_REF] Tan | Efficientnet: Rethinking model scaling for convolutional neural networks[END_REF] suit this purpose.

A common approach is to use an extracted feature vector from a pretrained network as the input for classification. However, the output at each layer in a pre-trained model offers different high-level information about the textures in the original input. Therefore, the authors propose to synergize the information extracted from different intermediate layers of different pre-trained networks by assembling feature vectors.

The authors choose intermediate layers instead of the last ones because features extracted in the middle layers would be more appropriate to represent information of the simple patterns on the texture input image. First, the authors pass a square image containing patterns into a pre-trained neural network. Then, the authors take the output tensor of a chosen intermediate layer of that network with the shape of (h, w, channelsize).

After that, the authors pass the tensor through a Global Average Pooling Layer to create a vector with a length of (channelsize,) used as a feature vector. By using Global Average Pooling, the authors pick up all requisite activated features without missing any of them as using Global Max Pooling and make the result more robust to spatial translation in the image. Finally, the authors multiply each feature vector by a parameter (see Section 6.1) and concatenate them into one single final feature vector.

A visual representation of the way authors ensemble the feature vectors from different layers in different models is shown in Figure 9 4. Creation of the Dis-similarity Matrix: After extracting features by the method described above, each object is represented as a feature vector. Such vectors are used to calculate distance between all pairs of objects (with metrics such as cosine similarity, L1 distance, L2 distance, etc.). Besides, the authors combine Average Query Expansion (AQE) [START_REF] Azad | Query expansion techniques for information retrieval: a survey[END_REF] with a view to helping our model to remove noises.

x y R(x) 1 R(x) 2 R(x) k x Direct distance
Modified distance with AQE The pipeline of DFE method is summarized in Figure 10. The authors considered many single pre-trained models.; the pre-trained models considered in the runs submitted to this track are described in Section 6.1.

Deep Patch Metric Learning (DPML) by Leonardo

Gigli, Santiago Velasco-Forero, Beatriz Marcotegui

This method works in two main steps. The first one involves the extraction of patch images from the mesh surfaces, to decorrelate information about relief from the global shape of the mesh. The second step uses these patches to train a Siamese Neural Network [START_REF] Chopra | Learning a similarity metric discriminatively, with application to face verification[END_REF] to learn a distance function between each pair of images.

1. Patch extraction: The goal is to extract images containing only the local texture. Let us define a triangle mesh S ⊂ R 3 , along with a graph G S = (V, E) associated to S, that is the graph whose nodes are the points (x 1 , . . . , x n ) of S. Two nodes are connected if and only if they are vertices of one of the triangles of S at the same time. With this setting, the method starts sampling a subset of points (x 1 , . . . , x n ) ∈ S using Poisson Disk Sampling [START_REF] Yuksel | Sample elimination for generating poisson disk sample sets[END_REF]. Then for each point x i , using the geodesic distance defined over the graph G S , a local neighborhood is defined. In particular, the geodesic distance d between two points x i and x j is the length of the shortest path connecting them. Thus, given r > 0, the local neighborhood is defined as N r (x i ) = {x j ∈ V|d(x i , x j ) ≤ r}. The goal is to project the local neighborhood over a plane and obtain an elevation image. For this reason, only the neighborhood that are as flat as possible are selected. To estimate such a property, the covariance based features are used. Those features are derived from the eigenvalues λ 1 ≤ λ 2 ≤ λ 3 of the neighborhood covariance matrix defined as:

cov(N r (x i )) = 1 |N r (x i )| x∈Nr(xi) (x -x)(x -x) T
and x is the centroid of the neighborhood N r (x i ). The following criteria are used to estimate if the neighborhood is flat enough:

• criterion on planarity: λ2-λ3 λ1 ≥ 0.5,

• criterion on the change of curvature: projected over the element and the tangent plane. In order to obtain a uniform sized patch the method crops them to obtain images of size 231 × 231 (equal to the smallest image extracted with this process). For each patch, crops are computed so that there is the minimum number of void pixels in each image. An overview of this process is reported in Figure 11.

Learning the embedding:

The selected images are used to train a Siamese neural network with the Triplet Loss. The architecture is composed of three CNNs sharing the same weights. In this case the VGG16 [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF], without fully connected layers, is chosen as CNN. The CNNs work in parallel taking as input a triplet of images and generating a comparable feature vectors, as shown in Figure 12. The Triplet Loss minimizes the distance between an anchor and a positive, both of which have the same identity, and maximizes the distance between the anchor and a negative of a different identity, i.e. an image from a different object [START_REF] Chechik | Large scale online learning of image similarity through ranking[END_REF]. The authors submitted two runs for this method. The different parameter settings are reported in Section 6.1.

Signature Quadratic Form Distance and PointNet (Point-5 Net+SQFD) by Ivan Sipiran and Benjamin Bustos

This method consists of computing the distance between two shapes using the Signature Quadratic Form Distance [START_REF] Beecks | Signature quadratic form distances for content-based similarity[END_REF] (SQFD) over descriptions of local patches. The SQFD distance has proven to be effective in large-scale retrieval problems where shapes are represented as sets of features [START_REF] Sipiran | Scalable 3d shape retrieval using local features and the signature quadratic form distance[END_REF]. This approach focuses the attention in the relief (instead of the entire shape) by decomposing the shape into local patches and describing the local patches using a neural network. Subsequently, authors compute aggregated features that keep the local variability of the patches. Finally, the SQFD distance is used to compare two signature collections.

Given the 3D shape M , the feature set F M contains descriptors for the shape. To use the SQFD distance, the feature set F M has to be clustered in a set of disjoint descriptors, such that F M = C 1 ∪ C 2 ∪ . . . C n . A signature is computed for each cluster, defined as S M = {(c M i , w M i ), i = 1, . . . , n}, where

c M i = d∈C i d |Ci| and w M i = |Ci| |F M | .
Each signature contains the average descriptor in the corresponding cluster and a weight that quantifies the representative power of the cluster with respect to the entire feature set. The clustering method uses an intra-cluster threshold (λ), and an inter-cluster threshold (β) and a minimum number of elements per cluster (N m) to perform the grouping. For more details about the local clustering method, see [START_REF] Sipiran | Scalable 3d shape retrieval using local features and the signature quadratic form distance[END_REF].

Given two objects M and N , and their corresponding signatures S M and S N , the SQFD distance is obtained as follows:

SQF D(S M , S N ) = (w M | -w N ) • A sim • (w M | -w N ) T ,
where (w M |w N ) denotes the concatenation of weight vectors. Matrix A sim stores the correlation between averaged descriptors in the signature. The correlation coefficient between two descriptors is defined as:

corr(c i , c j ) = exp(-αd 2 (c i , c j )).
Given an input shape, p local patches of diameter diam are sampled. The first seed vertex is randomly selected from the shape, while the remaining vertices are chosen using a farthest point sampling strategy over geodesic distances.

For each selected vertex, a local patch of diameter diam is computed, using a region growing method. Each local patch is used to obtain a point cloud that represents the patch. In all the submitted runs, a local patch is sampled with 2500 points. For the description of a given point cloud, a PointNet neural network [START_REF] Qi | Pointnet: Deep learning on point sets for 3d classification and segmentation[END_REF] is used. A PointNet network using the ModelNet-10 dataset [START_REF] Wu | 3d shapenets: A deep representation for volumetric shapes[END_REF] is pre-trained for the classification task. After training, the neural network is fed with the point clouds obtained from the previous procedure. The 1024dimensional feature obtained by PointNet is used before the classification of the network.

In the end, each shape in the dataset is represented by p descriptors of 1024 dimensions, which are finally used to compute the signatures and the SQFD.

The other parameters that characterize the runs are the number and diameter of the patches. More details on these settings are reported in Section 6.1.

Smooth-Rugged Normal Angle (SRNA) by Ioannis

Romanelis, Vlassis Fotis, Gerasimos Arvanitis, Konstantinos Moustakas

This approach outlines the geometric texture by using a per-vertex quantity and extracts a representative feature vector which is used to test against every other model in the database.

Consider a smooth planar surface on which a transformation matrix T 1 (v i ) is applied on each of its vertices, "bending" it in such a way, that it forms a smooth cylinder (see Figure 11(b, left)). As is to be expected, T 1 (v i ) is different depending on v i . By applying a pattern on the planar surface, while retaining one-to-one correspondence between the vertices, the surface of Figure 11(b, center) is obtained. The new vertices will have moved by some distance i from their original positions yielding vi = v i + i . This surface can also be morphed into a cylinder, using a transformation T 2 ( vi ). With sufficient vertex density it is possible to state that T 1 (v i ) ≈ T 2 ( vi ).

Since the transformation matrices affect not only the shape's vertices, but also its vertex normals, we can conclude that the angle between the normal vectorsn i (smooth) and ni (with pattern) is preserved on both the plane and the cylinder. This implies that the quantity θ i = ∠(n i , ni ) is not affected by the underlying geometry and depends solely on the pattern. A small error is introduced in cases where the vertex density is not sufficient, but the angles θ i remain a good descriptor of local features.

1. Laplacian Smoothing: The smoothing of the mesh is an iterative procedure which adjusts the position of each vertex based on the position of its neigborhood. The process is described by the following recursive equation:

p n i = p n-1 i + λL(p n-1 i ) L(p n-1 i ) = j∈Ni p n-1 j • w ij j∈Ni w ij -p n-1 i w ij = (p j -p i ) • (p j -p i ) T
Authors set 30 iterations with a smoothing factor λ = 0.7 in order to erase the pattern from the meshes. An example of the final output of this step is shown in Figure 13(a).

Theta Calculation:

The normal vectors of the original and the smooth models are computed as well as the angles between them. As can be seen from the visualization in Figure 13(c), the angles outline the local features with great precision. Thus, it can be concluded that the process can be generalized to more complex shapes than planes and cylinders.

Surface Segmentation:

In order to distinguish areas containing pure textures from those with little to no texture, authors use the magnitude of the saliency s i of the vertices, similarly to [START_REF] Moscoso Thompson | Feature Curve Extraction on Triangle Meshes[END_REF]. In particular, points with small saliency are considered to lie on flat areas. Points with high saliency are either part of a texture or lie in areas with significant geometric deformation. Generally, the latter points are few and far between, so they are not taken into consideration. More precisely, for each vertex v i of the mesh, a patch of the 20 closest geometrical neighbours (including v i ) is created, together with a matrix N i = (n T 1 n T 2 ... n T N ) of their normals.

Afterwards the co-variance matrix is formed as:

R i = N T i N i ∈ R 3×3 ∀ i = 1, • • • , n (7)
Decomposing the covariance matrix leads to

R i = UΛU -1
Finally, the saliency value of each vertex is computed as

s i = 1 λ 2 i1 + λ 2 i2 + λ 2 i3 ∀ i = 1, • • • , n (8)
where λ 1i , λ 2i , λ 3i are the elements of Λ. Finally, k-means is used to cluster the points in the two aforementioned categories: with or without texture. The points that belong to the cluster with the smallest centroid are considered to be part of flat areas (4 centroids were used in total). The points have now been labeled but they are randomly scattered along the surface of the mesh. A density based clustering helps unify them into large textureless areas. A variation of the DBSCAN algorithm [START_REF] Ester | A density-based algorithm for discovering clusters in large spatial databases with noise[END_REF] is used to find and connect neighboring flat points. In this variation the connectivity of the mesh is used to define a one-ring topological neighborhood instead of a geometrical one. An arbitrarily large threshold of points (in this case 1000) per area ensures that only large areas are classified as flat. The segmentation result is visualized in Figure 13(d).

4. Feature Vector Extraction: Finally, a feature vector needs to be computed for each model. The feature vector is a concatenation of 2 histograms H 1 , H 2 (see Table 1) multiplied by the weights w 1 , w 2 defined as follows:

w 1 = number of points in the "flat" areas total number of points [START_REF] Biasotti | Retrieval and classification on textured 3d models[END_REF] w 2 = number of points in the "texture" areas total number of points [START_REF] Biasotti | Retrieval and classification methods for textured 3D models: A comparative study[END_REF] Probability normalization is applied to both histograms to bring the values between models to the same order of magnitude. It is important to note that while flat areas may also contain some minor characteristics of the texture (see Figure 13(d)), they have to be taken into consideration during feature vector extraction. Finally, the feature vector is equal to:

F V = [w 1 • H 1 , w 2 • H 2 ]; (11)
The distances between each model are computed using the Manhattan distance, which makes the calculation very efficient computation-wise. Finally, the distance matrix obtained by comparing all the models is computed. While the method itself will take a significant amount of time to finish, it is highly parallelizable.

5.

Neighborhood angle standard deviation: The method described so far only depends on the set of θ angles of a mesh. As a variation of the previous steps, the authors included some neighborhood information in the feature vector. If two textures display similar angles but differ in their spatial distributions they would otherwise be classified as the same. This extra bit of information can help distinguish between them and improve the accuracy of the method. However, computing a complex, rotation-invariant spatial descriptor is by no means an easy task, so authors overcome this problem by using the standard deviation of angles in small topological neighborhoods (1 rings). If the normals of that area have a common orientation the value will be small, whereas irregular areas will display much larger values. Two more histograms H 3 , H 4 (see Table 1) with weights w 3 = w 1 , w 4 = w 2 for the flat and texture areas are added to produce the final feature vector. A distance matrix is finally computed as described in the step 4. Being based on a local patch analysis, this approach is intrinsically de-correlated from the global shape of the surface. These four steps are described down below.

1. Patches extraction from the objects surface: Up to six points are selected on the surface, obtained intersecting the mesh surface with the three eigenvectors of the covariance matrix centered at the center of mass of the object (see Figure 14(a)). This process, depending on the object shape, can detect 4 to 6 points on the mesh surface. Around each of these points a region is sampled (called patch), selecting only the vertices of the mesh within a given geodesical radius. Among those, the three patches with the largest ratio e2 e3 , where e 2 and e 3 are respectively the second and third eigenvalues associated to the eigenvectors of the covariance matrix of the patch. An example of the final outcome of this step is shown in Figure 14(b).

Regularization of the patches tessellation:

The three patches are then resampled by projection (PR) [START_REF] Werghi | Representing 3D texture on mesh manifolds for retrieval and recognition applications[END_REF]. At first, (a) PCA is used to determine the two main axes of the sample that define a 2D plane of projection; (b) a uniform 2D grid of points is generated, which is then triangulated using the Delaunay algorithm; (c) the points on the grid are projected back to the mesh surface using interpolation, while keeping their triangulation intact.

3. Descriptor computation: Using the ORF structure (see Figure 15), it is possible to compute LBP patterns and perform convolution-like operations locally and directly on the mesh. In particular, the authors use the ORF (as in [START_REF] Tortorici | Extending LBP and Convolution-Like Operations on the Mesh[END_REF]) as a tool to operate convolution on the mesh manifold, and represent them as Convolution Binary Pattern [START_REF] Tortorici | Extending LBP and Convolution-Like Operations on the Mesh[END_REF]. Leveraging on the ordered structure of the ORF, the authors redefine the convolution operator between a mesh M and a filter F in polar coordinates as follows:

(M * F) = r θ m r,θ • f r,θ , (12)
where m r,θ , and f r,θ are, respectively, a scalar function computed on the mesh and the filter values, both at radius r and angle θ. Subsequently, the response to the filter is used as input for the MeshLBP descriptor, thus obtaining a convolution binary pattern. Finally, the local descriptors are accumulated on a histogram computed over the entire patch surface.

Dissimilarity matrix computation:

To compare two models, authors compare all their patches together in a pair-wise manner using Bhattacharya distance. The dissimilarity between the two models is obtained by accumulating such distances.

Three runs have been submitted, changing the descriptor computed directly on the mesh manifold. The descriptors used in the runs are listed in Section 6.1

Correspondence matching based on kd-tree Fast Library for Approximate Nearest Neighbors (kd-tree FLANN) by Yoko Arteaga and Ramammorthy Luxman

This method is based on using the kd-tree Fast Library for Approximate Nearest Neighbors (FLANN [START_REF] Friedman | An algorithm for finding best matches in logarithmic expected time[END_REF]) correspondence matching to match the query of the other objects in the database. FLANN stands for Fast Library for Approximate Nearest Neighbors and it is a method used for evaluating the correspondence between two objects, by finding the distance between the extracted features. For this context, the kd-tree radius is set to be 0.15. This method works essentially in two steps: a pre-processing step that only extracts the surface information and the application of the kd-tree FLANN method on such information. For each patch, the authors detect points that are suitable for effective descrip- tion and matching, using uniform sampling keypoints detection method. Then, the local feature descriptor SHOT352 [START_REF] Tombari | Unique signatures of histograms for local surface description[END_REF] is applied for each of the detected keypoints. The obtained features are then matched using KD Tree FLANN correspondence matching method. Notice that, instead of analysing the global shape, this method matches the features from the representative patches in order to de-correlate relief from shape. The criteria for choosing the patches is that the curvature must be minimum so it belongs to a flatter section of the object.

The pre-processing step is done to speed up the retrieval and to ensure only texture information is used. First, 400 points within the object are chosen at random. From each of the 400 points, their 400 nearest neighbors are found.

Then, the mean curvature and the normals of each of the 400 patches are found.

The final patch used as the representative of the object is selected as follows: it must be the one with the lower mean curvature and the greater mean variance of the normals. That is because, if the mean curvature is low, the patch belongs to a flat area of the object with the least curvature from its global shape, while the highest mean variance of normals implies that this area has the highest distribution of peaks and valleys in the sample, i.e. more texture. An example of the extracted representative is shown in Figure 16.

The kd-tree FLANN method is used to match the model representatives.

Each entry of the dissimilarity matrix is equal to the inverse of the number of matches obtained as results from the kd-tree FLANN matching. If no matches are found, the value is set to 1.

Evaluation measures

We selected different evaluation measures for this SHREC track. The combination of these measures gives us a global view of the various methods, highlighting various properties (goodness of the method per model, class, overall based on multiple criteria). These measures are well known performance measures in information retrieval [START_REF] Rijsbergen | Information Retrieval[END_REF][START_REF] Baeza-Yates | Modern Information Retrieval[END_REF] and many of them are already used in related SHREC Normalized Discounted Cumulated Gain (nDCG). This measure is based on the assumption that relevant items are more useful if appearing earlier in the list of the retrieved items. The nDCG is based on the graded relevance of a result with respect to the query. Then, the value is normalized with respect to the ideal outcome of that query.

Average precision-recall curves, mAP and e-Measure (e). Precision is the fraction of retrieved items that are relevant to the query. Recall is the fraction of the items relevant to the query that are successfully retrieved. By plotting the precision value with respect to the recall value we obtain the so called recall vs. precision curve: the larger the area below such a curve, the better. In particular, the precision-recall curve of an ideal retrieval system would result in a constant curve equal to 1. For each query, we have a precision-recall (PR) curve. In our context, results are evaluated on the mean of all the PR curves. The mean Average Precision (mAP) corresponds to the area between the horizontal axis and the average precision-recall curve and ranges from 0 to 1. The higher, the better. The e-Measure (e) derives from the precision and recall for a fixed number of retrieved results (32 in our settings), [START_REF] Rijsbergen | Information Retrieval[END_REF]. For every query, the e-Measure considers the first 32 retrieved items and is defined as e = 1

1

P + 1 R
, where P and R represent the precision and recall values over those results, respectively.

Confusion matrix. To each run we associate also a confusion matrix CM , that is, a square matrix whose order is equal to the number of classes in the dataset. For a row i in CM , the element CM (i, i) gives the number of items which have been correctly classified as elements of the class i. The elements CM (i, j), with j = i, count the items of the class i which have been misclassified and j corresponds to the class in which they were wrongly classified. An ideal classification system should be a diagonal matrix. The sum j CM (i, j) equals the number of items in the class i. Generally, the confusion matrix is non-symmetric. [START_REF] He | Deep residual learning for image recognition[END_REF] multiplied by a scalar value that goes from 1 to the number of classes in the dataset [START_REF] Biasotti | Retrieval of surfaces with similar relief patterns: Shrec'17 track[END_REF]. The higher the curve is the better. A quick comparison between the methods based on the ROC curves can be derived also by the AUC value (namely the area under curve value), which is the measure of the area under the ROC curve. The higher this value is, the better. Anyway, note that an AUC value of 0.5 means that the corresponding method is not able to classify the models at all. In this work, we consider the mean of all ROC curves.

Description and evaluation of the submitted runs

In this Section, the settings of the runs submitted for evaluation are detailed.

Their outcome is presented with respect to the performance measures described in Section 5.

Run settings

In the following, the parameter settings of the runs submitted are listed. If an author sent a single run, the setting are those described in Section 4. The same happens if the runs of a single method differ more than "just" different parameters.

• APPFD-FK : two runs (APPFD-FK(run1) and APPFD-FK(run2) respec- Values goes from 0 (red), to 1 (green). The higher the value is, the better the method performs. than 0.7 and the nDCG greater than 0.8. Also, note that 2 methods have the ST score above 0.99 which, having all the classes 20 models each, means that the models with the same 3D texture as a query are generally found within the first 39 retrieved models, with very few exceptions.

For a better visual comparison of the methods, only the Confusion Matrix and Tier Image of the best run of each method are reported in Figure 18 and As also reflected by the area under the ROC curve, methods with AUC greater than 0.97 provide a better classification than other methods. For completeness, the PR plots and the ROC curves of the all the runs submitted are listed in the Appendix. This more complete overview of the runs highlights that the performances of a method show the same trend for the different runs, with small qualitative variations between the different parameter choices.

Figure 17: Overview of Precision-Recall plots of the best run for each method.

Discussions and concluding remarks

Overall, the best performances are obtained by the DFE method, which uses a pre-trained neural network. We observe that the NN, FT and ST scores for the methods based on transfer learning do not change significantly. This fact suggests that, if they have success, these methods have a larger capability of ranking the models that contains a texture similar to the query at the beginning of the list of the items retrieved, while the other methods drop around 0.3 from NN to FT. However, also methods that do not use learning techniques perform well (like the meshLBP, OH and SRNA). We notice that these methods are all based on feature vectors. Some methods share some background, for instance, the meshLBP-so run and the OH methods use of the Sobel Filter.

However, among the three meshLBP-based runs submitted to this track, the best performances are reached by the meshLBP run that is based on convolution-like operations extended to a triangle mesh.

A common characteristic of most of methods is the sampling of one or more representative patches as a pre-processing step. It consists of a single patch (like in the case of the DFE, OH, DPML, kd-tree FLANN runs) or multiple ones (like in the APPFD-FK, PointNet+SQDF, SRNA, meshLBP runs). In general, the selection of a single patch seems to lead to good results with the exception of the SRNA and meshLBP methods, which compute a more statistical approach on the representative patches.

Methods that convert the model into point clouds (APPFD-FK) or that are based on CNNs trained on point clouds (PointNet) seem to be sub-optimal for this task. Probably these methods lose information on local details (for instance, the sampling process in the APPFD-FK focuses on the representation of the global geometry) and do not capture the subtle geometry and structure variations of local patterns and reliefs. On a similar note, the authors of the kd-tree FLANN method suggest that the performance of their methods will probably be improved by considering a smaller representative patch. With the current size of the patch, the global geometry of the model is still kept in consideration and it biases the results. This fact highlights the importance of analysing a surface with reliefs by local approaches (but that are robust to noise).

From the Confusion Matrices we observe there is not a class (i.e. a realistic geometric relief that corresponds to a real texture) that is more complex to deal with at all. On the other hand, Tier Images highlight that some methods (DFE and meshLBP in particular) tend to confuse class 10 (straight horizontal lines with some thing double lines) and class 2 (just straight lines) or class 4 (bricks).

Indeed, all these classes have a set of horizontal and parallel lines which lead to some uncertainty in the classification (especially classes 10 and 2).

In conclusion, we have presented the results of the SHREC'20 contest track on "Retrieval of surface patches with similar geometric reliefs". The number of runs (twenty) and methods (eight) is significantly numerous and show the increasing efforts of the community in the effective characterization of all the aspects of a surface. The runs and the methods submitted to this track present a satisfactory variety, in terms of the diversification of the approach followed (feature-based and learning-based methods) and the type of description chosen (global vs local descriptions). Several methods use a transfer learning approach based on pre-trained, image-based neural networks. For instance, the best performances are obtained by the DFE method, which follows such a strategy.

With respect to the methods submitted to similar, previous SHREC contests, we can observe the rise of the machine learning based approaches specifically designed for and/or adapted to this track task. A future direction of investigation is to deepen the analysis of the performances of methods based on learning. To this end, it will be necessary to create larger data collections, opportunely equipped with a training set of models, even if the application of reliefs to a surface is not trivial. Indeed, at the moment it requires some manual cleaning of the models, in particular in correspondence of high curvature features like handles.

The way models are analysed by most methods, that is a local conversion of the surface into a kind of texture image, helps in removing the influence of the underlying surface from the reliefs. Still, corresponding to models that can be manufactured, the surfaces of the models proposed in this benchmark can be locally projected in a plane and therefore in an image. Further research is needed to deal with more challenging models and how these methods work on models with a more complex geometry and/or how they could be patched to deal with them.

Overall, this contest has received a good number of satisfactory solutions that highlight the progress of recent years in the field of geometric pattern retrieval. As a future research direction we envisage an increase of interest in the more complex task of pattern recognition on surfaces, i.e., addressing a problem similar to the challenge proposed in [START_REF] Biasotti | Recognition of Geometric Patterns Over 3D Models[END_REF], where the models were only partially covered by none, one, or many patterns. The goal of that track was to identify, from a set of sample patterns, if and where the same pattern was located on each model. At the time of that track [START_REF] Biasotti | Recognition of Geometric Patterns Over 3D Models[END_REF] there were no satisfactory solutions. In the near future, in the light of the progress achieved in the pattern retrieval problem and the progress made in the field of transfer learning, it would become interesting to understand what can be exploited also in the field of pattern recognition, too.

Figure 3 :

 3 Figure 3: (Left): the 20 base models on which the reliefs are applied. (Center): the 11 transformed textures used as height-fields on the base models (the brighter the color, the higher is the value of the field in that point). (Right): a sample of the final models of the dataset of the contest.

Figure 4 :

 4 Figure 4: Local Surface Patch (LSP), Pi with pairwise points (pi, pj) as part of asurflet-pair relation for (pi, ni) and (pj, nj), with pi being the origin. θ and φ are the angles of vectors projection about the origin, pi. θ is the projection angle from vector pi -pj to vector pi -pc while φ is the projection angle from vector pi -pj to vector pi -l . The LSP centre is given by pc, keypoint is given as p k i where i = 2. Finally, l is the vector position of p k i -pc.

Figure 5 :

 5 Figure 5: Overview of the APPFD-FK framework, which computes a global Fisher Vector (FV) for each 3D shape.

3 .Figure 6 :

 36 Figure 6: Overview of the pre-processing steps for the OH and DFE methods.

Figure 7 :

 7 Figure 7: Overview of the Average Query Expansion used in OH and DFE.

Figure 8 :Figure 9 :

 89 Figure 8: Overview of the third and fourth steps of the OH method.

  λ3 λ1+λ2+λ3 ≤ 0.03.

Figure 10 :

 10 Figure 10: Overview of the third and fourth steps of the Deep Feature Ensemble method (DFE).

Figure 11 :

 11 Figure 11: Pipeline of the first step of the DPML method.

Figure 12 :

 12 Figure 12: Overview of the network of the DPML. Such network consists of a batch input layer and a deep CNN which results in the image embedding by using a triplet loss during training.

  . of the angles in the "flat" areas [0, 0.5] H4 st. dev. of the angles in the "texture" areas [0, 0.5]

Table 1 :Figure 13 :

 113 Figure 13: Overview of the steps of the SRNA method. (a): on the left, the original models with texture, while on the right the smoothed models without texture. (b): On the left, a smooth cylinder; on the center, a plane with texture; on the right, a cylinder with texture. (c): Theta angles visualization on different models. (d): Thetas (on the left) and segmentation (on the right) of the same model.

4. 6

 6 Mesh Local Binary Pattern (meshLBP*), meshLBP-Sobel and meshLBP-Sharpen by Claudio Tortorici, Naoufel Werghi, Ahmad Shaker Obeid and Stefano Berretti This method comprises four stages: (i) the extraction of patches from the object surface, (ii) the regularization of the patches tessellation, (iii) the computation of the descriptors, and finally, (iv) the generation of the dissimilarity matrix.

Figure 14 :

 14 Figure 14: Patches extraction for the meshLBP-* method. (a) Intersection between the eigenvectors of the covariance matrix with the object surface to select the candidate points for the patch extraction. (b) A sample model is shown on the left, and the three extracted patches are reported on the right.

Figure 15 :

 15 Figure 15: ORF ordered structure, showing its construction procedure, the ordered rings and its extension to multiple rings.

Figure 16 :

 16 Figure 16: On the left, the final point cloud obtained after the conversion used for the kd-tree FLANN method. On the center, the representative patch (in blue) no the model. On the right, final extracted patch.

  Tier images. Similar to the confusion matrix, the tier image visualizes the matches of the NN, FT and ST. The value of the element T (i, j) is: black if j is the NN of i, red if j is among the (|C| -1) top matches (FT) and blue if j is among the 2(|C| -1) top matches (ST), where |C| is the number of elements of the class C. The models of a class are grouped along each axis so it is easier to interpret. With this configuration, the optimal tier image clusters the black/red square pixels on the diagonal. Receiver Operating Characteristic (ROC) curve and AUC value. ROC curves are largely used to evaluate the classification performance of a method and are suitable to assess retrieval issues, too. The ROC curve shows the ratio between False Positive Rate and True Positive Rate for each model at different classification thresholds. In our scenario, the classification thresholds are the number of models in each class

  tively) use a mono-dimensional, [0, 1] normalized histogram with 35 bins for each one of the feature-dimensions in the APPF; these histograms are concatenated to yield a final 6 x 35 = 210-dimensional local descriptor

Figure 19

 19 Figure 19 respectively. For completeness, the Confusion matrices and the Tier images of the all the runs submitted are listed in the Appendix. Precision-Recall plots of the best run for each method are shown in Figure17. Similarly, only the ROC curves of the best runs are shown in Figure20. As also reflected by the area under the ROC curve, methods with AUC greater than 0.97 provide a better classification than other methods. For completeness, the PR plots and the ROC curves of the all the runs submitted are listed in the Appendix. This more complete overview of the runs highlights that the performances of a method show the same trend for the different runs, with small qualitative variations between the different parameter choices.

Figure 18 :

 18 Figure18: Overview of the confusion matrices of the best run for all the methods.

Figure 19 :

 19 Figure 19: Overview of the tier images of the best run for all the methods.

Figure 20 :

 20 Figure 20: Overview of ROC curves of the best run for each method.

Cropped patterns Images' gradient vectors Histograms Feature vectors Dissimilarity matrix Calculate histogram Calculate gradient vectors Calculate distances with query expansion Angle of Gradient Vectors

		Represent
		feature vectors
	-𝝅	𝝅
	𝟐	𝟐

Table 2 :

 2 Nearest Neighborhood, First Tier, Second Tier, mAP, nDGC, emeasure and AUC value of all the submitter runs.

		NN	FT	ST mAP nDCG e	AUC
	APPFD-FK(run1)	0.186 0.204 0.332 0.235 0.523 0.211 0.672
	APPFD-FK(run2)	0.132 0.186 0.299 0.212 0.497 0.192 0.632
	APPFD-FK(run3)	0.186 0.192 0.318 0.228 0.507 0.203 0.682
	OH(run1)	0.791 0.406 0.567 0.470 0.737 0.377 0.817
	OH(run2)	0.750 0.374 0.517 0.418 0.709 0.341 0.779
	OH(run3)	0.714 0.405 0.575 0.469 0.732 0.382 0.818
	DFE(run1)	0.982 0.920 1.000 0.930 0.974 0.715 0.987
	DFE(run2)	0.982 0.913 1.000 0.926 0.973 0.714 0.986
	DFE(run3)	0.982 0.865 1.000 0.896 0.963 0.693 0.980
	DPML(run1)	0.900 0.836 0.990 0.868 0.941 0.686 0.974
	DPML(run2)	0.982 0.887 0.992 0.912 0.968 0.690 0.978
	PointNet+SQFD(run1) 0.095 0.095 0.184 0.168 0.440 0.113 0.569
	PointNet+SQFD(run2) 0.077 0.099 0.203 0.171 0.442 0.122 0.582
	PointNet+SQFD(run3) 0.173 0.119 0.225 0.190 0.470 0.137 0.605
	SRNA(run1)	0.905 0.493 0.670 0.548 0.802 0.447 0.869
	SRNA(run2)	0.923 0.494 0.683 0.563 0.811 0.453 0.882
	meshLBP-so	0.909 0.631 0.764 0.687 0.872 0.516 0.870
	meshLBP-sh	0.895 0.601 0.759 0.656 0.853 0.522 0.875
	meshLBP	0.905 0.671 0.832 0.726 0.884 0.570 0.909
	kd-tree FLANN	0.686 0.312 0.424 0.359 0.656 0.283 0.690

Finding the largest inscribed square: In order to remove the global shape of the object and focus on the local reliefs, the largest inscribed square of the object on the 2D image is extracted, which means selecting the largest region of only relief patterns. This is done by solving the problem of finding the largest square with no white points inside of it (because white points are background). The latter problem is resolved by using a simple binary search algorithm with the complexity of N 2 * log N (N is the greater value between the width and the height of an image). After

Acknowledgements

The authors thank the 3DOR 2020 Workshop and Program Chairs for helping us in the organization of our contest despite the current COVID-19 pandemic.

We also thank the anonymous reviewers for providing constructive comments on earlier drafts of the manuscript, which helped us to improve and clarify this work. This study was partially supported by the CNR-IMATI projects DIT.AD004.100 and DIT.AD021.080.001. Research for the team from University of Science, Ho Chi Minh city, Vietnam is supported by Vingroup Innovation Foundation (VINIF) in project code VINIF.2019.DA19. The team Y. Arteaga and R. Luxman research is funded by the Horizon 2020 programme of the European Union Grant #813789. The work of Ivan Sipiran has been supported by Proyecto de Mejoramiento y Ampliación de los Servicios del Sistema Nacional de Ciencia Tecnología e Innovación Tecnológica(Banco Mundial, Concytec), Nro. grant 062-2018-FONDECYT-BM-IADT-AV. The work of Benjamin Bustos is funded by the Millennium Institute Foundational Research on Data (IMFD).

for each LSP or key-point. In APPFD-FK(run3), the six-dimensional features in the APPF are discretized into a multi-dimensional histogram with 5 bins for each feature-dimension, the histogram is then flattened and normalized to give a 5 6 = 15625-dimensional local descriptor for each LSP or key-point. -OH(run3): Number of bins: N = 128 -metric: modified L1 norm.

• DFE : The runs differ in the models used for transfer learning. The models used in each run are listed in the following.

- The authors ensemble the three models above with the ratio of weights: model1 : model2 : model3 = 2 : 1 : 1, using the Cosine distance as metric.

-DFE(run2): Same model settings as DFE(run1), using Euclidean distance as metric.

- The authors ensemble the three models above with the ratio of weights: model1 : model2 : model3 = 2 : 1 : 2, using the Cosine distance as metric.

• DPML: The runs differ in the way the patches generated are pre-processed and in the use of data augmentation.

cropping to obtain images of size 231 × 231. No data augmentation has been used in this run.

-DPML(run2): Same model as in Run 1. This time instead of cropping patches, authors padded them with zero-values to the size of the biggest patch that is 836 × 836. Furthermore, during training data augmentation was applied rotating input image with random angles and also flipping vertically and horizontally.

• PointNet+SQFD: All the runs used the following clustering parameters: -PointNet+SQFD(run3): number of patches p = 500 of diameter diam = 0.025 of the diagonal of the bounding box of the shape.

In all the submitted runs, α = 0.9 and d is the L 2 distance.

• SRNA: Two runs have been proposed for this method:

-SRNA(run1); this run corresponds to the outcome of the first four steps of the method described in Section 4.5;

-SRNA(run2): this run corresponds to the variation of the SRNA method that includes also neighbour information as described in the step five of Section 4.5.

• meshLBP-* : the descriptors used in each of the three runs submitted are:

-(meshLBP-so): Sobel filter;

-(meshLBP-sh): sharpen filter;

-(meshLBP): MeshLBP.

Results

Table 2 summarizes the performances of all the twenty runs submitted for evaluation, namely each column of the