Florent Langenfeld
email: florent.langenfeld@lecnam.net

Tunde Aderinwale

Charles Christoffer

Woong-Hee Shin

Genki Terashi
email: gterashi@purdue.edu

Xiao Wang

Daisuke Kihara
email: dkihara@purdue.edu

Halim Benhabiles
email: halim.benhabiles@junia.com

Karim Hammoudi
email: karim.hammoudi@uha.fr

Adnane Cabani
email: adnane.cabani@esigelec.fr

Feryal Windal
email: feryal.windal@junia.com

Mahmoud Melkemi
email: mahmoud.melkemi@uha.fr

Ekpo Otu

Reyer Zwiggelaar

David Hunter

Yonghuai Liu

Léa Sirugue

Huu-Nghia H Nguyen

Tuan-Duy H Nguyen

Vinh-Thuyen Nguyen-Truong

Danh Le

Hai-Dang Nguyen

Minh-Triet Tran

Matthieu Montès
email: matthieu.montes@cnam.fr

Surface-based protein domains retrieval methods from a SHREC2021 challenge

Keywords: SHREC2021, Proteins surface, 2000 MSC: 92-08

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

A B S T R A C T

Proteins are essential to nearly all cellular mechanism and the effectors of the cells activities. As such, they often interact through their surface with other proteins or other cellular ligands such as ions or organic molecules. The evolution generates plenty of different proteins, with unique abilities, but also proteins with related functions hence similar 3D surface properties (shape, physico-chemical properties, . . . ). The protein surfaces are therefore of primary importance for their activity. In the present work, we assess the ability of different methods to detect such similarities based on the geometry of the protein surfaces (described as 3D meshes), using either their shape only, or their shape and the electrostatic potential (a biologically relevant property of proteins surface). Five different groups participated in this contest using the shape-only dataset, and one group extended its pre-existing method to handle the electrostatic potential. Our comparative study reveals both the ability of the methods to detect related proteins and their difficulties to distinguish between highly related proteins. Our study allows also to analyze the putative influence of electrostatic information in addition to the one of protein shapes alone. Finally, the discussion permits to expose the results with respect to ones obtained in the previous contests for the extended method. The source codes of each presented method have been made available online.

© 2021 Elsevier B.V. All rights reserved.

Introduction

Proteins are key molecular effectors at the cellular level.

Proteins are linear assemblies of amino-acids that fold in specific, energy-driven 3D structures [1, 2] linked to their activity.

Identifying similarities within protein structures is therefore of tremendous importance in various fields, from biochemistry to drug design. Numerous methods have been dedicated to structural similarity search of proteins in structural bioinformatics, that mainly rely on the analysis of the 3D point clouds defined by the 3D coordinates of their individual atoms [START_REF] Holm | Mapping the protein universe[END_REF]4,5,6,[START_REF] Mariani | lDDT: A lo-49 cal superposition-free score for comparing protein structures and mod-50 els using distance difference tests[END_REF].

These methods are mostly based on the conserved core structure of proteins, and therefore, may be inefficient to detect proteins sharing similar surface. The protein surface is a higher-level description of the protein structure that abstracts the underlying protein sequence, structure and fold into a continuous shape with geometric and chemical features that fingerprint its interactions with the other molecules of its environment (solvent, ligands, proteins, nucleic acids, . . . ) [8]. Methods able to detect protein surficial similarities are then of major importance.

Only a limited number of methods have been proposed so far:

• This method is able to use surficial physico-chemical properties like the electrostatic potential or the hydrophobicity [START_REF] Sael | Rapid comparison of properties on protein surface[END_REF].

• MaSIF (molecular surface interaction fingerprinting) is a geometric deep learning framework that allows to fingerprint biomolecular surfaces [START_REF] Gainza | Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning[END_REF]. Both geometric and chemical features are extracted and embedded into numerical vectors which is subsequently processed in an application-dependent manner.

Fig. 1. Structural similarity between the protein structure queries. The TM-score (in the (0, 1] range) measures the topological similarity between two protein structures: the higher the TM-score, the more similar the two structures. Scores below 0.17 correspond to unrelated proteins, while those above 0.5 usually indicate two structures having the same fold [START_REF] Zhang | Scoring function for automated assessment of protein structure template quality[END_REF].

coordinates, it uses the density distribution. A 3D Zernike 41 moment normalization procedure is applied to orient the 42 density volumes to be compared, allowing for for fast re-43 trieval of proteins/protein assemblies.

44

The aim of this track is therefore to assess the performance of 45 currently available methods and to stimulate the development of 46 novel methods. To this end, the dataset encompasses (1) A va-47 riety of protein domains, with some of them closely related, to 48 query the dataset. (2) A dataset of experimental structures that 49 contain one or more domains. (3) A few protein shapes cor-50 responding to protein that contains two of the query domains. 51 (4) Two versions of the same dataset, one made of the protein 52 shapes only, and the other with an additional physico-chemical 53 property, the electrostatic potential, encoded along the shapes. [START_REF] Fortune | Parallelism in random access machines[END_REF] We selected this surficial physico-chemical feature as it is the 55 main driving force in many biological recognition processes, 56 such protein-ligand and protein-protein interactions [START_REF] Zhang | On the role of electrostatics in protein-protein interactions[END_REF]. In the 57 present work, we detail the dataset proposed by the challenge 58 organizers to the participants and how it differs from the previ-59 ously proposed datasets in Section 2. In Section 3, we describe 60 the 5 methods submitted to the contest. The evaluation met-61 rics are briefly introduced in Section 4, and the performance of 62 the methods is presented in Section 5. Finally, we discuss the 63 outcomes of the different submitted methods in Section 6. The SHREC'21 protein dataset is based on the Pfam 33.1 67 database [START_REF] Mistry | Pfam: The protein families database in 2021[END_REF]. Basically, this database classifies protein se-68 quences into domains and families, that can be grouped into 69 clans whenever they are evolutionarily related. Protein do-70 mains of structures from the Protein Data Bank (PDB [START_REF] Berman | The Protein Data Bank[END_REF][START_REF] Berman | Announcing the worldwide Protein Data Bank[END_REF]) 71

33

For all structures (queries and dataset structures), we re-34 moved all hetero-atoms, and unwanted chains. The resulting PDB structures were then protonated using pdb2pqr [START_REF] Dolinsky | PDB2PQR: An automated pipeline for the setup of Poisson-Boltzmann electrostatics calculations[END_REF], using propka [START_REF] Søndergaard | Improved treatment of ligands and coupling effects in empirical calculation and rationalization of pKa values[END_REF][START_REF] Olsson | PROPKA3: Consistent treatment of internal and surface residues in empirical pKa predictions[END_REF] to compute the pKa values of the ionizable groups at pH=7.2. The solvent-excluded surface of all protonated structures were computed using the default parameters of EDTSurf [START_REF] Xu | Generating triangulated macromolecular surfaces by Euclidean distance transform[END_REF][START_REF] Xu | Protein depth calculation and the use for improving accuracy of protein fold recognition[END_REF] except that inner cavities were discarded. We then computed the electrostatics using APBS suite [START_REF] Baker | Electrostatics of nanosystems: Application to microtubules and the ribosome[END_REF], and used the multivalue to compute the electrostatic potential at the mesh vertices locations. Two datasets were then assembled, one with only the protein surface shapes (shape-only dataset) and one combining the protein surface shapes and electrostatics values (shape+electrostatic dataset). Similarly, two sets of query surfaces were produced (shape-only and shape+electrostatic). Each dataset (shape-only and shape+electrostatic) includes 554 molecular surfaces which were made available to the track participants, along with the 2 sets of 10 queries, on the track webpage (http://shrec2021.drugdesign.fr).

Regarding the dataset, it is important to note that SH3 and SH3_2 domains were annotated as similar according to HHsearch (a tool commonly used to detect homologous proteins [27]), as well as the PDZ, PDZ_6 and m50 domains. We present in Figure 1 the TM-scores matrix for all queries of the dataset. A TM-score above 0.5 indicates that the two structures are likely to share the same topology, while unrelated structures are usually associated to TM-scores below 0.17 [28]. SH3 and SH3_2 query structures show a TM-score of 0.84, while, PDZ and PDZ_6 query structures show a TM-score of 0.79. Surprisingly, the m50 query structure only has a TM-score of 0.28 and 0.32 with PDZ and PDZ_6 structures, respectively. A visual inspection of these structures confirmed that the peptidase M50 is topologically different from both the PDZ and PDZ_6 domains: while the peptidase M50 is mainly α-helical, PDZ and PDZ_6 are mixed αβ proteins. Overall, most query structures present an intermediate topological similarity with all other queries, as evidenced by the fact that all TM-scores range from 0.19 to 0.47, except for the aforementioned pairs of classes (SH3 / SH3_2 and PDZ / PDZ_6).

Finally, as the dataset encompasses multi-domain proteins, 22 dataset proteins display two of the query domains. Namely, 9 proteins of the dataset encompass both a SH2 and a SH3 domains, 6 proteins encompass both a SH2 and a Stat-binding domains, 4 proteins encompass both a bromodomain and a PHDfinger domain, 2 proteins encompass a PDZ_6 and a peptidase family M50 domains, and one structure encompass a PDZ and a peptidase m50 domains. The final structure of the dataset and the size of each class is summarized in Figure 2.

Comparison to previous SHREC datasets on proteins

Compared to previous SHREC datasets dealing with protein structures or surfaces [START_REF] Temerinac-Ott | SHREC'07 -Protein 12 Retrieval Challenge[END_REF]30,31,32,33,34], the SHREC'21 protein track dataset is characterized by three main aspects :

(1) The presence of two datasets representing the same set of proteins, one shape-only and one shape+electrostatic dataset.

(2) The close evolutionary relationship of some of the query domains, further characterized by a similar topology (Figure 1).

(3) The intermediate similarity of the domains topologies (Figure 1). And (4) The use of individual domains to query a dataset of single-as well as multi-domain proteins shapes.

The main novelty of the SHREC'21 track is arguably the 1 availability of protein surfaces with electrostatic values, which 2 has been shown to improve the retrieval performance of protein 3 surfaces [START_REF] Gainza | Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning[END_REF]35]. This additional feature might therefore allow 4 to better distinguish structurally related proteins based on their 5 surficial properties and improve the methods' performance. In our contest, the participants were asked, given each of the uses the ReLu activation function in all except the output of the fully connected layer where the sigmoid activation function is used to output the probability that the two proteins belong to the same protein-or species-level in the SCOPe dataset classification [37,[START_REF] Chandonia | SCOPe: Classification of 52 large macromolecular structures in the structural classification of pro-53 teins-extended database[END_REF]. The second framework (the end-to-end model) is similar to the first except the feature comparator layer is removed and the output of the encoder is directly connected to the fully connected layer.

The network is trained on the latest SCOPe dataset of 259,385 protein structures. 2,500 protein structures were set aside for network validation. Proteins in Class I (Artifacts) were removed. Each of the two network frameworks is trained with two datasets. The first dataset is 3DZDs of the surface shape of proteins and the second one is feature vectors that concatenate 3DZD of shape and 3DZD of the electrostatic properties.

Our group examined the performance of the networks on the validation dataset to determine which models to use. For the shape-only dataset, we submitted predictions generated by the Extractor model. For the shape+electrostatic dataset, we submitted the average predictions between the Extractor model and the end-to-end model.

For each protein in the provided dataset, our group performs a pre-processing step as follows: (1) The PLY mesh data file is converted to a volumetric skin representation (Situs file) where points within 1.7 grid intervals are assigned with values that are interpolated from the mesh [9]. For the electrostatic features, the interpolated values are the potentials at the mesh vertices. For the shape feature, a constant of 1 is assigned to grids that overlap with the surface. (2) The resulting Situs file is fed into the EM-Surfer pipeline [START_REF] Esquivel-Rodríguez | Navigating 3D electron microscopy maps with EM-SURFER[END_REF] to compute 3DZD.

Runtimes and computational cost

It takes approximately 12-13 min to pre-process each file. Generating 3DZD took ~8.00 s on average for each protein on an Intel ® Xeon ® CPU E5-2630 0 @ 2.30GHz. The training of each models took 12 h. Dissimilarity prediction between two proteins using the trained model took ~0.22 s on average on a Nvidia ® Titan X GPU. The averaging of the two matrices was almost instant and negligible. The code is available at the following url: https://github.com/kiharalab/ shrec_2021_shape_retrieval.

ProteinNet: Deep learning based protein characterization from 3D point clouds (ProteinNet) by Halim Benhabiles, Karim Hammoudi, Adnane Cabani, Feryal Windal & Mahmoud Melkemi

Our group proposes a deep learning approach to calculate a protein descriptor from its 3D point cloud. To this end, we developed a variant of PointNet [40] which is a point cloud deep architecture dedicated for 3D classification and segmentation. We adapted this architecture in order to learn an affine transformation matrix that allows to align the coordinates of the input 3D protein point cloud into a canonical representation. The new representation maintains interesting properties demonstrated in [40], including invariance to rigid geometric transformations as well as point order permutations. The Pro-teinNet deep architecture is illustrated in Figure 3. More specifically, the architecture is based on a PT-Net module (Protein The PT-Net module is trained to predict an affine transforma-3 tion matrix M that is constrained to be close to an orthogonal 4 matrix, namely |(M.M t )-I| = 0 (step 1 in Figure 3). The matrix 5 M is used to transform the input protein into its canonical repre-6 sentation (step 2 in Figure 3). A cosine similarity loss between 7 the original protein and the transformed one is then calculated 8 (step 3 in Figure 3) in order to back-propagate the error over the 9 network (step 4 in Figure 3) and optimize the matrix M.

Protein feature extractor

The trained ProteinNet model has then been exploited to calculate a protein feature descriptor, for each input protein, by extracting its intermediate Global Max Pooling hidden layer. This descriptor corresponds to a 1-dimension vector of 512 values.

Dissimilarity matrix computation

The dissimilarity matrix between the ten protein shape queries and the set of 554 protein shapes has been calculated using Euclidean distance between their respective 512 feature vectors.

Runtimes and computational cost

This framework has been developed in Python 3.7.6 using different libraries, namely Open3D 0.8.0.0, and Keras 2.2.4-tf on a TensorFlow-GPU 2.1.0 backend. The experiments have been conducted on an Intel Xeon ® Gold ® 5118 CPU@2.30 GHz with 128 GB of memory and NVIDIA ® Tesla ® T4 GPU with 16 GB of memory. The running times in sof each stage performed on CPU are reported in Table 1 for one protein. Table 2 shows the training times of the ProteinNet model trained on GPU. The code is available at the following url: https: //github.com/Benhabiles-JUNIA/ProteinNet. points APPFDs aggregation with FK and GMM described be-37 low. Figure 4 shows the processing pipeline of the APPFD-FK-GMM algorithm with complete implementation details provided in [START_REF] Otu | Code Implementation of Agglomeration of Point-pair Feature Descritor With Fisher Kernel and Gaussian Mixture Model (APPFD-FK-GMM)[END_REF]. The reader is referred to [START_REF] Thompson | SHREC'20 track: Retrieval of digital surfaces with similar geometric reliefs[END_REF], for further details regarding this method.

Stage (1) -Computing Local APPFDs. Following key points (p k i ) determination for each 3D protein surface, represented as point cloud (P), the 4-dimensional feature, f 1 = (α, β, γ, δ) in [START_REF] Wahl | Surflet-pair-relation histograms: A statistical 3d-shape representation for rapid classification[END_REF] is augmented with a locally-extracted 2-dimensional feature: f 2 (p i , p j ) = (ϕ, θ) for every possible combination of point pair, p i , p j (without their estimated normals, n i , n j ) in the local surface patch (LSP), {P i , i = 1 : K} around each key point {p k i , i = 1 : K} in P s , where K is the number of key points. The extraction of f 2 (see Figure 5) is because f 1 is not robust enough to capture the entire geometric details of the underlying surface, whereas, the PPF approach opens up possibilities for additional feature space.

The angular projections: θ and ϕ in Figure 5 are derived by taking the scalar products of (

- → S • -→ V 1 ) for ∠ 1 , and ( - → S • -→ V 2 ) for ∠ 2
about a point p i in a given LSP. Mathematically, scalar products defined in this manner are homogeneous (i.e. invariant) under scaling and rotation. Therefore, f 2 is considered rotation and scale invariant for 3D shapes under rigid and non-rigid transformations [34].

Finally, a 6-dimensional f 3 = ( f 2 + f 1 ) are locally obtained thus: f 3 (p i , p j ) = ( f 2 (p i , p j ), f 1 (p i , p j )) = (ϕ, θ, α, β, γ, δ), and binned into a 1-dimensional histogram with bins = 35 in each feature-dimension, normalized and concatenated to give 35 × 6 = 210-dimensional single local APPFDs per LSP.

Stage (2) -Key points APPFDs Aggregation with FK and GMM.

Here, the final descriptor (i.e. fisher-vector, FV) computation approach involves an initial step of training a GMM, given aggregated key points local APPFDs for all database 3D objects, then FK is applied on the trained model and a single protein's local APPFDs to derive a global signature (APPFD-FK-GMM) for the protein surface (see Figure 4).

Runtimes and computational cost

Our group submitted a dissimilarity matrix D = [10 × 554], where the entry D = [i, j] corresponds to the L 2 distance from i th FV in the query set to the j th FV in the collection set.

While implementing the APPFD-FK-GMM for this task, K is specified by the parameter, vs = 0.20, which is the voxel size for point cloud down-sampling, while the radius parameter, r = 0.50 specifies the size of P. Regarding point cloud size, P = 3, 500 points are sampled.

In conclusion, we present a pure Python 3.60 implementation code that computes the APPFD-FK-GMM method. All Local Surface Patch (LSP), P i with pairwise points (p i , p j ) as part of a surflet-pair relation for (p i , n i ) and (p j , n j ), with p i being the origin. θ and ϕ are the angles of vectors projection about the origin, p i . θ is the projection angle from vector ⟨p ip j ⟩ to vector ⟨p ip c ⟩ while ϕ is the projection angle from vector ⟨p ip j ⟩ to vector ⟨p i -l⟩. The LSP centre is given by p c , keypoint is given as p k i where i = 2. Finally, l is the vector position of p k ip c . [34] experiments were carried out under Windows ® 7 desktop PC with Intel ® Core ® i7-4790 CPU @ 3.60GHz, 32GB RAM. It takes an average of 30 s to compute the APPFD-FK-GMM. The implementation code is available at the following url: https: //tinyurl.com/shrec21.

Projected Wave Kernel Signature Maps (PWKSM) by Léa Sirugue & Matthieu Montès

This method is based on the 2D projection of the surface and the Wave Kernel Signature (WKS) descriptor. Wave Kernel Signature [START_REF] Aubry | The wave kernel signature: A quantum mechanical approach to shape analysis[END_REF] is an isometric invariant descriptor that has been extensively improved and used in the field of computer vision [START_REF] Rodolà | Dense non-rigid shape correspondence using random forests[END_REF][START_REF] Boscaini | Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks[END_REF][START_REF] Limberger | Feature encoding of spectral signatures for 3D non-rigid shape retrieval[END_REF][START_REF] Zeng | Convolutional neural network based multi-feature fusion for non-rigid 3D model retrieval[END_REF]. We have combined WKS with a 2D projection on a unit sphere [START_REF] Angenent | On the Laplace-Beltrami operator and brain surface flattening[END_REF]. Lowering one dimension of the space allows us to have a fast and dense comparison of the surface while having a smaller storage size for files.

Descriptor calculation. In a first step, the WKS descriptor is computed on the surface of the 3D object for each point of the mesh. The surface is flattened on the unit sphere using a confor-1 mal transformation [START_REF] Angenent | On the Laplace-Beltrami operator and brain surface flattening[END_REF]. Then, the 2D spherical coordinates of 2 the unit sphere are converted into 2D cartesian coordinates on 3 the plane [START_REF] Craciun | Shape Similarity System driven by Digital Elevation Models for Non-rigid Shape Retrieval[END_REF]. A maps of size (θ max -θ min )/δ, (ϕ max -ϕ min )/δ is 4 created. θ max and θ min are the maximum and minimum values of 5 θ on the unit sphere and the same with ϕ, each representing an 6 angle coordinate. δ is a coefficient to adapt for resolution. This

S T (T, V) = N T k T =1 min k V L(T (k T ), V(k V )) (1)
The final score is the average of S T and S V defined as fol- this neural network is used for the retrieval task in this track.

Data pre-processing

For the meshes in each 3D model of a protein surface, we first sample 512 points on the surfaces of the meshes based on the area of the meshes. Then, to re-assign the topological structures for sampled points, we connect each nodes with their k-Nearest Neighbors based on their original coordinates (k = 16).

Edge Convolution

In this geometry-only setting, the initial node features is the coordinates of sampled points. Each protein surface is represented by a k-Nearest Neighbors graph generated in the preprocessing step with 512 vertices (nodes).

The module that performs the graph message-passing function is the EdgeConv layer [START_REF] Wang | Dynamic graph CNN for learning on point clouds[END_REF]. In the EdgeConv layer, the information of a vertex i after layer l is calculated as follows:

x l+1 i = max j∈N h(x l i , x l j ) ( 3
)
where N is the neighboring vertices of vertex i with

h(x l i , x l j ) = ReLU(MLP(x l i ⊕ x l j )) ( 4
)
where ReLU is Rectified Linear Unit (in this implementation, we used LeakyReLU-a variant of ReLU), MLP is a standard multilayer perceptron (MLP), and ⊕ is the concatenation operator.

In this implementation, our group uses a dynamic variant of EdgeConv instead of the standard EdgeConv described above. At each Dynamic EdgeConv layer, each vertex's k-Nearest Neighbors is re-calculated in the feature space produced by the previous layer, before applying the standard EdgeConv operation. After the graph has been recomputed, standard EdgeConv operation is performed.

After the pre-processing phase, the vertex features first go through 4 layers of Dynamic EdgeConv. The dimensions of output features for each vertex after these first-4 layers are 64, 64, 128, and 256, respectively. Then, the outputs of these 4 layers are concatenated to become a 512-dimensional vector for each vertex. This 512-dimension vector is then fed through another Dynamic EdgeConv layer, creating the output vector v with 512 dimensions. The feature vector v is pooled using the concatenation of the outputs of a max-pooling and a mean pooling layer to generate the first graph-level feature vector. This vector is passed through two MLP blocks with BatchNorm, Leaky-ReLU, and Dropout layers. Finally, the vector is passed through a Fully-Connected layer for classification.

The latent representation of the graph is extracted as vectors by removing the last Fully-Connected layer from the network. The retrieval task is then performed by exploiting the L2-distances between these vectors.

Runtimes and computational cost

This method is implemented in Python 3.8 [START_REF] Van Rossum | Python 3.8.8 documentation[END_REF], using Pytorch [START_REF] Paszke | Pytorch: An imperative style, high-performance deep learning library[END_REF] and Pytorch Geometric [START_REF] Fey | Fast graph representation learning with pytorch geometric[END_REF] libraries. The experiments were carried out a machine with an Intel ® Core ® i7-8700K 6-core CPU Processor @3. [START_REF] Ponna | Structure of an unconventional SH3 domain from the postsynaptic density protein Shank3 at ultrahigh resolution[END_REF] We use common evaluation metrics to assess the perfor-6 mance of the proposed methods, most of which are used in 7 other SHREC tracks [START_REF] Thompson | Retrieval of digital surfaces with similar geometric reliefs[END_REF] or similar works evaluating the per-8 formance in retrieval [START_REF] Shilane | The Princeton Shape Benchmark[END_REF]. For each method, we compute the 9 overall metrics (i.e. the metrics averaged over all queries) and Both metrics range from 0 to 1. Precision-Recall curve plots the precision values at given recall values, which produces, in an ideal case, an horizontal line at P = 1 that spans the entire range of recall values.

Confusion matrix

A confusion matrix (CM) is a square matrix, whose columns represents the different classes of the dataset and rows the class of the query. For each row, each element CM(i, j) gives the number of objects from class i retrieved using the query j, considering the top k = C retrieved objects, C being the size of the class corresponding to query j. The elements CM(i, i) in the diagonal of the confusion matrix indicates the objects classified correctly, while the off-diagonal indicates mis-classified elements. To ease the comparison between classes of different sizes, the numbers were normalized over the class sizes C of the queries. Consequently, the sum j CM(i, j) of all elements of each row equals 1.

Reciprocal Rank and Mean Reciprocal Rank

The Reciprocal Rank (RR) measures the performance to find the first relevant item. For a given query, it equals to the inverse of the rank r of the first relevant item found: RR = 1/r. The Reciprocal Rank ranges from 0 (no relevant object found) to 1 (the first retrieved object is relevant). The Mean Reciprocal Rank (MRR) is the Reciprocal Rank averaged over all queries. This metric is useful as: (1) It is considered order-aware, contrary to the previous metrics. (2) Typical use cases only consider the few first retrieved items; therefore, the higher the reciprocal rank, the better the performance.

Results

Among the participants of the track, all teams returned a dissimilarity matrix for the shape-only dataset, and only one method (3DZD) was adapted to handle the shape+electrostatics dataset.

Shape-only challenge

The results for the shape-only dataset are presented in Table 4 and in Figures 8 and7. Table 4 summarizes the performances of all submitted matrices for the shape-only dataset. For each metric (Nearest Neighbor, First Tier, Second Tier and Mean Reciprocal Rank), the highest value is indicated in bold. Given the dataset structure and the selected query domains, the best ent pattern for a few classes (Figure7): see the PDZ class for the 3DZD method (green curve, top left plot) or the SH3 class for the APPFD-FK-GMM method (dark blue curve, middle left plot), for instance, whose corresponding curves display medium precision values at medium recall. The confusion matrices for all methods are shown in Figure 8. Combined with Figure 1, they allow us to put the performance into perspective. For instance, PDZ and PDZ_6 domains are topologically very similar (TM-score: 0.79, Figure 1) and therefore were expected to be confusing. When using the PDZ_6 query, ProteinNet retrieved only 1 (4%) of the 26 PDZ_6 shapes within the first 26 retrieved results, but also 12 (46%) shapes from the PDZ class (Figure 8, middle top confusion matrix). More strikingly, 3DZD only found 1 (3%) of the 33 SH3_2 shapes within the 33 first retrieved shapes using the SH3_2 query, but the other 32 retrieved shapes belong to the SH3 class (Figure 8, second row of the top left confusion matrix), which is closely related to the SH3_2 class (TM-score: 0.84, Figure 1). Similarly to the shape-only dataset, results for the shape+electrostatics dataset are presented in Table 5 and 5. The results show similar trends compared to the shape-only dataset, with a nearest neighbor of 0.5, a first tier value of 0.16, a second tier value of 0.321 and a mean reciprocal rank of 0.454. These metrics are similar to the results obtained from the shape-only dataset for the 3DZD method (the second tier value increased while the mean reciprocal rank decreased). The per-class metrics are shown in Appendix D (Tables D.11, D.12, D.13 and D.14).

Shape+electrostatics challenge

The precision-recall curves (Figure 9) show a similar overall behavior for the 3DZD method, whose performance improved significantly for the SH3 domain but decreased significantly for the PDZ domain (dark blue and green curves, respectively, left plot of Figure 9). The confusion matrix (Figure 10) is in line with the previous results, indicating that 3DZD performs similarly in terms of overall performance but with a few differences at the per-class results.

Discussion and concluding remarks

Shape-only dataset

Overall, the 3DZD method obtained the best results, in line with the previous tracks on protein shapes, where this group similarly obtained overall good results [33,34]. This method relies on the use of 3D Zernike polynomials, which has been successfully used to retrieve proteins based on their shapes [9] or their Cα coordinates [START_REF] Guzenko | Real time structural search of the Protein Data Bank[END_REF]. It then uses a neural network trained on the SCOPe [37,[START_REF] Chandonia | SCOPe: Classification of 52 large macromolecular structures in the structural classification of pro-53 teins-extended database[END_REF] database, whose classification largely overlaps with the classification of the Pfam database [START_REF] Mistry | Pfam: The protein families database in 2021[END_REF]. As instance, in the SCOPe databases, the SH3 domain and the SH3_2 domain are classified in two different SCOPe domains, similarly to the Pfam classification. The DGCNN used the data from another SHREC'21 track, the Retrieval and classification of protein surfaces equipped with physical & chemical properties track [35]. The organizers of this track, similarly to a previous track [33], proposed a set of shapes derived from NMR structures along their surficial physico-chemical properties to allow the participants to train their methods, and the resulting classification of the proteins was derived from the SCOPe database as well. The DGCNN and 3DZD methods were therefore trained on similar data, but produced different performance.

Another point to consider, is that the DGCNN method uses a sampling (down to ≈ 8, 000 points) of the initial point clouds (see 3.5.1) that potentially resulted in a loss of information 4).

12 While some methods were able to maintain medium preci-13 sion levels at medium recall values (see section 5.1), a few or the DGCNN group retrieved a significantly higher propor-28 tion of PDZ domains than PDZ_6 domains within the first re-29 sults using the PDZ_6 query. In these cases, the high level of similarity between the domains coupled to the imbalanced size 31 of the classes have negatively impacted the results. In the mean time, these results highlight the limits of the currently available methods to distinguish between the most closely related proteins. Also, we observed different results for order-aware (mean reciprocal rank) and order-unaware (nearest neighbor, first ter and second tier) metrics. While DGCNN, ProteinNet and PWKSM methods display similar values for order-unaware metrics, PWKSM displays a higher Mean Reciprocal Rank. When converted back to ranks, these results mean that, on average, PWKSM ranked better the first relevant match, but did find a similar amount of relevant items within the top results.

Shape+electrostatic dataset

While the shape of a protein is of tremendous importance, its surficial properties are important as well. Therefore, in this track, we generated the shape+electrostatic dataset which encompass both properties, in order to stimulate the development of such methods. However, only one groups returned a dissimilarity matrix for this dataset, namely the 3DZD group. Most groups that participated to this challenge come from the computer vision field. As such, most of the methods presented in this work are the result of methodological developments dedicated to the analysis of 3D point clouds. The development of new, specific methods to handle both shapes and electrostatics would require an amount of time far greater than the SHREC timeline. Nevertheless, we hope that this challenge along with the other challenge dedicated to protein shapes [35] would stimulate the development of such methods.

The overall results showed that the treatment of the electrostatics by the 3DZD marginally improved the results, compared to the shape-only results. Interestingly, the electrostatic potential impacted differently each class: it improved the ability of the 3DZD method to handle the SH3 or the Bromodomain classes, as evidenced by the precision-recall curves (Figures 7 and9, Tables in Appendix C and Appendix D). The 3DZD method is derived from previous attempts from the same team to couple shape and electrostatics analysis to classify protein surfaces [START_REF] Sael | Rapid comparison of properties on protein surface[END_REF]. As noted in this exploratory work, electrostatics may be suitable to compare closely related proteins, while our datasets was mainly composed of loosely related proteins (Fig 1). The electrostatics potential feature is likely to be more beneficial for protein surface comparison of local features rather than global shapes. In such local cases (like comparison of catalytic or binding sites), local electrostatic hot spots would represent the major local feature rather than one of the many features of the global protein surface, as it is the case for the MaSIF method [START_REF] Gainza | Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning[END_REF].

Current machine learning-based methods: pitfalls and challenges

Pitfalls of previously mentioned methods lies in the exploited protein datasets and their characteristics, notably the class imbalance as well as the high inter-class shape similarity. Indeed, the protein datasets are often highly imbalanced in terms of protein classes, which introduces a bias in the training process of these methods towards learning efficiently class representation. One common technique consists of using data augmentation to Preprint Submitted for review / Journal of Molecular Graphics & Modelling (2021) 13 overcome this lack of original data. Hence, it is important to bear in mind that several protein classes cannot be considered as representative groups of protein families. Moreover, our problematic tackles a large quantity of classes composed of protein, which are e.g. visually highly similar. In such a case, a challenge lies in the design of new methods with a high discriminating power that allows to extract the most significant features for distinguishing between protein classes. In this sense, other aspects of the proteins (in addition to the shape) such as molecular properties and electrostatic properties could be considered.

These parameters have to be carefully analyzed through experiments before envisaging method generalizations."

Concluding remarks

In conclusion, we have presented the results of the SHREC'21 challenge on Surface-based protein domains retrieval. The number of participants remained stable compared to the last two years, indicating a constant interest of the shape retrieval community towards biologically relevant problems.

Each group relied on different methods and theoretical background with respect to recommended modeling/machine learning practices [START_REF] Caruana | Intelligible and explainable machine learning: Best practices and practical chal-Preprint Submitted for review[END_REF][START_REF] Artrith | Best practices in machine learning for chemistry[END_REF] in order to solve the problem proposed by the organizers, and represent a variety of approaches to the same problem. As a step towards open science, all participants accepted to share their programs publicly with the community. Overall, the results are decreased compared to similar past tracks [34]. Indeed, two methods based on descriptors similar to 3DZD and APPFD-FK-GMM (3DZD and HAPPS, respectively) were presented in the SHREC'20 contest and performed very well (e.g. both methods exceeding 0.95 for the NN metric) on a problem similar to the shape-only problem (see Tables 6 and 7 of [34]). However, the adapted versions (3DZD and APPFD-FK-GMM) did not reach the same level of performance by exploiting this new, particular dataset of proteins. This decrease of performance (and low performances from the three other methods) reveals that this year dataset was particularly hard to analyze, and that there is still room for improvements.

Among the proposed methods, we observed that 3 over 5 used a learning-based protocol at some point. This proportion is in line with last year track, and show that such approaches continue to be investigated as they usually improve the results. To this regard, the SHREC'21 track on Retrieval and classification of protein surfaces equipped with physical & chemical properties might highlight some interesting points on the best architecture to learn protein surficial properties [35]. Similarly, the organizers of this track computed a set of additional chemical properties (electrostatic potential, location of potential hydrogen bond donors and acceptors, hydrophobicity). In this track, the participants first used the surface geometry then the combination of the geometry and physico-chemical features of the protein surfaces. The results showed that all methods improved their results when using both the geometric and physico-chemical data compared the the geometry only. Particularly, the results generated by the machine learning based methods increased more compared to the other methods. As each of the physicochemical feature was not considered individually, it remains hard to knowpredict whether one feature hasbeat a greater importance than the other. However, in their work, Gainza et al.

showed that the electrostatic potential have the greatest impact 58 of the physico-chemical features they computed. In our work, 59 the electrostatic potential was used by the 3DZD team as an 60 additional feature to help the retrieval task. The results reveals 61 only slightly improved results compared to the results from the 62 shape-only dataset. As noted in [START_REF] Sael | Rapid comparison of properties on protein surface[END_REF], the electrostatic potential 63 may be of better use to compare closely related proteins, rather 64 than comparing loosely related proteins, as it is the case in our 65 work. Alternatively, shapes and electrostatics may be used in 66 hierarchical way, i.e. using first the shape then the electrostatics 67 to achieve a better result.

68

This track reveals significantly lower results when compared 69 to past tracks [31,32,33,34]. However, satisfactory solutions 70 exist to distinguish between loosely related proteins, or to iden-71 tify identical proteins with different conformations, based on 72 their shapes only. Our work also reveals some limits of the 73 methods used by the participants for the challenge. Very closely 74 related proteins (such as SH3 and SH3_2 protein domains), 75 i.e. proteins displaying a high topological similarity and lim-76 ited variations of their amino-acid sequences, hence surfaces, 77 are still difficult to separate in different classes, but some meth-78 ods distinguish them from the other classes. Also, when we 79 consider the DNA-binding domain from the STAT proteins, no 80 method was able to produce satisfactory results. While the 81 DNA-binding domain used as a query has a globular shape, 82 the STAT proteins are significantly bigger, with a non-globular 83 shape, and have 3 additional domains (1 of which is a SH2 do-84 main, a domain included in the dataset), which means that only 85 partial matches compared to the query may be achieved. This 86 specific issue (the comparison of partially overlapping objects) 87 may require further development.

88

In the future, this latter point could be the subject of a ded-89 icated SHREC track, and a good indicator of the overall pro-90 gresses made in the field of the retrieval of proteins based on 91 their surfaces. Currently, most methods have difficulties to han-92 dle such cases, which are quite common. Solving this chal-93 lenge would be a step forward for the community. At the same 94 time, explainable artificial intelligence (XAI) methods [START_REF] Das | Opportunities and challenges in explainable artificial intelligence (XAI): A survey[END_REF] may 95 highlight the latent features responsible for good or bad pre-96 dictions, and help decipher the results of machine learning-97 based methods. XAI methods may help explain the perfor-98 mance difference observed for each class of protein, and pro-99 vide a human-interpretable representation of machine learning 100 descriptors, and therefore help identifying the current limits of 101 these algorithms. Finally, deciphering to which extend, if any, 102 the standard physico-chemical features (electrostatics potential, 103 charges distribution, hydrophobicity, etc.) improve the results 104 may be the main focus of the next SHREC tracks devoted to 105 protein surfaces.

  Sael et al. use 3D Zernike descriptors to detect either global or local similarities between protein's surfaces [9].

64 2. The Dataset 65 2

 6465 .1. Constitution of the SHREC'21 dataset 66

Fig. 2 . 20 nal

 220 Fig. 2. The Upset Plot of ten selected Pfam domains in SHREC2021 challenge datasets. The dataset is composed of 554 individual shapes, of which 22 bears two of the domains of the dataset.

25 ( 2 )

 252 Limit the dataset size to a tractable size given the track 26 timeline. Finally, only the best resolution structures for each 27 Uniprot entry were kept. When NMR structures were selected, 28 only the first model was considered. Unlike the query struc-29 tures, we kept the other domains present in these structures that 30 eventually constitute the dataset. Therefore, many dataset struc-31 tures display several domains, at least one of which is one of the 32 query domains.

62. 3 .

 3 Challenge proposed to the participants 7 SHREC, or 3D Shape Retrieval Challenges, are challenges 8 primarily organized in order to evaluate the effectiveness of 3D-9 shape retrieval algorithms. A group organizes a challenge by 10 building up a dataset, then proposes the challenge publicly to 11 the community, and finally gathers, analyses and verifies the re-12 sults. The theme of the challenge may vary from one to another, 13 but all challenges take place in a limited time, which ranges 14 from 1 to 1 ½ months.

15

 15

16 10

 16 query surfaces, to retrieve the molecular surfaces of pro-17 teins from the dataset that encompass the same domain as the 18 query. Each query-to-dataset-surface distance was expected to 19 be expressed as a dissimilarity score. The results were there-20 fore 10 × 554 matrices of dissimilarity scores. Each participant 21 was allowed to submit one dissimilarity matrix for each dataset: 22 one matrix for the shape-only dataset, and one matrix for the 23 shape+electrostatic dataset.

24 3 .

 3 Participants and methods25Among the seven groups that initially registered to this track, 26 only 5 were able to produce the results in time and returned a 27 shape-only dissimilarity matrix. Only one method (3DZD, see 28 3.1) returned a dissimilarity matrix for the shape+electrostatic 29 dataset. The other groups were not able to produce a satisfying 30 training dataset or willing to develop their algorithm to han-31 dle the electrostatics values. In the following subsections, each 32 group describes their new, respective methods.

  33

Fig. 3 .

 3 Fig. 3. ProteinNet deep architecture for protein point cloud transformation into canonical representation. Step (1): affine transformation matrix estimation. Step (2): protein point cloud transformation using the estimated affine matrix. Step (3): similarity calculation between the original protein point cloud (the input) and its transformed point cloud. Step (4): cosine similarity loss calculation between the original input protein point cloud and its transformation; and back-propagation over the network to optimize the estimation of the affine transformation matrix.

  2

10 3 .

 3 2.1. PT-Net module 11The module is composed of a sequence of 3 convolution 12 blocks (32, 64 and 512 layers) followed by a global max pool-13 ing layer and 3 successive dense layers (256, 128 and 9). As 14 shown in Figure 3, each convolution block as well as the dense 15 layers (except the last one) undergo a batch normalization and a 16 tangent hyperbolic activation function. The last dense layer of 17 9 units is reshaped to output the (3 × 3) M matrix (see details 18 in [40]).

19 3 .

 3 2.2. Data preparation and architecture training 20 All the proteins of the dataset of the track have been sampled 21 to 2,048 points using a Poisson disk sampling technique [41] 22 and then normalized into a zero-center unit sphere based on 23 their respective minimum bounding spheres [42]. The archi-24 tecture has then been trained using a batch size of 16 on 80% of 25 the dataset over 150 epochs and validated on the remaining 20% of the data. The training data were augmented on-the-fly (during the training process) by adding some geometric noise (e.g. random displacement of point coordinates in a limited interval).

Fig. 4 .

 4 Fig. 4. APPFD-FK-GMM processing pipeline involving Phase 1 (fitting a GMM to all the keypoints or LSPs descriptor, i.e. local APPFDs from each 3D protein surface and for all database protein surfaces) and Phase 2 (computing a single compact descriptor called fisher-vector (FV) for each 3D protein by aggregating all its keypoints or local APPFDs using the fisher kernel (FK) framework and the trained GMM in Phase 1. Within each LSP around a keypoint, six different geometric features are first extracted, and each feature-dimension is binned into a 1D histogram with 35 bins, where all histograms are combined to form a 210-dimensional descriptor, i.e. local APPFD for each LSP. All such LSP descriptors from each 3D protein are compacted into a 4210-dimensional FV for that protein model, as in Phase 2.

Fig. 5.

  Fig. 5. Local Surface Patch (LSP), P i with pairwise points (p i , p j ) as part of a surflet-pair relation for (p i , n i ) and (p j , n j ), with p i being the origin. θ and ϕ are the angles of vectors projection about the origin, p i . θ is the projection angle from vector ⟨p ip j ⟩ to vector ⟨p ip c ⟩ while ϕ is the projection angle from vector ⟨p ip j ⟩ to vector ⟨p i -l⟩. The LSP centre is given by p c , keypoint is given as p k i where i = 2. Finally, l is the vector position of p k ip c .[34]

7

  type of projection is similar to topographic maps, that is why 8 our group called this descriptor Projected Wave Kernel Signa-9 ture Maps (PWKSM). An interpolation in the space of discrete 10 integers is done to densify the maps. To reduce impact of de-11 formation at the poles when converting to 2D cartesian coordi-12 nates, we computed 7 different maps with different pole orien-13 tations. 14 Descriptor comparison. A dense comparison is made using 15 GPGPU sum reduction technique [54] [55] [56]. Each point's 16 WKS of a PWKSM is compared to all points' WKS of another 17 PWKSM. The Earth Mover's distance L is used to compare the 18 WKS descriptor of each point. Then, the smallest distance be-19 tween a point of a first PWKSM T and all points of a second 20 PWKSM V is selected. The sum of all the smallest distances 21 for each point of the first PWKSM are summed to create the 22 score S T . The same is done for computing S V .

23

 23

34 3 . 43 (

 343 5. Graph-based learning methods for Surface-based protein 35 domains retrieval (DGCNN) by Huu-Nghia H. Nguyen, 36 Tuan-Duy H. Nguyen, Vinh-Thuyen Nguyen-Truong, Danh 37 Le, Hai-Dang Nguyen & Minh-Triet Tran 38 In this deep learning method, our group exploits the avail-39 ability of protein class labels from [35] to optimize the rep-40 resentation of protein surfaces without any additional proper-41 ties. Particularly, we designed a message-passing graph convo-42 lutional neural network (MPGCNN) with the Edge Convolution EdgeConv) paradigm [57] for the protein classification objec-44 tive. Then, the latent representation of protein surfaces from 45

Fig. 6 .

 6 Fig. 6. Dynamic Edge Convolutional Neural Network

10 the

 10 individual metrics (i.e. the metrics for each query) to pro-11 vide a better understanding of the performance of each method 12 for each query. Two composite classes are also presented: the 13 SH3-like and the PDZ-like, which correspond to the grouping 14 of the SH3 and SH3_2 classes, and of the PDZ and PDZ_6 15 classes, respectively. To set the dissimilarity values for these 16 composite classes, for each entry of the dataset, we kept the 17 minimal dissimilarity value from the SH3 / SH3_2 queries and 18 from the PDZ / PDZ_6 queries.

19 4. 1 . 29 4

 19129 Nearest neighbor, First tier and Second Tier 20 These metrics measures the ratio of relevant objects among 21 the k retrieved objects, and ranges in the interval [0, 1]. For the 22 nearest neighbor (NN), only the first retrieved object is consid-23 ered (k = 1), while the top C objects are considered for the first 24 tier (FT), and the top 2 × C objects for the second tier (ST).25 Here, C represents the cardinal of the class under investigation, 26 i.e. the size of the class to which the query belongs. Higher val-27 ues of nearest neighbor, first tier and second tier indicate better 28 performance. .2. Precision-Recall curves 30 The Precision (P) represents the fraction of relevant ob-31 ject retrieved compared to the top k retrieved objects: P = 32 (relevant retrieved)/retrieved. Therefore, precision can be 33 evaluated at different intervals. The Recall (R) represents the 34 fraction of relevant objects retrieved compared to the size C of 35 the class of the query: P = (relevant retrieved)/relevant.

Fig. 7 .

 7 Fig. 7. Per-query precision-recall curves for the shape-only dataset, for each method. All plots are colored according to the legend on the bottom right of the figure.

2 0 . 7 C 10 ure 7 )

 07107 523 for the mean reciprocal rank. These results must be bal-3 anced by the fact that a few classes have only a small number 4 of models (namely, the Stat-binding and m50 classes only have 5 6 and 4 members, respectively, see Figure 2), and thus impact 6 negatively the averaged results. For completeness, Tables C.7, .8, C.9 and C.10 in Appendix C contain the per-class evalua-8 tion metrics for all methods. 9 The precision-recall curves for each individual classes (Figshow that most methods display a similar behavior for 11 all classes, characterized by a quick drop of the precision at 12 low recall values. A few methods, however, show a differ-13

  Figures 9 and 10. Only one team returned a dissimilarity matrix for the shape+electrostatics dataset. The evaluation metrics are listed in Table

Fig. 8 .

 8 Fig. 8. Confusion matrices of all methods for the shape-only dataset. The color-range is the same for all matrices. Confusion ranges from 0 (white background) to 1 (deep purple background). The original classes are separated from the composite classes (SH3-like and PDZ-like) by a black line.

Fig. 9 .

 9 Fig. 9. Per-query precision-recall curves for the shape+electrostatics dataset, for each method. All plots are colored according to the legend on the far right of the figure.

Fig. 10 .

 10 Fig. 10. Confusion matrix for the shape+electrostatics dataset. Confusion ranges from 0 (white background) to 1 (deep purple background). The original classes are separated from the composite classes (SH3-like and PDZ-like) by a black horizontal line.

14 queries

 14 were difficult to handle for all methods. For the DNA-15 binding domain from the STAT protein family or the peptidase 16 M50 domain, the low number of such surfaces (6 and 4, re-17 spectively) in the datasets explains the low performance ob-18 served for all methods. For the other queries, like the PDZ_6 domain or the SH3_2 domain, the explanation is the presence 20 of closely related domains, the PDZ and SH3 domains, respec-21 tively. These confusing classes are significantly more populated 22 (128 versus 26, for the PDZ / PDZ_6 domains, and 115 versus 23 33, for the SH3 / SH3_2 domains). This is supported by the con-24 fusion matrices, which showed that, for instance, the ProteinNet 25 group retrieved a great amount of SH3 domains (and almost no 26 SH3_2 domains) within the top results using the SH3_2 query,

  27

106

  Additional statments & Acknowledgments107The authors thank the 3DOR 2021 Workshop organizing 108 committee for maintaining this workshop despite the current 109 COVID-19 pandemic. Matthieu Montès and Florent Langen-110 feld thank Taoufik Labib for setting up and maintaining the 111 track webpage.

Funding. 3 Daisuke

 3 Léa Sirugue, Matthieu Montès and Florent Langen-1 feld are supported by the European Research Council Ex-2 ecutive Agency under the research grant number 640283. Kihara acknowledges supports from the National Insti-4 tutes of Health (R01GM133840, R01GM123055) and the National Science Foundation (DBI2003635, CMMI1825941, and 6 MCB1925643). Charles Christoffer is supported by NIGMS-

Table 1 .

 1 Running times in s using CPU for each stage of the ProteinNet framework obtained for one protein.

	Point cloud maximum and	Point cloud sampling (2,048)	Feature descriptor (512)	Distances from one protein to
	minimum sizes of two	and normalization	calculation of one protein	all protein dataset (554
	proteins			proteins)
	582,496 points 37,658 points	1.14 0.9	0.005	0.004

Table 2 .

 2 Running times in s using CPU for each stage of the ProteinNet framework obtained for one protein.

	Deep learning model ProteinNet
	Training data size	499
	Epochs	150
	Training time (s)	155
	3.3. Fisher Kernel agglomerated local Augmented Point Pair

1 Feature Descriptors, trained with Gaussian Mixture 2 Model (APPFD-FK-GMM) by Ekpo Otu, Reyer Zwigge-3 laar, David Hunter & Yonghuai Liu 4 Our group presents a novel APPFD-FK-GMM 3D shape re-5 trieval method (see Figure 4) based on Fisher Kernel (FK) and 6 Gaussian Mixture Model (GMM) agglomeration of the Aug-7 mented Point-pair Feature Descriptor (APPFD) [43]: a 3D 8 key point shape descriptor that robustly captures the physi-9 cal geometric characteristics of 3D surface regions. Previous 10 APPFD binning technique involves bucketting each of the 6-11 dimensional features of the APPFD into a multi-dimensional 12 histogram with at least 7 bins in each feature-dimension, result-13 ing to approximately 7 6 = 117649-dimensional final feature-14 vector (APPFD), which is very high-dimensional final descrip-15 tor. 16 In this work, we contribute a simpler approach, where each 17 of the 6-dimensional feature is binned into a 1-dimensional 18 histogram with 35 bins for each feature-dimension to pro-19 duce a 210-dimensional local descriptor (APPFD) for every key 20 point or local surface patch (LSP). Finally, the locally com-21 puted APPFDs are agglomerated into a compact code called 22 the Fisher Vector (FV) with 4210 dimensions, which is L 2 and 23 power-normalized, and represents a single protein model, using 24 the FK and GMM [44] framework.

25

The goal of the APPFD-FK-GMM method/contribution is to 32 face representation to 3, 500 points sample. 33 3.3.1. Summary of The APPFD-FK-GMM Method 34 Our method involves two main stages: (1) Computing local 35 APPFDs for selected key points on 3D protein surface. (2) Key

Table 3 .

 3 The 2 code is available at the following url: https://github.com/

	3
	huunghia160799/SHREC-protein-domains.

4

Table 3 .

 3 Time report of each step of the DGCNN method.

		Training	Test Set	Matrix	Total
			Extraction	Computa-
				tion
		≈ 1100 min	≈ 7 min	≈ 3 min	1173 min
	5	4. Evaluation metrics

Table 4 .

 4 Summary of the average evaluation metrics for the shape-only dataset. The composite classes are excluded from the average; they are presented in Tables C.[START_REF] Mariani | lDDT: A lo-49 cal superposition-free score for comparing protein structures and mod-50 els using distance difference tests[END_REF] to C.10.

		Method	Nearest	First Tier	Second	Mean Re-
			Neighbor		Tier	ciprocal
						Rank
		3DZD	0.5	0.160	0.292	0.523
		ProteinNet	0	0.088	0.195	0.126
		APPFD	0.3	0.136	0.237	0.410
		PWKSM	0.1	0.105	0.201	0.236
		DGCNN	0	0.098	0.189	0.193
	1	method achieves an overall level of 0.5 for the nearest neigh-
		bor metric, 0.160 for the first tier, 0.292 for the second tier and

Table 5 .

 5 Summary of the average evaluation metrics for the shape+electrostatics dataset.The composite classes are excluded from the average; they are presented in Tables D.11 to D.14.

	Method	Nearest	First Tier	Second	Mean Re-
		Neighbor		Tier	ciprocal
					Rank
	3DZD	0.5	0.160	0.321	0.454

Preprint Submitted for review / Journal of MolecularGraphics & Modelling (2021)

& Matthieu Montès; Methodology: Tunde Aderinwale, Charles

Christoffer, Woong-Hee Shin, Genki Terashi, XiaoWang &

Melkemi (ProteinNet), Ekpo Otu, Reyer Zwiggelaar, David

Hunter & Yonghuai Liu (APPFD-FK-GMM), Léa Sirugue &

Matthieu Montès (PWKSM), Huu-Nghia H. Nguyen, Tuan-

funded pre-doctoral fellowship (T32 GM132024). Huu-Nghia