Jon Haitz Legarreta

François Laurent Petit

François Rheault

Guillaume Theaud

Carl Lemaire

Maxime Descoteaux

Pierre-Marc Jodoin

Tractography filtering using autoencoders

Keywords: Representation Learning, Autoencoder, diffusion MRI, Tractography, Filtering

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Tractography is a diffusion Magnetic Resonance Imaging (dMRI) data feature extraction technique that provides information about the brain white matter structural connectivity. These connectivity pathways are composed of streamlines virtually representing fascicles of white matter fibers. Tractography methods face a number of challenges when propagating streamlines, including avoiding the early termination of the tracking procedure [START_REF] Smith | Anatomically-constrained tractography: Improved diffusion MRI streamlines tractography through effective use of anatomical information[END_REF][START_REF] Girard | Towards quantitative connectivity analysis: reducing tractography biases[END_REF], providing streamlines between gray matter regions that are known to be connected while avoiding spurious streamlines, ensuring the full occupancy of the white matter volume by the streamlines [START_REF] Rheault | Bundle-specific tractography with incorporated anatomical and orientational priors[END_REF], and a complete gray matter surface coverage when streamlines reach the cortex [START_REF] St-Onge | Surface-enhanced tractography (SET)[END_REF]. As a result, and despite the efforts to address these issues, white matter tracking procedures are known to produce a disproportionately large amount of invalid streamlines (referred to as "implausible"s in this work, as opposed to "plausible" streamlines). The category of invalid streamlines spans a broad group of streamlines that are known not to satisfy accepted anatomical constraints. These include streamlines that contain loops or sharp bends; streamlines that stop in non-white matter tissues, such as the cerebrospinal fluid (CSF); streamlines that prematurely stop in the white matter; or streamlines describing trajectories between gray matter regions that are not connected structurally. An illustration of "plausible" and "implausible" streamlines can be seen in figure 1. According to the ISMRM 2015 Tractography Challenge benchmark (Maier-Hein et al., 2017), tracking methods produce a non-negligible proportion of invalid streamlines as a trade-off between sensitivity and specificity. The study reported that non-existing, false positive connections were present in as many as 64% of the submissions. Hence, improved filtering strategies are required to detect and remove anatomically unfeasible streamlines and mitigate the undesired drifts of the streamline propagation methods. Filtering "implausible" streamlines generated by tracking methods is a necessary step to avoid biologically unrealistic connectivity patterns and biases in reconstructing the brain structural connectome from diffusion MRI data [START_REF] Sotiropoulos | Building connectomes using diffusion MRI: why, how and but[END_REF][START_REF] Yeh | Connectomes from streamlines tractography: Assigning streamlines to brain parcellations is not trivial but highly consequential[END_REF] (see section 1.1.1). The quality of the tractograms influence various aspects of the connectome analysis, including the connectivity density, the clustering degree or the existence of connections themselves, among others [START_REF] Yeh | Correction for diffusion MRI fibre tracking biases: The consequences for structural connectomic metrics[END_REF]. Hence, tractogram filtering, in its various forms, is essential towards providing a reliable imaging evidence using diffusion MRI of how the white matter fascicles are arranged in the brain.

Filtering in tractography can also be regarded as a procedure that separates streamlines that are useful to serve a certain task from those that are not. As such, some filtering methods involve a clustering operation which may be used to extract a predefined set of bundles (see section 1.1.2). While connectomics applications [START_REF] Sotiropoulos | Building connectomes using diffusion MRI: why, how and but[END_REF] require every anatomically "plausible" fiber to be recovered, other applications often need a more specialized set of fibers. For example, in the human versus macaque study by [START_REF] Takemura | Occipital white matter tracts in human and macaque[END_REF], only streamlines from the occipital white matter are considered. In [START_REF] Chenot | A population-based atlas of the human pyramidal tract in 410 healthy participants[END_REF] only fibers from the pyramidal tract are retained, while in [START_REF] Bullock | Associative white matter connecting the dorsal and ventral posterior human cortex[END_REF] the brain fibers connecting the dorsal and ventral posterior cortex are the ones of interest.

Streamline filtering can be seen as an application of choice for deep learning classification methods. Unfortunately, neural networks are not easily applicable to brain tractography, mostly because of the fundamental difficulty of building a labeled dataset. In fact, finding irreproachable ground truth streamlines is a very difficult endeavour, with significant differences in reproducibility measures even among well-trained expert neuroanatomists, as shown in Rheault et al. (2020a). Also, depending on the task at hand, the labeling of the streamlines might vary considerably, and hence may require time-consuming manual verification and/or relabeling. Classification neural networks are trained to predict a fixed number of classes, and thus cannot be used to predict a different set of classes without being retrained on a newly labeled set of data.

This work proposes a deep autoencoder for streamline-based tractography filtering. The method, named FINTA (Filtering in Tractography using Autoencoders), is trained on non-curated, raw tractograms. Once training is over, the learned latent space is a low-dimensional representation of the input streamlines where similar streamlines are located next to each other. The filtering process is then carried out by projecting to the latent space examples of reference streamlines. Then, the to-be-filtered streamlines are projected to the latent space and labeled according to a nearest neighbor test. We demonstrate the filtering ability of FINTA on both synthetic and human brain datasets. Additional quantitative evaluations show its superior performance compared to both anatomy-based tractography filtering methods and the state-of-the-art RecoBundles [START_REF] Garyfallidis | Recognition of white matter bundles using local and global streamline-based registration and clustering[END_REF][START_REF] Rheault | Analyse et reconstruction de faisceaux de la matière blanche[END_REF] streamline similarity filtering method. Similarly, FINTA is linear in terms of the streamline count at test time, allowing for a faster tractogram filtering.

1.1 Related work

Filtering plausible versus implausible streamlines

Diffusion MRI tractography methods generate tractograms that may contain several million candidate streamlines representing white matter fiber populations. Yet, a large proportion of such streamlines are anatomically "implausible", meaning that they do not comply with neuroanatomical constraints attributed to fiber populations. A regular tractogram including only anatomically "plausible" streamlines can contain in the order of 500 000 to 3 000 000 streamlines [START_REF] Presseau | A new compression format for fiber tracking datasets[END_REF]. Hence, having an automatic procedure to filter out "implausible" streamlines is critical.

Tractography filtering is currently approached based on five main patterns: (i) streamline geometry features; (ii) region-of-interest-driven streamline inclusion and exclusion; (iii) clustering; (iv) connectivity-based; and (v) diffusion signal mapping or forward models [START_REF] Jeurissen | Diffusion MRI fiber tractography of the brain[END_REF][START_REF] Sotiropoulos | Building connectomes using diffusion MRI: why, how and but[END_REF][START_REF] Jörgens | Challenges for tractogram filtering[END_REF]. The first category of criteria is based on anatomical features of individual streamlines, such as unfeasible streamline length or local curvature indices. Streamline inclusion and exclusion criteria incorporate white matter and/or tissue local and connectivity constraints for streamline traversal [START_REF] Schilling | Brain connections derived from diffusion MRI tractography can be highly anatomically accurate-if we know where white matter pathways start, where they end, and where they do not go[END_REF]. Such principles are usually concatenated into a sequence of restrictions to form a succession of step-wise refinement rules (Maier-Hein et al., 2017;[START_REF] Sarubbo | Uncovering the inferior fronto-occipital fascicle and its topological organization in non-human primates: the missing connection for language evolution[END_REF][START_REF] Jeurissen | Diffusion MRI fiber tractography of the brain[END_REF][START_REF] Jörgens | Challenges for tractogram filtering[END_REF]. Clustering approaches, such as QuickBundles (Garyfallidis et al., 2012), use a similarity measure to remove non-meaningful data issued by a tracking method. These methods are based on the assumption that a properly defined distance measure is able to split apart the relevant streamlines from the rest. These methods usually require a representative set of streamlines of the population that is being studied to be built so that they can be used as reference models to exclude the extraneous streamlines. Connectivity-based approaches, such as the one presented by Wang et al. (2018), propose to remove undesired streamlines usually imposing some regularization constraint on the tractogram-derived connectivity matrices. The forward models, such as SIFT/SIFT2 [START_REF] Smith | SIFT: Spherical-deconvolution informed filtering of tractograms[END_REF][START_REF] Smith | SIFT2: Enabling dense quantitative assessment of brain white matter connectivity using streamlines tractography[END_REF], LiFE [START_REF] Pestilli | Evaluation and statistical inference for human connectomes[END_REF] and COMMIT/COMMIT2 [START_REF] Daducci | COMMIT: Convex optimization modeling for microstructure informed tractography[END_REF][START_REF] Schiavi | A new method for accurate in vivo mapping of human brain connections using microstructural and anatomical information[END_REF], posit that only a subset of streamlines in a whole-brain tractogram are essential or relevant to explain the underlying diffusion signal. SIFT/SIFT2 is based on modifying the local fiber Orientation Distribution Function (fODF) based on the diffusion signal, whereas LiFE and COMMIT/COMMIT2 use specific local models to weigh the contribution of the signal to the streamline representation. Anatomical a priori constraints can be incorporated to improve the accuracy of the reconstructed pathways. Some of these methods have large computational demands in terms of the time required to filter the tractogram of a single subject (in the order of hours) [START_REF] O'donnell | Advances in computational and statistical diffusion MRI[END_REF].

A number of works use deep learning-based methods for tractography filtering. de Lucena (2018) used an autoencoder to learn features about the input tractogram, and then used a residual network to regress on those features. However, their results were only limited to the uncinate fascicle and no comparison to other filtering methods was provided.

More recently [START_REF] Astolfi | Tractogram filtering of anatomically non-plausible fibers with geometric deep learning[END_REF] proposed a geometrical deep learning method to filter tractograms. The method implements a supervised geometric model which builds on streamline graph structures.

Streamline clustering

White matter streamline clustering (also referred to as bundling) allows to assign every streamline in a tractogram to the bundle it belongs to [START_REF] O'donnell | Automatic tractography segmentation using a high-dimensional white matter atlas[END_REF][START_REF] Maddah | A unified framework for clustering and quantitative analysis of white matter fiber tracts[END_REF][START_REF] Li | A hybrid approach to automatic clustering of white matter fibers[END_REF][START_REF] Wassermann | Unsupervised white matter fiber clustering and tract probability map generation: Applications of a gaussian process framework for white matter fibers[END_REF][START_REF] Guevara | Robust clustering of massive tractography datasets[END_REF]Garyfallidis et al., 2012;[START_REF] Jin | Automatic clustering of white matter fibers in brain diffusion MRI with an application to genetics[END_REF][START_REF] Wassermann | The white matter query language: a novel approach for describing human white matter anatomy[END_REF][START_REF] Siless | AnatomiCuts: Hierarchical clustering of tractography streamlines based on anatomical similarity[END_REF][START_REF] Sharmin | White matter tract segmentation as multiple linear assignment problems[END_REF][START_REF] Garyfallidis | Recognition of white matter bundles using local and global streamline-based registration and clustering[END_REF]. It is a procedure where the streamlines of interest need to be identified from the rest of the streamlines (the rejection class). The task of attributing a streamline to a bundle needs to address the fundamental question of how streamlines are similar to, or different from, each other. One of the challenges lies in finding the appropriate measure to quantify the affinity of streamlines, the other major difficulty being computing such affinities or distances in a reasonable time.

Probabilistic model-based methods (e.g. [START_REF] Dayan | Unsupervised detection of white matter fiber bundles with stochastic neural networks[END_REF]) or traditional machine learning-based frameworks (e.g. [START_REF] Brun | Clustering fiber traces using normalized cuts[END_REF]; Wang and Shi (2018); [START_REF] Kumar | White matter fiber analysis using kernel dictionary learning and sparsity priors[END_REF]; [START_REF] Bertò | Classifyber, a robust streamline-based linear classifier for white matter bundle segmentation[END_REF]) have also been proposed to solve the clustering task. However, some of the findings in these works are limited to clustering a single bundle, require separate learning models for each bundle, are still affected by the streamline-wise distance computational complexity, or suffer from the trade-off of the granularity of the arrangement.

A number of different strategies have been developed recently to address the streamline clustering task using deep learning networks, including stacked, bi-directional siamese networks (FS2Net) (Patil et al., 2017), regular convolutional neural networks using different streamline re-parameterizations (FiberNet/FiberNet2.0) [START_REF] Gupta | FiberNET: An ensemble deep learning framework for clustering white matter fibers[END_REF][START_REF] Gupta | FIBERNET 2.0: An automatic neural network based tool for clustering white matter fibers in the brain[END_REF], or custom feature vectors (TRAFIC) [START_REF] Lam | TRAFIC: Fiber tract classification using deep learning[END_REF], (DeepWMA) [START_REF] Zhang | Deep white matter analysis (DeepWMA): Fast and consistent tractography segmentation[END_REF], ensemble models using fully connected neural networks [START_REF] Ugurlu | Supervised classification of white matter fibers based on neighborhood fiber orientation distributions using an ensemble of neural networks[END_REF], or representation learning methods [START_REF] Zhong | Auto-encoded latent representations of white matter streamlines[END_REF] based on recurrent autoencoders. Yet, many of these methods still work in a supervised fashion, and thus do not allow to generalize to a variable number of classes, require a distinctly trained network for each bundle, or present results limited to a reduced number of bundles.

As opposed to the previous deep learning approaches, a few other ones [START_REF] Wasserthal | TractSeg -fast and accurate white matter tract segmentation[END_REF]; [START_REF] Pomiecko | 3D convolutional neural network segmentation of white matter tract masks from MR diffusion anisotropy maps[END_REF]; [START_REF] Li | Neuro4Neuro: A neural network approach for neural tract segmentation using large-scale populationbased diffusion imaging[END_REF]) provide voxel-wise segmentations representing the locations traversed by a given bundle's streamlines.

Most of these approaches are based on a U-Net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] or another form of classification convolutional neural network using a specialized feature vector or parameterization. The major drawback of voxel-based bundle segmentation methods is the need of a separate model (including the need to train it) for each bundle, and hence, a variable number of models depending on the target classes.

In this work, we describe a novel unsupervised learning framework based on deep autoencoders to filter streamlines in diffusion MRI tractography. We demonstrate its performance on both synthetic and in vivo human brain tractography data, including partial tractograms. We provide evidence that the described deep learning setting can be applied to filter "plausible" and "implausible" streamlines as well as to multi-label classification problems without requiring to be retrained.

Material and methods

Autoencoders (AEs) are deep neural networks that have the ability to learn an efficient representation of the data in an unsupervised fashion using a dimensionality reduction approach [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF]. AEs are trained to reconstruct an input signal through a series of compressing operations (known as the encoder) followed by a series of decompression operations (referred to as the decoder). As shown in figure 2, between the encoder and the decoder is the so-called latent space, where each point is an encoded representation of an input data sample. In this work, the input data is a streamline, which is a sequence of three-dimensional points. A streamline is an ordered sequence of points s = {c 1 , c 2 , . . . , c ns }, c i ∈ R 3 that represents a package of similarly oriented fibers and describing a neural pathway within the brain white matter. A tractogram representing a set of M streamlines is mathematically expressed as T = {s 1 , s 2 , . . . , s M }.

AEs have two major advantages over other types of neural networks. First, they are trained on raw unlabeled data, which is a major asset in the context of brain tractography. Second, since the latent representation is obtained by a series of compressing operations, two neighboring points in the latent space correspond to input data instances that share similar characteristics [START_REF] Painchaud | Cardiac MRI segmentation with strong anatomical guarantees[END_REF]. Thus, the encoded representation can be used to redefine the notion of inter-data distance. In the context of tractography, instead of measuring the distance between two streamlines with a geometric distance function as is usually the case [START_REF] Siless | A comparison of metrics and algorithms for fiber clustering[END_REF], one can project the streamlines into the latent space and measure their Euclidean distance.

Our filtering approach, shown in figure 2, can be summarized by the following steps:

1. Train an autoencoder with a large, raw, uncurated tractogram.

2. From the training tractogram, label the streamlines of interest. These streamlines could be examples of anatomically "plausible" fibers, streamlines representing fibers belonging to predefined sets of bundles, or any sorts of streamlines that are of interest for some application. These streamlines are labeled as positive streamlines.

3. Project the positive streamlines into the latent space with the encoder network. The positive latent vectors are illustrated by the golden circles in figure 2. Note that in case of more than two classes (as for a multi-label bundling operation) streamlines of several classes could also be projected into the latent space. 4. Given a new tractogram that ought to be filtered, project its streamlines into the latent space (cf. the three bold circles in figure 2(b)) and label it according to the distance to its nearest neighbor. The proposed AE is a fully convolutional neural network whose overall structure is summarized in table 1. Our convolutional AE accepts a streamline at its input. Since raw streamlines have a different number of vertices (long streamlines have more vertices than shorter ones), we re-sample the streamlines so that they all have an equal number of 256 vertices. The latent space length was fixed at a value of 32 for all of our experiments.

The autoencoder was trained with an Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] with a mean squared-error loss. The hyper-parameters were adjusted using a Bayesian search method. The learning rate of the optimizer was fixed to a value of 6.68 × 10 -4 , and weight decaying regularization with a 0.13 valued parameter.

Data

Experiments were carried out using the "Fiber Cup" synthetic data [START_REF] Fillard | Quantitative evaluation of 10 tractography algorithms on a realistic diffusion MR phantom[END_REF][START_REF] Côté | Tractometer: Towards validation of tractography pipelines[END_REF], the ISMRM 2015 Tractography Challenge human-based synthetic dataset [START_REF] Maier-Hein | Tractography challenge ISMRM 2015 data[END_REF], as well as the BIL&GIN in vivo human subject brain dataset [START_REF] Mazoyer | BIL&GIN: A neuroimaging, cognitive, behavioral, and genetic database for the study of human brain lateralization[END_REF][START_REF] Chenot | A population-based atlas of the human pyramidal tract in 410 healthy participants[END_REF].

"Fiber Cup" synthetic data

A synthetic dataset re-generated (using Fiberfox [START_REF] Neher | Fiberfox: Facilitating the creation of realistic white matter software phantoms[END_REF]) from the "Fiber Cup" phantom [START_REF] Fillard | Quantitative evaluation of 10 tractography algorithms on a realistic diffusion MR phantom[END_REF][START_REF] Côté | Tractometer: Towards validation of tractography pipelines[END_REF] was used as a simplified test set mimicking the human brain white matter bundle anatomy. The "Fiber Cup" phantom contains 7 bundles, and the ground truth consists of a total of 7833 streamlines. The raw diffusion data were generated using 30 gradient directions, a diffusion gradient strength of 1000s/mm 2 , and a 3mm isotropic spatial resolution for a 64 × 64 × 3 volume. Figure 3(a) shows the ground truth tractogram for the synthetic "Fiber Cup" phantom.

ISMRM 2015 Tractography Challenge human-based synthetic data

The ISMRM 2015 Tractography Challenge dataset of [START_REF] Maier-Hein | Tractography challenge ISMRM 2015 data[END_REF] was used in a further step towards a more complex and realistic scenario. The dataset consists of a clinical-style, realistic single subject tractogram (approximately 200 000 ground truth streamlines) with 25 ground truth fiber bundles, raw diffusion-weighted data and a structural T1-weighted MRI volume generated using Fiberfox [START_REF] Neher | Fiberfox: Facilitating the creation of realistic white matter software phantoms[END_REF]. The raw dMRI data were generated at a 2mm isotropic spatial resolution, with 32 gradient directions, and a b-value of 1000s/mm 2 . Figure 3

Human data

A sample of 39 subjects of the BIL&GIN (Brain Imaging of Lateralization by the Groupe d'Imagerie Neurofonctionnelle) human brain dataset [START_REF] Mazoyer | BIL&GIN: A neuroimaging, cognitive, behavioral, and genetic database for the study of human brain lateralization[END_REF][START_REF] Chenot | A population-based atlas of the human pyramidal tract in 410 healthy participants[END_REF] was used to gauge the performance of FINTA.

Acquisitions were done with a Philips Achieva 3 Tesla MR scanner using 21 non-colinear diffusion gradient directions and a diffusion gradient encoding strength of 1000s/mm 2 with four averages and an isotropic spatial resolution of 2mm. In this work, only the corpus callosum was considered, and within the bundle, homotopic streamlines lying within any of 26 pairs of gyral-based segments were considered as "plausible" streamlines. Additional details about the gyral-based callosal segments used for the extraction of the streamlines, shown in figure 3(c), are provided in section A.5.

(a) (b) (c) (d) (e) (f)

Experimental

The same experiment design was implemented for the three datasets. Noise-free diffusion data were used for the "Fiber Cup" and ISMRM 2015 Tractography Challenge datasets, and no denoising procedure was required for the BIL&GIN dataset. A Constrained Spherical Deconvolution (CSD) [START_REF] Tournier | Robust determination of the fibre orientation distribution in diffusion MRI: Non-negativity constrained super-resolved spherical deconvolution[END_REF] method using a spherical harmonics order of 6 was used to extract the fiber ODFs from the raw dMRI data. The probabilistic setting of the Particle Filtering Tracking (PFT) method by [START_REF] Girard | Towards quantitative connectivity analysis: reducing tractography biases[END_REF] using white matter seeding and whole brain tracking were employed for the "Fiber Cup" and the ISMRM 2015 Tractography Challenge datasets to generate the sets for the autoencoder. The extracted streamlines were processed and scored according to the procedure in Maier-Hein et al. ( 2017) to obtain the "(anatomically) implausible streamline" and "(anatomically) plausible streamline" sets. For the ISMRM 2015 Tractography Challenge dataset, the anterior commissure (CA) and the posterior commissure (CP) bundles were left out of the experiments since the tracking algorithm was unable to extract them or the extracted bundles did not meet the scoring system criteria. Hence, 23 bundles were used in the experiments.

As a result of this procedure, the "implausible" sets on the "Fiber Cup" and ISMRM 2015 Tractography Challenge datasets were comprised of streamlines that did not reach the cortical surface (or the corresponding terminal regions in the "Fiber Cup"), streamlines whose trajectories described sharp bends, and streamlines connecting regions that were not actually connected in the ground truth (see examples of such trajectories in figure 1).

The probabilistic setting of the PFT method was likewise used for the BIL&GIN human brain dataset. The resulting tractograms were registered [START_REF] Avants | A reproducible evaluation of ants similarity metric performance in brain image registration[END_REF] to the MNI template common space using the MNI152 2009 standardspace T1-weighted average structural 1mm isotropic resolution template image [START_REF] Fonov | Unbiased average ageappropriate atlases for pediatric studies[END_REF]. For the experiments in this work, only the callosal subset of the generated tractograms was used. The "plausible" streamlines within this subset consisted of the streamlines within 26 gyral-based homotopic callosal segment pairs (see section A.5 for further details about their definition). The "implausible" set comprised, as for the "Fiber Cup" and ISMRM 2015 Tractography Challenge datasets, streamlines that did not reach the cortical surface, describing sharp bends, having a single endpoint in the callosal region, having endpoints in non-white matter tissues, or connecting both hemispheres through non-callosal segments or tissues (e.g. the brainstem). The procedure was supervised by a neuroanatomist. The procedure yielded, on average, 41838 streamlines per-subject (standard deviation: 5648) for the considered callosal dataset. The lower row in figure 3 shows the test set tractograms, including both "plausible" and "implausible" streamlines, corresponding to all three datasets.

The ratio of "implausible" streamlines over the resulting tractograms was in-line with the findings in previous studies: 79% for the "Fiber Cup" dataset (similar to the figures reported by [START_REF] Côté | Tractometer: Towards validation of tractography pipelines[END_REF] for probabilistic methods); 46% for the ISMRM 2015 Tractography Challenge dataset (see the valid connection ratios across submissions reported in Maier-Hein et al. ( 2017)); and 89% on average (standard deviation: 2%) for the callosal BIL&GIN dataset. All streamlines were resampled to contain an equal number of points ( 256) and their endpoints were aligned for each dataset. The obtained streamlines were split randomly into training/validation and test sets using an 80%/20% ratio. The proposed convolutional autoencoder was trained in an unsupervised manner using the training split of both the "plausible" and "implausible" streamlines issued by the probabilistic tracking methods. The filtering threshold was set by finding the optimal point in the receiver operating characteristic (ROC) curves evenly rewarding true positives and penalizing false positives.

Comparison to other filtering methods

The quantitative comparison was performed on the callosal BIL&GIN dataset. FINTA was compared to both anatomybased tractography filtering methods and two variants (single-bundle and multi-bundle) of RecoBundles [START_REF] Garyfallidis | Recognition of white matter bundles using local and global streamline-based registration and clustering[END_REF][START_REF] Rheault | Analyse et reconstruction de faisceaux de la matière blanche[END_REF], a state-of-the-art method applicable to partial data. The anatomy-based filtering methods were applied successively, each one filtering the previous step's output, as used in several works such as Maier-Hein et al. ( 2017); [START_REF] Sarubbo | Uncovering the inferior fronto-occipital fascicle and its topological organization in non-human primates: the missing connection for language evolution[END_REF]. The purpose of including the results of each step of this filtering pipeline (baseline filtering methods #1 to #4) was to show the successive improvement of the classification performance scores by using anatomical heuristics. The models of "plausible" streamlines used for the RecoBundles experiments were built according to [START_REF] Rheault | Analyse et reconstruction de faisceaux de la matière blanche[END_REF], and based on the 26 gyral-based homotopic callosal segment pairs (see section A.5). The multi-bundle RecoBundles experiment considered each gyrus-wise homotopic callosal streamline model as a separate, parameterizable entity, whereas the single-bundle RecoBundles method considered their concatenation as a single target group. This yielded a total of 6 tractography filtering methods as baselines:

1. filtering based on streamline length (named length within this text);

2. filtering with loop removal (no_loops);

3. filtering with early stops in cerebrospinal fluid (no_end_in_csf ); 4. filtering with early stops in the white matter (end_in_atlas); 5. single-bundle RecoBundles (recobundles_single);

6. multi-bundle RecoBundles (recobundles_multi).

To estimate the ability of the methods to preserve the underlying anatomy in terms of the gyrus-wise callosal streamline homotopic connections kept by the predicted positives, a valid gyrus-wise homotopic callosal streamline group count (shortened as VGW for the sake of compactness) rate was computed as the average ratio of the preserved streamline groups to the existing ones. The streamlines were recovered from the positives predicted by each method using the anatomical criteria defined for each segment. To estimate the task performance uniquely, a success rate (SR) measure was computed as the unweighted average of the classification measures and the valid gyrus-wise homotopic callosal streamline group count rate.

Time requirements

The filtering clock time of FINTA was measured on each of the datasets. For quantitative comparison purposes, tractograms of different sizes were generated from the ISMRM 2015 Tractography Challenge data using the same procedure described earlier. In total, six (6) different tractograms were generated containing 20 000, 40 000, 100 000, 200 000, 600 000, and 1 000 000 streamlines, respectively. The time required to filter each of these tractograms was measured for both FINTA and RecoBundles three (3) times, and the mean and standard deviation values were computed for each tractogram and method. In all cases, solely the time required for filtering was measured, excluding I/O operation time. All time tests were performed on a conventional desktop machine (Intel(R) Xeon(R) W-2133 CPU @ 3.60GHz 6 core processor; 16 GB RAM; NVIDIA GeForce GTX 1080 Ti 12 GB graphics card).

Results

Tractogram latent space

The autoencoder's latent space allows to re-interpret the streamline pair-wise distance. For illustrative purposes, a number of randomly picked "plausible" streamlines were assigned to their corresponding bundle (or gyrus-wise homotopic callosal streamline group for BIL&GIN), and together with some "implausible" streamlines, they were projected into the latent space of a trained autoencoder. Figure 4 shows a view of the latent spaces corresponding to each of the three datasets. The views were generated by applying a t-SNE dimensionality reduction (van der Maaten and [START_REF] Van Der Maaten | Visualizing data using t-SNE[END_REF] to the autoencoder's latent space vectors. The figure suggests that streamlines belonging to the same bundle (or streamline group) lie at a short distance from each other whereas "implausible" streamlines are uniformly distributed. Similarly, it provides evidence supporting that the autoencoder setting can be generalized and give way to multi-label streamline classification tasks such as bundling. Furthermore, the application of mathematical tools in the latent space is straightforward. Figure 5(a) shows the interpolation principle applied to the latent space. Interpolating in the latent space allows to explore the space through a series of intermediate representations between any number of input streamlines of choice. Figure 5(b) and (c) show streamlines interpolated in the latent space between streamline pairs corresponding to two distinct sets of bundles of the "Fiber Cup" dataset. This example stresses the underlying significance of the latent space streamline pair-wise distance. Our proposed algorithm relies on the underlying hypothesis that the Euclidean distance between the latent representation of two streamlines is a good proxy to measure their structural similarity. Thus, the fact that a linear interpolation between two latent vectors leads to a smooth transition between their reconstructed streamlines underlines that the learn manifolds are smooth and that the latent Euclidean distance is a good similarity metric.

The reconstruction of the streamlines at the output of the autoencoder involves decoding the samples in the latent space. Section A.1 shows the reconstructed tractograms corresponding to the "Fiber Cup" test set streamlines.

"Fiber Cup" synthetic data

Figure 6 (a) and (c) show the nearest neighbor latent space distance distribution histograms and within-class mean and variance for the "Fiber Cup" dataset. The figure illustrates the filtering principle of FINTA based on the boundary between the inliers and outliers: the "plausible" streamline within-class nearest neighbor distance is low (close to 0 for the "Fiber Cup") and shows a reduced variance. As a contrast, the "implausible" streamlines show a much higher mean distance to their nearest neighbor streamline, as well as a higher variance. Note that the distance measure is computed considering the "plausible" streamlines from the train set as the reference. As quantitatively shown in table 2, the filtering distance threshold obtained by means of the autoencoder-based strategy allows to filter out the undesired "implausible" streamlines with a high degree of success. The predicted positive streamlines for the "Fiber Cup" are shown in figure 7. These are the streamlines considered as "plausible"s by FINTA from the test set, and hence include both the true positives and the false positives. Qualitatively, looking at the tractogram bundle color code information, and in agreement with the quantitative results, the "implausible"s seem to have been substantially filtered out, implying a decreased influence of the false positives. Figures 8(a) and (b) show, respectively, the false positive and false negative streamlines as detected by FINTA for the "Fiber Cup" dataset, and color-coded according to the latent space nearest neighbor distance. From the figures and the corresponding highlighted areas it can be drawn that, at least for part of the streamlines, FINTA has identified that these streamlines were misclassified in the reference set: in figure 8(a) a large part of the highlighted streamlines seem to be close to an anatomically "plausible" streamline within the bundle at issue. Many of these false positives are labeled so because they end 1 or 2 voxels away from the terminal regions (which would correspond to the cortical surface or gray matter border on human subject brain data). Conversely, part of the highlighted streamlines in figure 8(b) describe trajectories that could have been identified as anatomically "implausible" due to erratic bends. This illustrates how close from the ground truth our results are.

FINTA takes around 0.2s to filter the "Fiber Cup" test tractogram.

ISMRM 2015 Tractography Challenge human-based synthetic data

The nearest neighbor latent space distance histograms and within-class mean and variance for the ISMRM 2015 Tractography Challenge dataset are shown in figure 6(b) and (d). As it is the case with the "Fiber Cup" dataset, although a larger degree of overlap between classes is observed, our autoencoder-based filtering strategy allows to obtain a threshold to filter out the undesired "implausible" streamlines with a degree of success of approximately 91% over all measures (see table 2). Here again, the latent space similarity metric shows the same behavior in both experiments: "implausible" streamlines have a larger distance to the reference nearest neighbor.

For qualitative visualization purposes, figure 9 shows the streamline-wise classification tractograms for some selected cases corresponding to the ISMRM 2015 Tractography Challenge dataset test set, color-coded according to their nearest neighbor distance in the latent space. In particular, "plausible" predictions corresponding to the the corticospinal tract (CST) and the optic radiation (OR) bundles are shown, together with an example corresponding to "implausible" streamlines. As it can be drawn from the scale bar, the latent space dissimilarity distances for the true positive samples (a) to (f) are below the distance threshold determined shown in figure 6(b, d) (13.634) while the distances for the "implausible" predictions shown are above that threshold. The filtered whole brain tractogram is shown in A.2.

FINTA takes around 12s to filter the ISMRM 2015 Tractography Challenge dataset test set.

(a) (b) (c) (d) (e) (f) (g) (h) (i)

Human data

FINTA's filtering ability on in vivo human data is consistent with the results on synthetic datasets. The quantitative results for the comparison to other filtering methods are shown in table 3, and plot in figure 10. As it can be drawn from these results, FINTA achieves a better classification performance than its competitors. Similarly, compared to any of the baseline methods, it is observed that FINTA shows a reduced variability in all measures consistently, with the sole exception of the valid gyrus-wise homotopic callosal (VGW) streamline group count rate. Supplementary results of the comparison to the baseline filtering methods on the callosal BIL&GIN dataset are reported in the appendix sections A.3 and A.4.

The recovery of the existing anatomy, measured by the gyrus-wise homotopic callosal streamline group recovery rate, requires to be interpreted with care. Although results show that the anatomy-based baseline methods show a superior ability in this aspect, this comes at the expense of significantly lower scores for the rest of the measures (see section 5.2 for a discussion). Yet, FINTA still achieves an average recovery rate of 80%, performing coherently across all measures. 11 shows that, compared to FINTA, the single-bundle RecoBundles method misses streamlines in the anterior, ventral part of the frontal lobe, and includes false positives in the occipital lobe (pointed regions).

Table 4 reports a gyrus-wise analysis on the callosal BIL&GIN dataset of the sensitivity measure on the predicted positives for FINTA. The reported data correspond to the averaged values over the test subjects. Given that the callosal BIL&GIN dataset gyral-based segments allow solely to extract homotopic streamlines, only the sensitivity is reported (see the appendix section A.6 for further details). The gyral-based Hippo and TPole segments are not reported since none of the test subjects contain "plausible" streamlines in such segments. Similarly, the mean streamline count being rounded to the nearest integer, some segments contain 0 streamlines on average (IOG, Ins, LFOG, MFOG, and PHG) and hence are not reported. Also, note that some others have a very low (as low as 1) streamline count, and thus missing a few streamlines bears a significant impact on the results. That is the case, for example, for the FuG segment, where 2 of the test subjects have both 2 streamlines, and another 2 have both 1 streamline, the remaining 4 lacking streamlines in that segment. As seen in the table, the sensitivity values for these are noticeably lower than for the segments containing a high streamline density, such as the MFG, PrCu, or SFG, all of the latter being above 86%. FINTA takes around 21s, on average, to filter a callosal tractogram corresponding to a BIL&GIN test subject. The single-bundle RecoBundles baseline method takes, on average, 66s, in our experiments.

Time requirements

Figure 12 compares the time required by FINTA and RecoBundles to filter tractograms of different sizes corresponding to the ISMRM 2015 Tractography Challenge data. For each method, the curves account for the mean filtering time (standard deviation values are below 10s in all cases). FINTA requires less time than RecoBundles to filter a tractogram of the same size. Also, it follows from the curves that FINTA is linear in terms of the streamline count in a tractogram, whereas RecoBundles requires a time that tends to be several order of magnitudes longer than FINTA as the streamline count increases.

Discussion

The results across the three different datasets show that FINTA is a state-of-the-art alternative for tractography filtering.

The latent space views suggest that streamlines belonging to the same class lie close to each other in the latent space, supporting the hypothesis that the autoencoder successfully clusters such streamlines in an unsupervised manner. The proposed unsupervised filtering method achieves good performance statistics on both synthetic and in vivo human brain data, and consistently ranks higher in the considered measures compared to the baseline methods.

Figure 12: Filtering time requirements for FINTA and RecoBundles. The curves demonstrate that FINTA is fast, having linear time complexity with the streamline count, while RecoBundles requires a significantly longer time to filter a given tractogram. Note that due to the vertical scale and reduced standard deviation values, the latter are hardly noticeable around the mean value. Similarly, note that the streamline counts are expressed with SI prefixes and engineering notation.

Performance on synthetic datasets

Although less convoluted than a tractogram derived from a human brain dataset, the "Fiber Cup" dataset still presents some challenges to tractography filtering methods, and hence it offers a good test bed to show the behavior of a given method. Our results show that FINTA provides almost perfect performances on it. Additionally, the dataset allows to gain further insight into the behavior of a given filtering method. Our findings illustrated in figure 8 suggest that:

(1) false positives and false negatives generated by FINTA are very close to true positives and true negatives; and

(2) in some cases, FINTA can reveal examples of streamlines that were misclassified by the scoring method. Such inconsistencies with respect to the reference highlight the ability our proposed method has to exploit the learned features from the streamlines. At the same time, it is a demonstration of some potential limitations of conventional filtering approaches, which employ voxel-based or anatomical criteria, and pair-wise streamline dissimilarity measures that parallel those employed by existing scoring methods.

The slight decrease in performance registered by FINTA on the ISMRM 2015 Tractography Challenge (also seen in the human in vivo callosal BIL&GIN dataset) compared to the "Fiber Cup" is explained by the complexity of the former, where a larger variety of bundle and streamline configurations are present. Nonetheless, with a mean accuracy of 91%, our method still gets an excellent score. Together, the results show that there is a good agreement between the performance on the ISMRM 2015 Tractography Challenge and callosal BIL&GIN datasets, suggesting that these results are fully descriptive of the power of the method on human in vivo tractogram settings.

Performance on human data

Although the anatomy-based tractography filtering methods (methods #1 to #4) were expected to record lower scores than their competitors, they are still an important part of routine tractography filtering pipelines prior to downstream tasks, such as connectivity studies [START_REF] Jeurissen | Diffusion MRI fiber tractography of the brain[END_REF][START_REF] Yeh | Mapping structural connectivity using diffusion MRI: challenges and opportunities[END_REF]. No additional steps, such as obtaining the gyral-based homotopic streamlines, were added to avoid reproducing the scoring method providing the reference.

Classical filtering methods obtain a better gyrus-wise streamline group recovery rate than FINTA at the cost of including an excessive number of false positives among the kept streamlines. Penalizing more severely missing groups would have had some impact on the reported success rates without harm to the rest of the filtering performance measures.

Similarly, the superior ability of the RecoBundles bundle recognition methods to recover the existing anatomy may be explained by the use of gyrus-wise homotopic callosal streamline models specifically built for that purpose. The single-bundle RecoBundles obtained slightly better scores than the multi-bundle version except for the valid gyrus-wise homotopic callosal streamline group count rate measure. This might be explained by a greed of the multi-bundle version to assimilate streamlines to every model of the corpus callosum gyral-based segment pairs, leading it to an excessive number of false positives. Compared to FINTA, the improved gyrus-wise streamline group recovery rate of the multi-bundle RecoBundles baseline comes at the cost of potentially recognizing bundles that are not contained in the data (i.e. invalid bundles).

Note that the valid gyrus-wise homotopic callosal streamline group count does not account for the potential invalid (i.e. not existing in the underlying anatomy) groups that some of the methods may recover. It must be noted that the 26 pairs of gyral-based callosal segments (see the appendix section A.5) are representative of the entire BIL&GIN population, but subjects may be missing streamlines on an entire segment. Furthermore, the streamline count on some segments may be as low as a single streamline for some subjects, which may impact considerably the gyrus-wise group count recovery rate.

In this work, true positives and false positives were evenly weighted to find the optimal filtering threshold. We found that maximizing only the accuracy penalized excessively the least numerous class in a class imbalance situation. Likewise, the results in figure 10 show that this strategy leads to a high specificity, and allows to keep false positives at low rates in FINTA. Meanwhile, the rest of methods suffer from an excessive number of false positives. Note that this effect is observed consistently in the measures where false positives are involved (accuracy, F1-score, and precision). This finding is relevant for connectome studies, where keeping a good specificity is essential [START_REF] Zalesky | Connectome sensitivity or specificity: which is more important?[END_REF].

The reduced variability in the measures shown by FINTA suggests that the method is less prone to be influenced by the local, potentially noisy streamline trajectories. Conventional methods' ability to filter streamlines is determined by local decisions based on either local (e.g. curvature, regions of interest, endpoint location) or non-local (e.g. length) features, and thus a single streamline coordinate not complying with the given criterion (e.g. falling outside a structural mask) is enough to filter out the streamline. This reaffirms the ability of FINTA to encode those features of the input data which are most relevant for an accurate reconstruction.

The corpus callosum spans a large part of the brain volume in the anterior-posterior axis across both hemispheres; it is one of the largest fiber systems in the white matter, both in terms of the absolute number of streamlines and the occupied brain volume. Compared to some other white matter systems, it presents a less compact configuration. As a consequence, it is challenging to precisely delineate the corpus callosum homotopies in the reference set [START_REF] Benedictis | New insights in the homotopic and heterotopic connectivity of the frontal portion of the human corpus callosum revealed by microdissection and diffusion tractography[END_REF]. Similarly, as previously noted, the 26 gyral segment pairs used in this work differ greatly in the streamline density they hold, and thus bear an impact on the gyrus-wise sensitivity scores shown in table 4. Yet, FINTA records high classification performance scores, and shows a reduced variability at the gyrus-wise streamline group scale.

General considerations

As opposed to the conventional filtering methods that assume that the estimated streamline population at every voxel should be proportionally supported by the diffusion data (which is the case of COMMIT/COMMIT2, LiFE, or SIFT/SIFT2), our method is able to work on tracking data only. This makes our method immune against well-known domain adaptation issues [START_REF] Perone | Unsupervised domain adaptation for medical imaging segmentation with self-ensembling[END_REF]. Furthermore, compared to a quadratic or super-quadratic complexity of some methods, FINTA is linear in terms of the streamline count at test time as demonstrated by the results in figure 12.

Compared to other deep learning approaches that may be specifically oriented to classification tasks (e.g. regular classification convolutional neural networks), FINTA offers the benefit of being an unsupervised approach. Thus, it does not depend on the number of classes in the input data as the output of the network does not look for maximizing the probability of a given class among the possible ones. As such, the method is better suited to the reality of tractography, where there is a limited knowledge about the ground truth [START_REF] Yeh | Correction for diffusion MRI fibre tracking biases: The consequences for structural connectomic metrics[END_REF][START_REF] Sotiropoulos | Building connectomes using diffusion MRI: why, how and but[END_REF], and where the analysis lends itself to different organizational levels, and thus different classification degrees. As such, our method naturally adapts to new sets of classes without the need of having to retrain the network.

A potential side-benefit of the proposed autoencoder approach is that the reconstructed streamlines describe a locally smooth trajectory. Although not explicitly used or exploited in the present work, downstream pipeline or visualization processes may benefit from it, providing a less complex or more realistic long-range apparent fiber trajectory representation.

How good is the ground truth in tractography?

Inter-subject variability, and the lack of a "gold standard" to reliably determine bundles on in vivo human brain data (Rheault et al., 2020b) add a degree of uncertainty to tractography and downstream tasks. Our work demonstrates the ability of a 1D convolutional autoencoder to reveal uncertainties inherent to current state-of-the-art tractography filtering methods. Our filtering method is a powerful tool to uncover disparities and biases in current data scoring practices that can otherwise go unnoticed. The insight gained from such a model can be useful to build upon existing knowledge in tractography. Re-defining some of the current tractography filtering approaches may be essential when dealing with human data, where consensus on the best predictors of the underlying white matter anatomy is still hard to achieve. Together, this also highlights the need of a larger set of validated, shared data to converge towards a more quantitative tractography approach.

Future work

The proposed architecture requires all streamlines to have an equal number of points. A limitation of this work is that this number is larger than the minimal number of points required to express a streamline in regular tractography pipelines [START_REF] Presseau | A new compression format for fiber tracking datasets[END_REF]. Further work is needed to determine a more efficient characterization of a streamline in terms of a representation learning framework. Similarly, we did not study the effect of the number of parameters of the network, and only focused on the optimization of the reconstruction accuracy. Additional investigations are required to quantify the impact of the network size on the reconstructed streamlines and filtering performance.

We did not investigate the streamline or tractography features the autoencoder-based approach focuses on. Further work is required to gain more insight on the data aspects that the model is able to capture or tends to discard. This is still an active area of research in deep learning, where many efforts are being put to explain how a model behaves in terms of explainable concepts related to the data and its features.

Additional work includes confirming the robustness of the method against particular anatomical conditions. This involves, for example, measuring the ability of our method to generalize on infant brain tractography data or on brain data presenting a condition, such as a tumor, potentially affecting the tractogram distribution, without having to re-train the system. These experiments so could lead to further classification categories or to the need of a conditional mechanism attached to the autoencoder.

Conclusions

We have proposed an unsupervised, deep learning-based dimensionality reduction method for dMRI tractography filtering. We have dubbed our method FINTA, Filtering in Tractography using Autoencoders. We have shown that, working with a simple architecture, FINTA is able to robustly learn the structure of streamlines in tractography, and have applied such a framework to discriminate between an input tractogram's "positive" and "negative" streamlines. These categories may target "plausible" and "implausible" streamlines or a subset of bundles versus the rest of the tractogram. For the "plausible" versus "implausible" setting, once the neural network has been trained on unlabeled tractograms, simply labeling a subset of the streamlines of interest to obtain the partitioning threshold suffices to use this value to filter new tractograms using FINTA. We have demonstrated that the method allows to successfully and reliably filter tractograms in this context with accuracies above the 90% bar, obtaining improved scores compared to state-of-the-art filtering methods. To this end, we have proposed a mean for redefining the notion of inter-streamline distance which can be similarly used for any streamline filtering tasks without the need of having to re-train the network.

The proposed FINTA framework offers a fast, robust, data-driven approach for tractography filtering. Its unique characteristics unfold the potential to improve the accuracy and reliability of downstream tractometry and connectomics derivatives. Thus, it may assist neuroanatomists in better describing the white matter anatomy using tractography, and hence bring a significant positive impact to research in neuroscience. FINTA is expected to be integrated into a specialized diffusion MRI analysis open source toolkit.

A Appendix

A.1 Reconstruction

Figure 13 demonstrates the result of the autoencoder reconstruction corresponding to a subset of the "Fiber Cup" dataset test set streamlines. The considered classes ("plausibles" and "implausibles") are shown separately according to the FINTA filtering procedure. The figures show that the "plausible" streamlines match the expected bundle-wise color code information from the synthetic ground truth streamlines (see figure 3(a)). The "implausible" set can be seen to contain streamlines whose tracking had prematurely stopped within the tissue of interest, as well as streamlines connecting regions that are not connected in the synthetic ground truth. Additionally, and as anticipated by the autoencoder theory, it can be seen that the reconstructions are a smoothed version of the input tractogram streamlines in figure 3(d) (also visible in figure 5) .

A.3 Human data filtering supplementary results

Figure 15 shows the ROC curves corresponding to the callosal BIL&GIN dataset subjects used to compute the filtering threshold. As it can be concluded from the curves, FINTA is consistent at filtering the tractograms across subjects. At the same time, results are in agreement with the experiments performed in the synthetic datasets. Pre-Central Gyrus (PrCG); Pre-Cuneus (PrCu); Rectus Gyrus (RG); Superior Frontal Gyrus (SFG); Supramarginal Gyrus (SMG); Superior Occipital Gyrus (SOG); Superior Parietal Gyrus (SPG); Superior Temporal Gyrus (STG); and Temporal Pole (TPole). The gyrus-wise homotopic callosal streamline models, used in the RecoBundles filtering experiments, were built using the aforementioned procedure and according to the method described in Rheault (( 2020)) over the entire BIL&GIN population (411 subjects). Figure 16 provides an overview of the considered models.

Figure 16: Callosal BIL&GIN dataset gyrus-wise homotopic streamline models. All sagittal left views.

A.6 Callosal BIL&GIN gyrus-wise analysis

The gyral segment pairs defined in section A.5 for the callosal BIL&GIN dataset solely allow the extraction of homotopic streamlines. Hence, by definition, no "implausible" streamline can be recovered using the constraints established by these criteria. As a consequence, the only measure that can be reported for the predictions over the gyral segments is the sensitivity (see table 4). The mean number of segments with a non-zero streamline count in the reference data was 17 for the test subjects. Similarly, there was a non-negligible number of segments whose streamline density was very low compared to the most heavily populated ones. Test subjects averaged 5287 (standard deviation: 1450) homotopic streamlines across all segments, and the number of segments that had a mean streamline count larger than 264 (5% of the mean homotopic streamline count) was 3: MFG; PrCu; and SFG.

Figure 1 :

 1 Figure 1: Schematic representation of "plausible" and "implausible" streamlines: (a) "plausible" streamlines belonging to the corpus callosum; (b) "plausible" streamlines belonging to the corticospinal tract; (c) "implausible" streamline containing a loop or a sharp bend; (d) "implausible" streamline stopped at the cerebrospinal fluid; (e) "implausible" streamline stopped in the white matter (or hitting a white matter or atlas boundary); (f): "implausible" streamline describing an invalid bundle.

Figure 2 :

 2 Figure 2: Schematic representation of tractography filtering with FINTA: (a) training time; (b) test time. Although 1D convolutions are used in the proposed framework, convolutional blocks are depicted using a 2D shape for illustrative purposes.

  (b) shows the sagittal left view of the ground truth tractogram for the ISMRM 2015 Tractography Challenge dataset.

Figure 3 :

 3 Figure 3: Tractograms corresponding to the considered datasets. Upper row: (a) ground truth tractogram from the "Fiber Cup" dataset; (b) ground truth tractogram from the ISMRM 2015 Tractography Challenge dataset; (c) sagittal left view of the gyrus-wise homotopic callosal streamline models corresponding to the BIL&GIN dataset. Lower row: test sets corresponding to the tracking performed on each of the three datasets: (d) view of the "Fiber Cup" test set containing 4963 streamlines; (e) sagittal left view of ISMRM 2015 Tractography Challenge test set containing 63615 streamlines; (f) sagittal left view of a randomly picked test subject corresponding to the BIL&GIN dataset and containing 55160 streamlines.

Figure 4 :

 4 Figure 4: Latent space views using a t-SNE dimensionality reduction. (a) "Fiber Cup" data; (b) ISMRM 2015 Tractography Challenge data; (c) Callosal BIL&GIN human brain data. The number of projected streamlines was limited in each case for visualization purposes. The callosal BIL&GIN data corresponds to a randomly picked subject which was missing all streamlines in 8 of the gyral-based segments.

Figure 5 :

 5 Figure 5: Latent space interpolation. (a) Schematic representation of latent space streamlines interpolation. The streamline representation in the latent space allows to choose a pair of encoded streamlines and fill the space between them through a series of interpolated streamlines. (b) and (c) "Fiber Cup" latent space streamline interpolation: streamlines interpolated between the instances at the boundaries of each series: (b) bundles 1 and 5; (c) bundles 4 and 7 (see the bundle numbering in Côté et al. (2013)). Only the structural shapes of the bundles used for the interpolation are shown for the sake of clearness. The streamlines are shown with a larger diameter with respect to the one used for the rest of the figures.

Figure 6 :

 6 Figure 6: Nearest neighbor latent space distance distribution: histogram (a) and within-class mean and variance (c) corresponding to the "Fiber Cup" dataset test set streamlines; histogram (b) and within-class mean and variance (d) corresponding to the ISMRM 2015 Tractography Challenge. The dashed lines indicate the filtering threshold distance in each case. Note that the vertical axes are in logarithmic scale.

Figure 7 :Figure 8 :

 78 Figure7: Predicted positives on the test set corresponding to the "Fiber Cup" dataset. Note that, in contrast to the input dataset shown in figure3(d), the filtered tractogram shows a bundle color code information that matches better the "Fiber Cup" ground truth shown in figure3(a), implying that "implausible" streamlines have been eliminated to a large extent.

Figure 9 :

 9 Figure 9: Streamline classification predictions corresponding to some cases of the ISMRM 2015 Tractography Challenge dataset test set: (a), (b), (c) corticospinal tract (CST); (d), (e), (f) optic radiation (OR); (g), (h), (i) example corresponding to "implausible" streamlines. All axial superior, coronal anterior and sagittal left views. Streamlines are color-coded according to their nearest neighbor distance in the latent space. The dashed white lines on the scale bars mark the filtering threshold. The streamlines corresponding to the "implausible" class are shown with a larger diameter with respect to the one used for the rest of the figures.

Figure 10 :Figure 11 :

 1011 Figure 10: Classification performance measures on the callosal BIL&GIN dataset. For each of the compared filtering methods, the mean and standard deviation of the measures are presented. The horizontal dotted lines indicate FINTA's minimum and maximum scores across the measures.

Figure 13

 13 Figure 13: "Fiber Cup" test set filtered streamlines as reconstructed by the autoencoder: (a) "plausible" streamlines; (b) "implausible" streamlines. The number of streamlines displayed is lower than the actual number in the test set for illustrative purposes.

Figure 14 :

 14 Figure 14: Predicted positives on the test set corresponding to the ISMRM 2015 Tractography Challenge dataset. (a) Axial superior view; (b) coronal anterior view; (c) sagittal left view. Although due to the streamline density the visualization can be difficult to interpret, the middle cerebellar peduncle (pointed bundle) clearly shows that the filtered tractogram has an orientation information that matches better the dataset ground truth shown in figure 3(b) in comparison to the raw, unfiltered tractogram in figure 3(e).

Figure 15 :

 15 Figure 15: Callosal BIL&GIN dataset receiver operating characteristic (ROC) curves. The ROC curve shows the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. The figure shows the curves for the subjects used to compute the filtering threshold. The area under the curve (AUC) is indicated for each subject.

Table 1 :

 1 FINTA autoencoder structure. The encoder uses strides of size 2, and the decoder uses strides of size 1. The decoder's upsampling stages use an upsampling factor of 2. The autoencoder uses ReLU activations throughout its convolutional layers.

	Part	Type	Features	Size
	input	-	-	3; 256
		1D conv	32	256
		1D conv	64	128
	encoder	1D conv 1D conv	128 256	64 32
		1D conv	512	16
		1D conv	1024	8
	latent space	fully connected	32
		upsampling + 1D conv	1024	8
		upsampling + 1D conv	512	16
	decoder	upsampling + 1D conv upsampling + 1D conv	256 128	32 64
		upsampling + 1D conv	64	128
		upsampling + 1D conv	32	256
	output	-	-	3; 256

Table 2 :

 2 Filtering performance for the "Fiber Cup" and the ISMRM 2015 Tractography Challenge datasets.

	Measure	Dataset
		"Fiber Cup" ISMRM 2015 Tractography Challenge
	Accuracy	0.99	0.91
	Sensitivity	0.99	0.91
	Precision	0.97	0.91
	F1-score	0.98	0.91

Table 3 :

 3 Callosal BIL&GIN dataset macro results. Macro mean (standard deviation) values over test subjects. The highest mean score is marked in bold face.

		Method	Accuracy m Sensitivity m Precision m F1-score m VGW rate SR m
	#1	length	0.14 (0.02)	0.51 (0.0)	0.56 (0.01) 0.13 (0.01) 0.98 (0.03) 0.46
	#2	no_loops	0.2 (0.02)	0.48 (0.02)	0.48 (0.02)	0.2 (0.02) 0.98 (0.03) 0.47
	#3	no_end_in_csf	0.44 (0.01)	0.56 (0.01)	0.53 (0.01)	0.4 (0.02) 0.93 (0.05) 0.57
	#4	end_in_atlas	0.74 (0.02)	0.61 (0.03)	0.56 (0.02) 0.57 (0.02) 0.92 (0.05) 0.68
	#5 recobundles_single 0.83 (0.02)	0.85 (0.03)	0.69 (0.02) 0.73 (0.02) 0.87 (0.1)	0.79
	#6 recobundles_multi 0.82 (0.03)	0.8 (0.03)	0.67 (0.01)	0.7 (0.02) 0.96 (0.04) 0.79
	#7	FINTA	0.91 (0.01)	0.91 (0.01)	0.78 (0.01) 0.83 (0.01) 0.8 (0.09)	0.84
	Qualitatively, figure

Table 4 :

 4

	Segment Count Sensitivity	Segment Count Sensitivity
	AG	14	0.14 (0.14) [0, 0.4]	PoCG	122	0.98 (0.03) [0.91, 1]
	Cing	23	0.81 (0.12) [0.63, 0.96]	PrCG	177	0.98 (0.01) [0.95, 1]
	Cu	6	0.08 (0.11) [0, 0.3]	PrCu	978	0.88 (0.05) [0.77, 0.94]
	FuG	1	0 (0) [0, 0]	RG	87	0.99 (0.01) [0.98, 1]
	IFG	12	0.5 (0.23) [0, 0.74]	SFG	3428	0.93 (0.01) [0.9, 0.95]
	ITG	1	0 (0) [0, 0]	SMG	10	0.64 (0.18) [0.45, 1]
	LG	4	0.2 (0.28) [0, 0.75]	SOG	23	0.52 (0.14) [0.25, 0.71]
	MFG	274	0.87 (0.28) [0.77, 0.93]	SPG	142	0.67 (0.22) [0.23, 0.91]
	MOG	9	0.21 (0.17) [0, 0.41]	STG	5	0.1 (0.17) [0, 0.5]
	MTG	4	0 (0) [0, 0]

Gyral segment-wise callosal BIL&GIN dataset sensitivity of FINTA. Mean (standard deviation; [min, max]) values over test subjects for FINTA. The Count column contains the mean streamline count for each segment across the test subjects.

Acknowledgments

This work has been partially supported by the Centre d'Imagerie Médicale de l'Université de Sherbrooke (CIMUS); the Axe d'Imagerie Médicale (AIM) of the Centre de Recherche du CHUS (CRCHUS); the Réseau de Bio-Imagerie du Québec (RBIQ)/Quebec Bio-imaging Network (QBIN) (FRSQ -Réseaux de recherche thématiques File: 35450); and the Samuel de Champlain 2019-2020 Program of the Conseil Franco-Québécois de Coopération Universitaire (CFQCU). This research was enabled in part by support provided by Calcul Québec (www.calculquebec.ca/en/) and Compute Canada (www.computecanada.ca). We also thank the research chair in Neuroinformatics of the Université de Sherbrooke.

A.4 Human data performance supplementary results

Tables 5,6, and 7 present supplementary results of the comparison to the baseline filtering methods on the callosal BIL&GIN human data: table 5 shows the weighted mean and standard deviation values, and tables 6 and 7 show the minimum and maximum values on the macro and weighted mean values. The weighted measures still show that FINTA provides an improved performance compared to the baseline methods. As mentioned, the latter yield an excessively large amount of false positives, and thus their valid gyrus-wise homotopic callosal streamline group count rates are generally higher than the values scored by FINTA. Nevertheless, FINTA still scores values over 0.68, with a maximum value of 0.94 across subjects.

A.5 The callosal BIL&GIN dataset

The callosal BIL&GIN dataset was composed of the streamlines passing through the corpus callosum in 39 participants of the BIL&GIN dataset. For each subject, callosal streamlines were extracted from a whole brain tractogram computed with the probabilistic setting of the Particle Filtering Tracking (PFT) method by [START_REF] Girard | Towards quantitative connectivity analysis: reducing tractography biases[END_REF]) as previously described in [START_REF] Chenot | A population-based atlas of the human pyramidal tract in 410 healthy participants[END_REF]). Each individual callosal tractogram was composed of anatomically "plausible" and "implausible" streamlines. The former were composed of gyral-based homotopic callosal streamlines, any other type of callosal streamline being considered as anatomically "implausible". Twenty-six (26) different types of gyral-based homotopic streamlines were considered depending on the homotopic gyri they belonged to, namely the 26 gyral regions of interest of the JHU template ( [START_REF] Zhang | Atlas-guided tract reconstruction for automated and comprehensive examination of the white matter anatomy[END_REF]). Each gyral ROI was composed of both cortical and superficial white matter parts of the gyrus as defined in the JHU template excepted for the insula, hippocampus and parahippocampal gyrus that have no superficial white matter part. The 26 gyral-based ROIs were therefore:


Andrei Ciortea
email: andrei.ciortea@unisg.ch

Simon Mayer
email: simon.mayer@unisg.ch

Olivier Boissier
email: olivier.boissier@emse.fr

Fabien Gandon
email: fabien.gandon@inria.fr

Exploiting Interaction Affordances: On Engineering Autonomous Systems for the Web of Things Position Paper for the Second W3C Workshop on the Web of Things

Keywords: Web of Things, Hypermedia, Multi-Agent Systems, Semantic Web

Interaction Affordances are central to the W3C Web of Things (WoT): they allow Consumers to identify and use the capabilities provided by Web Things. 1 Ideally, interaction affordances would allow consumers to arrive-and-operate in any W3C WoT environment: given an entry URI, consumers would be able to achieve their tasks in an autonomous manner by navigating the hypermedia and deciding among the various options presented to them at run time. A central challenge then, which is not typically within the scope of Web engineering, is how to design, program, debug, monitor, and regulate such autonomous consumers of Web Things. The engineering of similar autonomous systems has been studied to a large extent in research on multi-agent systems (MAS), and we believe that tapping into the large body of MAS research holds great promise for unlocking the full potential of the W3C WoT. In this position paper, we motivate and present our vision for autonomous systems in the WoT, and support this vision with a prototype for industrial manufacturing. We then discuss some of the challenges and opportunities raised by bringing autonomy to the WoT.

INTRODUCTION

The Web of Things (WoT) fosters innovation and rapid prototyping in the Internet of Things: developers can use standard Web technologies to build applications on top of mash-ups of devices and digital services (a.k.a. WoT mashups or physical mashups [START_REF] Guinard | Towards physical mashups in the web of things[END_REF]). Two paradigms that were very successful early-on in showcasing these benefits were inspired by dataflow programming and rulebased systems: in the former approach, developers and/or end-users would use a visual programming language to create directed graphs of devices and digital services (e.g., with Node-RED2 ), while in Second W3C Workshop on the WoT, June 3-5, 2019, Munich, Germany . the latter approach they would define event-condition-action rules where an event triggers the execution of one or more actions (e.g., with IFTTT3 ). Both approaches ease the development of reactive WoT systems that are very responsive to sensory input, but their main drawback is that -once deployed -these systems cannot adapt their behavior to changes in the environment or to new user requirements at run time: they are built on static WoT mashups that have to be defined and maintained manually. The W3C WoT helps mitigate these limitations to great extent through interaction affordances and hypermedia controls: it enables WoT mashups to be defined in terms of abstract interaction patterns rather than specific protocols and device APIs. The resulting WoT mashups are then more flexible as they are loosely coupled to the underlying device APIs, but they still have to be defined and maintained manually.

To avoid the use of static WoT mashups, some WoT researchers turned to automated planning [14,[START_REF] Mayer | Configuration of smart environments made simple: Combining visual modeling with semantic metadata and reasoning[END_REF]: given a design goal or a userspecified goal, it is left to the WoT system to infer how to achieve that goal using a reasoner and interaction affordances discovered at run time. Such systems are more adaptable (the WoT mashups are created at run time), but they are also less responsive: automated planning is computationally costly, and environmental changes are taken into account only before the planning phase (the inferred plan can become invalid during execution).

The W3C WoT already provides essential building blocks for engineering flexible WoT systems. We believe these building blocks, and in particular the introduction of interaction affordances, also hold the potential to unlock new practical use cases for autonomous systems on the Web [START_REF] Ciortea | A Decade in Hindsight: The Missing Bridge Between Multi-Agent Systems and the World Wide Web[END_REF]: many of the underlying research questions that we are now confronted with, such as how to balance reactive and goal-directed behavior in WoT systems or how to enable goal-directed behavior in resource-constrained systems, have already been explored to a large extent in the scientific literature on distributed artificial intelligence and, in particular, multi-agent systems (MAS) [START_REF] Weiss | Multiagent systems: a modern approach to distributed artificial intelligence[END_REF]. This area of research provides models, languages, and tools that could potentially be used to design, program, debug, monitor, and regulate systems of autonomous consumers of Web Things. In what follows, Section 2 briefly presents our vision for a new class of MAS for the WoT (and the Web in general), which we call Hypermedia MAS [START_REF] Ciortea | Engineering World-Wide Multi-Agent Systems with Hypermedia[END_REF][START_REF] Ciortea | A Decade in Hindsight: The Missing Bridge Between Multi-Agent Systems and the World Wide Web[END_REF] 4 . Section 3 then illustrates and supports this vision with a prototype for flexible industrial manufacturing [START_REF] Ciortea | Repurposing Manufacturing Lines on the Fly with Multi-agent Systems for the Web of Things[END_REF]. We discuss opportunities and challenges raised by bringing autonomy to the WoT in Section 4.

HYPERMEDIA MULTI-AGENT SYSTEMS

In AI research, an agent is commonly defined as "a computer system, situated in some environment, that is capable of flexible autonomous action in order to meet its design objectives" [START_REF] Jennings | Applications of Intelligent Agents[END_REF]. Autonomy is central to this definition and refers to the agent's ability to operate on its own, without the need of direct intervention from people or other agents. An autonomous agent thus has control over its actions and internal state. The agent is typically situated in an external environment that it can perceive via sensors and influence via actuators. 5 In distributed AI, a multi-agent system (MAS) is then a system conceptualized in terms of agents that are situated in a shared environment and interact with one another to achieve their design objectives [START_REF] Jennings | Applications of Intelligent Agents[END_REF][START_REF] Weiss | Multiagent systems: a modern approach to distributed artificial intelligence[END_REF].

We envision a new class of MAS that are aligned with the Web architecture to inherit the properties of the Web as a world-wide, open, and long-lived system. These envisioned Web-based MAS use hypermedia as a general mechanism to support uniform interaction among heterogeneous entities (e.g., agents, devices, digital services, knowledge repositories, organizations, datasets, any Web resource) -and we call them Hypermedia MAS [START_REF] Ciortea | Engineering World-Wide Multi-Agent Systems with Hypermedia[END_REF][START_REF] Ciortea | A Decade in Hindsight: The Missing Bridge Between Multi-Agent Systems and the World Wide Web[END_REF]. Figure 1 illustrates our vision: we conceive of Hypermedia MAS as socio-technical systems composed of people and autonomous agents -henceforth agents -that are situated and interact in a shared hypermedia environment distributed across the Web. To achieve their goals, agents navigate the hypermedia environment to discover other entities in the MAS that they could interact with and the interaction affordances provided by those entities. Agents could navigate and search the hypermedia environment themselves, or they could use hypermedia search engines (cf. Figure 1) -an approach that has proven successful on the Web.

Here, we consider the hypermedia environment as a first-class abstraction in the MAS, which is a view based on a long line of research in engineering MAS [START_REF] Ricci | Environment programming in multi-agent systems: an artifact-based perspective[END_REF][START_REF] Weyns | Environment as a first class abstraction in multiagent systems[END_REF]: it is a key component designed and programmed with clear-cut responsibilities, such as mediating interaction among agents and access to the deployment context (e.g., devices, digital services), and providing an abstraction layer for modeling, representing, and programming non-autonomous entities (e.g., artifacts that agents can use [START_REF] Ricci | Environment programming in multi-agent systems: an artifact-based perspective[END_REF]). In this vision, the Web provides the underpinning that interconnects all entities -within and across MAS.

DEMONSTRATOR: AGENT-BASED MANUFACTURING FOR THE WOT

To illustrate and support our vision for Hypermedia MAS in the context of the W3C WoT, in this section we present briefly a prototype for flexible industrial manufacturing that integrates automated planning with multi-agent oriented programming [4] for WoT systems. This prototype was developed at Siemens Corporate Technology and is described in detail in [START_REF] Ciortea | Repurposing Manufacturing Lines on the Fly with Multi-agent Systems for the Web of Things[END_REF]. A demonstrator video of our deployment is available online. 6

Flexible Assembly of Customized Furniture

The scenario considered for this prototype is the assembly of customized furniture. This market seems particularly appealing for flexible manufacturing systems both due to high potential and high demand for customization: even when restricting our scope to only a few different types of furniture, colors, leg configurations (e.g., three-legged vs. four-legged stools), and types of floor protector pads, the manufacturing line already faces several hundreds of product variations. This scenario thus warrants the use of manufacturing systems that are capable of switching between product variants and can be extended with additional industrial devices.

The concrete example we use is switching between the assembly of several different variants of a wooden stool: our system should be able to continuously manufacture stools with three legs; this behavior can then be interrupted by requests to assemble stools with a four-leg configuration, which does not require any adjustment of the manufacturing line itself but merely of the behaviors of involved agents (i.e., a manufacturing robot and a human worker, in our example); furthermore, the system should react appropriately to assembly requests for stools with padded legs (in any leg configuration) -in this case, the system should transparently add a third agent (e.g., a second robot) that delivers felt pads to the assembly station.

System Design and Implementation

Figure 2 depicts an overview of the layers of abstraction used in our approach. The various programming abstractions -agents, artifacts, organizations -were inspired by the JaCaMo meta-model for multiagent oriented programming [4]. We motivate and present in detail these abstractions in the context of the WoT in [START_REF] Ciortea | Beyond Physical Mashups: Autonomous Systems for the Web of Things[END_REF].

The agent environment (cf. Figure 2) is composed of workspaces, which are dynamic sets of artifacts that agents can use to achieve their goals. Artifacts can be both physical entities (e.g., devices on The various layers of abstraction in our agentbased manufacturing system for the WoT (sourced from [START_REF] Ciortea | Repurposing Manufacturing Lines on the Fly with Multi-agent Systems for the Web of Things[END_REF]).

the shop floor) and digital entities (e.g., an automated planner). Physical artifacts are loosely coupled to devices via W3C WoT Thing Descriptions (WoT TD). 7 To achieve their manufacturing goals, autonomous agents either use production plans that have already been programmed by an engineer, or they use an automated planner to synthesize plans from semantic descriptions of artifacts available in their environment at run time (the artifact manuals in Figure 2). All production plans, either programmed or inferred, are represented in AgentSpeak [START_REF] Rafael | Programming multi-agent systems in AgentSpeak using Jason[END_REF], a high-level programming language for cognitive agents. Production engineers can thus use the same programming language both for writing plans and for inspecting or editing plans synthesized by agents at run time.

We use multi-agent organizations [START_REF] Jomi | Instrumenting multi-agent organisations with organisational artifacts and agents[END_REF] (cf. Figure 2) to coordinate all agents in our system (i.e., factory workers and autonomous agents), and to hot-deploy manufacturing specifications at run time -which allows production engineers to repurpose the manufacturing system on-the-fly (see [START_REF] Ciortea | Repurposing Manufacturing Lines on the Fly with Multi-agent Systems for the Web of Things[END_REF] for details). Production engineers use a graphical Web front end to define and deploy manufacturing organizations, to configure and monitor the system, and to write, inspect, and edit production plans.

We implemented the autonomous agents, the artifacts they use, and the multi-agent organizations with the JaCaMo platform for MAS [4]. The semantic models used in our system (e.g., organizational specifications, agent descriptions) are stored and managed with the Open Semantic Framework (OSF) [START_REF] Mayer | An Open Semantic Framework for the Industrial Internet of Things[END_REF], a standardscompliant industrial knowledge management platform.

Deployment

We deployed the presented system in a prototypical production cell containing two handling robots that interact with human workers. The robots are controlled by systems that are representative for a state-of-the-art production cell in a real manufacturing environment. The first of the two handling robots is a Universal Robotics UR5 robot with an attached Robotiq 2-Finger Gripper that is connected to the rest of the system via ROS [START_REF] Quigley | ROS: an open-source Robot Operating System[END_REF], a Linux-based robot programming framework that includes modules for movement planning, 7 https://www.w3.org/TR/wot-thing-description/ optimization, and visualization. The second robot is a Fischertechnik ROBO TX Automation Robot that is controlled via an industrial controller (a Siemens S7-300 PLC) and is programmed by means of the STEP7 language using ladder logic. Both robot controllers expose REST HTTP APIs for integration with the rest of the system. 8 The robots interact with human workers through an augmented reality (AR) interface for Microsoft HoloLens 9 devices. 10 This setup allows us to demonstrate that our system enables collaborative robots (i.e., the UR5), which are widely perceived as important future participants in industrial production processes, to interact with people as well as with devices controlled via conventional factory automation systems such as PLCs. Furthermore, by building on top of the WoT, our system can be seamlessly extended with any Web-enabled devices that expose WoT TDs for their Web APIs.

AUTONOMY IN THE WOT: CHALLENGES AND OPPORTUNITIES

Weaving Hypermedia MAS into the open Web raises a broad range of technical, social, and ethical challenges (see also [START_REF] Ciortea | A Decade in Hindsight: The Missing Bridge Between Multi-Agent Systems and the World Wide Web[END_REF]). In this section, we discuss two immediate challenges that we believe are of particular interest in the context of the W3C WoT.

Interaction as a first-class abstraction

The W3C WoT TD and other recent initiatives such as Hydra 11 pave the way for declarative specifications of interactions on the Web: consumers of Web APIs no longer have to rely on hard-coded interactions, but instead they can reliably interpret and use interaction affordances provided to them at run time. Further evolutionary steps would be (i) to enable consumers of Web APIs to reason on how to achieve their design goals using the interaction affordances provided to them at run time, and (ii) to enable any software agent on the Web to discover and interact not only with Web APIs, but also with people and other agents on the Web. Ideally, interaction would become a first-class abstraction on the Web: both people and autonomous agents would be able to publish, discover, interpret, enact, and reason on interaction protocols, and when applicable to modify the protocols as they evolve throughout their lifetime. We believe this is particularly important in the context of the WoT due to the open and dynamic nature of WoT systems.

The declarative specification and enactment of interactions has been studied to large extent in MAS research (we refer the interested reader to [START_REF] Baldoni | Interaction Protocols[END_REF] for an introduction). One research opportunity would be to investigate the reuse of this previous work to further support and promote interaction as a first-class abstraction in the WoT (and on the Web in general). 8 For more technical details about the setup of the laboratory production cell we refer interested readers to [START_REF] Mayer | UberManufacturing: A Goal-Driven Collaborative Industrial Manufacturing Marketplace[END_REF]. 9 https://www.microsoft.com/en-us/hololens/ 10 Human workers are proxied by software agents that forward to humans all their obligations (e.g., to achieve a specific goal) and signal their fulfillment on behalf of humans. Obligations are displayed to human workers in textual form, and workers use voice commands to notify when they have fulfilled their obligations. 11 http://www.hydra-cg.com/spec/latest/core/

Regulation as a first-class abstraction

The Robots Exclusion Protocol (a.k.a. robots.txt) 12 allows website administrators to convey to crawlers policies for polite crawling. Rate limiting is also a common practice for Web APIs, but many times such policies are only presented in human-readable documentation. Another challenge for data consumers are licensing policies, which can hinder the reuse of data [START_REF] Villata | Licenses Compatibility and Composition in the Web of Data[END_REF]. We expect these challenges will remain relevant in the context of the WoT, in particular as consumers of Web Things become increasingly autonomous. To address these challenges, consumers should be able to reliably identify, interpret, and reason on any norms applicable to their interactions with Web Things and any other resources on the Web. If norms are expressed unambiguously and in machine-readable formats, autonomous behavior can then be monitored and regulated. Without such mechanisms, however, the benefits of bringing autonomy to the WoT may be negated by intentional, accidental or erroneous risks raised by autonomous agents.

Regulation has been studied to a large extent in research on normative MAS [START_REF] Andrighetto | Normative multi-agent systems[END_REF], where regulative norms [START_REF] Ossowski | Agreement technologies[END_REF] or prescriptions [START_REF] Balke | Norms in MAS: Definitions and Related Concepts[END_REF] are generally used to specify who does what, in what context, and as subject to what deontic modality (e.g., obligation, prohibition, permission). Regulative norms, therefore, affect agent behaviour in an indirect manner: agents can decide to conform or not to regulative norms, for instance in order to balance internal motivation versus external consequences. Therefore, in order to be effective, regulative norms require enforcement mechanisms, for instance to sanction norm violators. Two types of enforcement mechanisms that have been studied in MAS research are [START_REF] Ossowski | Agreement technologies[END_REF]: (i) social enforcement, where control emerges from the independent actions of agents (e.g., approaches based on computational trust and reputation [START_REF] Pinyol | Computational trust and reputation models for open multi-agent systems: a review[END_REF]), and (ii) normative organizations, where agent behavior is monitored and enforced in a top-down manner by an authority [START_REF] Jomi | Instrumenting multi-agent organisations with organisational artifacts and agents[END_REF]. The various regulation models and mechanisms proposed by MAS research over the past decade can now be used to support and promote regulation as a first-class abstraction in the WoT (and on the Web in general).

CONCLUSIONS

The W3C WoT defines the building blocks required to create more flexible WoT systems. In particular, we believe that the introduction of interaction affordances is an important step in the evolution of the Web that can promote and motivate the need for autonomous agents (cf. original Semantic Web vision [START_REF] Hendler | Agents and the semantic web[END_REF]). The last decade of research on MAS produced results that can now be directly applied to design, program, debug, monitor, and regulate autonomous agents for the WoT. All the elements required to build and deploy the Hypermedia MAS presented in our vision are already available. We believe that tapping into the large body of MAS research holds great promise for unlocking the full potential of the W3C WoT.

Figure 1 :

 1 Figure 1: Hypermedia MAS: Agents are situated in an environment designed as a distributed hypermedia application (sourced from [8]).

Figure 2 :

 2 Figure2: The various layers of abstraction in our agentbased manufacturing system for the WoT (sourced from[START_REF] Ciortea | Repurposing Manufacturing Lines on the Fly with Multi-agent Systems for the Web of Things[END_REF]).

This terminology is defined at W3C: https://www.w3.org/TR/wot-architecture/.

http://nodered.org/

http://www.ifttt.com/

https://hyperagents.org/

A distinctive feature of an autonomous agent is its flexibility in the pursuit of some design objectives[START_REF] Jennings | Applications of Intelligent Agents[END_REF]: the agent is reactive by responding to changes in the environment in a timely fashion, proactive by exhibiting goal-directed behavior and taking the initiative when appropriate, and social by interacting with humans or other agents in order to achieve complex tasks that would otherwise overcome its own capabilities.

https://youtu.be/tfAVDpYn_ow