Oriane Siméoni

Mateusz Budnik

Yannis Avrithis

Guillaume Gravier

Rethinking deep active learning: Using unlabeled data at model training

Active learning typically focuses on training a model on few labeled examples alone, while unlabeled ones are only used for acquisition. In this work we depart from this setting by using both labeled and unlabeled data during model training across active learning cycles. We do so by using unsupervised feature learning at the beginning of the active learning pipeline and semi-supervised learning at every active learning cycle, on all available data. The former has not been investigated before in active learning, while the study of latter in the context of deep learning is scarce and recent findings are not conclusive with respect to its benefit. Our idea is orthogonal to acquisition strategies by using more data, much like ensemble methods use more models. By systematically evaluating on a number of popular acquisition strategies and datasets, we find that the use of unlabeled data during model training brings a surprising accuracy improvement in image classification, compared to the differences between acquisition strategies. We thus explore smaller label budgets, even one label per class.

Introduction

Active learning [START_REF] Settles | Active learning literature survey[END_REF] is an important pillar of machine learning but it has not been explored much in the context of deep learning until recently [START_REF] Gal | Deep bayesian active learning with image data[END_REF][START_REF] William H Beluch | The power of ensembles for active learning in image classification[END_REF][START_REF] Wang | Cost-effective active learning for deep image classification[END_REF][START_REF] Geifman | Deep active learning over the long tail[END_REF][START_REF] Sener | Active learning for convolutional neural networks: A core-set approach[END_REF]. The standard active learning scenario focuses on training a model on few labeled examples alone, while unlabeled data are only used for acquisition, i.e., performing inference and selecting a subset for annotation. This is the opposite of what would normally work well when learning a deep model from scratch, i.e., training on a lot of data with some loss function that may need labels or not. At the same time, evidence is being accumulated that, when training powerful deep models, the difference in performance between acquisition strategies is small [START_REF] Gissin | Discriminative active learning[END_REF][START_REF] Chitta | Large-scale visual active learning with deep probabilistic ensembles[END_REF][START_REF] William H Beluch | The power of ensembles for active learning in image classification[END_REF].

In this work, focusing on image classification, we revisit active deep learning with the seminal idea of using all data, whether labeled or not, during model training at each active learning cycle. This departs from the standard scenario in that unlabeled data are now directly contributing to the cost function being minimized and to subsequent parameter updates, rather than just being used to perform inference for acquisition, whereby parameters are fixed. We implement our idea using two principles: unsupervised feature learning and semi-supervised learning. While both are well recognized in deep learning in general, we argue that their value has been unexplored or underestimated in the context of deep active learning.

Unsupervised feature learning or self-supervised learning is a very active area of research in deep learning, often taking the form of pre-training on artificial tasks with no human supervision for representation learning, followed by supervised fine-tuning on different target tasks like classification or object detection [START_REF] Doersch | Unsupervised visual representation learning by context prediction[END_REF][START_REF] Wang | Unsupervised learning of visual representations using videos[END_REF][START_REF] Gidaris | Unsupervised representation learning by predicting image rotations[END_REF][START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF]. To our knowledge, all deep active learning research so far considers training deep models from scratch. In this work, we perform unsupervised feature learning on all data once at the beginning of the active learning pipeline and use the resulting parameters to initialize the model at each active learning cycle. Relying on [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF], we show that such unsupervised pre-training improves accuracy in many cases at little additional cost.

Semi-supervised learning [START_REF] Chapelle | Semi-Supervised Learning[END_REF] and active learning can be seen as two facets of the same problem: the former focuses on most certain model predictions on unlabeled examples, while the latter on least certain ones. Combined approaches appeared quite early [START_REF] Mccallum | Employing em in pool-based active learning for text classification[END_REF][START_REF] Zhu | Combining active learning and semi-supervised learning using gaussian fields and harmonic functions[END_REF]. In the context of deep learning however, such combinations are scarce [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF] and have even been found harmful in cases [START_REF] Ducoffe | Adversarial active learning for deep networks: a margin based approach[END_REF]. It has also been argued that the two individual approaches have similar performance, while active learning has lower cost [START_REF] Gal | Deep bayesian active learning with image data[END_REF]. In the meantime, research on deep semi-supervised learning is very active, bringing significant progress [START_REF] Tarvainen | Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[END_REF][START_REF] Laine | Temporal ensembling for semisupervised learning[END_REF][START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF][START_REF] Verma | Interpolation consistency training for semi-supervised learning[END_REF]. In this work, we use semi-supervised learning on all data at every active learning cycle, replacing supervised learning on labeled examples alone. Relying on [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], and contrary to previous findings [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF][START_REF] Gal | Deep bayesian active learning with image data[END_REF], we show that this consistently brings an impressive accuracy improvement.

Since [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF] uses label propagation [START_REF] Zhou | Learning with local and global consistency[END_REF] to explore the manifold structure of the feature space, an important question is whether it is the manifold similarity or the use of un-1 labeled data during model training that actually helps. We address this question by introducing a new acquisition strategy that is based on label propagation.

In summary, we make the following contributions:

• We systematically benchmark a number of existing acquisition strategies, as well as a new one, on a number of datasets, evaluating the benefit of unsupervised pretraining and semi-supervised learning in all cases. • Contrary to previous findings, we show that using unlabeled data during model training can yield a striking gain compared to differences between acquisition strategies. • Armed with this finding, we explore a smaller budget (fewer labeled examples) than prior work, and we find that the random baseline may actually outperform all other acquisition strategies by a large margin in cases.

Related work

We focus on deep active and semi-supervised learning as well as their combination. Active learning. Geometric methods like core sets [START_REF] Geifman | Deep active learning over the long tail[END_REF][START_REF] Sener | Active learning for convolutional neural networks: A core-set approach[END_REF] select examples based on distances in the feature space. The goal is to select a subset of examples that best approximate the whole unlabeled set. We introduce a similar approach where Euclidean distances are replaced by manifold ranking. There are methods inspired by adversarial learning. For instance, a binary classifier can be trained to discriminate whether an example belongs to the labeled or unlabeled set [START_REF] Gissin | Discriminative active learning[END_REF][START_REF] Sinha | Variational adversarial active learning[END_REF]. Adversarial examples have been used, being matched to the nearest unlabeled example [START_REF] Mayer | Adversarial sampling for active learning[END_REF] or added to the labeled pool [START_REF] Ducoffe | Adversarial active learning for deep networks: a margin based approach[END_REF].

It has been observed however that deep networks can perform similarly regardless of the acquisition function [START_REF] Gissin | Discriminative active learning[END_REF][START_REF] Chitta | Large-scale visual active learning with deep probabilistic ensembles[END_REF], which we further investigate here. Ensemble and Bayesian methods [START_REF] Gal | Deep bayesian active learning with image data[END_REF][START_REF] William H Beluch | The power of ensembles for active learning in image classification[END_REF][START_REF] Chitta | Large-scale visual active learning with deep probabilistic ensembles[END_REF] target representing model uncertainty, which than can be used by different acquisition functions. This idea is orthogonal to acquisition strategies. In fact, [START_REF] William H Beluch | The power of ensembles for active learning in image classification[END_REF][START_REF] Chitta | Large-scale visual active learning with deep probabilistic ensembles[END_REF] show that the gain of ensemble models is more pronounced than the gain of any acquisition strategy. Of course, ensemble and Bayesian methods are more expensive than single models. Approximations include for instance a single model producing different outputs by dropout [START_REF] Gal | Deep bayesian active learning with image data[END_REF]. Our idea of using all data during model training is also orthogonal to acquisition strategies. It is also more expensive than using labeled data alone, but the gain is impressive in this case. This allows the use of much smaller label budget for the same accuracy, which is the essence of active learning. Semi-supervised active learning has a long history [START_REF] Mccallum | Employing em in pool-based active learning for text classification[END_REF][START_REF] Muslea | Active+ semi-supervised learning = robust multi-view learning[END_REF][START_REF] Zhu | Combining active learning and semi-supervised learning using gaussian fields and harmonic functions[END_REF][START_REF] Zhou | Exploiting unlabeled data in content-based image retrieval[END_REF][START_REF] Long | Graphbased active learning based on label propagation[END_REF]. A recent deep learning approach acquires the least certain unlabeled examples for labeling and at the same time assigns predicted pseudo-labels to most certain examples [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF]. This does not always help [START_REF] Ducoffe | Adversarial active learning for deep networks: a margin based approach[END_REF]. In some cases, semi-supervised algorithms are incorporated as part of an active learning evaluation [START_REF] Li | Ascent: Active supervision for semisupervised learning[END_REF][START_REF] Sener | Active learning for convolutional neural networks: A core-set approach[END_REF]. A comparative study suggests that semi-supervised learning does not significantly improve over active learning, despite its additional cost due to training on more data [START_REF] Gal | Deep bayesian active learning with image data[END_REF]. We show that this is clearly not the case, using a state of the art semisupervised method [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF] that is an inductive version of label propagation [START_REF] Zhou | Learning with local and global consistency[END_REF]. This is related to [START_REF] Zhu | Learning from labeled and unlabeled data with label propagation[END_REF][START_REF] Zhu | Combining active learning and semi-supervised learning using gaussian fields and harmonic functions[END_REF][START_REF] Long | Graphbased active learning based on label propagation[END_REF], which however are limited to transductive learning. Unsupervised feature learning. A number of unsupervised feature learning approaches pair matching images to learn the representation using a siamese architecture. These pairs can come as fragments of the same image [START_REF] Doersch | Unsupervised visual representation learning by context prediction[END_REF][START_REF] Noroozi | Unsupervised learning of visual representations by solving jigsaw puzzles[END_REF] or as a result of tracking in video [START_REF] Wang | Unsupervised learning of visual representations using videos[END_REF]. Alternatively, the network is trained on an artificial task like image rotation prediction [START_REF] Gidaris | Unsupervised representation learning by predicting image rotations[END_REF] or even matching images to a noisy target [START_REF] Bojanowski | Unsupervised learning by predicting noise[END_REF]. The latter is conceptually related to deep clustering [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF], the approach we use in this work, where the network learns targets resulting from unsupervised clustering. It is interesting that in the context of semi-supervised learning, unsupervised pre-training has been recently investigated by [START_REF] Sylvestre-Alvise | Semisupervised learning with scarce annotations[END_REF], with results are consistent with ours. However, the use of unsupervised pre-training in deep active learning remains unexplored. Let L j := L j-1 ∪ S j-1 ⊂ I be the set of indices of labeled examples in cycle j ≥ 1 and U j := I \ L j the indices of the unlabeled examples for j ≥ 0. Then y j := (y i ) i∈Lj are the labels in cycle j and S j ⊂ U j is selected from the unlabeled examples. To keep notation simple, we will refer to a single cycle in the following, dropping subscripts j. Classifier learning. The classifier f θ : X → R c with parameters θ, maps new examples to a vector of probabilities per class. Given x ∈ X , its prediction is the class of maximum probability

Problem formulation and background

π(p) := arg max k∈C p k , (1)
where p k is the k-th element of vector p := f θ (x). As a by-product of learning parameters θ, we have access to an embedding function φ θ : X → R d , mapping an example x ∈ X to a feature vector φ θ (x). For instance, f θ may be a linear classifier on top of features obtained by φ θ .

In a typical AL scenario, given a set of indices L of labeled examples and labels y, the parameters θ of the classifier are learned by minimizing the cost function

J(X, L, y; θ) := i∈L ℓ(f θ (x i ), y i ), (2)
on labeled examples x i for i ∈ L, where cross-entropy ℓ(p, y) :=log p y for p ∈ R c + , y ∈ C. Acquisition. Given the set of indices U of unlabeled examples and the parameters θ resulting from training, one typically acquires a new batch by initializing S ← ∅ and then greedily updating by

S ← S ∪ {a(X, L ∪ S, U \ S, y; θ)} (3) until |S| ≥ b.
Here a is an acquisition (or sampling) function, each time selecting one example from U \ S. For each i ∈ S, the corresponding example x i is then given as query to an oracle (often a human expert), who returns a label y i to be used in the next cycle.

Geometry. Given parameters θ, a simple acquisition strategy is to use the geometry of examples in the feature space F θ := φ θ (X ), without considering the classifier. Each example x i is represented by the feature vector φ θ (x i ) for i ∈ I. One particular example is the function [START_REF] Geifman | Deep active learning over the long tail[END_REF][START_REF] Sener | Active learning for convolutional neural networks: A core-set approach[END_REF] a(X, L, U, y; θ) := arg max

i∈U min k∈L φ θ (x i ), φ θ (x k ) , (4)
each time selecting the unlabeled example in U that is the most distant to its nearest labeled or previously acquired example in L. Such geometric approaches are inherently related to clustering. For instance, k-means++ [START_REF] Arthur | K-means++: the advantages of careful seeding[END_REF] is a probabilistic version of (4). Uncertainty. A common acquisition strategy that considers the classifier is some measure of uncertainty in its prediction. Given a vector of probabilities p, one such measure is the entropy

H(p) := - c k=1 p k log p k , (5)
taking values in [0, log c]. Given parameters θ, each example x i is represented by the vector of probabilities f θ (x i ) for i ∈ I. Then, acquisition is defined by

a(X, L, U, y; θ) := arg max i∈U H(f θ (x i )), (6)
effectively selecting the b most uncertain unlabeled examples for labeling. Pseudo-labels. It is possible to use more data than the labeled examples while learning. In [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF] for example, given indices L, U of labeled and unlabeled examples respectively and parameters θ, one represents example x i by

p i := f θ (x i ), selects the most certain unlabeled examples L := {i ∈ U : H(p i ) ≤ ǫ}, (7)
and assigns pseudo-label ŷi := π(p i ) by ( 1) for i ∈ L. The same cost function J defined by ( 2) can now be used by augmenting L to L ∪ L and y to (y, ŷ), where ŷ := (ŷ i ) i∈ L. This augmentation occurs once per cycle in [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF]. This is an example of active semi-supervised learning.

Transductive label propagation [START_REF] Zhou | Learning with local and global consistency[END_REF] refers to graph-based, semi-supervised learning. A nearest neighbor graph of the dataset X is used, represented by a symmetric non-negative n × n adjacency matrix W with zero diagonal. This matrix is symmetrically normalized as

W := D -1/2 W D -1/2
, where D := diag(W 1) is the degree matrix and 1 is the allones vector. The given labels y := (y i ) i∈L are represented by a n × c zero-one matrix Y := χ(L, y) where row i is a c-vector that is a one-hot encoding of label y i if example x i is labeled and zero otherwise,

χ(L, y) ik := 1, i ∈ L ∧ y i = k, 0, otherwise (8)
for i ∈ I and k ∈ C. [START_REF] Zhou | Learning with local and global consistency[END_REF] define the n × c matrix

P := η[h(Y )] 1 , where h(Y ) := (1 -α)(I -αW) -1 Y, (9)
I is the n × n identity matrix, and α ∈ [0, 1) is a parameter. The i-th row p i of P represents a vector of class probabilities of unlabeled example x i , and a prediction can be made by π(p i ) (1) for i ∈ U . This method is transductive because it cannot make predictions on previously unseen data without access to the original data X. Inductive label propagation. Although the previous methods do not apply to unseen data by themselves, the predictions made on X can again be used as pseudo-labels to train a classifier. This is done in [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], applied to semi-supervised learning. Like [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF], a pseudo-label is generated for unlabeled example x i as ŷi := π(p i ) by (1), only now p i is the i-th row of the result P of label propagation according to (9) rather than the classifier output f θ (x i ). Unlike [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF],

all unlabeled examples are pseudo-labeled and an additional cost term J w (X, U, ŷ; θ) := i∈U w i ℓ(f θ (x i ), ŷi ) applies to those examples, where ŷ := (ŷ i ) i∈U and w i := β(p i ) is a weight reflecting the certainty in the prediction of ŷi :

β(p) := 1 - H(p) log c . ( 10
)
Unlike [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF], the graph and the pseudo-labels are updated once per epoch during learning in [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], where there are no cycles.

Unsupervised feature learning. Finally, it is possible to train an embedding function in an unsupervised fashion. A simple method that does not make any assumption on the nature or structure of the data is [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF]. Simply put, starting by randomly initialized parameters θ, the data φ θ (X) are clustered by k-means, each example is assigned to the nearest centroid, clusters and assignments are treated as classes C and pseudo-labels ŷ respectively, and learning takes place according to J(X, I, ŷ, θ) [START_REF] William H Beluch | The power of ensembles for active learning in image classification[END_REF]. By updating the parameters θ, φ θ (X) is updated too. The method therefore alternates between clustering/pseudo-labeling and feature learning, typically once per epoch.

Training the model on unlabeled data

We argue that acquiring examples for labeling is not making the best use of unlabeled data: unlabeled data should be used during model training, appearing in the cost function that is being minimized. We choose two ways of doing so: unsupervised feature learning and semi-supervised learning. As outlined in Algorithm 1, we follow the standard active learning setup, adding unsupervised pre-training at the beginning and replacing supervised learning on L by semisupervised learning on L ∪ U at each cycle. The individual components are discussed in more detail below.

Unsupervised pre-training (PRE) takes place at the beginning of the algorithm. We follow [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF], randomly initializing θ and then alternating between clustering the features φ θ (X) by k-means and learning on cluster assignment pseudo-labels ŷ of X according to J(X, I, ŷ, θ) (2). The result is a set of parameters θ 0 used to initialize the classifier at every cycle.

Learning per cycle follows inductive label propagation [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]. This consists of supervised learning followed by alternating label propagation and semi-supervised learning on all examples L∪U at every epoch. The supervised learn-ing (SUP) is performed on the labeled examples L only using labels y, according to J(X, L, y, θ) (2), where the parameters θ are initialized by θ 0 .

Label propagation (LP) involves a reciprocal k-nearest neighbor graph on features φ θ (X) [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]. As in [START_REF] Zhou | Learning with local and global consistency[END_REF], the resulting affinity matrix W is normalized as W := D -1/2 W D -1/2 . Label propagation is then performed according to P = η[h(Y )] (9), by solving the corresponding linear system using the conjugate gradient (CG) method [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]. The label matrix Y := χ(L, y) (8) is defined on the true labeled examples L that remain fixed over epochs but grow over cycles. With p i being the ith row of P , a pseudo-label ŷi = π(p i ) (1) and a weight w i = β(p i ) (10) are defined for every i ∈ U [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF].

Semi-supervised learning (SEMI) takes place on all examples L ∪ U = I, where examples in L have true labels y and examples in U pseudo-labels ŷ := (ŷ i ) i∈U . Different than [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], we minimize the standard cost function J(X, L ∪ U, (y, ŷ), θ) (2), but we do take weights w := (w i ) i∈U into account in mini-batch sampling, ℓ 1normalized as η[w]. In particular, part of each mini-batch is drawn uniformly at random from L, while the other part is drawn with replacement from the discrete distribution η[w] on U : an example may be drawn more than once per epoch or never.

Discussion. The above probabilistic weighting decouples the size of the epoch from n and indeed we experiment with epochs smaller than n, accelerating learning compared to [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]. It is similar to importance sampling, which is typically based on loss values [START_REF] Katharopoulos | Biased importance sampling for deep neural network training[END_REF][START_REF] Cheng | Revisiting pre-training: An efficient training method for image classification[END_REF] or predicted class probabilities [START_REF] Yang | Dropsample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten chinese character recognition[END_REF]. Acceleration is important as training on all examples is more expensive than just the labeled ones, and is repeated at every cycle. On the contrary, unsupervised pre-trained only occurs once at the beginning.

The particular choice of components is not important: any unsupervised representation learning could replace [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF] in line 2 and any semi-supervised learning could replace [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF] in lines 4-7 of Algorithm 1. We keep the pipeline as simple as possible, facilitating comparisons with more effective choices in the future.

Investigating manifold similarity in the acquisition function

Label propagation [START_REF] Zhou | Learning with local and global consistency[END_REF][START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF] is based on the manifold structure of the feature space, as captured by the normalized affinity matrix W. Rather than just using this information for propagating labels to unlabeled examples, can we use it in the acquisition function as well? This is important in interpreting the effect of semi-supervised learning in Algorithm 1: is any gain due to the use of manifold similarity, or to training the model on more data?

Joint label propagation (jLP), introduced here, is an attempt to answer these questions. It is an acquisition function similar in nature to the geometric approach (4), with Euclidean distance replaced by manifold similarity. In particular, the n-vector Y 1 c , the row-wise sum of Y = χ(L, y) (8), can be expressed as

Y 1 c = δ(L) ∈ R n , where δ(L) i := 1, i ∈ L, 0, otherwise (11)
for i ∈ I. Hence, in the terminology of manifold ranking [START_REF] Zhou | Ranking on data manifolds[END_REF], vector Y 1 c represents a set of queries, one for each example x i for i ∈ L, and the i-th element of the nvector h(Y )1 c in (9) expresses the manifold similarity of x i to the queries for i ∈ I. Similar to (4), we acquire the example in U that is the least similar to examples in L that are labeled or previously acquired:

a(X, L, U, y; θ) := arg min i∈U (h(δ(L))) i . ( 12
)
This strategy is only geometric and bears similarities to discriminative active learning [START_REF] Gissin | Discriminative active learning[END_REF], which learns a binary classifier to discriminate labeled from unlabeled examples and acquires examples of least confidence in the "labeled" class.

6. Experiments

Experimental setup

Datasets. We conduct experiments on four datasets that are most often used in deep active learning: MNIST [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF],SVHN [START_REF] Netzer | Reading digits in natural images with unsupervised feature learning[END_REF], CIFAR-10 and CIFAR-100 [START_REF] Krizhevsky | Learning multiple layers of features from tiny images[END_REF].

Networks and training.

For all experiments we use a 13layer convolutional network used previously in [START_REF] Laine | Temporal ensembling for semisupervised learning[END_REF]. We train the model from scratch at each active learning cycle, using SGD with momentum of 0.9 for 200 epochs. An initial learning rate of 0.2 is decayed by cosine annealing [START_REF] Loshchilov | Sgdr: Stochastic gradient descent with warm restarts[END_REF], scheduled to reach zero at 210 epochs. The mini-batch size is 32 for standard training and 128 when SEMI is used, except for MNIST where the size of the mini-batch 10 and 64 with SEMI. All other parameters follow [START_REF] Tarvainen | Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[END_REF].

Unsupervised pre-training. We use k-means as the clustering algorithm and follow the settings of [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF]. The model is trained for 250 epochs on the respective datasets.

Semi-supervised learning. Following [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], we construct a reciprocal k-nearest neighbor graph on features φ θ (X), with k = 50 neighbors and similarity function

s(u, v) := [û ⊤ v] 3 + for u, v ∈ R d
, where û is the ℓ 2 -normalized counterpart of u, while α = 0.99 in [START_REF] Doersch | Unsupervised visual representation learning by context prediction[END_REF]. We follow [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF] in splitting mini-batches into two parts: 50 examples (10 for MNIST) are labeled and the remaining pseudo-labeled. For the latter, we draw examples using normalized weights as a discrete distribution. The epoch ends when 1 2 |U | pseudolabels have been drawn, that is the epoch is 50% compared to [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]. Given that |L| ≪ |U | in most cases, the labeled examples are typically repeated more than once. Acquisition strategies. We evaluate our new acquisition strategy jLP along with the following baselines: (a) Random; (b) Uncertainty based on entropy (5); (c) CEAL [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF], combining entropy with pseudo-labels ( 7); (d) the greedy version of CoreSet (4) [START_REF] Sener | Active learning for convolutional neural networks: A core-set approach[END_REF][START_REF] Geifman | Deep active learning over the long tail[END_REF]. Baselines. For all acquisition strategies, we show results of the complete Algorithm 1 as well as the the standard baseline, that is without pre-training and only fully supervised on labeled examples L, and unsupervised pre-training (PRE) alone without semi-supervised. In some cases, we show semi-supervised (SEMI) alone. For instance, in the scenario of 100 labels per class, the effect of pre-training is small, especially in the presence of semi-supervised. CEAL [START_REF] Wang | Cost-effective active learning for deep image classification[END_REF] is a baseline with its own pseudo-labels, so we do not combine it with semi-supervised. The length of the epoch is fixed for Algorithm 1 and increases with each cycle. Label budget and cycles. We consider three different sce- narios, as shown in Table 1. In the first, we use an initial balanced label set L 0 of 10 labels per class, translating into a total of 100 for CIFAR-10 and SVHN and 1000 for CIFAR-100. We use the same values as label budget b for all cycles. In the second, we use initially 100 labels per class in CIFAR-10 with b = 1000 per cycle; this is not interesting for CIFAR-100 as it results in complete labeling of the training set after 4 cycles. Finally, we investigate the use of one label per class both as the initial set and the label budget, on MNIST, translating to 10 labels per cycle. All experiments are carried out for 5 cycles and repeated 5 times using different initial label sets L 0 . We report average accuracy and standard deviation.

Standard baseline results

We first evaluate acquisition functions without using any unlabeled data. Figure 1 presents results on SVHN, CIFAR-10 and CIFAR-100. The differences between acquisition functions are not significant, except when compared to Random. On SVHN, Random appears to be considerably better than the other acquisition functions and worse on CIFAR-10 with b = 1000. All the other acquisition functions give near identical results; in particular, there is no clear winner in the case of 10 labels per class on CIFAR-10 and CIFAR-100 (Figure 1

(b) and (d), respectively).

This confirms similar observations made in [START_REF] Gissin | Discriminative active learning[END_REF] and [START_REF] Chitta | Large-scale visual active learning with deep probabilistic ensembles[END_REF]. Our jLP is no exception, giving similar results to the other acquisition functions. We study this phenomenon in Appendix A. In summary, we find that while the ranks of examples according to different strategies may be uncorrelated, the resulting predictions of label propagation mostly agree. Even in cases of disagreement, the corresponding examples have small weights, hence their contribution to the cost function is small. Since those predictions are used as pseudo-labels in [START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF], this can explain why the performance of the learned model is also similar in the presence of semisupervised learning.

The effect of unsupervised pre-training

As shown in Figure 2, pre-training can be beneficial. PRE by itself brings substantial gain on SVHN and CIFAR-10 with b = 100, up to 6%, while the improvements on CIFAR-100 are moderate. In addition, numerical results in Table 2 for our acquisition strategy jLP show that PRE is beneficial with or without SEMI in most cases. Pre-training provides a relatively easy and cost-effective improvement. It is performed only once at the beginning of the active learning process. While [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF] was originally tested on large datasets like ImageNet or YFCC100M, we show that it can be beneficial even on smaller datasets like CIFAR-10 or SVHN.

The effect of semi-supervised learning

Figure 3 shows results on different datasets and acquisition strategies like Figure 2, but including both PRE and PRE + SEMI. For the purpose of reproducibility, numeric results, including average and standard deviation measurements, are given in Appendix B for all cycles and datasets. The combination PRE + SEMI yields a further significant improvement over PRE and the standard baseline, on all acquisition functions and datasets. For instance, on CIFAR-10 with a budget of 100, the most noticeable improvement comes from Random, where the improvement of PRE + SEMI is around 15% over the standard baseline at all cycles. The improvement is around 10% in most other cases, which is by far greater than any potential difference between the acquisition methods. Also, noticeably, in the case of SVHN, Random with PRE + SEMI reaches nearly the fully supervised accuracy after just 2 cycles (300 labeled examples in total).

The gain from semi-supervised learning is striking in the few-labels regime of CIFAR-10 with b = 100. A single cycle with PRE + SEMI achieves the accuracy of 4 cycles of the standard baseline in this case, which translates to a significant reduction of cost for human annotation. In Table 2 we present the effect of all four combinations: with/without PRE and with/without SEMI. We focus on our jLP acquisition strategy, which has similar performance as all other strategies and uses manifold similarity just like SEMI. In most cases, PRE improves over SEMI alone by around 2%. The use of PRE appears to be particularly beneficial in the first cycles, while its impact decreases as the model performance improves.

It is worth noting that CEAL, which makes use of pseudo-labels, has a low performance. This has been observed before [START_REF] Ducoffe | Adversarial active learning for deep networks: a margin based approach[END_REF] and can be attributed to the fact that it is using the same set of pseudo-labels in every epoch. By contrast, pseudo-labels are updated in every epoch in our case.

One label per class

Since PRE and SEMI have a significant gain in classification accuracy, it is reasonable to attempt even fewer labeled examples than in previous work on active learning. We investigate the extreme case of one label per class using MNIST as a benchmark, that is, label budget at each cycle is equal to the number of classes. Figure 4 shows results on all acquisition strategies with and without SEMI. As in the previous experiments, there is no clear winner among the selection strategies alone, and accuracy remain below 80% after 5 cycles (50 labels in total) without SEMI. By contrast, Random with SEMI arrives at 90.89% accuracy after two cycles (20 labeled examples), which is 40% better than without SEMI.

Discussion

In this work, we have shown the benefit of using both labeled and unlabeled data during model training in deep active learning for image classification. This leads to a more accurate model while requiring less labeled data, which is in itself one of the main objectives of active learning. We have used two particular choices for unsupervised feature learning and semi-supervised learning as components in our pipeline. There are several state of the art methods that could be used for the same purpose, for instance [START_REF] Tarvainen | Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[END_REF][START_REF] Verma | Interpolation consistency training for semi-supervised learning[END_REF][START_REF] Berthelot | Mixmatch: A holistic approach to semi-supervised learning[END_REF][START_REF] Sylvestre-Alvise | Semisupervised learning with scarce annotations[END_REF] for semi-supervised learning. Our pipeline is as simple as possible, facilitating comparisons with more effective choices, which can only strengthen our results. While the improvement coming from recent acquisition strategies is marginal in many scenarios, an active learning approach that uses unlabeled data for training and not just acquisition appears to be a very good option for deep network models. Our findings can have an impact on how deep active learning is evaluated in the future. For instance, the relative performance of the random baseline to all other acquisition strategies depends strongly on the label budget, the cycle and the presence of pre-training and semi-supervised learning. It may of course be possible that two strategies with uncorrelated ranks still yield models of similar accuracy. To investigate this, we measure agreement as described above. Results are shown in Table 3. Uncertainty is used as a reference strategy, i.e. we train the model for a number of cycles using Uncertainty and then measure agreement and disagreement of another strategy to Uncertainty. After cycle 1, any two methods agree on around 80% of the pseudo-labels, while the remaining 20% have on average smaller weights compared to when the methods agree.

We reach the same conclusions from a similar experiment where we actually train the model rather than perform label propagation. Hence, although examples are ranked differently by different strategies, their effect on prediction, either by training or label propagation, is small.

B. All detailed results

In order to facilitate reproducibility, in this section we present all the detailed results in Table 4 and Table 5. We describe results obtained with the five methods presented before, namely Random, Uncertainty, CEAL, CoreSet and jLP. We evaluate them on CIFAR-10 with 10 and 100 labels per class (budget b = 100 and b = 1000 respectively), CIFAR-100 with b = 1000 in Table 4. We present results obtained on MNIST with only 1 label per class (b = 10) and SVHN with b = 100 in Table 5.

Problem.

  We are given a set X := {x i } i∈I ⊂ X of n examples where I := [n] := {1, . . . , n} and, initially, a collection y 0 := (y i ) i∈L0 of b labels y i ∈ C for i ∈ L 0 , where C := [c] is a set of c classes and L 0 ⊂ I a set of indices with |L 0 | = b ≪ n. The goal of active learning (AL) [31] is to train a classifier in cycles, where in cycle j = 0, 1, . . . we use a collection y j of labels for training, and then we acquire (or sample) a new batch S j of indices with |S j | = b to label the corresponding examples for the next cycle j + 1.

Algorithm 1 : 6 ( 7 θ 9 S ← ∅ 10 while 13 y

 16791013 Semi-supervised active learning Data: data X, indices of labeled examples L, labels y, batch size b 1 U ← I \ L ⊲ indices of unlabeled examples 2 θ 0 ← PRE(X ) ⊲ unsupervised pre-training 3 for j ∈ {0, . . . } do ⊲ active learning cycles 4 θ ← SUP(X, L, y; θ 0 ) ⊲ supervised learning on L only 5 for e ∈ {1, . . . } do ⊲ epochs ŷ, w) ← LP(X, L, y, θ) ⊲ pseudo-labels ŷ and labels w ← SEMI(X, L ∪ U, (y, ŷ), w; θ) ⊲ semi-supervised learning on all data 8 end |S| < b do ⊲ acquire a batch S ⊂ U for labeling 11 S ← S ∪ a(X, L ∪ S, U \ S, y; θ) 12 end ← (y, LABEL(S)) ⊲ obtain true labels on S by oracle 14 L ← L ∪ S; U ← U \ S ⊲ update indices 15 end

Figure 1 .

 1 Figure 1. Average accuracy vs. cycle on different setups and acquisition strategies.

Figure 2 .

 2 Figure 2. Average accuracy vs. cycle on different setups and acquisition strategies with PRE.

Figure 4 .

 4 Figure 4. Average accuracy vs. cycle on different setups and acquisition strategies.

METHOD CIFAR- 10 ,

 10 b = 100 CIFAR-10, b = 1000 CIFAR-100, b = 1000

Table 4 .

 4 Average accuracy and standard deviation for different label budget b and cycle on CIFAR-10 and CIFAR-100. Following Algorithm 1, we show the effect of unsupervised pre-training (PRE) and semi-supervised learning (SEMI) compared to the standard baseline. and jLP are not as heavily correlated compared to, for example, CoreSet and Uncertainty in Figure 5(b). The correlation between jLP and CoreSet is also quite low as shown in Figure 5(c).

Table 1

 1

		Data size	Image Mini-batch size Budget Total
		train / test	size	w/o SEMI/SEMI		labels
	MNIST	60k / 10k 28 × 28	10 / 64	10	50
	SVHN	73k / 26k 32 × 32	32 / 128	100	500
	CIFAR-10	50k / 10k 32 × 32	32 / 128	100	500
	CIFAR-10	50k / 10k 32 × 32	32 / 128	1k	5k
	CIFAR-100 50k / 10k 32 × 32	32 / 128	1k	5k

presents statistics of the datasets. Following

[START_REF] Tarvainen | Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results[END_REF][START_REF] Iscen | Label propagation for deep semi-supervised learning[END_REF]

, we augment input images by 4 × 4 random translations and random horizontal flips.

Table 1 .

 1 Datasets used in this paper, including the mini-batch sizes used in training with and without SEMI, acquisition size at each active learning step and the total number of labeled images.

Table 2 .

 2 Ablation study. Evaluation of results obtained with Random while adding PRE and/or SEMI.

	METHOD	CIFAR-10 CIFAR-100
	BUDGET	b = 100	b = 1000
	CYCLE 0
	jLP	29.17±1.62	19.63±0.99
	+ PRE	35.20±2.26	23.71±0.86
	+ SEMI	36.73±5.70	25.06±1.44
	+ PRE + SEMI 38.05±2.92	27.04±0.78
	CYCLE 1
	jLP	38.86±1.36	32.16±1.98
	+ PRE	42.07±0.74	33.48±0.52
	+ SEMI	46.76±3.27	37.99±2.47
	+ PRE + SEMI 48.66±2.64	40.30±1.53
	CYCLE 2
	jLP	42.30±1.61	40.65±1.21
	+ PRE	47.99±1.17	40.81±0.40
	+ SEMI	51.53±3.02	46.39±1.49
	+ PRE + SEMI 51.18±1.80	47.03±0.47

  91±1.63 50.63±0.59 62.37±1.41 82.33±0.21 86.66±0.21 47.85±0.84 47.54±0.63 50.38±0.25 Uncertainty 47.89±1.78 50.03±1.38 55.47±2.10 84.47±0.49 89.32±0.24 47.26±0.79 46.39±0.81 50.42±0.24 CoreSet 46.75±2.41 51.40±1.99 56.93±2.90 84.27±0.36 88.75±0.45 46.22±0.39 46.34±0.92 50.85±0.49±1.71 51.54±1.24 56.67±2.58 83.82±0.02 88.85±0.38 46.52±0.99 45.94±0.44 50.90±0.67 94±1.75 55.31±1.28 64.35±1.37 84.10±0.10 87.23±0.21 51.43±0.56 51.40±0.47 53.58±0.64 Uncertainty 49.73±2.29 53.17±1.52 60.71±2.77 86.49±0.19 90.42±0.28 50.83±0.31 49.90±0.82 52.20±0.50 CoreSet 50.11±1.40 54.17±0.40 62.94±2.41 86.39±0.36 90.33±0.13 50.48±0.84 49.54±0.95 53.67±1.29

	PRE
	SEMI
	CYCLE 0		100 LABELS		1K LABELS		1K LABELS
	Random	29.17±1.62 35.20±2.26 39.84±2.63 63.61±1.42 78.85±0.86 19.63±0.99 23.71±0.86 27.46±0.52
	CYCLE 1		200 LABELS		2K LABELS		2K LABELS
	Random	36.66±1.08 41.76±1.32 50.69±2.95 75.09±0.51 83.49±0.81 32.44±1.69 34.88±0.90 40.65±0.63
	Uncertainty 37.59±1.93 40.56±2.21 46.04±2.78 76.22±0.68 84.94±0.35 32.09±1.50 34.54±0.70 38.88±1.11
	CoreSet	39.23±1.17 43.04±0.92 48.08±1.64 76.44±0.34 84.98±0.19 32.05±1.40 33.95±0.57 39.63±0.70
	CEAL	38.92±2.00 39.74±1.72	-	76.52±0.73	-	31.59±0.93 33.78±0.39	-
	jLP (ours)	38.86±1.36 42.07±0.74 48.66±2.64 75.74±0.39 84.62±0.47 32.16±1.98 33.48±0.52 40.30±1.53
	CYCLE 2		300 LABELS		3K LABELS		3K LABELS
	Random	42.12±1.83 46.31±1.40 58.72±4.04 79.45±0.56 85.33±0.42 42.45±0.90 42.37±0.53 47.42±0.53
	Uncertainty 43.66±1.57 44.02±1.73 52.04±2.46 81.26±0.30 87.65±0.29 40.43±0.63 41.04±0.27 46.30±1.12
	CoreSet	43.01±2.14 47.00±2.57 50.85±4.23 81.11±0.61 87.21±0.31 41.32±0.70 40.47±0.38 46.74±1.00
	CEAL	41.74±1.15 44.92±2.09	-	81.37±0.54	-	41.19±0.41 41.55±0.45	-
	jLP (ours)	42.30±1.61 47.99±1.17 51.18±1.80 80.97±0.40 87.16±0.44 40.65±1.21 40.81±0.40 47.03±0.47
	CYCLE 3		400 LABELS		4K LABELS		4K LABELS
	Random	45.32
	CEAL	45.55±2.39 49.73±1.82	-	84.05±0.44	-	46.34±0.44 46.67±0.38	-
	jLP (ours) 45.CYCLE 4	500 LABELS		5K LABELS		5K LABELS
	Random 50.CEAL 48.14±1.24 53.46±1.27	-	86.31±0.23	-	50.62±0.28 50.18±0.60	-
	jLP (ours)	48.93±2.22 53.89±1.42 59.83±4.02 85.94±0.38 89.91±0.28 50.24±0.93 50.20±0.44 53.37±0.64

We denote by η[A] := diag(A1) -1 A and η[a] := (a ⊤ 1) -1 a the (row-wise) ℓ 1 -normalization of nonnegative matrix A and vector a respectively.

 3. Agreement results between acquisition strategies on CIFAR-10 with b = 1000 after cycles 1 and 2. All strategies are compared to Uncertainty as reference, which is also employed in the previous cycles. %agree is percentage of pseudo-labels agreeing to the reference. Accuracy is weighted according to [START_REF] Gidaris | Unsupervised representation learning by predicting image rotations[END_REF] and weights are according to [START_REF] Ducoffe | Adversarial active learning for deep networks: a margin based approach[END_REF]. Measurements denoted by = ( =) refer to the set of pseudo-labels that agree (disagree) with the reference.

A. Studying the agreement of acquisition strategies

It has been observed that most acquisition strategies do not provide a significant improvement over standard uncertainty when using deep neural networks; for instance, all strategies perform similarly on CIFAR-10 and CIFAR-100 according to [START_REF] Gissin | Discriminative active learning[END_REF] and [START_REF] Chitta | Large-scale visual active learning with deep probabilistic ensembles[END_REF]. To better understand the differences, the ranks of examples acquired by different strategies are compared pairwise by [START_REF] Gissin | Discriminative active learning[END_REF]. We make a step further in this direction, using label propagation as a tool.

A.1. Measuring agreement

After the classifier is trained at any cycle using any reference acquisition function a, we apply two different acquisition functions, say a (1) and a (2) , followed by labeling of acquired examples and label propagation, obtaining two different sets of predicted pseudo-labels ŷ(1) and ŷ(2) and weights w (1) and w (2) on the unlabeled examples U . We define the weighted accuracy

for z, z ′ ∈ R |U| , where δ is the Kronecker delta function. Using the average weights w := 1 2 (w (1) + w (2) ), we then measure the weighted accuracy A U,w (y (1) , y (2) ), expressing the agreement of the two strategies, as well as the weighted accuracy A U,w (y (k) , t) of a (k) relative to the true labels t on U for k = 1, 2. More measurements include weighted accuracies relative to true labels on subsets of U where the two strategies agree or disagree. This way, assuming knowledge of the true labels on the entire set X, we evaluate the quality of pseudo-labels used in semi-supervised learning in each cycle, casting label propagation as an efficient surrogate of the learning process.

A.2. Results

We show results on CIFAR-10 with b = 1000 in this study. Following the experiments of [START_REF] Gissin | Discriminative active learning[END_REF], we first investigate the correlation of the ranks of unlabeled examples obtained by two acquisition functions. As shown in Figure 5


Anne Mercier
email: anne.mercier@univ-amu.fr

Emilie Ricquebourg

Chouaib Kandouli
email: chouaib50@gmail.com

Marcel Culcasi
email: marcel.culcasi@univ-amu.fr

Sylvia Pietri
email: sylvia.pietri@univ-amu.fr

Protective action of saharian Anvillea radiata extracts against oxidative stress, inflammatory processes and diabetic neuropathy progression in high-fat-fed mice

Keywords: Anvillea radiata, traditional medicine, C57/BL6J mice, high fat diet, diabetic neuropathy, oxidative stress, inflammation, vascular damage, myeloperoxidase, advanced glycation endproducts, microglial cells, astrocytes

Diabetic neuropathies are of the major and earliest complications in diabetes and are associated with reactive oxygen species (ROS) production and inflammation in neurons and glial cells. We recently reported the antioxidant and antidiabetic properties of water-soluble extracts of the traditional medicine Anvillea radiata Coss & Dur (AR). The present study investigated whether AR aqueous and n-butanol polyphenol-enriched extracts could afford protection against diabetic neuropathy. In the in vitro part of the study, antiglycative and ROS-producing enzymes inhibition capacities were demonstrated. In further experiments driven on AR-treated rat cell cultures, inflammation markers (TNF, IL-6, IL-1were measured in lipopolysaccharides-induced microglial cells and cytoprotection was evaluated on methylglyoxal-exposed astrocytes. In the in vivo part of the study, both extracts were given orally for 12 weeks at the dose of 75 mg/kg to high-fat-diet-C57BL/6J diabetic mice. Significant reduction of hyperglycemia and oxidative stress were observed, together with strong decrease of: i) thermal withdrawal latency (-30%) and T cells infiltration in sciatic nerve (-40%), ii) vasoconstriction, superoxide and protein carbonyls production in aortic rings, iii) myeloperoxidase and xanthine oxidase overactivation in plasma and kidney, iv) lipid peroxidation. Taken together, AR extracts may prove to be of therapeutic value in the management of diabetes complications.

Introduction

Diabetes mellitus has become a major healthcare issue over the world and represents a panel of syndromes mainly characterized by chronic hyperglycemia and insulin disturbances. Among the main complications, diabetic neuropathies (DN), associating neuronal, glial and microvascular damage, are of the major and earliest ones in diabetic patients, taking complex and multiple forms [START_REF] Duby | Diabetic neuropathy: an intensive review[END_REF][START_REF] Dewanjee | Molecular mechanism of diabetic neuropathy and its pharmacotherapeutic targets[END_REF]]. Hyperglycemia is susceptible to affect multiple metabolic routes associated to hallmarks including reactive carbonyl species such as methylglyoxal, advanced glycation endproducts (AGEs) [START_REF] Semba | Serum carboxymethyl-lysine, an advanced glycation end product, is associated with increased aortic pulse wave velocity in adults[END_REF][START_REF] Vistoli | Advanced glycoxidation and lipoxidation end products (AGEs and ALEs): an overview of their mechanisms of formation[END_REF][START_REF] Chaudhuri | The role of advanced glycation end products in aging and metabolic diseases: bridging association and causality[END_REF]], protein carbonyls (PCs) [START_REF] Stocker | Fluorimetric screening assay for protein carbonyl evaluation in biological samples[END_REF]], and is also linked to changes in oxidative status and antioxidant pool [Yagihashi 2010[START_REF] Dewanjee | Molecular mechanism of diabetic neuropathy and its pharmacotherapeutic targets[END_REF]]. Hyperglycemia-induced oxidative stress may cause an increase of myeloperoxidase (MPO) levels associated to endothelial dysfunction and related vascular and renal damages [Rovira-Llopis 2013; [START_REF] Galijasevic | The development of myeloperoxidase inhibitors[END_REF]. Furthermore, the possible link between microglia activation, MPO expression and neuroinflammation as been reported [START_REF] Jucaite | Effect of the myeloperoxidase inhibitor AZD3241 on microglia: a PET study in Parkinson's disease[END_REF]]. Diabetic neuropathies are linked to mitochondrial dysfunction, increased production of reactive oxygen species (ROS), and a special focus has been done on their close relation with the inflammatory status, even at chronic subclinical grade [START_REF] Zhou | Inflammation: Therapeutic targets for diabetic neuropathies[END_REF]]. Furthermore, even at pre-diabetic states, neurofunctional abnormalities have been reported in high-fat-fed (HFD)-mice in relation with the accumulation of ROS and other various species such as sorbitol and lipoxygenase, leading to unsaturated fatty acids oxidation and involved in endothelial dysfunction and inflammation [START_REF] Obrosova | Highfat diet induced neuropathy of pre-diabetes and obesity: effects of "healthy" diet and aldose reductase inhibition[END_REF]].

Complex mechanisms link hyperglycemia to neurodysfunction, presumably involving a crosstalk between oxidative stress and inflammation. The early prevention and treatment of hyperglycemia and its neurological consequences is therefore essential for the protection of the peripheral nervous system [START_REF] Zhou | Inflammation: Therapeutic targets for diabetic neuropathies[END_REF][START_REF] Sandireddy | Neuroinflammation and oxidative stress in diabetic neuropathy: futuristic strategies based on these targets[END_REF].

Even if diabetic neuropathies curative solutions are still to be developed, therapeutic strategies could rely on both pathogenetically-based and symptomatologic approaches [START_REF] Dewanjee | Molecular mechanism of diabetic neuropathy and its pharmacotherapeutic targets[END_REF][START_REF] Pokhriyal | A review on diabetic neuropathy: Complications and treatment[END_REF]]. Pathogenesis-targeting therapies include -metabolic control by hypoglycemic agents and diet, -use of drugs active on DN ethiology with the aim to lower or prevent the oxidative stress and neurovascular damages. Among the best candidates are -benfotiamine that regulates both hexosamine and PKC pathways and increases glutathione (GSH) production [START_REF] Hammes | Benfotiamine blocks three major pathways of hyperglycemic damage and prevents experimental diabetic retinopathy[END_REF]],antioxidants, radical scavengers or NADPH oxidase inhibitors to reduce oxidative stress, -aldose reductase inhibitors that lower sorbitol production by the polyol pathway [START_REF] Varkonyi | Diabetic neuropathy: new strategies for treatment[END_REF][START_REF] Dewanjee | Molecular mechanism of diabetic neuropathy and its pharmacotherapeutic targets[END_REF], -inhibitors of AGEs formation or action, including polyphenols [START_REF] Tang | Curcumin eliminates the effect of advanced glycation end-products (AGEs) on the divergent regulation of gene expression of receptors of AGEs by interrupting leptin signaling[END_REF][START_REF] Khan | Inhibitory effect of phenolic compounds and plant extracts on the formation of advance glycation end products: A comprehensive review[END_REF]]. Benzoic acid hydrazides or thioxanthines belong to irreversible MPO inhibitors [START_REF] Galijasevic | The development of myeloperoxidase inhibitors[END_REF]], but competitive MPO substrates such as flavonoids may also be active [START_REF] Shiba | Flavonoids as substrates and inhibitors of myeloperoxidase: molecular actions of aglycone and metabolites[END_REF]]. Symptomatic therapies aim to alleviate DN-induced pain and rely mainly on antidepressant, anticonvulsant or analgesic molecules [START_REF] Varkonyi | Diabetic neuropathy: new strategies for treatment[END_REF]].

Numerous studies have been devoted to the antidiabetic action of plant extracts and natural compounds but their impact on neurological complications has less often been investigated [Galuppo 2011[START_REF] Singh | Management of diabetic complications: A chemical constituent based approach[END_REF]. Phenolics known for their antioxidant activity can lower oxidative stress related to diabetic status, but they may also exert other beneficial effects against diabetes-induced neuropathies. A large panel of polyphenols such as resveratrol, phenolic acids and numerous flavonoids from natural origin were reported as AGEs inhibitors [START_REF] Khan | Inhibitory effect of phenolic compounds and plant extracts on the formation of advance glycation end products: A comprehensive review[END_REF]]. Quercetin efficiently inhibits MPO activation [START_REF] Shiba | Flavonoids as substrates and inhibitors of myeloperoxidase: molecular actions of aglycone and metabolites[END_REF][START_REF] Loke | Quercetin and its in vivo metabolites inhibit neutrophil-mediated low-density lipoprotein oxidation[END_REF]] and Gingko biloba extract (EGb761) exerts an anti-inflammatory action on microglial cells [START_REF] Gargouri | Antineuroinflammatory effects of Ginkgo Biloba extract EGb761 in LPS-activated primary microglial cells[END_REF]]. Puerarin, an isoflavon glucoside used in traditional chinese medicine, improves nerve conduction velocity [START_REF] Xie | Efficacy and Safety of the Injection of the Traditional Chinese Medicine Puerarin for the Treatment of Diabetic Peripheral Neuropathy: A Systematic Review and Meta-Analysis of 53 Randomized Controlled Trials[END_REF]] and the prescription BAIMEI-SAN was found to be efficient against peripheral neuropathy, improving nerve conduction and decreasing enzyme release and neurite loss in cortical cell cultures [START_REF] Liu | Effective compounds group of Mongolian prescriptions BAIMAI-SAN protect against peripheral neuropathy in lower limbs of rats through neuro protective effect[END_REF]]. Other medicinal extracts were reported to exert protection against diabetic neuropathies in diabetic rats by promoting myelination processes [START_REF] Hao | The protective effect of the active components of ERPC on diabetic peripheral neuropathy in rats[END_REF]] and improving sciatic nerve structural and functional features [START_REF] Yang | Mechanism of Tang Luo Ning effect on attenuating of oxidative stress in sciatic nerve of STZ-induced diabetic rats[END_REF]].

We have recently investigated the antioxidant properties of polyphenol-enriched extracts of the Saharian traditional medicine Anvillea radiata Coss & Dur and showed that relevant biomarkers of type 2 diabetes and obesity were strongly improved by oral administration of these extracts in a model of HFD-fed mice [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]].

In the present work, we extended our focus area by investigating the properties of two watersoluble Anvillea radiata extracts in their protection against inflammation and diabetes-induced neuropathies. Afferent markers of diabetic neuropathies were studied in vitro including cellular models, and in vivo in HFD-fed mice, together with other biochemical and functional aspects characterizing diabetic renal and vascular complications

Materials and Methods

Standards, reagents and cell culture media

Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com) Buffers, reactants and enzymes, including glucose, hydrogen peroxide (H2O2), bovine serum albumin (BSA), xanthine oxidase (XO, from buttermilk), lipopolysaccharides from Salmonella enterica serotype typhimurium (LPS, ref L6143), L-phenylephrine hydrochloride, ACh perchlorate, diphenyleneiodonium (DPI), methylglyoxal, were purchased from Sigma Aldrich (Saint Quentin Fallavier, France). Purified myeloperoxidase (MPO, ref MY176) was supplied by Elastin Products Co (Owensville, USA) and 7-hydroxy-2-oxo-2H-chromene-8-carbaldehyde oxime (7-HCCO) was synthesized as previously reported [Stocker 2017]. HPLC grade solvents, including methanol, nbutanol and ethylacetate were from VWR Chemicals (Fontenay sous Bois, France). Media for cell cultures including CellTiter 96® Aqueous One Solution Cell Proliferation Assay kit (MTS) were purchased from Promega, France. CD3 antigen, Tissue-Tek O.C.T. Compound, anti-rabbit Alexa 555. Deionized water obtained from a Purelab station (Elga, France) was used throughout. Screening fluorescence assays were performed with a microplate spectrofluorimeter TECAN Infinite 200 (TECAN, Männedorf, Switzerland).

Plant material

The aerial parts of Anvillea radiata (flowers and leaves) were collected in the Algerian part of Sahara (El Oued, alt 69 m) in April 2013. The sample was identified by Prof. N. Khalfallah (Laboratoire de Génétique, Biochimie, et Biotechnologies Végétales, Université Frères Mentouri, Constantine, Algeria). A voucher specimen (LBE13/01) has been deposited in the herbarium of the Faculty of Natural and Life Sciences at Université des Frères Mentouri [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]]. Informations about the traditional use and preparation methods of the plant were obtained from local inhabitants.

Extraction of plant material

Extraction procedures leading to aqueous and organic extracts were driven as reported by Kandouli et al, 2017 [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]]. Preparation of n-butanolic extract (n-BuOHextr): briefly, the dried plant material (200 g) was first extracted at room temperature for 24 h with methanol (1.2 L) then filtered and the procedure repeated twice. This extract was concentrated at 37°C until dryness under reduced pressure and treated with hot water (300 mL) to dissolve polyphenols, then filtered and the aqueous layer was successively extracted (4x100 mL) by petroleum ether (40-60°C fraction) to remove lipids, ethyl acetate and n-butanol to extract polyphenols. After reconcentration, 2.9g (1.4% w/w) of n-BuOHextr were obtained. Lyophilized aqueous extract (AQLextr) was prepared from aqueous extraction of the dried plant material (10g) with hot water for 30 min. Filtration and lyophilisation at -80°C (Cryotec lyophiliser Cosmos, Saint-Gély-du-Fesc, France) afforded 15% w/w yield of AQL, which was stored at -20°C before use.

Inhibition of xanthine oxidase

Inhibiting effect of A. radiata extracts on xanthine oxidase (XO) was evaluated by microplate assay, measuring the inhibition of XO-catalyzed formation of uric acid from xanthine as previously reported [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]]. Briefly, 150 µL of sample solutions (18.5-600 µg/mL, final concentration) in phosphate buffer (PB, 200 mM, pH 7.5) were added in each well to 25 µL xanthine solution (50 µM, final concentration) and 25 µL EDTA solution (10 µM, final concentration) in PB (0.31% v/v DMSO). The reaction was started by addition of 50 µL of XO solution in PB (4.7 mU, final concentration) or buffer alone for control (CTRL). After 30 min incubation at 37 °C, absorbance was measured at 290 nm and inhibition of uric acid formation calculated as ((Abs CTRL -Abs sample) / Abs CTRL) x 100, where Abs CTRL and Abs sample are respectively the net absorbance in absence or in presence of tested samples. Allopurinol and quercetin were used as standards.

Inhibition of myeloperoxidase

The activity of myeloperoxidase was evaluated as previously reported by Stocker et al [Stocker 2017] using 7-HCCO as a fluorescent probe for the detection of HOCl produced after adding H2O2.

Myeloperoxidase was added to 7-HCCO (150 µM) in 1X PBS (pH 6.5) in 96 wells microplate at 37°C in the absence (control, CTRL) or presence of extracts (XX mg/mL) or standards (XX µM), H2O2 (100 µM) was then added to start the reaction. The fluorescence was measured every 3 min (λex=350 nm/ λem=470 nm); its variation (F) was directly proportional to the amount of active MPO and a standard curve was used to determine the concentration of the hypochlorous acid produced. One unit (U) of the enzyme corresponds to the formation of 1 µmol HOCl per min at 37°C calculated from the initial rate of reaction with H2O2 as substrate. Inhibition of the specific MPO activity (As) was expressed as the ratio ((As CTRL -As sample) / As CTRL) x 100, where As CTRL is the measured activity in absence and As sample in presence of tested samples at a given concentration.

Inhibition of advanced glycated end-products (AGEs) formation in presence of BSA

BSA-Glucose assay: The procedure was driven as previously reported [START_REF] Vidal | High throughput assay for evaluation of reactive carbonyl scavenging capacity[END_REF]]. Briefly, BSA (50 mg/ml) was incubated with glucose (20 mM) in phosphate buffer (300 mM, pH 7.4) at 37 °C for 24h together with tested extracts or reference samples. Control was performed with BSA alone. Fluorescence of the advanced glycated end-products was monitored using a microplate spectrofluorimeter (λex=330 nm / λem=394 nm). Half maximal inhibitory concentration (IC50) values of AGEs inhibition were expressed as aminoguanidine equivalents.

BSA-methyglyoxal assay: The procedure was driven as previously reported [START_REF] Vidal | High throughput assay for evaluation of reactive carbonyl scavenging capacity[END_REF]]. Briefly, BSA (50 mg/ml) was incubated with glucose (20 mM) in phosphate buffer (300 mM, pH 7.4) at 37 °C for 24h together with tested extracts or reference samples. Control was performed with BSA alone. Fluorescence of the advanced glycated end-products was monitored using a microplate spectrofluorimeter (λex=330 nm / λem=394 nm). Half maximal inhibitory concentration (IC50) values of AGEs inhibition were expressed as aminoguanidine equivalents.

BSA-Formaldehyde assay: The procedure was adapted from a previous study [START_REF] Mai | Metformin scavenges formaldehyde and attenuates formaldehyde-induced bovine serum albumin crosslinking and cellular DNA damage[END_REF]]. Briefly, BSA (XX mg/ml) was incubated with formaldehyde (10 mM) in phosphate buffer (300 mM, pH 7.4) at 37 °C for h together with tested extracts or standards. Control was performed with BSA alone. Fluorescence of the advanced glycated end-products was monitored using a microplate spectrofluorimeter (λex= nm / λem= nm)

Experiments on rat neonatal microglial cells

Sprague-Dawley neonatal rat primary microglial cells were prepared according to the protocol of Ni et al [START_REF] Ni | Neonatal rat primary microglia: isolation, culturing, and selected applications[END_REF]]. Experiments were adapted from procedures reported previously [START_REF] Gargouri | Antineuroinflammatory effects of Ginkgo Biloba extract EGb761 in LPS-activated primary microglial cells[END_REF]]. MTT cell viability assay: microglial cells (3 × 10 5 cells/well) were plated on 96-well microplates and incubated for 24h in 10% CO2 at 37°C with A. radiata extracts (10-20 µg/mL). After treatment, cells were incubated with MTS Assay kit (20 µl) for 4 h. After incubation for 24 h at 37 °C in an atmosphere of 5% CO2, the supernatant was removed and the formazan crystals formed in the viable cells were solubilized in dimethylsulfoxide (DMSO). Absorbance was measured at 570 nm by using an ELISA MRX micro-plate reader (Dynex Technologies, Chantilly, USA). MTT (0.5 mg/ml) was added to the medium, incubation was extended for 1 h and the conversion of MTT to a purple formazan precipitate was monitored at 570 nm [Culcasi 2012]. TNF, IL-6, and IL-1release by microglial cells: A LPS stock suspension was prepared in sterile phosphate-buffered saline (PBS, 5 mg/ml). Cultured primary microglial cells were pre-incubated with A. radiata extracts (XX µg/ml) for 30 min and then treated with or without (CTRL) LPS (10 ng/ml) for 24 h. Supernatants were then collected and centrifuged (1000 × g, 5 min, 4 °C). Release of TNF (A), IL-6 (B) and IL-1 (C) were measured by ELISA assay (source??). PGE2 and 8-iso-PGF2α production were assessed in the supernatants with a commercially available enzyme immunoassay (EIA) kits (Biotrend, Cologne, Germany or Cayman Chemicals, Ann Arbor, Michigan, USA, respectively). For PGE2, standards from 39-2500 pg/ml were used and the lower limit of quantification (LLQ) of the assay was 36.2 pg/ml. The 8-iso-PGF2α assay has a range of standards from 0.5-500 pg/ml with a LLQ of approx. 3 pg/ml. Values are presented as the mean ± SEM of 4 independent experiments in triplicate. Statistical analyses were carried out by using one-way ANOVA and Tukey posthoc test with *: p < 0.05 and ***: p < 0.001 against the LPS-stimulated positive control.

Activity of A. radiata extracts on MGO-exposed rat primary astrocytes

Primary astrocytes were isolated from 1 day Sprague-Dawley rats and treated according to the procedure reported by Chu et al [START_REF] Chu | Ginsenosides attenuate methylglyoxal-induced impairment of insulin signaling and subsequent apoptosis in primary astrocytes[END_REF]]. To assess the effect of MGO exposition and the activity of extracts, cells (3 × 10 5 cells/well) were seeded on 96 wells microplates in DMEM low glucose medium supplemented with 1% fetal bovine serum and MGO (700 µM) was added. Cells were then incubated for 24 h with extracts (XX mg/ml) or without. Astrocytes viability was estimated from an MTT assay (see paragraph 2.7) and compared to untreated cells.

Animal and study design

Standard diet (SD; 11% energy by fat) and high-fat diet (HFD, 60% energy by fat) were purchased from Safe (Augy, France). All diets contained 20% proteins. Animal care and experimental procedures were performed according to the rules of the European Union Council (Revised Directives 2010/63/EU). This protocol was subjected to the scrutinity of the local Animal Research Ethics Committee and was approved for the project FEDER-AdiabaOx (2008, N° 13851). Aix Marseille University and the CNRS have a license for animal housing and experimentation (agreement C13-055-06) and the study was under the supervision of a vet at the CNRS. Seventy-two 4 weeks-old female C57BL/6 J mice were purchased from CERJ (Janvier Labs, Le Genest St Isle, France) and maintained under conventional conditions with controlled temperature (22 ± 3 °C) and a 12 h light/ dark cycle and access to food and water ad libitum. An enriched environment, by using wheels, tunnels and toys, was given to promote physical and social activity. After one week acclimatization mice were assigned to the following experimental protocols: A first protocol examined the acute oral toxicity of AQL and n-butanol extracts at 2000 mg/kg on 16-weeks SD animals (n = 4/group) according to the procedure of Yamanaka et al. (1990). A second protocol consisted of a dose-response study of the acute hypoglycemic activity of the extracts on 16-weeks HFD animals in comparison with glibenclamide (n = 6/group) as detailed below. Based on our previous results on AQL and n-BuOH extracts toxicity (non toxic up to 2000 mg/kg) and acute hypoglycemic activity by oral administration on C57BL/6 J mice [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]], the dose of 75 mg/kg per day was retained for the present work. Mice (n = 72) were randomly divided into 6 experimental groups (n = 12): controls (given SD for 28 weeks); HFD group (given HFD for 28 weeks); [HFD+AQL75] group, given HFD for 16 weeks, followed by HFD supplemented with AQL (75 mg/kg) for 12 weeks; [HFD+n-BuOH75] group, given HFD for 16 weeks, followed by HDF supplemented with butanol extract (75 mg/kg) for 12 weeks; [HFD+Met150] group, given HFD for 16 weeks, followed by HFD supplemented with metformin (150 mg/kg) for 12 weeks and [HFD+Met75+AQL75] group, given HFD for 16 weeks, followed by HFD supplemented with a mixture of metformin and AQLextr (both at 75 mg/kg) for 12 weeks. Weekly weights measurements and daily estimates of food intake and clinical signs of suffering, weight loss and moribundity were recorded. At weeks 16 (before incorporation of extracts in the food) and 28, blood samples were collected from tail vein under local anaesthesia induced by 0.25% lidocaine application in ointment. Blood glucose concentration was measured using a commercial whole-blood glucose auto analyzer (Freestyle Optium Neo, Abbott)

Glucose and insulin plasma levels and lipid metabolism 2.11. Plantar Test

The thermal withdrawal latency was measured according to the protocol described by De Gregorio et al [START_REF] De Gregorio | Characterization of diabetic neuropathy progression in a mouse model of type 2 diabetes mellitus[END_REF]. In a room kept at 25°C, mice were placed in an acrylic box provided with an infrared (IR) light (UgoBasile Plantar Test, Varese, Italy) 20 min before experiment. The IR light (40% of power) was placed beneath of the mid-plantar surface of hind paws and the withdrawal responses were automatically recorded, with a cut-off latency of 15 s to prevent hind paw damage.

Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com)

The stimulation was repeated three times with an interval of 5 min and repeated three consecutive days. Data were expressed as the mean of withdrawal latency registered each day.

T cells infiltration in the sciatic nerve

T cells infiltration in the sciatic nerve was measured following the procedure of De Gregorio et al [START_REF] De Gregorio | Characterization of diabetic neuropathy progression in a mouse model of type 2 diabetes mellitus[END_REF]. Sciatic nerves were carefully excised at week 33 from diabetic and control mice, fixed in phosphate buffer (XX mM, pH, 4% paraformaldehyde) for 24 h at 4°C and then kept in sucrose (30%) for 24-72 h at 4°C and embedded in Tissue-Tek O.C.T. Compound. Longitudinal thick nerve cryosections were placed on silanized slides. The samples were then blocked for one hour in 5% fetal bovine serum, 0.025 Triton X-100, 0.5 M Tris buffer (pH) and incubated overnight with antimouse CD3 (Dako, Santa Clara, USA, 1:100). They were then washed three times with 0.025 Triton X-100, 0.5 M Tris buffer and incubated (25°C, 2h) with secondary antibody (1:300, anti-rabbit Alexa 555, Cell Signa Technology, Danvers, USA). The number of CD3+ cells was quantified and normalized versus nerve area.

Isolated aortic rings preparation and treatments

Protocols driven for the preparation and treatment of aortic rings were as previously reported by Cassien et al [START_REF] Cassien | On the vasoprotective mechanisms underlying novel β-phosphorylated nitrones: Focus on free radical characterization, scavenging and NO-donation in a biological model of oxidative stress[END_REF]]. At week 33, mice were deeply anesthetized by intraperitoneal injection of sodium pentobarbital (100 mg/kg; Ceva Santé Animale, Libourne, France) before thoracotomy. The aortas were removed, cleaned and cut into 3-mm ring segments, mounted between two stainless steel hooks and suspended at 37 °C in oxygenated, isolated 10 mL baths filled with a modified Krebs-Henseleit (KH) buffer (pH 7.35) containing (in mM): NaCl, 118; KCl, 4.7; KH2PO4, 1.5; CaCl2, 2.5; MgSO4, 1.2; NaHCO3, 25; EDTA, 0.5; glucose, 11. The medium was renewed every 20 min and continuously aerated with a 5% CO2-95% O2 gas mixture. Changes in tension were recorded using a standard apparatus (Harvard Apparatus, Les Ulis, France). Rings were first equilibrated for 60 min at 1 g of resting tension and then stretched step by step until optimal and reproducible reference contraction to high-potassium physiological salt solution (KPSS, 123 mM KCl) was obtained. Then, after a 20 min wash out, rings were contracted to 50% of the KPSS response by 10-7 M Lphenylephrine hydrochloride and endothelial integrity was tested by adding 10 -5 M ACh perchlorate (ou Cl-IBEMECA, cf Nishat 2016) to the medium. The endothelium was considered intact if ACh induced a relaxation of 80% or higher and the rings fulfilling this condition were reequilibrated for 30 min in KH buffer before the experiments.

We also examined the protection afforded by A. radiata extracts against tissue damage caused by endogenous O2 - release as a result of NADPH stimulation of endothelial NADPH oxidase. Rings were first preincubated for 30 min in KH buffer containing 0.1% DMSO containing the selected extracts (10 mg/ml) and 30 µM DTC (Sigma-Aldrich), then NADPH (1 mM) was added and incubation was prolonged up to 60 min. Two equal sets of individual rings were then transferred into separate wells prefilled with 0.3 mL of the same KH buffer/extracts or vehicle/NADPH mixture as described above, and further incubated for 30 min in the dark at 37°C. One subset of rings was kept for tissue protein carbonyls measurements and the other subset was used for EPR experiments. Briefly, to each well containing NADPH-stimulated rings in KH buffer was added an aliquot of the selected extracts in DMSO as to reach a final concentration of 15 mM PBN and 1% DMSO, and the mixture was incubated for 20 additional min. For each extract, three 300 mL-aliquot of the mixture were placed in 1 mL Nalgene® cryogenic vials and immediately stored in liquid nitrogen for delayed EPR analysis. Control wells were performed without NADPH. In additional experiments, NADPH oxidase was inhibited by adding DPI (15 mM) to the medium during the 30 min preincubation phase.

EPR analysis of PBN spin trapping in aortic rings

Previously frozen samples of aortic rings(see above) were thawed just prior to use, placed into capillary glass tubes and EPR spectra were recorded 45 sec after sample thawing, on a Bruker ESP 300 spectrometer (Karlsruhe, Germany) with the following instrument settings: modulation frequency 100KHz, microwave power 10 mW, field axis resolution 2048 pts, modulation amplitude 0.07 mT, time constant 81.92 ms, receiver gain 8 x 10 4 , scan rate 0.31 mT/s, sweep width, 13 mT, 10 accumulated scans.

Protein carbonyls determination in aortic tissue

Protein carbonyls content was evaluated in ring tissues homogenates according to the protocol of Stocker et al [START_REF] Stocker | Fluorimetric screening assay for protein carbonyl evaluation in biological samples[END_REF]] using the 7-hydrazino-4-nitrobenzo-2,1,3-oxadiazole (NBDH, Sigma-Aldrich) as a probe to form highly fluorescent derivatives with aldehydes via hydrazone formation. Homogenates were prepared by homogenizing 0.1 g of fresh aortic tissue powdered in liquid N2 in 1.15% KCl ice-cold solution. After centrifugation (3000×g, 40 min, 2-3 °C), supernatants were diluted 10-fold with PBS 0.1 X. Protein concentrations were determined using the BCA protein assay kit (Thermo Fisher Scientific). Then 100 µL of diluted samples were placed in 96-well microplates and reacted for 15 min at room temperature with an aliquot (100 µL) of the NBDH assay kit (Carbofax®; Yelen) and concentrations in NBDH-reacted carbonyls were determined by fluorimetry (TECAN Infinite 200, exc = 500 nm, em = 560 nm) relative to standard calibration curves built from oxidized BSA. The protein carbonyl content was expressed as nanomoles/mg protein.

Myeloperoxidase activity in mice plasma

MPO activity was evaluated as previously reported in Stocker et al [Stocker 2017] using 7-HCCO as a fluorescent probe for HOCl detection. Plasma were collected from control and diabetic mice at week 33, then diluted ten-fold in 1X PBS pH 6.5. Ten µl of 7-HCCO (100 µM, final concentration) and five µl H2O2 (100 µM, final concentration?) were added to 150 µl of diluted plasma and the fluorescence recorded each 3 min (ex = 350 nm, em = 470 nm). Active MPO content was estimated from the relative fluorescence using a calibration curve built with purified human MPO in plasma.

MDA-TBA assay in mice plasma

Lipid peroxidation was evaluated through malondialdehyde-thiobarbituric acid (MDA-TBA) formation assay [Stocker 2017]: Aliquots (50 µL) of plasma were added in test tubes to a mixture of 450 µL of thiobarbituric acid solution in orthophosphoric acid (0.3 M, pH 3.5) and 10 µL of butylated hydroxytoluene solution (5 mM) in ethanol. Samples were heated at 95 °C for 30 min, then cooled back to room temperature and centrifuged (1800 × g, 10 min). The supernatant content in MDA-TBA adduct was assayed by HPLC (Merck Hitachi/Lachrom HPLC system: autosampler, pump L-7100, fluorescence detector L-7480, interface D-7000, degasser L-7612, with Agilent EZChrom Elite Compact software (Pleasanton, USA)). Analyses were performed using a C18-bonded Macherey-Nagel silica column (250 mm × 4.6 mm, 5 mm) and fluorescence detection (ex = 532 nm, em = 553 nm). Samples (20 µL) underwent isocratic elution (A/B: 50/50), solvent A: 20 mM phosphate buffer (pH 6.9) and solvent B: methanol, at a flow rate of 1 ml/min for 10 min, with peak area quantification. MDA-TBA levels were calculated from a standard curve of MDA tetrabutylammonium and are the means of three independent experiments in each group made in triplicate. Data are expressed in nmole/mg protein.

Evaluation of myeloperoxidase and xanthine oxidase activities in kidney homogenates

Kidney sample were collected at week 33, washed with PBS 1X, then homogenized in RIPA buffer and centrifuged (12,000 rpm, 10 min) at 4°C and stored at -80°C as reported by Chowdury et al [Chowdury 2019]. MPO activity was measured as reported in paragraph 2.6. Evaluation of XO activity was performed by microplate assay by adding in each well 10 µl of tissue homogenate, 190 µl of phosphate buffer (50 mM, pH 7.5) and 100 µL of xanthine solution in PB (75 µM, final concentration). After incubation at XX°C for XX min, absorbance was measured at 290 nm and XO activity calculated as reported in paragraph 2.4.

Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com)

Results and Discussion

Anvillea radiata aerial parts extraction processes and phenolic content characterization

Extraction of the aerial parts of the plant A. radiata and HPLC analysis of their phenolic content were driven as reported in Kandouli et al [Kandouli]. Petroleum ether extraction allowed the removing of lipids from the initial methanolic extract, which was reconcentred and taken back with hot water. Then further successive extractions of the resulting aqueous layer with ethyl acetate and nbutanol (n-BuOH) allowed to separate less (EtOAcextr) and more (n-BuOHextr) polar fractions (Fig. 1). In our previous study on A. radiata extracts anti-diabetic properties, phenolic profile analyses indicated that the main organic (MeOH, EtOAc, n-BuOH) and aqueous (AQL) extracts were rich in flavonoids (~41-57%) and hydroxycinnamic acid derivatives (~40-58%), while rather poor in hydroxybenzoic acids (~2-4%). Furthermore, administration of extracts to HFD-fed mice has resulted in a significant decay of hyperglycemia and in a regulation of the lipid profile, together with a lowering of inflammation markers [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]]. In the present study, HPLC analysis led to similar results (data not shown) and we selected two of these extracts, the aqueous lyophilized (AQLextr) and the nbutanolic one (n-BuOHextr) from their phenolic profile, (flavonoids, mg RUE/g DW: AQLextr 14.4 ± 0.8, n-BuOHextr 33.3 ± 1.9; hydroxycinnamics, mg CAE/g DW: AQLextr 17.1 ± 1.3, n-BuOHextr 51.7 ± 4.0; hydroxybenzoics, mg GAE/g DW: AQLextr 1.5 ± 0.0, n-BuOHextr 3.8 ± 0.2), their antioxidant and digestive enzyme inhibition capacities, together with their low cytotoxicity and good solubility. These extracts have also been shown to improve metabolic parameters, reduce hyperglycemia and dyslipidemia on HFD-fed mice [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]].

The Asteraceae family was reported to be of potential interest in the management of diabetic complications, due to the high content in flavonoids, polyphenols, terpenoids, saponins, polysaccharides and alkaloids [START_REF] Singh | Management of diabetic complications: A chemical constituent based approach[END_REF]]. Flavonoids such as breviscapin from Erigon breviscapus and silibinin from Silybum marianum were found to protect respectively against cardiac dysfunctions, myocardial Ca 2+ impairment and nephropathy in streptozotocin-induced diabetic rats [START_REF] Singh | Management of diabetic complications: A chemical constituent based approach[END_REF]]. Anvillea radiata belongs to this Asteraceae family and is known for its traditional use against diabetes mellitus in Algerian Sahara. However, to our knowledge, its properties against diabetes related neuropathies have still not been investigated.

In vitro evaluation of xanthine oxidase and myeloperoxidase inhibition and anti-glycative effect (BSAglucose, BAS-methyglyoxal and BSA-formaldehyde systems) of extracts

Create PDF files without this message by purchasing novaPDF printer (http://www.novapdf.com)

Thermal withdrawal latency

A plantar sensitivity test was used to assess the symptomatic progression of diabetic neuropathy in HFD-fed mice and to measure the protection provided by A. radiata extracts. The paw withdrawal latency triggered by a thermal stimulation allowed us to evaluate the peripheral nerve functional loss/protection. The assay was repeated from week 5 to week 33, starting the administration of each extracts (75 mg/kg) at week 21. Diabetes-induced hypoalgesia, as observed from the strong latency increase observed for HFD-compared to SD-fed mice at week 33, was lowered by about 30% when extracts were given, which demonstrates their efficiency even after long term T2DM induction. If short term diabetes may induce hyperalgesia, long term exposition was reported to induce merely hypoalgesia [START_REF] Uddin | Exploring the Promise of Flavonoids to Combat Neuropathic Pain: From Molecular Mechanisms to Therapeutic Implications[END_REF]]. A. radiata polyphenolic extracts are characterized by a high representation of flavonoids and hydroxycinnamic acids. Flavonoids, depending of their structure, are susceptible to interact through many DN-related processes from reduction of oxidative stress, increased levels of antioxidant enzymes SOD, CAT, GPx (catechin, naringenin, kaemperol, luteolin, rutin, genistein…), to reduction of MDA (catechin, luteolin, rutin, genistein…) and AGEs formation (kaempferol) or lowering of proinflammatory cytokine levels (TNF, IL-1, IL-6…). Behavioral markers may also be improved by flavonoids as for example thermal hypoalgesia which was reduced by baicalein treatment of STZ-diabetic mice [START_REF] Basu | In vitro and in vivo effects of flavonoids on peripheral neuropathic pain[END_REF]]. Various phenolic acids possess antioxidant and antidiabetic activities. Gallic acid administered to STZ-induced rats improved insulin secretion, lipid profile and others metabolic parameters [START_REF] Latha | Insulin-secretagogue, antihyperlipidemic and other protective effects of gallic acid isolated from Terminalia bellerica Roxb. in streptozotocin-induced diabetic rats[END_REF]]. In the hydroxycinnamic series, ferulic acid was beneficial against diabetic neuropathy in STZ-rats by lowering both metabolic and neurologic alterations, improving the antioxidative status and reducing inflammation and apoptosis; combination with insulin enhanced the obtained protection [START_REF] Dhaliwal | Beneficial effects of ferulic acid alone and in combination with insulin in streptozotocin induced diabetic neuropathy in Sprague Dawley rats[END_REF]]. Zhang et al reported that chlorogenic acid (CA), an abundant food polyphenol, decreases voltage-gated potassium channels which are key regulators of membrane potential in sensory neurons, then decreasing neuronal excitability in neuropathic pain [START_REF] Zhang | Chlorogenic acid alters the voltage-gated potassium channel currents of trigeminal ganglion neurons[END_REF]]. CA also protects neuronal cells under MGO induced apoptosis [START_REF] Huang | Cytoprotective effects of phenolic acids on methylglyoxal-induced apoptosis in Neuro-2A cells[END_REF]]. The rosmarinic acid, an ester of caffeic acid, was shown to exert an anti-allodynic effect in STZ-induced diabetic rats [START_REF] Hasanein | Effects of rosmarinic acid on an experimental model of painful diabetic neuropathy in rats[END_REF]].

A B

7 T cell infiltration in sciatic nerve

In addition to hyperglycemia-induced processes, obesity-associated low-grade chronic inflammation may also occur through adipose tissue dysfunctions and subsequent overexpression of pro-inflammatory factors such as TNF from adipose or neural tissues, which may then elicit the recruitment of immune cells such as macrophages and T cells within the endoneurial area [START_REF] Kosacka | The role of dietary non-heme iron load and peripheral nerve inflammation in the development of peripheral neuropathy (PN) in obese non-diabetic leptin-deficient ob/ob mice[END_REF]]. We analyzed the infiltration of CD3 positive cells in the mice sciatic nerve as an inflammation marker in the peripheral nervous system. Sciatic nerves were carefully excised at week 33 from diabetic and control mice and an immunofluorescence assay was performed against the CD3 antigen (Fig. 3B). The high infiltration level observed for the HFD group as compared to the SD group was lowered by about 40 % when HFD-fed mice were treated by extracts at the dose of 75 mg/kg. Of importance is the best result obtained by treatment with combined half doses of AQL+ Metformin (75 mg/kg each), while meftformin alone… In literature, caffeic acid phenyl ester (CAPE), a widespread phytoconstituent, exhibiting a large range of properties including antioxidant, anti-inflammatory and immunomodulatory activities was shown to protect the sciatic nerve of STZ-induced diabetic rats, reducing the total antioxidant status together with MDA and NO levels [START_REF] Yucel | Protective Effect of Caffeic Acid Phenethyl Ester on Oxidative Stress in Diabetic Rat Sciatic Nerve[END_REF]].

Vasorelaxant properties of extracts on isolated mice aortic rings

The peripheral nervous system is rather poorly microvascularized at the endoneurial level and furthermore the length of its axons makes them insufficient to insure by themselves trophic and other molecules transport to distal extremities. Such characteristics may contribute to increase the susceptibility of distal parts of the PNS towards diabetes-induced troubles. DN are associated to vascular damages at both macro and micro vascular levels, where structural changes may occur with in particular a thickening of the basement membranes and a narrowing of the vascular lumina with possible ischemia. Oxidative stress and MPO activation are known to be involved in vascular endothelium damages, leading to disturbances in vascular tone and enhanced vasoconstriction. We evaluated here the endothelium protection afforded by A. radiata extracts on HFD-fed mice aortic rings precontracted by the 1-adrenoreceptor agonist phenylephrine (PE) (Fig 4). Vasoconstriction was measured in response to the agonist. The best vasorelaxant effect was observed for AQL extract, giving quite 30 % protection compared to untreated HFD group. MPO is known to play an important inflammatory role in diabetes and exerts a vasoconstrictive action by NO consuming [START_REF] Neogi | Are either or both hyperuricemia and xanthine oxidase directly toxic to the vasculature? A critical appraisal[END_REF]]. Actually, a positive correlation between MPO inhibition and vasorelaxation can be observed… for extracts.

Vascular protection against NADPH-induced superoxide in mice aortic rings

MPO inhibition and protection against lipid peroxidation in plasma

MPO activity has been reported to induce NO consumption and vascular endothelial dysfunctions. Nishat et al highlighted the link between MPO level enhancement and the overexpression of the adenosine A3 receptor, involved in vasoconstriction processes [START_REF] Nishat | Increased A3AR-dependent vasoconstriction in diabetic mice is promoted by myeloperoxidase[END_REF]]. Figure XX shows that plasmatic MPO levels where significantly enhanced in HFD-fed mice compared to SD group, while groups treated with extracts exhibited XX% reduction compared to HFD untreated mice, giving then the same trend than vasorelaxation results as shown in fig. XX.

Inhibition of XO and MPO in mice kidney homogenates.

The activity of XO in kidney homogenates was strongly increased in HFD-fed group compared to control mice. Treatment with both AQL and n-BuOH extracts considerably lowered this overactivation as shown in Fig. 6A. Moreover, administration of combined [AQL75 + MET75] afforded the best protection, while metformin alone…. The same trend was observed for the detection of MPO activation (Fig 6 B), with a …... In diabetic renal disease, ROS overproduction may arise from various sources including XO, NADPH oxidase, mitochondrial electron-transport chain or MPO [START_REF] Fakhruddin | Diabetes-Induced Reactive Oxygen Species: Mechanism of their generation and role in renal injury[END_REF]]. XO has been shown to promote vascular inflammation and of the known inhibitors of the enzyme, febuxostat was reported to exert a protection against diabetic nephropathy in mice, to attenuate oxidative stress in blood vessels and to lower IL-6 and IL-1 inflammatory cytokine levels, while allopurinol protected human macrophages from ROS induced degradation linked to atherosclerosis [START_REF] Mizuno | Xanthine Oxidase Inhibitor Febuxostat Exerts an Anti-Inflammatory Action and Protects against Diabetic Nephropathy Development in KK-Ay Obese Diabetic Mice[END_REF]]. Then, the protection afforded through XO inhibition on diabetic mice may then occur merely by reducing inflammatory processes rather than lowering hyperuricemia or hyperglycemia as reported on obese KK-Ay mice with diabetic nephropathy [START_REF] Mizuno | Xanthine Oxidase Inhibitor Febuxostat Exerts an Anti-Inflammatory Action and Protects against Diabetic Nephropathy Development in KK-Ay Obese Diabetic Mice[END_REF]]. However, urate production through XO activity may in turn activate MPO and thereby lead to subsequent impairments [START_REF] Neogi | Are either or both hyperuricemia and xanthine oxidase directly toxic to the vasculature? A critical appraisal[END_REF]].

Conclusions

Diabetic neuropathies represent a complex panel of multifactorial pathologies. Many therapeutic agents have been proposed to address the different pathways involved in DN progression, such as anti-glycative agents, aldose reductase inhibitors, anti-inflammatory and antioxidant molecules [START_REF] Dewanjee | Molecular mechanism of diabetic neuropathy and its pharmacotherapeutic targets[END_REF]], but at the present time, there is still a need of safe and multitarget therapies. Phytochemical resources are good candidates for such approaches, owing to their rich composition in phenolics and flavonoids and the traditional knowledge associated with their use. Here we showed through administration of water-soluble AQL and n-BuOH extracts to HFD-fed mice that A. radiata, which is active in metabolic protection against T2DM may also afford a protection against DN, one of its main complications, being efficient on multiple DN-linked pathways. These extracts were shown to reduce oxidative stress and inflammation processes related to glycemic and lipidic dysregulations. They efficiently prevent AGEs and PC accumulation, reduce ROS formation by lowering the activity of enzymes such as XO and MPO, prevent ROS-induced damages through their antioxidant activity [START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]], and enhance the antioxidant status, particularly by preventing GSH decay. As a result, even after long term T2DM induction on HFD-fed mice, we observed in mice treated by AQL and n-BuOH A. radiata extracts a vascular protection and an improvement of the peripheral nerve response, together with a reduction of its inflammation.

_______________________________________________________________________________________

Figure 1 .

 1 Figure 1. Preparation of Anvillea radiata aqueous and organic extracts.

Figure 3 :

 3 Figure 3: A/ Plantar test: measurement of the paw withdrawal latency (week 33) of treated C57BJ/6 HFD-fed mice compared to SD and HFD controls. B/ T Cell infiltration (week 33) in sciatic nerve of C57BJ/6 HFD-fed mice. Both AQL and nBuOH extract were given at 75 mg/kg body weight.

  Addition of excess NADPH to the aortic rings incubation medium resulted in NADPH oxidase activation and superoxide production. We then evaluated the protection afforded by A. radiata extracts by spin trapping experiments and protein carbonyl measurements. PBN spin trapping of NADPH-induced superoxide radical in aortic rings resulted in observation by EPR of PBN-OH spin adducts, derived from superoxide scavenging (Fig5A). Extracts afforded…Superoxide quenching properties…. Protein carbonyl content: The activation of NADPH oxidase led to a strong increase of protein carbonyl content, which was considerably lowered in …groups, this result being in good accordance (Fig 5B)with the antioxidant and superoxide quenching properties of A. radiata extracts[START_REF] Kandouli | Antioxidant and antidiabetic properties of water soluble extracts from Anvillea radiata in high-fat-fed mice[END_REF]].