Zeynep S ¸uvak
email: zsuvak@ed.ac.uk

Miguel F Anjos
email: anjos@stanfordalumni.org

Luce Brotcorne
email: luce.brotcorne@inria.fr

Diego Cattaruzza
email: diego.cattaruzza@centralelille.fr

Design of Poisoning Attacks on Linear Regression Using Bilevel Optimization

Keywords: Poisoning attacks, bilevel optimization, regression, adversarial machine learning

Poisoning attack is one of the attack types commonly studied in the field of adversarial machine learning. The adversary generating poison attacks is assumed to have access to the training process of a machine learning algorithm and aims to prevent the algorithm from functioning properly by injecting manipulative data while the algorithm is being trained. In this work, our focus is on poisoning attacks against linear regression models which target to weaken the prediction power of the attacked regression model. We propose a bilevel optimization problem to model this adversarial process between the attacker generating poisoning attacks and the learner which tries to learn the best predictive regression model. We give an alternative single level optimization problem by benefiting from the optimality conditions of the learner's problem. A commercial solver is used to solve the resulting single level optimization problem where we generate the whole set of poisoning attack samples at once. Besides, an iterative approach that allows to determine only a portion of poisoning attack samples at every iteration is introduced. The proposed attack strategies are shown to be superior than a benchmark algorithm from the literature by carrying out extensive experiments on two realistic datasets.

Introduction

Today, many applications from different contexts rely on machine learning (ML) algorithms to generate automated decisions since ML based systems constantly self improve without an exogenous intervention and are capable of adjusting themselves to the changing data. The widespread usage of ML algorithms has raised questions about how vulnerable they are against malicious data coming from untrusted sources [START_REF] Dalvi | Adversarial classification[END_REF][START_REF] Szegedy | Intriguing properties of neural networks[END_REF][START_REF] Su | One pixel attack for fooling deep neural networks[END_REF]. The input data used to train ML algorithms can be collected from various sources including humans, machines, internet and sensors. Therefore they can be erroneous as the data source is prone to make mistakes like humans. Moreover, the data may be distorted by an adversary to deliberately manipulate the learning phase of predictive models by directly influencing the training data or to affect the accuracy of the model on new data during testing.

In this work, we consider a specific type of attack called poisoning attack where an adversary inserts a small number of poisoned data samples into the training set to mislead the targeted ML process in the course of fitting the best predictive model. Our focus is on determining poisoning attack strategies against linear regression which is an important and powerful method for prediction. Linear regression is an essential class of supervised learning which can be used for different purposes such as to predict the future demand of a product [START_REF] Carbonneau | Application of machine learning techniques for supply chain demand forecasting[END_REF], to optimize the value or price of a new service [START_REF] Madhuri | House price prediction using regression techniques: A comparative study[END_REF], to calculate the risk of a loan [START_REF] Zhang | Comparisons of linear regression and survival analysis using single and mixture distributions approaches in modelling LGD[END_REF] or to personalize the dose of a medical treatment [START_REF] Gage | Use of pharmacogenetic and clinical factors to predict the therapeutic dose of warfarin[END_REF]. Linear regression can be used for classification as well by converting the real valued outputs to class labels as done in logistic regression. Poisoning attacks against classification algorithms have been thoroughly studied in the literature [START_REF] Biggio | Poisoning attacks against support vector machines[END_REF][START_REF] Mozaffari-Kermani | Systematic poisoning attacks on and defenses for machine learning in healthcare[END_REF][START_REF] Carnerero-Cano | Regularisation can mitigate poisoning attacks: A novel analysis based on multiobjective bilevel optimisation[END_REF][START_REF] Muñoz-González | Poisoning attacks with generative adversarial nets[END_REF]. However, there are few works considering poisoning attacks on linear regression learning although the latter is being used in a wide range of applications. This study aims to contribute by assessing how influential poisoning attacks can be on regression learning.

A poisoning attack is a good example of an adversarial game where the first player (the attacker) tries to confuse the second player (the learner) to a maximum extent by determining the poisoning attack samples whereas the second player minimizes its prediction error by learning from the poisoned training data and optimizes the regression parameters based on it. The described adversarial game can be played in such a manner that one of the players is the leader who makes a decision first and the other is the follower whose decision is optimized after observing the leader's action.

Upon the follower's reaction, the leader may want to change its decision as it is no longer the best decision. In two player-games with one leader and one follower, the leader has the advantage of anticipating the follower's reaction and makes the optimal move taking into account the possible response of the follower. In the context of poisoning attacks, the attacker can be considered as the leader while the learner plays the role of the follower. It is typical to model these type of games as bilevel optimization problems [START_REF] Dempe | Foundations of Bilevel Programming[END_REF].

Although previous works point out that a poisoning attack can be modeled as a bilevel optimization problem [START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF], none of them attempts to solve it explicitly. For the purpose of filling this gap, a bilevel formulation for poisoning attacks on linear regression learning and a solution approach are introduced in this paper. As the solution method, an alternative single level nonlinear problem is obtained using the optimality conditions of the follower's problem and the resulting problem is solved using a commercial solver. The poisoning attack samples are generated both by solving the proposed model at once and through an iterative approach by optimizing only a fraction of poisoning attack samples at a time. The performance of these methods is measured with their running times and the Mean Squared Errors (MSE) of fitted regression model on new test data. Our method is compared to a gradient descent algorithm from the literature which is reported to provide the highest average MSE for benchmarking purposes. Based on the experiments with two different datasets, our approach improved the MSE of the benchmark algorithm by 20.4%, on average, and is competitive in terms of running time.

The remainder of this paper is organized as follows. We present the background of poisoning attacks and a brief review of the relevant work from the literature in Section 2. We start Section 3 with the problem definition, continue with the bilevel formulation of poisoning attacks which is followed by the alternative single level problem representation. Section 4 provides the details and implementation of the adopted solution approach. Section 5 includes the computational results obtained for assessing and comparing the performance of the solution approach. Finally, concluding remarks are given in Section 6.

Literature Review

Adversarial machine learning is a specific field of ML focusing on two main research topics. One of them is the attacks that try to deceive ML models by introducing corrupt input in order to prevent the targeted ML model from functioning properly, and the other is the defense mechanisms of the ML models against these attacks. The defense mechanisms include designing algorithms that detect the malicious input as well as the the exploration of robust learning techniques which function acceptably well despite the attacks. Defense strategies are beyond the scope of this paper but the interested reader is referred to [START_REF] Biggio | Wild patterns: Ten years after the rise of adversarial machine learning[END_REF] for a recent survey.

Attacks against ML algorithms have been categorized in the early work of [START_REF] Barreno | Can machine learning be secure?[END_REF] considering how the attacker influences the ML algorithm, how specific the target of the attack is or what kind of malfunctioning is intended. An extended taxonomy of attacks regarding the attacker's goal and knowledge about the system and the extent to which the attacker can control or interfere with the targeted process can be found in the work by [START_REF] Xiao | Is feature selection secure against training data poisoning?[END_REF]. One of the most common attack types is called evasion attacks in which the attacker has no control over the determination of the model parameters but manipulates the test data to get the desired output from the ML model.

Evasion attacks are usually realized by generating adversarial examples to avoid being identified by the ML model [START_REF] Ren | Adversarial attacks and defenses in deep learning[END_REF] e.g. to bypass security checks like spam filters [START_REF] Lowd | Good word attacks on statistical spam filters[END_REF], biometric recognition [START_REF] Alegre | Evasion and obfuscation in speaker recognition surveillance and forensics[END_REF][START_REF] Qian | Spot evasion attacks: Adversarial examples for license plate recognition systems with convolutional neural networks[END_REF] or banking fraud detection systems [START_REF] Carminati | Evasion attacks against banking fraud detection systems[END_REF].

The poisoning attacks that we study in this work differ from the evasion attacks as the attacker has access to the learning/training phase of the ML model. In the literature, various contributions can be found for poisoning attacks against classification algorithms. Among them we mention the work by [START_REF] Biggio | Poisoning attacks against support vector machines[END_REF] which targets support vector machines (SVM) and proposes a gradient ascent method. It is shown to considerably weaken the classification accuracy of SVM on the MNIST database [START_REF] Lecun | Comparison of learning algorithms for handwritten digit recognition[END_REF] The studies on poisoning attacks against regression learning are relatively fewer and more recent compared to the ones against classification learning. The work by [START_REF] Xiao | Is feature selection secure against training data poisoning?[END_REF] is the first to introduce a gradient descent algorithm to generate poisoning attacks against lasso and ridge regression. The authors evaluate how regression coefficients and the importance of features change in the existence of poisoning attack samples. [START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF] extend this work by developing a much faster but less effective attack scheme which generates attack samples from the estimated distribution of the feature space and a defense algorithm that discards the data points with high prediction errors. [START_REF] Wen | With great dispersion comes greater resilience: Efficient poisoning attacks and defenses for online regression models[END_REF] modify the gradient descent poisoning attack algorithm described by [START_REF] Xiao | Is feature selection secure against training data poisoning?[END_REF] through maximizing the variance of the poisoned training data set instead of the total loss due to the prediction error.

Notations and Problem Definition

In the setting of supervised learning, the parameters of a linear regression model are learned on training data which include both feature vectors and observed response values. The training data can be represented as

D t r = {(x t r i , y tr i )} n i=1
where x t r i ∈ [0, 1] d is a d-dimensional feature vector and y tr i ∈ [0, 1] is the response variable. For the implementation of linear regression, the data must be standardized so that all the values are assumed to be between 0 and 1. After learning the parameters of the regression model, the obtained model is used to predict the value of the response variable on a completely new data set, namely test data, which must be standardized in exactly the same way as the training data.

The linear regression model is a linear function f (x, θ) = w x + b = d j=1 w j x j + b that outputs a prediction for the response variable y, i.e. the predicted response, denoted by ŷ, is equal to f (x, θ). The parameter set θ is equivalent to (w, b) ∈ R d+1 which consists of the feature weights w ∈ R d and the bias b ∈ R. Given the training data

D t r = {(x t r i , y tr i )} n i=1 , determining θ = (w, b
) is fundamentally an optimization problem which minimizes the following quadratic loss function:

L(D t r , θ) = 1 n        n i=1 ( f (x t r i , θ) -y tr i ) 2        + λΩ(w), (1)
where the first term is the MSE represented as

MSE(D t r , θ) = 1 n n i=1 ( f (x t r i , θ) -y tr i ) 2 . ( 2
)
The MSE measures the prediction error of the regression model f (., θ) and is calculated as the average of squared residuals over training data samples. The second term of expression (1) is a regularization term to prevent the overfitting of the regression model to a specific set of training data. A good regression fit must have low MSE not only for training data but also for any unseen test data. Using a regularization term is an approach to avoid an overfitted regression model as it might lead to high MSE on test data samples. Furthermore, it can be regarded as a natural defense mechanism of the model against poisoning attacks. The optimization of the regularization parameter, λ, in the existence of poisoning attacks is previously modeled as a multi-level optimization problem and shown to be a powerful defense method [START_REF] Carnerero-Cano | Regularisation can mitigate poisoning attacks: A novel analysis based on multiobjective bilevel optimisation[END_REF]. In our model, we consider ridge regression which uses l 2 -norm regularization, i.e.

Ω(w) = 1 2 ||w|| 2 2 = 1 2 d j=1 w 2 j .
Since the regularization term associated with ridge regression is differentiable over w ∈ R d , l 2 -norm regularization is preferred in our experiments.

A poisoning attack involves finding a set of data points denoted as D p = {(x p k , y p k )} q k=1 to be inserted into the original training data D t r . Once the training data is poisoned, the regression model f (x, θ) learns on D t r ∪ D p ; therefore the objective of the attacker is to generate D p = {(x p k , y p k )} q k=1 so that the prediction error, MSE(D t r , θ), is maximized. The effectiveness of a poisoning attack strategy is measured by two performance metrics: the difference of MSEs on test data obtained with the regression models fitted to poisoned and unpoisoned data, and the time required to generate D p .

Bilevel Optimization Formulation

In our setting, we assume that the attacker has full knowledge about the training data D t r which is classified as white-box attacks [START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF]. In the alternative scenario which is known as black-box attacks, the adversary does not know the exact training data but uses a substitute training data D which has the same distribution as D t r .

White-box poisoning attacks can be modeled as the following bilevel optimization problem which can also be used to formulate black-box attacks when

D t r is replaced with D . max MSE(D t r , θ * ) = 1 n n i=1 ( f (x t r i , θ * ) -y tr i ) 2 (3) s.t. x p k ∈ [0, 1] d k = 1, ..., q (4
)

θ * = argmin θ {L(D t r ∪ D p , θ) : θ = (w, b) ∈ R d+1 }. (5)
In the upper level of the bilevel formulation (3)-( 5), the attacker, also referred to as the leader, determines the feature vectors of poisoning samples, i.e. D p = {(x p k , y p k )} q k=1 . We assume that y p k are fixed prior to optimization and they are not treated as variables of the given problem (3)-( 5). The objective function of the attacker to be maximized is the MSE on D t r instead of the loss function L(D t r , θ) which is preferred in earlier works [START_REF] Xiao | Is feature selection secure against training data poisoning?[END_REF][START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF]. This choice guarantees that high MSE is achieved on real training data instances rather than on the poisoning data samples. Moreover, it prevents the possibility that the regularization term drives the maximization process instead of the prediction error term which is not desired by the attacker. Also note that the objective function (3) of the attacker does not depend on its own decision variables which are x p of the set D p . Instead, the decision variables of the lower level problem, also referred to as the follower, i.e. θ, are used in it. In fact, the constraint (5) that are equivalent to the follower's regression learning problem (or the lower level problem of the bilevel formulation) imply that only optimal values of θ, which is denoted as θ * , are allowed to be used in the upper level objective function. The attacker's choice of D p affects θ * in the lower level problem represented by (5). Hence, the attacker has an indirect control over θ * and objective function (3). The feature values of poisoning attacks must comply with the standardized form as indicated by (4).

In the lower level problem given as (5), θ is optimized assuming that D p is given, so that the quadratic loss function over the poisoned training data with a regularization term is minimized. The explicit form of the follower's objective function is written as

L(D t r ∪ D p , θ) = 1 n + q        n i=1 ( f (x t r i , θ) -y tr i ) 2 + q k=1 ( f (x p k , θ) -y p k ) 2        + λΩ(w). (6)

The Single Level Nonlinear Optimization Formulation

Karush-Kuhn-Tucker (KKT) optimality conditions of the lower level problem can be retrieved easily if it belongs to the class of convex optimization problems like linear optimization problems or unconstrained convex quadratic optimization problems. In these cases, it is a common practice to replace the lower level problem with its KKT conditions [START_REF] Dempe | Foundations of Bilevel Programming[END_REF]. Since KKT conditions correspond to a set of equations or inequalities, introducing them removes the bilevel nature of the formulation. This does not mean that the difficulty of the problem changes but the described alteration results in a single level formulation which allows using solution techniques suitable for single level problems.

For the purpose of obtaining a single level optimization problem, KKT conditions of the lower level problem ( 5) must be stated explicitly. This is an unconstrained convex quadratic problem with respect to the variable set θ. Then, every local minimum is a global one and the first order optimality conditions, i.e. the necessary optimality conditions, are sufficient for global optimality. In other words, the first order optimality conditions for the lower level problem (5) correspond to its KKT optimality conditions. The first order optimality conditions are the following:

∂L(D t r ∪ D p , θ) ∂w j = 2 n + q        n i=1 ( f (x t r i , θ) -y tr i )x tr i j + q k=1 ( f (x p k , θ) -y p k )x p k j        + λw j = 0 j = 1, ..., d (7) ∂L(D t r ∪ D p , θ) ∂b = 2 n + q        n i=1 ( f (x t r i , θ) -y tr i ) + q k=1 ( f (x p k , θ) -y p k )        = 0 (8)
which state that the partial derivatives of (6) with respect to w j , j = 1, ..., d, and b are equal to zero at optimality.

We substitute the lower level problem (5) with the conditions ( 7) and ( 8) in order to attain the following single level formulation:

max MSE(D t r , θ) = 1 n n i=1 ( f (x t r i , θ) -y tr i ) 2 (9) s.t. ∂L(D t r ∪ D p , θ) ∂w j = 0 j = 1, ..., d (10)
∂L(D t r ∪ D p , θ) ∂b = 0 (11) x p k ∈ [0, 1] d k = 1, ..., q (12) θ = (w, b) ∈ R d+1 . ( 13
)
Notice that the objective function ( 9) of the single level formulation is convex with respect to the variables w and b while the variables x p k , k + 1, ..., q appear only in the constraints ( 10) and ( 11). Therefore, the resulting problem is the maximization of a convex quadratic function with nonlinear constraints.

Solution Approach

This section describes the steps taken to solve the problem ( 9)-( 13) presented in Section 3.

Data Preprocessing

Preprocessing of any given data set is essential to be able to fit a regression model on it. The first condition that must be satisfied is that all feature values must be numerical. The original data set might contain features which are numerically represented like the year a house was built or weight of a person whereas others are categorical like the neighborhood a house is situated in or the ethnicity of a person. The categorical features must be one-hot encoded before applying regression. The one-hot encoding deletes the column of the categorical feature and instead inserts as many feature columns as the number of categories for that feature. Only one of the newly inserted columns, namely that of the applicable feature, is assigned 1 and the others are left as 0. Also, all the numerical values must be standardized to obtain a reasonable regression model by using a simple transformation like min-max scaling so that the range of the values is [0,1].

Initialization of Poisoning Attacks

Since the problem represented by ( 9)-( 13) is non-convex, the returned solution by the solver will be a local maximum that is not necessarily the global optimal solution. Nonlinear optimization solvers require initial values for the decision variables (which are set to zero if unspecified) and the initial choice of D p has a considerable impact on the quality of the returned local optimal solution. For initialization, the approach of [START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF] with rounded inverse labels is adopted. We randomly select q data points from the training data D t r and set them as x p k ∈ D p , k = 1, ..., q. We replace the value of the response y of the randomly picked q data points by the rounded values of (1y) to the nearest integer (either 0 or 1) and set them as y p k ∈ D p , k = 1, ..., q.

Solving the Optimization Problem

The problem ( 9)-( 13) can be solved by commercial nonlinear optimization solvers. One approach is determine values of x p k k = 1, ..., q, by calling the solver just once. This approach is called single attack strategy in the sequel. In the alternative approach, we partition the initialized D p into N disjoint equal sized subsets so that

D p = D p 1 ∪ ... ∪ D p N , D p u ∩ D p v = ∅
MSE(D t r 0 , θ) = 1 n n i=1 ( f (x t r i , θ) -y tr i ) 2 (14) s.t. ∂L(D t r l-1 ∪ D p l , θ) ∂w j = 0 j = 1, ..., d (15)
∂L(D t r l-1 ∪ D p l , θ) ∂b = 0 (16) x p k ∈ [0, 1] d k = 1, ..., K (17)
θ = (w, b) ∈ R d+1 . ( 18
)
Notice that the objective function ( 14) does not depend on the iteration index l and remains the same throughout the whole process whereas the follower relearns the regression parameters on the continuously expanding training data as shown in ( 15) and ( 16). This method is called iterative attack strategy and allows fixing a fraction of poisoning attack samples earlier than the others as opposed to the single attack strategy in which all poisoning attack samples are determined at once. In fact, single attack strategy is a special case of iterative attack strategy where N = 1 and K = q.

The overall iterative attack strategy (IAS) is listed as Algorithm 1.

Dealing With One-Hot Encoded Features

The input training data D t r contains both continuous and binary valued features since the categorical features are transformed into binary valued feature columns during preprocessing. Therefore, the poisoning attack samples must obey the binary definition of one-hot encoded features. This allows to fully decode these features to the original representation of the dataset once the poisoning attack samples are determined. In other words, the corresponding feature values must be strictly binary valued and only one category must be equal to one for a categorical feature. We consider four strategies to deal with the one-hot encoded features. These strategies are discussed here in the following.

Proper formulation of one-hot encoded features. The first option is to formulate the binary valued features properly.

Instead of giving the entire representation, we can describe how they are properly defined with a small example. Let us assume that features j ∈ 1, 2, 3 are the one-hot encoded feature columns corresponding to the categories of a single categorical feature. Then, x p k1 + x p k2 + x p k3 = 1 for any poisoning attack sample k to obtain a valid feature vector. This type of constraints are also known as set-partitioning constraints [START_REF] Padberg | Covering, packing and knapsack problems, in: Discrete Optimization I[END_REF]. Moreover, these features must be defined as binary variables: x p k1 , x p k2 , x p k3 ∈ {0, 1}, k ∈ {1, ..., q}. Binary definition of one-hot encoded features and the set partitioning constraints give a proper representation of one-hot encoded features. Since the proper formulation together with the single level formulation represented by ( 9)-( 13) result in a mixed integer nonlinear optimization formulation, it becomes much more difficult to solve and we will not consider it.

Postprocessing the relaxed binary one-hot encoded features. Another straightforward approach is to include the one-hot encoded features as continuous variables. They are postprocessed after optimization by setting the highest valued category to 1 and the others to 0. Since our initial experiments showed that this strategy does not result in good quality poisoning attacks samples, it is not considered further.

Relaxing the binary-encoded features but keeping set partitioning constraints. In this version, binary restrictions are relaxed but the set partitioning constraints are kept. The final values are postprocessed as described previously.

However, keeping the set partitioning constraints does not have a significant impact and postprocessed poisoning attack points are observed to be poor again based on the preliminary experiments. Therefore, this approach is not adopted to handle the binary valued features.

Leaving binary-encoded features out. The one-hot encoded variables can be fixed to their initial values and only the remaining numerical feature values of the poisoning attacks are optimized. This option reduces the solution space of the problem considerably hence decreasing the time required to find the poisoning attack samples. Although this approach may leave out good solutions including the global optimal one, our experiment results indicate that fixing the binary valued features to their initial values and leaving them outside the optimization procedure leads to generating effective poisoning attack samples. Thus, this strategy is adopted to deal with one-hot encoded features as we carry out the experiments detailed in Section 5.

Experimental Study

In this section we describe the datasets, the testing environment and the test instances used to carry out the computational tests. The detailed and aggregated performance indicator values are reported to assess the efficiency of attack strategies. Furthermore, the impact of poisoning attacks on individual data samples is discussed. The results of an additional set of tests are provided to observe the performance of the proposed attack strategies in black-box scenario. The section is concluded by discussing how the running times can be improved.

Datasets and Test Instances

The experiments are carried out on two regression datasets which are also used by [START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF] and [START_REF] Wen | With great dispersion comes greater resilience: Efficient poisoning attacks and defenses for online regression models[END_REF] and publicly available at https://github.com/jagielski/manip-ml. A brief description of these datasets is provided below.

House Price dataset. This dataset is used to predict the sale price of a house as a function of its features. The features include lot size of the house, type of the dwelling, types of utilities it has (Kaggle). The original dataset consists of 1460 houses and 81 features in total. Preprocessing by one-hot encoding increases the number of features to 275.

Healthcare dataset. Given the attributes of a patient, the therapeutic dose of a drug called Warfarin is estimated using this dataset. The attributes of a patient include physical features like age, weight and gender and other conditions like having diabetes or being a smoker. The preprocessed dataset contains 205 features and 4684 patients.

In order to evaluate the performance of the proposed attack methods, test instances are generated from the mentioned datasets. Each test instance consists of a randomly split training, validation and test data. They have 300, 250 and 500 data points, respectively. Initial poisoning attack sample set, which is a part of the input, is determined as previously described in Section 4.2. Poisoning rate is the ratio of the number of poisoning attack samples to the size of the training data. Five test instances are generated for the poisoning rates in {4%, 8%, 12%, 16%, 20%} which results in 25 test instances for each dataset. Poisoning rates higher than 20% are not considered as only small fractions of poisoning attacks can be injected in realistic settings.

Test Conditions

The training data and the initial poisoning attack samples are the inputs of the optimization problem. Prior to solving the problem, training and validation sets are used together to determine the value of the regularization parameter, λ, by applying 10-fold cross validation. The proposed optimization problems are modeled using GAMS 30.3.0 and solved by the commercial nonlinear solver KNITRO (version 11). KNITRO returns a local optimum due to the nature of the nonlinear optimization problem. The default settings of KNITRO to terminate the optimization process (e.g. time limit of 10 8 seconds, iteration limit of 10,000 iterations, optimality tolerance of 10 -6 ) are used and the solver is able to find local optimal solutions for all test instances. The output of the solver is a set of final poisoning attack samples.

In order to assess the quality of the solution methods proposed in this paper, we also run the gradient descent algorithm of [START_REF] Jagielski | Manipulating machine learning: Poisoning attacks and countermeasures for regression learning[END_REF] to generate poisoning attack samples. The settings of the gradient descent (GD) algorithm are chosen so that the response variable is not optimized. The training data is used to calculate the gradients instead of the validation data and the poisoning attack samples are initialized in the same way as described in Section 4.2.

These settings for GD are preferred since they correspond to exactly the same assumptions we make when we develop our solution approaches. Different settings might lead to draw biased conclusions. To exemplify, a different initial set of points can result in a better set of attack samples regardless of the attack strategy; any additional information coming from the validation data could be advantageous; the ability to play with the value of the response variable gives more flexibility and a larger solution space.

After the optimization problem is solved by iterative attack strategy (IAS), (i) the poisoning attack samples are determined, (ii) a regression model is learned on the poisoned training data and, (iii) the regression model is tested on the test and validation sets. The same steps are repeated for the training data poisoned with the attack samples generated by GD. For the implementation of regression learning, sklearn package of Python is used. All experiments are carried out on a computer running with Microsoft Windows 10 Education operating system and having an Intel Core CPU 1.7 GHz processor with 16.0 GB RAM and it was ensured that the algorithms are run in a single thread environment to make a fair comparison.

Efficiency of the Attack Strategies

For benchmarking purposes, we report the MSE values of the learned regression fits on test and validation data where higher MSE means better attacks. In Figure 1, we plot the MSE values on the different runs for each poisoning rate using House Price dataset. Each point on the plot is the average MSE value obtained over 5 instances for the same poisoning rate. The results show that any type of poisoning attack considerably increases the MSE of regression models. The MSEs with IAS are higher than the ones provided by GD for most of the test instances. We observe that there are cases where IAS is not as good as GD. The points below the GD line in Figure 1 correspond to the single attack strategy together with poisoning rates 16% and 20% indicating that single attack strategy is not better than GD for high poisoning rates. We also observe an exceptionally high MSE of GD for the poisoning rate of 16% in Figure 1(a) where IAS performs better than GD except for the cases K = 1, 2.

Figure 2 shows the plot of the average MSEs of 5 instances on Healthcare dataset for different poisoning rates. In this dataset, we make a similar observation so that the MSE performance of the single attack strategy is poorer than the baseline attack, GD, for poisoning rates 12%, 16% and 20%. The other versions of IAS outperform GD, in general.

These plots reveal that solving the optimization problem in more than one iteration substantially improves the quality of the generated attack samples particularly for high poisoning rates. The tabulated results used to prepare the plots in Figures 1 and2 are reported in the Appendix in Tables 4567.

In order to assess the relative performance of IAS, we calculate the change in MSE with respect to the MSE obtained with GD poisoning for each test instance t and attack strategy a as follows:

MSE ta change (%) = MSE ta -MSE t{GD} MSE t{GD} × 100.
Positive MSE ta change values indicate that algorithm a provides higher MSE than the baseline strategy while negative values point out worse performance in terms of MSE.

Table 1 summarizes the performance of the different attack strategies. The first column of Table 1 lists the attack strategies that we experiment with including the benchmark GD. Since the size of the training data is fixed and equal to 300, poisoning rates of 4%, 8%, 12%, 16% and 20% correspond to generating 12, 24, 36, 48 and 60 attack samples, respectively. Single attack strategy is the special case of IAS where all attack samples are generated at one iteration.

Recall that K represents the number of attack samples generated during each iteration for the remaining strategies. The last row contains the mean values of performance indicators over IAS and single attack strategies. In the second column, the average running time in seconds which is calculated over 50 instances is provided for each strategy. The running time of the methods is defined as the time elapsed to generate poisoning attacks. The time needed to calculate λ, initialization of attacks and learning the regression models are not taken into account since they are common for both IAS and GD and negligibly short. Notice that the single attack strategy is much faster than GD and the running time of IAS (K = 12) is quite competitive with GD. The running time of IAS increases as the value of K gets smaller, as expected. The third column of Table 1 lists MSE a change values representing the average of MSE ta change for each attack strategy a. In other words, MSE a change is the average improvement achieved in MSE by the attack strategy a with respect to GD. This value is calculated by averaging 100 MSE ta change values obtained over the validation and test sets of all instances taken from both datasets. From the third column, it can be seen that all strategies, except for the single attack strategy, achieve similar change of MSE with respect to GD poisoning. The MSE of single attack strategy is 5.7% better than GD while the others achieve at least 21.3% improvement on the MSE obtained with GD. IAS (K = 12) and IAS (K = 4) gives the highest MSE change with respect to GD poisoned regression. As we decrease K, the change in MSE value increases first and slightly decreases when K < 4 while the running times monotonically goes up with descending K values. The summary in Table 1 in seconds which are computed over 70 runs, respectively. The running times are positively correlated with the value of poisoning rates, as expected. The last column gives the relative MSE change with respect to GD poisoned regression averaged over 140 MSE ta change values obtained with the same poisoning rates. The change in MSE with respect to GD regression reveals that our attack strategies output higher MSEs than GD for any poisoning rate and emphasizes that the highest MSE difference is obtained for lower poisoning rates, namely for 4% and 8%. The detailed results which we use to prepare the summary tables, namely Tables 1 and2, can be found in Tables 4-9 of the Appendix .

MSE on Validation Data

Impact of Poisoning Attacks on Individual Predictions

In this section, we discuss the influence of poisoning attacks in real life using House Price and Healthcare datasets.

For this purpose, we focus on the test instances with 4% poisoning rate where the attack strategy is assumed to be IAS with K = 12. We observe that for 54.2% of the houses the price is estimated higher than their real sale price.

Hence, the poisoned regression model does not favor only low or high prices. One extreme example is a house with sale price of 67,000$; it is estimated to worth 409,655$. Similarly, one of the most expensive houses with 745,000$ sale price is predicted to worth only 34,900$. In 2.1% and 10.5% of the houses, the deviation between the real and predicted sale prices, which is calculated as 100 × |ŷ-y| y , are more than 200% and 100%, respectively. Moreover, the percentage of houses whose predicted sale price is at least 75% and 50% away from the original price is 20.3% and 38.2%, respectively. On the other hand, 17.5% of the houses are estimated to have sale prices within 10% of the real value. One third of the houses fall in the category whose prediction error is within 20% despite the poisoning attacks.

Therefore, the net worth of a considerable number of houses are estimated poorly even though the poisoning rate is as small as 4%.

The effect of 4% poisoning rate is even more significant on Healthcare dataset. We can state a patient whose drug dosage is changed from 3.5 to 43.5 mg per week and one whose weekly dosage is reduced from 98 mg to 2.1 mg as individual examples. For 9.3% of the patients, the estimated drug dosage is at least 200% away from the real dose. The drug dosage is predicted to be at least 100% different for 21.9% of the patients. Almost half of the patients (46.3%) and 62.8% of them have their dosage changed by at least 75% and 50%, respectively. Furthermore, the change in dosage is within 10% and 20% for only 7.9% and 14.9% of the patients, respectively. These numbers show that even a small-scale poisoning attack dramatically changes the dosage of a drug negatively affecting the well-being of patients.

Attacks in Shorter Times

The computational results of the experiments show that one disadvantage of IAS can be the running time of the method. The running time is directly related to the default termination settings of KNITRO solver. This section provides a comparison of running times when the optimality tolerance of KNITRO is changed from its default value (10 -6 ) to 10 -3 . We re-ran the experiments for IAS with K = 6 and K = 4. These attacks are repeated only for poisoning rate of 20% for which the average running time is the highest. The experiments with the default setting of optimality tolerance equal to 10 -6 take 745.3 seconds whereas the runs with increased optimality tolerance up to 10 -3 terminates in 525.0 seconds, on average. The average MSE with default and altered settings are equal to 0.088 and 0.087, respectively.

To sum up, experimenting with only one parameter of KNITRO results in 30% improvement in running times of IAS while not changing the MSE significantly. Therefore, IAS is a promising attack strategy for which shorter running times can be achieved by fine tuning the termination parameters of the solver. We also assess how IAS performs in black-box scenario described in Section 3.1. The experiments are carried out assuming that the training data used in the optimization problem is the substitute training dataset D , i.e. the guess of the attacker, since the attacker has limited knowledge about the real training data. We determine D by random selection from the entire dataset. We adopted this approach because we know that both D and D t r have the same distribution. However, it is not known how similar they are. The poisoning attack samples are optimized with respect to D . Then, they are added to D t r and a regression model is learned. Reported MSE is calculated on the real test and validation data of the learner. The performance of attack strategies in black-box scenario are summarized in Table 3. The MSE a change value is calculated by averaging over 100 MSE ta change values for every IAS and the benchmark GD. We observe that MSE obtained with IAS is 0.86% higher than of GD, on average. IAS provides slightly higher MSEs than GD except for the single attack strategy. Furthermore, IAS (K = 12) is the most successful attack strategy with black-box attacks with an average of 4.24% MSE increase. IAS (K = 1) and IAS (K = 2) are the other promising methods in black-box scenario. The summary in Table 3 is obtained from the detailed results provided in Tables 10111213of the Appendix.

Attacks

Conclusion and Future Research

This paper proposes a bilevel optimization formulation for poisoning attacks on ridge regression learning. To the best of our knowledge, it is the first study which explicitly solves a bilevel optimization problem to determine poisoning attacks on regression. To this end, a single level nonlinear formulation equivalent to the bilevel formulation is presented.

The resulting problem is solved using a commercial nonlinear optimization problem solver. An iterative solution approach is developed where a fraction of attacks is determined at every iteration in addition to the straightforward approach in which the optimization problem is solved at a single attempt. The performance of the newly proposed attack strategies is compared to a benchmark gradient descent algorithm from the literature. They are proven to perform significantly better than the benchmark in terms of MSE and they are competitively fast when the number of iterations are kept small. In the white-box scenario, our attack strategies achieve 20% higher MSE than the benchmark does. For small poisoning rates of 4% and 8%, the MSE obtained with the proposed attack strategies are significantly higher than MSE given by the benchmark attack algorithm. When the poisoning rate is 12%, 16% or 20%, the iterative attack strategy is far more successful than the single attack strategy and the benchmark. In the black-box scenario, the newly proposed attack strategies result in equally good attacks to the ones generated by the benchmark.

The attack strategies are promising as there is a room for improving the running times by tuning the parameters of the optimization solver. Future research will also consider applying the described strategies on lasso regression. The mixed integer modeling approach to handle categorical features would also be interesting to explore.

APPENDIX: Detailed Experiment Results

  used for handwriting digit recognition. Mozaffari-Kermani et al. (2015) consider poisoning attacks on a set of different classifiers including decision trees, naive Bayes classifiers and artificial neural networks. They develop a generic poisoning attack scheme which is suitable for any classification algorithm by benefiting from the properties of the attacked class. Koh et al. (2018) focus on generating stronger poisoning attacks against an SVM classifier assuming that the classifier implements various defense algorithms known as data sanitization techniques. An interesting recent study by Muñoz-González et al. (2019) introduces a generative adversarial network (GAN) to generate poisoning attacks against deep neural networks. The proposed GAN allows incorporating detectability constraints which seek to prevent the poisoning attack samples from being detected by defense algorithms.

  and |D p u | = |D p v | for any {u, v} ∈ {1, ..., N}, u v. Note that every subset of D p contains exactly K poisoning samples implying that K = q/N. The poisoning attack samples in D p are optimized in exactly N iterations where only one subset of D p is determined at each iteration. The poisoned training data at iteration l for l = 1, ..., N is denoted by D t r l and found by concatenating the training data associated with the previous iteration and the determined poisoning attack subset at the current iteration , i.e. D t r l = D t r l-1 ∪ D p l where D t r 0 = D t r . The nonlinear optimization problem to be solved in iteration l ∈ {1, ..., N} is given as max

Algorithm 1

 1 Iterative Attack Strategy (IAS) Input: D t r , initialized poisonining attack samples D p , subsets D p 1 , ..., D p N , K Output: Final poisoning attack samples D p * 1: Set l = 1, D t r 0 = D t r 2: while l ≤ N do Solve problem (14)-(18) and let D p l * be the returned solution. Set D t r l = D t r l-1 ∪ D p l * , l = l + 1 3: end while 4: Return D p * = D p 1 * ∪ ... ∪ D p N *

Figure 2 :

 2 Figure 2: The effect of poisoning attacks on MSE for Healthcare dataset

Table 1 :

 1 suggests that considering running time and MSE together, IAS (K = 12) Summary of the performance of the attack strategies is the best strategy. IAS (K = 4) can be considered as another efficient method when MSE is relatively more important than running times. Moreover, IAS with lower K values and the single attack strategy are not as efficient as them. All in all, MSE provided by GD is increased up to 25% with our proposed attack strategies most of which terminate in a reasonable amount of time.

	Attack Strategy Avg Running MSE a change
			Time (s)	(%)
	GD		105.2	0.0
	Single Attack	46.6	5.7
	IAS (K=12)	160.1	23.7
	IAS (K=6)	311.3	21.3
	IAS (K=4)	514.8	25.1
	IAS (K=3)	676.7	22.5
	IAS (K=2)	901.0	22.4
	IAS (K=1)	1,794.5	21.9
	IAS Average	629.3	20.4
	Poisoning	GD Running Avg Running Avg MSE change
	Rate	Time (s)	Time (s)	(%)
	0.04	74.7	163.3	38.4
	0.08	68.4	372.6	23.6
	0.12	74.7	630.5	17.6
	0.16	108.6	974.8	7.2
	0.2	199.8	1,005.3	14.9
	Grand Avg	105.2	629.3	20.4

Table 2 :

 2 Summary of the performance with respect to poisoning rates

Table 2

 2 gives a summary by aggregating the previously stated performance measures for each poisoning rate. The first column contains the poisoning rate values used in the experiments. The second and third columns represent the running time of GD averaged over 10 instances and the average running time of IAS (including single attack strategy)

Table 3 :

 3 Under Black-box Scenario Performance of strategies in black-box scenario

	Attack Strategy MSE a change (%)
	Single Attack	-5.24
	IAS (K=12)	4.24
	IAS (K=6)	0.73
	IAS (K=4)	0.37
	IAS (K=3)	0.11
	IAS (K=2)	2.45
	IAS (K=1)	3.39
	IAS Average	0.86

Table 4 -

 4 Table13include the computational results averaged over 5 instances. They are used to prepare the summary tables Table1-Table3and the plots in Figures1-2, in Section 5.

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.026	0.035	0.035 0.034 0.034 0.035 0.034 0.036
	0.08	0.003	0.045	0.054	0.069 0.055 0.065 0.062 0.060 0.060
	0.12	0.005	0.064	0.078	0.072 0.073 0.063 0.073 0.069 0.068
	0.16	0.003	0.086	0.058	0.092 0.097 0.091 0.085 0.082 0.071
	0.2	0.003	0.072	0.072	0.111 0.101 0.124 0.109 0.113 0.110

Table 4 :

 4 Average MSE values on validation data for House Price dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.025	0.034	0.034 0.032 0.032 0.035 0.035 0.036
	0.08	0.003	0.046	0.048	0.057 0.050 0.054 0.055 0.056 0.057
	0.12	0.003	0.051	0.088	0.066 0.072 0.067 0.069 0.067 0.067
	0.16	0.003	0.077	0.056	0.084 0.094 0.090 0.086 0.087 0.079
	0.2	0.003	0.089	0.065	0.102 0.097 0.108 0.107 0.107 0.104

Table 5 :

 5 Average MSE values on test data for House Price dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.015	0.020	0.020 0.018 0.020 0.020 0.020 0.020
	0.08	0.002	0.027	0.028	0.031 0.034 0.037 0.035 0.033 0.033
	0.12	0.003	0.044	0.033	0.047 0.042 0.044 0.044 0.046 0.045
	0.16	0.003	0.047	0.041	0.052 0.054 0.057 0.048 0.049 0.050
	0.20	0.003	0.061	0.054	0.073 0.071 0.068 0.070 0.077 0.075

Table 6 :

 6 Average MSE values on validation data for Healthcare dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.015	0.019	0.019 0.018 0.019 0.019 0.020 0.019
	0.08	0.002	0.029	0.026	0.030 0.031 0.033 0.032 0.031 0.031
	0.12	0.003	0.043	0.034	0.047 0.045 0.046 0.045 0.045 0.048
	0.16	0.003	0.048	0.044	0.057 0.060 0.062 0.049 0.051 0.054
	0.20	0.003	0.061	0.053	0.068 0.068 0.066 0.066 0.071 0.070

Table 7 :

 7 Average MSE values on test data for Healthcare dataset

	Poisoning	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate		Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	127.1	18.9	22.2	41.0	114.0 106.6	279.1	273.1
	0.08	96.2	32.0	49.4	80.3	94.8	184.5	288.0	867.2
	0.12	91.2	43.6	72.5	111.3 419.1 525.2	614.2	1,058.9
	0.16	148.7	75.4	294.6 222.7 198.0 598.6 1,037.4 1,590.9
	0.2	192.0	118.0	258.9 505.3 337.0 632.2	593.7	2,007.2

Table 8 :

 8 Average running times (in seconds) of different attack strategies for House Price dataset

	Poisoning	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate		Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	22.2	10.8	10.8	26.6	145.5	94.4	227.6	915.6
	0.08	40.6	18.8	80.7	227.2	227.7	505.9	936.2	1,623.9
	0.12	58.1	35.1	182.8	142.9	650.7	1,137.6	910.9	2,922.4
	0.16	68.4	47.7	386.6	691.9	1,628.8 1,656.1 1,913.0 3,305.4
	0.2	207.5	66.0	243.1 1,064.3 1,332.6 1,325.7 2,210.1 3,380.3

Table 9 :

 9 Average running times (in seconds) of different attack strategies for Healthcare dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.038	0.040	0.040 0.034 0.036 0.033 0.036 0.036
	0.08	0.003	0.061	0.059	0.055 0.056 0.060 0.057 0.063 0.062
	0.12	0.005	0.087	0.072	0.086 0.081 0.076 0.081 0.080 0.082
	0.16	0.003	0.105	0.063	0.098 0.091 0.088 0.092 0.091 0.111
	0.2	0.003	0.112	0.081	0.131 0.123 0.111 0.121 0.125 0.115

Table 10 :

 10 Black-box scenario: Average MSE values on validation data for House Price dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.038	0.038	0.038 0.035 0.033 0.037 0.035 0.037
	0.08	0.003	0.058	0.052	0.052 0.053 0.054 0.051 0.056 0.055
	0.12	0.003	0.082	0.074	0.085 0.082 0.072 0.078 0.077 0.083
	0.16	0.003	0.098	0.056	0.087 0.093 0.086 0.088 0.096 0.100
	0.2	0.003	0.119	0.074	0.134 0.127 0.123 0.127 0.127 0.112

Table 11 :

 11 Black-box scenario: Average MSE values on test data for House Price dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.019	0.024	0.024 0.025 0.025 0.023 0.025 0.024
	0.08	0.002	0.040	0.041	0.041 0.041 0.042 0.040 0.040 0.039
	0.12	0.003	0.049	0.043	0.044 0.044 0.046 0.046 0.045 0.042
	0.16	0.003	0.065	0.062	0.068 0.060 0.068 0.065 0.062 0.067
	0.2	0.003	0.081	0.080	0.081 0.079 0.075 0.079 0.080 0.086

Table 12 :

 12 Black-box scenario: Average MSE values on validation data for Healthcare dataset

	Poisoning Unpoisoned	GD	Single	IAS	IAS	IAS	IAS	IAS	IAS
	rate			Attack K=12	K=6	K=4	K=3	K=2	K=1
	0.04	0.003	0.019	0.024	0.024 0.024 0.024 0.023 0.024 0.024
	0.08	0.002	0.034	0.038	0.039 0.038 0.038 0.037 0.038 0.036
	0.12	0.003	0.047	0.043	0.047 0.045 0.045 0.046 0.046 0.044
	0.16	0.003	0.067	0.068	0.067 0.065 0.068 0.065 0.065 0.068
	0.2	0.003	0.084	0.081	0.076 0.072 0.073 0.073 0.074 0.080

Table 13 :

 13 Black-box scenario: Average MSE values on test data for Healthcare dataset


Ines Jammeli

Ahmed Chemori

Salwa Elloumi

Samer Mohammed

A Novel LQR-Based Cascaded Control Scheme of a Powered Knee Joint Orthosis for Rehabilitation

Keywords: Wearable robot, Linear quadratic regulator, Linearization, knee joint impairments, rehabilitation

In order to improve the healthcare services quality, various initiatives have been carried out to develop new assistive technologies such as wearable robots. This paper deals with the control of an actuated knee joint orthosis dedicated to assistive and rehabilitation purposes. The proposed control scheme is based on a nonlinear state feedback complied with a linear quadratic regulator (LQR) in a cascaded control architecture. The proposed control scheme has been validated in simulation for the control of an active orthosis in various operating conditions and compared with a PID controller. The obtained simulation results show clearly the efficiency of the proposed control scheme and its superiority with respect to the PID in terms of the tracking performance and the comfort of the user.

I. INTRODUCTION

The increasing number of people living with physical disability has become a global socioeconomic issue and this is largely due to the rising ageing rate. Elderly people with weakened muscle strength may lose their stability during walking, thus they require the help of other people. The daily living tasks could become really challenging for people with motor impairment. This brings considerable attention on how to provide assistance for the elderly as well as people with lower and/or upper limb pathologies in their daily life, especially concerning mobility and autonomy [START_REF] Yan | Review of Assistive Strategies in Powered Lower-Limb Orthoses and Exoskeletons[END_REF]. To reduce the burden on care services, many initiatives have been set up to promote research on developing assistive care technologies. With recent technological advances, physical assistive robotics is emerging as a promising solution for developing systems to facilitate and improve the daily life conditions for people with reduced mobility. In particular, wearable robots have gained great attention in the last decade for applications related to rehabilitation, assistance, and human capabilities augmentation. Wearable robots are mechatronic devices embodied by the human upper and/or lower limbs that are equipped with sensors and actuators. They are intended to strengthen human physical abilities in the upper/lower limbs, to support people with reduced mobility in daily living activities and to automatically rehabilitate human joints and muscles enabling the recovery 1

Ines

Jammeli and Salwa Elloumi are with LSA, EPT, University of Carthage, ENICarthage Tunis, Tunisia.

jammeli.ines@gmail.com,salwa.elloumi@gmail.com 2 Ahmed Chemori is with LIRMM, University of Montpellier, CNRS, Montpellier, France. Ahmed.Chemori@lirmm.fr 3 Samer Mohammed is with LISSI, Paris-Est Créteil University, Paris, France. samer.mohammed@u-pec.fr and the enhancement of the user's limb control [START_REF] Chen | Recent developments and challenges of lower extremity exoskeletons[END_REF]- [START_REF]Ageing and health[END_REF]. Over the previous few decades, several exoskeletons have been developed and commercialized. The BLEEX, is one of the first motorized lower-limb exoskeleton prototype designed to decrease metabolic consumption, reduce the risk of back and leg injuries, and improve the capabilities of soldiers, firefighters, and rescuers when transporting heavy loads over long distances and in difficult situations. Force sensors are fixed under the soles of both feet, and actuations are performed at the hip, knee, and ankle. This exoskeleton, with seven degrees-of-freedom (DoF) at each leg, adopts a hybrid hydraulic-electric portable power supply [START_REF] Ghan | Control and system identification for the Berkeley lower extremity exoskeleton (BLEEX)[END_REF]. The controller described in [START_REF] Kazerooni | On the control of the Berkeley lower extremity exoskeleton (BLEEX), Robotics and automation[END_REF] uses the inverse dynamics of the exoskeleton as a positive feedback controller. The RoboKnee is a single-degree-of-freedom lower-limb exoskeleton that acts on the knee joint to assist human movement. It allows the wearer to climb stairs and do deep knee bends while handling heavy load in their backpack. The intention of the user is determined through the knee joint angle and ground reaction forces. RoboKnee employs a hierarchical control strategy, which uses a straightforward mid-level force generation scheme based on a positive force feedback amplification coupled with a low-level closed-loop force based on a PD control loop [START_REF] Pratt | The RoboKnee: An Exoskeleton for Enhancing Strength and Endurance During Walking[END_REF]- [START_REF] Ansari | A Survey of Current Exoskeletons and Their Control Architectures and Algorithms[END_REF]. The ReWalk is an active lower limbs orthosis attached to the patient's legs, that allows paraplegics to stand, sit, and walk without assistance. A wireless remote control worn on the wrist allows the patient to select the activity he/she wants to perform. The ReWalk hip and knee joints are powered and guided along a predefined path [START_REF] Yan | Review of Assistive Strategies in Powered Lower-Limb Orthoses and Exoskeletons[END_REF]. The stability of the user during walking is ensured by crutches. The device, which is controlled by DC motors at the joints, is customized and sized for each patient [START_REF] Esquenazi | The rewalk powered exoskeleton to restore ambulatory function to individuals with thoraciclevel motor-complete spinal cord injury[END_REF]. The Hybrid Assistive Limb (HAL) is another well-known commercialized exoskeleton. It is a four-degree-of-freedom bipedal locomotion system designed to help elderly and paraplegics with daily tasks such as sit-to-stand and stand-to-sit. Its movements are based on the use of EMG (Electromyography) sensors to detect motion intention. Moreover, the hybrid lower limb determines the wearer's center of pressure using floor response force sensors to ensure his/her stability. Each joint of the HAL system is driven by a PD controller [START_REF] Suzuki | Intention-based walking support for paraplegia patients with robot suit hal[END_REF]. There are also treadmill-based exoskeletons designed for rehabilitation and assistance of people with reduced mobility, such as the Lokomat. This latter is a powered gait orthosis that automates and assists walking motions, allowing more efficient treadmill training and less physically demanding rehabilitation. This therapeutic tool, which is controlled by DC actuators at the knee and hip, is used to treat patients with cerebrovascular accident, spinal cord injury, traumatic brain injury, Parkinson disease or multiple sclerosis. Lokomat is only used in clinical training sessions that are tailored to each patient's particular needs in terms of desired models and duration [START_REF] Jezernik | Sliding mode closed-loop control of FES: controlling the shank movement[END_REF]- [START_REF] Figueiredo | Assistance and Rehabilitation of Gait Disorders using Active Lower Limb Orthoses[END_REF]. The control of the EICOSI (Exoskeleton The control of the EICOSI (Exoskeleton Intelligently COmmunicating and Sensitive to Intention) system using Linear Quadratic Regulation, is the subject of our research study. This prototype was developed at the Lissi Laboratory of University Paris-Est Créteil in France. It is an active lower limb orthosis with one degree of freedom that acts on the knee joint level, designed for assistive and rehabilitation purposes. Several controls, including PID, sliding mode control, and adaptive control, have been applied to EICOSI. In this paper, we propose to control it using an exact input output liearization feedback and an LQR control. The LQR appears to be a good choice in our case since it has been proven that good tracking of the therapist's trajectory accelerates patient recovery and that the robustness and stability of the controller are required when dealing with robots in direct interaction with humans. Since PID controllers are commonly used in academia and industry, the results of simulations using the proposed approach were compared to those obtained using a PID controller. The shank foot was freely moving around the knee joint with no interaction with the ground. The following is a breakdown of the paper's structure: the EICOSI active lower limb orthosis is described in Section II. The suggested LQR control method is described in Section III. Section IV covers the simulation results of flexion/extension movements performed on a healthy subject in a sitting position with no contact with the ground, as well as a comparison study with a PID controller to illustrate the efficiency of our approach. Some conclusions and future scopes are drawn in section V.

II. DESCRIPTION OF THE EICOSI ACTIVE ORTHOSIS

In this study, the EICOSI active orthosis was used to provide power assistance at the user's knee joint level for flexion/extension exercises. The orthosis illustrated in Fig. 1, is driven using a high-power brushless DC motor (Maxon, Switzerland). A small transmission system with a gear motor, a ball screw, a transmission belt, and a cable drive which are designed to ensure an efficient and portable system with a relatively high output torque (up to approximately 18 N.m). The brushless DC motor is equipped with an incremental encoder that measures the motor rotation angle and serves to compute the knee joint angle. A numerical derivative of the measured joint angle yields the angular velocity [START_REF] Mohammed | Nested saturation based control of an actuated knee joint orthosis[END_REF]- [START_REF] Mohammed | Adaptive Control of a Human-Driven Knee Joint Orthosis[END_REF].

This 1 DoF active orthosis allows the user to conduct flexion and extension motions with a relative angle ranging from 0 • to 135 • , 0 • for full knee joint-orthosis extension, 135 • for maximum knee joint-orthosis flexion, and 90 • for resting position. The studied system represents a person in a sitting position with no interaction with the ground, wearing the actuated orthosis consisting of two segments attached separately to the thigh and shank of the wearer using appropriate braces. The shank-foot of the wearer is considered as a single rigid segment and can freely move around the knee joint. The entire system (human shank and embodied actuated orthosis) is illustrated by the following second-order equation:

J θ(t) = τ g cos(θ(t)) -f s sign( θ(t)) -f v θ(t) + u(t) + τ h (t)
(1) where θ, θ, and θ are respectively the angular position, velocity and acceleration, f s and f v are the solid and viscous friction coefficients, and J is the inertia of the whole system (the active orthosis and its wearer lower limb). τ g , τ h and u are respectively the gravitational torque, the human's torque, and the input applied to the system [START_REF] Alouane | Mohammed Cooperative Control for Knee Joint Flexion-Extension Movement Restoration[END_REF]- [START_REF] Roula | On Control Design for a Lower Limb Orthosis: A Comparative Study in Different Operating Conditions[END_REF].

III. LQR CONTROLLER DESIGN FOR THE EICOSI ORTHOSIS

The linear quadratic regulator is an optimal control theorybased feedback controller:

u(t) = -K(x(t) -x ref (t)) (2)
It computes the feedback gain matrix K using the state space representation of the controlled system and the weighting matrices Q and R supplied by the user [START_REF] Martins | Linear Quadratic Regulator for Trajectory Tracking of a Quadrotor[END_REF]- [START_REF] Anurag | Design of LQR-PID controller for linearized magnetic levitation system[END_REF]:

ẋ(t) = Ax(t) + Bu(t) y(t) = Cx(t) (3)
where A is the state matrix, B is the input matrix, and C is the output matrix. The cost function that collects the mathematically defined target performances is minimized, whether on an infinite or finite horizon (depending on the user's preference). The used infinite horizon cost function is illustrated by the following equation (4):

V (t) = ∞ 0 [[x(t)-x ref (t)] T Q[x(t)-x ref (t)]+u T (t)Ru(t)]dt (4)
The gain matrix K of the feedback control law that minimizes V (t) is defined by equation ( 5):

K = R -1 B T P ( 5
)
where P is calculated by solving the continuous time algebraic Riccati equation ( 6):

A T P + P A -P BR -1 B T P + Q = 0 (6)
As it can be seen from the equations ( 5) and ( 6), the weighting matrices Q and R intervene in the definition of the gain matrix K. How to choose these matrices (Q and R) is one of the most important concerns in LQR design.

In general, they are chosen as diagonal matrices in order to simplify the tuning. We can assign greater weight values to states that should be kept tiny using such matrices. Similarly, we can penalize an entry against states and other entries by adjusting its corresponding weight [START_REF] Thwin | Application of LQR control for two-wheel self-balancing robot[END_REF]- [START_REF] Malte | Analytical study of PID and LQR: Application to robotic manipulator[END_REF].

Q =    q 1 0 . . . 0 q n    , R =    r 1 0 . . . 0 r m    (7)
Since the LQR is a model-based controller, the shankorthosis parameters were obtained using a nonlinear least square optimization method. The regression equations of Winter were used to compute the shank mass and the gravity center, taking into consideration the subject's weight and height and the remaining parameters were identified by a passive pendulum test. The acquired parameters illustrated in the TABLE I, are used in the simulation scenarios presented in section IV. We used the exact input-output linearization method in order to be able to apply the LQR with our nonlinear system as illustrated in Fig. 2. Through a change of variables and appropriate nonlinear feedback, the nonlinear system can be transformed into a linear equivalent one as shown in the state space representation below:

ẋ(t) = Ax(t) + Bv(t) y(t) = Cx(t) (8)
with A = 0 1 0 0 , B= 0 1 and C= 1 0

As it can be seen from ( 8), a linear input-output map was established between the new input v(t) and the output y(t) by considering the following change of variables: IV. SIMULATION RESULTS : A COMPARISON BETWEEN PID AND LQR CONTROLLERS In this section, the simulation results that were performed to evaluate the LQR controller's performances while comparing them to those of a PID controller, are described and analyzed. This latter is tuned through trial and error method until achieving the best settling time, precise tracking, and stability. The simulation results are performed in the case of a sitting position with no interaction with the ground. Four separate scenarios are used to validate the proposed LQR control approach. The considered reference trajectory in the first case is configured to a chirp signal in passive mode while it is equal to a square wave used for flexion and extension movements, in the second scenario. Unlike the two previous scenarios, the torque τ h is taken into consideration in the two last ones. During scenario 3, the human torque is considered as white noise while a human walking cycle trajectory is used as reference. During the last scenario, τ h is used to represent the assistive (the orthosis user is delivering an assistance effort) and resistive modes (the subject is delivering an effort in the opposite direction of the input control u(t)), while a sinusoidal input signal with constant frequency and amplitude is used as a reference trajectory.

u(t) = Jv(t) -τ g cos(θ)(t) + f s sign( θ(t)) + f v θ(t) -τ h (t) (9)

Scenario 1: Passive Mode -Chirp reference trajectory

During this scenario, τ h is considered null. The reference trajectory is taken equal to a chirp signal. The goal of this scenario is to see if the controllers are able to track the reference trajectory as it accelerates. The PID controller's curves are depicted in blue, the LQR controller's curves are depicted in black, while the reference curves are depicted in red. The absolute values of the tracking errors for angular positions and velocities, illustrated by curves 2 and 4 in Fig. 3, indicate that the PID's tracking errors are significantly greater than the LQR ones while the delivered torques are approximately the same. The RMSEs (Root Mean Square Errors), whose formula is given by equation [START_REF] Esquenazi | The rewalk powered exoskeleton to restore ambulatory function to individuals with thoraciclevel motor-complete spinal cord injury[END_REF], are calculated and compared in both situations to corroborate these results.

RM SE = n i=1 (z ref (i) -z(i)) 2 n ( 10
)
with n: the number of the considered samples and z either representing the angular position or the angular velocity.

In the case of the LQR, the angular position RMSE is 1.37 10 -6 rad, while in the case of the PID, it is 5.21 10 -5 rad. These results confirm the LQR's superiority over the PID controller. In terms of angular velocity RMSE computation, we found that the value of the LQR RMSE was equal to 7.7 10 -6 (rad/s), whereas the value for the PID was equal to 1.06 10 -4 (rad/s). These calculations back up the observations made based on curves shown in Fig. 3. To illustrate the flexion/extension exercise performed for the rehabilitation of patients with reduced mobility, we use a reference trajectory equivalent to a square signal. During this scenario, the torque τ h is also considered null. The curves depicted in Fig. 4 show that the LQR performs better than the PID in this scenario as well. Indeed, the LQR allowed for better trajectory tracking without resorting to overshoot while delivering lower torque values compared to the PID controller. It can be observed that the PID has reached the motor's maximum values, resulting in saturation, according to the zoom performed around time 20s. Repeatedly engaging in such activity can damage the active orthosis motor. The reference trajectory used in this scenario is the human walking cycle. The orthosis wearer is considered to be passive, however to simulate real experimental conditions, we set τ h equal to a white noise. The two controllers were able to reject the noise caused by the human torque, as demonstrated by the curves in Fig. 5. Although the torques generated by the LQR and the PID are nearly identical, the LQR's trajectory tracking in terms of angular position and angular velocity is far superior to the PID tracking. Curves 2 and 4 in Fig. 5 clearly illustrate this observation.

For the two controllers, the computed RMSEs yielded to the following values 4.15 10 -4 (rad) and 2.5 10 -3 (rad/s) for PID and 1.54 10 -5 (rad) and 7.87 10 -5 (rad/s) for LQR. This scenario reveals once again the superiority of LQR over PID. The reference trajectory in this scenario (shown in Fig. 6) is sinusoidal with constant frequency and amplitude. The τ h torque is also taken into consideration. We consider the passive mode for the first 30 seconds, where τ h is zero then we set it to an assistive effort between 30s and 60s. In this scenario, we want to see if the controllers can compensate for human torque and assist the person only when needed. During the rehabilitation phase, this behavior is important, and should be considered while designing a wearable robot. In fact, involving the patient in rehabilitation exercises can help the user heal faster.

To illustrate a resistive force, τ h is taken equal to a force exerted in the opposite direction of the torque delivered by the orthosis during the last 30 seconds. Torques generated in both cases are significantly reduced in assistive mode, allowing reference trajectory tracking when the human is expected to provide an assistance force, whereas torques generated in resistive mode are larger, allowing the subject to compensate for resistance force. The LQR controller is able to compensate for the required torque during the final phase, resulting in better performances than the PID. We gathered the computed RMSEs during the four scenarios in one table, to make comparisons between the two controllers easier. We can clearly see that the LQR controller surpasses the PID controller in terms of position tracking error and angular velocity smoothness almost during all scenarios as shown in TABLE II. This smoothness makes the active orthosis more comfortable to wear and offers more protection to the human leg.

V. CONCLUSION AND FUTURE WORK

In this paper, a linear quadratic regulator was used to control the EICOSI exoskeleton, a one degree-of-freedom active orthosis acting on the knee joint, used for assistive and rehabilitation purposes. Since the studied system is stated as a nonlinear second order model, an exact input-output linearization feedback was performed, allowing the application of the LQR controller on the knee active orthosis. Following the results of four separate experimental scenarios, a comparison between simulation results achieved while using the PID controller and the LQR controller was developed and detailed. Hence, the results confirm that the LQR outperforms the PID controller in terms of position tracking error, energy consumption, and smoothness of the resulting angular velocity. Executing these experiments with distinct scenarios on subjects wearing the active orthosis in a sitting position will be the next stage for this research.

Fig. 1 .

 1 Fig. 1. View of EICOSI active lower limb orthosis.

Fig. 2 .

 2 Fig. 2. Block diagram of the proposed control scheme applied to EICOSI orthosis.

5 |Fig. 3 .

 53 Fig. 3. Scenario 1: PID/LQR controllers : null τ h torque with a chirp reference trajectory.

Scenario 3 :|θ 5 |Fig. 4 .

 354 Fig. 4. Scenario 2: PID/LQR controllers : no effort exerted by the patient -square reference trajectory.

Scenario 4 :

 4 Passive Mode / Assistive Mode / Resistive Mode -A sinusoidal reference trajectory with a constant frequency.

|θ 2 |Fig. 5 .

 25 Fig. 5. Scenario 3: PID/LQR controllers : no effort exerted by the patient -human walking cycle reference trajectory.

1 |Fig. 6 .

 16 Fig.[START_REF] Pratt | The RoboKnee: An Exoskeleton for Enhancing Strength and Endurance During Walking[END_REF]. Scenario 4: PID/LQR controllers : no effort during the first 30s, exerting an assistive effort between 30s and 60s and exerting a resistive effort during the last 30s.

TABLE I SUMMARY

 I OF THE DYNAMIC PARAMETERS OF THE SYSTEM.

	Parameter	Symbol	Value
	Solid friction coefficient	fs	0.4069 N.m
	Viscous friction coefficient	fv	2.3708 N.m.s.rad -1
	Inertia	J	0.256 Kg.m 2
	Gravity torque	τg	3.4689 N.m

TABLE II SUMMARY

 II OF THE OBTAINED RMSE'S.

		P ID RM SE θ	LQR RM SE θ	P ID RM SE θ	LQR RM SE θ
	Scenario	(rad)	(rad)	(rad/s)	(rad/s)
	1	5.21 10 -5	9.29 10 -6	1.06 10 -4	1.76 10 -5
	2	8.24 10 -4	8.84 10 -4	3.6 10 -3	2.8 10 -3
	3	4.15 10 -4	1.54 10 -5	2.5 10 -3	7.87 10 -5
	4	2.89 10 -4	1.56 10 -5	3.52 10 -4	2.87 10 -5