Susana Faria
email: sfaria@math.uminho.pt

Gilda Soromenho

Comparison of EM and SEM Algorithms in Poisson Regression Models: a simulation study

Keywords: Maximum likelihood estimation, EM algorithm, Stochastic EM algorithm, Mixture Poisson regression models, Simulation study Maximum likelihood estimation, EM algorithm, Stochastic EM algorithm, Mixture Poisson regression models, Simulation study

In this work, we propose to compare two algorithms to compute maximum likelihood estimates for the parameters of a mixture Poisson regression model: the EM algorithm and the Stochastic EM algorithm. The comparison of the two procedures was done through a simulation study of the performance of these approaches on simulated data sets and real data sets. Simulation results show that the choice of the approach depends essentially on the overlap of the regression lines. In the real data case, we show that the Stochastic EM algorithm resulted in model estimates that best fit the regression model.

Introduction

Finite mixture models are a well-known method for modelling data that arise from a heterogeneous population (see e.g. [START_REF] Mclachlan | Finite Mixture Models[END_REF][START_REF] Fruhwirth-Schnatter | Finite Mixture and Markov Switching Models[END_REF] for a review). The 2 study of these models is a well-established and active area of statistical research and mixtures of regressions have also been studied fairly extensively. In particular, Poisson mixture regression models are commonly used to analyze heterogeneous count data. [START_REF] Wedel | A latent class poisson regression model for heterogeneous count data[END_REF] proposed a class Poisson regression model and an EM algorithm for estimation was described. [START_REF] Wang | Mixed poisson regression models with covariate dependent rates[END_REF] studied mixed Poisson regression models and maximum likelihood estimates of the parameters were obtained by combining EM and quasi-Newton algorithms.

In this work, we study the procedure for fitting Poisson mixture regression models by means of maximum likelihood (ML). We apply two maximization algorithms to obtain the maximum likelihood estimates: the Expectation Maximization (EM) algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] and the Stochastic Expectation Maximization (SEM) algorithm [START_REF] Celeux | The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem[END_REF].

The comparison of EM and SEM approaches in a mixture of distributions is well known. [START_REF] Celeux | Stochastic versions of the EM algorithm: an experimental study in the mixture case[END_REF] have investigated the practical behaviour of these algorithms through intensive Monte Carlo numerical simulations and a real data study. [START_REF] Dias | An empirical comparison of EM, SEM and MCMC performance for problematic Gaussian mixture likelihoods[END_REF] have compared EM and SEM algorithms to estimate the parameters of Gaussian mixture model. [START_REF] Faria | Fitting mixtures of linear regressions[END_REF] have performed a simulation study to compare the performance of these two approaches on Gaussian mixtures of linear regressions. This paper is organized as follows: Section 2 describes the model. Parameter estimation based on the EM algorithm and the Stochastic EM algorithm is discussed in Section 3. Section 4 provides a simulation study investigating the performance of these algorithms for fitting two and three component mixtures of Poisson regression models. We also study the performance of algorithms in real data sets in section 5. In Section 6 the conclusions of our study are drawn.

F

( ) ∑ = = J j i i j j i i x y f x y h 1 | ) , | ( π θ (1)
where ( ) ( )

J j n i y x y f i y ij ij i i j i ,..., 1 , ,..., 1 , ! ) exp( | = = - = λ λ (2) and ) exp( i T j ij x β λ = , with T jp j j j ) ,..., , ( 1 0 β β β β =
denoting the (p+1)-dimensional vector of regression coefficients for jth component and To be able to reliably estimate the parameters of mixture models we require identifiability. That is, two sets of parameters do not yield the same mixture distribution. Finite mixtures of Poisson distributions are identifiable (see Teicher, 1960 for details). [START_REF] Fruhwirth-Schnatter | Finite Mixture and Markov Switching Models[END_REF] shows that if the covariate matrix is of full rank and the mixing proportions, j π , are all different, then the Poisson mixture regression model is identifiable. Among the various estimation methods considered in the literature for finite mixture models, the maximum likelihood (ML) has dominated the field.

For a given number of J components, the task is to estimate the vector of parameters ( )

J J β β π π θ ,..., , ,..., 1 1 = that maximizes the log-likelihood ( ) ( ) ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ = ∑ = ∑ = J j i i j j n i n n x y f y y x x L 1 1 1 1 | log ,..., , ,..., | π θ (3)
The standard tool for finding maximum likelihood solution is the Expectation Maximization (EM) algorithm. However, it suffers from slow convergence and may converge to local maxima or saddle points. The Stochastic Expectation Maximization (SEM) algorithm is a viable alternative to find the ML estimates of the parameters of a mixture model. The SEM algorithm by using random drawing at each iteration, prevents from being trapped in local optima. It has some advantages over the EM algorithm: it does not get stuck; it often provides more information about the data (see [START_REF] Diebolt | Stochastic EM: method and application[END_REF], for instance when parameters cannot be estimated; and in certain conditions behaves better than EM algorithm (see [START_REF] Celeux | Stochastic versions of the EM algorithm: an experimental study in the mixture case[END_REF].

The EM algorithm

The EM algorithm is a broadly applicable approach to the iterative computation of maximumlikelihood estimates when the observations can be viewed as incomplete data. The idea here is to

think of the data as consisting of triples n i z i x y i i , , 1 ), , , ( K = where J j n i z z z T ij i i K K K , 1 , , , 1 , ) , , ( 1 = = =
is the unobserved indicator that specifies the mixture compo- The log likelihood for the complete data is

( ) ( ) ( ) ( ) ∑ = ∑ = ∑ = ∑ = + = n i n i J j i i j ij J j j ij n n x y f z z y y x x L 1 1 1 1 1 | log log ,..., , ,..., | π θ (4)
The EM algorithm is easy to program and proceeds iteratively in two steps, E (for expectation) and M (for maximization). At the E-step, it replaces the missing data by its expectation conditional on the observed data. At the M-step, it finds the parameter estimates which maximize the expected log likelihood for the complete data, conditional on the expected values of the missing data. This procedure can be stated as follows. (  )

E-step: Given the current parameter estimates

2 1 1 | Q Q Q r r + = + θ θ (6)
under the restriction for the component weights and where

( ) ( ) ∑ ∑ = = + = n i J j r j ij w Q 1 1 1 1 log π (7)
∑ = ∑ = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ = + n i J j j i i j ij r x y f w Q 1 2 1 , | log β (8)
The maximization of 1 Q under the restriction for the component weights,

∑ = = J j j 1 1 π , is equivalent to maximizing the function ( ) ( ) ( ) ∑∑ ∑ = = = + + ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ - - = n i J j J j r j r j ij w Q 1 1 1 1 1 * 1 1 log π μ π
where μ is the Lagrangian multiplier. Setting the derivative of function * 1 Q with respect to ( ) and2 Q is maximized separately for each J j , , 1 K = using weighted ML estimation of generalized linear models (GLM).

1 + r j π equal to zero yields ( ) ( ) J j n w n i r ij r j , , 1 , ˆ1 1 K = = ∑ = + π (9)

The Stochastic EM algorithm

We also apply a procedure for fitting Poisson mixture regression models using a stochastic version of the EM algorithm, the so-called SEM algorithm. The SEM algorithm is an improvement of the EM algorithm that incorporates a stochastic step (S-step) between the E-and M-steps of EM.

Starting from an initial parameter 0 θ , an iteration of SEM algorithm consists of three steps.

J j n i w r ij , , 1 , , , 1 , K K = =
, are computed for the current value of θ as done in the stan- dard EM.

S-step:

A partition ( ) ( ) ( ) ) , , ( 1 1 1 1 + + + = r J r r P P P L of ) , ( , ), 1 , ( 1 n x y x y n K
is designed by assigning each observation at random to one of the mixture components according to the multinomial distribution with parameter ( )

J j n i w r ij , , 1 , , , 1 , K K = =
, given by ( 5). If one of the P(r+1) is empty or has only one observation, it must be considered that the mixture has J-1 components instead of J and the estimation process begins with J -1 components. Yet, in this case, it provides a bias towards uniform j π parameters.

M-step:

The ML estimate of θ is updated using the sub-samples ( )

1 + r J P
. It follows that on the M-step of the (r+1)th iteration, the parameter estimates ( )

1 + r j π are given by ( ) J j n n j r j , , 1 , ˆ1 K = = + π (10)
where j n is the total number of observations arising from component j and the maximization of

( ) ∑ = ∑ = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ = + J j z i j i i j ij r x y f Q 1 } 1 | { * 2 1 , | log β (11) where { } 1 | = ij z i
is the set of observations arising from the jth mixture component, gives ( ) To investigate the statistical behaviour of the proposed methods in fitting Poisson mixture regression models, a simulation study was performed. The simulation is designed to evaluate the model performance considering the effects of sample sizes and the initialization of the algorithms as well as the configuration of the regression lines. The scope was limited to the study of two and three components. We used the freeware R to develop the simulation program.

1 + r j β .

Initial Conditions

Two different approaches for choosing initial values are compared in the study. In the first strategy, we use the true parameter values of the model by generating the observations as initial values in order to determine the performance of the algorithm in the best case. In the other strategy we ran the algorithm 20 times from random initial position and we selected the solution out of 20 runs which provided the best value of the optimized criterion [START_REF] Celeux | Comparison of the mixture and the classification maximum likelihood in cluster analysis[END_REF].

Stopping Rules

For the EM algorithm, iterations were stopped when the relative change in log-likelihood between two successive iterations were less than 20 10 -. However, since SEM does not converge pointwise and it generates a Markov chain whose stationary distribution is more or less concentrated around the ML parameter estimate, we used as stopping rule for the SEM algorithm the total number of iterations required for convergence by the EM algorithm.

Number of Samples

For each type of simulated data set, we generated 500 samples of size n. distribution and then we have

( ) i j j ij x 1 0 exp β β λ + =
. Finally, we simulate the value ( )

ij i P y λ ~.

Measure of Algorithm Performance

In order to examine the performance of two algorithms, we report the Euclidean distance between estimated parameters and true parameter values.

Quality of the fit

In order to compare the quality of the fit of two algorithms, we report the root mean squared error of prediction (MRSEP): cross validation, which is given by:

( ) ( ) ∑ = ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ - = n i i i i m V y n RMSEP 1 2 ˆ1 μ μ where ∑ = = J j ij j i 1 λ π μ and ( ) 2 1 1 2 1 ˆ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ - + = ∑ ∑ ∑ = = = J j ij j J j ij j J j ij j i V λ π λ π λ π μ
For the K-fold cross validation, we have chosen K = 5 and K = 10 ( Hastie el al., 2001, p.214). [START_REF] Yang | Mixture poisson regression models for heterogeneous count data based on latent and fuzzy class analysis[END_REF][START_REF] Yang | Mixture poisson regression models for heterogeneous count data based on latent and fuzzy class analysis[END_REF][START_REF] Leisch | FlexMix: A general framework for finite mixture models and latent class regression in R[END_REF]. For instance, we present in Figure 1 typical scatter plots for samples with size 500. Note that the cases considered correspond to varying degrees of overlapping. Case A3 has the highest overlapping and data from A1 show the lowest overlapping.

Figure 2 shows boxplots of the Euclidean distance between estimated and true parameters over the 500 replications using the EM and SEM algorithm for fitting two component mixtures of Poisson regression models.

Figure 2 shows that the three algorithms have practically the same behaviour. However, when the overlap is high (A3) EM outperforms SEM by producing estimates of the parameters that have smaller estimation error. As expected, estimation error decreases when the sample size increases.

The resulting values of MRSEP based on 10-fold cross validation, for each of the configurations of the true regression lines are plotted in Figure 3 and Figure 4. Similar results were obtained calculating MRSEP based on 5-fold cross validation. Figure 3 and Figure 4 show that, in generality, the SEM algorithm performs better than the EM algorithm.

Simulation results: three component mixture of Poisson regressions

For three-component models, samples of three different sizes ( ) Figure 5 shows boxplots of the Euclidean distance between estimated and true parameters over the 500 replications using the EM and SEM algorithm for fitting three component mixtures of Poisson regression models. Figure 5 shows that EM outperforms SEM by producing estimates of the parameters that have lower estimation error, especially when the overlap is higher (B4). Also, as expected, estimation error tends to decrease as the sample size increases.

The resulting values of MRSEP based on 10-fold cross validation, for each of the configurations of the true regression lines are shown in Table 3 andTable 4. Similar results were obtained calculating MRSEP based on 5-fold cross validation. Table 3 andTable 4 show that, in generality, the SEM algorithm performs better than the EM algorithm.

Real Data Sets

We now compare the performance of the EM algorithm and the SEM algorithm for fitting Poisson mixture regression models in two real data sets.

Fabric faults

The "Fabric Faults'' data set consists of 32 observations of number of faults in rolls of fabric of different length. The dataset is analysed using a finite mixture of Poisson regression models in [START_REF] Aitkin | A general maximum likelihood analysis of overdispersion in generalized linear models[END_REF]. The response variable is the number of faults and the covariate is the length of role in meters. The data set can be loaded into R with the command data ("fabricfault", pack-age=''flexmix'').

We fitted a component Poisson mixture regression model using EM algorithm and SEM algorithm where the logarithm of lengths is used as independent variable. The algorithms were initiated by random numbers (second strategy) and the stopping criterion was the same used in the simulation study. For each algorithm, the optimal number of components was selected using the proposed procedure:

Step 1: Set j=2 and calculate the value of the MRSEP based on k-fold cross validation for a two component model. Let this value be denoted by MIN.

Step 2: Set j=j+1 and calculate the value of the MRSEP based on k-fold cross validation for a j component model.

Step 3: If the new value of the MRSEP is lower than MIN then set MIN equal to the new value of the MRSEP and go to Step 2, else deduce that the optimal number of components is j-1 and stop.

Table 5 presents the MRSEP based on 10-fold cross validation computed for each algorithm and the results show that the mixture with 2 components is selected. We can also observe that the SEM algorithm performs always better in fitting Poisson mixture regression model to the "Fabric Faults'' data.

Patent

The patent data given in [START_REF] Wang | Mixed poisson regression models with covariate dependent rates[END_REF] random numbers (second strategy), the stopping criterion was the same used in the simulation study and the optimal number of components was selected using the proposed procedure described in section 5.1.

Table 6 presents the MRSEP based on 10-fold cross validation computed for each algorithm and the results show that the mixture with 3 components is selected. We can also observe that the SEM algorithm performs always better in fitting Poisson mixture regression model to the patent data.

Conclusion

In this paper, we compare the performance of two algorithms to compute maximum likelihood estimates of a mixture Poisson regression models, the EM algorithm and the Stochastic EM algorithm (SEM).

The results of simulation show that the choice of approach depends essentially on the overlap of the regression lines. For some severely overlapping mixtures, the EM algorithm outperforms the SEM algorithm by producing estimates of the parameters that have smaller estimation error. However, simulation results indicated that the Stochastic EM Algorithm provides in general best estimates for those parameters in the sense of the best fit for the regression model.

In the real data case, we also show that the SEM algorithm resulted in model estimates that best fit the regression model. As we expected, the SEM algorithm and the EM algorithm can converge to the different estimates. EM convergence is very dependent upon the type of starting values and the stopping rule used, so the EM algorithm may converge to local maxima or saddle points. Page 4. Line 3, sentence beginning "Given a set of independent…" I don't believe that this statement is true at all. The exact conditions for existence of maximum likelihood estimates is a very difficult statement to make. Perhaps you can say that MLE's can sometimes be estimated from this likelihood function provided such estimates exist.

We rewrite the text in Section 3. The changes we make in the manuscript are in coloured text.

3. Page 4, Equation (4): Conditional on the lambda parameters, only the pi's are estimated. The real condition is y given x, but not lambda here.

We rewrite the equation ( 4) and also equation ( 1), ( 2) and (3).

4. Page 5, equation ( 8): This is confusing. I thought that the parameters we were estimating were the beta's, not the lambda's.

We rewrite the equation ( 8) and also equation ( 5) and (11).

5. Page 7, top paragraph: Please make a comment that this provides a bias towards uniform pi parameters.

We make a comment. The changes we make in the manuscript are in coloured text.

6. Page 9: Refer to page number in books such as [START_REF] Hastie | Elements of Statistical Learning[END_REF].

We refer the page number. We eliminate the end of Section 5.1 and 5.2 (and Figure 6 and Figure 7). The results in Table 5 and Table 6 show that the SEM algorithm perform always better in fitting Poisson mixture regression model to the data.

8. References: These are in different styles. Some are all capitals in titles, others are not. Please refer to the style requirements for this journal.

We rewrite some references. The changes we make in the manuscript are in coloured text.

  as the unconditional probabilities that an individual belongs to component j of the mixture.

  i.e., ij z equals 1 if observation i comes from component j and 0 otherwise.

θ

  in the rth iteration, replace the missing data ij z by the estimated probabilities that the i observation belongs to the jth component of the mixture, : Given the estimates for the probabilities( )

  : The estimated probabilities that the i observation belongs to the jth component of the mixture,( )

  generated by the following scheme. First, a uniform [0,1] random number i c was generated and its value is used to select a particular component j from mixture of regressions model. Next, ix was randomly generated from a uniform [ ]

  mean squared error of prediction of the mth replication based on K-fold

  results: two component mixture of Poisson regressions For two-component models, samples of four different sizes ( ) set of true parameter values ( ) β π , shown on Table 1

  consist of 70 observations on patent applications, research-and-development (R&D) spending and sales in millions of dollar from pharmaceutical and biomedical companies in 1976 taken from the National Bureau of Economic Research R&D Master file. To model this data, Wang et. al (1996) used several covariates including logarithm of R&D spending and/or squared logarithm of R&D spending for different models. The data set can be loaded into R with the command data ("patent", package=''flexmix'').We fitted a Poisson mixture regression model using EM algorithm and SEM algorithm where the logarithm of R&D spending is used as independent variable. The algorithms were initiated by

Figure 1 .Figure 2 .Figure 3 .Figure 4 .Figure 5 .

 12345 Figure 1. Scatter plot of samples from 2 component models with n = 500 (A1, A2 and A3)

7.

  Page 12: End Section 5.1 by explaining which method is better. What do you conclude from this example? The two methods converge to different estimates

ca Communications in Statistics -Simulation and Computation

	Page 12 of 25
	F o r
	P
	e
	e r
	R
	e
	v i e
	w
	O n l
	y
	12

URL: http://mc.manuscriptcentral.com/lssp E-mail: comstat@univmail.cis.mcmaster.

Table 1 .

 1 True parameter values for the essays with a 2 component mixtures of Poisson regression

	The SEM

Table 2 .

 2 True parameter values for the essays with a 3 component mixtures of Poisson regression

			Cases	10 β	11 β	β	20	β	21	β	30	β	31	π	1	π	2
					10 β
			B1	3	-0.5	4	0.5	3	0.5 0.4 0.4
			B2	6	-0.5	4	-0.5	2	0.5 0.3 0.3
			B3	4	-0.5	4	0.5	2	0.8 0.3 0.2
			B4	2.8 -2.9 2.6	0.4	3.6	0.2 0.3 0.2
	Table 3. MRSEP by 10-fold cross-validation for 3 component models when the algorithms were
	initiated by random numbers
				B 1												B 2
			n= 100	n= 500		n= 1000			n= 100			n= 500	n= 1000
	π	1	2

Table 4 .

 4 MRSEP by 10-fold cross-validation for 3 component models when the algorithms were initiated by random numbers

	Table 5. MRSEP based on 10-fold cross validation for "fabric faults'' dataset

URL: http://mc.manuscriptcentral.com/lssp E-mail: comstat@univmail.cis.

mcmaster.ca Communications in Statistics -Simulation and Computation

ca Communications in Statistics -Simulation and Computation

  First paragraph of Section 3: This material needs to be very carefully rewritten. Most of the statements made here are just not true. It is not clear what is intended. Does the SEM algorithm always converge? Does it converge if MLE's and/or EM estimates can not be obtained?

	Page 24 of 25
	Communication in Statistics -Simulation and Computation
	LSSP 2010 -0452
	Comparison of EM and SEM Algorithms in Poisson Regression Models: a
	simulation study
	Susana Faria and Gilda Soromenho
	Comments to the issues raised by the referees:
	1. Page 3, 5 lines from below: Text should read "… identifiability.
	That is, two sets of parameters do not yield the same…"
	We rewrite the text in page 3. The changes we make in the manuscript
	are in coloured text. F r o 2. Page 4:
	P
	e
	e r
	R
	e
	v i e
	w
	O n l
	y

URL: http://mc.manuscriptcentral.com/lssp E-mail: comstat@univmail.cis.mcmaster.

URL: http://mc.manuscriptcentral.com/lssp E-mail: comstat@univmail.cis.mcmaster.caCommunications in Statistics -Simulation and Computation

Communications in Statistics -Simulation and Computation


Stéphane Régnier

Aude Bolopion

Jean-Ochin Abrahamians

An overview of multiple DoF magnetic actuated micro-robots Soukeyna Bouchebout •

Keywords: Soukeyna Bouchebout, Aude Bolopion, Jean-Ochin Abrahamians, micro-robot, magnetic actuation, control Soukeyna Bouchebout

HAL is

Introduction

The promising field of micro and nano technologies based on wirelessly powered and maneuvered submillimeter devices has recently been developed. As these micro-robots have a small size and are untethered, they can fit in fluidic systems and manipulate micro-objects, likewise they can experience very high velocities and carry out many actions in a short time. Several areas will benefit from these advanced technologies.

As for potential applications, advanced manufacturing relies on the micro-assembly of small mechanical components, using micro-agents manipulating micro-objects in order to build assembled systems. For example, Micro-Opto-Electro-Mechanical Systems (MOEMS) sense or manipulate optical signals on a very small size scale using integrated mechanical, optical, and electrical systems, therefore their fabrication requires a precise micro-assembly system. Another example is the fabrication, packaging and interconnection of different micro-components in the MEMS (Micro-Electro-Mechanical-Systems) where micro-assembly is vital. Modular robotics can also benefit from micro-robotics. Reducing the size of individual robotic modules increases the spatial resolution of reconfigurable robots, hence it adds to their versatility and range of tasks. Industrial biotechnology and medicine are also suitable fields for micro-robot use. Indeed, micro-robots operate at the same scale as organic cells, allowing efficient in vitro interaction in order to move and/or sort cells. Hence they can fit in lab-on-chips and perform adaptable tasks on corpuscles such as blood cells sized from 6 to 10 µm. Minimally Invasive Surgery (MIS) greatly reduces patient trauma and time recovery through the use of smaller tools, in the future micron-sized robots could like cameras [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF], X-rays, laser scanning, MRI [START_REF] Folio | MRI magnetic signature imaging, tracking and navigation for targeted micro/nano-capsule therapeutics[END_REF] [24] [START_REF] Martel | MRI-based Medical Nanorobotic Platform for the Control of Magnetic Nanoparticles and Flagellated Bacteria for Target Interventions in Human Capillaries[END_REF], and fluoroscopy. The choice of the imaging systems is important and depends on the environment of the micro-device, range of detection field, cost, tracking speed and flexibility. Perception feedback can be exploited by a user in a teleoperation mode while the control system enhances manipulating precision at a micro scale, as is ideally the case for MIS. Camera frames can also be used in a fully automated manner, which is required for autonomous industrial micro-assembly or operations on cells due to the high number of repetitive tasks.

This work is organized as follows. Section 2 details the major challenges that need to be addressed. Sections 3, 4, 5, 6 illustrate how four micro-robot systems, the Mag-µBot from Carnegie Mellon University, the OctoMag of the ETH Zurich, a paramagnetic particles control system from the University of Twente, and the MagPieR of the ISIR and FEMTO-st laboratories, approach micro-robotic challenges, with focus on the micro-robot platforms, motion mechanisms and force/position controls. This review is concluded in section 7 with a comparison between the discussed systems.

Challenges of magnetically actuated micro-robots

Usually, the studied magnetic micro-robots systems consist in electromagnets surrounding the micro-robot workspace, as depicted in fig. 1. The micro-robot evolves in a workspace we will call the arena, which can include a fluid (liquid or gas). The scene is observed using a camera fitted with a microscope objective. In this section the main issues in designing, actuating and controlling such systems are discussed. Fig. 1 An example of a magnetic actuated micro-robot

Challenges of magnetic micro-robot actuation

An object with a non-zero internal magnetization experiences magnetic forces and torques when exposed to a magnetic field B, and thus exhibits motion [START_REF] Nagy | Modeling Assembled-MEMS Microrobots for Wireless Magnetic Control[END_REF]. B [T] is characterized by its directions represented as field lines, and by its value called flux density. It is a function of the magnetic H [T] field called field strength. It is proportional to the magnetization field M [A m] of its source, which is the density vector field of local magnetic moments inside its volume V. The magnetic moment m [A m 2 ] is a vector quantity resulting from the motion of charges within a magnetized object, and describes its ability to magnetically interact with other magnetized objects or magnetic fields.

Several types of magnetized materials can be used [START_REF] Palagi | Design and development of a soft magnetically-propelled swimming microrobot[END_REF]. In ferromagnetic and ferrimagnetic materials, magnetic domains locally regroup large numbers of atomic magnetic moments, which describes an element's ability to magnetically interact with magnetized objects or magnetic fields, pointing towards the same directions. These magnetic domains result in an overall magnetic moment in permanent magnets, or compensate each other in other materials. An external magnetic field can realign these magnetic domains according to its directions which may result in forces and torques on the object : these materials are attracted towards stronger values of the magnetic field. The stronger the exciting field is, the more magnetic domains realign, until it reaches magnetic saturation as all the domains point towards the same direction. Thus, a ferromagnetic object which produces no magnetic field on its own can also be used to enhance an external magnetic field. Residual magnetization may remain when the exciting field is switched off, and may be permanent. Ferromagnetic and ferrimagnetic materials are classified as soft magnetic if the remaining magnetization is small and dissipates very fast, which is suitable for use in microrobotics, or hard magnetic when residual magnetization is significant, which can produce permanent magnets. The advantage of using ferromagnetic micro-robots is to generate significant forces that can be used in the assembly or handling of micro-components.

Other materials are susceptible to magnetic fields, like paramagnetic and diamagnetic materials. These materials do not possess magnetic domains and are not subject to magnetic remanence, but magnetic moments linked to their unpaired electrons for paramagnetic materials, or paired electron orbital motions for diamagnetic materials, can likewise realign in the presence of a magnetic field. Paramagnetic materials realign their magnetic moments in the direction of B, while diamagnetic materials realign their magnetic moments in the opposite direction of B. Hence, a diamagnetic micro-device can benefit from a point of equilibrium; which is suitable for the control. Diamagnetism can be observed in all materials, but is very weak except in superconductors : when paramagnetic properties exist they prevail over diamagnetic repulsion. Paramagnetic materials themselves present a much weaker attraction towards stronger values of the magnetic field than ferromagnetic materials, and therefore require a stronger magnetic source to achieve similar results.

In practice, to express the forces exerted by a magnetic field on a ferromagnetic object, constant experimental values can be used as a uniform magnetization M for unsaturated materials. While, on a paramagnetic body, the local magnetic moment is proportional to the applied magnetic field, where α is the proportionality constant measured by a magnetometer. The magnetic force F m relies on the non-uniformity of the magnetic field B and is proportional to the magnetic field gradient. The magnetic torque T m is proportional to the magnetic field and acts to bring the internal magnetization of an object into alignment with the field. The magnetic forces and torques are expressed in Tab. 1for both ferromagnetic and paramagnetic material.

Material m F m T m Ferromagentic m = MV F m = V(M • ∇)B (1) T m = VM × B (2) Paramagnetic m = αVB F m = αV • ∇(B 2 ) (3) T m = 0
Table 1 Summary of the magnetic forces and torques on ferromagnetic and paramagnetic materials

Regarding the effect of B on magnetic materials, the magnetic field can be employed to actuate micro-robots made of a magnetic material. B fields can be used to actuate micro-robots in several ways. Stick-slip motion induced by magnetic torques can produce translation [START_REF] Floyd | Dynamic modeling of stick slip motion in an untethered magnetic micro-robot[END_REF]. In addition to the stick-slip method, magnetic forces can produce a rolling motion on spherical magnetic micro-bodies [START_REF] Jiang | Development of rolling magnetic microrobots[END_REF]. Using magnetic forces as a direct propulsion method requires a stronger magnetic field to move the micro-device [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF], but generates higher magnetic forces when interacting with micro-objects. Besides, magnetic forces can be used to create vibrations in a micro-robot [START_REF] Vollmers | Wireless resonant magnetic microactuator for untethered mobile microrobots[END_REF] [31] [START_REF] Hsi-Wen | Polymer-based Wireless Resonant Magnetic microrobots[END_REF], leading to motion at its resonance mode. In the case of the magnetically actuated systems purposed for in vivo or in vitro biomedical applications, the biocompatibility of magnetic materials is important for such magnetic systems. Coating the magnetic material with polymer layers [START_REF] Raju | Synthesis and characterisation of polymer-coated metallic magnetic materials[END_REF] can be used, to prevent the magnetic materials from corrosion and leaching. Moreover, some biocompatible magnetic materials exist [START_REF] Pouponneau | Biocompatibility of candidate materials for the realization of medical microdevices[END_REF] but there biocompatibility is limited in time.

Challenges in designing magnetic actuation platforms and micro-robots

The addressed magnetic actuation systems are based on electromagnets which generate a magnetic field B by applying a current I. The linearity and the strength of B depends on the input I and the coil parameters, like the number of loops, size, geometry and core of the electromagnet. Designing magnetic micro-robots involves the optimization of the electromagnets' parameters, thus maximizing the workspace and DoF of the micro-robots, the generated forces while manipulating micro-objects, as well as minimizing the latency and non-linearities of the coils for a fast and robust control of the micro-robots.

As B can be controlled by I, its intensity can be increased or turned off. Besides, the current signal can be modulated through software and can exhibit different forms: sinusoidal, sawtooth, square, triangle etc.. Thus, different magnetic fields can be produced to obtain trajectory control. The maximum admissible current intensity in the electromagnet depends on how much heat the conductive material can withstand and how fast it can be dissipated by the coil. However, cooling systems [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF], or square waveform current instead of continuous current [START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] can be used.

Besides, the strength and linearity of the magnetic field depend on the core of the coil. The field generated by a coil without a core (air-core) Fig. 2(a) is directly proportional to the applied current intensity. The generated magnetic field is strongest inside the coil, and falls off rapidly outside of its volume. It is then important to keep the micro-robot as close as possible to the coil, which restrains its workspace. However, it is possible to generate a stronger magnetic field outside of the electromagnet by adding a soft magnetic core at its center Fig. 2(b). If the core reaches magnetic saturation the magnetic field is no longer proportional to the current intensity input. In order to improve the micro-robot DoF, several fixed electromagnets can be controlled to produce different fields one at a time, or more complex fields when combined. The resulting field can then be considered as the linear superposition of the effects of each individual electromagnet for air-core coils, even though a very weak mutual induction exists between the coils. This approximation can also be applied to ferromagnetic core coils, as long as the sources are far enough from each other for the field generated by an electromagnet not to affect another's core [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF]. The challenge in choosing the coils configuration and micro-robot DoF is to reduce the distance between the electromagnets and the micro-robot to increase the available magnetic field strength, while keeping a certain distance to avoid overwhelming nonlinearities. Moreover, as one coil induces displacement of the micro-robot in one direction, the number of the used coils and their configuration influences the DoF of the micro-robot. Thus, to obtain controlled 2D motion at least 4 coils are needed, and for 3D fully controlled motion a minimum of 6 coils is required. Some typical configurations can be used in the design. For example, in a method of actuation relying on propulsion, Helmholtz coils2 generate a uniform magnetic field intensity near the center of the coil to align the micro-robot located near the center of the coil along the coil's axis, and Maxwell coils3 are used to generate a uniform gradient magnetic flux and propel the micro-robot in the desired direction [START_REF] Jeong | Remote Controlled Micro-Robots Using Electromagnetic Actuation (EMA) Systems[END_REF]. However, due to the electromagnets' size, the admissible configurations are restrained. This can be tackled by using a dynamic configuration of the coils i.e. moving the electromagnets [START_REF] Hagiwara | High speed microrobot actuation in a microfluidic chip by levitated structure with riblet surface[END_REF].

Once the macro scale actuation system is designed, the micro-robot fabrication can be addressed. Magnetic micro-robots are usually made of ferromagnetic materials and have more or less simple geometric shapes depend-ing on their intended purpose; for example they can have a crescent shape to clasp micro-devices. Micro-robots are sized from tens to hundreds µm. Since adhesion forces at this scale are very high and depend on surface roughness, the manufacturing method should offer accurately controlled micro-robots surfaces. To machine such micro-devices, macro scale processes can be used such as laser fabrication. Micro-fabrication process like lithography can also be used in machining micro-devices. Laser fabrication consists in cutting a large stock of a material using a small laser spot, whereas lithography-based fabrication combines operations of selective deposition and growth of materials, like sputtering, chemical vapor deposition, electroforming, spin-coating, etc., and selective removal, such as deep-reactive ion etching (DRIE). Lithography allows machining micro-objects with layers of different materials, and guarantees a smoother surface than laser fabrication, however the latter preserves magnetization and density values of the material which is not the case with a lithography-based process. The challenge for future magnetic micro-robots is to include internal DoF, like active clamps which can be actuated independently to perform complex micro-manipulation tasks.

Challenges in controlling magnetic micro-robots

The first step in controlling micro-robots is to perform open loop control. The user applies a given I to bring the micro-robot to a desired position P des = x des y des T in 2D 4 . An I applied through a coil generates a magnetic field B, the magnetic field applies a magnetic torque on the micro-robot following Eq. ( 2) and a magnetic force according to Eq. ( 1).The orientation of the device θ curr is commonly assumed to be equivalent to the magnetic field orientation [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF], but the translation of the micro-device needs to be identified. The control of the magnetic micro-robot can be based on I or B. In both cases, knowledge of the relation between the micro-robot velocity and the electric current input I or the magnetic field B is essential. Controlling magnetic micro-robots with B relies on the evaluation of B using hall sensors. In this case these sensors can be integrated all over the arena surface which is costly. However, they can be placed next to the coils and by interpolation, estimate B. Besides, the latency of the hall sensors should be minimal to perform fast control. When controlling the micro-robot through I, the identification of the generated B by a current I flowing in the coil is important and must include the latency and the non linearity of the coil. This identification can be performed by an analytic model, which does not take all the coil properties into account, or by a finite elements model (FEM), which is calculated off-line and cannot be used in a real-time fashion. However, the combination of the two methods is possible, i.e. simulating the response off-line and then integrating it in an analytic model for real-time computation.

In addition to the identification of the coil parameters, the computation of the direct/inverse model system, i.e the relation between the micro-robot velocity and B/I, or vice versa, is one of the main issues in controlling magnetic micro-robots. Actually, magnetic forces can induce the same accelerations regardless of the object's size, therefore a micro-robot can experience high accelerations relative to its reduced size However, depending on the environment, the micro-robot's behavior is disturbed by high adhesion forces in the air [START_REF] Abbott | Robotics in the Small, Part I: Microrobotics[END_REF], which depends closely on the surface of the arena and fabrication of the micro-robot. Adhesion forces can be reduced at this scale by using vibrations generated from a piezoelectric layer [START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF], or levitation using the electrostatic effect. In a liquid environment, the micro-robot experiences fluid drag forces [START_REF] Abbott | Robotics in the Small, Part I: Microrobotics[END_REF], which increase with its velocity; as the micro-robot operates, the liquid state changes and leads to system disturbances. Besides liquid drag forces, the micro-robot can experience adhesion forces if it sticks to the surface. To reduce or avoid surface adhesion forces in liquid, the surface of the micro-robot can be adapted to make it float [START_REF] Hagiwara | High speed microrobot actuation in a microfluidic chip by levitated structure with riblet surface[END_REF]; or it can be maintained in the fluid above the surface by capillary forces [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF]. The challenge in identificating the behavior of magnetic micro-robot systems lies in the significant influence of non-linearities, although systems parameters can be evaluated theoretically and corrected by several repeatable experiences. Other methods can be used such as neural networks [START_REF] Calderon | Nonlinear Dynamic System Identification with Dynamic Recurrent Neural Networks[END_REF] or learning machines [START_REF] Li | Nonlinear System Identification Using Extreme Learning Machine[END_REF], which showed good performance in identifying macro scale nonlinear systems.

To provide autonomy and time gain to the user, a closed loop system is required. In this case, the controller automatically computes the I needed to reach the desired position P des . Due to the reduced size of the robot, mounted sensors are impractical. The micro-robot position P curr = x curr y curr T can be detected using various sensors. Although, cameras are widely used due to the low cost and fast computational time. The camera should be fast enough (several hundreds of Hertz at least) to ensure a stable control law. The image processing can benefit from the contrast between the micro-robot and its background environment, for example by applying threshold and/or morphological operations on arena captions. Predictive control is suitable to speed up the control loop. Besides, other detection methods can be used such as event-based sensors, which are faster but have a more complex processing and need to be calibrated with a classic camera. Event-based sensors mimic biological visual systems and react to changes of contrast that are converted in a stream of asynchronous time-stamped events [START_REF] Lichtsteiner | [END_REF]. The controller uses the value returned by the detection method, and according to the desired position sets the input to the electromagnets. It thus minimizes the error ε *5 , which is the difference between the current and desired position values. The control of the micro-robot system is achieved by specific software according to the tasks to be executed. Thus, the control of magnetic micro-robots, besides avoiding obstacles and performing trajectories, has to be adapted to the micro-robots' tasks. Clearly, micro-objects manipulation differs depending on the environment. In the air, the micro-robot needs to generate important forces to push/pull micro-devices and contract the adhesion forces, whereas in liquid the control of the micro-robot needs to adapt to the fluid movement carrying the micro-objects. Besides the control of one micro-object, controlling multiple micro-robots in cooperation may allow to perform more complex tasks, like handling a bigger object, and cooperatively handle an object.

In the following sections, four magnetic micro-robots are presented, and the approaches proposed to handle the above mentioned challenges are highlighted.

3 Mag-µBot (Magnetic micro roBot)

Micro-robot design

The Mag-µBot was introduced in [20] [START_REF] Diller | Control of multiple heterogeneous magnetic microrobots in two dimensions on nonspecialized surfaces[END_REF]. The system contains six identical electromagnetic coils in an orthogonal configuration. The coils surround the arena, wherein resides the magnetic micro-robot (Mag-µBot) Fig. 3(a). The maximum generated magnetic field and gradient field are respectively around 15 mT and 0.65 T.m -1 , these values being measured at the center of the arena. A camera is located inside the top electromagnet. It is a classic CCD camera coupled with a variable magnification microscope lens providing a field of view of 26 mm × 20 mm.

A Mag-µBot is a composite of Neodymium-Iron-Boron. Two fabrication processes can be used in machining Mag-µBots. First, a laser micro-machining system has been used in the fabrication of the Mag-µBot [START_REF] Pawashe | Modeling and experimental characterization of an untethered magnetic micro-robot[END_REF]. The second process relies on batch manufacturing using molding techniques. The micro size moulds are produced by a lithography based process. The process allows the production of large numbers of Mag-µBots. To handle micro-components, the micro-robot's shape can be adapted. Thus, the Mag-µBots is fabricated in several shapes (parallelepiped, cylinder etc.). Fig. 3 Fig. 3 The Mag-µBot system design

Micro-robot actuation

The Mag-µBot's movement is produced by applying a time varying magnetic field. Thus the Mag-µBot exhibits a non-uniform rocking behavior, which results in a stick-slip motion across the arena surface as shown in Fig. 4. However, this composite motion limits the velocity of the Mag-µBot. The orientation of the micro-robot is produced by one of the in plane coils, while the Mag-µBot is held down by the magnetic field generated by the bottom coil. The translation of the micro-robot is achieved by out-of-plane pulses. The top coil applies an alternating magnetic field leading the micro-robot to rock downward when the field decreases, and upward when it increases, and due to surface friction the micro-robot experiences stick and slip phases which result in translating movement. As the motion of the Mag-µBot is composed of rotations, the torques are responsible for the microrobot going forward or rotating. The magnetic torques are given by Eq. ( 2). As the movement of the Mag-µBot is based only upon magnetic torques, the system requires a much smaller magnetic field strength than magnetic gradient-based systems (i.e magnetic force-based systems). The velocity of the Mag-µBot can reach 25 mm.s -1 , as performed at the NIST IEEE Mobile Microrobotics 2010 Challenge [44].

Fig. 4 Mag-µBot performing a translation. The solid line is the initial micro-robot position, the dotted line is the position of the micro robot after movement. In the stick phase the micro-robot is stationary because of surface friction, as soon as the contact between the micro-robot and the surface slips, the Mag-µBot translates. [START_REF] Floyd | Dynamic modeling of stick slip motion in an untethered magnetic micro-robot[END_REF] .

Motion control

In order to servo the system, a correlation is set up between the current signal input and the micro-robot's behavior. Simulations and experiments were done to determine the velocity of the micro-robot as a function of the signal frequency and surface roughness. Different waveform current signals (shape, frequency and amplitude varying) were applied to the system and the resulting velocities were measured from the camera frames. Thus, multiple surfaces were tested: glass, rough silicone, smooth silicone. Besides, air, vacuum and nitrogen atmospheres were tested. The Mag-µBot was also used in water, where precautions were taken to avoid the solid/liquid interface from getting the micro-robot stuck on the surface. The Mag-µBot showed lower velocities in water than in air, since the drag forces are more important in water. As the velocity of the micro-robot depends on the frequency of the current signal, values are obtained from experimental frequencies relating to the surface it operates on. Fig. 5 shows some results of these experiments and simulations. The effect of the magnetic torques is predominant compared to other forces, thus the orientation of the Mag-µBot is assumed to be equivalent to the orientation of the magnetic field in the -x,y-plane.

(a) Sawtooth and reverse sawtooth waveforms signal input [START_REF] Pawashe | Modeling and experimental characterization of an untethered magnetic micro-robot[END_REF].

(b) Square and sine waveforms signal input [START_REF] Pawashe | Modeling and experimental characterization of an untethered magnetic micro-robot[END_REF].

Fig. 5 The Mag-µBot velocities as a function of the current signal frequency, both experimental and simulated results. The Mag-µBot is traveling on a rough silicon surface in air.

The control is performed using visual feedback. The image processing returns the current position of the micro-robot to the controller. Given the goal position, the controller sets the adequate current signal frequency and amplitude. The image processing is a particle filtering algorithm coupled with state prediction [START_REF] Isard | Condensation -conditional density propagation for visual tracking[END_REF]. The controller is a proportional-integrator-derivative (PID), with additional imposed nonlinear logic. The nonlinear logic regulates the orientation of the magnetic field and the velocity of the robot. When too high angular accelerations are applied, the Mag-µBot can flip on its edge and its behavior becomes unknown. The aim of this block is to improve the stability of the micro-robot motion by limiting these accelerations. Fig. 6 shows the block diagram of the Mag-µBot control.

Experimental results and discussion

The reliable control of the micro-robot allows a higher level of Mag-µBot autonomy. The system is provided with a motion planner. This module computes a free collision path from an initial position to a goal position, in a clustered environment. In this case, the user defines in advance the obstacles in the environment of the robot. The planner sends to the controller the positions to follow to reach the goal and the controller generates the needed current signal to the coils. The path is computed using the Wavefront algorithm [START_REF] Steven | Planning Algorithms[END_REF]. The control of the Mag-µBot generates sufficient forces and is accurate enough to perform interaction with micro-objects, such as pushing a 50 µm diameter sphere underwater. Moreover, as the Mag-µBot operates in liquid, the manipulation of micro-objects is performed using two strategies. Non-contact handling is performed using the flow-field generated by the Mag-µBot [START_REF] Pawashe | Two-Dimensional Autonomous Microparticle Manipulation Strategies for Magnetic Microrobots in Fluidic Environments[END_REF]. A micro-object can be also moved by direct contact, in this case the displacement of the micro-objects before contact due the flow field is significant and needs to be taken into account.

The Mag-µBot system also offers the ability to control multiple micro-robots, either by adapting the surface with electrodes and using the electrostatic effect to clamp all micro-robots except the one to move [START_REF] Floyd | Microparticle manipulation using multiple untethered magnetic micro-robots on an electrostatic surface[END_REF] [49], or by using micro-robots of different sizes, in which case each micro-robot has a specific actuation signal frequency [START_REF] Diller | Control of multiple heterogeneous magnetic microrobots in two dimensions on nonspecialized surfaces[END_REF]. To control the micro-robot, it must keep contact with the surface. Indeed, if the contact is broken, the robot flies during much of its travel due to the magnetic gradient. The micro-robot will then experience high velocities which are impractical for use from a controllability point of view. This behavior occurs when using the top coil, which orients the micro-robot upwards and towards the in plane coils, making the micro-robot lose contact with the surface. Hence, with this method the Mag-µBot system is restricted to 2D motion.

OctoMag

Micro-robot design

The OctoMag [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF] micro-robot, an elliptical shaped micro-device made of Nickel-Cobalt, is surrounded by 8 soft-magnetic-core electromagnets Fig. 7. These coils can withstand a current intensity of 15 A provided by the amplifiers. The temperature in typical operating conditions is around 60 o without a cooling system, and can rise excessively if maximum magnetic forces are applied. Thus, the coils require a cooling system. The disposition of the coils were defined in order to optimize the force capability generation throughout the arena. To observe the micro-robot, two cameras are disposed with additional optic zoom, one on the top of the arena and the other at the side of the scene. The micro-robot operates in a wet environment. The OctoMag micro-robot is a threedimensional structure made of two micro-assembled individual parts. Micro-robot parts have been machined with electroplated nickel using a lithography process then laser cut steel [START_REF] Berk Yeşin | Modeling and control of Untethered Biomicrorobots in a Fluidic environment using Electromagnetic Fields[END_REF].

Fig. 7 The OctoMag electromagnetic system. 8 soft-magnetic-core electromagnets surround the arena containing the micro-robot. A camera is fitted down the central axis to image the micro-robot [START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF].

Micro-robot actuation

The design of the OctoMag system allows the micro-robot to perform 6 DOF movements. However, the magnetic torque tends to align the longest axis of the micro-device with the magnetic field. Therefore, rotations along this axis are not practical, leading to the loss of one DOF. Hence, the control of the micro-robot is restricted to 5 DOF : 3 axis translation Eq. ( 1) and 2 rotations Eq.( 2). For -n-electromagnets, each electromagnet -e-generates a magnetic field B e that can be precomputed for every point P throughout the arena Eq. ( 4).

B e (P) = Be (P)i e [START_REF] Jing | A magnetic thin film microrobot with two operating modes[END_REF] where Be [T.A -1 ] is a unit-current vector and i e [A] is a scalar current value. As the field generated by a coil is known and it is assumed that the soft magnetic core used in the electromagnets is ideal, the superposition theorem can be applied i.e the field generated by several coils is the sum of the magnetic fields generated by each coil. Thus, the magnetic field can be identified at any point of the arena as function of the current input.

B(P) = n e=1
Be (P)i e = B(P)I

where I is a vector of the unit current i e . As, the Octomag's micro-robot is force controlled and the force is related to B with Eq. ( 1), the derivative of the magnetic field is calculated for a given direction -x,y,z-at every point in the workspace. According to Eq. ( 5), the magnetic field and force is mapped through the arena for a given current signal I by the following equation:

B F m =               B(P) M T B x (P) M T B y (P) M T B z (P)                           i 1 . . . i n             = A(M, P) I (6)

Motion control

The system is controlled using micro-robot position feedback from a camera. The OctoMag's micro-robot has 5 DoF, thus detection of the micro-robot in space requires two cameras. The x,y position are deduced from a top camera and the z position from a side camera. The tracking scheme of the OctoMag benefits from the use of adaptive thresholding and morphological operators, such as erosion and dilatation. The orientation of the microrobot is controlled by an open loop assuming that if the orientation of the magnetic field is changed slowly, the orientation of the micro-robot will be the same as the orientation of the magnetic field. Hence, the micro-robot's orientation is not measured. The magnetic field and forces are proportional to I Eq.( 6).For a desired magnetic field and force, the current can be computed for each coil by inverting the OctoMag model:

I = A(M, P) + B des F des (7)
Fig. 8 depicts the block diagram of the OctoMag.

Experimental results and discussion

The main advantage of the OctoMag is to perform 3D motion, however only 5 DoF are fully controlled. To evaluate the precision of the OctoMag's positioning, the Octomag was controlled from a random position in the workspace to the center of the arena. Positioning of the micro-robot showed a standard deviation of 6.313, 4.757, and 8.951 µm along the x, y, and z axes, respectively, and a maximum Cartesian deviation of 29.77 µm. The positioning error was measured from 400 frames of the camera, collected at 30 Hz and the Octomag system was tested by keeping the absolute value of B constant (|B|=15 mT) to guarantee the linearization of the controller. The following figure shows a 3D ∞ shaped trajectory performed in an automated mode. Due to the positioning error, the micro-robot deviates from the desired path in some parts, however the micro-robot recovers from this drift and follows the trajectory. One of the potential application of the OctoMag system is to perform in vivo ophthalmic procedures, like the puncturing of retinal veins to inject thrombolytic drugs. Thus, the system was tested in vitro. It successfully performed wireless vessel puncture of blood vessels of the chorioallantoic membrane (CAM) on a growing chicken embryo, which is considered a valid test setup for applications on human retinal vessels. In this experiment, another shape of the micro-robot was used: two NdFeB cubes, of 800 µm edge, and a needle tip were glued. The overall size of the robot is more than 1600 µm in length, the device can then apply enough force to puncture vessels.

Magnetic control of paramagnetic particles

Micro-robot design

An image-based magnetic control of paramagnetic micro-particles system was presented in [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF] and [START_REF] Islam | Wireless magnetic-based control of paramagnetic microparticles[END_REF]. In this work, the micro-robot is a sphere of diameter ∼ 100 µm made of a paramagnetic material (iron oxide). The paramagnetic particles are standards and commercially available. Four identical electromagnets are responsible for the actuation of the micro-spheres, which lie in an arena Fig. 10(a) containing a liquid (water), and are maintained in the water-to-air boundary. This configuration of the micro-spheres avoids the presence of surface adhesion forces as the micro-spheres are not in contact with the bottom of the arena. The contained liquid forms a meniscus at the center of the arena. Thus, to reduce this liquid deformation the arena size is increased and only a parcel of the arena is observed. The particles are observed by a camera with a 100 ms image period, fitted with a microscope objective.

(a) Scheme of the system, four coils surrounding an arena [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF].

(b) Picture of the controlled paramagnetic particles [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF].

Micro-robot actuation

As the particles are made of a paramagnetic material, they exhibit a particular behavior under the influence of a magnetic field. The magnetic forces Eq. ( 3) on these particles depend on the gradient field squared. Besides, the micro-spheres are held in the water-to-air layer by capillary forces, hence it is assumed the only force opposing the micro-spheres' movement is the drag force induced by the liquid environment. The drag force increases with the velocity of the particle, thus the maximum speed of the particle is reached when the magnetic force equals the drag force.

Motion control

The control of paramagnetic particles relies on using a tracking algorithm to detect particle motion. To benefit from a fast detection method, the tracking algorithm uses a region of interest, the center of which is either defined by the user or assumed to be the last position of the particle. Moreover, to overcome the lighting disturbances, an adaptive thresholding is then applied on this region. Afterwards, an erosion filter is applied to reduce noise on the frame. As several micro-particles are used once in a time, the particle is assumed to be the biggest object on the frames. A size test is also applied to verify that the detected particle has an acceptable size, otherwise the particle is considered lost. In the experiment, the environment was carefully prepared to avoid any disturbances. The image processing algorithm reaches its limit when several particles are used and more than one particle is in the region of interest : the algorithm returns an inexact position, which is the average of the particle positions inside the region.

The tracking algorithm provides PI6 controllers -one for each axis (x,y) -with the difference between the actual position and the desired position of the particle. The PI controllers have the same gains for each axis Fig. 11.

Fig. 11 Schematic diagram of the control loop. Two PI controllers are used to control the particle in both X and Y directions.

Experimental results and discussion

The experiments demonstrated in [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF] show that controlling a particle towards a position takes around 15 s to reach a steady state. The current intensity applied to the coils is 0.8 A which generates a gradient field squared of 1.8 mT 2 .m -1 . Under this condition, the speed of a particle of 100 µm is 235 µm.s -1 . Fig. 12(a) represents the positioning of a particle, the standard deviation according to x and y is about 8.4 µm. Fig. 12(b) shows movement of a particle around a path of preset points. The movement of the particle is not smooth due to significant overshoot in particle positioning. Moreover, it is assumed that the controlled particles are always held in the air-to-water boundary, avoiding buoyancy forces. Thus, the experiments need to be carefully prepared, otherwise the proposed model of the system is not valid anymore.

(a) Positioning of a particle according to x and y directions [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF] .

(b) The particles following a preset of points. The path is obtained from the image processing algorithm [START_REF] Jasper | Image-based magnetic control of paramagnetic microparticles in water[END_REF] . 6 MagPieR (Magnetic Piezo Robot)

Micro-robot design

The MagPieR is a micro-parallelepiped [START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF], composed of two heterostructure layers: a ferromagnetic material (Nickel) layer for magnetic driving on top of a bulk of PMN-PT (Pb(Mg 1/3 Nb 2/3 )O 3 -PbTiO 3 : a composite of Lead-Magnesium-Niobium-Titane), a piezoelectric material used to overcome surface friction Fig. 13(a).

The MagPieR is fabricated by electrodeposition of a Ni layer on PMN-PT substrate and then by saw dicing into small rectangular samples. In the Ni layer two trenches are cut in order to align the micro-robot with the magnetic field along the trenches. The MagPieR moves inside a capacitor. The bottom electrode is the arena substrate itself, (a) Image from Scanning Electronic Microscope of MagPieR [START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] (b) Disposition of the coils surrounding the MagPieR arena [START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] Fig. 13 The MagPieR design and the top electrode is an optically transparent conductive glass (ITO glass) allowing a visualization of the scene from above. The observation of the MagPieR is done by a top situated high speed camera (1000 frames per second). The micro-robot then moves in a dry environment. The arena is surrounded by four identical electromagnets Fig. 13(b), the coils require less than 1 A to activate the micro-robot. The coils generate a magnetic field that allows 3 DoF to the micro-robot: 2 translations in the x and y directions, and a rotation in the -x,y-plane.

Micro-robot actuation

At the micro scale, the surface adhesion force is the main factor limiting the movement of a micro-device in the air. To overcome this force the MagPier uses vertical vibration. High frequency, high voltage pulses on the bottom electrode actuate the piezoelectric effect to free the micro-robot from adhesion Fig. 14(a). In-plane motion (translations and rotation) is then obtained by applying an external magnetic field generated by the coil. Only one electromagnet at a time is used to actuate the robot. The magnetic field gradient induces a magnetic force according to Eq. ( 1), which results in the micro-robot's translation. The magnetic field exerts a magnetic torque on the micro-robot by Eq. ( 2) orientating the robot in a given direction. The signal input to the coils is an impulse Fig. 14(b). The velocity of the micro-robot varies according to the amplitude h and length u of this impulse.

Motion control

The behavior of the MagPieR is identified experimentally, thus the relation between the current intensity and the displacement of the micro-robot is known. As the micro-robots operates in a dry environment, adhesion forces are predominant and influence the model of the system. In fact, the adhesion forces vary throughout the surface of the arena at this scale, hence it is difficult to simulate the micro-robot behavior with a high resolution. The controller is based on driving the micro-robot to a region the center of which is the desired position. The boundaries of this region are marked by X min /X max and Y min /Y max , which are preset values obtained the identification of the system. In this work, the visual feedback returns an (x,y) position of the micro-robot in the workspace. The difference between the current position and the desired position is given to the PID controllers. The same PID controller is used for each coil. Moreover, every coil controls the micro-robot in one direction on a given (x-y) axis. Once the position error is given to the controller, for each axis the controller activates a coil, depending on the range of the error. Usable control laws are thus limited; simultaneous control of several coils should improve the motion of the MagPieR. Depiction Fig. 15 represents the block diagram of the controlling system.

Experimental results and discussion

The MagPieR showed good velocity performance, indeed it won the "2mm dash" competition twice at the NIST IEEE Mobile Microrobotics Challenge, being the fastest micro-robot by performing the 2 mm race in 15 ms. Fig. 16(a)represents the resulting response of serving the MagPier to an x-y position. The figure demonstrates clearly that the x and y variables are coupled. For example, at time 100 ms, the x position has reached its desired position but the y position has not. When a signal is sent to reduce the y error, the generated magnetic force is in fact oblique and leads to an additional movement of the micro-robot in the x direction.

The second figure Fig. 16(b) shows the control of the micro-robot through an ∞-shaped trajectory. The simulated trajectory is the continuous line, and the experimental trajectory is the dotted line. The difference between the trajectories is due to the coupling of the -x,y-directions. Moreover, the control scheme is based upon reaching a region rather than a trajectory, the resulting motion is irregular and lacks precision. The use of the gradient magnetic field in a dry environment allows the micro-robot to experience high accelerations, though significant non-linearities induced by this environment limit motion accuracy. Moreover, the close arrangement of the coils leads to large field gradients, but also leads to nonlinear field distributions. Thus, accurate control of the microrobot is hard to achieve.

Comparison and conclusion

Characteristics of the previously presented magnetic micro-robots are summarized in Tab. 2. Magnetic actuated micro-robots are made of ferromagnetic or paramagnetic materials. The shape is designed according to the application, and the robots usually have parallelepiped, cylinder or claw geometries. The above systems' coil setups respect the same configuration : four orthogonal electromagnets are used to produce 2D in plane motion; out of planes can be added to increase the DoF of the micro-robots or to perform 2D using only the magnetic torques. Besides, the average magnetic field strength generated by the considered systems is sufficient to induce motion in the magnetic micro-robot (∼10 mT). Some systems use ferromagnetic cores in their coils to increase the magnetic field strength, though the saturation of the core needs to be taken into consideration in these systems. The maximum current intensity admissible by a coil typically increases with its dimensions. However, the coils are subjected to excessive heating, and require a cooling system when a significant gradient field is needed.

The systems presented above use several types of actuation signals, such as continuous or periodic signals. Magnetic actuated micro-robots are mainly based on the gradient of the magnetic field which induces magnetic forces to translate the micro-device, the magnetic field being used to produce its orientations. However, the translation of micro-robots can also be induced by magnetic torques, which requires lower magnetic field strengths but generates limited velocities. In a dry environment, allow micro-robot experiences high accelerations, unlike in a liquid environment where the drag forces decrease the micro-robot velocity. Although, in a dry environment the adhesion forces are significant and lead to non-linearity in the system in addition to the non-linearity of the magnetic field distribution. For most of the systems, the dynamic model of the micro-robot is computed theoretically and corrected with experimental data.

Micro-robotic systems show high velocities compared to their reduced size. The controllers of the above systems perform a relatively accurate positioning, though the following of trajectories presents highly discontinuous behavior and irregularities. The control systems are, for the moment, insufficient to lead repeatable tasks at a high speed with a high reliability. The dynamics of the magnetic field and the micro-robot, established in an experimental way, do not reliably represent the micro-robot's behavior and non linearities.

The next challenges for untethered magnetic micro-robots are to benefit from 6 DoF navigation to fully explore the environment, and to be less sensitive to atmospheric variations. The coil setups can also benefit from optimized configurations in order to improve the time response in generating the magnetic field and to increase the workspace of the micro-robot. In the future, these untethered micro-robots will have a great potential for many medical, biological and industrial applications, thanks to their tiny size, wireless power and control.

  (a) Air core electromagnet magnetic field lines.(b) Iron core electromagnet magnetic field lines.

Fig. 2

 2 Fig. 2 Field lines comparison between an air core and iron core electromagnet. Data obtained by FEMM software 1 .

  (b) depicts the Mag-µBots with claws to manipulate micro-objects.

  (a) Picture of the Mag-µBot system, A: Camera, B: Microscope,C: Top coil, D: Upright coil, E: The arena, F: Bottom coil [42]. (b) The claw model of Mag-µBot: 300 µm x 300 µm x 100 µm [43].

Fig. 6

 6 Fig.6Block diagram depicting the method of control for positioning a Mag-µBot to a desired goal location.

Fig. 8

 8 Fig. 8 Block diagram representing the method of control for positioning the OctoMag.

  (a) The top and side view of the OctoMag's microrobot controlled to follow an ∞-shaped trajectory. The micro-robot's orientation is kept toward the center of the sphere. (b) This trajectory was performed in an average time of 8.3 s.The • are set path points and the + are the tracked positions. The direction of displacement is shown by the arrows.

Fig. 9

 9 Fig.9OctoMag control experiments[START_REF] Kummer | Octomag: An electromagnetic system for 5-dof wireless micromanipulation[END_REF]

Fig. 10

 10 Fig. 10 Design of the paramagnetic particles controlling system.

Fig. 12

 12 Fig. 12 The control system performance in (a) positioning and in (b) following preset point paths.

  (a) Externally induced vibration actuation[START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] .(b) Signal current sent to the coil[START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] .

Fig. 14

 14 Fig. 14 MagPieR actuation

Fig. 15

 15 Fig. 15 Controlling scheme of the MagPieR

  (a) Time responses to an x-y step input. The solid line is the desired position and the dashed line is the experimental result[START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] .(b) An ∞-shaped controlled trajectory of the micro-robot, the solid line is the planned trajectory and the dashed one is the trajectory executed by the robot[START_REF] Ivan | First experiment on magpier : a planar wireless magnetic and piezoelectric microrobot[END_REF] .

Fig. 16

 16 Fig. 16 MagPieR control results. One pixel is ∼ 20 µm

http://www.femm.info/

Helmholtz coil pair: two identical circular magnetic coils placed symmetrically one on each side along a common axis, and separated by a distance equal to the radius of the coil. Each coil carries an equal electrical current flowing in the same direction.

Maxwell coil pair: two identical circular magnetic coils placed symmetrically one on each side along a common axis, and separated by 1.73 times their radius. Each coil carries an equal electrical current flowing in the opposite direction.

Respectively P des = x des y des z des T in 3D

ε x , ε y , ε z , ε θ , are respectively the error along x, y, z direction and the orientation.

Proportional Integrator