Réjean Plamondon
email: rejean.plamondon@polymtl.ca

Christian O'reilly

Javier Galbally

Abdullah Almaksour

Éric Anquetil

Recent Developments in the Study of Rapid Human Movements with the Kinematic Theory: Applications to Handwriting and Signature Synthesis

Human movement modeling can be of great interest for the design of pattern recognition systems relying on the understanding of the fine motor control (such as online handwriting recognition or signature verification) as well as for the development of intelligent systems involving in a way or another the processing of human movements. In this paper, we briefly list the different models that have been proposed in order to characterize the handwriting process and focus on a representation involving a vectorial summation of lognormal functions: the Sigma-Lognormal model. Then, from a practical perspective, we describe a new stroke extraction algorithm suitable for the reverse engineering of handwriting signals. In the following section it is shown how the resulting representation can be used to study the writer and signer variability. We then report on two joint projects dealing with the automatic generation of synthetic specimens for the creation of large databases. The first application concerns the automatic generation of totally synthetic signature specimens for the training and evaluation of verification performances of automatic signature recognition systems. The second application deals with the synthesis of handwritten gestures for speeding up the learning process in customizable on-line recognition systems to be integrated in electronic pen pads.

Introduction

Human movement modeling can be of great interest for the design of pattern recognition systems relying on the understanding of the fine motor control, like on-line handwriting recognition and signature verification, as well as in the development of intelligent systems involving in some way the analysis of human movements. Among other things, this general approach aims at elaborating a theoretical background for any handwriting processing application as well as providing some basic knowledge that can be integrated in the development of automatic systems. So far, many models have been proposed to study human movement production in general and handwriting in particular : models relying on neural networks [START_REF] Bullock | The VITE model: a neural command circuit for generating arm and articulator trajectories[END_REF][START_REF] Schomaker | Simulation and recognition of handwriting movement: a vertical approach to modeling human motor behavior[END_REF][START_REF] Gangadhar | An oscillatory neuromotor model of handwriting generation[END_REF][START_REF] Kalveram | A neural oscillator model learning given trajectories, or how an allo-imitation algorithm can be implemented into a motor controller[END_REF], equilibrium point models [START_REF] Feldman | Functional tuning of the nervous system with control of movement or maintenance of a steady posture[END_REF][START_REF] Feldman | Testing hypotheses and the advancement of science: recent attempts to falsify the equilibrium point hypothesis[END_REF][START_REF] Bizzi | Effect of Load Disturbances During Centrally Initiated Movements[END_REF]1992), behavioral models [START_REF] Schmidt | Motor Control and Learning: A behavioral Emphasis[END_REF][START_REF] Thomassen | Motor aspects of handwriting[END_REF][START_REF] Van Galen | The independent monitoring of form and scale factors in Handwriting[END_REF], coupled oscillator models [START_REF] Hollerbach | An oscillation theory of handwriting[END_REF][START_REF] Kelso | Dynamic patterns: The self-organization of brain and behavior[END_REF]Zazone et al., 2005), kinematic models (Plamondon, 1995;[START_REF] Plamondon | A multi-level representation paradigm for handwriting stroke generation[END_REF], and models exploiting minimization principles [START_REF] Wada | A theory for cursive handwriting based on the minimization principle[END_REF][START_REF] Engelbrecht | Minimum principles in motor control[END_REF] : minimization of the acceleration [START_REF] Neilson | The problem of redundancy in movement control: the adaptive model theory approach[END_REF][START_REF] Neilson | An overview of adaptive model theory: solving the problems of redundancy, resources and nonlinear interactions in human movement control[END_REF], of the energy [START_REF] Nelson | Physical principles for economies of skilled movements[END_REF], of the time [START_REF] Tanaka | An optimization principle for determining movement duration[END_REF][START_REF] Enderle | Time-optimal control of saccadic eye movements[END_REF][START_REF] Hermes | Functional analysis and time optimal control[END_REF], of the jerk [START_REF] Hogan | An organization principle for a class of voluntary movements[END_REF][START_REF] Flash | The coordination of Arm Movements: An experimentally confirmed mathematical model[END_REF], of the snap [START_REF] Edelman | A model of handwriting[END_REF], of the torque changes [START_REF] Uno | Formation, Control of optimal trajectory in human multijoint arm movement[END_REF] and of the sensory-motor noise [START_REF] Harris | Signal-dependent noise determines motor planning[END_REF]. Finally, many models exploit the properties of various functions to reproduce human movements: exponentials [START_REF] Plamondon | Modelization of Handwriting: A System approach[END_REF], second order systems (Denier van der [START_REF] Van Der Gon | The guiding of human movements[END_REF][START_REF] Dooijes | Analysis of handwriting movements[END_REF], gaussians [START_REF] Leclerc | Des gaussiennes pour la modélisation des signatures et la segmentation des tracés manuscrits[END_REF], beta functions [START_REF] Alimi | Beta neuro-fuzzy systems[END_REF], splines [START_REF] Morasso | How a discontinuous mechanism can produce continuous patterns in trajectory formation and handwriting[END_REF] and trigonometrical functions [START_REF] Maarse | The Study of Handwriting Movement: Peripheral Models and Signal Processing Tehcniques[END_REF].

Among the models which provide analytical representations, the Kinematic Theory of rapid human movements (Plamondon, 1995a(Plamondon, , 1995b;;[START_REF] Plamondon | A multi-level representation paradigm for handwriting stroke generation[END_REF] and its Delta-and Sigma-lognormal models have been used to explain most of the basic phenomena reported in classical studies on human motor control [START_REF] Plamondon | Speed/Accuracy tradeoffs in target-directed movements[END_REF] and to study several factors involved in the fine motricity [START_REF] Djioua | A new methodology to improve myoelectric signal processing using handwriting[END_REF][START_REF] O'reilly | Prototype-based methodology for the statistical analysis of local features in stereotypical handwriting tasks[END_REF]Woch et al., 2010). Apart from these fundamental studies, the theory has been used, directly or indirectly, in many practical applications like the design of a signature verification system [START_REF] Plamondon | The Design of An On-Line Signature Verification System: From Theory to Practice[END_REF], the development of tools to help children learning handwriting [START_REF] Djeziri | Learning handwriting with pen-based systems: computational issues[END_REF], as well as of biomedical set ups to detect fine motor control problems associated with brain strokes (O'Reilly and [START_REF] O'reilly | Impact of the principal stroke risk factors on human movements[END_REF][START_REF] O'reilly | Design of a Neuromuscular Disorders Diagnostic System Using Human Movement Analysis[END_REF].

In this paper, we report on two new and original case studies dealing with the automatic generation of synthetic handwritten specimens for the creation of large databases. The first application addresses the automatic generation of totally synthetic signature specimens which may be used for the training and evaluation of the verification performances of automatic recognition systems as well as for the quality assessment of specimens. The second application regards the synthesis of handwritten gesture for speeding up the learning process in customizable on-line recognition systems to be integrated in electronic pen pads. Sections 5 and 6 reports detailed results about these two genuine applications, which at the time of the ICFHR 2010 keynote address presented by the first author, were the first trial of using the Kinematic Theory for the generation of synthetic trajectories to be used in signature verification and gesture recognition experiments.

To better understand these applications and estimate their potential interest, as well as making the present paper self-consistent a brief survey of the Kinematic Theory is presented in section 2, two algorithms used for sigma-lognormal parameter extraction are outlined described in section 3 and the main results on previous studies of handwriting variability are summarized in section 4. These sections present in a condensed and goal oriented way, the main concepts and strategies that have been explored over the years and that are necessary to understand the present applications, without coming back to these complete and often more exhaustive studies.

The Kinematic Theory of Rapid Human Movement and its Sigma-

Lognormal Model

One key feature of the Kinematic Theory is that it relies on strong and robust mathematical grounds. All the models that are used under this paradigm are based on the lognormal function which has been proved to be the ideal curve for describing asymptotically the impulse response of a neuromuscular network made up of a large number of coupled subsystems controlling the velocity of a movement [START_REF] Plamondon | A kinematic theory of rapid human movement. Part IV: a formal mathematical proof and new insights[END_REF]. For simple reaching or pointing gestures, a target is specified and two of these networks are needed to control a trajectory, an agonist network which is acting in the target direction and one antagonist, acting in the opposite direction. Overall, the speed profile is then described by a Delta-lognormal equation, a weighted difference of two lognormals (Plamondon, 1995a(Plamondon, , 1995b)). When more complex trajectories have to be generated, like in handwriting or in signing, a sequence of targets has to be reached and, globally, the trajectory of the pen tip can then be described by a vectorial summation of lognormals, hereinafter called sigma-lognormal equations, which takes into account the various changes of direction.

In this vectorial summation context, the production of a word or of a signature requires the definition beforehand of an action plan that is made up of virtual targets, which are linked in pairs with an arc of circle. This map of paired target points represents a sequence of discontinuous strokes. This plan triggers a motor command generator that produces a series of impulses activating the neuromuscular systems characterized by their lognormal impulse response [START_REF] Plamondon | A Neural Model for Generating and Learning a Rapid Movement Sequence[END_REF]. For each impulse, a lognormal velocity profile is generated at the pen tip and the time superimposition of these strokes results in a smooth and well controlled trajectory. According to this representation, the original strokes are thus hidden in the signal.

Mathematically, the Sigma-Lognormal model considers the velocity of the pen tip, , as described by a vectorial summation of N lognormal primitives:

2 0 11 ( ) ( ) ( ) ( , , , ); 2 NN i i i i i i ii v t v t D t t t N        (1)
Each lognormal in this summation defines a stroke scaled in amplitude by a command parameter ( ) and time-shifted by the time occurrence of this command ( ), any individual stroke pattern being described by a lognormal time function:

  2 0 2 0 2 0 ln( ) 1 ( ; , , ) exp 2 2 ( ) ii i i i i ii tt tt tt              (2)
Each of these primitives is also assumed to occur around a pivot, and the evolution of the angular position of the trajectory can be calculated using an error function (erf):

0 ( ) ln( ) ( ) 1 2 2 ei si i i i si i tt t erf                    (3)
where si  and ei  refer, respectively, to the starting and ending angular direction of each stroke. In equations ( 2) and (3), i  and i  represent correspondingly the logtime delay and the logresponse time of the neuromuscular system as it reacts to the i th command [START_REF] Plamondon | A multi-level representation paradigm for handwriting stroke generation[END_REF].

Under these conditions, the synergy produced by the interaction and coupling of many of these neuromuscular systems results in the sequential generation of a complex handwriting sample or a signature pattern.

Sigma-lognormal Parameter Extraction

To use the Sigma-Lognormal model for analyzing human movements, it is necessary to have an algorithm to solve the inverse problem in a fully automatic fashion, that is, to extract the lognormal parameters that most adequately fit the experimental data. The Sigma-Lognormal parameters are considered to be well estimated and fitted for statistical analysis if the SNR, defined in (4), is over 20dB.

In this equation, are the experimental (numerical) velocity signals and are the velocity signals of the sigma-lognormal reconstruction.

In the last years, two complementary algorithms have been proposed to solve this nonlinear regression problem, the Robust Xzero based algorithm and the prototype based algorithm. The next subsections briefly overview the state-of-the-art regarding these parameter extractors.

The Robust Xzero based extractor

The Robust Xzero (RX0) based extractor is a powerful algorithm that provides an accurate set of sigma-lognormal parameters describing the end-effector trajectory (e.g.

the pen tip trajectory in handwriting studies) of arbitrarily complex motions without any a priori knowledge regarding the nature of the movement. In the following text, an outline of the algorithm is presented. A more comprehensive description can be found in [START_REF] O'reilly | Development of a Sigma-Lognormal representation for on-line signatures[END_REF][START_REF] O'reilly | Développement d'outils d'analyse de la motricité fine pour l'investigation de troubles neuromusculaires: Théorie, prototype et mise en application dans le contexte des accidents vasculaires cérébraux[END_REF].

To implement this algorithm, sequences of five characteristic points (i=1,2,..,5) must be located in the original speed signal . Following a time occurrence order, these points are: a local minimum, an inflexion point, a local maximum, an inflexion point and a local minimum. The sigma-lognormal representation of these points can be written as in equations (5-6) with the parameters defined in (7).

(

) (6) (7) 5
Nine estimators can be obtained for the values of the kinematic parameters corresponding to the nine different combinations (with , , ) in the equations ( 8)-( 11).

(

) (9) (10) (11) 8
The angular parameters associated with each estimation of the kinematic parameter set is obtained using ( 12)-( 13), where (with erfc being the complementary error function defined as ) and is direction angle of the trajectory with respect to time.

(12) (13)

A choice can be made between the nine estimations of the six sigma-lognormal parameters by keeping the solution minimizing the error function ( 14).

(

) 14
It should be noticed that, before using the values in the previous expressions, some preprocessing should be applied to get proper estimations 1 . To extract the parameters of a whole velocity signal, good results have been obtained by, first, extracting sequentially (i.e. in increasing order of their time occurrence) the lognormal components. For that matter, each lognormal is extracted and subtracted from the original signal before proceeding to the next component. Then, a global non-linear optimization process can be applied to improve the estimated values. If this approach results in an unsatisfactory reconstruction SNR, more lognormal components can be extracted by processing them by decreasing importance of their impact (assessed here by their relative size) on the signal.

The latest improvements included in this extraction system have resulted in a significant increase of the SNR fitting accuracy. For example, on a 683 signatures database 1 The details of these preliminary computations are presented in [START_REF] O'reilly | Development of a Sigma-Lognormal representation for on-line signatures[END_REF][START_REF] O'reilly | Développement d'outils d'analyse de la motricité fine pour l'investigation de troubles neuromusculaires: Théorie, prototype et mise en application dans le contexte des accidents vasculaires cérébraux[END_REF].

comprising 124 subjects, an average increase of 7.9 dB has been obtained, passing from 17.4 dB in [START_REF] O'reilly | Development of a Sigma-Lognormal representation for on-line signatures[END_REF] to 25.3 dB. Fig. 1 gives an example of a complex movement, in this case a signature, fitted using the proposed RX0 approach (SNR=22.2dB).

The prototype based extractor

Although the system based on the Robust Xzero estimator gives very satisfying extraction results on complex movements, an alternative extraction strategy may be of great use under certain experimental scenarios. For example, this is the case of the analysis of stereotypical movements such as those often involved in psychophysical experiments. This type of experiments present some specific characteristics that can make an alternative extraction method better fitted for the task than RX0. First, a lot of a priori information on the nature of the movement is available which may be very helpful during the extraction process (the RX0 algorithm is not designed to take advantage of this knowledge). Second, in this kind of experimental framework, researchers may want to perform statistical testing of hypotheses on the value of local parameters. This may be difficult with the solution obtained by an extractor such as the one based on RX0 because, in this case, there is no clear correspondence between the parameters extracted from various movements.

These reasons supported the development of the prototype based extractor presented in (O'Reilly and [START_REF] O'reilly | Prototype-based methodology for the statistical analysis of local features in stereotypical handwriting tasks[END_REF]. The advantages of such an extractor can be seen, for example, in (O'Reilly and [START_REF] O'reilly | Impact of the principal stroke risk factors on human movements[END_REF] where it has been used to assess the neuromuscular health of subjects on the basis of the sigma-lognormal parameters of their movements.

This extractor applies a three step methodology: 1) synthesis of a sigma-lognormal prototype of the stereotypical movements, 2) time scaling and offsetting of the prototype to make it more closely correspond to the experimental movement, and 3) global nonlinear optimization of the scaled and offset prototype to improve the fitting. These three steps can be briefly described as follow:

-

Automatic Generation of Trajectories: Variability Issues

As we have seen, according to the Kinematic Theory, a complex movement results from the superposition of a set of elementary movements (corresponding to single curved strokes), localized both in time and space. So, the large variability observed in handwriting patterns can be interpreted as caused both by the intrinsic variability of the individual strokes and by fluctuations occurring in the time plan of the superimposition process as controlled by the central nervous system.

The local variability observed in handwriting and signing can thus come from various sources. At the central level, a movement is represented by an action plan, a sequence of virtual targets describing a piece-wise discontinuous trajectory. In parameter terms, this plan is a timed sequence of arcs described by their length, directions and time of activation. This discontinuous pattern, once instantiated, stimulates a set of neuromuscular networks that react to each of these fundamental primitives with specific time delays and response times.

In this context, at least three basic sources of variability can be identified: Overall, these studies have shown the existence of a direct relationship between the fluctuations of the sigma-lognormal parameters and the space and time warping of a pen tip trajectory, which suggests the feasibility of using this model as a new tool for the design of synthetic human like movements. This will be shown in the next two sections,

where the relative importance of the three sources of variability will be critical for the successful design of large synthetic databases.

Application 1: Automatic On-line Signature Database Generation

One of the main obstacles that the biometric technology has found, and still finds, to become one of the leading solutions in the security market, is the lack of large real biometric databases which may serve as common benchmarks for the development of this thriving technology. Two main reasons may explain such a scarcity of biometric data. On the one hand, biometric database collection is not at all an easy job, involving a lot of effort in terms of time and resources in order to reflect the variability present in biometric traits (both inter-and intra-class). This process includes a number of pre-acquisition and post-acquisition demanding tasks such as the recruitment of donors, the supervision of collected data, error correction or labeling [START_REF] Flynn | Handbook of Biometrics[END_REF]. On the other hand, biometric traits are classified as personal data, and as such they are subdued to the different personal data protection laws existing in each country, which makes the acquisition (donor's consent), and later storage and distribution (licensing) of these data very difficult.

Such a complex context has promoted over recent years the apparition of new algorithms for the generation of synthetic biometric databases [START_REF] Cappelli | Synthetic Fingerprint Generation[END_REF][START_REF] Zuo | On generation and analysis of synthetic iris images[END_REF]. These synthetically produced datasets are not affected by the acquisition and legal issues mentioned before: i) first, once the appropriate generation method has been developed, they are effortless to be produced, avoiding this way the arduous acquisition campaigns, and ii) second, the synthetic samples which conform these databases cannot be considered personal data as they have not been produced by a person, and so they may be freely distributed in order to be used as common evaluation benchmarks. These desirable characteristics make synthetic databases very powerful tools for the performance assessment of biometric recognition systems, and have already been used in international competitive evaluation campaigns [START_REF] Cappelli | Performance evaluation of fingerprint verification systems[END_REF].

In spite of presenting some very interesting features, the use of synthetic biometric databases is not yet generalized as the production of realistic synthetic samples still remains a challenging problem: modeling the information contained in a certain biometric trait as well as the inter-class and intra-class variation found in real databases (i.e., variation between samples of different subjects, and variation between samples of the same subject). Accepted solutions have been proposed for fingerprint [START_REF] Cappelli | Synthetic Fingerprint Generation[END_REF] or iris [START_REF] Zuo | On generation and analysis of synthetic iris images[END_REF][START_REF] Shah | Generating synthetic irises by feature agglomeration[END_REF], but still no consistent method has been given for the generation of synthetic handwritten signature databases.

As presented in previous sections, the Kinematic Theory of rapid human movements provides a powerful theoretical framework which models in a precise and compact manner the kinematic information involved in most of human writing processes, including signature. Thus, the Kinematic Theory and its associated Sigma-Lognormal model constitute a very high potential instrument for many different applications and have been applied in the present work to the development of an algorithm for the generation of fully synthetic on-line signatures.

The generation method

Two main parameters are involved in the design of real biometric databases and, hence, should also be critical in the generation of synthetically produced datasets: i) number of users comprised in the database, and ii) number of samples per user to be acquired. As can be seen in Fig. 2, the generation method of synthetic signature databases proposed in the present work is constituted of two different algorithms in order to produce: i) the first sample of fully synthetic individuals (i.e., it allows to control the number of users in the database), and ii) different samples derived from that original master signature (i.e., permitting to fix the number of samples per user). The objective of this initial stage of the generation algorithm is to produce samples from different synthetic signers (i.e., this algorithm is responsible for controlling the number of users in the database and for the inter-class variability present in the dataset).

In a first approach, a signature-like graphic is generated following the spectral approach described in [START_REF] Galbally | Synthetic Generation of Handwritten Signatures Based on Spectral Analysis[END_REF]. Although this first specimen has approximately the appearance of a genuine signature, it does not possess many of the humanly produced kinematic characteristics of real writing. Thus, in order to confer this preliminary master signature with the velocity and acceleration properties of human strokes, it is processed using the Sigma-Lognormal model in two consecutive stages, as shown in Fig. 3: -Stage 1: Extraction of the sigma-lognormal parameters using the RX0 system. In this phase, the velocity function of the initial synthetic master signature (v I ) is decomposed in singular strokes and the sigma-lognormal parameters (t 0 , D, µ, σ, θ s , θ e ) which best fit each of the individual strokes.

-

Generating the duplicated samples

Once the time sequences [x[n] y[n]

] defining the master signature of a synthetic user have been produced following the method described in Sect. 5.1.1, the next phase for the automatic generation of synthetic on-line signature databases is the creation of duplicated samples starting from that master sample (as is shown in Fig. 2).

Therefore, the objective of this part of the proposed method is to produce different samples of one same synthetic individual following the intra-variability found in real signatures (i.e., existing variability among signatures produced by the same user).

For this purpose, the velocity function v of the master signature is decomposed into single tendency of the function x (as it corresponds to left-to-right occidental signatures); 3) large fluctuation at the end of the x and y signals in some of the signatures (corresponding to some sort of round-like flourish); 4) degree of correlation between some of the most relevant maxima and minima points in the x and y directions. Furthermore, even though it is a model-based approach, some recognizable characters may be distinguished in the synthetic samples.

Validation experiments

The experimental validation of the proposed generation method is aimed at determining if the performance of signature verification systems is similar when it is evaluated on real and synthetic databases. If the error rates presented by signature-based recognition applications is comparable in both scenarios (i.e., performance evaluation with real and synthetic signatures), it would mean that, from a computer-based perspective, the synthetic signatures present a very similar behavior to that of real samples and that they can be used to obtain a fair estimation of a system performance, avoiding this way the different problems linked to real databases (i.e., high resource-consuming acquisition campaigns and legal issues regarding their acquisition and distribution).

In order to achieve this objective, the performance of two signature verification systems, using totally different feature sets and matchers, has been evaluated on the MCYT real database [START_REF] Ortega-Garcia | MCYT baseline corpus: A bimodal biometric database[END_REF] and on a Synthetic DB (SDB) produced using the proposed synthetic generation method.

The MCYT dataset has been selected as real test set since it has no overlap with the BiosecurID database used as development set for the estimation of the generation method parameters. This way we ensure to obtain totally unbiased results. The SDB has been created with the same number of users (300) and samples per user (25) as MCYT in order to permit the use of the same evaluation protocol for both scenarios.

The two on-line verification systems evaluated in the experiments are:

 System A: function based + HMM [START_REF] Fierrez | HMM-based on-line signature verification: feature extraction and signature modeling[END_REF]. This function-based verification system applies a regional approach using a statistical model built using Hidden Markov Models (HMMs) to a set of 10 time sequences selected applying the SFFS feature selection algorithm [START_REF] Pudil | Flotating search methods in feature selection[END_REF] to the total set of 34 functions defined in (Martinez-Diaz, 2009a).

 System B: function-based + DTW (Martinez-Diaz, 2009b). In this function-based local approach, a subset of nine time functions (selected using the SFFS from the total 34 feature set as in the case of system B) are directly matched using the elastic technique Dynamic Time Warping (DTW) [START_REF] Kholmatov | Identity authentication using improved online signature verification method[END_REF].

The performance results (Detection Error Trade-off, DET, curves) obtained for both verification systems are shown in Fig. 6. We can observe that the curves of the two systems present a very high degree of resemblance, both from a quantitative (EERs) and qualitative (general behavior) point of view, for the case of real and synthetic signatures.

The results derived from this validation experiment confirm the great potential of the Kinematic Theory of Rapid Human Movements applied to the generation of synthetic online signature databases, and the suitability of such datasets to obtain reliable estimations of the performance of signature verification systems.

Application 2: Synthetic Gesture Generation for Evolving

Handwriting Classifiers

Motivated by the increasing spread of many types of devices equipped with pen-based interfaces, such as PDAs, e-book, Tablet PCs, Whiteboards, etc., more emphasis is placed on the development of efficient recognition systems that can correctly interpret the gestures sketched by the user and then translate them either into computerized text or into some specific commands. Nowadays, the recognition systems in use are always pretrained on a fixed, predefined and a limited group of gestures, which usually contains the Latin letters and a few specific gestures. These systems do not allow users to add gestures in order to assign them to new commands or shortcuts, or to replace default gestures mapped to existing commands. In order to meet this important functionality, the static handwriting recognition systems that have been used so far must be replaced by novel dynamic ones where the knowledge base can constantly evolve during the use of the system. The evolving nature comes from the fact that the system must be able to integrate at any moment a new class (gesture in our context), and must also continue its adaptation to the existing classes using the new available data. Although the dynamic nature of evolving classifiers offers many important advantages, the operation of these systems suffers from the lack of learning data. The training process is done directly by the final user in an online and interactive manner, so that the quantity of teaching samples is limited because it is impractical to ask the user to enter a large number of samples in order to obtain a functional classifier. Therefore, the main challenge in the conception of incremental learning algorithms of evolving classification systems consists in reaching high recognition performance as fast as possible; i.e. with the minimum number of samples. Besides the beginning of the incremental learning from scratch, the problem of lack of data samples appears again during the adaptation process when new classes are added to the classifier. The evolving system is supposed to be able to learn these new classes without forgetting the old ones. However, it is difficult to completely avoid perturbations on the global performance of the classifier when adding new classes and the efforts must be focused on reducing as much as possible these perturbations.

In addition to the efforts of improving the classification systems and the training algorithms, the incremental learning process can be further accelerated and enhanced by generating artificial data based on some knowledge related to the application domain. For handwritten gesture recognition problems, this idea can be implemented by generating synthetic gestures from the available real ones after applying on them some deformations in a realistic and significant manner. Thus, when a new class of gestures is introduced to the system with few samples provided by the user, many artificial samples can be generated. Geometric distortions are usually applied on real handwritten symbols in order to generate synthetic ones [START_REF] Mitoma | Online character recognition based on elastic matching and quadratic discrimination[END_REF][START_REF] Wang | Combining shape and physical models for online cursive handwriting synthesis[END_REF][START_REF] Lin | Style-preserving English handwriting synthesis[END_REF][START_REF] Mouchère | Synthetic on-line handwriting generation by distortions and analogy[END_REF].

These deformations can be either based on class-dependent models of gesture variability and require a learning phase, or on class-independent general strategies without specific deformation models.

In this work, we incorporate a handwriting generation technique using class-independent lognormal-based deformations in the incremental learning of evolving handwritten gesture classifiers. Thanks to the RX0 based extractor, the ΣΛ parameter extraction and the data generation is performed automatically as a part of the adaptation process. Motion pattern variability rooted in the motor representation space of the handwritten gestures is regarded to be more realistic than geometric distortions and thus more valuable in the training process. In addition to the great advantage of integrating the lognormal-based handwriting generation technique in our evolving handwriting classifier, an objective and numerical evaluation of the quality of generated data is provided for the first time, to the best of our knowledge. The generated handwritten samples are considered realistic as much as they help the classifier to predict future real samples from the same class of gestures. The capacity of prediction is translated by the improvement of recognition performance of the evolving classifier.

Acceleration of the learning process using synthetic data

As aforementioned, we believe that distortions obtained by applying some variations on lognormal parameters are more realistic that those obtained using direct geometrical distortions. The idea is to extract the sigma-lognormal profiles of a real handwritten gesture provided by the user. Then, we apply some variation on the extracted parameters within some specific ranges, and we regenerate artificial gestures using the modified profiles. The resemblance between the synthetic and the real gestures is controlled by the variation intervals. Thus, a suitable setting of these intervals is required in order to avoid over-deformed gestures. We show in Fig. 7 some examples of the artificial gestures that can be generated by applying modifications on the sigma-lognormal parameters. We can note that the real gestures can be almost predicted from the synthetic ones. In the context of incremental learning of evolving systems, one can overcome the problem of lack of samples at the beginning of the inclusion of a new class by generating, in an adequate manner, a number of synthetic samples. For an evolving handwriting classifier, the abovementioned sigma-lognormal based technique for synthetic gesture generation can be incorporated into the incremental training process. The handwriting generation is automatically performed transparently, with no user intervention. Fig. 8 shows the different steps of the generation method. Frist, the sigma-lognormal parameters of each incoming sample are first extracted. These parameters are then modified and a number of synthetic samples are generated. The original sample and the synthetic ones are then introduced in the incremental learning algorithm.

Using SimScript (O'Reilly and Plamondon, 2007), a visualization interface developed by the Scribens laboratory that allows an interactive modification of the sigma-lognormal parameters of a given handwritten gesture, we have experimentally studied the valid variation intervals of the six parameters within which the generated gesture is generally considered similar (from a human viewpoint) to the original one.

Experimental results

Experiments have been performed on a dataset of on-line handwritten gestures. It was composed of 11 different gestures drawn by 7 writers on a Tablet PC. Each writer has drawn 100 samples of each gesture, i.e. 1,100 gestures in each writer specific dataset.

Each gesture was described by a set of 10 features. The presented results are the average of 7 different tests for the 7 writers. In order to avoid the data order inducing a bias into the outcome, we repeated 40 times the experiment for each writer with different random data orders and only the averages are reported. We have used about 40% of the dataset for the incremental training and the rest is used to estimate the evolution of the performance during the learning process. We generated 10 synthetic samples (gestures) for each real sample. The evolving classifier used in these experiments is based on a first- order Takagi-Sugeno (TS) fuzzy inference system, and taught with our original incremental learning approach "Evolve++" [START_REF] Almaksour | Improving premise structure in evolving Takagi-Sugeno neurofuzzy classifiers[END_REF].

We compared the lognormal-based handwriting generation method to the geometric distortions explained in [START_REF] Mouchère | Synthetic on-line handwriting generation by distortions and analogy[END_REF]. Therefore, three performance curves are presented in the figures: I. Evolve++: our evolving classification approach with Evolve++ algorithm presented in [START_REF] Almaksour | Improving premise structure in evolving Takagi-Sugeno neurofuzzy classifiers[END_REF]. Only real samples are considered (no synthetic data); II. Evolve++&Geo: synthetic samples are generated by applying geometric distortions and used along with real ones to train Evolve++ system; III. Evolve++&Sigma: synthetic samples are generated using the lognormal-based method.

The results are presented for two different experimental scenarios: the 11 gestures were introduced together in the first case, while the gestures were progressively introduced in the second one. We measure in the former the impact of the synthetic samples at the beginning of the learning process from scratch, while the latter scenario aims at showing the impact of these synthetic samples when introducing new gestures. The results of the first set-up are presented in Fig. 9.a and those of the second in Fig. As it can be seen in Fig. 9.a, there is an important impact of the synthetic gestures at the beginning of the incremental learning process. For example, the misclassification rate is reduced by 50% for 10 real samples per class when the training is enriched with synthetic samples. It must also be noted from the same figure that distortions applied on the sigmalognormal parameters produces a more realistic variability in the synthetic gestures as compared to direct geometrical distortions. Thanks to the realistic human-like distortions, the synthetic samples present a significant ability to predict the appearance of future real samples, which significantly accelerates the adaptation process. Fig. 9.b shows that using synthetic samples, the classifier resists much better when introducing new classes. It is able to rapidly re-estimate all its parameters and to improve the recognition performance for the old and the new gestures. Again, the superiority of lognormal-based deformations over the traditional geometrical ones is quite apparent.

These experimental results show that sigma-lognormal based synthetic samples play an important role in improving the classification performance and accelerating the learning process both when it starts from scratch and also when new gestures are introduced. One interest of the present methodology is also that it does not depend on the way reference gestures are defined and collected. For example, gestures that are recorded on hand held mobile devices while the user is standing up, writing vertically, or while he is sitting in a moving vehicle could also be used to train the system without really affecting the results.

Conclusion

In the last few years, great advances have been done on the problem of parameter extraction of the Sigma-Lognormal model. New algorithms have been designed which now allow an automatic representation of complex human motions such as those involved in on-line signature and handwriting. The availability of these new systems and a better understanding of the variability of the sigma-lognormal parameters have paved the way for the use of this model in the context of automatic generation of synthetic databases of human movements. This paper has summarized the promising results of two different investigations on that topic.

Needless to say that further research is still needed on the various topic addressed in this paper, but as can already be seen, the use of the Sigma-Lognormal model for the generation of human like movements offers very interesting perspectives for the field of pattern recognition and the development of verification and recognition systems based on human movements.

Fig. 1 .

 1 Fig. 1. Example of a signature reconstruction (SNR=22.2dB) following the proposed

Step 1 :

 1 Synthesis of the prototype. The initial prototype can be built from i) the results of the RX0 extractor in order to find what is the expected sigma-lognormal decomposition of the stereotypical movement or ii) from a sigma-lognormal synthesizer such as SimScript (O'Reilly and Plamondon, 2007). -Step 2: Movement scaling and offsetting. It is performed by finding the value of the scaling ( ) and the offsetting ( ) factors that maximize the reconstruction SNR between the experimental data and the prototype signals modified in such a way that the original μ i and t 0i parameters are changed for the scaled and shifted parameters μ ia and t 0ia according to (15)-(16).

-

  Step 3: Non-linear optimization. It can be performed according to any suitable optimization algorithm. For our work, we used a custom implementation of a direct search optimization (O'Reilly, 2012) which monotonically increases the SNR without risk of divergence or of finding solutions that are too far away from the original prototypes. This preserves the psychophysical correspondence of the lognormal components among movements. Using this algorithm on a database of 1440 triangular movements performed by 120 subjects (see (O'Reilly & Plamondon, 2011) for a more complete description of this dataset), we obtained a mean SNR of 20.8 dB for the prototype based extractor compared to a 22.1 dB for the RX0 based system. Although its SNR is slightly lower, the prototype based extractor has the clear advantage of producing solutions with fixed number of lognormals which enables the comparison among movements with any standard tool of statistical analysis of variance.

D

  1-a time variability associated with the temporal information contained in the activation sequence of the different commands, 2-a spatial variability associated with the geometrical information contained in commands themselves (the magnitude i D and direction ( neuromuscular variability reflected in the timing properties ,ii  of the neuromuscular networks reacting to these commands.Those sources of variability have been investigated using a semi-automatic sigmalognormal parameter extraction methodology somewhat similar to the one described for the prototype based extraction. Especially, we have shown that the Sigma-Lognormal model, can explain the great variability of individual stroke trajectories and their corresponding velocity profiles(Plamondon and Djioua 2005, 2006). We have also applied this paradigm to study the possible sources of handwriting deformations caused both by the disruptions in the motor control and the neuromuscular systems(Djioua and Plamondon 2007, 2009). Particularly, we have shown that, without altering the rest of the factors involved in handwriting, the distortion of the shapes of a handwritten word is very sensitive to slight changes of the time plan, represented by the sequence of command time occurrences  0i t . This stresses out the fact that, to write a readable word or to generate a consistent signature, the production of strokes composing that word or signature must be planned in advance in order to keep almost constant the timing of the learned original plan. In contrast, the command parameters, that affect the direction ( of the strokes, seem to be less critical. And, finally, the changes induced by the neuromuscular timing parameters ,ii  seem to have an even smaller influence on the final variability of the trajectory.

Fig. 2 .

 2 Fig. 2. General architecture of the algorithm proposed for the generation of synthetic

Stage 2 :

 2 Fig. 3. General diagram of the generation process of a synthetic on-line master signature

  strokes following the Sigma-Lognormal model where each stroke is defined by the set of features p=[t 0 , D, µ, σ, θ s , θ e ]. The duplicated samples are then generated adding to each of the single strokes a certain amount of noise which is modeled by a vector w=[w t0 , w D , w µ , w σ , w θs , w θe ] where w t0 is extracted from a uniform distribution max max tt ww    which is estimated according to the intra-user variability found in the development database BiosecurID[START_REF] Fierrez | BiosecurID: A Multimodal Biometric Database[END_REF] (analogously for the rest of distortion elements comprised in the vector w). After the distortion stage, the new velocity function v n is computed, and in a subsequent step the new coordinates x n and y n are recovered from that velocity information (see Fig.4).

Fig. 4 .

 4 Fig. 4. General diagram of the generation process of duplicated samples starting from a

Fig. 6 .

 6 Fig. 6. Performance evaluation of systems A and B, on a real (MCYT, grey DET curve)

Fig. 7 .

 7 Fig. 7. Some examples of generated gestures using sigma-lognormal based deformations.

Fig. 8 .

 8 Fig. 8. Incorporating lognormal-based data generation into the learning stage of an evolving

  Fig. 9 (a) Performance improvement using synthetic data generation.

In this context, one concern that might be raised is dealing with the potential use of this methodology to fabricate forged signatures. This might be an issue except that for making realistic forgeries, the forger would need to have access to the complete on-line information about the target signature and if he does, there would be no need to use a complex methodology like the one described in this paper. The addition of some noise might be sufficient. However, the complete on-line information is generally not available in real life systems since it is generally not stored in the reference database.

Acknowledgements

This work was supported by NSERC grant RGPIN-915 to Réjean Plamondon and a NSERC PGS D scholarships to Christian O'Reilly. This work was also partially supported by projects Contexts (S2009/TIC-1485) from CAM and Bio-Challenge (TEC2009-11186) from Spanish MICINN.


Maurizio Gabbrielli

Saverio Giallorenzo

Fabrizio Montesi

Service-Oriented Architectures: From Design to Production Exploiting Workflow Patterns

In Service-Oriented Architectures (SOA), services are composed by coordinating their communications into a flow of interactions. Coloured Petri nets (CPN) offer a formal yet easy tool for modelling interactions in SOAs, however mapping abstract SOAs into executable ones requires a non-trivial and time-costly analysis. Here, we propose a methodology that maps CPN-modelled SOAs into Jolie SOAs (our target language), exploiting a collection of recurring control-flow patterns, called Workflow Patterns, as composable blocks of the translation. We validate our approach with a realistic use case. In addition, we pragmatically asses the expressiveness of Jolie wrt the considered WPs.

Introduction

Service-Oriented Computing (SOC) is a design methodology focused on the realisation of systems by composing autonomous entities called services. In a Service-Oriented Architecture [START_REF] Erl | Service-Oriented Architecture: Concepts[END_REF] (SOA), services are composed by coordinating their communications into a flow of interactions. Several tools have been presented [START_REF] Omg | Service oriented architecture Modeling Language[END_REF][START_REF]Reference architecture foundation for SOA version 1[END_REF][START_REF] Mayer | The UML4SOA Profile[END_REF] to assist the process of SOA design, each focusing on a particular aspect of the system, e.g., the architectural composition, the interaction among components, etc. Coloured Petri nets [START_REF] Jensen | Coloured Petri Nets -Modelling and Validation of Concurrent Systems[END_REF] (CPNs) are a formal yet intuitive graphical tool, largely employed in business process modelling [START_REF] Van Der Aalst | Workflow patterns[END_REF] and suitable for SOA specification. Although it is easy to understand the interactions of a CPN model, it is unclear which components form the system, which implement the described logic or whether it be spread among the components or centralised.

Therefore the aim of this work is to provide a methodology that allows the translation of CPN-modelled SOAs into executable ones.

The Workflow Patterns Initiative (WPI) studied and collected a comprehensive set of recurring patterns of process-aware information systems, dubbed Workflow Patterns [START_REF] Van Der Aalst | Workflow patterns[END_REF] (WP). In particular we remark the exhaustive set of patterns of interaction, dubbed Control-Flow Workflow Patterns [START_REF] Russell | Workflow control-flow patterns: A revised view[END_REF] 1 , modelled as CPNs.

Since CPNs are composable, our idea, depicted by the scheme in Fig. 1, is that an SOA, modelled as a CPN, can be described in terms of the Workflow Patterns it is made of. Once the SOA is defined by a composition of WPs, the developer only has to refer to the implementation of each WP to build the whole system. To realise our proposal, we provide the implementation of a substantial set of WPs. Such implementation is not immediate since WPs are abstract specifications and it is unclear how they map into executable code for service coordination. Moreover, although the same WP applies to different subnets of interactions, its implementation may differ sensibly depending on whether its logic is centralised in a single component or distributed among several ones. Centralised and distributed approaches suit different contexts. E.g., if a vendor wants to monitor its application he might prefer a single point of control to track the whole system. On the other hand, some scenarios strictly require a distributed approach, e.g., an interaction that comprehends different parties. In § 3 we consider a realistic use case that combines the two approaches. We translate both the centralised and distributed versions of WPs as composable and executable SOAs. In order to provide a consistent translation we also define a procedure in § 2. Notably, such procedure might directly map a CPN-modelled SOA to an executable one, thus skipping the said in-between translation to a WP-modelled SOA. However, the behaviour of some WPs needs ad-hoc solutions (see Table 1) not directly mapped by the presented procedure. Thus, although providing an automatic procedure is an interesting challenge, in this work we focus on the practical implications of enabling developers translate CPN-modelled SOAs into executable ones by referring to our collection of Workflow Patterns. Our procedure applies to any service-oriented language, e.g., BPEL [START_REF]BPEL[END_REF] but we choose to implement WPs in Jolie [9, 10] for two main reasons. First Jolie supports several communication and serialisation protocols, thus the same implementation applies to different application domains. Second Jolie is based on a formal process calculus [START_REF] Guidi | Sock: A calculus for service oriented computing[END_REF] which we plan to use to prove relevant correctness properties of translated SOAs. For reasons of presentation we delegate to the full paper [START_REF] Gabbrielli | Executable SOAs exploiting workflow patterns[END_REF] the analysis and the implementation of all considered WPs (basic and advanced branching and synchronisation patterns).

From Coloured Petri Nets to Jolie SOAs

Background. We assume the reader is familiar with the basic concepts of Petri nets on which Coloured Petri nets add expressions on arcs and types to tokens 2 . In order to understand our translation procedure we briefly describe the messagepassing operations in Jolie. The language supports two kinds of message-passing operations which send or receive the content of a variable v. Sending operations specify the location L the message is sent to. One-Way operations (OWs) send or receive a message and immediately pass the thread of control to the subsequent activity in the process. The syntax of a receiving OW is simply op name(v), whilst a sending OW adds the location of the request op name@L(v). Request-Response operations (RRs) either send a request and keep the thread of control until they receive a response or they receive a message, do some computation, and send a response. The syntax of a sending RR is op name@L(v)(v), whilst a receiving RR adds a pair of brackets that enclose the code run before sending the response to the invoker op name(v)(v){code}. Jolie provides also an input choice with syntax [η 1 ]{B 1 } . . . [η n ]{B n }. When an input operation η i is triggered it disables all {η 1 , . . . , η n }\n i and executes. Then B i executes.

Workflow Patterns in Jolie. In this section we present a procedure for translating a CPN-modelled Workflow Pattern into an executable Jolie SOA. The translation follows five principles: (i) transitions are services; (ii) places are message-passing operations (i.e., communications); (iii) communications carry typed messages, as coloured tokens do; (iv ) arcs are properties on communications: they express the type of carried messages and the conditions that fire the communication; (v ) a CPN models a WP composed by several services running in parallel. Following these principles, we translate CPN models of Workflow Patterns into Jolie SOAs as follows. We map input, internal, and output places into One-Way (OW) operations (principle ii). When it is compatible with the behaviour of the pattern, we coalesce two round-trip OW operations between two services into one Request-Response (RR) for brevity. Since in Jolie output operations define the service they communicate to, we map output places into OWs on default locations DefaultOutput1,.... This enables composition of the implemented patterns by binding their locations. As exposed in § 1 we translate both the centralised and the distributed versions of WPs. In the centralised approach a master service, called orchestrator encodes the whole behaviour of a WP coordinating the interactions among the services participating to the SOA. BPEL [START_REF]BPEL[END_REF] is the most known technology for this approach, called orchestration. By convention the orchestrator of a WP is the only service that receives and sends messages outside the SOA. The distributed approach recalls that of choreography languages like WS-CDL [START_REF] W3c | Web services choreography description language version 1.0[END_REF]. Recent works [START_REF] Carbone | Deadlock freedom by design: multiparty asynchronous global programming[END_REF][START_REF] Dalla Preda | Deadlock freedom by construction for distributed adaptative applications[END_REF] introduced automatic techniques to project executable services of an SOA from a choreographic specification. Following a similar approach in the distributed version of a WP we maintain a direct relation between transitions and services, imposing no restrictions on the scope of external input and output operations.

The Upload Service Use Case

This section shows how a realistic SOA, modelled as a Coloured Petri net, translates into an executable SOA. The goal of this section is twofold. (i) We show the benefits of CPNs as SOA design tool. CPNs let the designer model the system focusing on interactions, whilst the control on message flow is left to the developer. (ii) We exhibit how a developer can easily map a CPN into a composition of WPs. With just the flow of interactions as model, the developer can mix distributed and centralised implementations of patterns and build the system.

The use case describes a typical interaction between a User, a file upload Service Provider, and an Identity Provider. In particular the Identity Provider offers an OpenID-like [START_REF]Specifications[END_REF] authentication with a multi-factor mechanism [START_REF]Multi-factor authentication[END_REF] Fig. 3. The CPN model of the interactions in the Upload Service SOA whilst the Service Provider offers a multipart upload procedure. Fig. 3 depicts the CPN model of the use case. The two double-line bordered boxes in the figure act as placeholders for the two subnets related to the multi-factor authentication and the Multipart Upload procedure, the latter modelled in Fig. 4.

Let us identify the WPs3 that compose the CPN-modelled SOA. Interaction starts from the User that requests the service. This basic interaction is a distributed Sequence that passes the thread of control to the Service Provider. The Service Provider employs a distributed Parallel Split to start the multifactor authentication and redirects the authentication request to the User. The Identity Provider allows users to identify themselves with a multi-factor authentication. Let us suppose that the Identity Provider demands a minimum number of different authentication procedures. Such M-out-of-N mechanism maps to the centralised version of the Cancelling Partial Join pattern being completely controlled by the Identity Provider. For reasons of presentation we choose not to discuss about the implementation of this pattern as it maps directly and does not show interesting interactions with other patterns. For the model and the code relative to the multi-factor authentication refer to [START_REF] Gabbrielli | Executable SOAs exploiting workflow patterns[END_REF]. After the successful authentication, the thread of control passes back to the Service Provider with another distributed Sequence which notifies the User (s)he can proceed to upload the file. The User and the Service Provider enter the Multipart Upload interaction whose behaviour results from the composition of several patterns. Fig. 4 depicts such interactions and highlights the most relevant WPs. Fig. 5 depicts the architectural view of the translation following the same informal representation used in Fig. 2. The User-controlled part of the interaction mixes centralised and distributed WPs. Listing 1.2 reports the code relative to the services orchestrator and SendChunks at User's side. When the uploadRequest arrives (Line 1), the orchestrator requires the User to select a file, passing the thread of control as a centralised Sequence to service SelectFile (Line 2). At file selection, the thread of control returns to the orchestrator which passes it to service CreateChunks (Line 3). The service employs a centralised Thread Split (A) to split the file into n chunks. Then the orchestrator implements a centralised Thread Merge (B) to collect triplets of chunks and send them to service SendChunks (Lines 5-7). Notably, since the orchestrator passes the thread of control to the invoked service and waits for its response, we can coalesce the OW operations between them into one RequestResponse. SendChunks implements a distributed Parallel Split to forward each chunk in parallel to the Service Provider (Lines 11-13). At Service Provider's side the service StoreChunks employs a centralised Generalised AND-Join (C) to receive the chunks (Listing 1.3 Lines 1-13). When the nth chunk reaches the service, it passes the thread of control with a distributed Sequence to service ComposeFile which employs a centralised Thread Merge (D) to restore the chunks into a single file. Finally a distributed Sequence returns the thread of control to the User, notifying the success of the upload procedure.

Conclusions

Contributions of this work are: (i) the definition of a methodology for translating CPN-modelled SOAs into composable and executable ones and (ii) the creation of a collection of implemented Workflow Patterns (reported in [START_REF] Gabbrielli | Executable SOAs exploiting workflow patterns[END_REF]). Such implementations follow both a centralised and a distributed approach to allow developers the flexibility to choose and mix them. A realistic use case proves our claim that the patterns obtained in this way can be used for building real SOAs starting from abstract specifications. In addition, (iii) our work also allows us to provide a pragmatic assessment on the expressiveness of the Jolie language. Table 1 summarizes the results of such an assessment. For each pattern, we indicate in the second column the kind of support offered by Jolie: "+" means direct support, i.e., the implementation of the pattern either uses some specific primitives provided by the language or is a composition of directly supported patterns. "+/-" indicates a "non direct" support, i.e., the translation of the CPN of the pattern does not completely follow the rules described in § 2 although it complies with the general structure of the pattern. In the third column of Table 1 we indicate the specific Jolie primitive and/or the other patterns used to implement a given pattern. Note that we report both the centralised and distributed implementations which, as expected, in some cases vary. As shown in Table 1 we can conclude that Jolie allows to implement most WPs. Related Work. A close concept to Workflow Pattern is that of service interaction pattern, introduced in [START_REF] Barros | Service interaction patterns[END_REF]. Service interaction patterns define recurring interaction patterns among services but, differently from Workflow Patterns, they are informally specified and therefore not employable in this work. Variants of Petri nets have been used for system modelling [START_REF] Mendes | Composition of petri nets models in service-oriented industrial automation[END_REF] and static analysis [START_REF] Lohmann | Analyzing bpel4chor: verification and participant synthesis[END_REF]. Finally WPI used WPs as a tool to evaluate the expressive power of business process languages, in particular for the cases of BPEL [START_REF] Wohed | Analysis of web services composition languages: The case of bpel4ws[END_REF] and BPML [START_REF] Van Der Aalst | Patternbased analysis of BPML (and WSCI)[END_REF].

Future Work. We plan to provide a formal definition of our technique for translating CPNs into Jolie code. Such a formalisation would enable to mechanically translate CPN-modelled SOAs into executable ones, also applying known methodologies of static analysis to assess properties of SOAs implemented in Jolie. We also plan to investigate the remaining patterns described by the WPI and to use the implemented WP to offer pattern composition as APIs [START_REF] Guidi | Towards a composition-based APIaaS layer[END_REF].

Fig. 1 .

 1 Fig. 1. The scheme of translation from abstract to executable SOAs

Fig. 2

 2 reports an example of a CPN in box A and informally depicts the architectural view of the translated centralised (B) and distributed (C) implementations. Listing 1.1 reports the corresponding code of, respectively, the orchestrator of the centralised version and of the services in the distributed one. In B the orchestrator (O) receives the input message as a OW operation i1 (Line 1). O sends via RR the content c to service A which returns the condition cond1 (Line 2). Based on cond1, O either redirects a request to B or C (Lines 4-8). O sends c to D, which returns its response. (Line 9) Finally O sends c as output to the DefaultOutput1.(Line 10). Notably, O uses RRs to invoke operations on the services its composes, waiting for their responses. C maintains a direct relation between transitions and services which pass the thread of control using OW operations. Service A receives the input message i1 (Line 1), evaluates condition cond1 and redirects c to either service B (Line 3) or C (Line 4) which, in turn, forwards c to service D. Finally D receives c on operation p3 and sends it to the output location DefaultOutput1 (Line 10).

Fig. 2 . 1 . 1 . 4 Ð

 2114 Fig. 2. (A) a CPN model, its centralised (B), and distributed (C) realisations

Fig. 4 .Fig. 5 .

 45 Fig. 4. Multipart Upload modelled as composition of Workflow Patterns

Table 1 .

 1 Support of basic and advanced branching and synchronisation WPs in Jolie

Here referred as Workflow Patterns for simplicity. S. Omatu et al. (eds.), Distributed Computing and Artificial Intelligence,

11th International Conference, Advances in Intelligent Systems and Computing 290, DOI: 10.1007/978-3-319-07593-8_17, c Springer International Publishing Switzerland 2014

[START_REF] Gabbrielli | Executable SOAs exploiting workflow patterns[END_REF] reports a comprehensive introduction on Coloured Petri nets.

[START_REF] Gabbrielli | Executable SOAs exploiting workflow patterns[END_REF] reports the description of the considered patterns and their implementations.


Kavya Anjani

Marie Lhomme

Nataliya Sokolovska

Christine Poitou

Judith Aron-Wisnewsky

Jean-Luc Bouillot

Philippe Lesnik

Pierre Bedossa

Anatol Kontush

Karine Clement

Judith Aron- Wisnewsky

Isabelle Dugail
email: isabelle.dugail@inserm.fr

Joan Tordjman
email: joan.tordjman@crc.jussieu.fr

Circulating phospholipid profiling identifies portal contribution to NASH signature in obesity

Keywords: NAFLD, non alcoholic fatty liver disease, NASH, nonalcoholic steatohepatitis, LC-MS, liquid chromatography coupled mass spectrometry, PL, glycerophospholipids, SL, sphingolipids, PC, glycerophosphatidylcholines, LPC, lyso-glycerophosphatidylcholines, PE, glycerophosphatidylethanolamines, PI, glycerophosphatidylinositols Lipidomics, Visceral Adipose tissue, Glycerophosphatidylglycerols, Glycerophosphatidylethanolamines, bariatric surgery

Introduction

Non-alcoholic fatty liver disease (NAFLD) comprises steatosis to nonalcoholic steatohepatitis (NASH). Insulin resistance and obesity are some of the major risk factors for NAFLD [START_REF] Ruhl | Determinants of the association of overweight with elevated serum alanine aminotransferase activity in the United States[END_REF] along with age [START_REF] Targher | Prevalence of nonalcoholic fatty liver disease and its association with cardiovascular disease among type 2 diabetic patients[END_REF], gender [START_REF] Arun | The prevalence of nonalcoholic steatohepatitis is greater in morbidly obese men compared to women[END_REF], ethnicity [START_REF] Mohanty | Influence of ethnicity on histological differences in non-alcoholic fatty liver disease[END_REF] [START_REF] Bambha | Ethnicity and nonalcoholic fatty liver disease[END_REF] and genetic variants such as patatin-like phospholipase domain-containing 3 (PNPLA3) [START_REF] Romeo | Genetic variation in PNPLA3 confers susceptibility to nonalcoholic fatty liver disease[END_REF]. Incidence of NAFLD in the general population is estimated in the range of 6 -14% whereas in specific population such as in morbid obesity it is reported to be as high as 84 -96%, comprising 25 -55% with NASH and 34 -47% with liver fibrosis [START_REF] Clark | The epidemiology of nonalcoholic fatty liver disease in adults[END_REF].

NAFLD is associated with elevated cardiovascular risk [START_REF] Bhatia | Non-alcoholic fatty liver disease: a new and important cardiovascular risk factor?[END_REF][9] [START_REF] Verrijken | Prothrombotic factors in histologically proven NAFLD and NASH[END_REF] with increased fat mass as an important mediator. We showed that accumulation of inflammatory cells in visceral and not subcutaneous adipose tissue is associated with severity of NAFLD in morbid obesity [START_REF] Tordjman | Structural and inflammatory heterogeneity in subcutaneous adipose tissue: relation with liver histopathology in morbid obesity[END_REF] [START_REF] Cancello | Increased infiltration of macrophages in omental adipose tissue is associated with marked hepatic lesions in morbid human obesity[END_REF].

Extent of hepatic inflammation and fibrosis also associated with visceral fat independent of insulin resistance and hepatic steatosis [START_REF] Van Der Poorten | Visceral fat: a key mediator of steatohepatitis in metabolic liver disease[END_REF][14] [START_REF] Eguchi | The pathological role of visceral fat accumulation in steatosis, inflammation, and progression of nonalcoholic fatty liver disease[END_REF]. These findings indicate potential contribution of visceral fat to worsening liver condition through release of bioactive molecules such as cytokines into hepatic portal system draining into liver, thus supporting the portal hypothesis. Increased liver load with molecules including free fatty acids (FFA) [START_REF] Bjorntorp | Portal" adipose tissue as a generator of risk factors for cardiovascular disease and diabetes[END_REF] [START_REF] Bergman | Free fatty acids and pathogenesis of type 2 diabetes mellitus[END_REF] and inflammatory mediators [START_REF] Rytka | The Portal Theory Supported by Venous Drainage -Selective Fat Transplantation[END_REF] from visceral fat were shown to contribute to altered hepatic function [START_REF] Hotamisligil | Mechanisms of TNF-alpha-induced insulin resistance[END_REF] [START_REF] Steppan | The hormone resistin links obesity to diabetes[END_REF][21] [START_REF] Kabir | Molecular evidence supporting the portal theory: a causative link between visceral adiposity and hepatic insulin resistance[END_REF] [START_REF] Jensen | Role of body fat distribution and the metabolic complications of obesity[END_REF].

Lipids are obvious players in NAFLD. Mass spectrometry based lipidomic approaches offer detailed analysis of disease-associated changes in lipid profiles. Early studies reported perturbations in hepatic lipid classes during progression from normal to fatty liver and NASH, accumulation of triacylglycerols (TAG), diacylglycerols (DAG), free cholesterols, elevated Lyso-Glycerophosphocholines (LPC) and a shift towards more saturated fatty acids [START_REF] Puri | A lipidomic analysis of nonalcoholic fatty liver disease[END_REF]. Gorden 6 et al. [START_REF] Gorden | Increased diacylglycerols characterize hepatic lipid changes in progression of human nonalcoholic fatty liver disease; comparison to a murine model[END_REF] also observed increased DAG in steatotic compared to normal liver, but reported mostly unchanged; Glycerophosphocholines (PC), Glycerophosphoethanolamines (PE) and Glycerophosphoinositols (PI). Despite a potential for pathophysiological and non-invasive diagnostic information, analysis of circulatory lipidome in NAFLD is limited to date. Puri et al. found decreased plasma essential polyunsaturated fatty acids (PUFAs), increased (MUFAs) and lipoxygenase metabolites in NASH subjects [START_REF] Puri | The plasma lipidomic signature of nonalcoholic steatohepatitis[END_REF]. Recently, variation in circulatory TAG profiles served as a basis to define phenotypic heterogeneity in NASH i.e. PNPLA3 or obesity-associated NASH [START_REF] Hyysalo | Circulating Triacylglycerol Signatures in Nonalcoholic Fatty Liver Disease Associated With the I148M Variant in PNPLA3 and With Obesity[END_REF].

Though comprehensive analysis of all lipids still remain a challenge, we focused on Phospholipids (PL) and Sphingolipids (SL), which represent nearly half of the plasma lipidome in humans [START_REF] Quehenberger | Lipidomics reveals a remarkable diversity of lipids in human plasma[END_REF]. We performed MS-based lipidomic analysis on systemic and hepatic portal circulation simultaneously to characterize obese NASH subjects and examine portal lipid input to liver. Contribution of visceral adipose tissue to NASH was also investigated, as well as improvement in NASH associated lipid alterations following gastric bypass-induced weight loss.

Advanced statistical machine learning method was used for integration of lipidomic and clinical data to define a NASH signature in morbid obesity, visualized as a Bayesian network.

Patients and methods

Study cohort and sample collection

Forty-six morbidly obese women, candidates for gastric bypass surgery were recruited at Pitié-Salpêtrière Hospital (ICAN, Institute of Cardiometabolism and Nutrition). All subjects gave written informed consent. Local ethics committee approved the present clinical investigations.

Exclusion criteria include autoimmune, inflammatory or infectious diseases, viral hepatitis, cancer, known alcohol consumption (>20 g/day) or kidney diseases. Among patients with NASH, 19 were diabetics and treated with: Metformin (n=13), Insulin (n=1), Sulfamide (n=1), incretinomimetic drug (i.e. GLP1 analogs, n=2) and 2 without treatment. Sera were collected from systemic blood following an over-night fast. Blood from right epiploic vein connected to portal trunk, therefore termed "portal blood" was collected during surgery. Biopsies of omental, subcutaneous and mesenteric adipose tissues and liver were obtained from all subjects during surgery and stored at -80°C or fixed until further analyses. Clinical characteristics of the cohort are summarized in Table 1.

Biochemical analyses

Serum glucose, triglycerides, total and HDL-cholesterol, alanine aminotransferase (ALT), aspartate aminotransferase (AST), gamma-glutamyl transpeptidase (GGT) and fibrinogen were measured. Serum insulin concentrations were determined by Bi-INSULIN IRMA (CisBio International, Gif-sur-Yvette, France), leptin and adiponectin were assayed by radioimmunoassay (Linco Research, Saint Louis, MI, USA), interleukin-6 (IL-6) was measured by ELISA (QuantikineUS, R&D System Europe Ltd, Abingdon, UK), high-sensitive C-reactive protein (hsCRP) was measured using an IMMAGE automatic immunoassay system (Beckman-Coulter, Fullerton, CA, USA).

Liver histopathology

The liver biopsy was formalin-fixed, paraffin-embedded and serial sections were stained.

Minimal staining included Haematoxylin and Eosin, PicroSirius Red and Perls. All biopsies were reviewed by a single liver pathologist and graded on SAF score, absence or presence of NASH was evaluated according to diagnostic algorithm for NASH [START_REF] Bedossa | Histopathological algorithm and scoring system for evaluation of liver lesions in morbidly obese patients[END_REF]. No NASH subjects had no histological signs of steatosis, lobular and portal inflammation and fibrosis.

Lipidomics

Lipidomic measurements were based on the method by Camont et al. [START_REF] Camont | Small, dense high-density lipoprotein-3 particles are enriched in negatively charged phospholipids: relevance to cellular cholesterol efflux, antioxidative, antithrombotic, antiinflammatory, and antiapoptotic functionalities[END_REF]. Briefly, lipids were extracted by cold acidified methanol: chloroform with internal standard for each lipid class. All standards were purchased from Avanti Polar Lipids (Alabaster, AL, USA). Eight PL subclasses; PC, LPC, PE, PI, PG, PA, Lyso-glycerophosphoethanolamines (LPE), Glycerophosphoserines (PS) and two SL subclasses Sphingomyelins (SM) and Ceramides (Cer) comprising 158 species were assayed and quantified by LC/MS/MS (Shimadzu, Kyoto, Japan, and AB Sciex, Framingham, MA, USA).

Bayesian network

We employed computational biology for combined analysis of clinical and lipidomic data. Supervised machine learning using sqsvm (Squared Hindge Loss Classification) algorithm from gglasso package in R environment was used. Optimal computational model based on minimal 10-cross folds validation error rate was chosen. Interdependencies between clinical and lipidomic data was revealed by learning a Bayesian network using the R package bnlearn, where nodes represent variables of interest and edges between the nodes represent relationships between them. The resulting network was visualized using Cytoscape, open source software.

Statistical analysis

All statistical analyses were performed in R (http://www.r-project.org/). Data are expressed as mean ± SEM. In most analysis, significant differences between groups were estimated using unpaired Wilcoxon rank sum test while paired Wilcoxon rank sum test was performed to compare same group at different time points. For age adjustment, we used linear models after log transformation of the variables and have used the function lm() for fitting linear models of R package "stats" (R version 3.0.1 ). A p-value of less than 0.05 was considered to be statistically significant.

Results

Characterization of morbidly obese women with NASH

Morbidly obese women (BMI matched) were grouped as NASH or no NASH based on histological assessment of liver biopsies. NASH subjects were significantly older than no NASH with 68% being type 2 diabetic with higher fasting glucose and HbA1c. NASH subjects had higher levels of AST, ALT, GGT, fasting triglycerides, serum IL-6 and lower fat mass and adiponectin (Table 1).

Change in circulatory phospholipid profile in obese women with NASH

Mass spectrometry-based approach was used to profile PL and SL species in serum obtained from patients' systemic blood under fasting condition. A total of 158 lipid species were identified and quantified from PC, LPC, PE, LPE, PI, PS, PG, PA, SM and Cer. Lipidomic data projected onto the plane depicting maximum inertia (total variance) of the dataset i.e., the first two principal components explain 45% of variance in the data and was able to distinguish between NASH and no NASH subjects (Fig. 1A). Type 2 diabetic and non-diabetic subjects discriminated to a lesser extent (supp Fig. 1). Significant NASH-associated lipid changes were visualized as heat map presenting ratio of each lipid concentration in individual NASH subject vs. mean of no NASH subjects, and revealed a large number of altered lipids in NASH subjects (Fig. 1B). Sum of concentrations of individual lipid species within same class showed significant increase in PC, PE, PI, PG and Cer (Fig. 1C). After adjustment for age, which differed between the two groups, PI and Cer were not found altered. LPC was not found elevated in the NASH group but positively correlated with age and negatively with obesity duration (Suppl Fig 2 ). A characteristic feature of circulating lipidome is the wide concentration range over three log amplitude between different lipid classes. For example, most abundant PC and SM circulate within a 50-100 mg/dL range, whereas PE and LPC are found at 10 fold lower concentration, followed by PI, Cer (1-2 mg/dL) and PS, PG, PA with concentrations below 0.1 mg/dL (Fig. 1C). Individual species within a class also vary in concentration range, those bearing the common C16 and C18 fatty acids being the most abundant. Therefore, to evaluate the contribution of individual lipid species we plotted their relative abundance against their ratio NASH/no NASH concentration. This revealed that NASH associates with modifications of not only high but also of several low abundant species (Fig. 1D).

Gastric bypass results in partial reversal of phospholipid alteration in obese women with NASH

Gastric bypass surgery-induced weight loss associated with improvements in BMI, fat mass, fasting glucose, insulin, HbA1c, triglycerides, HDL-C and hsCRP (Table 1). NASH subjects showed significant decrease in serum levels of liver enzymes 12 months post surgery that reached values similar to those of no NASH subjects, suggesting amelioration of liver condition (Fig. 2A). Despite this improvement, we cannot conclude on liver rescue in this population as there is no biopsy proven diagnosis of NASH post surgery. Therefore, we retained the initial grouping of subjects based on their baseline biopsy-proven diagnosis for lipid profiling. Post surgery lipidomic data indicate that several lipids, mainly PG and Cer, remain significantly elevated in systemic circulation of NASH subjects (Fig. 2B andC) in addition to a few intermediate abundant PC and PE (Fig. 2D). Overall lipid profile before and after surgery visualized as volcano plots of log mean ratio NASH/no NASH against p-value (ratio over indicates increase in NASH), shows much fewer alteration after surgery. Twelve months following gastric bypass surgery was associated with lower number of affected lipids in NASH subjects compared to baseline (Fig. 2E). Detailed analysis showed that PI was mostly resolved while PC and PE were partially resolved but PG and Cer remained elevated in NASH subjects months after gastric bypass surgery (Fig. 2F).

Partial contribution of adipose tissue depots to NASH-associated phospholipid alterations

We investigated potential contribution of adipose tissue to NASH-associated alterations in lipid profiles by estimating ex vivo lipid efflux. For this purpose calibrated omental, mesenteric and subcutaneous adipose tissue biopsies from NASH and no NASH subjects obtained at the time of surgery were incubated for 24 hours in media containing BSA as a lipid acceptor. PL and SL efflux into the medium was quantified and individual lipids were expressed as mol%.

Interestingly, we were able to detect all PL (identified in serum) under these conditions of ex vivo adipose tissue efflux. Compared to serum, the distribution of lipids effluxed from three adipose tissue depots mainly differed in LPC: PC ratio and higher efflux of PG and PA (Supp Fig. 3A). Comparison of profiles of adipose tissue lipid efflux from NASH and no NASH subjects showed very few alterations. Only few species, not overlapping in the three adipose tissue depots were less efficiently effluxed from adipose tissue depots of NASH subjects (Supp Fig. 3B). Remarkably, PG (36:1) from omental and subcutaneous and PG (34:1) from omental adipose tissue were among the lipids effluxed at significantly higher rate from adipose tissue of NASH subjects (Supp Fig. 3C). Overall, these data suggest weak contribution of adipose tissue towards circulating PL alterations in obese women with NASH.

Lipid alterations in the hepatic portal system and association with liver pathology in morbid obesity

We next investigated lipid profiles from the hepatic portal system, known to transport molecules from visceral organs to liver. Portal blood was collected from right epiploic vein during gastric bypass surgery and plasma was used for lipidomic analysis. Compared to systemic circulation, fewer alterations were observed in portal phospholipidome of obese women with NASH (Fig. 3A vs. Fig. 2E baseline). Visualization of heat map indicates lipid alterations in PC, PE, PI and PG in hepatic portal circulation of NASH subjects (Fig. 3B) with major alterations in PE and PG classes (Fig. 3C). The most abundant PE(36:2) along with 6 other low abundant species and 2 high and 2 low abundant PG species were significantly increased in NASH subjects (supplementary Fig. 4). As a lipid class, PC and PI did not appear altered (Fig. 3C

NASH signature in morbid obesity

Since bioclinical and lipidomic data contribute to NASH characterization, we learnt a Bayesian network to identify the most relevant associations with NASH. The network was constructed from baseline values from 15 bioclinical variables including mesenteric and omental adipocyte size and 61 and 24 lipidomic variables from systemic and portal circulation respectively. The nodes of the network were pre-selected by a sparse support vector machine algorithm. Directly interacting nodes with NASH, considered Markov blanket of NASH were bioclinical variables; age, fasting glycemia, serum adiponectin and fibrinogen, tissue phenotypes; mesenteric and omental fat cell size (FCS) and lipid species; systemic PC(40:4), portal pPC(40:4), PI(40:5) and portal pCer(d18:2/14:0). These variables thus influenced or were influenced directly by NASH.

Width of lines in the network are proportional to the weight of dependencies and suggest that relationships between NASH and PC(40:4) (systemic and portal) and omental FCS are stronger than with other nodes. The influence of age on NASH status also appeared as an important factor. The Bayesian network also revealed strong dependencies between the portal lipids pPC(40:4), pSM(42:3) and pCer(d18:2/14:0) and between systemic PG(34:1) and PI (40:5). As expected fasting glycemia, directly linked to NASH has a strong interaction with HbA1c and ALT (Fig. 4A). Altogether these data highlight that circulating (systemic and also portal blood) phospholipids interact with clinical and tissue parameters to define NASH signature in morbid obesity. Because portal blood lipids is not available in routine patient investigation, we tested for added value of lipid profiling systemic blood only in NASH prediction. Using variables selected from machine learning algorithms, ROC curves indicated that selected lipids alone had a poor predictive value over clinical variables, but combination of selected lipids and biological parameters slightly improved NASH prediction (Fig 4B ).

Discussion

Liver disease is commonly associated with obesity, but not all obese subjects develop NASH.

This study explored morbidly obese subjects with or without liver alterations in a combined analysis of not only systemic but also portal lipids. A main outcome is that, in this context of morbid obesity with patients stratified for NASH, the composition of PL in the portal circulation differs, indicating selective alterations in lipid input to the liver. Nevertheless, fewer lipid species are impacted in portal circulation compared to massive changes that dominate systemic blood, reported in this and previous studies [START_REF] Puri | The plasma lipidomic signature of nonalcoholic steatohepatitis[END_REF].

Several circulating protein markers have previously been defined in NASH and associated insulin-resistance such as cytokeratin 18 or Fetuin A [START_REF] Anty | A new composite model including metabolic syndrome, alanine aminotransferase and cytokeratin-18 for the diagnosis of non-alcoholic steatohepatitis in morbidly obese patients[END_REF] [START_REF] Stefan | Circulating fetuin-A and free fatty acids interact to predict insulin resistance in humans[END_REF]. Focusing on lipids, we identified 4 PL significantly contributing to the NASH signature. Therefore, these lipids represent putative candidates in NASH-associated liver deterioration. In particular, we have identified NASHrelated changes in several prominent PG species indicating that phosphatidylglycerols might have physiopathological relevance to NASH. The integration of lipidomic and clinical variables to construct a Bayesian network identified 15 most significant contributors and also revealed their interdependencies. Among those, there were 7 lipid species (4 from portal and 3 from systemic circulation) and 8 clinical features. This approach demonstrates the interest in combining usual clinical variables and large-scale measurements to identify complex NASH signatures and potentially provides new basis for hypothesis in NASH development in morbid obesity.

We confirmed the importance of fasting glycemia, HbA1c and adiponectinemia, linked to glucose homeostasis and NASH [START_REF] Gupte | Nonalcoholic steatohepatitis in type 2 diabetes mellitus[END_REF] [START_REF] Ma | Independent association of HbA1c and nonalcoholic fatty liver disease in an elderly Chinese population[END_REF]. Insulin resistance is common feature in NASH [START_REF] Musso | Nonalcoholic steatohepatitis versus steatosis: adipose tissue insulin resistance and dysfunctional response to fat ingestion predict liver injury and altered glucose and lipoprotein metabolism[END_REF] [START_REF] Boza | Predictors of nonalcoholic steatohepatitis (NASH) in obese patients undergoing gastric bypass[END_REF]. Nevertheless, in the present study fasting insulin levels are equally elevated in NASH and no NASH subjects. This may be due to specific features, particularly the high proportion of NASH patients with diabetes (68%) in our cohort, especially the high proportion of patient receiving anti-diabetic insulin treatment (and mainly metformin). This particularity, in addition to the well-known insulin-resistance of morbidly obese patients might explain why LPCs, a lipid class associated with insulin sensitivity, does not show up in our study. Despite normalization of liver enzymes after gastric bypass surgery [START_REF] Mathurin | Prospective study of the long-term effects of bariatric surgery on liver injury in patients without advanced disease[END_REF], we showed that weight loss can only partially attenuate NASH-related alterations in systemic PL profiles at least one year after surgery.

Persistence of some lipid alterations after surgery could be linked to unresolved glycemic status (only 4 among 17 subjects resolved their type 2 diabetes following gastric bypass surgery). It remains to explore these patients after longer-term follow-up to examine whether the targeted lipid species further change or stabilize in parallel to the evolution in clinical parameters. The older age of NASH subjects could also affect liver outcome but we could not find age-matched subjects in the two clinical groups. However adjustment for age of systemic lipid concentrations confirms significant PL alterations (i.e.PE, PG, PC) in NASH patients.

Regarding lipid alterations, the Bayesian network revealed a surprisingly high dependency of NASH to portal PC(40:4), a low abundant species with potential role as a signaling molecule that may warrant further investigation. Also, SM(42:3) and a short chain (C14) Ceramide are the only SL with intriguingly decreased concentrations in portal circulation of NASH subjects and showed strong links with NASH but the biological significance is yet to be evaluated.

Most interestingly, about half of lipid species associated with NASH showed altered concentrations in the portal circulation. We initially hypothesized that in obesity, both omental 16 and mesenteric fat tissues drained by portal blood could be determinant NASH contributors by releasing lipids. However our results may rather highlight the importance of other visceral organs, than visceral fat. Indeed our ex vivo observations revealed limited changes in adipose tissue lipid efflux from NASH and no NASH subjects. Moreover, in the Bayesian network, none of the lipid nodes showed strong dependencies with omental fat cell size, whereas this tissue phenotype per se appeared as a strong contributor in NASH signature. This suggests that omental fat involvement is through fatty acid sequestration rather than release of specific lipid species.

On the other, association of adipose fat cell size with NASH in the network could reflect increased portal release of inflammatory mediators by hypertrophied fat cells that need to be further investigated. In addition, dysregulated hepatokines could also participate as reported recently by Stefan et al. [START_REF] Stefan | The role of hepatokines in metabolism[END_REF]. PE and PG are the two major lipid classes elevated in portal circulation of NASH subjects, and two species (36:2 and 34:1) with high abundance among PG both contribute to NASH signature.

Remarkably, PG and PE are mostly represented in bacterial membranes as opposed to eukaryotic cells predominantly composed of PC. Besides a role for gut microbiota dysbiosis in obesity [START_REF] Ley | Obesity alters gut microbial ecology[END_REF] and the importance of microbial diversity in individual responses of obese subjects to nutritional intervention [START_REF] Cotillard | Dietary intervention impact on gut microbial gene richness[END_REF], changes in gut microbiome of children with or without NASH [START_REF] Zhu | Characterization of gut microbiomes in nonalcoholic steatohepatitis (NASH) patients: a connection between endogenous alcohol and NASH[END_REF] have been reported. Therefore it is possible that elevated portal PG and PE in NASH could be linked to altered gut microbiota and, represent circulating microbial products possibly derived from bacterial translocation. Since PE and PG are elevated not only in portal but also in systemic circulation of obese NASH subjects, we propose the concept of gut microbiota-related hyperphospholipidemia as a potential new actor in NAFLD.

In conclusion, we showed that specific PL alteration in hepatic portal circulation, also observed in systemic circulation is characteristic of NASH in morbid obesity. Since portal lipid deregulation is weakly contributed by visceral adipose tissue, this suggests a possible contribution from gut microbiome.

  ) even if few species such as PC(36:1, 38:5, 40:4, 40:5) and PI(32:0, 32:1, 34:2, 36:4, 40:4, 40:5) were significantly increased in NASH (supplementary Fig. 4). Cer that was altered systemically was unaltered in the portal circulation of NASH subjects (Fig. 1B and 3B). Low abundant species predominantly from PG and PE and few PC and PI dominated NASH associated lipid alteration in hepatic portal circulation (Fig 3D).

Fig. 2 .Fig. 3 .

 23 Fig. 2. Gastric bypass surgery associated serum lipid alteration (A) AST, ALT and GGT

Fig. 4 .

 4 Fig. 4. NASH signature based on lipidomic and clinical data. A: Interdependencies among

Table 1 No NASH (n = 24) NASH (n = 22)

 1 no NASH vs. NASH at baseline (non parametric unpaired Wilcoxon test) in women candidate for bariatric surgery. $ no NASH vs. NASH at 12 months post surgery (non parametric unpaired Wilcoxon test). * Baseline vs.

		Baseline	12 months	Baseline	12 months
	Age (years)	36.42 ± 2.4		48.55 ± 2.5 ##
	BMI (kg/m 2 )	46.93 ± 1.0	31.61 ± 1.2**** 45.06 ± 1.2	33.75 ± 1.3****
	Percent Fat mass (%)	50.32 ± 0.7	38.70 ± 1.3**** 46.81 ± 0.8 ## 40.96 ± 1.1****
	Fasting Insulin (µIU/ml)	17.21 ± 2.0	8.18 ± 1.1**	25.91 ± 3.6	12.72 ± 4.1**
	Fasting glucose (mM)	4.93 ± 0.1	4.55 ± 0.1***	6.88 ± 0.4 #### 5.63 ± 0.3*** $$
	HbA1c (%)	5.58 ± 0.1	5.25 ± 0.1****	6.91 ± 0.3 ### 5.84 ± 0.2**** $$
	Triglycerides (mM)	1.02 ± 0.1	0.83 ± 0.1**	1.89 ± 0.2 ### 1.24 ± 0.1** $$
	Cholesterol (mM)	4.67 ± 0.2	4.36 ± 0.1	4.68 ± 0.2	4.60 ± 0.2
	HDL-C (mM)	1.32 ± 0.1	1.62 ± 0.1***	1.17 ± 0.1	1.54 ± 0.1***
	ApoA1 (mM)	1.54 ± 0.1	1.56 ± 0.1	1.39 ± 0.1	1.66 ± 0.1*
	ApoB (mM)	0.89 ± 0.03	0.75 ± 0.03**	0.94 ± 0.1	0.82 ± 0.05
	ASAT (IU/L)	21.46 ± 0.9	23.62 ± 1.2	32.91 ± 3.1 ## 25.20 ± 1.3*
	ALAT (IU/L)	21.25 ± 1.6	23.46 ± 2.3	38.36 ± 4.1 ## 22.45 ± 1.8**
	GGT (mg/dl)	32.21 ± 6.9	25.75 ± 7.4**	47.91 ± 5.8 ## 17.95 ± 2.1****
	Leptin (ng/ml)	63.00 ± 6.3	23.81 ± 3.9**** 47.29 ± 3.8	26.63 ± 4.0***
	Adiponectin (µg/ml)	4.46 ± 0.4	8.95 ± 0.8****	3.48 ± 0.4 #	5.98 ± 0.6***
	IL-6 (pg/ml)	3.11 ± 0.3	4.19 ± 1.6	4.28 ± 0.4 #	4.38 ± 0.8
	hsCRP (mg/L)	0.97 ± 0.2	0.17 ± 0.03**** 1.06 ± 0.2	0.22 ± 0.06***
	Fibrinogen (g/l)	4.58 ± 0.2	4.19 ± 0.2	3.80 ± 0.2 #	4.16 ± 0.2

#

Acknowledgments: We thank Dr. Florence Marchelli (Center for Research on Human Nutrition, Pitié-Salpêtriére Hospital, Paris, France), for patient recruitment, data collection, and sampling.

We thank Patricia Bonjour for histochemistry. We greatly thank Omran Allatif for his help in data presentation. We thank Mrs. Nathalie Colnot for providing technical assistance for tissue processing.

We thank Assistance Publique -Hôpitaux de Paris (APHP) and the Direction of Clinical Research, the Programme Hospitalier de Recherche Clinique for supporting clinical investigations (AOR 02076 to K.C.). This work was also supported by the European Commission (Collaborative Project FP6 ''Hepatic and adipose tissue and functions in the metabolic syndrome'' HEPADIP, see http://www.hepadip.org/, contract LSHM-CT-2005-018734). The research leading to these results received funding from the European Union Seventh Framework Program (FP7/2007-2013) under grant agreement no. Health-F2-2009-241762, for the FLIP project. We thank the Société Française de Nutrition (SFN) for financial support (JAW). performed the experiments and data analysis, drafted the manuscript, and prepared figures. ML performed LC-MS/MS analysis, NS performed machine learning analysis and