text:
Madalena Chaves
email: madalena.chaves@inria.fr

Jean-Luc Gouzé
email: jean-luc.gouze@inria.fr

2D piecewise affine models approximate real continuous dynamics up to invariant sets

Keywords: Piecewise affine models, ordinary differential equations, hybrid systems, approximation, invariant sets

Piecewise affine models often provide a good approximation to describe continuous systems, but may involve a high degree of simplification. To compare solutions of the continuous and piecewise affine models, it is important to quantify the differences between solutions in each region of the state space. As an approach, we will use enveloping "bands" to characterize continuous activation or inhibition functions, and then describe the differences between continuous and piecewise affine solutions in terms of the width δ of these bands. As a case study, we will consider the negative feedback loop, a classical motif in two dimensions which results in oscillating behaviour. For this example, it is shown that the two types of models may differ only on a compact invariant set (the interior of a limit cycle), whose diameter is a function of the band width δ.

INTRODUCTION

Piecewise affine models consist on systems of differential equations whose vector fields are based on a combination (sums of products) of step functions and are amenable to the use of theoretical analysis tools. This modeling framework has been introduced by L. Glass and co-authors [START_REF] Glass | The logical analysis of continuous, nonlinear biochemical control networks[END_REF], to describe and represent the dynamics of biological (e.g., genetic) regulatory networks. They have developed several qualitative results, including a classification of systems according the existence of oscillatory behavior [START_REF] Glass | Stable oscillations in mathematical models of biological control systems[END_REF]. Applications to genetic networks include the study of the carbon stress response in Escherichia coli [START_REF] Ropers | Qualitative simulation of the carbon starvation response in Escherichia coli[END_REF] and the segment polarity genes family in Drosophila Melanogaster [START_REF] Chaves | Methods of robustness analysis for boolean models of gene control networks[END_REF].

To understand the connection between solutions of a piecewise affine model and those of a continuous system, several approaches have been suggested, based on a continuous approximation of the step function and then studying the limit [START_REF] De | Modeling and simulation of genetic regulatory systems: A literature review[END_REF][START_REF] Plahte | Global analysis of steady points for systems of differential equations with sigmoid interactions[END_REF].

A more general formulation is suggested here (see Section 2.2), where the continuous system is composed of activation and inhibition functions; these are represented by two constants (zero and one) separated by a band (of width δ) enveloping a threshold region in the phase space, which allows a continuous transition from zero to one (no particular mathematical expression is assumed for this transition region) or from one to zero. Similar approaches This work was supported in part by the projects GeMCo (ANR 2010 BLANC020101), RESET (Bioinformatique, ANR-11-BINF-0005), and by the LABEX SIGNALIFE (ANR-11-LABX-0028-01).

have been proposed before in [START_REF] Chaves | Bistable biological systems: a characterization through local compact input-to-state stability[END_REF]; [START_REF] Plahte | Global analysis of steady points for systems of differential equations with sigmoid interactions[END_REF].

To study and apply this formalism, we will focus on a two-dimensional negative feedback loop (see Section 3), which represents a frequent motif in complex regulatory networks [START_REF] Thomas | Biological feedback[END_REF], in particular it is at the core of the E. coli carbon starvation model developed in [START_REF] Ropers | Qualitative simulation of the carbon starvation response in Escherichia coli[END_REF]. Our objective is to characterize the regions of the state space where the trajectories of the continuous and piecewise affine systems have the same or similar qualitative dynamics, as well as those regions where their dynamics diverges.

Two main results from our analysis are to be pointed out (see Sections 4 and 5): (i) first, the region where qualitative differences occur between the behavior of continuous and piecewise affine models can be quantified in terms of the band width δ; (ii) second, trajectories of both systems converge towards this region; while the continuous system may indeed exhibit some dynamics not modeled by the piecewise affine systems, these are confined to a compact invariant set C, which can be quantified and has a diameter which is a function of δ and tends to zero with δ.

Our tools are the construction of two outer and inner piecewise linear systems that will enclose the smooth system. We study the dynamical behaviour of these two systems, and obtain bounds for the smooth one, for a given width δ of the band. Unlike other works [START_REF] Plahte | Global analysis of steady points for systems of differential equations with sigmoid interactions[END_REF], our approach (similar to [START_REF] Chaves | Bistable biological systems: a characterization through local compact input-to-state stability[END_REF]) is not a limit approach, and gives results for a fixed δ. A related idea is found in [START_REF] Shu | Dynamical properties of a two-gene network with hysteresis[END_REF], where the work is oriented toward hybrid systems and hysteresis.

CONTINUOUS AND PIECEWISE AFFINE MODELS

Protein and gene interactions in genetic networks are often well described by sigmoid-like functions [START_REF] Yagil | On the relation between effector concentration and the rate of induced enzyme synthesis[END_REF], where abrupt changes in transcription (or synthesis rates) are observed once a given transcription factor has reached a certain threshold concentration. Continuous models of large genetic networks can be very complex: while numerical simulations provide good results, theoretical analysis is rarely possible without simplifying the mathematical system. A possible approach is to consider interactions with only two states On or Off [START_REF] Thomas | Boolean formalization of genetic control circuits[END_REF][START_REF] Glass | The logical analysis of continuous, nonlinear biochemical control networks[END_REF], thus approximating sigmoidlike functions by piecewise constant step functions.

PWA systems

This approximation leads to the formulation of piecewise affine (PWA) differential equations models (de Jong, 2002) and, in particular, it has led to the development of a qualitative method for analysis of genetic networks [START_REF] De Jong | Genetic network analyzer : Qualitative simulation of genetic regulatory networks[END_REF]. We very briefly review the PWA formalism and notation (for more details see [START_REF] Casey | Piecewiselinear models of genetic regulatory networks: equilibria and their stability[END_REF]. Define the increasing (or activation) step function as:

s + (x, θ) = 0, x < θ 1, x > θ (1)
and the decreasing (or inhibition) step function as

s -(x, θ) = 1 -s + (x, θ).
The equations for variable x i ∈ [0, +∞) (i = 1, . . . , n) is: ẋi = f i (x; θ j1,i , . . . , θ jp n ,i ) -γ i x i where functions f i represent the dependence of the rate of synthesis of a protein encoded by gene i on the concentrations x j of the other proteins in the cell, and θ j are thresholds, with:

f i (x) = l∈I κ il b il (x), (2)
where κ il > 0 is a rate parameter, b il : R n ≥0 → {0, 1} is a Boolean-valued regulation function, and I is an index set. The regulation functions b il summarize the incoming interactions at node i, and are typically written as combinations (sums of products) of step functions with thresholds θ j . The γ i are constant degradation rates.

The thresholds partition the state space into several rectangular regions, or regular domains. Inside each regular domain, D J , the vector field is linear and decoupled, and solutions of the system can be explicitly computed. As t tends to infinity, the solution would tend to the point φ ij = κ ij /γ i , with φ = (φ 1j1 , . . . , φ njn ) called the focal point of domain D J . However, as soon as the trajectories hit the boundary of D J , the vector field switches. The full trajectory is thus a concatenation of the solutions in each successive regular domain. Note that the vector field is discontinuous and not defined at the threshold values, x i = θ i,j for all i = 1, . . . , n, j = 1, . . . , p i . However, solutions of the system (2) "across" or "along" a threshold can still be defined in the sense of Filippov, as the solutions of differential inclusions (see [START_REF] Casey | Piecewiselinear models of genetic regulatory networks: equilibria and their stability[END_REF] for more details). We do not need these notions in the paper.

A general formulation for continuous systems

Now we describe the continuous systems associated to these PWA systems. Two widely used continuous models for regulatory networks substitute the step functions either by Hill functions [START_REF] De | Modeling and simulation of genetic regulatory systems: A literature review[END_REF] or logoids (saturated linear functions) [START_REF] Plahte | Global analysis of steady points for systems of differential equations with sigmoid interactions[END_REF].

We now propose a more generally formulated function which coincides with the step function outside a (small) interval around the threshold value, and may be any differentiable function in the transition interval (see also [START_REF] Chaves | Bistable biological systems: a characterization through local compact input-to-state stability[END_REF]). Given positive constants θ, κ, and δ with

A1. δ ∈ (0, θ) such that κ/γ -(θ ± δ) > 0;
and a transition function

µ trans : [θ -δ, θ + δ] → [0, 1] with the following properties: A2. µ trans (θ -δ) = 0, µ trans (θ + δ) = 1, and µ trans (x) ∈ (0, 1) for all x ∈ (θ -δ, θ + δ); A3. µ trans is continuously differentiable; define: µ + (x, θ; δ) = 0, 0 ≤ x ≤ θ -δ µ trans (x), θ -δ ≤ x ≤ θ + δ 1, θ + δ ≤ x. (3)
Note that (3) contains both the Hill1 and logoid formulation (with

µ trans (x) = 1 2δ (x -(θ -δ))
), but in general the transition function can be very freely chosen, for instance, it doesn't have to be monotonic. A similar definition holds for inhibition functions µ -(x, θ; δ) = 1 -µ + (x, θ; δ). For the comparison between PWA and continuous systems, we will focus on the case of small δ, and will not deal with the case δ → 0. The form of the model (2) remains unchanged, replace only the step functions by the corresponding, more general, functions of the form (3).

THE (SHIFTED) 2D NEGATIVE LOOP

As an example, consider the two-node network {x → y, y x} which represents a frequent motif in complex regulatory networks. It is one of the most classical motifs in gene networks, together with the bistable switch [START_REF] Thomas | Biological feedback[END_REF]. For instance, the model of the E. coli nutritional stress response developed in [START_REF] Ropers | Qualitative simulation of the carbon starvation response in Escherichia coli[END_REF] can be reduced to a two-dimensional negative feedback loop, in the presence of glucose (as shown in [START_REF] Grognard | Piecewise-linear models of genetic regulatory networks: theory and example[END_REF]).

Let κ 1 , κ 2 , γ 1 , and γ 2 be positive constants, and θ 1 , and θ 2 the thresholds associated with each interaction. Assume that κ i /γ i -θ i > 0. Using step functions, the 2-dimensional negative feedback loop is modeled as:

Σ : ẋ = κ 1 s -(y, θ 2 ) -γ 1 x ẏ = κ 2 s + (x, θ 1 ) -γ 2 y, (4)
It has been shown that system (4) converges towards the point (θ 1 , θ 2 ) [START_REF] Glass | Stable oscillations in mathematical models of biological control systems[END_REF]. The corresponding smooth system is:

Σ δ : ẋ = κ 1 µ -(y, θ 2 ; δ) -γ 1 x ẏ = κ 2 µ + (x, θ 1 ; δ) -γ 2 y (5)
To analyze (5), we first study a limiting case, obtained by choosing functions µ -and µ + which provide upper and lower bounds for trajectories of (5). Systems ( 5) and ( 4) have exactly the same vector fields outside the bands enveloping the threshold values, that is outside

[θ 1 -δ, θ 1 + δ] × [0, ∞) and [0, ∞) × [θ 2 -δ, θ 2 + δ].
So the qualitative dynamics of the continuous and PWA systems are the same outside these regions. Within these regions, trajectories of ( 5) deviate from trajectories of (4).

Maximal deviation (the upper bound) corresponds to a system where the vector field (5) switches only at the "last possible value": for instance, in the left half of the plane (x < θ 1 -δ), let µ + (y, θ y ) = 0 for all y > θ 2 -δ, while in the right half of the plane (x > θ 1 + δ), let µ + (y, θ y ) = 0 for all y > θ 2 + δ). This gives rise to an outer system of the form (2), which generates a shifted negative feedback loop, defined for all (x, y) /

∈ [θ - 1 , θ + 1 ] × [θ - 2 , θ + 2 ]: ẋ = κ 1 s -(y, θ + 2 )s + (x, θ + 1 ) + s -(y, θ - 2 )s -(x, θ + 1 ) -γ 1 x ẏ = κ 2 s + (x, θ + 1 )s -(y, θ + 2 ) + s + (x, θ - 1 )s + (y, θ + 2 ) -γ 2 y (6)
(where θ - i = θ i -δ and θ + i = θ i + δ) whose trajectories "enclose" the trajectories of any system of the form (5) (see Fig. 1). The phase plane is thus partitioned into four orthants separated by bands of width 2δ and a small square region in the middle. The four shifted orthants are

Q 01 = {(x, y) ∈ R 2 : x ∈ [0, θ 1 -δ), y ∈ (θ 2 -δ, ∞)} Q 00 = {(x, y) ∈ R 2 : x ∈ [0, θ 1 + δ), y ∈ [0, θ 2 -δ)} Q 10 = {(x, y) ∈ R 2 : x ∈ (θ 1 + δ, ∞), y ∈ [0, θ 2 + δ)} Q 11 = {(x, y) ∈ R 2 : x ∈ (θ 1 -δ, ∞), y ∈ (θ 2 + δ, ∞)}.
The fact that the transition thresholds are not aligned with respect to left/right or up/bottom half planes, implies that system (6) is not of the form studied in [START_REF] Glass | Stable oscillations in mathematical models of biological control systems[END_REF], even though it is still a negative feedback loop. In fact, we will next show that system (6) converges to a limit cycle around the middle region

[θ 1 - δ, θ 1 + δ] × [θ 2 -δ, θ 2 + δ].
In a similar way, we define a Fig. 1. The limit cycle (red) and a trajectory (black) of system (6). The parameters are: κ 1 = 3, κ 2 = 5, θ 1 = 1.7, θ 2 = 2.2, γ 1 = 0.7, γ 2 = 1.2, and δ = 0.16.

lower bound or inner PWA approximation, which switches at the "first possible value" and will be shown to converge toward (θ 1 , θ 2 ).

EXISTENCE OF A LIMIT CYCLE

Now, we will compute the first return map for (6), and show that it has a unique fixed point (see Fig. 1). In addition, we will show that this fixed point is outside the middle region, and that trajectories starting outside converge towards this point. To simplify notation, define

A i = κ i γ i , θ + i = θ i + δ, θ - i = θ i -δ (i = 1, 2), and γ = γ 1 γ 2 .
For technical reasons (which will become clear in Lemma 4.4 below), assume that:

θ - 2 A2 γ < θ - 1 .
Morever, assumptions A1 and A2 hold for every index i. In the following results, it will be assumed that trajectories of ( 6) are evolving in [0, A 1 ]×[0, A 2 ], since this is an invariant set for the system. Define the functions (invertible in their domains):

h 1 (r) = θ - 1 r -A 1 θ + 1 -A 1 , h 2 (r) = θ + 2 -A 2 θ - 2 1 r 1/γ -A 2 γ , with domains h 1 : [0, A 1 /θ - 1 ] → [0, A 1 /(A 1 -θ + 1 )] and h 2 : ((θ - 2 /A 2 ) γ , ∞) → ((A 2 -θ + 2 )/A 2 , ∞)
. Some basic properties of these functions are next summarized:

Lemma 4.1. Assume θ + i < A i (i = 1, 2) and (θ - 2 /A 2 ) γ < θ -
1 . The functions h 1 and h 2 satisfy:

(a) dh 1 dr = θ - 1 θ + 1 -A 1 < 0, h 1 (0) = A 1 A 1 -θ + 1 , h 1 A 1 θ - 1 = 0; (b) dh 2 dr = -θ - 2 h 2 (r) 1 r 1 A 2 r 1/γ -θ - 2 < 0, lim r→(θ - 2 /A 2 ) γ h 2 (r) = ∞, lim r→∞ h 2 (r) = A 2 -θ + 2 A 2 γ ; (c) h 1 (r) > r if and only if r ∈ 0, A 1 A 1 -(θ + 1 -θ - 1 ) ; (d) dh 1 • h 2 dr (s) > dh 2 • h 1 dr (s) , ∀ s ∈ L = θ - 2 A 2 γ , 1 .
To compute the Poincaré map for this system, we will follow the trajectory of a point starting on the segment

Y 0 = θ + 2 with θ + 1 < X 0 < A 1 : (X 0 , θ + 2 ) ; (θ - 1 , Y A ) ; (X B , θ - 2 ) ; (θ + 1 , Y C ) ; (X F , θ + 2 ). Lemma 4.2. The Poincaré map of system (6) is given by f : [0, A 1 ] → [0, A 1 ]: f (x) = θ - 1 h -1 1 • h 2 • h 1 • h -1 2 x θ - 1 . ( 7
)
Proof: In general, solving the linear equations ẋ = γ 1 (φ 1x) and ẏ = γ 2 (φ 2 -x) in each quadrant, the trajectories from a point P a = (x a , y a ) to a point P b = (x b , y b ), satisfy:

x b -φ 1 x a -φ 1 = e -γ1(t b -ta) , y b -φ 2 y a -φ 2 = e -γ2(t b -ta) , ( 8
)
where φ i ∈ {0, A i } denotes the appropriate focal point. Thus,

x b -φ 1 x a -φ 1 = y b -φ 2 y a -φ 2 γ .
The expression of the Poincaré map follows after some computations.

The existence of a fixed point of the Poincaré map can be deduced from the following Lemma. Lemma 4.3. There is a unique solution

r * of h 1 • h 2 (r * ) = h 2 • h 1 (r * ) in the interval L = ((θ - 2 /A 2 ) γ , 1).
Proof: To prove this result, let ε > 0 be a small number and choose a real number z such that:

z = θ - 2 /A 2 γ + ε < 1, h 2 (z) = A 1 /θ - 1
(this choice is possible, by Lemma 4.1). We will show that:

(a) h 1 • h 2 (z) < h 2 • h 1 (z); (b) h 1 • h 2 (1) > h 2 • h 1 (1); (c) dh1•h2 dr (s) > dh2•h1
dr (s) , for all s ∈ L. First, observe that h 1 and h 2 are both strictly decreasing (Lemma 4.1). Next, note that

h 1 • h 2 (z) = h 1 (A 1 /θ - 1 ) = 0 and that, for any z, h 2 • h 1 (z) > A2-θ + 2 A2
so, for some appropriately small ε > 0, part (a) is true. It is easy to check that (b) is also true:

h 1 • h 2 (1) = h 1 θ + 2 -A 2 θ - 2 -A 2 γ > θ - 1 -A 1 θ + 1 -A 1 > 1 h 2 • h 1 (1) = h 2 θ - 1 -A 1 θ + 1 -A 1 < θ + 2 -A 2 θ - 2 -A 2 γ < 1.
Finally, statement (c) follows directly from Lemma 4.1(d).

From (a) and (b), we conclude that h 1 • h 2 and h 2 • h 1 intersect at least once, in the interval L: as a composition of two decreasing functions, both h 1 • h 2 and h 2 • h 1 are strictly increasing; from (c), h 1 •h 2 increases strictly faster than h 2 • h 1 ; and from (a) h 1 • h 2 starts at a lower value than h 2 • h 1 , so that the functions can only intersect once, as claimed.

Lemma 4.4. The Poincaré map (7) has a unique fixed point, X * , and it is stable.

Proof: First, we will show the map (7) has a unique fixed point, and then that |df /dx| < 1, to show that it is stable.

h 1 x 0 θ - 1 = h 1 • h 2 x B θ - 1 and h 1 x F θ - 1 = h 2 • h 1 x B θ - 1 Note that x 0 /θ - 1 is in the domain of h 1 (since x 0 ∈ [0, A 1
]); and also x B ∈ (0, θ - 1 ) (see also Fig. 1). Finding a fixed point of the Poincaré map is thus equivalent to finding x B that satisfies x 0 = x F , that is:

h 1 • h 2 x B θ - 1 = h 2 • h 1 x B θ - 1 .
Since x B < θ - 1 and by definition of the domain of h 2 , any fixed point of f must be in the interval θ - 2 /A 2 γ , θ - 1 . Then, by Lemma 4.3, there is a unique fixed point of f and it is of the form X * = r * θ - 1 . To check stability, the chain rule and Lemma 4.1 show that |df /dx(X * )| < 1.

The existence of a unique, stable, fixed point for the Poincaré map implies the next result. Theorem 1. For each 0 < δ < min{θ 1 , θ 2 }, the system (6) admits a limit cycle, C δ , passing through the point (X * , θ 2 + δ), where X * = X * (δ) is the unique fixed point of the Poincaré map f . Trajectories starting outside the region enclosed by C δ converge towards it. As δ → 0, lim δ→0 X * (δ) = θ 1 , and lim

δ→0 C δ → {(θ 1 , θ 2 )}.
Proof: To prove the results as δ → 0, observe that the interval where the fixed point is located can be reduced further to:

(z 1 , z 2 ) := θ - 2 θ + 2 ν , θ - 2 θ + 2 γ ,
where ν > γ can be chosen such that (using Lemma 4.1),

z 1 = θ - 2 /θ + 2 ν and h 2 (z 1 ) = A 1 /θ - 1 . This implies h 1 • h 2 (z 1 ) = 0, so that h 1 • h 2 (z 1 ) < (A 2 - θ + 2 )/A 2 < h 2 • h 1 (z 1 ), as in the proof of Lemma 4.3, part(a). Next, one has h 2 (z 2 ) = 1 and h 1 (1) = (A 1 - θ - 1 )/(A 1 -θ + 1 ) > 1, that is h 1 •h 2 (z 1 ) > 1; while h 1 (z 2 ) > 1 implies h 2 • h 1 (z 2 ) < 1. We then have h 2 • h 1 (z 2 ) < h 1 • h 2 (z 2 )
. Therefore, as in the argument of Lemma 4.3, we conclude that the functions h 2 • h 1 and h 1 • h 2 intersect exactly once in the interval (z 1 , z 2 ), which also implies r * ∈ (z 1 , z 2 ). Since θ ± 2 = θ 2 ± δ, the interval for r * tends to {1}, as δ → 0. Hence, the point (X * , θ 2 + δ) (and the curve C δ ) tend to (θ 1 , θ 2 ).

Similarly, the study of the inner PWA approximation can be done, and it is easy to see that it has an attractor at (θ 1 , θ 2 ).

We are now interested in the results for a finite fixed δ.

CONVERGENCE TO A COMPACT SET

The four segments constituting the limit cycle C δ (see Fig. 1) can be parametrized as: C = {(g q (z), z) : z ∈ I q , q ∈ {01, 00, 10, 11}} where q denotes one of the four orthants defined above (Q 01 , Q 00 , Q 10 , or Q 11 ), and the functions g q are defined using expressions (8), as follows. Let x B denote the intersection of C δ with the segment (θ 1 , A 1 ) and y = θ + 2 , and y A (resp., y C ) denote the intersection of C δ with the segment (θ 2 , A 2 ) and x = θ - 1 (resp., (0, θ 2 ) and x = θ + 1 ).

I 01 = [θ - 2 , y A ], g 01 (z) = θ - 1 z y A γ I 00 = [y C , θ - 2 ], g 00 (z) = A 1 -(A 1 -x B ) z θ - 2 γ I 10 = [y C , θ + 2 ], g 10 (z) = A 1 -(A 1 -θ + 1 ) A 2 -z A 2 -y C γ I 11 = [θ + 2 , y A ], g 11 (z) = x B A 2 -z A 2 -θ + 2 γ
Let C denote the compact set enclosed by the closed curved C δ . To show that trajectories of (5) converge towards this compact set, we will use a function that measures the distance to the set C, from each point in the plane. By definition, such a function will be zero at any point in the set C. At each point (x, y) ∈ R 2 ≥0 define function V (x, y):

min z∈I 01 |y -z| + 1 γ θ - 2 θ - 1 |x -g 01 (z)| , if (x, y) ∈ Q 01 \ C min z∈I 00 A 1 -θ + 1 θ - 1 {|y -z| + 1 γ θ - 2 θ - 1 |x -g 00 (z)| , if (x, y) ∈ Q 00 \ C min z∈I 10 A 1 -θ + 1 θ - 1 |y -z| + 1 γ A 2 -θ + 2 θ - 1 |x -g 10 (z)| , if (x, y) ∈ Q 10 \ C min z∈I 11 |y -z| + 1 γ A 2 -θ + 2 θ - 1 |x -g 11 (z)| , if (x, y) ∈ Q 11 \ C 0, if (x, y) ∈ C . (9)
We will next show that V is a "flat" Lyapunov function for system (5). It is clear that V (x, y) is zero if and only if (x, y) ∈ C. Indeed, since (g q (z), z) parametrizes the boundary of C in orthant Q q , V (x, y) is zero whenever (x, y) belongs to the boundary of C. Otherwise, V is strictly positive, as the sum of positive values. It is not difficult to show that V is continuous. A more detailed characterization of V , including explicit expressions, can be obtained based on minimizing the function W with respect to z:

W (r, s, z) = S |s -z| + R |r -g(z)| .
where S = 1 and

R = 1 γ θ - 2 θ - 1 in Q 01 . Proposition 1 below
characterizes the decrease of V along trajectories evolving in Q 01 . Let V 01 denote the function V in the orthant Q 01 . The same arguments will apply to the other orthants, under appropriate changes of variables. For this reason, in this Section, a "general" set of coordinates will be adopted, (r, s), with: Recall also that the boundary of C in Q 01 is defined by s = g -1 (r), so that we are interested in pairs (r, s) ∈ Q 01 \ C, that is s > max{ θs , g -1 (r)}.

Proposition 1. Given any δ > 0, there exists a constant c = c(γ r , γ s ) such that:

∇V 01 f (r, s; δ) < -cV 01 (r, s), ∀ (r, s) ∈ Q 01 \ C.
Proof: Fix any δ > 0. Recall the definition of Q 01 (9), and note that f r (r, s; δ) ≤ µ - 1 (s, θ s ; δ) -γ r r and f s (r, s; δ) = -γ s s that is, coordinate s is strictly decreasing on Q 01 , while coordinate r is decreasing as long as s ≥ θ s + δ, but may be increasing for θ s -δ ≤ s < θ s + δ (depending on the function µ - 1 ).

It can be shown that V 01 = V i depending on which of the four regions of the Q 01 orthant (r, s) belongs to. We will show that ∇V i f (r, s; δ) < -cV i (r, s) holds for each i = 1, . . . , 4, in its corresponding region. Assume first that s > s α and consider V 1 (r, s) = R( θr -r) + S(s -s α ): where the second equality was obtained by first putting γ s in evidence, and then adding and subtracting the quantity γR θr -Ss α . The inequality in the third line follows by using -1 γs Rµ - 1 (r, θ r ; δ) ≤ 0 and observing that γR θr -Ss α = S θs -Ss α < 0. The cases for the other V i are treated similarly.

∇V 1 f (r, s; δ) = -R(µ - 1 (
As a corollary, we conclude that V satisfies a similar estimate for all (x, y) ∈ R 2 ≥0 . Proposition 2. There exist a constant c = c(γ 1 , γ 2 ) such that

∇V f (x, y; δ) < -cV (x, y), ∀ (x, y) ∈ R 2 ≥0 \ C
and ∇V f (x, y; δ) = 0, for all (x, y) ∈ C.

Proof: It is enough to apply Proposition 1 to each orthant using appropriate variables (these correspond to a clockwise rotation, until the desired orthant coincides with Q 01 ). We cannot detail here.

The convergence of the trajectories of system (5) can now be shown. Theorem 2. Trajectories of (5) converge towards the compact set enclosed by the limit cycle C.

Proof: First, note that, for any ξ

∈ R 2 ≥0 dist(ξ, C) ≤ max 1, θ - 1 A 1 -θ + 1 , γ θ - 1 θ - 2 , γ θ - 1 A 2 -θ + 2 V (ξ) = c 3 V (ξ).
Now consider a solution of ( 5), (x(t), y(t)) and let ψ(t) = V (x(t), y(t)). Apply Proposition 2 to conclude that

dψ dt = ∇V f (x, y; δ) < -cψ(t),
for some positive constant c. Integration yields ψ(t) ≤ ψ(0) e -ct , implying that the distance between (x(t), y(t)) and C tends to zero as time tends to infinity.

SIZE OF THE LIMIT CYCLE

It is interesting to explore the links between the band width δ and the size of the limit cycle. First, in the proof of Theorem 1, it is easy to obtain (from the estimate of X * ) that a rough estimate of the size of one radius of the limit cycle along the axis y

= θ 2 -δ is R(δ) = θ 1 -(θ - 1 ) θ - 2 θ + 2 γ ( 10
)
Of course, R(0) = 0 and R(θ 1 ) = θ 1 , and function R is increasing. The first term of the Taylor expansion around δ = 0 is (2θ 1 γ/θ 2 + 1). If we choose (to simplify) θ 1 γ/θ 2 = 1, γ = θ 1 /θ 2 = 1, then function R/θ 1 is concave between 0 and 1, and admits a maximum for δ = 1, with an initial slope of 3.

We checked numerically, for arbitrary values of the parameters, these findings (cf. Fig2). It is shown on the figure that the estimate is close to the real value of the radius of the limit cycle, except around zero, and also the concave form of the curve in function of δ.

EXTENSION TO N-DIMENSIONAL LOOPS

These results can be extended for higher dimensional negative loops. Assuming that the system is of a simple form {x → z, z → y, y x}, with a single threshold associated to each variable, a similar idea can be applied, by considering functions µ ± instead of step functions. In this case, one question is whether trajectories of the general system will still converge towards a compact set. For instance (under an appropriate parameter choice) the 3D negative loop is known to exhibit a limit cycle [START_REF] Farcot | Periodic solutions of piecewise affine gene network models with non uniform decay rates: The case of a negative feedback loop[END_REF], so it is expected that trajectories of the general system using functions µ ± will stay in an "annular" region enclosing the nominal limit cycle (see [START_REF] Poignard | Periodic Oscillations for Non Monotonic Smooth Negative Feedback Circuits[END_REF]).

CONCLUSIONS

This work shows how a continuous system can be bounded by two piecewise affine systems, which generate outer and inner bounds for its trajectories. Application to a 2-dimensional negative loop shows that the qualitative behavior of the continuous and PWA systems are quite similar except on a compact invariant set. Trajectories of both systems will converge towards this set (Theorems 1 and 2). Inside this compact set, the behavior of the continuous system depends on the transition part of the continuous system, and cannot be predicted by the PWA systems. However, the diameter of this set decreases with the width δ of the transition interval, and shrinks to a single point in the phase space when δ → 0.

In practice, many genetic and signaling interactions have switch-like properties which translate to narrow transition regions, and we think that this approach may give new tools to study continuous systems with the help of their "enveloping" PWA systems. The resulting behaviour are similar, up to some invariant sets, where anything can happen; but their size is small when band width δ is small.

  = y A , r β = g( θs) and g(s) = θr (s/sα) γ and g -1 (r) = sα r/ θr 1 γ .

  r, θr; δ) -γrr) + S(-γss) θr; δ) + γRr -γR θr -Ss + Ssα +γs(γR θr -Ssα) < -γs min{1, γ} R( θr -r) + S(s -sα) < -γs min{1, γ}V 1 (r, s)

Fig. 2 .

 2 Fig. 2. Size of the limit cycle w.r.t. band width δ: blue: numerical results; red curve: estimates with (10); dashed red: first order approximation.

In fact, it is not exactly the Hill function, but a function taking the values zero and one outside the central band where it is a Hill formulation chosen to obtain continuity.


David Lechevalier
email: david.lechevalier@etu.u-bourgogne.fr

Ronay Ak
email: ronay.ak@nist.govyung-tsun.lee@nist.gov

Y Tina Lee

Steven Hudak
email: hudak1@umbc.edu

Sebti Foufou
email: sfoufou@qu.edu.qa

A Neural Network Meta-Model and its Application for Manufacturing

Keywords: neural network, meta-model, data analytics, PMML, manufacturing, I

Manufacturing generates a vast amount of data both from operations and simulation. Extracting appropriate information from this data can provide insights to increase a manufacturer's competitive advantage through improved sustainability, productivity, and flexibility of their operations. Manufacturers, as well as other industries, have successfully applied a promising statistical learning technique, called neural networks (NNs), to extract meaningful information from large data sets, so called big data. However, the application of NN to manufacturing problems remains limited because it involves the specialized skills of a data scientist.

This paper introduces an approach to automate the application of analytical models to manufacturing problems. We present an NN meta-model (MM), which defines a set of concepts, rules, and constraints to represent NNs. An NN model can be automatically generated and manipulated based on the specifications of the NN MM. In addition, we present an algorithm to generate a predictive model from an NN and available data. The predictive model is represented in either Predictive Model Markup Language (PMML) or Portable Format for Analytics (PFA). Then we illustrate the approach in the context of a specific manufacturing system. Finally, we identify future steps planned towards later implementation of the proposed approach.

INTRODUCTION

The manufacturing industry generates a large amount of data [START_REF] Young | What businesses can learn from big data and high performance analytics in the manufacturing industry[END_REF]. At each manufacturing level (i.e., from the shop floor to the enterprise level), data is produced and recorded to monitor the operations that occur. One way to extract useful information from "big data" is through the application of data analytics. Applying data analytics on manufacturing data is a promising way to improve the efficiency of the manufacturing system as well as reducing the cost of production at every manufacturing level [START_REF] Manyika | Big data: The next frontier for innovation, competition and productivity[END_REF]. In particular, neural networks (NN), a statistical technique, has been widely used in data analytics. Examples of the NN applications are available at the process level [START_REF] Benardos | Prediction of surface roughness in CNC face milling using neural networks and Taguchi's design of experiments[END_REF], [START_REF] Wc | A neural network-based approach for dynamic quality prediction in a plastic injection molding process[END_REF], at the machine level [START_REF] Dornfeld | Neural network sensor fusion for tool condition monitoring[END_REF], [START_REF] Ghosh | Estimation of tool wear during CNC milling using neural network-based sensor fusion[END_REF], at the factory level [START_REF] Lin | Using an artificial neural network prediction model to optimize work-in-process inventory level for wafer fabrication[END_REF], [START_REF] Kj Wang | A hybrid knowledge discovery model using decision tree and neural network for selecting dispatching rules of a semiconductor final testing factory[END_REF] and at the supply chain level [START_REF] Chiu | Collaborative supply chain planning using the artificial neural network approach[END_REF], [START_REF] Carbonneau | Application of machine learning techniques for supply chain demand forecasting[END_REF]. These applications demonstrate how manufacturers can make their systems smarter using neural networks. However, developing an NN model requires data science knowledge that manufacturers often do not have. As a contribution to automating the application of neural networks in manufacturing, we propose an NN meta-model (NNMM) that encapsulates data scientist knowledge about neural networks. We introduce an algorithm to automatically manipulate a neural network model (NNM), built using the meta-model and illustrate how manufacturers can leverage the meta-model to apply neural networks to their manufacturing systems.

The paper is organized as follows. Section II provides the use case scenario and required background about both neural networks and meta-models. Section III presents the NNMM and an algorithm to manipulate an NNM created using this metamodel. Section IV introduces a manufacturing use case to illustrate the capabilities of using the meta-model in the manufacturing area. We conclude this paper by presenting future work that supports automation of the neural networks application on a manufacturing system. This paper will be of interest to manufacturers that look to apply neural networks on their manufacturing systems, and schema developers who look to represent data science knowledge.

II. CONTEXT AND BACKGROUND

In this section, we introduce the use case scenario and we provide background about both neural networks and metamodels.

A. Use case scenario

This section describes a case study that focuses on predicting the energy consumed by a milling machine tool. The goal of this case study is to predict estimated energy consumption corresponding to the machine's operational parameters. The following three input variables are used to estimate the required energy to manufacture a certain workpiece: Feed rate: the velocity at which the tool is fed Spindle speed: rotational speed of the tool Depth of cut: the actual depth of material that the tool is removing A milling machine tool obviously involves more parameters. For simplicity, we decided to only keep these three parameters which have the biggest impact on the energy consumption in the milling process.

B. Regression Analysis and Neural Network (NN)

We present the characteristics of regression analysis and neural networks. Regression analysis is a statistical process used to investigate the relationships between variables [START_REF] Golberg | Introduction to Regression Analysis[END_REF]. Regression techniques can be roughly divided into "linear" and "non-linear" regressions. A dataset is defined as a 𝒟 = {(𝑥 𝑖1 , 𝑥 𝑖2 , … , 𝑥 𝑖𝑝 , 𝑦 𝑖 )|𝑖 = 1, … , 𝑛}, where p and n indicate the number of input variables and the number of samples in our dataset, respectively. 𝑦 𝑖 is the corresponding output target scalar for each input vector 𝒙 𝑖 [START_REF] Hastie | The Elements of Statistical Learning: Data Mining, Inference, and Prediction[END_REF].

NNs are ooriginally inspired by the function of the neurons in the brain. They are typically used for classification and forecasting. They are frequently used as an alternative to standard nonlinear regression and cluster analysis techniques. NNs are composed of computing units (called neurons or nodes) operating in parallel. These units are arranged in different layers and interconnected by weighed edges (called synapses). A layer is the term used for a vertical row of neurons. Each of these computing units performs a few simple operations and communicates the results to its neighboring units. From a mathematical viewpoint, NNs consist of a set of nonlinear (e.g., sigmoidal) basis functions with free parameters; i.e., w or weights, that are adjusted. The objective of the adjustment is to minimize the error associated with regression in an iterative process called training using a dataset. The basis functions are called activation functions.

There are many types of NNs. Feed-forward neural networks (FNNs) and recurrent neural networks (RNNs) are the two main types. A RNN has neurons that transport a signal back through the network (at least one feedback connection in recurrent network) whereas FNNs feed outputs from individual neurons forward to one or more neurons or layers in the network [START_REF] Du | Neural Networks and Statistical Learning[END_REF], [START_REF] Rojas | Neural Networks: A Systematic Introduction[END_REF]. Multilayer perceptron (MLP) neural networks and Radial basis function (RBF) neural networks are two of the most common types of FNNs used as empirical nonlinear regression models. RBF networks use a radial basis function, i.e., a Gaussian kernel, as the activation function. RBFs networks have similar universal approximation capabilities as MLP networks. For the theory and application of the RBF networks, we refer the readers to [START_REF] Du | Neural Networks and Statistical Learning[END_REF], [START_REF] Howlett | Radial Basis Function Networks 2: New Advances in Design[END_REF].

A feed-forward neural network that represents our use case is illustrated in Fig. 1. The neural network is composed of an input layer, a hidden layer and an output layer. The input layer is the layer containing the input neurons. In our example, the input neurons represent the three input parameters: feed rate, spindle speed and depth of cut. The hidden layer contains the hidden neurons. The hidden layers are used to represent the relationships between the input layer and the output layer. The hidden layers allow a representation of the relationships among the variables when these relationships cannot be captured in a regression model. At both the input and hidden layers, there is a bias neuron. The bias neuron, which allows shifting the activation function to the left or right for improving the learning process, is a neuron with a constant output. This neuron is treated as a regular neuron in the associated layer. Finally, the output layer contains the output neuron that represents the predicted energy in our example. The free parameters w called weights are assigned to the edges that connect the different layers. During the training, these weights are adjusted to minimize the error between the output value of the NN and the real output value for a given data sample. In essence the weights represent the sensitivity of the output to the input variable. Equation (1) introduces how to compute the output value using the weights.

Each layer receives input signals generated by the previous layer, produces output signals through an activation function (e.g. a sigmoid function) and distributes them to the subsequent layer through the neurons. The network output is given by the following expression:

𝑓(𝑿; 𝒘) = 𝑦 ̂(𝑿) = 𝜓(𝑤 0𝑙 𝑏 + ∑ 𝑤 𝑗𝑙 𝜙(𝑤 0𝑗 𝑏 + ∑ 𝑤 𝑖𝑗 𝒙 𝑘 𝑝 𝑘=1 ) ℎ 𝑗=1 ) , (1)
where 𝑿 is an input vector with p entries, 𝑿 = (𝒙 1 , 𝒙 2 , … , 𝒙 𝑝 ), p is the number of input signals (variables), 𝒙 𝑘 is the k th input signal, w is the weight vector, ℎ indicates the number of hidden neurons, 𝑤 𝑖𝑗 is the synaptic weight from ith neuron to jth neuron, 𝜙() and 𝜓() are the activation (transfer) functions from input layer to hidden, and hidden layer to output layer, respectively, and b stands for the bias factor.

An estimate 𝒘 ̂ of 𝒘 can be obtained by a training procedure aimed at minimizing the quadratic error function, E, on a training set:

𝐸(𝑤) = ∑ (𝑦 ̂𝑖 -𝑦 𝑖 ) 2 𝑛 𝑖=1 , (2)
where 𝑦 ̂𝑖 = 𝑓(𝑿 𝑖 ; 𝑤 ̂) represents the output provided by the NN in correspondence of the input 𝑿 𝑖 , and n is the number of the training samples.

C. Motivation and Requirements for a Neural Network Metamodel in Manufacturing

This section presents the motivation and requirements to develop an NNMM for manufacturing. The survey presented in the section I shows that neural networks are widely used at different manufacturing applications. For each application, a neural network needs to be built based on the manufacturing knowledge involved. This manual process requires data science knowledge, in addition to manufacturing knowledge. A manufacturer that does not have sufficient data science knowledge will need external help to apply data analytics on a manufacturing system. In [START_REF] Lechevalier | Towards a domainspecific framework for predictive analytics in manufacturing[END_REF], authors describe a framework to automatically apply data analytics on a manufacturing system. The suggested framework uses manufacturing and data science knowledge to establish relationships between these two worlds. One central component of the framework is a meta-model repository where knowledge of manufacturing and data science are described.

A meta-model is a set of concepts, rules and constraints used to describe a specific domain [START_REF] Ledeczi | Composing domain-specific design environments[END_REF]. Thus, a meta-model provides domain-specific abstractions representing all of the relevant domain-specific core components. Such abstractions offer capabilities to maintain consistency across an entire domain, as well as capabilities to understand models of that domain in a common way. Models, as instances of a given meta-model, instantiate objects of the meta-model to represent a system inside the specific domain. If a model conforms to a domainspecific meta-model, it can be used to generate other models by simply manipulating the model objects in accordance with the meta-model specifications and meaning. Moreover, libraries instantiating meta-model components can be provided to facilitate the representation of the system inside a model.

An NNMM can contribute to the described framework by providing the capabilities to represent and communicate neural networks. An MM can provide a common way to represent NNs regardless of the available data or the specific software or programming languages used to generate the neural network. The NNMM can also facilitate the exchange and the manipulation of models that are built based on the NNMM. Using an NNMM, an algorithm can automatically create an NNM to represent an NN. Another algorithm can manipulate the generated model according to the meta-model specification and, finally, achieve a specific data analytics task using an NN as represented in the NN model. One example of task is the training of a neural network. Manufacturers can leverage these capabilities to generate neural network models representing manufacturing systems and then apply data analytics for their systems. Fig. 2 illustrates a workflow that does this, assuming that meta-models representing manufacturing knowledge and analytical knowledge are available. A manufacturer can represent a manufacturing system in a model using available concepts defined in a manufacturing meta-model. Once the manufacturing model is built, the manufacturer defines the target of the analysis and provides collected data to an algorithm. Defining the target of the analysis consists in defining the studied metric and the type of analytics the manufacturer wants to run (i.e., classification or regression). From the manufacturing model, the target of the analysis and the data, the algorithm automatically generates a neural network model using the concepts defined in the NNMM. In our use example, a manufacturer could have a milling machine model as its manufacturing model. The manufacturer would treat energy as the studied metric and regression as the type of analytics. The algorithm would then generate the NNM representing an NN as described in Fig. 1. Section V presents initial capabilities that this algorithm should provide.

In this workflow, it is important to understand that there are several types of manufacturing models. One model could be a simulation model while another model could be a production scheduling model. In both cases, these models can be very complex. Running data analytics directly from these models requires 1) developing an algorithm to apply neural networks specifically for each type of model and 2) modifying one's algorithm every time these models are modified. Providing an NNMM enables generation of an NNM from the manufacturing model. Once an NNM is generated, the manufacturer executes data analytics from the NNM. If a manufacturing model is modified, it does not imply that the NNM needs to be modify as well and the manufacturer does not need to repeat the step of generating the NNM. A manufacturer can keep the existing NNM model and run data analytics from this NNM until the new manufacturing model requires the NNM to be modified. Thus, manufacturers save time and money by reusing the NNM. In addition, an important modification of the manufacturing model The NNM created in the modeling environment represents only the structure of the neural network. Using this NNM, the data given as inputs by the manufacturer, and available tools or libraries implementing machine learning techniques, another algorithm can execute data analytic tasks. The choice of the executed task depends on the type of analysis that the manufacturer defines at the beginning of the workflow. The algorithm can also generate analytical models in standard format to apply scoring using these models and existing software that understands this format. Scoring is the process of using a model to make predictions about behavior.

III. DESCRIPTION OF THE NEURAL NETWORK META-MODEL

As described in Section II.B, a variety of neural networks are available. Creating an NNMM must facilitate representing any neural network. Moreover, using an NNMM, models have to be easy to understand and manipulate. To achieve these requirements, the meta-model needs to provide capabilities to represent any specificity of any neural network. The meta-model needs to provide explicit and clear concepts to make the model generation, understanding and manipulation simple. This paper will present our NNMM to represent neural networks and meet these requirements. It also describes the architecture of the second algorithm mentioned in the workflow.

A. Description of the Neural Network Meta-model

In this section, an NNMM developed in the Generic Modeling Environment (GME) [START_REF] Ledeczi | The generic modeling environment[END_REF] is proposed. GME is a tool for creating custom, domain-specific modeling environments. The domain-specific modeling environment is used to create high-level descriptive models of objects in a given domain. GME is also used to build models of real world objects in that domain. The NNMM represents different types of neural networks, including FNN, through various abstractions. In this work, we exclude the representation of RNN. An RNN 1) allows an edge to have the same neuron as its source and its destination and 2) requires higher-level abstractions than the abstractions presented in this NNMM. Including RNN will be a future step to allow the representation of all kinds of neural networks.

The meta-model needs to be easy to use, thus we have chosen a simplified representation of each concept. Fig. 3 shows the NNMM. A NeuralNetworkModel concept is composed of Neuron and Edge concepts. Neuron is an abstract concept and is extended by four concepts: InputNeuron, HiddenNeuron, BiasNeuron and OutputNeuron concepts. Edge is an abstract concept that is extended by the VisibleEdge and HiddenEdge concepts. A VisibleEdge is used to represent an edge between an input neuron and a hidden neuron, between a hidden neuron and an output neuron and between a bias neuron and an output neuron. Edges between two hidden neurons and between a bias neuron and a hidden neuron are represented using a HiddenEdge.

This NNMM is built to facilitate its adoption by allowing the extension of the meta-model to meet new requirements. For instance, in the proposed NNMM, a layer is represented by a neuron attribute . However, one can create a concept called Layer as a new concept as part of a NeuralNetworkModel.

B. Algorithm Generating a Predictive Model from an NNM and Data

To illustrate the capabilities of an NNMM, we implement an algorithm that automatically generates a predictive model using a neural network model and data. Four tasks must be executed to complete the workflow from an NNM and data to a predictive model in a specific format e.g., Predictive Model Markup Language (PMML) [START_REF] Guazzelli | PMML: An open standard for sharing models[END_REF] or Portable Format for Analytics (PFA) [START_REF] Pivarski | PFA specification v0.7[END_REF]. Tasks, as shown in Fig. 4, are 1) extract the available information specified in the NNM 2) build a data structure representation of the NNM using the extracted information, 3) train this data structure with the given data and updating the NNM, and 4) convert the trained data structure into a standardized or neutral format. Fig. 4 represents the process flow of the algorithm as an example of an analytical model interpreter presented in the workflow in Fig. 2. The NNM is implemented using GME. GME allows users to develop an algorithm that can directly manipulate the implemented model. First, the algorithm executes the first task. The relevant information extracted from the NNM is the type of neural network, the number of hidden layers, the number of hidden neurons per layer, the number of inputs neurons, the number of outputs neurons, and the names the NNM uses to refer to each neuron and connection within the network. This information is provided as the input of the second task. The second and third tasks are accomplished using Weka [START_REF] Hall | The WEKA data mining software: an update[END_REF], a Java machine-learning library. Using the information given as inputs, the algorithm can instantiate a Weka data structure representing the neural network specified in the model. Using data provided at the beginning of the workflow, the algorithm trains the neural network using functions implemented in the Weka library. The algorithm provides capabilities to customize the training by modifying several parameters of the training: learning rate, momentum, number of iterations, size of validation and random seed that is the number used to initialize the pseudorandom number generator. Default training parameter values are suggested as well. Finally, the NNM is updated to include the neural network edge weights that have been computed during the training. The final task is accomplished using Java PMML (JPMML) application program interface (API) [22], a Java library for producing and scoring PMML documents, and using Titus [23], a Python library for producing and scoring PFA documents.

Using these libraries and the Weka data structure trained during the previous task, a PMML or PFA document can be generated for scoring. The conversion from the Weka data structure, which represents the trained neural network to a PMML or PFA document involved considerable programming because Weka does not contain a method for exporting a neural network into either of these formats. This programming requires understanding of the Weka library and the PMML and PFA specifications. The algorithm is developed to provide an automatic way to build, train and manipulate a neural network from a neural network model.

IV. USE CASE

In this section, we will present the NNM and an implementation of the algorithm generating a predictive model for the use case we described in Section II.A.

A. Milling Process Analysis: Process Description and Neural Network Representation

The data set used in this case study was generated for [START_REF] Park | A generalized data-driven energy prediction model with uncertainty for a milling machine tool using Gaussian Process[END_REF]. To generate the data, Park et al. describe an intelligent machine monitoring framework that consists of two agents: i) a data management and extraction agent, and ii) a data-driven machine-learning and knowledge-extraction agent. The former agent consists of an MTConnect agent [START_REF] Vijayaraghavan | Improving Machine Tool Interoperability Using Standardized Interface Protocols: MT Connect[END_REF] and a data postprocessor. The MTConnect agent retrieves raw sensor data from a machine tool and systematically converts and organizes the data into semantically meaningful input features and response outputs.

The raw data includes the timestamp, power demand, feed rate, spindle speed, and numerical control (NC) code block (an NC code block corresponds to a specific cutting operation). The raw data is processed to obtain derived data, such as the average feed rate, average spindle speed, and cumulative energy consumption for each NC code block. The data set also includes simulated data, such as the depth of cut and cutting strategy (i.e., climb versus conventional milling). This data is determined by comparing two sequential tool positions reported by the MTConnect agent relative to the actual dimensions and location of the workpiece. The data set was generated from a part machined with different cutting operations such as face milling, contouring, slotting, pocketing, and drilling. Refer to [START_REF] Park | A generalized data-driven energy prediction model with uncertainty for a milling machine tool using Gaussian Process[END_REF], [START_REF] Bhinge | An intelligent machine monitoring system for energy prediction using a Gaussian Process regression[END_REF], [START_REF] Helu | Development of a Machine Tool Platform to Support Data Mining and Statistical Modeling for Machining Processes[END_REF] for detailed information about the experimental design and data processing techniques used to generate the data set.

With three input variables and one output variable, the NN structure therefore consists of three input neurons and one output neuron. For simplicity, in this example, we have used only one hidden layer and three hidden neurons.

B. Using the Neural Network Model and Data to Generate PMML file

Using the NNMM, an NNM that represents our case study is created. Fig. 5 shows the NNM created in GME to represent our use case. The algorithm described in the previous section provides an interface to provide the data and customize the parameters of the training if needed. Fig. 6 presents the graphical user interface (GUI) that pops up when a user calls the algorithm. The predictive model file generated from the algorithm allows the predictive model to be used by other software that understands the format of the file. Doing so users of that software can score new data against the predictive model. A manufacturer can create this predictive model and use it to establish diagnostics on the manufacturing system. The predictive model can also be used in different manufacturing facilities that use the same milling machine tool. Each factory can establish diagnostics by using their own data against this model. This flow can facilitate the application of data analytics in the manufacturing industry.

V. CONCLUSION AND FUTURE WORK

In order to improve systems that are already quite efficient, manufacturers can take advantage of data analytics [START_REF] Manyika | Big data: The next frontier for innovation, competition and productivity[END_REF]. A neural network is one such technique that allows them to run regression analysis or classification to establish diagnostic or prognostic. To facilitate the representation and manipulation of neural networks, we propose a neural network meta-model. We introduce the specification of this meta-model and an algorithm describing the manipulation of the neural network to generate predictive models. The algorithm illustrates how a neural network model, created using the meta-model, is used as input with data to generate a predictive model in a standard or neutral format. Using the algorithm, a manufacturer can apply neural networks on a manufacturing system without much data science knowledge. The described use case shows an application of the algorithm to one particular system, a milling process.

We also describe a full workflow to apply neural networks on a manufacturing system from a manufacturing model. Our current work contributes to this workflow through the generation of NNMs using the NNMM using an algorithm to manipulate this NNM. To allow manufacturers to automatically generate neural network models, an algorithm should be able to transform a manufacturing model (that can be a simulation model, a mathematical model, etc.) into a neural network model using the neural network meta-model as described in the referenced framework.

Future work includes (1) providing a manufacturing metamodel to represent manufacturing systems at different levels and (2) designing and implementing a step by step process to transform a manufacturing model into a neural network model to develop a manufacturing model interpreter as described in Fig. 2. The first task requires defining concepts at different manufacturing levels. Classifications are already available at the process level such as in [START_REF] Todd | Manufacturing processes reference guide[END_REF]. By taking advantage of the available classifications, one can define a manufacturing metamodel to represent processes, machines, factories, etc. The second task requires defining the steps to transform a manufacturing model into a neural network model. This second task requires one to identify the type and the target of the analysis, and the manufacturing knowledge represented in the manufacturing model. The algorithm designed for the second task will need to define the inputs and outputs of the neural network depending on the scenario. Extracting information using a neural network model will be different at the process level than it is at the factory assembly line level because of the complexity. The number of hidden layers and hidden neurons will change from one level to the next. Research has been done to define the most appropriate structure (i.e., the number of hidden layers and hidden neurons) of an NN and can be implemented in the algorithm developed in the second task [START_REF] Sk | Review on methods to fix number of hidden neurons in neural networks[END_REF].Achieving these two tasks seems promising to obtain a fully automated workflow to apply neural networks for manufacturing systems. Finally, the workflow can be generalized by including other meta-models that represent other techniques for applying data analytics. Developing these metamodels and the required algorithms to ensure the transformation between models will provide a full framework for predictive analytics in manufacturing as described in [START_REF] Lechevalier | Towards a domainspecific framework for predictive analytics in manufacturing[END_REF].

Fig. 1 .

 1 Fig. 1. Example of a MLP NN for estimating a quantity of interest

Fig. 2 .

 2 Fig. 2. Workflow to automatically apply data analytics on a manufacturing system

Fig. 3 .

 3 Fig. 3. Neural Network Meta-Model

Fig. 4 .

 4 Fig. 4. Process flow to generate a predictive model from an NNM and data

Fig. 5 .

 5 Fig. 5. NNM to study the energy consumed by a milling machine tool The input layer contains three input neurons representing the three parameters given as inputs: feed rate, spindle speed and depth of cut. The input layer includes a bias neuron called BiasNeuron1. The hidden layer including the bias neuron called BiasNeuron2 is represented. Finally the output neuron represents the power prediction.

Fig. 6 .

 6 Fig. 6. Interface to provide the needed inputs to the algorithm As mentioned in the previous section, the user does not have to change the parameters of the neural network training and can use the suggested values for the training. Data has to be provided to the algorithm to enable the training. The user can decide the format (with two options, PMML and PFA) used to export the predictive model that the algorithm generates. By clicking on the appropriate button from the GUI, the user launches the first task described in Section III.B. Following the process described in Section III.B, the algorithm trains a neural network as specified by the NNM. The algorithm updates the weight of every edge inside the NNM in GME as shown in Fig. 7. Finally, it creates a predictive model file in the requested format.

Fig. 7 .

 7 Fig. 7. Example of edge weight instantiation

Acknowledgement This work was supported by National Institute of Standards and Technology's Foreign Guest Researcher Program and the Ecole Supérieure d'Electricité (SUPELEC), sponsor of Ronay Ak, and the NIST Student Undergraduate Research Fellowship program.

We thank our colleagues Anantha Narayanan and Sudarsan Rachuri from NIST who provided insight and expertise that greatly assisted the research. We also thank our colleagues KC Morris, Albert Jones and Sharon Kemmerer for their valuable comments that helped to improve the quality of the paper.


David Lechevalier
email: david_lechevalier@etu.u-bourgogne.fr

Anantha Narayanan
email: anantha@umd.edu

Sudarsan Rachuri
email: sudarsan.rachuri@hq.doe.gov

Sebti Foufou
email: sfoufou@qu.edu.qa

Y Tina Lee

Model-based Engineering for the Integration of Manufacturing Systems with Advanced Analytics

Keywords: Data analytics, meta-model, neural network, manufacturing process, predictive modeling 1

To employ data analytics effectively and efficiently on manufacturing systems, engineers and data scientists need to collaborate closely to bring their domain knowledge together. In this paper, we introduce a domain-specific modeling approach to integrate a manufacturing system model with advanced analytics, in particular neural networks, to model predictions. Our approach combines a set of meta-models and transformation rules based on the domain knowledge of manufacturing engineers and data scientists. Our approach uses a model of a manufacturing process and its associated data as inputs, and generates a trained neural network model as an output to predict a quantity of interest. This paper presents the domain-specific knowledge that the approach should employ, the formal workflow of the approach, and a milling process use case to illustrate the proposed approach. We also discuss potential extensions of the approach.

Introduction

The manufacturing industry generates large amounts of data on products, processes, and resources, among other things. Data analytics provide the capabilities needed to extract insights and make predictions from these data. The potential impacts of data analytics on manufacturing-systems efficiency include a reduction of production cost and time across all manufacturing levels [START_REF] Manyika | Big data: The next frontier for innovation, competition, and productivity[END_REF][START_REF] Brown | Are you ready for the era of 'big data[END_REF]. Data scientists and manufacturing engineers often collaborate when using data analytics to solve process-specific problems to improve product quality [START_REF] Erzurumlu | Comparison of response surface model with neural network in determining the surface quality of moulded parts[END_REF][START_REF] Zhai | Feature extraction using rough set theory and genetic algorithms-an application for the simplification of product quality evaluation[END_REF], equipment efficiency [START_REF] Dabbas | Mining semiconductor manufacturing data for productivity improvement-an integrated relational database approach[END_REF][START_REF] Chien | A data mining approach for analyzing semiconductor MES and FDC data to enhance overall usage effectiveness (OUE)[END_REF], and resource efficiency [START_REF] Shin | Predictive analytics model for power consumption in manufacturing[END_REF][START_REF] Gupta | Energy sensitive machining parameter optimisation[END_REF]. However, these collaborations require a significant amount of time and effort to merge the expertise from these two domains. In [START_REF] Lechevalier | Towards a domainspecific framework for predictive analytics in manufacturing[END_REF], the authors present a domain-specific framework to address this challenge. The framework 1) identifies the main components and interfaces that must be implemented to improve communication between these domains and 2) facilitates the application of data analytics in manufacturing. In this paper, we introduce an implementation of some of the components and interfaces that will be a part of this framework.

Our approach focuses on using data analytics -specifically neural networks (NNs) -for predicting a set of manufacturing-process-related performance metrics. There are three main contributions of this paper. First, we provide meta-models to represent manufacturing processes and NNs. Second, we describe an algorithm to generate a trained NN automatically from a manufacturing process model and data. Third, we discuss a tool to export the NN in two standard formats: the Predictive Model Markup Language (PMML) [START_REF][END_REF] and the Portable Format for Analytics (PFA) [11].

The paper is organized as follows. Section 2 presents the domain-specific knowledge required from the manufacturing and data-science domains to generate NNs for manufacturing processes. It also introduces the approach to generate NNs automatically. Section 3 describes, in more detail, two components of the proposed approach: a manufacturing meta-model and transformation rules to generate an NN. Section 4 presents a process-level manufacturing use case to illustrate the capabilities of the approach.

Domain Specific Knowledge from Neural Networks and Manufacturing Processes

In this section, we discuss the knowledge required from manufacturing engineers and data scientists to apply NNs to manufacturing processes. We review applications of NNs in manufacturing processes, and devise a methodology based on the common practice of data scientists.

Manufacturing Domain Knowledge

To identify the required manufacturing-domain knowledge, we studied several research efforts on the applications of data analytics (DA) to manufacturing processes. In [START_REF] Wolbrecht | Monitoring and diagnosis of a multistage manufacturing process using Bayesian networks[END_REF], the authors apply analytics to detect faults in the alignment of a cap to the base part of a product. In [START_REF] Correa | A Bayesian network model for surface roughness prediction in the machining process[END_REF], [START_REF] Abouelatta | Surface roughness prediction based on cutting parameters and tool vibrations in turning operations[END_REF], and [START_REF] Tsai | An in-process surface recognition system based on neural networks in end milling cutting operations[END_REF], the authors predict product quality using three DA algorithms: Bayesian networks (BNs), linear regression, and NNs. In [START_REF] Kurz | Dynamic maintenance in semiconductor manufacturing using Bayesian networks[END_REF], the authors describe a way to predict the need for equipment repair using BNs. In [START_REF] Tsai | An in-process surface recognition system based on neural networks in end milling cutting operations[END_REF], the authors used NNs to study surface roughness in a milling process. They identified surface roughness as the performance metric of interest. They also identified spindle speed, feed rate, depth of cut, and the vibration average per revolution as the process variables that have the most impact on surface roughness. They collected 492 data samples to train and validate the NN. Each application followed a similar workflow: 1) identify the performance metric to be studied, 2) identify the variables that impact this target quantity, and 3) use test data to build an analytical model to predict the performance metric from the process variables. We used this same workflow in our work.

Data Science Domain Knowledge

To understand the knowledge required from a data scientist to apply data analytics techniques to build an NN, it is important to understand how an NN is built. Figure 1 presents the main elements and the structure of an NN. An NN is composed of an input layer, zero or more hidden layers, and an output layer. Each layer contains at least one neuron. All layers except the output layer contain a bias neuron (shown in black). Weighted edges fully connect neurons in different layers. From a mathematical viewpoint, NNs can be viewed as a set of nonlinear basis functions (the activation functions), with free parameters (the adjustable weights). Training the NN is about adjusting the weights to minimize the error between the output value of the NN and the known, real, output value for a given data sample [START_REF] Haykin | Neural Network A comprehensive foundation[END_REF].

As noted above, the first step in building the NN involves selecting the input variables relevant to the performance metric. This step is called feature selection and defines the number of input neurons of the NN. There is one input neuron for each input variable. The second step is to determine the number of hidden layers and the number of neurons in each layer. In general, one hidden layer is sufficient [START_REF] Heaton | Introduction to neural networks with Java[END_REF] for the class of problems related to manufacturing processes. The number of hidden neurons has an impact on the NN accuracy, thus data scientists define this number very carefully. Finally, the output neuron represents the variable that we are trying to predict, which we call the quantity of interest. For example, a performance metric such as energy consumption may be the quantity of interest in a manufacturing scenario.

Based on these reviews, our approach needs to define the input neurons based on the process variables, define the optimal number of hidden neurons for an NN with one hidden layer, and finally define the output neuron for the quantity of interest.

Integration of Manufacturing and Data Science Domain Models

After identifying the required knowledge from manufacturing engineers and data scientists, we describe our approach and how it contributes to the framework defined in [START_REF] Lechevalier | Towards a domainspecific framework for predictive analytics in manufacturing[END_REF]. Figure 2 summarizes the workflow of our approach. In this figure, meta-models (Ⓐ and Ⓑ) are in gray, models (,  and ) are in yellow, and software solutions (,  and ) are in blue. The dashed arrows represent actions defined in the related label. The solid arrows show the use of models as input or output of the software solutions.

Figure 1. Structure of a Neural Network

Box Ⓐ represents our manufacturing meta-model that captures the manufacturing knowledge. This meta-model defines the concepts, rules and constraints needed to represent a manufacturing process. Using the meta-model, a manufacturing engineer is able to build a manufacturing process model  to define the quantities of interest and the variables involved in the manufacturing process. Note, we provide an interface to collect data related to the manufacturing process.

Taking the manufacturing process model and data as inputs, an NN model builder  embeds a set of algorithms to run a feature selection that 1) optimizes the number of input neurons, 2) computes the optimal number of hidden neurons, and 3) builds the optimal structure of the NN . This NN structure is recorded using an NN meta-model contained in the meta-model repository. The NN meta-model and NN model interpreter are presented in [START_REF] Lechevalier | A neural network meta-model and its application for manufacturing[END_REF]. The NN model interpreter  generates a trained NN. This NN is exported as a PMML or PFA file  that is ready to use for prediction with new data. A scoring engine  provides predictions  using the PMML file and new data. Scoring is the process of using a model to make predictions about the behavior of a quantity of interest. A manufacturing engineer makes decisions based on these predictions to control the manufacturing process under investigation.

Manufacturing Meta-Models and Transformation Rules of the Neural Network Builder

In this section, we describe the components, Ⓐ, Ⓑ and  in Figure 2, to generate NNs from manufacturing process descriptions automatically. We also describe our implementations of these components.

Figure 2. Formal workflow of the approach

Meta-Model for Manufacturing Processes

A meta-model is a graphical description of concepts and their relationships, which can be used to describe objects or instances of those concepts in a particular domain. We developed a meta-model for describing manufacturing processes in a way that is helpful to build an NN. A manufacturing engineer builds a manufacturing model using the meta-model to provide the required knowledge identified in Section 2.1. Since the purpose of the approach is to use data-driven techniques (in this case NNs), there are no physics-based equations associated with the model. Figure 3 presents the main concepts of the manufacturing meta-model. Please note that this is a simple but a reasonable representation of the domain model. The notation in Figure 3 and Figure 4 is based on Unified Modeling Language Class Diagrams [START_REF]Unified Modeling Language[END_REF], where the rectangles represent concepts occurring in the domain, and the lines represent relationships between the concepts. A line with a solid diamond represents a containment relationship, with a numerical range at one end denoting the number of allowed instances. For example, in Figure 3, a ManufacturingModel can contain 0 or more instances of ManufacturingProcess.

The annotation <<Model>> is used to identify first class objects, while the annotation <<Connection>> is used to represent edges, flows, or associations.

ManufacturingModel is a high level concept that allows the description of a manufacturing model that is composed of Flows and ManufacturingProcess concepts.

The Flow concept represents connections between instances of the ManufacturingProcess concept. A ManufacturingProcess is composed of Resource and Equipment concepts, which allow the manufacturer to include resource or equipment parameters as variables of the manufacturing process. ManufacturingProcess also contains the concepts of Parameter and Metric. Metric is used to define a quantity of interest in the manufacturing process. Parameters are the variables that can impact the metric for a manufacturing process.

Figure 3. Manufacturing meta-model

In the UML notation, an empty triangle is used to denote specialization, where one concept may be specialized into many sub-concepts. As shown in Figure 3, the Resource concept is extended to define different types of resources: energy, water, and material. The manufacturing meta-model can also be extended to define other kinds of resources such as labor.

Meta-Model for Neural Networks

Figure 4 shows the neural network meta-model (NNMM) presented in [START_REF] Lechevalier | A neural network meta-model and its application for manufacturing[END_REF]. The NNMM represents different types of NNs through various abstractions. A NeuralNetworkModel concept is composed of Neuron and Edge concepts. A Neuron can be one of four types: InputNeuron, HiddenNeuron, BiasNeuron, and OutputNeuron. An Edge can be a VisibleEdge or a HiddenEdge. A VisibleEdge is used to represent an edge between an input neuron and a hidden neuron, between a hidden neuron and an output neuron, or between a bias neuron and an output neuron. Edges between two hidden neurons, or between a bias neuron and a hidden neuron are represented using HiddenEdge.

Transformation Rules to Generate an NN from a Manufacturing Model

We developed a set of transformation rules to generate an NN model from a manufacturing model. Together, these rules represent a step-by-step process to build an NN from the input model and the data provided by a manufacturing engineer. We embedded these rules into the NN model builder, so that they can be applied to any type of manufacturing process model. The result of applying these rules is an untrained NN model, which is built based from the NN meta-model described above, and an input The NN model builder takes the manufacturing process model and data provided by the manufacturing engineer as inputs. In the builder, Step 1 identifies those variables that the manufacturer listed as impacting the quantity of interest in the manufacturing model. The identified variables are compared with the variables present in the data set. The variables that are not identified in the manufacturing model are then removed. Step 1 takes advantage of the manufacturing expertise that the manufacturing engineer provides in the model.

Step 2 uses the feature-selection algorithm and a real data set to identify those variables that do not contribute to the quantity of interest based on a data set. This algorithm takes the variables provided from Step 1 and removes the variables that do not contribute from the list.

During Step 3, the builder runs an algorithm to optimize the number of hidden neurons for the NN. Several reports document the studies associated with optimizing the number of hidden neurons and putting them all into a single hidden layer. Sheela et al. [START_REF] Sheela | Review on methods to fix number of hidden neurons in neural networks[END_REF], for example, analyzes the performance of the different optimization methods described in different reports -that is, their ability to predict the actual optimal number of hidden neurons. Our algorithm applies these different methods and computes the number that appeared most frequently. That number is the one selected for the NN. As we mentioned previously, one hidden layer is sufficient for manufacturing processrelated problems, and the algorithm is implemented to build one hidden layer. This algorithm, however, can easily be modified to build NNs with more than one hidden layer.

In Step 4, the builder generates the NN instance model and a data set as outputs. The NN instance model describes the structure of the NN, and must be trained in order to predict the quantity of interest. The output data set is a subset of the input data set. The input data set variables that do not impact the quantity of interest are not included in the output data set.

Using the output data set, the NN model interpreter [START_REF] Lechevalier | A neural network meta-model and its application for manufacturing[END_REF] performs the NN training and updates the weights on the NN instance model. It also generates a PMML or PFA file containing the trained NN model.

Use Case

In this section, we show how our implementation of the proposed approach is used in a typical manufacturing scenario. For our approach, manufacturing engineers build Figure 5. Workflow of the NN model builder a model of the process they wish to study using the meta-model described earlier. Next, they collect test data by conducting experiments or from other sources. Finally, they use the automated tools described in Figure 2 to generate an NN for their process. This NN can be used to make future decisions without having to conduct physical experiments to determine target values.

Scenario Description

This case study focuses on predicting the energy consumed by a milling machine tool. The data set used in this case study was generated in [START_REF] Park | A generalized data-driven energy prediction model with uncertainty for a milling machine tool using Gaussian Process[END_REF] from a total of 18 parts machined with 196 face milling, 108 contouring, 54 slotting and pocketing, 16 spiraling and 32 drilling operations. We focused on face milling in this use case. The series of machining operations were performed. Data was collected using power meters and different sensors, and then stored in a database. We use the collected data as test data to build an NN model to predict power consumption for different combinations of machining parameters.

The test data includes the timestamp, power demand, feed rate, spindle speed, depth of cut, cutting direction, cutting strategy, cutting ratio, cutting volume, and length of cut in the 3 axis, referred as cutX, cutY and cutZ. As described below, we first built a manufacturing process model based on our case study, then identified the optimal process parameters and the metric of interest.

Building the Manufacturing Process Model

Figure 6 shows the manufacturing process model that we built for the milling process. This model contains the parameters that the manufacturing engineer has specified as contributing to the quantity of interest. In this model, we defined power as the quantity of interest. We defined feed rate, spindle speed, depth of cut, cutting ratio, cutting volume, cutX, cutY, and cutZ as parameters that impact the power consumption. During this step, the manufacturers use their domain expertise to list only those parameters that they think will have a significant impact on their power consumption. The test data and the manufacturing model are the inputs to our next step, which performs feature selection and generates the NN.

Generating the Neural Network for Prediction

In the next step, the manufacturing engineer executes the NN model builder using both the manufacturing model (in Figure 6) and the test data as inputs. Our algorithm, The NN model builder identifies the quantity of interest (the selected performance metric) from the manufacturing model. In this case, it is power. The NN model builder prunes the data set by removing the data that were omitted in the manufacturing model. In our case, it removes the cutting direction and cutting strategy variables from the data set since these are not present in the manufacturing model in Figure 6. Next, the feature selection algorithm is executed. It uses the test data to identify and remove parameters that have an insignificant impact on the target variable. In our example, feed rate, depth of cut, cutX, and cutY were found not to have a significant effect on power; therefore, these parameters are not considered when building the NN.

The NN model builder then displays which variables were removed 1) based on the manufacturing model and 2) using the feature selection algorithm. Then the builder saves the new data set in a location identified by the manufacturing engineer. Figure 7 shows the resulting NN model, an automatically generated instance of the NNMM which is shown in Figure 4.

In this NN model, the NN model builder keeps four variables: spindle speed, cutting ratio, cutting volume and cutZ. They are defined as input neurons in the NN. The algorithm computes that two hidden neurons are optimal in this model. Power is defined as the output neuron in the NN. Finally, the builder adds a bias neuron for every layer except the output layer to build a correct NN.

The NN model builder generates the structure model the NN. It still needs to be trained (i.e. weights must be assigned to the edges to make correct predictions). To generate the weights, the structural NN must be trained with the test data. The NN model and the test data are inputs to the NN model interpreter. The interpreter generates a trained NN based on the structure described in the NN model and the test data. The NN is generated as a standard PMML file. Several off-the-shelf data analytics tools can read this PMML file.

The manufacturer can now use this NN to predict the energy consumption of the milling machine under different conditions. This allows the manufacturer to perform different tests and make decisions, without having to physically execute experiments on the machine.

Summary and Future Work

In this paper, we proposed an approach to generate an NN to predict performance metrics for manufacturing processes. This approach provides capabilities to collect the required manufacturing knowledge and to use that knowledge to build NN models to We first reviewed the applications of data analytics to manufacturing processes for identifying the steps taken by data scientists to create NNs. We then developed and implemented the components needed to provide the capabilities required by this approach. Part of that approach is developing a manufacturing meta-model. The metamodel allows manufacturing engineers to provide a set of the most important process parameters -those have the most impact on performance -in a manufacturing model. In addition to this meta-model, we implemented an NN model builder to automatically build an NN model from a manufacturing model and data provided by manufacturing engineers. The NN model builder provides 1) a feature-selection algorithm based on the test data and 2) an NN model generator that generates the structure of the NN. From the generated NN structure, an NN model interpreter produces a trained NN in a standard format. Using a scoring engine, the trained NN can then be used to predict the quantity of interest.

We illustrated the capabilities of our implementation using a realistic manufacturing scenario. In this scenario, an NN is trained to predict energy use during a particular milling process. A manufacturing engineer provides a manufacturing model used as input to the NN builder. The implemented algorithms finally generate a trained NN that can be used with new data for predicting energy consumption.

This paper presented an initial description and implementation of an approach to generate predictive models for manufacturing applications. We implemented a translator (the NN model builder) to generate neural networks automatically. More translators will be implemented in future work to generate other types of predictive models. In practice, manufacturing processes and their interactions with their surrounding environment are complex. In order to generate reliable prediction models for practical scenarios, our meta-models and translators must be extended to account for other parameters and constraints that affect manufacturing processes. Future work lies in four directions. The first is to extend the manufacturing meta-model to enable the representation of problems in greater detail, and at different manufacturing levels such as assembly. Next, add new steps to the NN model builder to improve its accuracy. Third, include a scoring engine. Fourth, extend the framework to include different analytical techniques such as Bayesian networks. Capabilities to build BN models could enable the application of uncertainty quantification in manufacturing [START_REF] Nannapaneni | Uncertainty quantification in performance evaluation of manufacturing processes[END_REF].

Figure 4 .

 4 Figure 4. Neural network meta-model [19]

Figure 5

 5 presents the workflow and the transformation rules of the NN model builder, identified as  in Figure 2.

Figure 6 .

 6 Figure 6. Manufacturing model

Figure 7 .

 7 Figure 7. Neural network model

Acknowledgement

The research in this paper was supported by National Institute of Standards and Technology's Foreign Guest Researcher Program, and Cooperative Agreement No. 70NANB14H250.