Ouassim Bara
email: obara@utk.edu

Michel Fliess
email: michel.fliess@polytechnique.edu

Cédric Join
email: cedric.join@univ-lorraine.fr

Judy Day
email: judyday@utk.edu

Seddik M Djouadi
email: mdjouadi@utk.edu

Toward a model-free feedback control synthesis for treating acute inflammation A mathematical perspective

Keywords: Immune systems, Inflammatory response, Model-free control, Intelligent controllers

published or not. The documents may come

Toward a model-free feedback control synthesis for treating acute inflammation

Introduction

Inflammation is a key biomedical subject (see, e.g., [START_REF] Nathan | Points of control in inflammation[END_REF][START_REF] Vodovotz | Solving immunology?[END_REF]) with fascinating connections to diseases like cancer (see, e.g., [START_REF] Balkwill | Inflammation and cancer: back to Virchow?[END_REF]), AIDS (see, e.g., [START_REF] Deeks | HIV infection, inflammation, immunosenescence, and aging[END_REF]) and psychiatry (see, e.g., [START_REF] Miller | The role of inflammation in depression: from evolutionary imperative to modern treatment target[END_REF]). Mathematical and computational models investigating these biological systems have provided a greater understanding of the dynamics and key mechanisms of these processes (see, e.g., [START_REF]Complex Systems and Computational Biology Approaches to Acute Inflammation[END_REF]). The very content of this particular study leads citations of mathematical models where differential equations play a prominent role (see, e.g., [START_REF] Arazi | Modeling immune complex-mediated autoimmune inflammation[END_REF][START_REF] Arciero | Using a mathematical model to analyze the role of probiotics and inflammation in necrotizing enterocolitis[END_REF][START_REF] Asachenkov | Disease Dynamics[END_REF][START_REF] Barber | A three-dimensional mathematical and computational model of necrotizing enterocolitis[END_REF][START_REF] Day | Modeling the immune rheostat of macrophages in the lung in response to infection[END_REF][START_REF] Day | Mathematical modeling of early cellular innate and adaptive immune responses to ischemia/reperfusion injury and solid organ allotransplantation[END_REF][START_REF] Day | A reduced mathematical model of the acute inflammatory response II. Capturing scenarios of repeated endotoxin administration[END_REF][START_REF] Russo | A mathematical model of inflammation during ischemic stroke[END_REF][START_REF] Dunster | The resolution of inflammation: A mathematical model of neutrophil and macrophage interactions[END_REF][START_REF] Eftimie | Mathematical models for immunology: Current state of the art and future research directions[END_REF][START_REF] Ho | A model of neutrophil dynamics in response to inflammatory and cancer chemotherapy challenges[END_REF][START_REF] Kumar | The dynamics of acute inflammation[END_REF][START_REF] Mathew | Global sensitivity analysis of a mathematical model of acute inflammation identifies nonlinear dependence of cumulative tissue damage on host interleukin-6 responses[END_REF][START_REF] Perelson | Immunology for physicists[END_REF][START_REF] Prodanov | A model of space-fractional-order diffusion in the glial scar[END_REF][START_REF] Reynolds | Mathematical Models of Acute Inflammation and Full Lung Model of Gas exchange under inflammatory stress[END_REF][START_REF] Reynolds | A mathematical model of pulmonary gas exchange under inflammatory stress[END_REF][START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF][START_REF] Song | Ensemble models of neutrophil trafficking in severe sepsis[END_REF][START_REF] Torres | Mathematical modelling of posthemorrhage inflammation in mice: Studies using a novel, computer-controlled, closed-loop hemorrhage apparatus[END_REF][START_REF] Yiu | Dynamics of a cytokine storm[END_REF]). The usefulness of those equations for simulation, prediction purposes, and, more generally, for understanding the intimate mechanisms is indisputable. In addition, some models have also been used in order to provide a real-time feedback control synthesis (see, e.g., [START_REF] Åström | Feedback Systems: An Introduction for Scientists and Engineers[END_REF] for an excellent introduction to this important engineering topic) for treating acute inflammation due to severe infection. Insightful results were obtained via two main model-based approaches:

optimal control [START_REF] Bara | Optimal control of an inflammatory immune response model[END_REF][START_REF] Bara | Immune Therapy using optimal control with L 1 type objective[END_REF][START_REF] Bara | Immune therapeutic strategies using optimal controls with L 1 and L 2 type objectives[END_REF][START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF][START_REF] Stengel | Stochastic optimal therapy for enhanced immune response[END_REF][START_REF] Stengel | Optimal enhancement of immune response[END_REF][START_REF] Stengel | Optimal control of innate immune response[END_REF][START_REF] Tan | Optimal control strategy for abnormal innate immune response[END_REF], model predictive control [START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF][START_REF] Hogg | Acute inflammation treatment via particle filter state estimation and MPC[END_REF][START_REF] Radosavljevic | A data-driven acute inflammation therapy[END_REF][START_REF] Zitelli | Combining robust state estimation with nonlinear model predictive control to regulate the acute inflammatory response to pathogen[END_REF].

Our work in [START_REF] Bara | Optimal control of an inflammatory immune response model[END_REF][START_REF] Bara | Immune Therapy using optimal control with L 1 type objective[END_REF][START_REF] Bara | Immune therapeutic strategies using optimal controls with L 1 and L 2 type objectives[END_REF][START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF][START_REF] Zitelli | Combining robust state estimation with nonlinear model predictive control to regulate the acute inflammatory response to pathogen[END_REF] made use of the low dimensional system of ordinary differential equations (ODE) derived in [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF] (see also [START_REF] Day | A reduced mathematical model of the acute inflammatory response II. Capturing scenarios of repeated endotoxin administration[END_REF]). This four variable model possesses the following characteristics:

-The model is based on biological first principles, the non-specific mechanisms of the innate immune response to a generic gram-negative bacterial pathogen. -A variable representing anti-inflammatory mediators e.g. Interleukin-10, Transforming Growth Factor-β ) is included and plays an important role in mitigating the negative effects of inflammation to avoid excessive tissue damage.

-Though a qualitative model of acute inflammation, it reproduces several clinically relevant outcomes: a healthy resolution and two death outcomes.

The calibration of a system of differential equations can be quite difficult since the identification of various rate parameters requires specific data in sufficient quantities, which may not be feasible. Additionally, there is much heterogeneity to account for between patient responses such as the initiating circumstances, patient co-morbidities and personal characteristics, like genetics, age, gender, . . . . In spite of promising preliminary results in [START_REF] Bara | Nonlinear state estimation for complex immune responses[END_REF][START_REF] Bara | Parameter estimation for nonlinear immune response model using EM[END_REF][START_REF] Zitelli | Combining robust state estimation with nonlinear model predictive control to regulate the acute inflammatory response to pathogen[END_REF], state estimation and parameter identification of highly nonlinear models may still require more data than can be reasonably collected. These roadblocks hamper the use of model-based control strategies in clinical practice in spite of recent mathematical advances. Here, another route, i.e, model-free control (MFC) and the corresponding "intelligent" feedback controllers [START_REF] Fliess | Model-free control[END_REF], are therefore explored. 1 We briefly introduce the method before discussing its application to the scenario of controlling the inflammatory response to pathogen. We begin by replacing the poorly known global description by the ultra-local model given by:

ẏ = F + αu, (1)
where the control and output variables are u and y, respectively; the derivation order of y is 1 like in most concrete situations; α ∈ R is chosen by the practitioner such that αu and ẏ are of the same magnitude; -F is estimated via the measurements of u and y; -F subsumes not only the unknown system structure but also any perturbation.

Remark 1

The following comparison with computer graphics is borrowed from [START_REF] Fliess | Model-free control[END_REF].

To produce an image of a complex curve in space, the equations defining that curve are not actually used but, instead an approximation of the curve is made with short straight line segments. Equation ( 1), which might be viewed as an analogue of such a segment, should hence not be considered as a global description but instead as a rough linear approximation.

Remark 2

The estimation of the fundamental quantity F in Equation ( 1) via the control and output variables u and y will be detailed in Section 2.2. It connects our approach to the data-driven viewpoint which has been adopted in control engineering (see, e.g., [START_REF] Formentin | A comparison of model-based and data-driven controller tuning[END_REF][START_REF] Hou | From model-based control to data-driven control: Survey, classification and perspective[END_REF][START_REF] Roman | Multi-input multi-output system experimental validation of model-free control and virtual reference feedback tuning techniques[END_REF][START_REF] Roman | Data-driven model-free adaptive control tuned by virtual reference feedback tuning[END_REF]) and in studies about inflammation (see, e.g., [START_REF] Azhar | Integrating data-driven and mechanistic models of the inflammatory response in sepsis and trauma[END_REF][START_REF] Brause | Data driven automatic model selection and parameter adaptation -a case study for septic shock[END_REF][START_REF] Radosavljevic | A data-driven acute inflammation therapy[END_REF][START_REF] Vodovotz | Computational modelling of the inflammatory response in trauma, sepsis and wound healing: implications for modelling resilience[END_REF]).

Ideally, data associated with the time courses of the inflammatory response variables would be generated by measurements from real patients. In our case, it would be the pro-inflammatory and anti-inflammatory variables of the model (patient) which we would want to track; and therefore, define these as the reference trajectories (available data) for the model-free setup. Once the quantity F est is obtained, the loop is closed by an intelligent proportional controller, or iP:

u = - F est -ẏ * + K P e α , (2)
where -F est is an estimate of F; y is the reference trajectory; e = yy is the tracking error; and -K P is an usual tuning gain.

With a "good" estimate F est of F, i.e., F -F est 0, Equations ( 1)-(2) yield ė + K P e = F -F est 0

1 This new viewpoint in control engineering has been successfully illustrated in many concrete casestudies (see, e.g., the references in [START_REF] Fliess | Model-free control[END_REF], and [START_REF] Abouaïssa | Energy saving for building heating via a simple and efficient model-free control design: First steps with computer simulations[END_REF][START_REF] Abouaïssa | On ramp metering: Towards a better understanding of ALINEA via model-free control[END_REF][START_REF] Agee | Tip trajectory control of a flexible-link manipulator using an intelligent proportional integral (iPI) controller[END_REF][START_REF] Agee | Intelligent proportional-integral (iPI) control of a single link flexible joint manipulator[END_REF][START_REF] Bara | Model-free load control for high penetration of solar photovoltaic generation[END_REF][START_REF] Chand | Non-linear model-free control of flapping wing flying robot using iPID[END_REF][START_REF] Join | A simple and efficient feedback control strategy for wastewater denitrification[END_REF][START_REF] Lafont | A model-free control strategy for an experimental greenhouse with an application to fault accommodation[END_REF][START_REF] Li | Direct power control of DFIG wind turbine systems based on an intelligent proportional-integral sliding mode control[END_REF][START_REF] Madoński | Model-free control of a two-dimensional system based on uncertainty reconstruction and attenuation[END_REF][START_REF] Menhour | An efficient modelfree setting for longitudinal and lateral vehicle control. Validation through the interconnected pro-SiVIC/RTMaps prototyping platform[END_REF][START_REF] Michel | Commande "sans modèle" pour l'asservissement numérique d'un banc de caractérisation magnétique[END_REF][START_REF] Michel | Model-free based digital control for magnetic measurements[END_REF][START_REF] Mohammadridha | Model free iPID control for glycemia regulation of type-1 diabetes[END_REF][START_REF] De Miras | Active magnetic bearing: A new step for model-free control[END_REF][START_REF] Rodriguez-Fortun | Model-free control of a 3-DOF piezoelectric nanopositioning platform[END_REF][START_REF] Schwalb Moraes | Model-free control of magnetic levitation systems through algebraic derivative estimation[END_REF][START_REF] Tebbani | Model-based versus model-free control designs for improving microalgae growth in a closed photobioreactor: Some preliminary comparisons[END_REF][START_REF] Ticherfatine | Model-free approach based intelligent PD controller for vertical motion reduction in fast ferries[END_REF][START_REF] Wang | ZMP theory-based gait planning and model-free trajectory tracking control of lower limb carrying exoskeleton system[END_REF][START_REF] Wang | Event-driven model-free control in motion control with comparisons[END_REF][START_REF] Wang | Model-free based terminal SMC of quadrotor attitude and position[END_REF][START_REF] Xu | Robustness study on the model-free control and the control with restricted model of a high performance electro-hydraulic system[END_REF][START_REF] Yaseen | Attack-tolerant networked control system: an approach for detection the controller stealthy hijacking attack[END_REF][START_REF] Al-Younes | Robust model-free control applied to a quadrotor UAV[END_REF][START_REF] Zhou | Model-free deadbeat predictive current control of a surfacemounted permanent magnet synchronous motor drive systems[END_REF]). Some of the methods have been patented and some have been applied to life sciences [START_REF] Fliess | Dynamic compensation and homeostasis: a feedback control perspective[END_REF][START_REF] Join | A simple and efficient feedback control strategy for wastewater denitrification[END_REF][START_REF] Lafont | A model-free control strategy for an experimental greenhouse with an application to fault accommodation[END_REF][START_REF] Mohammadridha | Model free iPID control for glycemia regulation of type-1 diabetes[END_REF][START_REF] Tebbani | Model-based versus model-free control designs for improving microalgae growth in a closed photobioreactor: Some preliminary comparisons[END_REF].

Thus e(t) e(0) exp(-K P t), which implies that lim t→+∞ y(t) y (t) if and only if, K P > 0. In other words, the scheme ensures an excellent tracking of the reference trajectory. This tracking is moreover quite robust with respect to uncertainties and disturbances which can be numerous in a medical setting such as considered here. This robustness feature is explained by the fact that F in Equation (1) encompasses "everything," without trying to distinguish between its different components. In our application, sensorless outputs must be driven in order to correct dysfunctional immune responses of the patient. Here, this difficult problem is solved by assigning suitable reference trajectories to those systems variables which can be measured. This feedforward viewpoint is borrowed from the flatness-based control setting [START_REF] Fliess | Flatness and defect of non-linear systems: introductory theory and examples[END_REF] (see also [START_REF] Åström | Feedback Systems: An Introduction for Scientists and Engineers[END_REF][START_REF] Lévine | Analysis and Control of Nonlinear Systems -A flatness-based approach[END_REF][START_REF] Sira-Ramírez | Differentially Flat Systems[END_REF]).

After justifying model-free control in Section 2, Section 3 presents results from applying the method to a heterogeneous in silico virtual patient population generated in [START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF]. The cohort of virtual patients are summarized in Section 3.1. The computer simulations demonstrate the great robustness of the model-free control strategy with respect to noise corruption, as demonstrated in Section 3.4. Concluding remarks in Section 4 discuss some of the potential as well as the remaining challenges of the approach in the setting of controlling complex immune responses.

A first draft has already been presented in [START_REF] Bara | Model-free immune therapy: A control approach to acute inflammation[END_REF].

2 Justification of the model-free approach: A brief sketch

Justification of the ultra-local model

We first justify the ultra-local model given in [START_REF] Abouaïssa | Energy saving for building heating via a simple and efficient model-free control design: First steps with computer simulations[END_REF]. For notational simplicity, we restrict to a system with a single control variable u and a single output variable y. Assume that the system is a causal, or non-anticipative functional; In other words, for any time instant t > 0, let

y(t) = F (u(τ) | 0 ≤ τ ≤ t) , (3)
where F depends on the past and present but not the future, various perturbations, and initial conditions at t = 0.

Example 1 A representation of rather general nonlinear functionals, also popular in the biological sciences, is provided by a Volterra series (see, e.g., [START_REF] Korenberg | The identification of nonlinear biological systems: Volterra kernel approaches[END_REF]):

y(t) =h 0 (t) + t 0 h 1 (t, τ)u(τ)dτ+ t 0 t 0 h 2 (t, τ 2 , τ 1 )u(τ 2 )u(τ 1 )dτ 2 dτ 1 + . . . t 0 . . . t 0 h ν (t, τ ν , . . . τ 1 )u(τ ν ) . . . u(τ 1 )dτ ν . . . dτ 1 + . . .
Solutions of quite arbitrary ordinary differential equations, related to input-output behaviors, may be expressed as a Volterra series (see, e.g., [START_REF] Fliess | An algebraic approach to nonlinear functional expansions[END_REF]).

Let

-I ⊂ [0, +∞[ be a compact subset and -C ⊂ C 0 (I ) be a compact subset, where C 0 (I ) is the space of continuous functions I → R, which is equipped with the topology of uniform convergence.

Consider the Banach R-algebra S of continuous causal functionals (3)

I × C → R.
If a subalgebra contains a non-zero constant element and separates points in I × C , then it is dense in S according to the classic Stone-Weierstraß theorem (see, e.g., [START_REF] Rudin | Functional Analysis[END_REF]). Let A ⊂ S be the set of functionals which satisfy an algebraic differential equation of the form E(y, ẏ, . . . , y (a) , u, u, . . . ,

u (b) ) = 0, ( 4
)
where E is a polynomial function of its arguments with real coefficients. It has been proven in [START_REF] Fliess | Model-free control[END_REF] that with this, the conditions of the Stone-Weierstraß theorem are satisfied and, therefore, A is dense in S.

Assume therefore that our system is "well" approximated by a system defined by Equation (4). Let ν be an integer, 1 ≤ ν ≤ a, such that

∂ E ∂ y (ν) ≡ 0.
The implicit function theorem yields locally y (ν) = E (y, ẏ, . . . , y (ν-1) , y (ν+1) , . . . , y (a) , u, u, . . . , u (b) ).

This may be rewritten as

y (ν) = F + αu. (5)
In most concrete situations such as the one here, the order ν = 1 of derivation, as in Equation( 1), is enough. See [START_REF] Fliess | Model-free control[END_REF] for an explanation and for some examples where ν = 2.

Closing the loop

If ν = 1 in Equation ( 5), we are back to Equation (1). The loop is closed with the intelligent proportional controller (2).

Estimation of F

Any rather general function [a, b] → R, a, b ∈ R, a < b, may be approximated by a step function F approx , i.e., a piecewise constant function (see, e.g., [START_REF] Rudin | Real and Complex Analysis[END_REF]). Therefore, for estimating a suitable approximation of F in Equation ( 5), the question reduces to the identification of the constant parameter Φ in

ẏ = Φ + αu. (6)
Here a recent real-time algebraic estimation/identification techniques are employed ( [START_REF] Fliess | Closed-loop parametric identification for continuous-time linear systems via new algebraic techniques[END_REF][START_REF] Sira-Ramírez | Algebraic Identification and Estimation Methods in Feedback Control Systems[END_REF]). With respect to the well-known notations of operational calculus (see, e.g., [START_REF] Erdélyi | Operational Calculus and Generalized Functions[END_REF][START_REF] Yosida | Operational Calculus -A Theory of Hyperfunctions[END_REF]) (which are identical to those of the classic Laplace transform taught everywhere in engineering e.g., [START_REF] Doetsch | Introduction to the Theory and Application of the Laplace Transformation (translated from the German)[END_REF], [START_REF] Åström | Feedback Systems: An Introduction for Scientists and Engineers[END_REF]), Equation ( 6) yields:

sY = Φ s + αU + y(0),
where U and Y are the operational analogues of u and y. In the literature, U and Y are often called the Laplace transforms of u and y, and s is the Laplace variable (see, e.g., [START_REF] Åström | Feedback Systems: An Introduction for Scientists and Engineers[END_REF]).

We eliminate the initial condition y(0) by left-multiplying both sides by d ds or, in other words, by differentiating both sides with respect to s:

Y + s dY ds = - Φ s 2 + α dU ds .
The product by s corresponds in the time domain to the derivation with respect to time. Such a derivation is known to be most sensitive to noise corruptions. Therefore, multiply both sides on the left by s -2 in order to replace derivations by integrations with respect to time, which are quite robust with respect to noise (see [START_REF] Fliess | Analyse non standard du bruit[END_REF] for more explanations). Recall that d ι ds ι , where ι ≥ 1 is an integer, corresponds in the time domain to the multiplication by (-t) ι . Then

F est (t) = - 6 τ 3 t t-τ [(τ -2σ )y(σ ) + ασ (τ -σ )u(σ )] dσ , (7)
where τ > 0 might be quite small. This integral may of course be replaced in practice by a classic digital filter.

There are other formulas one can use for obtaining an estimate of F. For instance, closing the loop with the iP (2) yields:

F est (t) = 1 τ t t-τ ( ẏ -αu -K P e) dσ . (8)
Remark 3 Measurement devices are always corrupted by various noise sources (see, e.g., [START_REF] Tagawa | Biomedical Sensors and Instruments[END_REF]). The noise is usually described via probabilistic/statistical laws that are difficult to write down in most concrete situations. Following [START_REF] Fliess | Analyse non standard du bruit[END_REF] where nonstandard analysis is used, the noise is related to quick fluctuations around zero [START_REF] Cartier | Integration over finite sets[END_REF]. Such a fluctuation is a Lebesgue-integrable real-valued time function F which is characterized by the following property: the integral of F over any finite time interval,

τ f τ i F (τ)dτ, is infinitesimal.
Therefore, noise is attenuated thanks to the integrals in formulas ( 7)-( 8).

3 Computer Simulation

Virtual patients

In [START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF], a cohort of virtual patients were defined by using the ODE model of [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF] for the underlying immune response dynamics for each patient and with each differing in the value of six of the rate parameters and two of the initial conditions. This same cohort was used in this study as well. The ODE model describes an abstract dynamical representation of an acute inflammatory response to pathogenic infection:

Ṗ(t) = k pg P(t) 1 - P(t) p ∞ - k pm s m P(t) µ m + k mp P(t) -k pn f (N(t))P(t) (9)
Ṅ(t) = s nr R(P(t), N(t), D(t))

µ nr + R(P(t), N(t), D(t)) -µ n N(t) + u p (t) (10)
Ḋ(t) = k dn f (N(t)) 6 x 6 dn + f (N(t)) 6 -µ d D(t) (11)
Ċa (t) = s c + k cn f (N(t) + k cnd D(t)) 1 + f (N(t) + k cnd D(t)) -µ c C a (t) + u a (t), (12)
where

R(P, N, D) = f (k np P(t) + k nn N(t) + k nd D(t)) and f (x) = x 1 + C a (t) c ∞ 2 .
-Equation ( 9) represents the evolution of the bacterial pathogen population P that causes the inflammation. -Equation ( 10) governs the dynamics of the concentration of a collection of early pro-inflammatory mediators such as activated phagocytes and the pro-inflammatory cytokines produced by N. -Equation ( 11) corresponds to tissue damage (D), which helps to determine response outcomes. -Equation ( 12) describes the evolution of the concentration of a collection of antiinflammatory mediators C a .

As explained in [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF], f (x) represents a Hill function that models the impact of activated phagocytes and their by-products (N) on the creation of damaged tissue. With this modeling construct, tissue damage (D) increases in a switch-like sigmoidal fashion as N increases such that it takes sufficiently high levels of N to incite a moderate increase in damage and that the increase in damage saturates with sufficiently elevated and sustained N levels. The hill coefficient (exponent) 6 was chosen to model this aspect which also ensured that the healthy equilibrium had a reasonable basin of attraction for the N/D subsystem.

For the reference set of parameter values which is given in Table I of [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF], the above model possesses three (positive) stable equilibria which can be qualitatively interpreted as the following clinical outcomes:

-Healthy outcome: equilibrium in which P = N = D = 0 and C a is at a background level. -Aseptic death outcome: equilibrium in which all mediators N, C a , and D are at elevated levels, while pathogen, P, has been eliminated.

-Septic death outcome: equilibrium in which all mediators N, C a , and D together with the pathogen P are at elevated levels (higher than in the aseptic death equilibrium).

Fig. 1 Diagram of the mediators of the acute inflammatory response to pathogen as abstractly modeled in [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF]. Solid lines with arrow heads and dashed lines with nodes/circular heads represent upregulation and inhibition, respectively. P: replicating pathogen, N: early pro-inflammatory immune mediators, D: marker of tissue damage/dysfunction caused by inflammatory response, C a : inhibitory anti-inflammatory mediators, u a and u p : time-varying input controls for the anti-and pro-inflammatory therapy, respectively.

Note that the model was formulated to represent a most abstract form of the complex processes involved in the acute inflammatory response. Hence, as explained in [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF] the variables N and C a represent multiple mediators with similar inflammatory characteristics, and D is an abstract representation of collateral tissue damage caused by inflammatory by-products. This abstraction reduces the description to four essential variables which also allows for tractable mathematical analysis. Therefore, the units of these variables are in arbitrary units of N-units, C a -units, D-units, since they represent various types of cells and thus, they qualitatively, rather than quantitatively, describe the response of the inflammatory mediators and their by-products. Pathogen, P, units are more closely related to numbers of pathogens or colony forming units (CFU), but abstract units P-units are simply used as well and this population is scaled by 10 6 /cc. More details about the model development can be found in [START_REF] Reynolds | A reduced mathematical model of the acute inflammatory response I. Derivation of model and analysis of antiinflammation[END_REF]. The diagram in Figure 1 characterizes the different interactions between the states of the inflammatory model. A solid line with an arrow head indicates an up-regulation, whereas a dashed line with circular head indicates inhibition or down-regulation of a process. For instance, early pro-inflammatory mediators, N, respond to the presence of pathogen, P, by initiating self-recruitment of additional inflammatory mediators and N is therefore up-regulated by the interaction with P to attempt to efficiently eliminate the pathogen. The self up-regulation that exists for P is due to replication. Furthermore, N inhibits P by eliminating it at some rate. The inflammation caused by N, however, results in tissue damage, D, which can provide a positive feedback into the early inflammatory mediators depending on their intensity. To balance this, anti-inflammatory mediators, such as cortisol, IL-10, and TGF-β , can mitigate the inflammation and its harmful effect by suppressing the response by N and the effects of D in various ways. The C a variable maintains a small positive background level at equilibrium in the presence of no pathogen. Following the setup used in [START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF], the reference parameter value for C a (0) is set to 0.125 and virtual patients have a value that is ±25% of the reference value. In addition, the values of six other parameters as well as the initial condition for P are set to have differing (positive) values from the reference set. In particular, the values of these parameters and initial conditions were generated from a uniform distribution on defined parameter ranges or on a range that was +/-25% of the (mean) reference value. The remaining parameters retained the same values as those in the reference set. These differences distinguish one virtual patient from another.

We use the set of 1000 virtual patients generated by [START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF] in the way described above to evaluate the performance of the proposed control strategy. The set of patients was classified with respect to their outcome after an open loop simulation for a long enough time to numerically determine outcome without ambiguity. Of the 1000 virtual patients, 369 did not resolve the infection and/or inflammatory response on their own and succumbed to a septic (141) or aseptic (228) death death outcome. On the other hand, 631 exhibited a healthy outcome of which there were two distinct subsets:

1. 379 of the 631 healthy virtual patients did not necessitate treatment intervention because their inflammatory levels did not exceed a specified threshold (defined to be N(t) ≤ 0.05, set in [START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF]). These virtual patients were excluded from receiving treatment and from our in silico study. 2. The remaining 252 of these virtual patients did surpass the specified threshold, N(t) ≥ 0.05, and are included in the cohort that receives treatment. However, these virtual patients would be able to resolve to health on their own, in the absence of treatment intervention. An important issue for these particular virtual patients is not to harm them with treatment.

Thus, 621 of the 1000 generated virtual patients receive treatment via our control design. Once a suitable reference trajectory is provided for the states with sensors, the derivation of the control part is straightforward which we now discuss.

Control design

As in previous control studies using this model, we assume that the state components P and D in Equations ( 9) and ( 12) are not measurable; whereas, the states N and C a in Equations ( 10) and ( 12), respectively, are:

easily measured and influenced by the control variables u p and u a , respectively.

We then introduce two equations of type (1):

Ṅ = F 1 + α p u p (t) (13) Ċa = F 2 + α a u a (t). (14)
We emphasize that, like in [START_REF] Lafont | A model-free control strategy for an experimental greenhouse with an application to fault accommodation[END_REF], the above two ultra-local systems may be "decoupled" so that they can be considered as monovariable systems. It should be nevertheless clear from a purely mathematical standpoint that F 1 (resp. F 2 ) is not necessarily independent of u a (resp. u p ). The two corresponding iPs (2) then read

u p = - F 1,est -Ṅ * + K P1 e p α p (15)
u a = - F 2,est -Ċ * a + K P2 e a α a . ( 16
)
The tracking errors are defined by

e p = N -N * and e a = C a -C * a ,
where N * and C * a represent the reference trajectories corresponding to pro and antiinflammatory measurements N and C a , respectively. Knowing that F encapsulates all the model uncertainties and disturbances as already explained in the introduction, a good estimate F est provides exponential local stability of the closed-loop system. The following algorithm 1 provides a good summary on the functioning of the proposed methodology for immune regulation:

Algorithm 1 Model-free Control

Step 1: Initialization, k = 0 u p (0) = 0, define reference trajectories N * , initialize K p and α, fix the sampling time T e ; For 1 ≤ k ≤ T f

Step 2, : Get measurements of N and u p ;

Step 3 Estimation of F: Estimate F according to a discrete implementation of equation ( 7);

Step 4: Close the loop according to equation ( 15) and return to Step 2.

Note that the same design procedure can lead to the derivation of the control u a ; however, this time we associate the measurement C a with the control u a (See equation ( 14)). The interesting fact about this approach is that we do not need to control the state variables P and D, which are not measurable. Solving the tracking problem consisting of following closely the reference trajectory of N and C a is enough to drive the pathogen and damage to values in the basin of attraction of the healthy equilibrium where they would converge to this state as time progressed, thereby 'curing' the patient.

Results without noise corruption

We first examine the performance of the control approach with respect to the set of virtual patients and their individual corresponding initial conditions. The robustness of the control law with the addition of corrupting measurement noise will be discussed afterward. In what follows, the reference trajectories which are inspired from [START_REF] Bara | Immune therapeutic strategies using optimal controls with L 1 and L 2 type objectives[END_REF], correspond to the measurable states N and C a . They will be highlighted in dashed lines. The simulations for all patients were performed under the following conditions:

a sampling time of 1 minute, α p = 1, α a = 10 in Equations ( 13)-( 14), -K P1 = K P2 = 0.5 in Equations ( 15)-( 16), and -250 hours simulation time to numerically determine outcomes without ambiguity; though we note stress that our control objectives were reached in less than 250 hours.

The use of the same reference trajectory for all simulations emphasizes the robustness of the proposed control approach with respect to the variability among virtual patient parameter values and initial conditions. Figure 2 represents a successful outcomes related to 92 out of the 141 septic patients who were cured when applying a Fig. 3 Time evolution of the control u p and u a for the set of septic patients on which the strategy was implemented. The zoomed-in plot for u p provides more details on the duration of the control dose, where the x-axis is shown for only two hours since it is zero for the remaining time control given by Figure 3. The patients that converged to the septic death equilibrium (as explained in Section 3.1) are obviously the ones who were not cured with the approach. The criteria to classify successful therapeutic control is to determine if the levels of pathogen (P) and damage (D) are reduced to very low values (< 0.2). All virtual patients not meeting this criteria were either classified as septic death outcomes if, in addition, the pathogen state did not also approach zero or aseptic death outcomes otherwise. These two latter cases correspond to virtual patients not saved by the applied dosage.

A closer look during the first hours in Figure 3 shows that the amplitude of the control variables is the main difference between different dosing profiles. Similar remarks apply fo u a . Analyzing u a shows that it is applied for a longer period of time than u p , but with smaller amplitude. This was not observed in the optimal control setting of [START_REF] Bara | Immune therapeutic strategies using optimal controls with L 1 and L 2 type objectives[END_REF], where dosing strategy ended at 30 hrs. It is thus interesting to see if purposefully restraining the dose quantity would have a sizable impact on the result. Surprisingly, however, we observe that forcing the anti-inflammatory control input, u a , to be zero after 28 hours does not affect the number of cured patients in this current study. This is an important insight to have in order to prevent unnecessary and lengthy dosing protocols. Whereas the maximum duration of the derived optimal control doses in [START_REF] Bara | Immune therapeutic strategies using optimal controls with L 1 and L 2 type objectives[END_REF] is 30 hours, it is much longer in the model-free control simulations. An extended duration in the model-free setting is the price to pay. On the other hand, the model-free control tracks a single given reference trajectory for all the virtual patients whereas the optimal control strategy [START_REF] Bara | Immune therapeutic strategies using optimal controls with L 1 and L 2 type objectives[END_REF] strives to infer the trajectories from a mathematical model that is required to be a 'good' model for all the virtual patients.

Table 1 displays the results from our study for the 621 patients that qualified for therapy because of sufficiently elevated inflammation. The first column displays the outcomes in the absence of intervention, labeled the placebo outcome. Without intervention, 40% (252) will resolve to a healthy outcome, while the remaining 60% (369) fall into one of the two unhealthy outcome categories. We use the total of 369 unhealthy placebo outcomes to determine the percentage of those that the treatment rescued. Likewise, we use the total of 252 healthy placebo outcomes to determine the percentage of those harmed (i.e. they would have resolved to healthy without treatment but converged to one of the death states instead after receiving treatment). Figures 2 and3 display the time courses for the sensorless states, P and D. These were guided via the reference trajectories for the states with sensors, N and C a , along with the corresponding control input. The results are reminiscent of [START_REF] Bara | Optimal control of an inflammatory immune response model[END_REF][START_REF] Bara | Immune Therapy using optimal control with L 1 type objective[END_REF][START_REF] Day | Using nonlinear model predictive control to find optimal therapeutic strategies to modulate inflammation[END_REF]: first apply a large dose of pro-inflammatory therapy, u p , followed by an anti-inflammatory dose, u a . The latter attempts to prevent excessive tissue damage resulting from the additional pro-inflammatory signals form the first dose. The information we can derive from Table 1 is that the control strategy obviously improves the percentage of cured patients when compared to the placebo case. Our therapy rescued 85.66% of the total patient population (621) and 75.88% of the combined septic and aseptic population (369). Additionally, 0% of the healthy patients are harmed. Figure 4 shows the evolution of the unobservable state P and D together with the measured states corresponding to N and C a for a set of 228 aseptic patients. Of these, 188 were able to recover from an aseptic placebo outcome, when the generated controls in Figure 5 are applied, driving the pathogen P and the level of damage D to zero. Again, one can observe from Figure 4 that some trajectories diverge to the unhealthy aseptic region, where the pathogen is known to have a zero value but the other state variables remain elevated.

Overall, the simulation results with respect to successful control of the number of outcomes for both the septic and aseptic placebo outcomes are very encouraging when one considers that only a unique reference trajectory was used for the heterogeneous population. The absence of perfect tracking should not be seen as a weakness of the model free control approach, since the control objective has been attained in most scenarios.

One of the important features of the presented data driven control approach is the necessity to have suitable choice(s) for the reference trajectories. To be more explicit, consider a naive choice of the reference trajectories: a trajectory exponential decaying to zero for N re f and another trajectory exponentially decaying to the C A steady state value 0.125. This would not satisfy the control objective since the generated control doses are negative and the level of pathogen will converge to its maximum allowable value. The reason for this behavior can be explained by the fact that the iP controller is only concerned about reducing the tracking error without imposing any constraints on the control inputs. Constraints on the control are not implemented simply because the model-free approach is not formulated as an optimization problem. 2 However, choosing a reference trajectory that will account for the correct time-varying dynamics of the inflammatory response will eventually generate the correct doses. That is, if we chose, for example, a reference trajectory with a smaller amplitude or with slower rising dynamics, other than what is presented in this work, then it is highly probable that the patient would not converge to the healthy state with respect to the generated control doses received. Similar remarks can be made to Figure 5 as discussed previously for the set of placebo septic patients. It is not surprising to notice a very close pattern with respect to the generated control doses for both sets of septic and aseptic patients. This can be explained in part by a common control objective consisting of tracking the same reference trajectory and also because of what has been discussed before regarding how the inflammatory immune system needs to react in order to eliminate the pathogen without incurring a significant damage.

Results with noise corruption

Consider the effects of corrupting measurement noise on our control problem. Here, a white Gaussian noise is taken into account as in many academic studies, (see, e.g., [START_REF] Blanc-Lapierre | Théorie des fonctions aléatoires -Applications à divers phénomènes de fluctuation[END_REF][START_REF] Rabiner | Theory and Application of Digital Signal Processing[END_REF]). Otherwise, the same setting as the previous section is kept. Figures 6 and7 display the states and the corresponding controls for the set of 141 septic placebo patients in which 90 were cured.

The addition of measurement noise with a standard deviation equal to 10 -3 only changes the outcome for two of the septic patients, when compared to the initial simulations where no noise was included. However, for the aseptic set of patients, there is a difference of 16 additional patients that did not survive when measurement noise is considered.

Remark 4 For the model-free simulation with measurement noise, there are mainly two important remarks to make with regard to the discussion of the previous Section. First, for the case of septic patients, restraining the control u p and u a to be zero after 2 hours and 28 hours, respectively, will not considerably affect the number of cured patient since 90 patients were cured. One would fail to obtain a similar result when altering the control in the same way for the aseptic case. Although not shown here, a decrease of around 45 patients was observed when compared to the 172 who were cured without restraining control inputs.

Concluding remarks

In this study we propose a new data-driven control approach in order to appropriately regulate the state of an inflammatory immune response in the presence of pathogenic infection. The performance of the proposed control strategy is investigated in the context of a set of 621 heterogenous model-based virtual patient population having model rate parameter variability. The results of the model-free strategy presented here in the presence of measurement noise are also explored and discussed. The robustness of the approach to parameter variability and noise disturbances is seen in the fact that a single reference trajectory was used to inform the approach about desirable inflammatory dynamics and from this, the individual dosing strategies found largely produced healthy outcomes. The downside of the proposed control approach to this specific application is the necessity to apply the control for a longer period of time although with small doses. However, we have seen that artificially restricting this small dose from being provided does not affect the outcome of the states in the case when no measurement noise is used; thought it did in the scenarios with measurement noise. We want to emphasize the importance of a suitable choice for the reference trajectory and further studies may provide better insights in this direction.

Past successes of the model-free control feedback approach in other realistic casestudies should certainly be viewed as encouraging for the future development of our approach to the treatment of inflammation. Additionally, the model-free control approach seems to be both theoretically and practically simpler when compared to model-based control designs. This newer viewpoint for control problems in biomedicine needs to be further analyzed in order to confirm its applicability in these complex dynamic systems where the ability to realistically obtain frequent measurement information is limited.

Fig. 2

 2 Fig. 2 Dashed (--) curves in the panels for the variables N and C a denote the reference trajectories used in the simulation. The various colored curves display the closed loop state responses for the set of 141 septic patients, of which 92 resolved to the healthy outcome.

Fig. 4

 4 Fig. 4 Dashed (--) curves in the panels for the variables N and C a denote the reference trajectories used in the simulation. The various colored curves display the closed loop state responses results for the set of 228 aseptic placebo patients, of which 188 were cured.

Fig. 5

 5 Fig.5Time evolution of the control inputs u p and u a for the set of aseptic patients shown in Figure4. The zoomed-in plot for u p provides a better perspective on the duration of the control dose, where the x-axis is shown for two hours only since it is zero afterward.

Fig. 6

 6 Fig. 6 Dashed (--) curves in the panels for the variables N and C a denote the reference trajectories used in the simulation. The various colored curves display the closed loop state responses for the set of 141 septic placebo patients, of which 90 were cured. Note that the measurements N and C a were corrupted with Gaussian noise.

Fig. 7

 7 Fig. 7 Time evolution of the control u p and u a for the set of septic placebo patients when the measurements N and C a were corrupted with Gaussian noise. The zoomed-in plot for u p provides more detail on the duration of the control dose, with an x-axis shown only for two hours since the doses are zero afterward.

Fig. 8

 8 Fig. 8 Dashed (--) curves in the panels showing the time courses of the variables N and C a denote the reference trajectories used in the simulation. The various colored curves display the closed loop state responses for the set of 228 aseptic placebo patients, 172 of which were cured. Note that the measurements N and C a were corrupted with Gaussian noise.

Fig. 9

 9 Fig. 9 Time evolution of the control u p and u a for the set of 228 aseptic placebo patients when the measurements N and C a were corrupted with Gaussian noise. The zoomed-in plot for u p provides more details on the duration of the control dose, with an x-axis shown only for two hours since the doses are zero afterward.

Table 1

 1 Results of the model-free immune therapy strategy without measurement noise compared to the placebo outcomes.

	Therapy Type:	Placebo	Model-free control therapy
	Percentage Healthy:	40% (252)	85.66% (518)
	Percentage Aseptic:	37% (228)	6.4% (40)
	Percentage Septic:	23% (141)	7.8% (49)
	Percentage Harmed (out of 252)	n/a	0% (0/252)
	Percentage Rescued (out of 369)	n/a	75.88% (280/369)

Table 2

 2 Results of the model-free immune therapy strategy with measurement noise compared to the placebo outcomes.

	Therapy Type:	Placebo	Model-free control therapy
	Percentage Healthy:	40% (252)	82.76% (514)
	Percentage Aseptic:	37% (228)	9.02% (56)
	Percentage Septic:	23% (141)	8.21% (51)
	Percentage Harmed (out of 252)	n/a	0% (0/252)
	Percentage Rescued (out of 369)	n/a	71% (262/369)

Allowing the control to be only positive semidefinite will result in a zero control all the time.

Work partially supported by the NSF-DMS Award 1122462 and a Fullbright Scholarship.