Ivan Yegorov
email: ivanyegorov@gmail.com

Francis Mairet

Hidde De Jong

Jean-Luc Gouzé

Egorov F Mairet

J.-L Gouzé

Optimal control of bacterial growth for the maximization of metabolite production

Keywords: Optimal control, Nonlinear dynamical systems, Mathematical modelling, Bacterial growth, Biotechnology

Microorganisms have evolved complex strategies for controlling the distribution of available resources over cellular functions. Biotechnology aims at interfering with these strategies, so as to optimize the production of metabolites and other compounds of interest, by (re)engineering the underlying regulatory networks of the cell. The resulting reallocation of resources can be described by simple so-called self-replicator models and the maximization of the synthesis of a product of interest formulated as a dynamic optimal control problem. Motivated by recent experimental work, we are specifically interested in the maximization of metabolite production in cases where growth can be switched off through an external control signal. We study various optimal control problems for the corresponding self-replicator models by means of a combination of analytical and computational techniques. We show that the optimal solutions for biomass maximization and product maximization are very similar in the case of unlimited nutrient supply, but diverge when nutrients are limited. Moreover, external growth control overrides natural feedback growth control and leads to an optimal scheme consisting of a first phase of growth maximization followed by a second phase of product maximization. This twophase scheme agrees with strategies that have been proposed in metabolic engineering. More generally, our work shows the potential of optimal control

Introduction

The fitness of microorganisms is defined by their capacity to propagate in environments hosting a variety of competitors for available resources. The situation faced by these single-cell organisms can be seen as an optimization problem: the allocation of resources to different cellular functions so as to maximize fitness. In many situations, microbial fitness amounts to maximizing the growth rate of the population, allowing the cells to outgrow competitors. However, depending on the structure of the environment and the properties of metabolic pathways, other fitness criteria may apply as well [START_REF] Schuetz | Systematic evaluation of objective functions for predicting intracellular fluxes in Escherichia coli[END_REF][START_REF] Schuster | Is maximization of molar yield in metabolic networks favoured by evolution?[END_REF]. While experimental conditions can be kept constant in the laboratory, in most natural environments microorganisms are exposed to changing conditions [START_REF] Savageau | Escherichia coli habitats, cell types, and molecular mechanisms of gene control[END_REF][START_REF] Van Elsas | Survival of Escherichia coli in the environment: Fundamental and public health aspects[END_REF]. This requires microbial cells to continually adapt their functioning, and the allocation of resources to maximize fitness can be viewed as a dynamical optimization problem (see [Banga, 2008, Lenhart and[START_REF] Lenhart | Optimal Control Applied to Biological Models[END_REF] for general introductions to dynamical optimization problems in biology).

Self-replicator models [START_REF] Hinshelwood | On the chemical kinetics of autosynthetic systems[END_REF][START_REF] Koch | Why can't a cell grow infinitely fast?[END_REF] have been shown to provide a fruitful description of microbial growth. Despite their simplicity, they are remarkably expressive and are capable of accounting for observed correlations between variables in microbial physiology [START_REF] Molenaar | Shifts in growth strategies reflect tradeoffs in cellular economics[END_REF]. In recent years, self-replicator models have been used to formulate resource allocation in microorganisms as an optimal control problem [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF][START_REF] Pavlov | Optimal control of gene expression for fast proteome adaptation to environmental change[END_REF][START_REF] Scott | Emergence of robust growth laws from optimal regulation of ribosome synthesis[END_REF][START_REF] Van Den Berg | Optimal allocation between nutrient uptake and growth in a microbial trichome[END_REF], Yegorov et al., 2017b[START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF]. For the purpose of optimal control, self-replicator models are of great interest, since they reduce the mathematical and computational complexity of the problem. The optimal solutions thus obtained may give new insights into the functioning of microorganisms, by comparing them with control strategies that microorganisms have evolved to distribute available resources over cellular functions, involving complex networks of regulatory interactions on the molecular level [START_REF] Chubukov | Coordination of microbial metabolism[END_REF], Schaechter et al., 2006].

In addition to gaining a better understanding of naturally-evolved resource allocation, optimal control theory is also beneficial in biotechnology, where the objective is not to optimize microbial growth, but to exploit the synthetic capacities of microorganisms for maximizing the production of compounds of interest [START_REF] Stephanopoulos | Metabolic Engineering: Principles and Methodologies[END_REF]. Control theory has contributed a rich variety of mathematical and computational tools for achieving this, notably through on-line estimation, adaptive control and optimization of process conditions [START_REF] Banga | Dynamic optimization of bioprocesses: Efficient and robust numerical strategies[END_REF][START_REF] Bastin | On-line Estimation and Adaptive Control of Bioreactors[END_REF][START_REF] Venkateswarlu | Advances in monitoring and state estimation of bioreactors[END_REF].

The emergence of systems biology and synthetic biology has provided new tools for the implementation of control strategies, by increasing our understanding of the functioning of regulatory networks on the molecular level and facilitating their (re)engineering [Chubukov et al., 2014, Khalil and[START_REF] Khalil | Synthetic biology: applications come of age[END_REF].

An example of a synthetic biology application in biotechnology is the development of an E. coli strain that allows growth to be switched on and off externally by adjusting the availability of RNA polymerase, a major component of the gene expression machinery. It was shown that by arresting the expression of the ββ subunits of RNA polymerase, resources are shuttled away from protein synthesis (growth) to the production of a metabolite of interest (glycerol) [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF]. Other examples of synthetic growth-rate control, with possible applications in biotechnology, have been published in the literature recently [START_REF] Ceroni | Sensing the right time to be productive[END_REF][START_REF] Lo | A two-layer gene circuit for decoupling cell growth from metabolite production[END_REF][START_REF] Milias-Argeitis | Automated optogenetic feedback control for precise and robust regulation of gene expression and cell growth[END_REF].

Self-replicator models provide a useful conceptual framework for expressing the dynamical reallocation of resources in microorganisms as an optimal control problem [ de Jong et al., 2017b]. In this study, we aim at exploiting this potential by reformulating the redirection of resources from growth to metabolite production via the growth switch as a dynamical optimal control problem. We thus hope to answer the question which strategy for the dynamical regulation of RNA polymerase expression maximizes metabolite production and how this strategy compares with the approach followed in the original publication [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF]. Whereas previous studies using self-replicator models focused on natural growth control [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF][START_REF] Pavlov | Optimal control of gene expression for fast proteome adaptation to environmental change[END_REF][START_REF] Scott | Emergence of robust growth laws from optimal regulation of ribosome synthesis[END_REF][START_REF] Van Den Berg | Optimal allocation between nutrient uptake and growth in a microbial trichome[END_REF][START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF], we here explore their use for biotechnological applications. The emphasis on resource allocation using a coarse-grained, whole-cell model of microbial growth distinguishes our work from previous application of (optimal) control in biotechnology [START_REF] Banga | Dynamic optimization of bioprocesses: Efficient and robust numerical strategies[END_REF][START_REF] Bastin | On-line Estimation and Adaptive Control of Bioreactors[END_REF][START_REF] Venkateswarlu | Advances in monitoring and state estimation of bioreactors[END_REF].

We first formulate a model of a self-replicator extended with a pathway for the production of a metabolite of interest as well as external control of the synthesis of components of the gene expression machinery. We consider the scenarios of limited and unlimited nutrient availability, corresponding roughly to batch and fed-batch cultivation modes, respectively. Given the control input for these models, modulation of the synthesis rate of components of the gene expression machinery, we then formulate two dynamical optimal control problems, the first one involving maximization of metabolite production, and the second one involving maximization of biomass production. We solve the optimal control problems by analytical and numerical means, and notably find that the optimal solutions for product maximization and biomass maximization are very similar in the case of the model with unlimited nutrient supply, but become essentially different when the available nutrients are limited.

The solutions thus obtained provide a gold standard in the sense that they represent the optimum that can be theoretically achieved, under the as-

S P R M A B (1-α)V R S P R M X I V M V S αV R V S V M (1-α)V R αV R V X
Fig. 1 Self-replicator models of bacterial growth and metabolite production. A: Original self-replicator model from [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], describing the synthesis of precursor metabolite P from external substrate S, and the utilization of P for the synthesis of gene expression machinery R and metabolic machinery M, at proportions α and 1 -α, respectively.

V M and V R denote the rates of precursor and protein synthesis, respectively. M catalyzes precursor synthesis and R protein synthesis. B : Extended self-replicator model in which a heterologous pathway allows the cell to produce metabolite X from precursor P, at a rate V X . Moreover, following [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF], the synthesis of gene expression machinery R can be modulated by an external signal I.

sumption that external control of the gene expression machinery completely overrides natural feedback growth control in bacteria. Since the two are expected to interfere in biotechnological applications, we adapt the model, in a final step, to account for natural feedback growth control. We show that the gold standard can be closely approximated when assuming a simple two-step control scheme: a first phase with maximal induction of the gene expression machinery (and low production), followed by a second phase with no induction of the gene expression machinery (and high production). The practical relevance of this control scheme is assessed by comparing it with established strategies of dynamical process control [START_REF] Venayak | Engineering metabolism through dynamic control[END_REF].

2 Self-replicator models of the control of metabolite production Figure 1A is a schematic representation of the self-replicator model of bacterial growth proposed by [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF]. Incoming nutrients S are transformed into precursor metabolites P, which are then consumed by several cellular functions, here limited to gene expression and metabolism. More precisely, the precursor metabolites are utilized for ribosomes and other components of the gene expression machinery (R) as well as the enzymes making up the metabolic machinery (M). M enables the conversion of external substrates into precursors, while R is responsible for the production of M and R itself.

The (auto-)catalytic activity of M and R thus allows the cell to replicate its protein contents, the major constituent of biomass. The allocation of resources over the two different cellular functions considered here, metabolism and gene expression, is captured by the time-varying function α = α(t) ∈ [0, 1], which indicates the proportional utilization of precursors for the synthesis of R and M.

In order to reengineer natural resource allocation strategies for biotechnological applications, we consider extended self-replicators including a heterologous (non-natural) metabolic pathway for the production of a metabolite of interest X (Figure 1B ). X may be a drug precursor or a biofuel. Moreover, we extend the self-replicator with the growth switch as described by [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF], which allows the synthesis of (components of) the gene expression machinery, in particular RNA polymerase, to be shut off by means an external control signal I. The original growth switch uses a chemical signal, but alternatively a light signal could be employed [Milias-Argeitis et al., 2016, Olson and[START_REF] Olson | Optogenetic characterization methods overcome key challenges in synthetic and systems biology[END_REF]. While a chemical inducer can be simply added to the growth medium of a bacterial population, it cannot be easily removed, contrary to a light signal which can be switched on and off. The basic idea underlying the growth switch is that upon arresting synthesis of (components of) the gene expression machinery, precursors P are decreasingly utilized for protein synthesis which causes growth to slow down. As a consequence, P becomes available for the production of X and, under the assumption that the enzymes necessary for producing P and X are abundant and stable, so that metabolism remains active, resources are reallocated from growth to metabolite production.

The extended self-replicator of Figure 1B is described in terms of the timevarying variables S, P, M, R, X representing the masses of S, P, M, R, X [g], respectively. Their dynamics are given by the following system of ordinary differential equations:

                                     dS(t) dt = V S (t) -V M (t), dP (t) dt = V M (t) -V R (t) -V X (t), dM (t) dt = (1 -u(t)) V R (t), dR(t) dt = u(t) V R (t), dX(t) dt = V X (t), t ∈ [0, T ].
(1)

Here t [h] is the time variable, T ∈ (0, +∞) [h] is a finite time horizon, and the quantities

V S = V S (t) g h -1 , V M = V M (t) g h -1 , V R = V R (t) g h -1 , V X = V X (t) g h -1
represent the rates of supplying the external substrate, producing the precursors by metabolism, utilizing the precursors for protein synthesis, and producing the metabolite of interest, respectively. The precursor mass used for protein synthesis is allocated between the gene expression and metabolic machineries. The dimensionless resource allocation (control

) function u = u(t) ∈ [0, 1]
specifies the proportion utilized for gene expression, while 1 -u(t) is the proportion allocated to metabolism. We set

u(t) = I(t) α(t), (2)
that is, the input u aggregates the external control I and the control α exerted by the natural resource allocation systems of the cells. By (2), the natural and synthetic controls are supposed to act independently. Assume that α = α(t) ∈ [α min , α max ] and I = I(t) ∈ [I min , I max ], where the constant parameters I min , I max , α min , α max satisfy the inequalities

0 I min < 1, I min < I max 1, 0 α min < 1, α min < α max 1.
(3)

Then the lower and upper bounds for u are given by

u min def = I min α min ∈ [0, 1), u max def = I max α max ∈ (u min , 1]. ( 4
)
The model presented above makes a number of simplifications. First, protein degradation is ignored, i. e., we suppose that the macromolecules are stable and that their degradation rates are negligible with respect to the rates of the reactions described above. This is a reasonable assumption for normal physiological conditions, in which most bacterial proteins have a half-life > 10 h [START_REF] Larrabee | The relative rates of protein synthesis and degradation in a growing culture of Escherichia coli[END_REF][START_REF] Mosteller | Metabolism of individual proteins in exponentially growing Escherichia coli[END_REF]. At high temperatures, however, the influence of degradation may become significant [START_REF] Farewell | Effect of temperature on in vivo protein synthetic capacity in Escherichia coli[END_REF]Neidhardt, 1998, Johnson and[START_REF] Johnson | The growth rate of E. coli in relation to temperature, quinine and coenzyme[END_REF], as was also shown by [START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF] for a variant of the self-replicator model of Figure 1A. Second, only two classes of macromolecules, corresponding to the metabolic and gene expression machineries, are taken into account. We thus neglect the use of precursors for the synthesis of proteins involved in, for example, cell maintenance [START_REF] Scott | Interdependence of cell growth and gene expression: Origins and consequences[END_REF], as well as in sensing and regulation [START_REF] Kalisky | Cost-benefit theory and optimal design of gene regulation functions[END_REF][START_REF] Poelwijk | Tradeoffs and optimality in the evolution of gene regulation[END_REF]. Third, we describe the growing cell population on the level of the aggregated masses of its components P, M, R, X. The use of such nonsegregated models, as opposed to segregated models taking individual cells into account [START_REF] Surovstev | Whole-cell modeling framework in which biochemical dynamics impact aspects of cellular geometry[END_REF], relies on the implicit assumption that individual cells in the population are sufficiently similar to each other [ de Jong et al., 2017a].

The above model can be further developed by making a number of additional assumptions and transformations.

First, in line with [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], we assume that the cytoplasmic density of the cells in the population is constant, and β > 0 L g -1 denotes the inverse density. Second, the growing structural volume of the bacterial population is defined as

V = V(t) def = β • (M (t) + R(t)) [L].
(5)

In other words, the volume is proportional to the macromolecules constituting the metabolic and gene expression machineries, consistent with the fact that macromolecules make up most of the biomass of microbial cells [START_REF] Bremer | Modulation of chemical composition and other parameters of the cell at different exponential growth rates[END_REF]. Third, the relative increase in the volume determines the growth rate

µ = µ(t) def = 1 V(t) dV(t) dt . (6)
Fourth, we introduce variables that express the mass quantities P, R, M, V M , V R per unit population volume:

p = p(t) def = P (t) V(t) g L -1 , r = r(t) def = R(t) V(t) g L -1 , m = m(t) def = M (t) V(t) g L -1 , v M = v M (t) def = V M (t) V(t) g L -1 h -1 , v R = v R (t) def = V R (t) V(t) g L -1 h -1 . (7)
The variables p, r, m thus represent the concentrations of precursors, gene expression machinery, and metabolic machinery, respectively, in the growing bacterial population, and v M , v R the rate of production of precursors and proteins, respectively. From the relation (5), one can see that the sum p(t) + r(t) is constant and equal to 1/β, so that

m(t) = 1 β -r(t). (8)
Fifth, consider the quantities

x = x(t) def = X(t) V(t) g L -1 , v X = v X (t) def = V X (t) V(t) g L -1 h -1 . (9)
They are needed mainly for the mathematical investigation below and do not have a clear biological interpretation (because the metabolite of interest is assumed to be excreted from the cells). Sixth, we define the nutrient concentration

s = s(t) def = S(t) V ext g L -1 (10)
with respect to a constant external volume V ext > 0 [L] and set

v S = v S (t) def = V S (t) V ext g L -1 h -1 . ( 11
)
With the above definitions, the system (1) can be rewritten in the form

                                             ds(t) dt = v S (t) -v M (t) V(t) V ext , dp(t) dt = v M (t) -v R (t) -v X (t) -µ(t) p(t), dr(t) dt = u(t) v R (t) -µ(t) r(t), dx(t) dt = v X (t) -µ(t) x(t), dV(t) dt = µ(t) V(t), m(t) = 1 β -r(t), X(t) = x(t) V(t), t ∈ [0, T ]. ( 12
)
Following [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], we suppose that

v M (t) def = m(t) k M s(t) K M + s(t) , v R (t) def = r(t) k R p(t) K R + p(t) . (13)
Here k M > 0 h -1 and K M > 0 g L -1 are respectively the rate and half-saturation constants for the metabolic macroreaction, while k R > 0 h -1 and K R > 0 g L -1 are those for the protein synthesis reaction. By using the relations (1), ( 5)-( 7) and ( 13), one can reformulate the growth rate as

µ(t) = β v R (t) = β r(t) k R p(t) K R + p(t) . (14)
We also set

v X (t) def = m(t) k X p(t) K X + p(t) , (15)
where k X > 0 h -1 and K X > 0 g L -1 are the rate and half-saturation constants for the reaction producing the metabolite of interest. The expressions for the reaction rates v M , v R , v X and the growth rate µ complete the system (12).

For the analysis below, it is convenient to introduce the following new variables and parameters (most of which are dimensionless):

t def = k R t, T def = k R T, p t def = β p(t), r t def = β r(t), m t def = β m(t) = 1 -r t , X t def = β X(t), x t def = β x(t), ŝ t def = β s(t), vS t def = β k R v S (t), K def = β K R , K 1 def = β K X , K 2 def = β K M , k 1 def = k X k R , k 2 def = k M k R . ( 16
)
We also consider the structural volume V and control strategy u as functions of the new time variable t: V = V t , u = u t . Moreover, the rescaled growth rate becomes

μ t = µ(t) k R = p t r t K + p t . (17)
With these definitions, (12) transforms into the following nonlinear system:

                                                                 dŝ t d t = vS t -k 2 ŝ t 1 -r t K 2 + ŝ t V t V ext , dp t d t = k 2 ŝ t 1 -r t K 2 + ŝ t -1 + p t p t r t K + p t - -k 1 p t 1 -r t K 1 + p t , dr t d t = u t -r t p t r t K + p t , dx t d t = k 1 p t 1 -r t K 1 + p t - p t r t K + p t x t , dV t d t = p t r t K + p t V t , m t = 1 -r t , X t = x t V t , t ∈ 0, T . (18)
We will study two particular cases of this general model. First, as in [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF][START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF], assume that the environmental conditions do not change over the observed time interval, either because the nutrient concentration s is approximately constant or because s K M (when the substrate is available in excess). Then we consider the quantity

e M def = k M s K M + s h -1 (19)
as a constant environmental input and naturally suppose its positivity: e M = const > 0. Hence, the first formula in ( 13) is rewritten as

v M (t) = e M m(t) = e M • 1 β -r(t) , (20)
and we arrive at the following model.

Definition 2.1 (Model 1: constant environmental conditions) In case e M = const > 0, the system (18) is reduced to Model 1 defined by

                                                       dp t d t = E M 1 -r t -1 + p t p t r t K + p t - -k 1 p t 1 -r t K 1 + p t , dr t d t = u t -r t p t r t K + p t , dx t d t = k 1 p t 1 -r t K 1 + p t - p t r t K + p t x t , dV t d t = p t r t K + p t V t , m t = 1 -r t , X t = x t V t , t ∈ 0, T , (21)
with the additional dimensionless parameter

E M def = e M k R . ( 22
)
Model 1 may describe fed-batch cultivation during which the nutrient concentration is either high enough or maintained at a certain level, so that nearly exponential bacterial growth is achieved [START_REF] Cinar | Batch Fermentation: Modeling, Monitoring, and Control[END_REF].

Definition 2.2 (Model 2: substrate depletion) In case v S ≡ 0, substrates are only depleted and not replenished from the outside during the observed time interval. As a consequence, the system (18) is reduced to Model 2 defined by

                                                                 dŝ t d t = -k 2 ŝ t 1 -r t K 2 + ŝ t V t V ext , dp t d t = k 2 ŝ t 1 -r t K 2 + ŝ t -1 + p t p t r t K + p t - -k 1 p t 1 -r t K 1 + p t , dr t d t = u t -r t p t r t K + p t , dV t d t = p t r t K + p t V t , d X t d t = k 1 p t 1 -r t K 1 + p t V t , m t = 1 -r t , t ∈ 0, T . ( 23
)
Model 2 is appropriate for describing batch cultivation [START_REF] Cinar | Batch Fermentation: Modeling, Monitoring, and Control[END_REF]. Note that, in Model 1, the equation for x is included instead of the equation for X. As will be shown in the next section, this allows for a useful equivalent transformation of the product maximization criterion in Model 1, but is not appropriate for Model 2, due to the preservation of the equation for ŝ in the latter model.

Statement of the optimal control problems with aggregated input

For Models 1 and 2, we are interested first in finding an aggregated control input u that maximizes the total product output, i. e., the total amount of metabolite produced. Recall from (2) that u is given by a combination of natural and synthetic mechanisms of growth control.

We fix the initial state as p(0) = p0 > 0, r(0) = r0 ∈ (0, 1),

X(0) = X0 0, V(0) = V 0 > 0, ŝ(0) = ŝ0 > 0, x(0) = x0 = X0 V 0 0 (24)
and recall the control constraint

u min u t u max ∀ t ∈ 0, T . (25)
Definition 3.1 (Product maximization problem) For the fixed initial state (24), the product maximization problem is specifed by the criterion

X T -X0 -→ max u(•) ∈ U
, where U is the set of all admissible open-loop control strategies, which are Lebesgue measurable real-valued functions defined on the time interval 0, T and satisfying the constraint (25). By using the dynamical equation for X (which is the same in Models 1 and 2), this criterion can be written as

X T -X0 = T 0 k 1 p t 1 -r t K 1 + p t V t d t -→ max u(•) ∈ U . ( 26
)
For later use, as a baseline for comparison, we also consider the biomass maximization problem [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF][START_REF] Molenaar | Shifts in growth strategies reflect tradeoffs in cellular economics[END_REF][START_REF] Scott | Interdependence of cell growth and gene expression: Origins and consequences[END_REF][START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF]. Definition 3.2 (Biomass maximization problem) For the fixed initial state (24), the biomass maximization problem is specifed by the criterion

V T -→ max u(•) ∈ U
, where the control class U is the same as in Definition 3.1. By using the dynamical equation for V (which is the same in Models 1 and 2) and also the relation (17), this criterion can be written as

T 0 µ(t) dt = T 0 p t r t K + p t d t -→ max u(•) ∈ U . ( 27
)
Note that, in these two problems, the aggregated control input u is determined by (2) as the product of the natural growth control α and the synthetic growth control I. In practice, natural growth control is fixed, in the sense that the mechanisms underlying α, which have been selected by evolution, are endogeneous and cannot be easily modified. The experimenter can only vary I, corresponding to induction of the synthesis of the gene expression machinery. In Section 5, we will refine the problems by distinguishing between natural and synthetic growth control. However, we remark that the solutions obtained by considering u as the control input have a clear interpretation: they provide the optimum that can be obtained independently of constraints imposed by natural growth control mechanisms. They thus form a gold standard against which more realistic strategies accessible in a biotechnological context can be compared.

The product maximization criterion (26) allows for an equivalent transformation. Since X does not appear in the expression for d X/d t, the initial value X0 can be chosen arbitrarily. Let us take X0 > 0. Then X t > 0 and x t = X t /V t > 0 for all t ∈ 0, T (recall that V 0 > 0 due to (24), and, therefore, V t > 0 on the whole observed time interval). Hence, one can write the relation

ln X T = ln x T + ln V T = = ln x0 + T 0 1 x t dx t d t d t + + ln V 0 + T 0 1 V t dV t d t d t = = T 0 k 1 x t p t 1 -r t K 1 + p t - p t r t K + p t d t + + T 0 p t r t K + p t d t + ln x0 + ln V 0 = = T 0 k 1 x t p t 1 -r t K 1 + p t d t + ln x0 + ln V 0 . (28)
This leads to an equivalence property that will be found useful below.

Property 3.3 If X0 > 0, the product maximization problem as stated in Definition 3.1 can be equivalently reformulated as

T 0 k 1 x t p t 1 -r t K 1 + p t d t -→ max u(•) ∈ U . (29)
In what follows, when speaking about the product maximization criterion, we indiscriminately refer to ( 26) and ( 29). The condition X0 > 0 underlying the equivalence property can be imposed without loss of generality, and it will be assumed valid henceforth.

Assumption 3.4 X0 > 0.
Even though the reasoning underlying the derivation of (29) applies to both Models 1 and 2, the proposed transformation is useful only for Model 1, where it becomes possible to exclude the equation for V from the system (21) when considering the product maximization criterion (26) or the integral representation of the biomass maximization criterion (27). For Model 2, the equation for V cannot simply be excluded from the system (23), because the equation for ŝ includes V.

The product maximization and biomass maximization problems for both Models 1 and 2 will be investigated in the next section.

4 Investigation of the optimal control problems with aggregated input 4.1 Model 1 (constant environmental conditions) First, we consider Model 1. For the related product maximization problem, one can analyze the signs of the right-hand sides of the equations in ( 21) and thereby verify that

G def = {(p, r, x) : p > 0, 0 < r < 1, x > 0} (30)
is a strongly invariant domain [Clarke et al., 1998, Chapter 4, §3]. That is, for any admissible control function u(•) ∈ U, the corresponding state trajectory of Model 1 in the space (p, r, x) cannot leave G if the initial state lies in G.

For the biomass maximization problem, the equations for p, r and also the criterion do not contain x, i. e., x can be excluded from consideration, and the strongly invariant domain (30) therefore reduces to

G 1 def = {(p, r) : p > 0, 0 < r < 1} . (31)
Furthermore, for any (p 0 , r0 , x0 ) ∈ G and u( The estimates (32) and strong invariance of the domain (30) ensure that, for any initial state coordinates (p 0 , r0 , x0 ) ∈ G, V 0 > 0 and control input u(•) ∈ U, there exists a unique solution to the Cauchy problem ( 21), ( 24) and that this solution can be extended to the whole time interval [0, +∞). From the general existence result in [Cesari, 1983, §9.3] (see also [START_REF] Yong | Stochastic Controls: Hamiltonian Systems and HJB Equations[END_REF], Chapter 2, §5.1]), it can be concluded that, for every fixed (p 0 , r0 , x0 ) ∈ G, there exist optimal control functions in the class U for the product maximization and biomass maximization problems for Model 1.

Before investigating the dynamical optimal control problems, we provide the steady-state analysis.

Steady-state analysis for Model 1

For Model 1, we would like to find a steady state p * opt , r * opt , x * opt ∈ G for which the integrand in the product maximization criterion (29) takes its maximum value.

For any steady state (p * , r * , x * ) ∈ G of Model 1 such that

E M -k 1 p * K 1 + p * > 0, (33)
one has

u * = r * = E M -k 1 p * K1+ p * E M -k 1 p * K1+ p * + (1 + p * ) p * K+ p * ∈ (0, 1), x * = k 1 1 -r * r * K + p * K 1 + p * = k 1 (1 + p * ) p * E M -k 1 p * K1+ p * (K 1 + p * ) , (34)
i. e., the coordinates r * , x * and control u * are represented as functions of p * , while the integrand in the product maximization criterion (29) can be written as

k 1 x * p * (1 -r * ) K 1 + p * = p * r * K + p * = = p * K + p * E M -k 1 p * K1+ p * E M -k 1 p * K1+ p * + (1 + p * ) p * K+ p * . ( 35
)
The condition (33) guarantees the well-posedness and positivity of the expressions in ( 34) and ( 35). From ( 35), we arrive at the following property. By (35), the steady-state production and growth rates can be determined as a function of p * . In order to find its maximum, we search for zeros of its derivative, which has the same sign as

χ (p * ) def = (K + p * ) 2 E M -k 1 p * K 1 + p * + (1 + p * ) p * K + p * 2 • • d dp * k 1 x * (p * ) p * (1 -r * (p * )) K 1 + p * = = E M KE M -(p * ) 2 -2k 1 KE M p * K 1 + p * + + k 2 1 K (p * ) 2 (K 1 + p * ) 2 + k 1 (p * ) 3 K 1 + p * - -k 1 K 1 (1 + p * ) (p * ) 2 (K 1 + p * ) 2 . ( 36
)
Solving χ (p * ) = 0 yields p * opt . This problem can be reduced to searching for the roots of a fourth-order polynomial. The next assumption states the existence and uniqueness of such a root on the interval 0 < p * pmax up (p 0 ) , where pmax

up (p 0 ) def = max t ∈ [0, T ] pup t; p0 (37)
with pup (•; p0 ) appearing as the time-varying upper bound for p(•) in ( 32), and ensures the maximum condition.

Assumption 4.2 It holds that

E M -k 1 p * K 1 + p * > 0 ∀ p * ∈ 0, pmax up (p 0 ) (38)
(which ensures the well-posedness and positivity of (34) and ( 35)). Moreover, there exists a unique zero p * opt of the function (36) on the interval 0 p * pmax up (p 0 ) , and the following properties hold:

0 < p * opt < pmax up (p 0 ) , χ (p * ) > 0 ∀ p * ∈ 0, p * opt , χ (p * ) < 0 ∀ p * ∈ p * opt , pmax up (p 0 ) , u * opt = u * p * opt ∈ (u min , u max ).
If analytical verification of Assumption 4.2 appears to be difficult, one can easily check it numerically.

Once p * opt is obtained, the other components r * opt = r * p * opt , x * opt = x * p * opt of the optimal steady state together with the corresponding constant control u * opt = u * p * opt can be determined by (34). In order to investigate the stability of the optimal steady state, we write the Jacobian matrix for the system of the first three equations in (21) (here the partial derivatives of the right-hand sides are taken with respect to p, r, x, while u is treated as a constant parameter):

    -r K+ p p + K(1+ p) K+ p -k1K1 (1-r) (K1+ p) 2 -E M -(1+ p) p K+ p + k1 p K1+ p 0 K r (u-r) (K+ p) 2 p (u-2r) K+ p 0 k1K1 (1-r) (K1+ p) 2 -K r x (K+ p) 2 -k1 p K1+ p -px K+ p -pr K+ p     .
For any steady state (p, r, x) ∈ G, we get u = r, and, after substituting this equality, one can easily verify that the eigenvalues of the resulting matrix are real and negative. Hence, all such steady states including the optimal one are asymptotically stable nodes. We summarize the obtained results in the form of a theorem. Below we study product maximization and biomass maximization from a dynamical perspective.

Dynamical product maximization for Model 1

Now consider the dynamical product maximization problem for Model 1.

For the sake of convenience, introduce the new state variable ŷ = ŷ t def = ln x t (39) instead of x = x t . Then the third equation in ( 21) is replaced with

dŷ t d t = k 1 e -ŷ( t) p t 1 -r t K 1 + p t - p t r t K + p t , (40)
and the integrand in the product maximization criterion (29) transforms into the first term on the right-hand side of (40). The initial condition for (39) is

ŷ(0) = ŷ0 = ln x0 . (41)
In optimal control theory, first-order necessary optimality conditions for open-loop control strategies are given by Pontryagin's maximum principle or, in short, PMP [START_REF] Cesari | Optimization -Theory and Applications: Problems with Ordinary Differential Equations[END_REF][START_REF] Pontryagin | The Mathematical Theory of Optimal Processes[END_REF]. For the product maximization problem for Model 1, PMP leads to the Hamiltonian

H (p, r, ŷ, u, ψ 0 , ψ 1 , ψ 2 , ψ 3 ) = E M (1 -r) -(1 + p) p r K + p - -k 1 p (1 -r) K 1 + p ψ 1 + (u -r) p r K + p ψ 2 + + k 1 e -ŷ p (1 -r) K 1 + p - p r K + p ψ 3 -k 1 e -ŷ p (1 -r) K 1 + p ψ 0 , (42)
and to the adjoint system

                                                                                                       ψ 0 ≡ -1 or ψ 0 ≡ 0, dψ 1 t d t = - ∂H ∂ p p t , r t , ŷ t , u t , ψ 0 , ψ 1 t , ψ 2 t , ψ 3 t = = K r t K + p t 2 ψ 1 t 1 + p t + ψ 2 t r t -u t + + ψ 3 t + ψ 1 t p t r t K + p t + k 1 K 1 1 -r t K 1 + p t 2 • • ψ 1 t -ψ 3 t -ψ 0 e -ŷ( t) , dψ 2 t d t = - ∂H ∂ r p t , r t , ŷ t , u t , ψ 0 , ψ 1 t , ψ 2 t , ψ 3 t = = E M ψ 1 t + p t K + p t ψ 1 t 1 + p t + + ψ 2 t 2 r t -u t + ψ 3 t -k 1 p t K 1 + p t • • ψ 1 t -ψ 3 t -ψ 0 e -ŷ( t) , dψ 3 t d t = - ∂H ∂ ŷ p t , r t , ŷ t , u t , ψ 0 , ψ 1 t , ψ 2 t , ψ 3 t = = ψ 3 t -ψ 0 k 1 e -ŷ( t) p t 1 -r t K 1 + p t , t ∈ 0, T , (43)
with appropriate boundary conditions, while an optimal control strategy should fulfill the Hamiltonian maximum condition almost everywhere on 0, T (with respect to Lebesgue measure). The latter reduces to

u t = u min , ψ 2 t < 0, u max , ψ 2 t > 0. ( 44
)
Since there are no constraints on the terminal state at t = T , the terminal adjoint vector should satisfy

ψ 0 ≡ -1, ψ 1 T = ψ 2 T = ψ 3 T = 0. ( 45
)
In general, if both initial and terminal states are fixed, the case ψ 0 = 0 cannot be excluded and there are no restrictions on the terminal adjoint vector. An admissible process

(u(•), p(•), r(•), ŷ(•), ψ 0 , ψ 1 (•), ψ 2 (•), ψ 3 (•)) ( 46
)
satisfying PMP is called extremal. It is called normal if ψ 0 = 0 (which holds in the studied case ( 45)) and abnormal if ψ 0 = 0. Since the controlled system of Model 1 is autonomous (i. e., the right-hand sides of the related state equations depend on time only through the state and control variables), the Hamiltonian ( 42) is conserved along any extremal process (as also follows from the general formulation of PMP [START_REF] Cesari | Optimization -Theory and Applications: Problems with Ordinary Differential Equations[END_REF][START_REF] Pontryagin | The Mathematical Theory of Optimal Processes[END_REF]).

Property 4.4 The third adjoint equation in (43) directly implies

ψ 3 t -ψ 0 = ψ 3 T -ψ 0 • • exp   - T t k 1 e -ŷ(ξ) p(ξ) (1 -r(ξ)) K 1 + p(ξ) dξ    ∀ t ∈ 0, T . (47)
Therefore, if ψ 3 T -ψ 0 > 0 (which holds for (45)), then the function ψ 3 (•) -ψ 0 is positive and strictly increasing everywhere on 0, T .

The Hamiltonian maximum condition ( 44) is written in terms of the switching function ψ 2 (•). If ψ 2 (•) vanishes over some time subinterval, the corresponding control and arc of the related extremal state trajectory are called singular [START_REF] Gabasov | Singular Optimal Control[END_REF]. Multiple fundamental works [START_REF] Bonnard | Singular Trajectories and their Role in Control Theory[END_REF][START_REF] Clark | Mathematical Bioeconomics: The Optimal Management of Renewable Resources[END_REF][START_REF] Schattler | Geometric Optimal Control: Theory, Methods and Examples[END_REF], 2015[START_REF] Zelikin | Theory of Chattering Control with Applications to Astronautics[END_REF], 2005] indicate the central role of singular control regimes for dynamical optimization problems arising in many application domains, such as economics, engineering, ecology, biology, and medical sciences.

If one consecutively differentiates the switching function ψ 2 = ψ 2 t along the system of the state and adjoint equations ( 21), ( 40), ( 43) and substitutes the singular arc condition ψ 2 t = 0, then the control u = u t may first occur only in the expression for an even-order derivative. In our case, this is the fourth-order derivative of the switching function. By checking the sign of the corresponding coefficient for the control, one can also conclude that the Kelley condition or, in other words, the generalized Legendre-Clebsch condition [Schattler andLedzewicz, 2012, Zelikin and[START_REF] Zelikin | Theory of Chattering Control with Applications to Astronautics[END_REF] (which is necessary for optimality of singular arcs) holds there in the form of a strict inequality. According to the theory of chattering control [Schattler andLedzewicz, 2012, Zelikin and[START_REF] Zelikin | Theory of Chattering Control with Applications to Astronautics[END_REF], a chattering phenomenon (involving an infinite number of bang-bang switchings between the control bounds u min , u max on a finite time interval) should take place when entering or leaving a singular arc. Thus, we arrive at the following result.

Theorem 4.5 Let Assumption 3.4 hold. A singular arc in the product maximization problem for Model 1 can be entered or left only by chattering.

Complete characterization of the singular arcs by the direct analysis of the PMP boundary value problem ( 21), ( 24), ( 40), ( 41), ( 43)-( 45) turns out to be rather complicated. However, in contrast to what was previously obtained for biomass maximization problems without heterologous metabolite production [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF][START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF], the current problem does not allow for singular arcs with steady coordinates p and r.

Theorem 4.6 Let Assumption 3.4 and the condition (38) in Assumption 4.2 hold. A singular arc in the product maximization problem for Model 1 cannot have steady coordinates p and r, that is, the corresponding solution to the PMP system (21), ( 24), ( 40), ( 41), ( 43)-( 45) cannot satisfy dp t /d t = dr t /d t = ψ 2 t = 0 on a subinterval of 0, T .

Theorem 4.6 is proved in Appendix B. For illustrating the above results, we numerically solve the product maximization problem for Model 1 by using the software package BOCOP [START_REF] Bonnans | Bocop 2.0.5 -user guide[END_REF]. * The parameter values

u min = 0, u max = 1, (48)
β = 0.003 L g -1 , k R = 3.6 h -1 , K R = 1 g L -1 , (49)
e M = 3.6 h -1 (50)
are chosen in line with [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], and they are completed with reasonable values for the parameters characterizing the production pathway:

k X = 0.5 h -1 , K X = 1 g L -1 . ( 51
)
The dimensionless parameters in Model 1 are then determined by ( 16). In order to clearly observe the dynamics, the initial coordinates p0 , r0 are taken sufficiently far away from the optimal steady state components p * opt , r * opt . More precisely, we set

(p 0 , r0 ) = p * opt, k1 = 0 , r * opt, k1 = 0 e M = 0.72 h -1 , (52)
where p * opt, k1 = 0 , r * opt, k1 = 0 e M = 0.72 h -1

is the optimal steady state for the biomass maximization problem of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF] (whose dynamical system can be obtained from the first two equations of (21) by taking k 1 = 0 and u ≡ α) with the lower environmental input e M = 0.72 h -1 and with the other parameters as in ( 48), ( 49). This can be interpreted as a nutrient upshift after transferring from a poor environment to a richer one. The initial coordinate x0 = e ŷ0 is selected as

x0 = x * opt (53)
(it can in fact be chosen arbitrarily, because the initial product amount X0 can be fixed as any positive number, as was discussed in Section 3). Assumption 4.2 (including conditions on functions of single variables with values in bounded intervals) has been numerically verified for these data.

Fig. 2 The optimal control input and state variables for the product maximization problem for Model 1 computed by using BOCOP. The parameter values and initial conditions are given by ( 48)-( 53). The time horizon is taken as T = 15.

Figs. 2 and 3 indicate the corresponding numerical simulation results, for the time horizons T = 15 and T = 50, respectively. In BOCOP, we set the number of time steps to 5000 and the tolerance to 10 -6 , and we use the sixthorder Lobatto III C discretization rule [START_REF] Bonnans | Bocop 2.0.5 -user guide[END_REF]. Note also that BOCOP supplies smoothing techniques for handling chattering control. One can see that the singular arcs are entered and left by chattering, as expected from Theorem 4.5. With the increase of the time horizon T , the absolute and also relative duration of the singular control regime increases, and the singular arcs closer approach the optimal steady state p * opt , r * opt , x * opt , without completely coinciding with the latter, as per Theorem 4.6. Leaving the singular control regime some time before the final instant is typical for finite-horizon optimal resource allocation problems in different areas [START_REF] Akhmetzhanov | Optimal life-history strategies in seasonal consumer-resource dynamics[END_REF][START_REF] Grigorieva | Optimal control of a nonlinear model of economic growth[END_REF][START_REF] Yegorov | Synthesis of optimal control in a mathematical model of economic growth under R&D investments[END_REF], 2017a]. It should be noted, however, that even for the small horizon T = 15 (Fig. 2), and a fortiori for larger finite horizons (Fig. 3), the final chattering regime gives only a slight relative addition (< 1%) to the total product output in comparison with the practically more convenient strategy of extending the singular regime until the end of the observed time interval.

Dynamical biomass maximization for Model 1

The dynamics of the optimal control inputs, as shown in Figs. 2 and3, resembles the bang-bang-singular strategies for the biomass maximization problems studied in [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], Yegorov et al., 2017b[START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF]. This motivates an optimal control analysis under the biomass maximization criterion of Definition 3.2, in order to determine if biomass maximization is a good Fig. 3 The optimal control input and state variables for the product maximization problem for Model 1 computed by using BOCOP. The parameter values and initial conditions are given by ( 48)-( 53). The time horizon is taken as T = 50.

approximation of product maximization. As a further motivation, we remind that, by virtue of Property 4.1, the integrands in the product maximization and biomass maximization criteria coincide with each other at steady states. As a consequence, the optimal steady-state coordinates p * opt , r * opt for the product maximization problem also lead to the maximal steady-state biomass production. The singular arcs in Figs. 2 and3, which make up most of the optimal trajectories, are close to the optimal steady state, thus suggesting that biomass maximization may approximate product maximization.

First, it is useful to construct the third adjoint component ψ 3 (•) for the product maximization problem in Model 1, based on the formula (47), terminal conditions (45) and the optimal state trajectory computed via BOCOP. The related graphs in the two cases T = 15 and T = 50 are shown in Fig. 4. One can see that, in reverse time, this function decreases by starting at ψ 3 T = 0 and approaching the number ψ 0 = -1, which is done relatively fast if T is large enough.

In view of the above, we formally set ψ 3 t ≡ ψ 0 in the adjoint system (43) for the product maximization problem. Then, after introducing the new notations η 0 , η 1 , η 2 instead of ψ 0 , ψ 1 , ψ 2 , respectively, we arrive at the following Fig. 4 The third adjoint component ψ 3 = ψ 3 t for the product maximization problem in Model 1, constructed from the formula (47), terminal conditions (45) and the optimal state trajectory computed via BOCOP. The parameter values and initial conditions are given by ( 48)-( 53), and the two cases T = 15 and T = 50 are considered.

two-dimensional adjoint system:

                                                         η 0 ≡ -1 (normal case) or η 0 ≡ 0 (abnormal case), dη 1 t d t = K r t K + p t 2 η 1 t 1 + p t + + η 2 t r t -u t + η 0 + + η 1 t p t r t K + p t + k 1 K 1 1 -r t K 1 + p t 2 , dη 2 t d t = E M η 1 t + p t K + p t η 1 t 1 + p t + + η 2 t 2 t -u η 0 - -η 1 t k 1 p t K 1 + p t , t ∈ 0, T . (54)
The latter is the same as the adjoint system obtained by formulating PMP for the biomass maximization problem for Model 1, where the state can be reduced to only two variables p and r.

Similarly to the formulations in [Yegorov et al., 2018, Subsection 2.2] regarding biomass maximization problems for self-replicator models without a heterologous metabolic pathway, we reasonably exclude the final chattering regime for the biomass maximization problem for Model 1 by imposing the terminal constraint p T , r T = p * opt , r * opt .

(55)

Note that the time horizon T ∈ (0, +∞) is fixed and taken large enough in order to guarantee the existence of admissible trajectories satisfying this constraint.

The following existence result can be established similarly to [Yegorov et al., 2018, Theorem 5.1].

Theorem 4.7 Let Assumptions 3.4 and 4.2 hold. There exists an optimal open-loop control strategy in the biomass maximization problem for Model 1 under the terminal constraint (55) if the time horizon T is sufficiently large.

The minimum suitable horizon T in Theorem 4.7 depends on the initial state (p 0 , r0 ) ∈ G 1 . A heuristic way of selecting an appropriate horizon is to integrate the first two equations in (21) with u ≡ u * opt numerically until a sufficiently small neighborhood of p * opt , r * opt is reached and then to take T somewhat greater than the final time of the integration.

The next theorem can be proved by using PMP similarly to the results of [Yegorov et al., 2018, Section 6].

Theorem 4.8 Let Assumptions 3.4 and 4.2 hold. Consider the biomass maximization problem for Model 1 under the terminal constraint (55). Abnormal extremal processes in this problem do not allow for singular regimes. Any singular arc of a normal extremal process in this problem stays at the optimal steady state p * opt , r * opt and can be entered or left only by chattering. For a related chattering arc, bang-bang switchings from u = u max to u = u min (in forward time) occur only in the domain

{(p, r) ∈ G 1 : r > r * (p)} , (56)
and switchings in the opposite direction occur only in the domain

{(p, r) G 1 : r < r * (p)} , (57)
where r * = r * (p) is defined by (34).

Furthermore, we have developed a numerical method for constructing the switching curve of the optimal feedback control law in the biomass maximization problem for Model 1 under the terminal constraint (55). The related description and illustrations are given in Appendix C. As in [START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF], we exclude abnormal extremal processes (with η 0 ≡ 0) from consideration due to the fact that they do not allow for singular regimes (as mentioned in Theorem 4.8) and therefore do not admit a clear biological interpretation.

Figs. 5 and 6 show the results of solving the biomass maximization problem for Model 1 under the terminal constraint (55) by means of BOCOP, with the same parameters and initial states as in the previous subsection. For comparison, the results for the product maximization problem from Figs. 2 and 3 are also reproduced. In agreement with our theoretical results, the state variables p, r in Figs. 5 and 6 reach the optimal steady-state coordinates p * opt , r * opt by chattering and then stay in the singular regime until the final time.

The second chattering arc does not appear by virtue of (55). Fig. 5 The optimal control input and state variables for the biomass maximization problem for Model 1 under the terminal constraint (55), computed by using BOCOP (black dashed curves). The parameter values and initial conditions are given by ( 48)-( 53). The time horizon is taken as T = 15. The solutions of the corresponding product maximization problem for Model 1 (Fig. 2) are also indicated for comparison (gray solid curves). Fig. 6 The optimal control input and state variables for the biomass maximization problem for Model 1 under the terminal constraint (55), computed by using BOCOP (black dashed curves). The parameter values and initial conditions are given by ( 48)-( 53). The time horizon is taken as T = 50. The solutions of the corresponding product maximization problem for Model 1 (Fig. 3) are also indicated for comparison (gray solid curves).

Product maximization, Model 1 Biomass maximization, Model 1

Product maximization, Model 1 Biomass maximization, Model 1 Fig. 7 Comparison of the values of the product output characteristic (58) for the product maximization problem for Model 1 (gray solid curves), and for the biomass maximization problem for Model 1 under the terminal constraint (55) (black dashed curves). The functions are approximated from the numerical solutions indicated in Figs. 2, 3, 5, 6. The parameter values are given by ( 48)-( 53), and the two cases T = 15 and T = 50 are considered.

Fig. 7 compares the values of the integral

t 0 k 1 x(ξ) p(ξ) (1 -r(ξ)) K 1 + p(ξ) dξ, (58)
which serves as a characteristic of the product output over the time interval 0, t according to Property 3.3. The integral computed from the numerically constructed optimal solutions of the product maximization and biomass maximization problems. The amount of X produced by the optimal control strategy for the biomass maximization problem, even when imposing the terminal constraint (55), is seen to be very close to that obtained for the product maximization problem. This confirms that, for Model 1, where the environmental conditions remain constant, biomass maximization results in a good approximation of product maximization. This can be intuitively understood by the fact that the amount of metabolite X produced is somewhat proportional to the exponentially growing amount of biomass. As a consequence, when an unlimited amount of substrate is available, it becomes profitable to produce as much biomass as possible, even if this leads to a lower metabolite production per unit of biomass, since most of the substrate is utilized for the synthesis of biomass (and not product).

Model 2 (substrate depletion)

In biotechnological problems of practical interest, substrates are expensive and not available in unlimited amounts. Rather, the problem consists in converting a limited amount of substrate into the maximum amount of metabolite of interest within a given time interval. A priori, one would expect that, in order to maximize metabolite production over the time interval, a trade-off has to be found in promoting either biomass synthesis or product synthesis. This can be understood by considering the extreme of high metabolic machinery content, enhancing precursor and product synthesis, but leading eventually to low growth. In this case, the little biomass produced does not allow all substrate to be converted into the metabolite of interest within the observed time interval. On the other hand, promoting growth does not necessarily maximize the yield of conversion of the available substrate into the metabolite of interest.

In order to obtain the optimal trade-off, we study the product maximization problem for Model 2 given by Definitions 2.2 and 3.1. In Model 2, contrary to Model 1, the dynamics of the substrate and volume variables have to be taken into account, and the transformation of the optimization criterion in Property 3.3 is therefore not useful. This makes it very difficult to derive analytical results via PMP, and we hence resort directly to numerical optimization by means of BOCOP.

In addition to the parameter values ( 48), ( 49), ( 51) and the initial state coordinates (52) already introduced for Model 1, we need to specify the parameters k M , K M characterizing the macroreaction converting the substrate into precursors, as well as the initial state coordinates s 0 , X 0 , V 0 and the external volume V ext . For consistency, we set the initial uptake rate k M s 0 / (K M + s 0 ) equal to the value (50) of the lumped environmental parameter chosen for Model 1. Bearing in mind the variable and parameter transformations ( 16), we take k

M = 1.2 k R , K M = 0.1/β, i. e., k 2 = 1.2, K 2 = 0.1, ( 59
)
and specify the remaining initial conditions as ŝ0 = 0.5, ln for T = 50. Note that a larger V ext for the same ŝ0 = βS 0 /V ext leads to a higher initial substrate amount S 0 . The solutions of the product maximization problem for Model 2 are shown in Figs. 8-10. For comparison, the solutions of the corresponding biomass maximization problem are plotted as well. One can see that, while the control input profiles for the product maximization and biomass maximization problems are quite similar in the beginning, they start to diverge after some time. For the product maximization problem, the intermediate singular regime is left noticeably earlier, and u is thereafter set to 0 until the end of the observed time interval. That is, the synthesis of the gene expression machinery (R) is shut off, and the nutrients are used only for the synthesis of the metabolic machinery (M) and for producing the metabolite of interest (X). Put in a schematic way, the optimal control strategy for the product maximization problem seems to Fig. 8 The optimal control input and state variables for the product maximization problem for Model 2, computed by using BOCOP (gray solid curves). The parameter values and initial conditions are given by ( 48), ( 49), ( 51), ( 52), ( 59)-( 61). The time horizon is taken as T = 15. We also indicate the solutions of the corresponding biomass maximization problem for Model 2, computed by using BOCOP (black dashed curves). For convenience, the logarithmic scales ln V and ln 1 + X are used for illustrating the dynamics of the structural volume and product output.

V 0 = 0 L, X0 = 0 L. ( 60
consist in the first phase of maximal growth followed by the second phase of maximal production. The difference in the control input profiles also leads to different state dynamics. In particular, for product maximization, the concentration r of the gene expression machinery decreases due to growth dilution after u is set to 0. As a consequence, the total amount of biomass generated is lower than for biomass maximization, but the total production is higher.

We thus conclude that, whereas biomass maximization is a good approximation of product maximization for Model 1, this is not true for Model 2. In the latter case, the arrest of the gene expression machinery (and therefore growth) via the external control input leads to a sizeable increase in metabolite production. Fig. 9 The optimal control input and state variables for the product maximization problem for Model 2, computed by using BOCOP (gray solid curves). The parameter values and initial conditions are given by ( 48), ( 49), ( 51), ( 52), ( 59)-( 61). The time horizon is taken as = 20. We also indicate the solutions of the corresponding biomass maximization problem for Model 2, computed by using BOCOP (black dashed curves). For convenience, the logarithmic scales ln V and ln 1 + X are used for illustrating the dynamics of the structural volume and product output.

Dynamical product maximization with natural and synthetic growth controls

We have thus far considered models in which both natural and synthetic growth controls are aggregated into a single input (as discussed in Section 3). This is clearly an idealized situation, since we have implicitly assumed that the synthetic growth control I can completely override the natural growth control α. In reality, the two are expected to interfere, which may therefore reduce the efficacy of the efforts to reorient resources from growth to metabolite production. Following (2), we henceforth assume that the two effects are multiplicative, i. e., u t = I t α t .

How can we model natural growth control? In [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], it was shown for the self-replicator in Fig. 1A (without a heterologous pathway and without external growth control) that a certain discontinuous feedback Fig. 10 The optimal control input and state variables for the product maximization problem for Model 2, computed by using BOCOP (gray solid curves). The parameter values and initial conditions are given by ( 48), ( 49), ( 51), ( 52), ( 59), ( 60), ( 62). The time horizon is taken as T = 50. We also indicate the solutions of the corresponding biomass maximization problem for Model 2, computed by using BOCOP (black dashed curves). For convenience, the logarithmic scales ln V and ln 1 + X are used for illustrating the dynamics of the structural volume and product output.

strategy leads to a reasonable approximation of the optimal policy for biomass maximization (see Fig. 11A). This so-called on-off strategy maps values of p and r into values of α:

α t = α p t , r t . (64)
Moreover, the on-off strategy was shown to resemble the action of a major regulatory system involved in growth control in E. coli, the ppGpp system [START_REF] Bosdriesz | How fast-growing bacteria robustly tune their ribosome concentration to approximate growthrate maximization[END_REF]. In this section, we introduce Model 3, a version of Model 2 with the aggregated control (63) and with the natural feedback control (64) constructed from the on-off strategy of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF] (see Fig. 11B ).

The self-replicator model, for which the on-off strategy was originally proposed (under the hypothesis that the self-replicators have evolved to maximize the production of biomass [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF]), does not include the heterologous (non-natural) pathway for the production of metabolite X (see

S P R M A B (1-α)V R S P R M X I V M V S αV R V S V M (1-α)V R αV R V X
Fig. 11 Self-replicator models of bacterial growth and metabolite production taking into account natural and synthetic growth controls. A: Original self-replicator model from Figure 1A, including natural feedback growth control following the on-off strategy proposed by [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF]. This on-off strategy mimics the effect of ppGpp regulation in E. coli [START_REF] Bosdriesz | How fast-growing bacteria robustly tune their ribosome concentration to approximate growthrate maximization[END_REF]. B : Idem for the extended self-replicator model from Figure 1B, with the pathway for metabolite production. Growth control in these self-replicators involves a combination of natural growth control via α and synthetic control via the external inducer I.

Fig. 11A). As a consequence, the steady state with the maximal growth rate and the corresponding constant control input differ between the models with and without the heterologous pathway. Production of X can be completely shut off by setting k 1 = 0, and we therefore use the notations p * opt, k1=0 , r * opt, k1=0 , and α * opt, k1=0 for the optimal steady state and for the corresponding control in the model of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF] without metabolite production.

Model 3 also revisits another hypothesis, namely that the external signal can be switched on and off at will. This may be possible for optogenetic regulation of gene expression [Milias-Argeitis et al., 2016, Olson and[START_REF] Olson | Optogenetic characterization methods overcome key challenges in synthetic and systems biology[END_REF], but does not hold for the chemical induction system used by [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF]. We make the simplifying assumption that RNA polymerase synthesis is initially fully active, and then switched off at a time instant tsw ∈ 0, T . This reduces the range of control strategies considered, but allows us to answer the question to which extent the approach implemented by [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF], where a phase with maximal induction of RNA polymerase expression is followed by a phase with no induction, can provide a reasonable approximation of the optimal solutions obtained in Subsection 4.2.

Hence, we define Model 3 and the corresponding product maximization problem as follows.

Definition 5.1 (Model 3: natural and synthetic growth controls) Consider the system of equations (23) defining Model 2. Model 3 extends them by adopting (63), (64) with

I t = I 0 , t ∈ 0, tsw , 0, t ∈ tsw , T , tsw ∈ 0, T , I ∈ [0, I max ] , (65)
and

α (p, r) def =        0, r > g (p) , 1, r < g (p) , α * opt, k1 = 0 , (p, r) = p * opt, k1 = 0 , r * opt, k1 = 0 , ∀ (p, r) ∈ G 1 , (66)
where p * opt, k1=0 , r * opt, k1=0 , α * opt, k1=0 are as mentioned above, and

g (p) def = p p + K 1+ p K+ p ∀ p > 0, (67)
in line with [Giordano et al., 2016, Appendix S1].

The use of Model 3 instead of Model 2 requires a modification of the statement of the optimization problem as well, since the control input is no longer u(•), but consists of the pair of scalars tsw , I 0 , that is, the time at which synthesis of the gene expression machinery is switched off and the initial induction level, respectively. Definition 5.2 (Revised product maximization problem) Adopt the representations (63)-( 67). For the fixed initial state (24), the revised product maximization problem is specifed by the criterion

X T -→ max ( tsw,I0) ∈ [0, T ] × [0,Imax] .
Here optimization is performed over the switch-off times of the gene expression machinery and over the initial induction levels.

While there is not much hope for solving the revised product maximization problem for Model 3 analytically, the numerical solution can easily be obtained by testing different combinations of tsw and I 0 on a two-dimensional grid for the rectangle 0, T × [0, I max ]. We have developed a dedicated C++ program to this end. * We set the maximal induction level and the grid stepsizes to

I max = 1, ∆ tsw = 0.01, ∆I 0 = 0.01. ( 68
)
In order to reduce errors in numerical integration of the dynamical equations (23), we replace the discontinuous law (66) with its continuously differentiable approximation determined by

α (p, r) def =        0, r g (p) + ε, 1, r g (p) -ε, ϕ r -g (p) ε , |r -g (p)| ε, ∀ (p, r) ∈ G 1 , (69)
* The code has been uploaded to http://www-sop.inria.fr/members/Jean-Luc.Gouze/JMBcode/code.zip where ε > 0 is a sufficiently small parameter, and the function ϕ : [-1, 1] → [0, 1] is selected as the third-order polynomial

ϕ(x) def = x 3 4 - 3x 4 + 1 2 ∀ x ∈ [-1, 1] (70)
satisfying ϕ(-1) = 1, ϕ(1) = 0, ϕ (-1) = ϕ (1) = 0.

We take

ε = 5 • 10 -3 . ( 71
)
The other parameters and initial states are chosen as in Subsection 4.2 for Model 2. In particular, the three time horizons T = 15, T = 20, T = 50 are considered, and the external volume V ext is determined by ( 61) in the cases T = 15, T = 20, and by ( 62) for T = 50. The corresponding numerical results for the (revised) product maximization problem for Model 3 are shown in Figs. 12-14 (blue curves). For comparison, the solutions of the product maximization and biomass maximization problems for Model 2 (Figs. 8-10) are also indicated (gray solid and black dashed curves, respectively). First, one can see that the profiles of the aggregated control strategy u for the product maximization problems for Models 2 and 3 look similar to each other. The major qualitative difference here is that leaving the singular regime at t = tsw, opt no longer involves chattering in case of Model 3. However, the effect of this difference on the optimal amount of metabolite X produced is negligible. Second, the optimal value I 0, opt in Model 3 equals I max , meaning that the optimal strategy essentially consists in a growth phase followed by a production phase, like for the product maximization problem for Model 2. We conclude from these results that, for the purpose of product maximization, synthetic growth control can effectively override natural growth control, and that even the simple induction strategy, consisting of a single shift from maximal induction to zero induction of synthesis of the gene expression machinery, like in [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF], leads to a near-optimal solution.

Discussion

The functioning of microbial cells on the molecular level involves complex regulatory networks controlling the biochemical reactions responsible for, among other things, the transformation of external substrates into precursor metabolites, and the conversion of the precursors into proteins (biomass). The understanding of these regulatory networks in control-theoretical terms has become well-established [START_REF] Iglesias | Control Theory and Systems Biology[END_REF]Ingalls, 2010, Sontag, 2005]. For example, the robustness of the response of microbial cells to environmental perturbations has been identified with the operation of integral feedback controllers on the molecular level [START_REF] Yi | Robust perfect adaptation in bacterial chemotaxis through integral feedback control[END_REF], see [START_REF] El-Samad | Surviving heat shock: Control strategies for robustness and performance[END_REF], [START_REF] Sauro | Control and regulation of pathways via negative feedback[END_REF] Fig. 12 The optimal control input and state variables for the (revised) product maximization problem for Model 3 (blue curves). The parameter values and initial conditions are given by ( 49), ( 51), ( 52), ( 59)-( 61), ( 68), (71). The time horizon is T = 15. The dynamics of the natural feedback growth control strategy approximated by ( 69) is illustrated as well. We also indicate the solutions of the corresponding product maximization and biomass maximization problems for Model 2, taken from Fig. 8 (gray solid and black dashed curves, respectively).

examples). Moreover, control theory has the potential of guiding the reengineering and control of regulatory networks for biotechnological or biomedical purposes [START_REF] Iglesias | Control Theory and Systems Biology[END_REF]Ingalls, 2010, Vecchio et al., 2016], notably by exploiting advances in synthetic biology to rewire interactions in the network [START_REF] Khalil | Synthetic biology: applications come of age[END_REF]]. An example is a theoretical study of the dynamical control of metabolic pathways by a synthetic gene circuit ( [START_REF] Oyarzún | Synthetic gene circuits for metabolic control: design trade-offs and constraints[END_REF], see also [START_REF] Chaves | Exact control of genetic networks in a qualitative framework[END_REF] Fig. 13 The optimal control input and state variables for the (revised) product maximization problem for Model 3 (blue curves). The parameter values and initial conditions are given by ( 49), ( 51), ( 52), ( 59)-( 61), ( 68), (71). The time horizon is T = 20. The dynamics of the natural feedback growth control strategy approximated by ( 69) is illustrated as well. We also indicate the solutions of the corresponding product maximization and biomass maximization problems for Model 2, taken from Fig. 9 (gray solid and black dashed curves, respectively).

In this paper, we have studied the reallocation of cellular resources via external control of the gene expression machinery from the perspective of dynamical optimization. While inspired by previous works based on the use of self-replicator models for the analysis of microbial growth [START_REF] Ewald | Optimality principles reveal a complex interplay of intermediate toxicity and kinetic efficiency in the regulation of prokaryotic metabolism[END_REF][START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF][START_REF] Molenaar | Shifts in growth strategies reflect tradeoffs in cellular economics[END_REF][START_REF] Pavlov | Optimal control of gene expression for fast proteome adaptation to environmental change[END_REF][START_REF] Scott | Emergence of robust growth laws from optimal regulation of ribosome synthesis[END_REF][START_REF] Van Den Berg | Optimal allocation between nutrient uptake and growth in a microbial trichome[END_REF][START_REF] Weiße | Mechanistic links between cellular trade-offs, gene expression, and growth[END_REF][START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF], we have focused on engineered rather than naturally evolved resource allocation strategies. In particular, we have developed extended self-replicator Fig. 14 The optimal control input and state variables for the (revised) product maximization problem for Model 3 (blue curves). The parameter values and initial conditions are given by ( 49), ( 51), ( 52), ( 59), ( 60), ( 62), ( 68), (71). The time horizon is T = 50. The dynamics of the natural feedback growth control strategy approximated by ( 69) is illustrated as well. We also indicate the solutions of the corresponding product maximization and biomass maximization problems for Model 2, taken from Fig. 10 (gray solid and black dashed curves, respectively).

models that include a pathway for the production of a metabolite of biotechnological interest as well as a synthetic circuit for externally controlling the expression of RNA polymerase. The motivation for this system has come from a recent experimental work on a synthetic growth switch in E. coli [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF], in which growth was arrested by externally shutting off the expression of RNA polymerase, thus enabling the passive redirection of incoming nutrient fluxes from biomass synthesis to the production of a metabolite of interest.

Three model variants of this extended self-replicator have been considered. Model 1 assumes that substrate availability is unlimited and that the substrate concentration in the medium is essentially constant over the time interval considered. This is not the case for Model 2, which takes into account the diminution of the substrate concentration due to the consumption of the limited substrate supply by the growing bacterial population. Whereas Models 1 and 2 both assume that the synthesis of the gene expression machinery is fully determined by the external control signal, Model 3 makes the more realistic assumption that this control interferes with natural feedback growth control mechanisms. A simple representation of the latter mechanisms, proposed by [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], has been included in Model 3 as an extension of Model 2. The model variants have been studied by a combination of analytical and computational techniques to find the external control signal leading to either product or biomass maximization.

When resources are unlimited (Model 1), essentially the same amount of product is obtained by product maximization and biomass maximization. In other words, external growth arrest by switching off synthesis of the gene expression machinery does not bring any advantage in this case. Since resources are unlimited, there is no penalty for spending resources on biomass, so that a population with a high exponential growth rate will be more productive than a population with a low exponential growth rate. This is not true in the case of limited resources (Model 2), where in order to maximize the amount of product obtained, for a given duration of the production process, a trade-off needs to be found between high productivity (total rate of conversion of substrate into product, depending on the size of the population) and high yield (efficiency of the conversion of substrate into product). This yield/productivity trade-off is well-known in metabolic engineering [START_REF] Venayak | Engineering metabolism through dynamic control[END_REF]. The optimal solution for product maximization in case of Model 2 consists of a two-stage procedure: a first stage of maximal biomass production followed by a second stage of maximal product synthesis. Imposing this two-stage procedure on Model 3 shows no significant decrease in the amount of product obtained, suggesting that the efficacity of external growth control is not affected by natural feedback growth control, at least for the self-replicator models considered here.

The self-replicator models used in this paper are very simplified representations of the biological processes taking place in living cells, as discussed in Section 2. Among other things, the models neglect protein degradation, lump macromolecules into only two classes of proteins, and disregard cell division. Some of the simplifications could affect the qualitative nature of the optimal control solutions found. However, the conclusion that the two-stage scheme of biomass maximization followed by product maximization is optimal corresponds well with established practice in biotechnology [START_REF] Anesiadis | Dynamic metabolic engineering for increasing bioprocess productivity[END_REF][START_REF] Venayak | Engineering metabolism through dynamic control[END_REF]. As a matter of fact, the experimental test of the growth switch by [START_REF] Izard | A synthetic growth switch based on controlled expression of RNA polymerase[END_REF] also followed this scheme, in that bacteria from a preculture grown under maximal induction of RNA polymerase expression (growth phase) were rediluted into the same medium without inducer and thus without RNA polymerase expression (production phase), increasing the yield of glycerol production from glucose. The results of the present study suggest a way to optimize this procedure, namely to choose the time of redilution (or equivalently, the redilution factor) in such a way that the amount of glycerol produced over a given time interval is maximal.

In the case of biomass maximization for the original self-replicator model in Fig. 1A [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], the addition of protein degradation did not fundamentally change the conclusions of the dynamical optimal control analysis for physiological temperatures [START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF]. This may not always be the case for the extended self-replicators in this paper, especially for long time intervals of metabolite production, since enzyme degradation will decrease the maximum possible rate of metabolite production. As a consequence, periodic strategies with more than one on/off switch of the gene expression machinery may become relevant, allowing a restart of the gene expression machinery when enzyme degradation becomes a limiting factor. Such repeated on/off switches would be much facilitated by light induction rather than chemical induction, as it requires the ability to decrease and not only to increase the inducer signal.

These dynamical scenarios raise interesting optimal control problems, but also questions about the actual implementation of the strategies. Optimal open-loop strategies as provided by the analysis in this paper may inform the design of more robust quasi-optimal closed-loop strategies, by engineering adaptive feedback from real-time observations of the process behavior (growth rate, amount of gene expression machinery R) to the temporal induction of the gene expression machinery. Some interesting work on feedback control at the interface of control engineering and synthetic biology has appeared in recent years [START_REF] Fracassi | Automatic control of gene expression in mammalian cells[END_REF][START_REF] Milias-Argeitis | Automated optogenetic feedback control for precise and robust regulation of gene expression and cell growth[END_REF][START_REF] Uhlendorf | Long-term model predictive control of gene expression at the population and single-cell levels[END_REF].

From the second adjoint equation in ( 43) and the condition (75), one obtains

ψ 1 = - 1 E M + (1 + p) p K+ p -k 1 p K 1 + p • • (ψ 3 -ψ 0 ) p K + p + k 1 e -ŷ p K 1 + p + ψ 0 p K + p . (78)
The third adjoint equation in ( 43) and the equation ( 40) lead to

d d t (ψ 3 -ψ 0 ) = dψ 3 d t = (ψ 3 -ψ 0 ) k 1 e -ŷ p (1 -r) K 1 + p , d d t (ψ 3 -ψ 0 ) e -ŷ = (ψ 3 -ψ 0 ) e -ŷ k 1 e -ŷ p (1 -r) K 1 + p - -k 1 e -ŷ p (1 -r) K 1 + p + p r K + p = (ψ 3 -ψ 0 ) e -ŷ p r K + p . (79)
From ( 76), ( 78) and ( 79), one arrives at

dψ 1 d t = - 1 E M + (1 + p) p K+ p -k 1 p K 1 + p (ψ 3 -ψ 0 ) k 1 e -ŷ • • p (1 -r) K 1 + p p K + p + (ψ 3 -ψ 0 ) e -ŷ p r K + p k 1 p K 1 + p = = - (ψ 3 -ψ 0 ) k 1 e -ŷ E M + (1 + p) p K+ p -k 1 p K 1 + p p2 (K + p) (K 1 + p) , d 2 ψ 1 d t2 = dψ 1 d t p r K + p = = - (ψ 3 -ψ 0 ) k 1 e -ŷ E M + (1 + p) p K+ p -k 1 p K 1 + p p3 r (K + p) 2 (K 1 + p) . (80)
On the other hand, the first adjoint equation in (43) and the condition (75) give

dψ 1 d t = K 1 + p (K + p) 2 + p K + p r + k 1 K 1 (K 1 + p) 2 (1 -r) ψ 1 - -k 1 K 1 e -ŷ 1 - r (K 1 + p) 2 -K r (K + p) 2 (ψ 3 -ψ 0 ) + + ψ 0 K r (K + p) 2 ,
which implies

d 2 ψ 1 d t2 - K 1 + p (K + p) 2 + p K + p r + k 1 K 1 (K 1 + p) 2 (1 -r) dψ 1 d t = = -k 1 K 1 1 - r (K 1 + p) 2 (ψ 3 -ψ 0 ) e -ŷ p r K + p + + K r (K + p) 2 (ψ 3 -ψ 0 ) k 1 e -ŷ p (1 -r) K 1 + p = = p r (1 -r) (K + p) (K 1 + p) (ψ 3 -ψ 0 ) k 1 e -ŷ K K + p - K 1 K 1 + p (81)
u = umax to u = u min (in forward time) occur only on Σ 0 and switchings in the opposite direction occur only on Σ 1 . We do not include the optimal steady state in Σ 0 and Σ 1 . According to Theorem 4.8, Σ 0 lies in the set (56), and Σ 1 is contained in the set (57). The optimal feedback strategy equals u min on the upper side of Σ and umax on the lower side of Σ. These properties can be seen in Fig. 15 below.

Since u does not explicitly appear in the first equation of ( 21), the first component of the corresponding velocity vector field is continuous regardless of u. From one arrives at the hypothesis that the tangent line to Σ at p * opt , r * opt is unique and vertical (as also observed in Fig. 15 below). For a rigorous verification, one can use the following property (whose general formulation can be found in [START_REF] Naumov | Construction of the switching curve for optimal control problems with chattering control[END_REF] coincide with each other, then this is also the direction of the unique tangent line to Σ at p * opt , r * opt . Therefore, the latter is vertical by virtue of (83). Note that such tangent lines are also vertical for optimal chattering regimes in the well-known Fuller and Marshal problems [Schattler andLedzewicz, 2012, Zelikin and[START_REF] Zelikin | Theory of Chattering Control with Applications to Astronautics[END_REF]].

Thus, one can choose a sufficiently small δ > 0 such that, in the closed neighborhood (88)

Let the process (86) contain a chattering arc that enters the singular regime at the optimal steady state p * opt , r * opt . Following [START_REF] Yegorov | Optimal feedback strategies for bacterial growth with degradation, recycling and effect of temperature[END_REF], we exclude abnormal extremal processes (with η 0 ≡ 0) from consideration, since they do not allow for singular regimes (due to Theorem 4.8) and therefore do not admit a clear biological interpretation. Then η 0 ≡ -1, and the conserved Hamiltonian value for the process (86) equals allows to find the bang-bang switching points in the state space (at the time instants when η 2 changes its sign) and thereby to approximate Σ 0 and Σ 1 . It remains to recall that Σ = Σ 0 ∪ p * opt , r * opt ∪ Σ 1 . Note that this method of approximating Σ in principle does not depend on a particular finite time horizon T , although some upper time bound for integrating the reverse-time system of the state and adjoint equations has to be selected. Fig. 15 shows the switching curve Σ approximated via the described algorithm. The global optimal feedback control law in the studied biomass maximization problem is in fact illustrated (the feedback representation combines all possible initial states together, as opposed to the time-dependent open-loop form). The corresponding parameter values (48)-( 51) are the same as in Subsections 4.1.2 and 4.1.3. Fig. 16 compares Σ with the switching curve Σ k 1 = 0 for the biomass maximization problem of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF] (whose dynamical system can be obtained from the first two equations of (21) by taking k 1 = 0 and u ≡ α). The parameters (49)-( 51) are still the same, while different control constraints are now considered. For Σ, the control bounds u min = 0.01 and umax = 0.81 correspond to the case I min = α min = 0.1, Imax = αmax = 0.9. For Σ k 1 = 0 , we have u ≡ α and, hence, u min = α min , umax = αmax (here we set I ≡ 1 Fig. 15 The optimal feedback control law in the biomass maximization problem for Model 1 under the terminal constraint (55). The parameter values are given by ( 48)-( 51).

16 The chattering switching curves Σ and Σ k 1 = 0 in the biomass maximization problem for Model 1 under the terminal constraint (55), and in the biomass maximization problem of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF]. Different control constraints are considered, and the other parameter values are given by ( 49)-( 51).

for the model of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF], because it does not contain an inducer). Moreover, p * opt, k 1 = 0 , r * opt, k 1 = 0 denotes the optimal steady state for the problem of [START_REF] Giordano | Dynamical allocation of cellular resources as an optimal control problem: Novel insights into microbial growth strategies[END_REF].

Property 4. 1

 1 At a steady state (p * , r * , x * ) ∈ G of Model 1 satisfying (33), the integrand in the product maximization criterion (29) coincides with the dimensionless growth rate (17), i. e., with the integrand in the biomass maximization criterion (27).

)

  We also consider the three different time horizons T = 15, T = 20, and T = 50. The logarithm of the external volume is selected as ln V ext = 8

  steady state, it is reasonable to approximate the switching curve Σ as the vertical •), p(•), r(•), η 0 , η 1 (•), η 2 (•)) .(86)It satisfies the adjoint system (54) and the maximum conditionu t = u min , η 2 t < 0, umax, η 2 t > 0,(87)for the HamiltonianH 1 (p, r, u, η 0 , η 1 , η 2 ) = E M (1 -r) -(1

  tsw be one of bang-bang switching instants for a chattering arc of (86). If dp tsw /d t = 0, then r tsw = r * p tsw (the function r * = r * (p) is defined by (34)) and, from the Hamiltonian conservation property, 4.1 and Theorem 4.3, i. e., this is already a singular arc at t = tsw. Therefore, the case dp tsw /d t = 0 can be excluded from the consideration of chattering arcs. Then the switching condition η 2 tsw = 0 and Hamiltonian conservation property yield a correct representation (without division by zero) for the first adjoint component at t = tsw: 89) can be used for approximate setting of the values of η 1 at the points (p, r) ∈ • Lδ def = L δ \ p * opt , r * opt (as was mentioned above, Σ is close to L δ in the ball (84)). Denote the corresponding representation by η 1,δ = η 1,δ (p, r). For computational purposes, it is reasonable to discretize • Lδ by the grid consisting of the points n, -n + 1, . . . , n -1, n} \ {0}, for a sufficiently large n ∈ N. Let us rewrite the state and adjoint equations under the condition (87) in reverse time. The numerical integration of the four resulting reverse-time equations from the starting positions η 1 = η 1,δ (p, r) , η 2 = 0, (p, r) ∈ {Q i } i∈In

  Theorem 4.3 Let Assumptions 3.4 and 4.2 hold. Model 1 admits a unique optimal steady state p * (p 0 ) and maximizing the integrand in the product maximization criterion (29). The first coordinate p * opt is a unique zero of the function (36) on the interval 0, pmax up (p 0 ) , and the other coordinates together with the corresponding constant control can be determined by (34). Moreover, this steady state is an asymptotically stable node.

	Due to Property 4.1 and Theorem 4.3, the pair p * opt , r * opt also maximizes the integrand in the biomass criterion (27) over all steady states (p * , r * ) ∈ G 1
	satisfying 0	p *	pmax up (p 0 ). In other words, for Model 1 (describing bac-
	terial growth in a constant environment), product maximization and biomass
	maximization are identical at steady states.

opt , r * opt , x * opt ∈ G satisfying the inequality 0 p * opt pmax up

  for other

	Product maximization, Model 3	Product maximization, Model 3
	Product maximization, Model 2
	Biomass maximization, Model 2
	Product maximization, Model 3
		Product maximization, Model 2
		Biomass maximization, Model 2
		Product maximization, Model 3
	Product maximization, Model 2	Product maximization, Model 2
	Biomass maximization, Model 2	Biomass maximization, Model 2
	Product maximization, Model 3	Product maximization, Model 3
	Product maximization, Model 2	Product maximization, Model 2
	Biomass maximization, Model 2	Biomass maximization, Model 2
	Product maximization, Model 3	Product maximization, Model 3

  , de Hijas-Liste et al.[2014],[START_REF] Waldherr | Dynamic optimization of metabolic networks coupled with gene expression[END_REF]).

	Product maximization, Model 3	Product maximization, Model 3
	Product maximization, Model 2
	Biomass maximization, Model 2
	Product maximization, Model 3
		Product maximization, Model 2
		Biomass maximization, Model 2
		Product maximization, Model 3
	Product maximization, Model 2
	Biomass maximization, Model 2
	Product maximization, Model 3
		Product maximization, Model 2
		Biomass maximization, Model 2
		Product maximization, Model 3
		Product maximization, Model 2
		Biomass maximization, Model 2
		Product maximization, Model 3
	Product maximization, Model 2
	Biomass maximization, Model 2
	Product maximization, Model 3

Acknowledgements

This work was supported in part by the PIA project Reset (ANR-11-BINF-0005), ANR project Maximic (ANR-17-CE40-0024-01), Inria IPL AlgaeInSilico, and Labex SIGNALIFE (ANR-11-LABX-0028-01). The authors thank Johannes Geiselmann and Eugenio Cinquemani for discussions and comments on the manuscript.

Appendix A Time-varying bounds for state variables in Model 1

As can be directly verified, the unique solutions to the following Cauchy problems can serve as time-varying bounds (32) for the state variables p, x in Model 1:

Although the right-hand side of the system ( 72) is nonlinear, one can see that it satisfies the sublinear growth property (see, e. g., [Markley, 2004, §2.3]) due to the boundedness of the fractions involved. This ensures the extendability of the unique solution of the Cauchy problem to the whole observed time interval. The existence, uniqueness, and extendability of the solution of ( 72) obviously imply those for ( 73) and ( 74).

B Proof of Theorem 4.6

Consider an extremal process (46) and a time subinterval where

(a singular regime). Assume that p and r are steady, i. e.,

(x and ŷ are not necessarily steady). It suffices to show that these conditions lead to ψ 3ψ 0 ≡ 0, because the latter contradicts with (45). Note that, in line with Property 4.4, the fulfillment of the equality ψ 3 -ψ 0 = 0 for at least one instant implies its fulfillment in the whole considered time interval. The condition (38) and the first state equation in ( 21) yield

due to ( 76) and (79). By substituting (80) into (81), one gets

This is simplified to

and trivially holds for ψ 3 -ψ 0 = 0. Now let ψ 3 -ψ 0 = 0. Then, after substituting ( 77) into ( 82) and canceling ψ 3 -ψ 0 , one consecutively derives p

The obtained contradiction with E M > 0 completes the proof.

C Optimal feedback control in the biomass maximization problem for Model 1 under the terminal constraint (55)

In this section, we adopt Assumptions 3.4, 4.2 and consider the biomass maximization problem for Model 1 under the terminal constraint (55).

First, we develop a numerical method for constructing the chattering switching curve Σ of the optimal feedback control law in the space of the state variables p, r. A similar approach was proposed in [Yegorov et al., 2018, Section 7] regarding biomass maximization problems for self-replicator models without a heterologous metabolic pathway.

The chattering arcs enter the singular regime at the optimal steady state p * opt , r * opt . The latter divides Σ into two subcurves Σ 0 and Σ 1 , so that bang-bang switchings from